LangChain

LangChain is a framework for developing applications that are powered by Large Language Models (LLMs). It implements a standard interface for large language models and related technologies, such as embedding models and vector stores, and integrates with hundreds of providers.

Prerequisites

In order to use the AI Gateway with any LangChain powered app you will need to complete these steps first:

Configure LangChain

In this example we will configure LangChain to work with OpenAI (or OpenAI compatible models).

To work with OpenAI in LangChain it's recommended to use their ChatOpenAI integration.

Code
import os
from langchain_openai import ChatOpenAI

def init_chat_model():
    """Initialize the ChatOpenAI model"""
    api_key = os.getenv("ZUPLO_AI_GATEWAY_API_KEY")
    if not api_key:
        print("❌ Error: Please set your ZUPLO_AI_GATEWAY_API_KEY in a .env file")
        exit(1)

    # Check for custom BASE_URL - this is the app's gateway URL from Zuplo
    base_url = os.getenv("BASE_URL")

    if not base_url:
        print("❌ Error: Please set BASE_URL to your app's gateway URL")
        exit(1)

    return ChatOpenAI(
        api_key=api_key, model="openai/gpt-6-luna", base_url=base_url
    )

In the code above, checks are performed for two environment variables:

• ZUPLO_AI_GATEWAY_API_KEY - This is the API key of the app you have configured to use with LangChain in Zuplo
• BASE_URL - This is your app's API URL with /v1 appended, for example https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1. The app page in the Zuplo Portal shows the API URL, which ends in the app's ID

Both of these values are passed to ChatOpenAI when it's instantiated, switching the configuration from using default OpenAI APIs to using Zuplo's AI Gateway.

Always name the model as providerName/model, where providerName is the provider name configured in your gateway. The gateway needs the prefix to know which provider to route to, and a request with no model, or a model without that prefix, gets a 400. Adding the Model Filtering policy restricts an app to certain models, and an allow list also supplies a default so requests may omit the model.