π Expose Langchain Agent (Langgraph) result as an OpenAI-compatible API π
A FastAPI + Langchain / langgraph extension to expose agent result as an OpenAI-compatible API.
Use any OpenAI-compatible UI or UI framework with your custom Langchain Agent.
- β
Chat Completions API
- β Invoke
- β Stream
- β
Assistant API - Feature in progress
- β Run Stream
- β Threads
- β Messages
- β Run
- β Tools step stream
- π§ Human In The Loop
- β
OpenAI
- β Stream
- β Multimodal
- β Auto tool choice
- β
Anthropic
- β Stream
- β Multimodal
- β Auto tool choice
- β
Groq
- β Stream
- β Multimodal
- β Auto tool choice
- β
LLamaCPP local inference
- β Stream
- β Multimodal
- β Auto tool choice
If you find this project useful, please give it a star β!
pip install langchain-openai-api-bridgepoetry add langchain-openai-api-bridge# Assistant Bridge as OpenAI Compatible API
from fastapi.middleware.cors import CORSMiddleware
from fastapi import FastAPI
from dotenv import load_dotenv, find_dotenv
import uvicorn
from langchain_openai_api_bridge.assistant import (
InMemoryMessageRepository,
InMemoryRunRepository,
InMemoryThreadRepository,
)
from langchain_openai_api_bridge.fastapi.langchain_openai_api_bridge_fastapi import (
LangchainOpenaiApiBridgeFastAPI,
)
from tests.test_functional.fastapi_assistant_agent_openai.my_agent_factory import (
MyAgentFactory,
)
_ = load_dotenv(find_dotenv())
app = FastAPI(
title="Langchain Agent OpenAI API Bridge",
version="1.0",
description="OpenAI API exposing langchain agent",
)
in_memory_thread_repository = InMemoryThreadRepository()
in_memory_message_repository = InMemoryMessageRepository()
in_memory_run_repository = InMemoryRunRepository()
bridge = LangchainOpenaiApiBridgeFastAPI(
app=app, agent_factory_provider=lambda: MyAgentFactory()
)
bridge.bind_openai_assistant_api(
thread_repository_provider=in_memory_thread_repository,
message_repository_provider=in_memory_message_repository,
run_repository_provider=in_memory_run_repository,
prefix="/my-assistant",
)
if __name__ == "__main__":
uvicorn.run(app, host="localhost")# Agent Creation
@tool
def magic_number_tool(input: int) -> int:
"""Applies a magic function to an input."""
return input + 2
class MyAgentFactory(AgentFactory):
def create_agent(self, dto: CreateLLMDto) -> Runnable:
llm = self.create_llm(dto=dto)
return create_react_agent(
llm,
[magic_number_tool],
messages_modifier="""You are a helpful assistant.""",
)
def create_llm(self, dto: CreateLLMDto) -> Runnable:
return ChatOpenAI(
model=dto.model,
api_key=dto.api_key,
streaming=True,
temperature=dto.temperature,
)Full example:
# Server
from langchain_openai_api_bridge.assistant import (
AssistantApp,
InMemoryMessageRepository,
InMemoryRunRepository,
InMemoryThreadRepository,
)
from langchain_openai_api_bridge.fastapi import include_chat_completion
app = FastAPI(
title="Langchain Agent OpenAI API Bridge",
version="1.0",
description="OpenAI API exposing langchain agent",
)
bridge = LangchainOpenaiApiBridgeFastAPI(
app=app, agent_factory_provider=lambda: MyAnthropicAgentFactory()
)
bridge.bind_openai_chat_completion(prefix="/my-custom-path/anthropic")
if __name__ == "__main__":
uvicorn.run(app, host="localhost")# Client
openai_client = OpenAI(
base_url="http://my-server/my-custom-path/anthropic/openai/v1",
)
chat_completion = openai_client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "user",
"content": 'Say "This is a test"',
}
],
)
print(chat_completion.choices[0].message.content)
#> "This is a test"Full example:
// Vercel AI sdk - example
// ************************
// app/api/my-chat/route.ts
import { NextRequest } from "next/server";
import { z } from "zod";
import { type CoreMessage, streamText } from "ai";
import { createOpenAI } from "@ai-sdk/openai";
export const ChatMessageSchema = z.object({
id: z.string(),
role: z.string(),
createdAt: z.date().optional(),
content: z.string(),
});
const BodySchema = z.object({
messages: z.array(ChatMessageSchema),
});
export type AssistantStreamBody = z.infer<typeof BodySchema>;
const langchain = createOpenAI({
//baseURL: "https://my-project/my-custom-path/openai/v1",
baseURL: "http://localhost:8000/my-custom-path/openai/v1",
});
export async function POST(request: NextRequest) {
const { messages }: { messages: CoreMessage[] } = await request.json();
const result = await streamText({
model: langchain("gpt-4o"),
messages,
});
return result.toAIStreamResponse();
}More examples can be found in tests/test_functional directory.
This project is not limited to OpenAIβs models; some examples demonstrate the use of Anthropicβs language models. Anthropic is just one example, and any LangChain-supported vendor is also supported by this library.
Define OPENAI_API_KEY or ANTHROPIC_API_KEY on your system.
Examples will take token from environment variable or .env at root of the project.
If you want to contribute to this project, you can follow this guideline:
- Fork this project
- Create a new branch
- Implement your feature or bug fix
- Send a pull request
poetry install
poetry env use ./.venv/bin/python| Command | Command |
|---|---|
| Run Tests | poetry run pytest |
-
Chat Completions Tools
- Functions cannot be passed through open ai API. Every functions need to be defined as a tool in langchain. Usage Example
-
LLM Usage Info
- Returned usage info is innacurate. This is due to a Langchain/Langgraph limitation where usage info isn't available when calling a Langgraph Agent.
