[Docs] Litellm add docs vertex ai engine (#18027)
* new provider doc * add to sidebar * stash docs * docs fix * docs vertex agent engine
This commit is contained in:
Ishaan Jaff
GitHub
parent
13c0ab8985
commit
244d83ff47
@ -16,7 +16,7 @@ Add A2A Agents on LiteLLM AI Gateway, Invoke agents in A2A Protocol, track reque
|
||||
|
||||
| Feature | Supported |
|
||||
|---------|-----------|
|
||||
| Supported Agent Providers | A2A, Pydantic AI, LangGraph, Azure AI Foundry, Bedrock AgentCore |
|
||||
| Supported Agent Providers | A2A, Vertex AI Agent Engine, LangGraph, Azure AI Foundry, Bedrock AgentCore, Pydantic AI |
|
||||
| Logging | ✅ |
|
||||
| Load Balancing | ✅ |
|
||||
| Streaming | ✅ |
|
||||
@ -50,14 +50,18 @@ The URL should be the invocation URL for your A2A agent (e.g., `http://localhost
|
||||
|
||||
Follow [this guide, to add your azure ai foundry agent to LiteLLM Agent Gateway](./providers/azure_ai_agents#litellm-a2a-gateway)
|
||||
|
||||
### Add LangGraph Agents
|
||||
### Add Vertex AI Agent Engine
|
||||
|
||||
Follow [this guide, to add your langgraph agent to LiteLLM Agent Gateway](./providers/langgraph#litellm-a2a-gateway)
|
||||
Follow [this guide, to add your Vertex AI Agent Engine to LiteLLM Agent Gateway](./providers/vertex_ai_agent_engine)
|
||||
|
||||
### Add Bedrock AgentCore Agents
|
||||
|
||||
Follow [this guide, to add your bedrock agentcore agent to LiteLLM Agent Gateway](./providers/bedrock_agentcore#litellm-a2a-gateway)
|
||||
|
||||
### Add LangGraph Agents
|
||||
|
||||
Follow [this guide, to add your langgraph agent to LiteLLM Agent Gateway](./providers/langgraph#litellm-a2a-gateway)
|
||||
|
||||
### Add Pydantic AI Agents
|
||||
|
||||
Follow [this guide, to add your pydantic ai agent to LiteLLM Agent Gateway](./providers/pydantic_ai_agent#litellm-a2a-gateway)
|
||||
|
||||
216
docs/my-website/docs/providers/vertex_ai_agent_engine.md
Normal file
216
docs/my-website/docs/providers/vertex_ai_agent_engine.md
Normal file
@ -0,0 +1,216 @@
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
# Vertex AI Agent Engine
|
||||
|
||||
Call Vertex AI Agent Engine (Reasoning Engines) in the OpenAI Request/Response format.
|
||||
|
||||
| Property | Details |
|
||||
|----------|---------|
|
||||
| Description | Vertex AI Agent Engine provides hosted agent runtimes that can execute agentic workflows with foundation models, tools, and custom logic. |
|
||||
| Provider Route on LiteLLM | `vertex_ai/agent_engine/{RESOURCE_NAME}` |
|
||||
| Supported Endpoints | `/chat/completions`, `/v1/messages`, `/v1/responses`, `/v1/a2a/message/send` |
|
||||
| Provider Doc | [Vertex AI Agent Engine ↗](https://cloud.google.com/vertex-ai/generative-ai/docs/reasoning-engine/overview) |
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Model Format
|
||||
|
||||
```shell showLineNumbers title="Model Format"
|
||||
vertex_ai/agent_engine/{RESOURCE_NAME}
|
||||
```
|
||||
|
||||
**Example:**
|
||||
- `vertex_ai/agent_engine/projects/1060139831167/locations/us-central1/reasoningEngines/8263861224643493888`
|
||||
|
||||
### LiteLLM Python SDK
|
||||
|
||||
```python showLineNumbers title="Basic Agent Completion"
|
||||
import litellm
|
||||
|
||||
response = litellm.completion(
|
||||
model="vertex_ai/agent_engine/projects/1060139831167/locations/us-central1/reasoningEngines/8263861224643493888",
|
||||
messages=[
|
||||
{"role": "user", "content": "Explain machine learning in simple terms"}
|
||||
],
|
||||
)
|
||||
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
```python showLineNumbers title="Streaming Agent Responses"
|
||||
import litellm
|
||||
|
||||
response = await litellm.acompletion(
|
||||
model="vertex_ai/agent_engine/projects/1060139831167/locations/us-central1/reasoningEngines/8263861224643493888",
|
||||
messages=[
|
||||
{"role": "user", "content": "What are the key principles of software architecture?"}
|
||||
],
|
||||
stream=True,
|
||||
)
|
||||
|
||||
async for chunk in response:
|
||||
if chunk.choices[0].delta.content:
|
||||
print(chunk.choices[0].delta.content, end="")
|
||||
```
|
||||
|
||||
### LiteLLM Proxy
|
||||
|
||||
#### 1. Configure your model in config.yaml
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="config-yaml" label="config.yaml">
|
||||
|
||||
```yaml showLineNumbers title="LiteLLM Proxy Configuration"
|
||||
model_list:
|
||||
- model_name: vertex-agent-1
|
||||
litellm_params:
|
||||
model: vertex_ai/agent_engine/projects/1060139831167/locations/us-central1/reasoningEngines/8263861224643493888
|
||||
vertex_project: your-project-id
|
||||
vertex_location: us-central1
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
#### 2. Start the LiteLLM Proxy
|
||||
|
||||
```bash showLineNumbers title="Start LiteLLM Proxy"
|
||||
litellm --config config.yaml
|
||||
```
|
||||
|
||||
#### 3. Make requests to your Vertex AI Agent Engine
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="curl" label="Curl">
|
||||
|
||||
```bash showLineNumbers title="Basic Agent Request"
|
||||
curl http://localhost:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||
-d '{
|
||||
"model": "vertex-agent-1",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Summarize the main benefits of cloud computing"}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="openai-sdk" label="OpenAI Python SDK">
|
||||
|
||||
```python showLineNumbers title="Using OpenAI SDK with LiteLLM Proxy"
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(
|
||||
base_url="http://localhost:4000",
|
||||
api_key="your-litellm-api-key"
|
||||
)
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model="vertex-agent-1",
|
||||
messages=[
|
||||
{"role": "user", "content": "What are best practices for API design?"}
|
||||
]
|
||||
)
|
||||
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## LiteLLM A2A Gateway
|
||||
|
||||
You can also connect to Vertex AI Agent Engine through LiteLLM's A2A (Agent-to-Agent) Gateway UI. This provides a visual way to register and test agents without writing code.
|
||||
|
||||
### 1. Navigate to Agents
|
||||
|
||||
From the sidebar, click "Agents" to open the agent management page, then click "+ Add New Agent".
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### 2. Select Vertex AI Agent Engine Type
|
||||
|
||||
Click "A2A Standard" to see available agent types, then select "Vertex AI Agent Engine".
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### 3. Configure the Agent
|
||||
|
||||
Fill in the following fields:
|
||||
|
||||
- **Agent Name** - A friendly name for your agent (e.g., `my-vertex-agent`)
|
||||
- **Reasoning Engine Resource ID** - The full resource path from Google Cloud Console (e.g., `projects/1060139831167/locations/us-central1/reasoningEngines/8263861224643493888`)
|
||||
- **Vertex Project** - Your Google Cloud project ID
|
||||
- **Vertex Location** - The region where your agent is deployed (e.g., `us-central1`)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
You can find the Resource ID in Google Cloud Console under Vertex AI > Agent Engine:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
You can find the Project ID in Google Cloud Console:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### 4. Create Agent
|
||||
|
||||
Click "Create Agent" to save your configuration.
|
||||
|
||||
|
||||
|
||||
### 5. Test in Playground
|
||||
|
||||
Go to "Playground" in the sidebar to test your agent.
|
||||
|
||||
|
||||
|
||||
### 6. Select A2A Endpoint
|
||||
|
||||
Click the endpoint dropdown and select `/v1/a2a/message/send`.
|
||||
|
||||
|
||||
|
||||
### 7. Select Your Agent and Send a Message
|
||||
|
||||
Pick your Vertex AI Agent Engine from the dropdown and send a test message.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `GOOGLE_APPLICATION_CREDENTIALS` | Path to service account JSON key file |
|
||||
| `VERTEXAI_PROJECT` | Google Cloud project ID |
|
||||
| `VERTEXAI_LOCATION` | Google Cloud region (default: `us-central1`) |
|
||||
|
||||
```bash
|
||||
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
|
||||
export VERTEXAI_PROJECT="your-project-id"
|
||||
export VERTEXAI_LOCATION="us-central1"
|
||||
```
|
||||
|
||||
## Further Reading
|
||||
|
||||
- [Vertex AI Agent Engine Documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/reasoning-engine/overview)
|
||||
- [Create a Reasoning Engine](https://cloud.google.com/vertex-ai/generative-ai/docs/reasoning-engine/create)
|
||||
- [A2A Agent Gateway](../a2a.md)
|
||||
- [Vertex AI Provider](./vertex.md)
|
||||
@ -632,6 +632,7 @@ const sidebars = {
|
||||
"providers/vertex_speech",
|
||||
"providers/vertex_batch",
|
||||
"providers/vertex_ocr",
|
||||
"providers/vertex_ai_agent_engine",
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@ -1947,6 +1947,23 @@
|
||||
"a2a": true
|
||||
}
|
||||
},
|
||||
"vertex_ai/agent_engine": {
|
||||
"display_name": "Vertex AI Agent Engine (`vertex_ai/agent_engine`)",
|
||||
"url": "https://docs.litellm.ai/docs/providers/vertex_ai_agent_engine",
|
||||
"endpoints": {
|
||||
"chat_completions": true,
|
||||
"messages": true,
|
||||
"responses": true,
|
||||
"embeddings": false,
|
||||
"image_generations": false,
|
||||
"audio_transcriptions": false,
|
||||
"audio_speech": false,
|
||||
"moderations": false,
|
||||
"batches": false,
|
||||
"rerank": false,
|
||||
"a2a": true
|
||||
}
|
||||
},
|
||||
"pydantic_ai_agents": {
|
||||
"display_name": "Pydantic AI Agents (`pydantic_ai_agents`)",
|
||||
"url": "https://docs.litellm.ai/docs/providers/pydantic_ai_agent",
|
||||
|
||||
Loading…
Reference in New Issue
Block a user