ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update Claude Code integration guides (#19415)

Browse Source 
* docs: document Claude Code default models and env var overrides

- Update config example with current Claude Code 2.1.x model names
- Add section documenting default models (sonnet/haiku) that Claude Code requests
- Document env var overrides (ANTHROPIC_DEFAULT_SONNET_MODEL, etc.)
- Show how model_name alias can route to any provider (Bedrock, Vertex, etc.)

* Update docs

Removed warning about changing model names in Claude Code versions.

* docs: add 1M context support and improve Claude Code quickstart guide

- Add comprehensive 1M context window documentation
- Document [1m] suffix usage and shell escaping requirements
- Clarify that LiteLLM config should NOT include [1m] in model names
- Add standalone claude_code_1m_context.md guide
- Improve model selection documentation with environment variables
- Add section on default models used by Claude Code v2.1.14
- Add troubleshooting for 1M context issues
- Reorganize to emphasize environment variables approach

Addresses GitHub issue #14444

* docs: reorder model selection options - prioritize --model over env vars

- Move command line/session model selection to Option 1 (most reliable)
- Move environment variables to Option 2
- Add note that env vars may be cached from previous session
- Emphasize that --model always uses exact model specified

* docs: reorganize 1M context section - separate command line from env vars

- Split 1M context examples into two clear sections
- Show command line usage first (--model and /model)
- Show environment variables as alternative approach
- Improves readability and emphasizes most reliable method

* docs: remove misleading default models section from website tutorial

- Remove 'Default Models Used by Claude Code' section (misleading)
- Remove claim that config must match exact default model names
- Update config comment to be more general
- Add claude-opus-4-5-20251101 to example config
- Keep authentication section as-is

* docs: correct model selection in website tutorial

- Remove incorrect claim that Claude Code automatically uses proxy models
- Add explicit model selection examples with --model and /model
- Show environment variables as alternative approach
- Remove misleading comment about 'multiple configured'

* docs: add 1M context section to website tutorial

- Add section on using [1m] suffix for 1 million token context
- Include warning about shell escaping (quotes required)
- Explain how Claude Code handles [1m] internally
- Add /context verification command
- Note that LiteLLM config should NOT include [1m]

* docs: add tip about using .env for API keys

- Add note that ANTHROPIC_API_KEY can be stored in .env file
- Clarifies alternative to exporting environment variables
This commit is contained in:
Cesar Garcia 2026-01-22 01:11:06 -03:00 committed by GitHub
parent ab606c9a73
commit 815e6fcf76
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 182 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

136
cookbook/ai_coding_tool_guides/claude_code_quickstart/guide.md
View File

@ -97,17 +97,75 @@ export ANTHROPIC_AUTH_TOKEN="$LITELLM_MASTER_KEY"
## Step 5: Use Claude Code
Start Claude Code and it will automatically use your configured models:
### Choosing Your Model
You have two options for specifying which model Claude Code uses:
#### Option 1: Command Line / Session Model Selection
Specify the model directly when starting Claude Code or during a session:
```bash
# Claude Code will use the models configured in your LiteLLM proxy
claude
# Or specify a model if you have multiple configured
# Specify model at startup
claude --model claude-3-5-sonnet-20241022
claude --model claude-3-5-haiku-20241022
# Or change model during a session
/model claude-3-5-haiku-20241022
```
This method uses the exact model you specify.
#### Option 2: Environment Variables
Configure default models using environment variables:
```bash
# Tell Claude Code which models to use by default
export ANTHROPIC_DEFAULT_SONNET_MODEL=claude-3-5-sonnet-20241022
export ANTHROPIC_DEFAULT_HAIKU_MODEL=claude-3-5-haiku-20241022
export ANTHROPIC_DEFAULT_OPUS_MODEL=claude-opus-3-5-20240229
claude # Will use the models specified above
```
**Note:** Claude Code may cache the model from a previous session. If environment variables don't take effect, use Option 1 to explicitly set the model.
**Important:** The `model_name` in your LiteLLM config must match what Claude Code requests (either from env vars or command line).
### Using 1M Context Window
Claude Code supports extended context (1 million tokens) using the `[1m]` suffix with Claude 4+ models:
```bash
# Use Sonnet 4.5 with 1M context (requires quotes for shell)
claude --model 'claude-sonnet-4-5-20250929[1m]'
# Inside a Claude Code session (no quotes needed)
/model claude-sonnet-4-5-20250929[1m]
```
**Important:** When using `--model` with `[1m]` in the shell, you must use quotes to prevent the shell from interpreting the brackets.
Alternatively, set as default with environment variables:
```bash
export ANTHROPIC_DEFAULT_SONNET_MODEL='claude-sonnet-4-5-20250929[1m]'
claude
```
**How it works:**
- Claude Code strips the `[1m]` suffix before sending to LiteLLM
- Claude Code automatically adds the header `anthropic-beta: context-1m-2025-08-07`
- Your LiteLLM config should **NOT** include `[1m]` in model names
**Verify 1M context is active:**
```bash
/context
# Should show: 21k/1000k tokens (2%)
```
**Pricing:** Models using 1M context have different pricing. Input tokens above 200k are charged at a higher rate.
## Troubleshooting
Common issues and solutions:
@ -123,18 +181,25 @@ Common issues and solutions:
- Ensure the `ANTHROPIC_AUTH_TOKEN` matches your LiteLLM master key
**Model not found:**
- Ensure the model name in Claude Code matches exactly with your `config.yaml`
- Check LiteLLM logs for detailed error messages
- Check what model Claude Code is requesting in LiteLLM logs
- Ensure your `config.yaml` has a matching `model_name` entry
- If using environment variables, verify they're set: `echo $ANTHROPIC_DEFAULT_SONNET_MODEL`
**1M context not working (showing 200k instead of 1000k):**
- Verify you're using the `[1m]` suffix: `/model your-model-name[1m]`
- Check LiteLLM logs for the header `context-1m-2025-08-07` in the request
- Ensure your model supports 1M context (only certain Claude models do)
- Your LiteLLM config should **NOT** include `[1m]` in the `model_name`
## Using Multiple Models and Providers
Expand your configuration to support multiple providers and models:
You can configure LiteLLM to route to any supported provider. Here's an example with multiple providers:
```yaml
model_list:
# OpenAI models
- model_name: codex-mini
litellm_params:
litellm_params:
model: openai/codex-mini
api_key: os.environ/OPENAI_API_KEY
api_base: https://api.openai.com/v1
@ -156,7 +221,7 @@ model_list:
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: claude-3-5-haiku-20241022
litellm_params:
model: anthropic/claude-3-5-haiku-20241022
@ -174,19 +239,54 @@ litellm_settings:
master_key: os.environ/LITELLM_MASTER_KEY
```
**Note:** The `model_name` can be anything you choose. Claude Code will request whatever model you specify (via env vars or command line), and LiteLLM will route to the `model` configured in `litellm_params`.
Switch between models seamlessly:
```bash
# Use Claude for complex reasoning
claude --model claude-3-5-sonnet-20241022
# Use environment variables to set defaults
export ANTHROPIC_DEFAULT_SONNET_MODEL=claude-3-5-sonnet-20241022
export ANTHROPIC_DEFAULT_HAIKU_MODEL=claude-3-5-haiku-20241022
# Use Haiku for fast responses
claude --model claude-3-5-haiku-20241022
# Use Bedrock deployment
claude --model claude-bedrock
# Or specify directly
claude --model claude-3-5-sonnet-20241022 # Complex reasoning
claude --model claude-3-5-haiku-20241022 # Fast responses
claude --model claude-bedrock # Bedrock deployment
```
## Default Models Used by Claude Code
If you **don't** set environment variables, Claude Code uses these default model names:
| Purpose | Default Model Name (v2.1.14) |
|---------|------------------------------|
| Main model | `claude-sonnet-4-5-20250929` |
| Light tasks (subagents, summaries) | `claude-haiku-4-5-20251001` |
| Planning mode | `claude-opus-4-5-20251101` |
Your LiteLLM config should include these model names if you want Claude Code to work without setting environment variables:
```yaml
model_list:
- model_name: claude-sonnet-4-5-20250929
litellm_params:
# Can be any provider - Anthropic, Bedrock, Vertex AI, etc.
model: anthropic/claude-sonnet-4-5-20250929
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: claude-haiku-4-5-20251001
litellm_params:
model: anthropic/claude-haiku-4-5-20251001
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: claude-opus-4-5-20251101
litellm_params:
model: anthropic/claude-opus-4-5-20251101
api_key: os.environ/ANTHROPIC_API_KEY
```
**Warning:** These default model names may change with new Claude Code versions. Check LiteLLM proxy logs for "model not found" errors to identify what Claude Code is requesting.
## Additional Resources
- [LiteLLM Documentation](https://docs.litellm.ai/)

79
docs/my-website/docs/tutorials/claude_responses_api.md
View File

@ -37,18 +37,22 @@ Create a secure configuration using environment variables:
```yaml
model_list:
# Claude models
- model_name: claude-3-5-sonnet-20241022
# Configure the models you want to use
- model_name: claude-sonnet-4-5-20250929
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: claude-3-5-haiku-20241022
litellm_params:
model: anthropic/claude-3-5-haiku-20241022
model: anthropic/claude-sonnet-4-5-20250929
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: claude-haiku-4-5-20251001
litellm_params:
model: anthropic/claude-haiku-4-5-20251001
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: claude-opus-4-5-20251101
litellm_params:
model: anthropic/claude-opus-4-5-20251101
api_key: os.environ/ANTHROPIC_API_KEY
litellm_settings:
master_key: os.environ/LITELLM_MASTER_KEY
```
@ -60,6 +64,10 @@ export ANTHROPIC_API_KEY="your-anthropic-api-key"
export LITELLM_MASTER_KEY="sk-1234567890" # Generate a secure key
```
:::tip
Alternatively, you can store `ANTHROPIC_API_KEY` in a `.env` file in your proxy directory. LiteLLM will automatically load it when starting.
:::
### 2. Start proxy
```bash
@ -111,15 +119,55 @@ export ANTHROPIC_AUTH_TOKEN="$LITELLM_MASTER_KEY"
### 5. Use Claude Code
Start Claude Code and it will automatically use your configured models:
Start Claude Code with the model you want to use:
```bash
# Claude Code will use the models configured in your LiteLLM proxy
claude
# Specify model at startup
claude --model claude-sonnet-4-5-20250929
# Or specify a model if you have multiple configured
claude --model claude-3-5-sonnet-20241022
claude --model claude-3-5-haiku-20241022
# Or specify a different model
claude --model claude-haiku-4-5-20251001
claude --model claude-opus-4-5-20251101
# Or change model during a session
claude
/model claude-sonnet-4-5-20250929
```
Alternatively, set default models with environment variables:
```bash
export ANTHROPIC_DEFAULT_SONNET_MODEL=claude-sonnet-4-5-20250929
export ANTHROPIC_DEFAULT_HAIKU_MODEL=claude-haiku-4-5-20251001
export ANTHROPIC_DEFAULT_OPUS_MODEL=claude-opus-4-5-20251101
claude
```
### Using 1M Context Window
Claude Code supports extended context (1 million tokens) using the `[1m]` suffix:
```bash
# Use Sonnet with 1M context (requires quotes in shell)
claude --model 'claude-sonnet-4-5-20250929[1m]'
# Inside a Claude Code session (no quotes needed)
/model claude-sonnet-4-5-20250929[1m]
```
:::warning
**Important:** When using `--model` with `[1m]` in the shell, you must use quotes to prevent the shell from interpreting the brackets.
:::
**How it works:**
- Claude Code strips the `[1m]` suffix before sending to LiteLLM
- Claude Code automatically adds the header `anthropic-beta: context-1m-2025-08-07`
- Your LiteLLM config should **NOT** include `[1m]` in model names
**Verify 1M context is active:**
```bash
/context
# Should show: 21k/1000k tokens (2%)
```
Example conversation:
@ -140,6 +188,7 @@ Common issues and solutions:
**Model not found:**
- Ensure the model name in Claude Code matches exactly with your `config.yaml`
- Use `--model` flag or environment variables to specify the model
- Check LiteLLM logs for detailed error messages
## Using Bedrock/Vertex AI/Azure Foundry Models