From 815e6fcf760202a1b71c2c537a6aba9dba5afbda Mon Sep 17 00:00:00 2001 From: Cesar Garcia <128240629+Chesars@users.noreply.github.com> Date: Thu, 22 Jan 2026 01:11:06 -0300 Subject: [PATCH] docs: update Claude Code integration guides (#19415) * docs: document Claude Code default models and env var overrides - Update config example with current Claude Code 2.1.x model names - Add section documenting default models (sonnet/haiku) that Claude Code requests - Document env var overrides (ANTHROPIC_DEFAULT_SONNET_MODEL, etc.) - Show how model_name alias can route to any provider (Bedrock, Vertex, etc.) * Update docs Removed warning about changing model names in Claude Code versions. * docs: add 1M context support and improve Claude Code quickstart guide - Add comprehensive 1M context window documentation - Document [1m] suffix usage and shell escaping requirements - Clarify that LiteLLM config should NOT include [1m] in model names - Add standalone claude_code_1m_context.md guide - Improve model selection documentation with environment variables - Add section on default models used by Claude Code v2.1.14 - Add troubleshooting for 1M context issues - Reorganize to emphasize environment variables approach Addresses GitHub issue #14444 * docs: reorder model selection options - prioritize --model over env vars - Move command line/session model selection to Option 1 (most reliable) - Move environment variables to Option 2 - Add note that env vars may be cached from previous session - Emphasize that --model always uses exact model specified * docs: reorganize 1M context section - separate command line from env vars - Split 1M context examples into two clear sections - Show command line usage first (--model and /model) - Show environment variables as alternative approach - Improves readability and emphasizes most reliable method * docs: remove misleading default models section from website tutorial - Remove 'Default Models Used by Claude Code' section (misleading) - Remove claim that config must match exact default model names - Update config comment to be more general - Add claude-opus-4-5-20251101 to example config - Keep authentication section as-is * docs: correct model selection in website tutorial - Remove incorrect claim that Claude Code automatically uses proxy models - Add explicit model selection examples with --model and /model - Show environment variables as alternative approach - Remove misleading comment about 'multiple configured' * docs: add 1M context section to website tutorial - Add section on using [1m] suffix for 1 million token context - Include warning about shell escaping (quotes required) - Explain how Claude Code handles [1m] internally - Add /context verification command - Note that LiteLLM config should NOT include [1m] * docs: add tip about using .env for API keys - Add note that ANTHROPIC_API_KEY can be stored in .env file - Clarifies alternative to exporting environment variables --- .../claude_code_quickstart/guide.md | 136 +++++++++++++++--- .../docs/tutorials/claude_responses_api.md | 79 ++++++++-- 2 files changed, 182 insertions(+), 33 deletions(-) diff --git a/cookbook/ai_coding_tool_guides/claude_code_quickstart/guide.md b/cookbook/ai_coding_tool_guides/claude_code_quickstart/guide.md index ad86c2b7b1..3d6c75498b 100644 --- a/cookbook/ai_coding_tool_guides/claude_code_quickstart/guide.md +++ b/cookbook/ai_coding_tool_guides/claude_code_quickstart/guide.md @@ -97,17 +97,75 @@ export ANTHROPIC_AUTH_TOKEN="$LITELLM_MASTER_KEY" ## Step 5: Use Claude Code -Start Claude Code and it will automatically use your configured models: +### Choosing Your Model + +You have two options for specifying which model Claude Code uses: + +#### Option 1: Command Line / Session Model Selection + +Specify the model directly when starting Claude Code or during a session: ```bash -# Claude Code will use the models configured in your LiteLLM proxy -claude - -# Or specify a model if you have multiple configured +# Specify model at startup claude --model claude-3-5-sonnet-20241022 -claude --model claude-3-5-haiku-20241022 + +# Or change model during a session +/model claude-3-5-haiku-20241022 ``` +This method uses the exact model you specify. + +#### Option 2: Environment Variables + +Configure default models using environment variables: + +```bash +# Tell Claude Code which models to use by default +export ANTHROPIC_DEFAULT_SONNET_MODEL=claude-3-5-sonnet-20241022 +export ANTHROPIC_DEFAULT_HAIKU_MODEL=claude-3-5-haiku-20241022 +export ANTHROPIC_DEFAULT_OPUS_MODEL=claude-opus-3-5-20240229 + +claude # Will use the models specified above +``` + +**Note:** Claude Code may cache the model from a previous session. If environment variables don't take effect, use Option 1 to explicitly set the model. + +**Important:** The `model_name` in your LiteLLM config must match what Claude Code requests (either from env vars or command line). + +### Using 1M Context Window + +Claude Code supports extended context (1 million tokens) using the `[1m]` suffix with Claude 4+ models: + +```bash +# Use Sonnet 4.5 with 1M context (requires quotes for shell) +claude --model 'claude-sonnet-4-5-20250929[1m]' + +# Inside a Claude Code session (no quotes needed) +/model claude-sonnet-4-5-20250929[1m] +``` + +**Important:** When using `--model` with `[1m]` in the shell, you must use quotes to prevent the shell from interpreting the brackets. + +Alternatively, set as default with environment variables: + +```bash +export ANTHROPIC_DEFAULT_SONNET_MODEL='claude-sonnet-4-5-20250929[1m]' +claude +``` + +**How it works:** +- Claude Code strips the `[1m]` suffix before sending to LiteLLM +- Claude Code automatically adds the header `anthropic-beta: context-1m-2025-08-07` +- Your LiteLLM config should **NOT** include `[1m]` in model names + +**Verify 1M context is active:** +```bash +/context +# Should show: 21k/1000k tokens (2%) +``` + +**Pricing:** Models using 1M context have different pricing. Input tokens above 200k are charged at a higher rate. + ## Troubleshooting Common issues and solutions: @@ -123,18 +181,25 @@ Common issues and solutions: - Ensure the `ANTHROPIC_AUTH_TOKEN` matches your LiteLLM master key **Model not found:** -- Ensure the model name in Claude Code matches exactly with your `config.yaml` -- Check LiteLLM logs for detailed error messages +- Check what model Claude Code is requesting in LiteLLM logs +- Ensure your `config.yaml` has a matching `model_name` entry +- If using environment variables, verify they're set: `echo $ANTHROPIC_DEFAULT_SONNET_MODEL` + +**1M context not working (showing 200k instead of 1000k):** +- Verify you're using the `[1m]` suffix: `/model your-model-name[1m]` +- Check LiteLLM logs for the header `context-1m-2025-08-07` in the request +- Ensure your model supports 1M context (only certain Claude models do) +- Your LiteLLM config should **NOT** include `[1m]` in the `model_name` ## Using Multiple Models and Providers -Expand your configuration to support multiple providers and models: +You can configure LiteLLM to route to any supported provider. Here's an example with multiple providers: ```yaml model_list: # OpenAI models - model_name: codex-mini - litellm_params: + litellm_params: model: openai/codex-mini api_key: os.environ/OPENAI_API_KEY api_base: https://api.openai.com/v1 @@ -156,7 +221,7 @@ model_list: litellm_params: model: anthropic/claude-3-5-sonnet-20241022 api_key: os.environ/ANTHROPIC_API_KEY - + - model_name: claude-3-5-haiku-20241022 litellm_params: model: anthropic/claude-3-5-haiku-20241022 @@ -174,19 +239,54 @@ litellm_settings: master_key: os.environ/LITELLM_MASTER_KEY ``` +**Note:** The `model_name` can be anything you choose. Claude Code will request whatever model you specify (via env vars or command line), and LiteLLM will route to the `model` configured in `litellm_params`. + Switch between models seamlessly: ```bash -# Use Claude for complex reasoning -claude --model claude-3-5-sonnet-20241022 +# Use environment variables to set defaults +export ANTHROPIC_DEFAULT_SONNET_MODEL=claude-3-5-sonnet-20241022 +export ANTHROPIC_DEFAULT_HAIKU_MODEL=claude-3-5-haiku-20241022 -# Use Haiku for fast responses -claude --model claude-3-5-haiku-20241022 - -# Use Bedrock deployment -claude --model claude-bedrock +# Or specify directly +claude --model claude-3-5-sonnet-20241022 # Complex reasoning +claude --model claude-3-5-haiku-20241022 # Fast responses +claude --model claude-bedrock # Bedrock deployment ``` +## Default Models Used by Claude Code + +If you **don't** set environment variables, Claude Code uses these default model names: + +| Purpose | Default Model Name (v2.1.14) | +|---------|------------------------------| +| Main model | `claude-sonnet-4-5-20250929` | +| Light tasks (subagents, summaries) | `claude-haiku-4-5-20251001` | +| Planning mode | `claude-opus-4-5-20251101` | + +Your LiteLLM config should include these model names if you want Claude Code to work without setting environment variables: + +```yaml +model_list: + - model_name: claude-sonnet-4-5-20250929 + litellm_params: + # Can be any provider - Anthropic, Bedrock, Vertex AI, etc. + model: anthropic/claude-sonnet-4-5-20250929 + api_key: os.environ/ANTHROPIC_API_KEY + + - model_name: claude-haiku-4-5-20251001 + litellm_params: + model: anthropic/claude-haiku-4-5-20251001 + api_key: os.environ/ANTHROPIC_API_KEY + + - model_name: claude-opus-4-5-20251101 + litellm_params: + model: anthropic/claude-opus-4-5-20251101 + api_key: os.environ/ANTHROPIC_API_KEY +``` + +**Warning:** These default model names may change with new Claude Code versions. Check LiteLLM proxy logs for "model not found" errors to identify what Claude Code is requesting. + ## Additional Resources - [LiteLLM Documentation](https://docs.litellm.ai/) diff --git a/docs/my-website/docs/tutorials/claude_responses_api.md b/docs/my-website/docs/tutorials/claude_responses_api.md index 6b681d93a8..03ac9935fd 100644 --- a/docs/my-website/docs/tutorials/claude_responses_api.md +++ b/docs/my-website/docs/tutorials/claude_responses_api.md @@ -37,18 +37,22 @@ Create a secure configuration using environment variables: ```yaml model_list: - # Claude models - - model_name: claude-3-5-sonnet-20241022 + # Configure the models you want to use + - model_name: claude-sonnet-4-5-20250929 litellm_params: - model: anthropic/claude-3-5-sonnet-20241022 - api_key: os.environ/ANTHROPIC_API_KEY - - - model_name: claude-3-5-haiku-20241022 - litellm_params: - model: anthropic/claude-3-5-haiku-20241022 + model: anthropic/claude-sonnet-4-5-20250929 + api_key: os.environ/ANTHROPIC_API_KEY + + - model_name: claude-haiku-4-5-20251001 + litellm_params: + model: anthropic/claude-haiku-4-5-20251001 + api_key: os.environ/ANTHROPIC_API_KEY + + - model_name: claude-opus-4-5-20251101 + litellm_params: + model: anthropic/claude-opus-4-5-20251101 api_key: os.environ/ANTHROPIC_API_KEY - litellm_settings: master_key: os.environ/LITELLM_MASTER_KEY ``` @@ -60,6 +64,10 @@ export ANTHROPIC_API_KEY="your-anthropic-api-key" export LITELLM_MASTER_KEY="sk-1234567890" # Generate a secure key ``` +:::tip +Alternatively, you can store `ANTHROPIC_API_KEY` in a `.env` file in your proxy directory. LiteLLM will automatically load it when starting. +::: + ### 2. Start proxy ```bash @@ -111,15 +119,55 @@ export ANTHROPIC_AUTH_TOKEN="$LITELLM_MASTER_KEY" ### 5. Use Claude Code -Start Claude Code and it will automatically use your configured models: +Start Claude Code with the model you want to use: ```bash -# Claude Code will use the models configured in your LiteLLM proxy -claude +# Specify model at startup +claude --model claude-sonnet-4-5-20250929 -# Or specify a model if you have multiple configured -claude --model claude-3-5-sonnet-20241022 -claude --model claude-3-5-haiku-20241022 +# Or specify a different model +claude --model claude-haiku-4-5-20251001 +claude --model claude-opus-4-5-20251101 + +# Or change model during a session +claude +/model claude-sonnet-4-5-20250929 +``` + +Alternatively, set default models with environment variables: + +```bash +export ANTHROPIC_DEFAULT_SONNET_MODEL=claude-sonnet-4-5-20250929 +export ANTHROPIC_DEFAULT_HAIKU_MODEL=claude-haiku-4-5-20251001 +export ANTHROPIC_DEFAULT_OPUS_MODEL=claude-opus-4-5-20251101 +claude +``` + +### Using 1M Context Window + +Claude Code supports extended context (1 million tokens) using the `[1m]` suffix: + +```bash +# Use Sonnet with 1M context (requires quotes in shell) +claude --model 'claude-sonnet-4-5-20250929[1m]' + +# Inside a Claude Code session (no quotes needed) +/model claude-sonnet-4-5-20250929[1m] +``` + +:::warning +**Important:** When using `--model` with `[1m]` in the shell, you must use quotes to prevent the shell from interpreting the brackets. +::: + +**How it works:** +- Claude Code strips the `[1m]` suffix before sending to LiteLLM +- Claude Code automatically adds the header `anthropic-beta: context-1m-2025-08-07` +- Your LiteLLM config should **NOT** include `[1m]` in model names + +**Verify 1M context is active:** +```bash +/context +# Should show: 21k/1000k tokens (2%) ``` Example conversation: @@ -140,6 +188,7 @@ Common issues and solutions: **Model not found:** - Ensure the model name in Claude Code matches exactly with your `config.yaml` +- Use `--model` flag or environment variables to specify the model - Check LiteLLM logs for detailed error messages ## Using Bedrock/Vertex AI/Azure Foundry Models