History

user 5bafa8b3a2 Drop dep bumps + black-26 reformat to clear fork CI policy PR was blocked by .github/workflows/guard-fork-dependencies.yml: fork PRs cannot modify uv.lock. Reverting: - uv.lock + pyproject.toml black bump (24.10.0 -> 26.3.1) and the 295 files of mechanical Black 26 reformat coupled to it - pyproject.toml diskcache extra change (kept the runtime mitigation in litellm/caching/disk_cache.py via JSONDisk) Kept: - Dockerfile cache narrowing (drops ~660 MB of uv build cache that surfaced cached setuptools as CVE findings) - litellm/caching/disk_cache.py: dc.JSONDisk to neutralize CVE-2025-69872 - ui/litellm-dashboard/package-lock.json + litellm-js/spend-logs/package-lock.json: next/postcss/hono/uuid CVE bumps (these are not blocked by the fork guard) - tests/test_litellm/caching/test_disk_cache.py - tests/code_coverage_tests/liccheck.ini: harmless black authorization Black + gitpython + langchain dep upgrades will need a follow-up from a maintainer pushing a branch in the canonical BerriAI/litellm repo. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>		2026-05-07 23:04:52 +00:00
..
config.example.yaml	[Feat] Add xAI /realtime API Support - works with LiveKitSDK (#20381 )	2026-02-03 19:58:28 -08:00
main.py	Drop dep bumps + black-26 reformat to clear fork CI policy	2026-05-07 23:04:52 +00:00
README.md	[Feat] Add xAI /realtime API Support - works with LiveKitSDK (#20381 )	2026-02-03 19:58:28 -08:00
requirements.txt	[Feat] Add xAI /realtime API Support - works with LiveKitSDK (#20381 )	2026-02-03 19:58:28 -08:00

README.md

LiveKit Voice Agent with LiteLLM Gateway

Simple example showing how to use LiveKit's xAI realtime plugin with LiteLLM as a proxy. This lets you switch between xAI, OpenAI, and Azure realtime APIs without changing your code.

Quick Start

1. Install dependencies

pip install livekit-agents[xai] websockets

2. Start LiteLLM proxy

# With xAI
export XAI_API_KEY="your-xai-key"
litellm --config config.yaml --port 4000

3. Run the voice agent

python main.py

Type your message and get a voice response from Grok!

Configuration

Set these environment variables if needed:

export LITELLM_PROXY_URL="http://localhost:4000"
export LITELLM_API_KEY="sk-1234"
export LITELLM_MODEL="grok-voice-agent"

Or use the defaults - connects to http://localhost:4000 by default.

Example Config File

Create a config.yaml with your realtime models:

model_list:
  - model_name: grok-voice-agent
    litellm_params:
      model: xai/grok-2-vision-1212
      api_key: os.environ/XAI_API_KEY
    model_info:
      mode: realtime

  - model_name: openai-voice-agent
    litellm_params:
      model: gpt-4o-realtime-preview
      api_key: os.environ/OPENAI_API_KEY
    model_info:
      mode: realtime

general_settings:
  master_key: sk-1234

Then start: litellm --config config.yaml --port 4000

How It Works

LiveKit's xAI plugin connects through LiteLLM proxy by setting base_url:

from livekit.plugins import xai

model = xai.realtime.RealtimeModel(
    voice="ara",
    api_key="sk-1234",              # LiteLLM proxy key
    base_url="http://localhost:4000", # Point to LiteLLM
)

Switching Providers

Just change the model in your config - no code changes needed:

xAI Grok:

model: xai/grok-2-vision-1212

OpenAI:

model: gpt-4o-realtime-preview

Azure OpenAI:

model: azure/gpt-4o-realtime-preview
api_base: https://your-endpoint.openai.azure.com/

Why Use LiteLLM?

✅ Switch providers without changing agent code
✅ Cost tracking across all voice sessions
✅ Rate limiting and budgets
✅ Load balancing across multiple API keys
✅ Fallbacks to backup models