feat: Add abliteration.ai provider (#18678)
* feat: Add abliteration.ai provider * adding signoz integration to observability docs * Fixing build * Adding timeout for flaky test * Fixing e2e * add team member budget duration in team/update * Reusable Duration Select and update team member budget UI --------- Co-authored-by: Goutham Karthi <goutham@signoz.io> Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: YutaSaito <36355491+uc4w6c@users.noreply.github.com>
This commit is contained in:
Abliteration AI
GitHub
parent
8ce4eea88f
commit
dc4ce7c5a2
@ -262,6 +262,7 @@ Support for more providers. Missing a provider or LLM Platform, raise a [feature
|
||||
|
||||
| Provider | `/chat/completions` | `/messages` | `/responses` | `/embeddings` | `/image/generations` | `/audio/transcriptions` | `/audio/speech` | `/moderations` | `/batches` | `/rerank` |
|
||||
|-------------------------------------------------------------------------------------|---------------------|-------------|--------------|---------------|----------------------|-------------------------|-----------------|----------------|-----------|-----------|
|
||||
| [Abliteration (`abliteration`)](https://docs.litellm.ai/docs/providers/abliteration) | ✅ | | | | | | | | | |
|
||||
| [AI/ML API (`aiml`)](https://docs.litellm.ai/docs/providers/aiml) | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | |
|
||||
| [AI21 (`ai21`)](https://docs.litellm.ai/docs/providers/ai21) | ✅ | ✅ | ✅ | | | | | | | |
|
||||
| [AI21 Chat (`ai21_chat`)](https://docs.litellm.ai/docs/providers/ai21) | ✅ | ✅ | ✅ | | | | | | | |
|
||||
@ -455,4 +456,3 @@ All these checks must pass before your PR can be merged.
|
||||
<img src="https://contrib.rocks/image?repo=BerriAI/litellm" />
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
109
docs/my-website/docs/providers/abliteration.md
Normal file
109
docs/my-website/docs/providers/abliteration.md
Normal file
@ -0,0 +1,109 @@
|
||||
# Abliteration
|
||||
|
||||
## Overview
|
||||
|
||||
| Property | Details |
|
||||
|-------|-------|
|
||||
| Description | Abliteration provides an OpenAI-compatible `/chat/completions` endpoint. |
|
||||
| Provider Route on LiteLLM | `abliteration/` |
|
||||
| Link to Provider Doc | [Abliteration](https://abliteration.ai) |
|
||||
| Base URL | `https://api.abliteration.ai/v1` |
|
||||
| Supported Operations | [`/chat/completions`](#sample-usage) |
|
||||
|
||||
<br />
|
||||
|
||||
## Required Variables
|
||||
|
||||
```python showLineNumbers title="Environment Variables"
|
||||
os.environ["ABLITERATION_API_KEY"] = "" # your Abliteration API key
|
||||
```
|
||||
|
||||
## Sample Usage
|
||||
|
||||
```python showLineNumbers title="Abliteration Completion"
|
||||
import os
|
||||
from litellm import completion
|
||||
|
||||
os.environ["ABLITERATION_API_KEY"] = ""
|
||||
|
||||
response = completion(
|
||||
model="abliteration/abliterated-model",
|
||||
messages=[{"role": "user", "content": "Hello from LiteLLM"}],
|
||||
)
|
||||
|
||||
print(response)
|
||||
```
|
||||
|
||||
## Sample Usage - Streaming
|
||||
|
||||
```python showLineNumbers title="Abliteration Streaming Completion"
|
||||
import os
|
||||
from litellm import completion
|
||||
|
||||
os.environ["ABLITERATION_API_KEY"] = ""
|
||||
|
||||
response = completion(
|
||||
model="abliteration/abliterated-model",
|
||||
messages=[{"role": "user", "content": "Stream a short reply"}],
|
||||
stream=True,
|
||||
)
|
||||
|
||||
for chunk in response:
|
||||
print(chunk)
|
||||
```
|
||||
|
||||
## Usage with LiteLLM Proxy Server
|
||||
|
||||
1. Add the model to your proxy config:
|
||||
|
||||
```yaml showLineNumbers title="config.yaml"
|
||||
model_list:
|
||||
- model_name: abliteration-chat
|
||||
litellm_params:
|
||||
model: abliteration/abliterated-model
|
||||
api_key: os.environ/ABLITERATION_API_KEY
|
||||
```
|
||||
|
||||
2. Start the proxy:
|
||||
|
||||
```bash
|
||||
litellm --config /path/to/config.yaml
|
||||
```
|
||||
|
||||
## Direct API Usage (Bearer Token)
|
||||
|
||||
Use the environment variable as a Bearer token against the OpenAI-compatible endpoint:
|
||||
`https://api.abliteration.ai/v1/chat/completions`.
|
||||
|
||||
```bash showLineNumbers title="cURL"
|
||||
export ABLITERATION_API_KEY=""
|
||||
curl https://api.abliteration.ai/v1/chat/completions \
|
||||
-H "Authorization: Bearer ${ABLITERATION_API_KEY}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "abliterated-model",
|
||||
"messages": [{"role": "user", "content": "Hello from Abliteration"}]
|
||||
}'
|
||||
```
|
||||
|
||||
```python showLineNumbers title="Python (requests)"
|
||||
import os
|
||||
import requests
|
||||
|
||||
api_key = os.environ["ABLITERATION_API_KEY"]
|
||||
|
||||
response = requests.post(
|
||||
"https://api.abliteration.ai/v1/chat/completions",
|
||||
headers={
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
json={
|
||||
"model": "abliterated-model",
|
||||
"messages": [{"role": "user", "content": "Hello from Abliteration"}],
|
||||
},
|
||||
timeout=60,
|
||||
)
|
||||
|
||||
print(response.json())
|
||||
```
|
||||
@ -683,12 +683,13 @@ const sidebars = {
|
||||
"providers/bedrock_writer",
|
||||
"providers/bedrock_batches",
|
||||
"providers/aws_polly",
|
||||
"providers/bedrock_vector_store",
|
||||
]
|
||||
},
|
||||
"providers/litellm_proxy",
|
||||
"providers/ai21",
|
||||
"providers/aiml",
|
||||
"providers/bedrock_vector_store",
|
||||
]
|
||||
},
|
||||
"providers/litellm_proxy",
|
||||
"providers/abliteration",
|
||||
"providers/ai21",
|
||||
"providers/aiml",
|
||||
"providers/aleph_alpha",
|
||||
"providers/amazon_nova",
|
||||
"providers/anyscale",
|
||||
|
||||
@ -61,6 +61,10 @@
|
||||
"max_completion_tokens": "max_tokens"
|
||||
}
|
||||
},
|
||||
"abliteration": {
|
||||
"base_url": "https://api.abliteration.ai/v1",
|
||||
"api_key_env": "ABLITERATION_API_KEY"
|
||||
},
|
||||
"llamagate": {
|
||||
"base_url": "https://api.llamagate.dev/v1",
|
||||
"api_key_env": "LLAMAGATE_API_KEY",
|
||||
|
||||
@ -34,6 +34,23 @@
|
||||
}
|
||||
},
|
||||
"providers": {
|
||||
"abliteration": {
|
||||
"display_name": "Abliteration (`abliteration`)",
|
||||
"url": "https://docs.litellm.ai/docs/providers/abliteration",
|
||||
"endpoints": {
|
||||
"chat_completions": true,
|
||||
"messages": false,
|
||||
"responses": false,
|
||||
"embeddings": false,
|
||||
"image_generations": false,
|
||||
"audio_transcriptions": false,
|
||||
"audio_speech": false,
|
||||
"moderations": false,
|
||||
"batches": false,
|
||||
"rerank": false,
|
||||
"a2a": false
|
||||
}
|
||||
},
|
||||
"aiml": {
|
||||
"display_name": "AI/ML API (`aiml`)",
|
||||
"url": "https://docs.litellm.ai/docs/providers/aiml",
|
||||
|
||||
50
tests/litellm/llms/openai_like/test_abliteration_provider.py
Normal file
50
tests/litellm/llms/openai_like/test_abliteration_provider.py
Normal file
@ -0,0 +1,50 @@
|
||||
"""
|
||||
Unit tests for the Abliteration OpenAI-like provider.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(
|
||||
0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../.."))
|
||||
)
|
||||
|
||||
from litellm.llms.openai_like.dynamic_config import create_config_class
|
||||
from litellm.llms.openai_like.json_loader import JSONProviderRegistry
|
||||
|
||||
ABLITERATION_BASE_URL = "https://api.abliteration.ai/v1"
|
||||
|
||||
|
||||
def _get_config():
|
||||
provider = JSONProviderRegistry.get("abliteration")
|
||||
assert provider is not None
|
||||
config_class = create_config_class(provider)
|
||||
return config_class()
|
||||
|
||||
|
||||
def test_abliteration_provider_registered():
|
||||
provider = JSONProviderRegistry.get("abliteration")
|
||||
assert provider is not None
|
||||
assert provider.base_url == ABLITERATION_BASE_URL
|
||||
assert provider.api_key_env == "ABLITERATION_API_KEY"
|
||||
|
||||
|
||||
def test_abliteration_resolves_env_api_key(monkeypatch):
|
||||
config = _get_config()
|
||||
monkeypatch.setenv("ABLITERATION_API_KEY", "test-key")
|
||||
api_base, api_key = config._get_openai_compatible_provider_info(None, None)
|
||||
assert api_base == ABLITERATION_BASE_URL
|
||||
assert api_key == "test-key"
|
||||
|
||||
|
||||
def test_abliteration_complete_url_appends_endpoint():
|
||||
config = _get_config()
|
||||
url = config.get_complete_url(
|
||||
api_base=ABLITERATION_BASE_URL,
|
||||
api_key="test-key",
|
||||
model="abliteration/abliterated-model",
|
||||
optional_params={},
|
||||
litellm_params={},
|
||||
stream=False,
|
||||
)
|
||||
assert url == f"{ABLITERATION_BASE_URL}/chat/completions"
|
||||
Loading…
Reference in New Issue
Block a user