From 30565581be0e0b9d407036354f7cf3192d576f3a Mon Sep 17 00:00:00 2001 From: Yuneng Jiang Date: Mon, 6 Apr 2026 22:53:23 -0700 Subject: [PATCH] [Infra] Pin cosign.pub verification to initial commit hash Pin all cosign public key references to the immutable commit hash (0112e53) that first introduced the key, instead of fetching it from the release tag. This addresses the concern that an attacker with push access could replace the key on main/tags and re-sign tampered images. Docs now show two verification methods: commit hash (recommended) and release tag (convenience), with explanation of why the hash is stronger. Co-Authored-By: Claude Opus 4.6 (1M context) --- .github/workflows/create-release.yml | 16 +++++++++++- README.md | 26 +++++++++++++++++++ .../blog/ci_cd_v2_improvements/index.md | 16 +++++++++++- .../blog/security_townhall_updates/index.md | 16 +++++++++++- .../blog/security_update_march_2026/index.md | 16 +++++++++++- docs/my-website/docs/proxy/deploy.md | 16 +++++++++++- 6 files changed, 101 insertions(+), 5 deletions(-) diff --git a/.github/workflows/create-release.yml b/.github/workflows/create-release.yml index 2ae01823a9..b863397985 100644 --- a/.github/workflows/create-release.yml +++ b/.github/workflows/create-release.yml @@ -48,7 +48,21 @@ jobs: const cosignSection = [ `## Verify Docker Image Signature`, ``, - `All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). To verify the integrity of an image before deploying:`, + `All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit \`0112e53\`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).`, + ``, + `**Verify using the pinned commit hash (recommended):**`, + ``, + `A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:`, + ``, + '```bash', + `cosign verify \\`, + ` --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \\`, + ` ghcr.io/berriai/litellm:${tag}`, + '```', + ``, + `**Verify using the release tag (convenience):**`, + ``, + `Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:`, ``, '```bash', `cosign verify \\`, diff --git a/README.md b/README.md index 7d910617f7..d7b8bad69f 100644 --- a/README.md +++ b/README.md @@ -404,6 +404,32 @@ Support for more providers. Missing a provider or LLM Platform, raise a [feature 2. Install dependencies `npm install` 3. Run `npm run dev` to start the dashboard +# Verify Docker Image Signatures + +All LiteLLM Docker images published to GHCR are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). + +**Verify using the pinned commit hash (recommended):** + +A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: + +```bash +cosign verify \ + --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ + ghcr.io/berriai/litellm: +``` + +**Verify using a release tag (convenience):** + +Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: + +```bash +cosign verify \ + --key https://raw.githubusercontent.com/BerriAI/litellm//cosign.pub \ + ghcr.io/berriai/litellm: +``` + +Replace `` with the version you are deploying (e.g. `v1.83.0-stable`). + # Enterprise For companies that need better security, user management and professional support diff --git a/docs/my-website/blog/ci_cd_v2_improvements/index.md b/docs/my-website/blog/ci_cd_v2_improvements/index.md index fb9a6609c9..8558114396 100644 --- a/docs/my-website/blog/ci_cd_v2_improvements/index.md +++ b/docs/my-website/blog/ci_cd_v2_improvements/index.md @@ -31,7 +31,21 @@ Building on the roadmap from our [security incident](https://docs.litellm.ai/blo ## Verify Docker image signatures -Starting from `v1.83.0-nightly`, all LiteLLM Docker images published to GHCR are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). To verify the integrity of an image before deploying: +Starting from `v1.83.0-nightly`, all LiteLLM Docker images published to GHCR are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). + +**Verify using the pinned commit hash (recommended):** + +A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: + +```bash +cosign verify \ + --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ + ghcr.io/berriai/litellm: +``` + +**Verify using a release tag (convenience):** + +Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ diff --git a/docs/my-website/blog/security_townhall_updates/index.md b/docs/my-website/blog/security_townhall_updates/index.md index 633e97df3e..39db096c53 100644 --- a/docs/my-website/blog/security_townhall_updates/index.md +++ b/docs/my-website/blog/security_townhall_updates/index.md @@ -147,7 +147,21 @@ We believe that [Cosign](https://github.com/sigstore/cosign) is a good fit for t #### How to verify a Docker image with Cosign -Starting from `v1.83.0-nightly`, all LiteLLM Docker images published to GHCR are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). To verify the integrity of an image before deploying: +Starting from `v1.83.0-nightly`, all LiteLLM Docker images published to GHCR are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key that was introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). + +**Verify using the pinned commit hash (recommended):** + +A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: + +```bash +cosign verify \ + --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ + ghcr.io/berriai/litellm: +``` + +**Verify using a release tag (convenience):** + +Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ diff --git a/docs/my-website/blog/security_update_march_2026/index.md b/docs/my-website/blog/security_update_march_2026/index.md index 628e26f0c6..6e7b77d1e4 100644 --- a/docs/my-website/blog/security_update_march_2026/index.md +++ b/docs/my-website/blog/security_update_march_2026/index.md @@ -710,7 +710,21 @@ The LiteLLM AI Gateway team has already taken the following steps: ## Verify Docker image signatures -Starting from `v1.83.0-nightly`, all LiteLLM Docker images published to GHCR are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). To verify the integrity of an image before deploying: +Starting from `v1.83.0-nightly`, all LiteLLM Docker images published to GHCR are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). + +**Verify using the pinned commit hash (recommended):** + +A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: + +```bash +cosign verify \ + --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ + ghcr.io/berriai/litellm: +``` + +**Verify using a release tag (convenience):** + +Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ diff --git a/docs/my-website/docs/proxy/deploy.md b/docs/my-website/docs/proxy/deploy.md index d4f02afbb1..4b087afd84 100644 --- a/docs/my-website/docs/proxy/deploy.md +++ b/docs/my-website/docs/proxy/deploy.md @@ -67,7 +67,21 @@ docker compose up ### Verify Docker image signatures -All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). You can verify the integrity of an image before deploying: +All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). + +**Verify using the pinned commit hash (recommended):** + +A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: + +```bash +cosign verify \ + --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ + ghcr.io/berriai/litellm: +``` + +**Verify using a release tag (convenience):** + +Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \