LiteLLMsupply chain securityPyPIAI gatewayincident response

LiteLLM 1.107.0 Compromise: How to Check, Clean Up and Rotate Keys

person
LaunchBoosts Research Desk·AI-assisted research
8 min read

Researched and drafted with AI assistance from the 6 public sources listed at the end of this article, then published after automated editorial checks. Spotted an error? Tell us at support@launchboosts.com.

LiteLLM 1.107.0 Compromise: How to Check, Clean Up and Rotate Keys — LaunchBoosts

On October 21, 2026, a malicious build of LiteLLM, version 1.107.0, went out on PyPI and in the project's official Docker images. It installed a credential stealer and an hourly backdoor that run whenever Python starts. If you installed litellm==1.107.0, or pulled latest, main-latest, main-stable or v1.107.0 during a window of roughly two to three hours that day, treat the host as compromised. Remove the planted files without starting Python, rebuild from a clean image or environment pinned to 1.106.2, and rotate every secret that machine could reach.

LiteLLM is one of the most common ways teams put a single API in front of OpenAI, Anthropic, Gemini and other providers. That makes its hosts unusually valuable: they hold provider keys for the whole company. Below is what happened, how to check whether you were exposed, how to clean up in the right order, and what to change in your own pipeline so the next poisoned dependency does less damage.

What happened, in order

LiteLLM's official incident report gives a detailed timeline. All times are UTC on October 21, 2026:

Time (UTC) Event
10:08 Branch protection rejects an unauthorized change to the private release workflow. Five seconds later, a privileged developer credential overrides the protection
10:46 The modified workflow runs a staging build from the lookalike repository Berri-AI/litellm
11:11 A second developer's credential passes the production approval gate
11:39–11:40 Seven malicious litellm==1.107.0 wheels are uploaded to PyPI
11:46 Container publication runs; the Docker images contain the malware
11:53–11:54 A third developer credential deletes workflow run records, which limits the logs available
by 13:23 PyPI removes 1.107.0
by 13:51 Affected GHCR tags are unavailable
14:20 All affected Docker Hub tags are removed or repointed to clean images

Third-party write-ups give slightly different times. Datadog Security Labs puts the PyPI quarantine at 13:20 and the Docker removal at 13:53. StepSecurity says BerriAI confirmed the images were gone at 14:32. To be safe, use the widest window: anything installed from PyPI between 11:39 and 13:23, and any Docker pull between 11:46 and about 14:30.

StepSecurity also found that this was planned in advance. The litellm.dev exfiltration domain was registered on October 13, and the attacker's GitHub dead-drop repository was created on October 18.

How the attacker got in

The preliminary root cause in LiteLLM's report is a chain of failures:

  1. GitHub CLI OAuth tokens belonging to developers were exposed. The report says Claude Code can include a user's GitHub CLI OAuth token in a response body, and those tokens could end up in LiteLLM's logs when response bodies were logged.
  2. The attacker reached a user account on an internal LiteLLM deployment that was exposed to the public internet. The team suspects a recently patched vulnerability was the entry point but has not confirmed it.
  3. With one GitHub organization owner's token, the attacker disabled branch protections and pointed the release workflow at a lookalike repository. That name differs from the real BerriAI/litellm by one hyphen.
  4. The official pipeline then built, signed and published the poisoned artifacts.

The GitHub v1.107.0 tag itself is clean. Only the built artifacts were malicious. In The Hacker News' report, LiteLLM's director of security, Oliver Jensen, says the team has not yet determined how the tokens were first obtained. LiteLLM revoked the tokens involved and temporarily removed all but one unaffected user from its GitHub organization.

This is LiteLLM's second supply-chain incident this year. Versions 1.82.7 and 1.82.8 were compromised in March 2026 in activity Datadog linked to TeamPCP. Datadog says no link between the two incidents has been established. LiteLLM had hardened its releases after March with a private release repo, trusted publishing, immutable Docker tags and signed images. Its report acknowledges that this attack beat several of those controls at once.

What the malware does

SafeDep's technical analysis shows that each malicious wheel drops three files into site-packages:

  • cook.pth (12 bytes, containing import cook). Python runs .pth import lines at interpreter startup, so the stealer runs in every Python process in that environment, even if nothing imports LiteLLM.
  • cook.py: the credential stealer, which StepSecurity calls "privcollect".
  • sitecustomize.py: also loaded at startup. It forks a detached daemon that checks an attacker-controlled GitHub file every hour for a domain, runs the stealer again, then downloads http://DOMAIN/codepy and executes it.

The stealer collects a lot. According to SafeDep, it takes:

  • SSH keys and AWS, Azure, gcloud, kubeconfig and Docker credentials
  • GitHub CLI tokens, Vault and Terraform credentials, and CI secrets
  • Browser passwords and history, and shell history
  • Private key files under $HOME
  • Config and tokens for AI coding tools, including Claude Code, Cursor, Windsurf, Copilot, Gemini CLI, Codex, Aider and VS Code's mcp.json
  • The environment variables of other processes, read through /proc/PID/environ
  • AWS instance-metadata role credentials

On LiteLLM proxy hosts it reads DATABASE_URL and dumps the tables that hold your LLM provider API keys. The data is encrypted with AES-256 plus RSA-4096 and POSTed to www.litellm[.]dev/healthcheck.

The dead-drop GitHub account returned 404 by 13:40 UTC, so the backdoor currently has nowhere to call home. But any host that ran the loader before then may have fetched and executed second-stage code. SafeDep could not recover what that code did.

Were you exposed? Check without starting Python

The most important detail in LiteLLM's remediation guidance is easy to miss: do not run pip or python in a suspect environment first. Every interpreter start runs the malware again. Use filesystem and container tooling instead.

  1. Search lockfiles and requirements. Grep for 1.107.0 next to litellm in requirements*.txt, poetry.lock, uv.lock and Pipfile.lock. Also check CI logs from October 21 for unpinned pip install litellm or litellm[proxy] runs.
  2. Search the filesystem for the planted files. On Linux or macOS: find / -name cook.pth -o -name "litellm-1.107.0.dist-info" 2>/dev/null. On Windows: Get-ChildItem C:\ -Recurse -Filter cook.pth -ErrorAction SilentlyContinue. Don't forget virtualenvs, conda environments, and CI runner caches.
  3. Check the host markers. /tmp/.pc_once_72129118c59f5636c0bb means the stealer finished an upload. /tmp/remote_code_loader.lock means the backdoor ran. lsof /tmp/remote_code_loader.lock shows a live loader process, as StepSecurity notes.
  4. Check images by digest, not tag. The Docker Hub and GHCR litellm index digest is sha256:1bbcb1443bc962e45f5aad53fe3d2c30600e0379715fb6c664cdd3ac126c339e. The report also lists separate digests for litellm-database and litellm-non_root. Run docker images --digests, and on Kubernetes list pod imageIDs and grep for these digests. Pods with imagePullPolicy: Always that restarted during the window are especially suspect.
  5. Check network logs. Look for traffic to litellm[.]dev, requests carrying X-Token: XWiA8gZNa7NcX2XRoAEb, fetches of raw.githubusercontent.com/letters9813/, any path ending in /codepy, and Python processes calling 169.254.169.254. A /codepy hit means attacker code ran on that host.

The exposure is narrow. If you pin versions, run the LiteLLM Enterprise image from the private registry (which LiteLLM says was not affected), or didn't install or pull during the window, you are most likely fine. Confirm with steps 1 and 4 anyway.

Clean-up and rotation, in the right order

If you found a hit, follow this order, which combines LiteLLM's guidance with StepSecurity's and Datadog's:

  1. Contain. Cut the host's outbound network access if you can, and block litellm.dev at DNS or egress.
  2. Kill the daemon. Kill the process holding /tmp/remote_code_loader.lock. It is a detached Python process with parent PID 1 and working directory /.
  3. Delete the planted files. Remove cook.pth, cook.py, sitecustomize.py and the matching __pycache__/cook*.pyc and sitecustomize*.pyc from every affected site-packages. Uninstalling litellm does not remove them.
  4. Rebuild, don't patch in place. Recreate virtualenvs and container images from scratch, pinned to litellm==1.106.2 (or the v1.106.2 image by digest). Purge 1.107.0 from internal PyPI mirrors, Artifactory or registry caches, and CI caches.
  5. Rotate from a clean machine. Assume everything the host could read is gone. LiteLLM's report says to assume exfiltration if the host had outbound internet access, even without the marker file.
  6. Audit for use. In AWS, LiteLLM suggests checking CloudTrail for GetCallerIdentity calls from the instance role around install time, and for the role's credentials being used from outside your infrastructure. Do the same in your LLM providers' usage dashboards: unexplained spikes on a rotated key are a strong signal.

A note on version advice: LiteLLM, StepSecurity, Datadog and The Hacker News all recommend 1.106.2, and LiteLLM's report says any explicitly pinned version published before the incident is safe. Some vendor posts name other versions. Follow the official report.

What to rotate first

Priority Secret Why
1 LLM provider keys in the LiteLLM proxy DB, plus the proxy master key, virtual keys and salt key Directly targeted; billable and abusable within minutes
1 Cloud IAM role or task credentials, and static cloud keys Taken via instance metadata and config files; lead to wider access
2 DATABASE_URL password and any secret in any process's environment on the host The stealer reads other processes' environments
2 GitHub CLI and PAT tokens, CI/CD, Vault, Kubernetes and registry tokens Tokens like these started this incident; they enable the next supply-chain hop
3 SSH, GPG and TLS private keys Collected from disk
3 AI coding tool tokens and MCP configs, plus browser-saved passwords on developer laptops Explicitly targeted on workstations

What builders should change after this

This attack worked against a project that had already hardened its releases. Your own defenses need to assume that signed, official artifacts can still be malicious.

  • Pin by hash and by digest. Use lockfiles with hashes (pip install --require-hashes, uv.lock, poetry.lock) and reference container images by @sha256: digest. Floating tags like latest and main-stable are how many people got hit here.
  • Add a release-age delay. A compromised release that gets pulled within hours can't reach you if your tooling refuses packages newer than a few days. Resolvers like uv support an exclude-newer cutoff, and many internal mirrors can enforce a quarantine period.
  • Restrict egress from AI gateways. A LiteLLM proxy needs to reach your model providers and its database, not arbitrary domains. An egress allowlist would have stopped both the exfiltration and the /codepy download.
  • Lock down instance metadata. Require IMDSv2 and set the hop limit to 1 so containers can't easily pick up the node's role credentials.
  • Stop logging full LLM request and response bodies, or redact them. The root cause here began with tokens showing up in gateway logs. Agent traffic can carry credentials, so treat prompt and response logs as secret stores. Redact, keep them for a short time, and never put them on an internet-facing internal tool.
  • Treat developer CLI tokens as production credentials. Use short-lived tokens, give as few people org-owner rights as possible, and require approval rules that one person holding two compromised accounts can't satisfy. Datadog also recommends locking release workflows to approved repositories.
  • Watch build provenance. Datadog notes that the provenance recorded the lookalike fork as the source. Checking an attestation's source repository, not just whether a signature exists, catches exactly this pattern.

If you're re-evaluating your gateway or agent tooling after this, the AI coding assistants and AI tools directory are a starting point. Whatever you choose, the same pinning and egress rules apply.

What to watch next

LiteLLM's report says the investigation is ongoing, with fuller indicator-of-compromise guidance still to come. Watch the incident page and GitHub issue #47826 for three things: confirmation of how the developer tokens were first stolen, whether the internet-exposed deployment was breached through a known CVE, and any recovered second-stage payload.

For now:

  1. Search for cook.pth and the bad image digests today.
  2. If you find a hit, rebuild on 1.106.2 and rotate provider keys and cloud credentials first.
  3. Then put hash pinning, digest pinning and egress allowlists on your AI gateway so the next poisoned release does less damage.

For more on keeping a startup's AI stack safe and affordable, see the LaunchBoosts blog.

Frequently asked questions

Which LiteLLM versions are compromised?

The October 21, 2026 incident affected litellm==1.107.0 on PyPI (all seven wheels) and the Docker tags v1.107.0, latest, main-latest and main-stable on Docker Hub and GHCR during the exposure window. LiteLLM says all earlier versions are safe and recommends pinning 1.106.2. Separately, versions 1.82.7 and 1.82.8 were compromised in an earlier incident in March 2026.

I never import litellm in my code. Can I still be affected?

Yes. The malicious wheels add cook.pth and sitecustomize.py to site-packages, and Python runs both automatically whenever any interpreter starts in that environment. If 1.107.0 was installed anywhere in the environment, any Python process there could have run the stealer, even one that never touches LiteLLM.

Is uninstalling LiteLLM 1.107.0 enough?

No. Removing the litellm package does not delete the planted files, and running pip or python in that environment runs the malware again. LiteLLM recommends deleting cook.pth, cook.py, sitecustomize.py and their cached .pyc files, killing the background daemon, and preferably rebuilding the environment or image from scratch. Then rotate every credential the host could reach.

Are LiteLLM Enterprise customers affected?

According to LiteLLM's incident report, enterprise customers pulling from LiteLLM's private registry were not affected, and the private enterprise release was not touched. Anyone who installed the public PyPI package or pulled the public Docker tags during the window should follow the public remediation steps.

How do I know whether data was actually stolen?

A marker file at /tmp/.pc_once_72129118c59f5636c0bb means the stealer completed an upload, but LiteLLM warns that a missing marker does not prove the host is clean. If 1.107.0 ran on a host with outbound internet access, assume its credentials were taken. Look for outbound traffic to litellm[.]dev and any request ending in /codepy, which would mean second-stage code ran.

Sources

  1. LiteLLM official incident report: Oct 21, 2026 release v1.107.0 compromised— docs.litellm.ai
  2. SafeDep: Compromised LiteLLM 1.107.0 on PyPI installs a credential stealer and backdoor— safedep.io
  3. StepSecurity: litellm 1.107.0 compromised on PyPI – backdoored wheels and hourly remote code loader— stepsecurity.io
  4. The Hacker News: LiteLLM compromised in supply chain attack delivering credential stealer and backdoor— thehackernews.com
  5. Datadog Security Labs: LiteLLM supply chain compromise – what you need to know— securitylabs.datadoghq.com
  6. GitHub issue BerriAI/litellm #47826: security report on compromised 1.107.0— github.com
person

LaunchBoosts Research Desk

AI-assisted research

Explainers on software and AI industry trends, drafted with AI assistance from the public sources cited in each article and published after automated editorial checks for length, independent sourcing and originality.