- Rust 81.6%
- Swift 7.4%
- Shell 7.3%
- TypeScript 3.3%
- CSS 0.3%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
|
|
||
| .github/workflows | ||
| docs | ||
| macos/RelayBarGlass | ||
| mdm | ||
| scripts | ||
| src | ||
| tests | ||
| ui | ||
| .dockerignore | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| Dockerfile | ||
| LICENSE | ||
| README.md | ||
LiteLLM Relay
LiteLLM Relay is a proxy you install on employee machines through your MDM. It does two things.
First, it sets up your developers' AI tools for them. On install, Relay auto-detects the AI tools already on the machine — Claude Desktop, Claude Code, and Codex (CLI, VS Code, and the macOS app) — and wires each one to your LiteLLM AI Gateway automatically. There is no per-tool opt-in: if Relay recognizes a tool, it routes it through the Gateway. Developers just launch the tool and sign in with their corporate identity, with no provider API key and no manual setup. See Auto-configuration and the AI Tool Guides below.
Second, it catches shadow AI. Relay detects AI traffic from tools like Notion AI, Perplexity, and OpenClaw and routes it to the Gateway too, making it a single pane of glass for all AI usage in your company.
Usage
- Install LiteLLM Relay on all your employee devices, using supported MDM
- Employees use AI tools as they normally would, such as Notion AI.
- Every request, response, and usage event is captured in LiteLLM.
Auto-configuration
Relay is opt-out, not opt-in. When you install it (or run relay setup), Relay
scans the device for supported AI tools and wires every one it finds to the
Gateway in a single pass — you never enumerate tools per machine.
| Tool | Detected by | Config Relay writes |
|---|---|---|
| Claude Code CLI | claude on PATH or ~/.claude |
~/.claude/settings.json |
| Claude Desktop | /Applications/Claude.app or its app-support dir |
/Library/Managed Preferences/com.anthropic.claudefordesktop.plist on macOS, /etc/claude-desktop/managed-settings.json on Linux |
| Codex (CLI, VS Code, macOS app) | codex on PATH, Codex.app, the openai.chatgpt VS Code extension, or ~/.codex |
~/.codex/config.toml |
Detection also runs on a schedule, so a tool installed after Relay gets wired
to the Gateway automatically — no manual re-run. Because each tool's config
lives in a different place, the installer registers two periodic jobs (both at
login and every RELAY_AUTOCONFIGURE_INTERVAL seconds, default 3600):
| Job | Runs as | Tools | Why |
|---|---|---|---|
ai.litellm.relay.autoconfigure (LaunchAgent) |
you | Claude Code, Codex | configs are user-writable (~/.claude, ~/.codex) |
ai.litellm.relay.autoconfigure-desktop (LaunchDaemon) |
root | Claude Desktop | its managed plist lives in the root-owned /Library/Managed Preferences |
The root daemon runs a root-owned copy of the Relay binary at
/Library/PrivilegedHelperTools/ai.litellm.relay.autoconfigure-desktop, which
each install refreshes, and pins HOME to the installing user so it reads that
user's Relay config while running as root. It also runs whenever /Library/Managed Preferences
changes: on an MDM-enrolled Mac a managed-preferences refresh (login, a profile
push) regenerates that directory from the installed profiles and drops the plist,
and the daemon writes it back within seconds, leaving it alone when it is already
current. Installing it needs root; install.sh uses sudo
when not already root, and the macOS .pkg postinstall runs install.sh as the
console user, so there the daemon is installed only when that user has
passwordless sudo. If it can't get root, Claude Code and Codex still
auto-configure and Relay prints a warning for Claude Desktop.
On macOS Claude Code, Claude Desktop, and Codex get their credential from the
Relay daemon, so relay onboard, relay onboard-codex,
relay onboard-claude-desktop, the setup wizard, and each auto-configure pass
also install and start the ai.litellm.relay LaunchAgent when no daemon is
answering. A default install needs no --background for that, and on a
--background install the scheduled pass brings the daemon back after you stop
it unless ai.litellm.relay.autoconfigure is stopped too; see
Credential broker.
You can still run detection on demand, and scope it with --only:
relay autoconfigure # every detected tool
relay autoconfigure --only claude-desktop # just one (repeatable)
Unset flags fall back to the saved Relay config, so a managed config.yaml
seeded by your MDM is enough to configure a device with no arguments. Pass
--team, --api-key, or the --oidc-* flags (--oidc-issuer,
--oidc-client-id, --oidc-scopes, --oidc-redirect-port) to override.
Only detected tools are touched, and one tool failing never blocks the others.
Pass --skip-autoconfigure to install.sh to disable it.
Before it writes a static Gateway key into any tool, autoconfigure verifies
that key against the Gateway with GET /v1/models. If the Gateway rejects it
with 401 (for example the session credential from relay setup has expired) or
cannot be reached, the run leaves every tool config untouched, prints
Run litellm-relay setup to sign in again, and exits non-zero, so the scheduled
job fails visibly instead of rewriting a dead key every hour. The one exception
is a saved key that is only Claude Desktop's fallback behind the IdP exchange:
the run reports the refusal, keeps configuring, and Claude Desktop gets the
exchanged credential or nothing, never the refused key. A key the Gateway
authenticates but answers 403 for (its allowed_routes or key_type leaves out
/v1/models) is still written, with a warning that quotes the Gateway. The
check only runs once a supported tool is detected. When the Gateway reported an
expiry at sign-in, the run also warns once the credential is within 24 hours of
it. The same check is exposed on /api/status as a credential block
(configured says whether a Gateway key is saved; state is valid,
restricted, rejected, unverifiable, missing, or unchecked when a key
is saved but no check has finished yet; expiry is unknown, ok,
expiring_soon, or expired; plus detail, checked_at, enrolled_at, and
expires_at), and the dashboard shows it as the Gateway credential row.
While the daemon holds an IdP session, the block describes that session
instead (configured true, state valid, and expires_at the expiry of the
key the daemon minted). The
status endpoint answers at once with the last check, which is up to 60 seconds
old, and refreshes it in the background, so the first call after serve
starts reads unchecked until that check finishes (at most 10 seconds)
Developers see their own usage, locally
Relay also ships an optional macOS menu bar app, RelayBar, that sits on top of the daemon: it shows whether the developer is signed in and as whom, how long the key every coding tool is using stays valid and that the daemon renews it, the selected team's spend against its budget as the Gateway reports it, the MCP servers the daemon serves, and a Team and an Environment picker that move Claude Code, Codex, and the local proxy at once through relay switch-team and relay switch-environment (Claude Desktop's requests fail after a switch until it restarts, and the restart moves it to the new Gateway only where the .pkg or a sudo install added the root Claude Desktop agent). Build it from macos/RelayBarGlass (./build.sh && open RelayBarGlass.app) or ship it in the .pkg with scripts/build-macos-pkg.sh --relaybar.
AI Tool Guides
Relay onboards each AI coding tool onto the LiteLLM AI Gateway with zero developer setup — the developer just launches the tool. Pick your tool for the step-by-step guide:
Supported MDMs
Deploy LiteLLM Relay with your existing device-management process:
- Jamf
- Microsoft Intune
- Kandji
- Mosyle
- VMware Workspace ONE
- Addigy
- Custom shell scripts or internal deployment workflows
See the MDM rollout guide for the deployable .pkg, the PAC
configuration profile, and step-by-step Jamf and Intune runbooks.
Features
- Detect shadow AI usage across employee devices and company traffic sources
- Route AI traffic through LiteLLM AI Gateway for central visibility
- Log AI activity from desktop apps, browser AI, coding tools, agents, MCP clients, and LLM APIs
- Apply one set of Gateway controls for audit, access, provider routing, and policy
Relay does not log cookies or authorization headers. Payload previews are truncated and headers are redacted.
The dashboard's event table is the proxy's metadata log: CONNECT tunnels and
their attribution, optional payload captures, shadow completions, and
collector runs. Requests a configured tool sends to Relay's local inference
endpoint (ANTHROPIC_BASE_URL, Codex's base_url) are forwarded to the
Gateway with the developer's credential and are accounted there, in the
Gateway logs and the menu bar usage card, not as rows in this table.
The dashboard's data calls under /api/ answer a page opened at an IP address,
localhost, relay.host, or this Mac's own name (<name> or <name>.local),
on any port, so http://0.0.0.0:4142 and a Docker port mapping keep working.
Any other Host (a page that rebinds its own domain to the Mac), a request
whose Origin is another site, and a browser request whose Sec-Fetch-Site is
anything but same-origin or none (such as an image another page embeds) get
a 403.
Install
Relay has no tagged release yet, so install it from GitHub. The installer
builds Relay from source, so you need Rust (cargo)
installed first. Relay currently supports macOS.
Clone the repository and run the installer:
git clone https://github.com/LiteLLM-Labs/litellm-relay.git
cd litellm-relay
./src/install.sh
Or install directly from main without cloning:
curl -fsSL https://raw.githubusercontent.com/LiteLLM-Labs/litellm-relay/main/src/install.sh | \
RELAY_ALLOW_UNPINNED_MAIN=1 bash
The installer builds the relay command, adds it to your PATH, and trusts
the local Relay CA so AI app payloads can be captured; macOS asks for your
account password in a Certificate Trust Settings sheet for that step. Pass
--skip-trust-ca to install without trusting the CA. A managed install
(--config-file or --skip-setup, which is what the .pkg runs) trusts the
CA only when its config sets capture.payloads: true, so a fleet on the
default metadata-only capture never sees that sheet; see
CA trust on managed devices.
Then open a new terminal and run:
relay
Once tagged releases are published, production deployments will be able to pin a version tag and verify the source archive checksum:
curl -fsSL https://raw.githubusercontent.com/LiteLLM-Labs/litellm-relay/main/src/install.sh | \
RELAY_VERSION=vX.Y.Z \
RELAY_SHA256=<release-tarball-sha256> \
bash