Shepherd Model Gateway CLI
The CLI runs a model gateway that routes requests across backends and executes MCP tools under configured policies.
- 701 ★ on the repository
- Last commit
- Apache-2.0
Broad access
Why this level
- Without a restrictive policy, connected MCP tools are allowed, including code execution when such a server is configured.
Install
Manual install
docker pull lightseekorg/smg:latestFirst Quick Start installation option: downloads the image without starting workers or the gateway.
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- Without a restrictive policy, connected MCP tools are allowed, including code execution when such a server is configured.
This is third-party code. Review the repository files before installing.
What it does
Shepherd Model Gateway, or SMG, routes application requests among model backends. Administrators connect self-hosted engines or cloud providers and select a balancing policy. Cache-aware routing tries to reuse previously processed prefixes. Applications can also execute connected MCP tools through SMG. The gateway acts as an MCP client, applies approval policies and returns results through the model API.
Who it is for. LLM platform engineers operating several model backends and centralizing tool execution.
Good fit when
- Several model workers need to share incoming traffic.
- An application needs one endpoint for local and cloud models.
- Tool execution needs per-server and per-action policies.
- Latency investigation needs request metrics and traces.
Not a fit when
- You need a model without configuring a backend or provider.
- Every MCP call must always produce an interactive approval prompt.
- You cannot operate the gateway and protect its credentials.
Example request
Prepare SMG for two test model workers and show a policy denying MCP tools until explicitly allowed.Limitations
The gateway does not supply the models; prepare inference workers or provider access first. MCP tools are allowed by default without a separate policy, so explicitly restrict them before connecting work services. Interactive approval is available through the Responses API; in other modes, the gateway allows or denies actions under configured policies. Cloud routes send requests to the selected provider even when conversation history is stored within your infrastructure.
How to disable. Stop the gateway and remove its endpoint from applications. Revoke unused provider and MCP credentials, and manage stored history separately.
FAQ
What does the first Docker command do?
It downloads the image. You still need to run the gateway and supply worker addresses.
Where does the example accept requests?
On local port 30000; match it to your actual launch configuration.
Can I connect vLLM and SGLang?
Yes. Both are supported model backends.
How are duplicate tool names handled?
SMG qualifies tools by server and name; configuration can add application-facing aliases.
Where can I inspect load?
Connect the gateway’s Prometheus metrics and OpenTelemetry traces.
Discussion
No questions or reviews yet.
Related
CLI toolsA CLI runs a model and MCP-tool gateway with owned API keys, format translation, routing and request logs.
A self-hosted gateway routes internal MCP tools, authenticates users and records requests for later review.
AIHelms is an organizational MCP gateway that attributes calls to employee keys and tracks configured per-tool costs.
An IBM gateway and registry that federates MCP servers, A2A agents and REST or gRPC APIs behind one endpoint with auth and observability
A self-hosted gateway that merges several MCP servers into one endpoint with auth and tool selection
MCP Gateway
mcp-gateway
A reverse proxy and lifecycle management for MCP servers on Kubernetes and Azure from Microsoft: routing, authorization, roles. For platform teams
README in short
The gateway supports Kubernetes worker discovery and several history backends. WebAssembly plugins can customize processing.