jyao-SUSE-power-group/dsh-provider-rate-limit

dsh-provider-rate-limit adds per-provider and per-model rate limiting to DSH LLM traffic: a reservation-based token bucket per (provider, model) route with wait or reject modes and strict FIFO admission, plus gateway identity rules that rewrite User-Agent or inject static headers for gateways that validate client identity, a settings card, an SSE-pushed live stats line and stats HTTP endpoints.

dsh-provider-rate-limit adds per-provider and per-model rate limiting to DeepSeek Harness LLM traffic, plus gateway identity rules for restricted free-tier gateways. It applies token-bucket rate limiting per (provider, model) route with smooth refill, burst support and idle-time recovery, with wait or reject modes when a bucket is empty and strict FIFO admission from a reservation-based design. A full settings card exposes the configuration from the harness settings page, a compact live stats line sits in the composer dock and is pushed over SSE with a per-route provider/model breakdown on hover, and GET /api/provider-rate-limit.stats returns aggregate and per-route figures while GET /api/provider-rate-limit.events streams snapshots.

Models & Providers ★ 4
View on GitHub ↗

Install

dsh plugin --profile web add github:jyao-SUSE-power-group/dsh-provider-rate-limit

Copied verbatim from the README's Install block. No npm package documented, so the GitHub form is shipped.

Compatibility

A reservation-based token-bucket design guarantees strict FIFO admission without polling; when a bucket is empty the plugin can either hold the request up to maxWaitMs (transparent queueing) or short-circuit immediately with a synthetic RATE_LIMIT response carrying providerRetryAfterMs. A master switch passes all traffic instantly with no listener re-registration. Live stats are SSE-pushed (no polling) with a 15 s heartbeat and automatic reconnection, and the gateway identity rules rewrite User-Agent or inject static headers only for URLs matching a pattern.

Details

Alternatives

1624318455/dsh-plugin-adapter · aa2246740/dsh-antigravity-oauth · AmmarByFar/dsh-trustedrouter

More plugins in Models & Providers

Browse more in Models & Providers

Guides for Models & Providers plugins