Vmarcelo49/dsh-local-models

dsh-local-models adds a Local Models tab to the DSH Web GUI: pick a .gguf file, tune context, KV-cache quantization and MoE placement, watch a live VRAM estimate, load it through stock upstream llama-server, and register the running server as an llm-pi-ai provider route in one click - with named launch profiles and an OpenAI-compatible router mode.

dsh-local-models adds a Local Models tab to the DeepSeek Harness Web GUI for running GGUF models through llama-server. An in-app file browser lists directories and .gguf files with a header-only GGUF parse (architecture, quant, layers, context length, MoE detection). Launch options cover a context-length slider (8K steps, capped at the model's trained context) plus fine-tune, KV-cache quantization selectors for K and V (every type llama-server accepts, with bytes per element), a fixed MTP draft depth, a thinking level (off / low / medium / xhigh) with a preserve-thinking toggle, an optional vision mmproj, and MoE expert placement with a fit-to-VRAM helper. A live VRAM estimate compares weights plus the selected K/V cache types, recurrent state, compute/graph and overhead against the detected GPU total, with fits / safe-margin / max-ctx-that-fits rows. Named launch profiles are saved and reloaded in one click, a router mode serves all saved profiles from one OpenAI-compatible endpoint, the ready server is registered in dsh as an llm-pi-ai provider route in one click, and a terminal overlay tails the llama-server log.

Models & Providers ★ 0 updated 2026-10-03
View on GitHub ↗

Install

dsh plugin --profile web add dsh-local-models

Copied verbatim from the README's Install block (the npm-registry form). The README also documents dsh plugin --profile web add github:Vmarcelo49/dsh-local-models and a link:/path/... development form. npm package dsh-local-models at 0.5.0 (registry check 2026-10-16; the registry repository field resolves back to this repo).

Compatibility

A dsh plugin lives inside a profile (a pnpm project under $DSH_HOME/profiles/<name>), and dsh plugin add writes the dependency and appends the package to dsh.profile.bundles so nothing needs editing by hand - a web-process restart mounts it. The VRAM budget is detected (nvidia-smi for NVIDIA, amdgpu sysfs for AMD, all visible GPUs summed) with a 16 GB fallback and a safety margin defined at the top of lib/client.js, and there is a KNOWN_ISSUES.md noting Gemma-family accuracy caveats. pnpm must be on PATH.

Details

Alternatives

1624318455/dsh-plugin-adapter · aa2246740/dsh-antigravity-oauth · AmmarByFar/dsh-trustedrouter

More plugins in Models & Providers

Browse more in Models & Providers

Guides for Models & Providers plugins