DeepSeek V4 vs Claude vs Local LLM Benchmark Kit
A standardized Python benchmark suite with a simple results dashboard that lets Mac Studio and local LLM users compare DeepSeek V4, Ollama models, and Claude API on real-world coding, reasoning, and agentic tasks.
- ▸benchmark/run.py — CLI entrypoint; run all suites or target a single model in one command
- ▸benchmark/models.py — provider adapters for DeepSeek API, Anthropic API, and Ollama (local)
- ▸benchmark/cost.py — pricing table and token-to-USD calculator, updatable when rates change
- ▸tasks/coding/ — 4 tasks: debug, refactor, write unit tests, explain a traceback
- ▸tasks/reasoning/ — 4 tasks: logic puzzle, multi-step math, constraint scheduling, causal chain
- ▸tasks/agentic/ — 4 tasks: plan a script, decompose a feature, write a bash pipeline, generate a PR description
- ▸dashboard/ — terminal leaderboard + static HTML report (no CDN, fully offline)
- ▸docs/task-authoring-guide.md — add your own tasks in minutes
- ▸docs/model-pricing-table.md — current pricing reference for all supported models
## goes-well-with
From the same shelf
Agentic Migration Playbook: API Endpoint Modernization
For DevOps and platform engineers with a real migration deadline — the complete Claude Code workflow you'd otherwise spend months reverse-engineering.
MAESTRO Security Framework for Claude Code Projects
Apply the MAESTRO 7-layer threat-modeling framework to any autonomous Claude Code / Agent SDK project — enumerate threats across the whole chain and fill in a reusable threat model for your own stack.
MCP Tunnel Explainer: Private Network MCP Without Public Endpoints
A hands-on tutorial and code walkthrough showing how to wire a local Ollama MCP server behind an outbound-only encrypted tunnel — no public IP, no inbound firewall rules, yet agents still reach private enterprise tools.
## not-ready-to-buy
Take the field notes instead
One practical write-up a week from the same workbench these kits come from — plus reader pricing when new kits ship.