BuilderGraph: A Hardware-Aware Local AI Coding Agent

If you have a computer with a GPU, you can run your own AI coding agent. No cloud APIs, no subscriptions, no data leaving your machine. That is the premise behind BuilderGraph.

The Problem#

Most developers who want AI-assisted coding have two choices: pay for cloud APIs (sending their code to someone else's server) or try to run models locally and deal with "out of memory" errors, slow inference, and no tooling.

The gap is not the models. Open-source coding models like Qwen2.5-Coder and DeepSeek-Coder are genuinely good. The gap is knowing which model fits YOUR specific hardware and having a proper coding agent that works with local models.

What BuilderGraph Does#

BuilderGraph is a hardware-aware personal AI-building advisor. It has two operational layers:

Layer 1: Stack Advisor profiles your machine (GPU, VRAM, RAM, CPU), filters a 20-model registry using a KV-cache-aware memory calculator, and ranks models by coding quality. It knows that a 7B model at Q4_K_M quantization with 16K context needs exactly X GB of VRAM including KV-cache overhead. Not approximate. Exact.

Layer 2: Builder Runtime connects to the model, provides file/shell tools (read, write, edit, search, run commands), and gives you an interactive coding session with conversation memory. Think Claude Code or Cursor, but running entirely on your local GPU.

How It Works Under the Hood#

Prompt-Based Tool Calling#

Local 7B models cannot reliably use native function-calling APIs. They hallucinate JSON schemas, skip required fields, or call nonexistent tools. BuilderGraph uses prompt-based tool calling instead: the model outputs <tool_call> XML blocks in plain text, and the agent parses them deterministically.

This approach works with ANY instruction-following model. No special fine-tuning, no function-calling adapters, no model-specific hacks.

Security Sandbox#

Here is where it gets interesting. The agent runs commands in your project directory. A local model could theoretically be jailbroken by a malicious file in the repo. BuilderGraph treats the model as adversarial by default.

30+ attack vectors are blocked before any command executes:

• Shell operators (no command chaining, no pipes, no redirects)
• Interpreter script execution (no python script.py)
• Package managers with side effects (no pip install, no npm run)
• Git write operations (no push, no remote manipulation, no config changes)
• Build tools, .git directory writes, Windows Alternate Data Streams

The sandbox is designed so that even a fully compromised model cannot exfiltrate data, execute arbitrary code, or destroy your workspace.

Context Window Management#

Sessions can run for hours. A 7B model typically has 16K-32K context. BuilderGraph automatically prunes the oldest conversation messages while keeping the system prompt and the most recent exchanges intact. The session stays functional indefinitely without manual intervention.

Numbers#

Running on an RTX 4060 Laptop (8 GB VRAM) with Qwen2.5-Coder-7B-Instruct at Q4_K_M quantization:

• 40.3 tokens/second generation speed
• 2.12 seconds time-to-first-token
• 318 tests (all stdlib unittest, zero third-party test dependencies)
• Zero pip dependencies in the entire project

The recommendation engine handles the hardware math: it knows this specific GPU can fit a 7B Q4_K_M model with 16K context and ~2.8 GB KV-cache headroom, but cannot fit a 13B model at the same quantization without aggressive context reduction.

The Interactive REPL#

The main interface is buildergraph chat. It gives you a persistent coding session with:

• Git branch display in the prompt
• Conversation memory across turns
• Slash commands (/mode, /clear, /compact, /diff, /info)
• Shell escape (prefix with ! to run commands directly)
• Multi-line input (end lines with \)
• Colored tool-use indicators

You work with it the same way you would work with any AI coding assistant. Ask it to find bugs, write tests, refactor code, explain logic. The difference is everything runs on your hardware, touches only your files, and costs nothing per token.

Design Choices#

Zero dependencies. The entire project uses Python stdlib only. urllib for HTTP, ctypes for RAM detection, subprocess for nvidia-smi and the LM Studio CLI, json for serialization. No supply chain risk, runs anywhere Python 3.9+ exists.

Headless model management. buildergraph up goes from zero to a running coding session in one command. It picks the best model for your hardware, downloads it, loads it into LM Studio headlessly (no GUI), and starts serving. Pure terminal workflow.

KV-cache-aware fitting. Most VRAM calculators just check model weight size. BuilderGraph calculates the actual runtime memory including KV-cache at your target context length. A model that "fits" in VRAM at rest might OOM during a long coding session. BuilderGraph prevents that.

Try It#

pip install -e .
buildergraph diagnose      # see your hardware
buildergraph recommend     # get model recommendations
buildergraph up            # download + load + serve the best model
buildergraph chat          # start coding

The source code is available on GitHub. Python 3.9+, zero dependencies, works on Windows/Linux/macOS.

If you have a computer, you can run AI. No excuses.