Skip to content

AI Agent Security

Audit Local LLM Agents Like Runtimes

CLAWAUDIT is a useful reminder that local agents are privileged runtimes, not chat widgets. Audit prompt assembly, parsers, tool dispatch, skill loading, memory writes, network clients, and permission gates.

June 24, 2026 · Dominic Hückmann

Short Answer

Local LLM agents can touch shells, files, browsers, credentials, memory, and messaging tools. Treat their runtime layer as source code worth auditing, then turn static findings into a manual review queue instead of automatic verdicts.

Short answer

Local LLM agents are not just chatbots with a few helpful tools.

They are privileged runtimes.

They sit between natural-language intent and host-level actions: shell commands, files, browser sessions, credentials, memory, network clients, and messages. That makes the runtime layer a real security boundary, not a prompt detail.

The useful idea in Local LLM Agents as Vulnerable Runtimes is to audit that layer as source code. Not “can the model be tricked?” but “where does this runtime assemble instructions, parse model output, grant tools, load skills, write memory, call the network, or check permissions?”

7
runtime surfaces to audit before trusting a local agent
47 + 30
Semgrep rules and CodeQL queries reported by CLAWAUDIT
0
automatic vulnerability verdicts from static findings alone

The mistake

Most agent security talk still starts at prompt injection, malicious webpages, and poisoned skills. Good questions. Incomplete questions.

The runtime decides what model text becomes an action: how tool calls are parsed, which names are valid, whether skills can reshape priority, whether memory writes are trusted, and whether permission checks happen before side effects.

If that code is sloppy, a better prompt will not save you.

Agent runtime audit loop

source first, verdict later

  1. 01
    Map surfaces
  2. 02
    Run static rules
  3. 03
    Create review queue
  4. 04
    Triage by exploit path
  5. 05
    Patch control point
  6. 06
    Add regression check

A workflow to steal

Take one local agent runtime and make a four-column audit table:

Surface | Example risk | Evidence to inspect | Safe control

Use these rows:

  1. Prompt builder: untrusted text entering high-priority instruction slots.
  2. Output parser: fallback parsing, JSON repair, hidden command extraction, and partial parse success.
  3. Tool dispatcher: stringly typed routing, broad allowlists, and tools that can call other tools.
  4. Skill loader: remote install paths, weak provenance, and skill text that implies new authority.
  5. Memory writer: unverified promotion, cross-user bleed, and attacker-controlled lessons.
  6. Network client: implicit credential use, SSRF-shaped fetches, and weak destination checks.
  7. Permission gate: checks after side effects, approval stored as plain text, and bypasses for “internal” calls.

For each finding, write the exploit path in boring language:

Untrusted source -> runtime transform -> privileged action -> missing gate

If you cannot write that path, keep the item as a hardening note, not a vulnerability claim.

The practical rule

  • Audit the code path that turns language into authority.
  • Treat static findings as a review queue, not proof.
  • Patch the control point closest to the side effect.
  • Add a regression test that fails when the bypass returns.

Static checks help, but they are noisy

CLAWAUDIT reports a clear signal: agent-specific static rules can catch things general rules miss. The abstract says held-out recall rose from 21.7% to 66.8% for Semgrep and from 13.8% to 75.1% for CodeQL.

That is useful.

It is also not the same as “run scanner, ship fix.” Recall-oriented rules over-report by design. A preliminary live-code audit in the paper still required manual triage and semantic filtering before production use.

Weak vs. safer runtime audit

Weak

  • Review only prompts and policy prose.
  • Treat every scanner hit as a bug.
  • Patch the prompt after a runtime bug.
  • Trust user approval stored in conversation text.

Safer

  • Review prompt assembly, parsing, dispatch, memory, network, and gates.
  • Rank hits by exploit path and reachable authority.
  • Patch the parser, dispatcher, loader, writer, client, or gate.
  • Bind approval to a typed decision object checked before side effects.

Tiny audit prompt

Use this after a scan or source read:

Review this local-agent runtime finding.

Return:
1. Surface: prompt builder / parser / dispatcher / skill loader / memory writer / network client / permission gate.
2. Untrusted input source.
3. Privileged action it may influence.
4. Missing or late control.
5. Minimal exploit path in one line.
6. False-positive reason, if likely.
7. Best fix location in code.
8. Regression test that would catch it next time.

Do and don’t

Runtime audit habits

Do

  • ✓ Map authority boundaries before reading individual findings.
  • ✓ Manually triage scanner output by reachable side effect.
  • ✓ Require explicit gates for shell, browser, messaging, memory, credentials, and production writes.
  • ✓ Keep adversarial regression tests beside the runtime code.

Don't

  • × Assume prompt policy is an enforcement mechanism.
  • × Let skills, memory, or web content enter high-priority instruction slots without labeling.
  • × Allow broad tool dispatch because the agent is local.
  • × Publish scanner numbers as safety proof without triage evidence.

Where this fails

This is not a full security program. It will miss bugs that require live state, timing, auth context, or multi-step behavior. It also gets noisy if every tool-shaped string becomes a finding.

So keep the bar practical: a strong finding needs a source, a transform, a reachable privileged action, and a missing control. Local agents deserve that discipline because they run where the damage is close.

Sources

FAQ

Why audit a local LLM agent runtime?

Because the runtime mediates between model text and host actions: shell commands, file writes, browser sessions, memory updates, stored credentials, and messages.

Is static scanning enough for agent security?

No. Static rules are useful for recall and review queues, but agent-runtime findings need manual triage, semantic filtering, and runtime controls before production use.

Where should a small team start?

Start with the seven runtime surfaces: prompt builder, output parser, tool dispatcher, skill loader, memory writer, network client, and permission gate.

Need AI-first architecture support?

Send me a short note about your project or technical bottleneck.

Get in touch