Consistency Review Agent

Part of Vault Agents. Runs nightly to review documentation within each vault for internal consistency, applies safe fixes automatically, and escalates ambiguous issues as action items.

Purpose

Vaults accumulate notes over time written in different sessions, contexts, and levels of detail. This agent reads all notes in a vault together and checks whether they tell a coherent, consistent story — catching things like contradictions, outdated references, terminology drift, and structural gaps that a human wouldn’t notice note-by-note.

Trigger

Scheduled: nightly at 3am
Scope: one vault per run (loops through all vaults)

Inputs

All .md files within a vault (excluding _agent/ files)
The vault’s _agent/context.md (for topic, key themes, user preferences)

Outputs

Output	Destination	When
In-place edits to notes	The vault itself	Safe, unambiguous corrections
Change log entry	`_agent/log.md` in the vault	Every run
Action items for user	`todos` vault	When agent is uncertain or decision needed

What “Consistency” Means

The agent checks for:

Factual / Content Consistency

Contradictory statements across notes (e.g. two notes disagreeing on a setting or spec)
Outdated information superseded by a newer note (e.g. old config replaced by a new one)
References to things that no longer exist (deleted notes, renamed files, changed entity IDs)

Terminology Drift

Same concept referred to by different names across notes (e.g. “blind” vs “shade” vs “cover”)
Inconsistent naming of hardware, services, or variables

Structural / Formatting Consistency

Frontmatter gaps or wrong date formats
Notes missing required fields (title, tags, date)
Section headings that don’t match the vault’s established style

Completeness

Notes that reference something that should exist but doesn’t (e.g. “see the config note” but no config note exists)
Placeholder text left unresolved (TBD, TODO, entity ID TBD, etc.)

Decision Logic

for each vault:
  read all content notes + context.md
  identify consistency issues

  for each issue:
    if safe to fix automatically:
      apply edit
      log change to _agent/log.md
    else:
      write action item to todos vault
      include: vault name, note, issue description, what decision is needed

”Safe to fix automatically” criteria

Frontmatter gaps (adding missing title, tags, date)
Obvious typos in terminology (single consistent term used 9/10 times; fix the outlier)
Broken wikilinks where the target has been renamed (if rename is unambiguous)
Formatting normalization (heading levels, list styles)

“Escalate to user” criteria

Contradictory facts where it’s unclear which is correct
Placeholders that require real-world information (entity IDs, passwords, URLs)
Structural decisions (should this be one note or two?)
Any deletion of content

Action Item Format

When writing to the todos vault, each item should follow this structure:

- [ ] [Consistency Agent] <vault-name> — <brief issue title>
  - Note: `<filename>`
  - Issue: <description of the inconsistency>
  - Options: <what choices the user needs to make>

Open Questions

Granularity: Should the agent run per-vault or across all vaults in one pass? (Per-vault is safer and easier to debug)
Tone of edits: Should auto-edits preserve the original author’s voice, or normalize to a standard style?
Frequency: Does every vault need to run every night, or only vaults modified since last run?
Confirmation mode: Option to log proposed changes without applying them, for user review first?
Logging verbosity: Full diff in log, or just a summary?

Running via Claude Code CLI

The agent runs using the claude CLI (Claude Code) in non-interactive mode. No separate API key or script is needed — it uses your existing Claude Code session.

Key CLI flags

claude --dangerously-skip-permissions -p "your prompt"

-p / --print — non-interactive mode: runs the prompt and exits
--dangerously-skip-permissions — skips all tool permission prompts, required for unattended/cron use

Runner script

Save as /vault/scripts/consistency-agent.sh and make it executable (chmod +x):

#!/bin/bash
 
VAULT_DIR="/vault/content"
LOG_FILE="/vault/scripts/consistency-agent.log"
SKIP=("vault-template")
 
echo "" >> "$LOG_FILE"
echo "=== $(date '+%Y-%m-%d %H:%M') ===" >> "$LOG_FILE"
 
for vault_path in "$VAULT_DIR"/*/; do
  vault_name=$(basename "$vault_path")
 
  # Skip reserved vaults
  for skip in "${SKIP[@]}"; do
    [[ "$vault_name" == "$skip" ]] && continue 2
  done
 
  # Skip if not a real vault (no _agent dir)
  [[ ! -d "$vault_path/_agent" ]] && continue
 
  echo "Processing: $vault_name" >> "$LOG_FILE"
 
  claude --dangerously-skip-permissions -p "
You are the Consistency Review Agent running in automated mode.
 
Vault to process: $vault_name
Vault path: $vault_path
 
Instructions:
1. Read all .md files in $vault_path (skip files inside _agent/)
2. Read $vault_path/_agent/context.md for vault topic and themes
3. Check all notes for consistency issues as defined in /vault/content/vault-agents/consistency-agent.md
4. For each issue:
   - If safe to fix automatically: apply the edit directly to the file
   - If uncertain or requires user decision: append an action item to /vault/content/todos/todos.md using the format defined in consistency-agent.md
5. Append a summary of what you did (edits made, items escalated, or 'no issues found') to $vault_path/_agent/log.md under today's date
 
Be conservative. When in doubt, escalate rather than edit.
" >> "$LOG_FILE" 2>&1
 
  echo "Done: $vault_name" >> "$LOG_FILE"
done
 
echo "=== Complete ===" >> "$LOG_FILE"

Cron setup

Open your crontab:

crontab -e

Add this line to run at 3am daily:

0 3 * * * /vault/scripts/consistency-agent.sh

To verify it’s registered:

crontab -l

Testing manually

Run a single vault without cron to verify it works:

claude --dangerously-skip-permissions -p "
You are the Consistency Review Agent. Process only the vault: todos
(paste the full prompt from the script above)
"

Or run the full script immediately:

/vault/scripts/consistency-agent.sh

Then check:

cat /vault/scripts/consistency-agent.log

Notes

Claude Code must be installed and authenticated (claude must work in your terminal)
The script runs each vault sequentially; a full run across all vaults takes a few minutes
--dangerously-skip-permissions gives Claude full file read/write access — only use this in the /vault context where that is intentional
Log output is appended to /vault/scripts/consistency-agent.log for review

Status

Create /vault/scripts/ directory
Save and test runner script on one vault
Verify action items appear correctly in todos vault
Set up cron job
Monitor first few nightly runs and tune the prompt

Tony's Notes

Explorer

Consistency Review Agent — Plan