Catch silent model downgrades in Claude Code

How do you know Claude Code doesn’t silently downgrade Fable 5 to Opus for some of your tasks?

Recently Anthropic restored access to Fable 5 with a twist. From their blog post understood that some requests might get blocked and requests sent to Opus 4.8.

How do you know how many requests were sent to Opus 4.8 when the model is set to Fable5? I started debugging with that question in mind a few days ago. The built-in commands don’t answer it. /usage shows quota, /cost shows spend, /context shows tokens, but none of them tell you which model actually produced each response.

The ground truth is already sitting on your machine. Claude Code writes every session transcript to ~/.claude/projects/, and every assistant message in those files carries a “model” field with the exact model ID that generated it. A per-message audit trail, hiding in plain sight.

I turned that debugging session into a Claude Code skill called model-audit.

It scans your session history and reports which models handled your work, with main agent and subagents listed separately. Subagents legitimately run on different models, Haiku for cheap scans and so on, but your main conversation turns should stay on the model you selected. If a session was downgraded midway, you see it immediately as mixed model counts in a single row.

Install it as a plugin:

/plugin marketplace add vijaykodam/model-audit
/plugin install model-audit@model-audit

Then run /model-audit to audit the last 7 days, which is the default. You can pass any number of days, or all for your full session history.

Example output

Claude Code sessions active in the last 7 day(s)

LAST ACTIVE        PROJECT                                SESSION                               MODELS (responses)
2026-07-03 09:47   -vijay-projects-project3    85f5d9a6-ec9f-4dc2-...                claude-fable-5(24)
2026-07-02 11:18   -vijay-projects-project1    4a347153-4458-47f8-...                claude-fable-5(278)
2026-07-02 07:53   -vijay-projects-projecy1    1a89419d/agent-ab3a2f9e5caae8685      claude-opus-4-8(21)
2026-06-30 18:06   -vijay-projects-project3    a4b9c607-b1e8-4c92-...                claude-opus-4-8(128)

Totals across all sessions:
     598  claude-fable-5
     504  claude-opus-4-8
      51  claude-haiku-4-5-20251001

Reading the results

When invoked as a skill, /model-audit runs the audit and presents the session table and totals with these interpretation rules applied; they hold equally if you read the script output yourself.

• MODELS counts are per response, so a session that changed model mid-way shows both models with their proportions. A downgrade shows up as claude-opus-4-8 (or another model) counts inside a session you expected to be Fable 5.
• Subagent work appears as separate rows (SESSION looks like 1a89419d/agent-..., prefixed with the parent session that spawned it). Subagents legitimately run on different models but never handle the main conversation turns — so a main-session row showing 100% one model means every conversation turn ran on that model.
• Resumed or forked sessions get a new session file that includes a copy of the prior history, so the same responses can appear in two rows (same message IDs in both files). Totals across sessions therefore overcount conversations that were resumed; per-session rows are still accurate.
• "model":"<synthetic>" entries in transcripts are harness-injected error messages, not real model responses; the audit filters them out.

Related built-in tools

Claude Code’s built-in usage commands were checked first, but none of them report model provenance — which model actually produced each response. That is the gap /model-audit fills. How each differs:

• /usage shows plan quota consumption (session/weekly limits) — no model IDs, no per-project or per-session breakdown.
• /cost and /context show token and cost figures for the current session only — no history across sessions, no model provenance.
• The session-report plugin aggregates token usage by project, skill, and subagent type from the same transcripts — but does not report model IDs.
• /model-audit reads the per-message "model" field from the transcripts: per-session, per-response model IDs across all projects and any time window.

Note

Two things I cared about while building it:

Everything runs locally. It is a small shell script reading files on your own machine, and nothing is sent over the internet.

No dependencies beyond standard bash, grep, and awk. Works on macOS and Linux.

Final Thoughts

For what it’s worth, my audit showed no silent downgrades. But now I can verify that instead of assuming it.

Give it a try and let me know if you have any feature requests.

Repo: https://github.com/vijaykodam/model-audit