Catch silent model downgrades in Claude Code

Sat, 04 Jul 2026 10:00:00 +0300

How do you know Claude Code doesn’t silently downgrade Fable 5 to Opus for some of your tasks?

Recently Anthropic restored access to Fable 5 with a twist. From their blog post understood that some requests might get blocked and requests sent to Opus 4.8.

How do you know how many requests were sent to Opus 4.8 when the model is set to Fable5? I started debugging with that question in mind a few days ago. The built-in commands don’t answer it. /usage shows quota, /cost shows spend, /context shows tokens, but none of them tell you which model actually produced each response.

The ground truth is already sitting on your machine. Claude Code writes every session transcript to ~/.claude/projects/, and every assistant message in those files carries a “model” field with the exact model ID that generated it. A per-message audit trail, hiding in plain sight.

I turned that debugging session into a Claude Code skill called model-audit.

It scans your session history and reports which models handled your work, with main agent and subagents listed separately. Subagents legitimately run on different models, Haiku for cheap scans and so on, but your main conversation turns should stay on the model you selected. If a session was downgraded midway, you see it immediately as mixed model counts in a single row.

Install it as a plugin:

/plugin marketplace add vijaykodam/model-audit
/plugin install model-audit@model-audit

Then run /model-audit to audit the last 7 days, which is the default. You can pass any number of days, or all for your full session history.

Example output

Claude Code sessions active in the last 7 day(s)

LAST ACTIVE PROJECT SESSION MODELS (responses)
2026-07-03 09:47 -vijay-projects-project3 85f5d9a6-ec9f-4dc2-... claude-fable-5(24)
2026-07-02 11:18 -vijay-projects-project1 4a347153-4458-47f8-... claude-fable-5(278)
2026-07-02 07:53 -vijay-projects-projecy1 1a89419d/agent-ab3a2f9e5caae8685 claude-opus-4-8(21)
2026-06-30 18:06 -vijay-projects-project3 a4b9c607-b1e8-4c92-... claude-opus-4-8(128)

Totals across all sessions:
 598 claude-fable-5
 504 claude-opus-4-8
 51 claude-haiku-4-5-20251001

Reading the results

When invoked as a skill, /model-audit runs the audit and presents the session table and totals with these interpretation rules applied; they hold equally if you read the script output yourself.

MODELS counts are per response, so a session that changed model mid-way shows both models with their proportions. A downgrade shows up as claude-opus-4-8 (or another model) counts inside a session you expected to be Fable 5.
Subagent work appears as separate rows (SESSION looks like 1a89419d/agent-..., prefixed with the parent session that spawned it). Subagents legitimately run on different models but never handle the main conversation turns — so a main-session row showing 100% one model means every conversation turn ran on that model.
Resumed or forked sessions get a new session file that includes a copy of the prior history, so the same responses can appear in two rows (same message IDs in both files). Totals across sessions therefore overcount conversations that were resumed; per-session rows are still accurate.
"model":"<synthetic>" entries in transcripts are harness-injected error messages, not real model responses; the audit filters them out.

Claude Code’s built-in usage commands were checked first, but none of them report model provenance — which model actually produced each response. That is the gap /model-audit fills. How each differs:

/usage shows plan quota consumption (session/weekly limits) — no model IDs, no per-project or per-session breakdown.
/cost and /context show token and cost figures for the current session only — no history across sessions, no model provenance.
The session-report plugin aggregates token usage by project, skill, and subagent type from the same transcripts — but does not report model IDs.
/model-audit reads the per-message "model" field from the transcripts: per-session, per-response model IDs across all projects and any time window.

Note

Two things I cared about while building it:

Everything runs locally. It is a small shell script reading files on your own machine, and nothing is sent over the internet.

No dependencies beyond standard bash, grep, and awk. Works on macOS and Linux.

Final Thoughts

For what it’s worth, my audit showed no silent downgrades. But now I can verify that instead of assuming it.

Give it a try and let me know if you have any feature requests.

Repo: https://github.com/vijaykodam/model-audit

model-audit Claude Code SKILL

Sat, 04 Jul 2026 10:00:00 +0300

A Claude Code skill, installable as a plugin, that audits which Claude model handled your sessions. Use it to confirm your coding requests ran on the model you selected (Fable 5, Opus 4.8, and so on) and were not silently served by a different one.

View on GitHub

Why

Claude Code’s built-in commands do not answer “which model produced this response?”. /usage shows plan quota, /cost shows spend, /context shows tokens — none report model provenance.

The ground truth is already on your machine. Claude Code writes every session transcript to ~/.claude/projects/, and every assistant message carries a "model" field with the exact model ID that generated it. model-audit reads that per-message audit trail and rolls it up per session.

Install

As a plugin (recommended):

/plugin marketplace add vijaykodam/model-audit
/plugin install model-audit@model-audit

Or from your shell, outside a session:

claude plugin marketplace add vijaykodam/model-audit
claude plugin install model-audit@model-audit

Run /reload-plugins (or restart Claude Code) to activate.

Use

/model-audit # audit the last 7 days (default)
/model-audit 30 # any number of days
/model-audit all # full session history

Example output

Claude Code sessions active in the last 7 day(s)
LAST ACTIVE PROJECT SESSION MODELS (responses)
2026-07-03 09:47 -vijay-projects-project3 85f5d9a6-ec9f-4dc2-... claude-fable-5(24)
2026-07-02 11:18 -vijay-projects-project1 4a347153-4458-47f8-... claude-fable-5(278)
2026-07-02 07:53 -vijay-projects-project1 1a89419d/agent-ab3a2f9e5caae claude-opus-4-8(21)
2026-06-30 18:06 -vijay-projects-project3 a4b9c607-b1e8-4c92-... claude-opus-4-8(128)
Totals across all sessions:
598 claude-fable-5
504 claude-opus-4-8
51 claude-haiku-4-5-20251001

Model counts are per response, so a session that changed model mid-way shows both models with their proportions. A silent downgrade shows up as an unexpected model count inside a session you expected to be on one model. Subagents appear as separate rows (the SESSION looks like 1a89419d/agent-...) — they legitimately run on different models but never handle the main conversation turns.

Notes

Everything runs locally. It is a small shell script reading files on your own machine; nothing is sent over the internet.
No dependencies beyond standard tools (bash, find, grep, sed, awk). Works on macOS and Linux.
Apache-2.0 licensed.

For full documentation and the source, see the GitHub repo.

Plugins on Vijay Kodam

Catch silent model downgrades in Claude Code

Example output

Reading the results

Related built-in tools

Note

Final Thoughts

model-audit Claude Code SKILL

Why

Install

Use

Example output

Notes