Back to blog
Claude Code

A quality score for the way you prompt your agent

Two prompts asking for the same change can cost 10× apart. SpecShip now grades every prompt in a session — does it name a concrete target, is it scoped, is it cache-friendly, did it go structural or brute-force — and hands you the one-line fix. Rule-based, no LLM, and it never invents a number it can’t measure.

Two engineers ask the agent for the same change. One types "why is login slow"; the other types "trace the call path from authenticate to the DB query and tell me where the N+1 is." Same intent, wildly different bills — the first sends the agent fanning out across files; the second hands it a target it can resolve in one structural hop. The gap between those two prompts is the single biggest lever you have over what a session costs, and until now nothing measured it.

Open any session's prompt timeline and each prompt now carries a small quality dot. Open a prompt's page and you get the whole read: a 0–100 score, the factors behind it, and the one-line fix.

Four factors, no model

The score is deterministic — pure rules over the prompt text and the turn's tool calls. No second LLM grading your first one, no temperature, no surprise bill. It weighs four things:

Target
names a symbol, file, or REQ-id the agent can resolve
Scope
a concrete ask, not "explain / look / why…"
Cache
the prompt's cache-read rate was healthy
Structural
leaned on a query, not a wall of Read/grep

Each factor nudges a base score up or down; the total grades out as Excellent, Good, Fair, or Needs work. The dot on the timeline takes its color from the same number, so you can scan a hundred-prompt session and spot the three turns worth tightening without reading any of them.

It tells you what to change

A score on its own is a scold. The useful part is the How to improve list underneath — concrete, and where it can be, copyable. If a turn leaned on a big grep, the card doesn't just say "be more structural"; it hands you the replacement:

suggested fix · copy
specship_explore --symbol validateSession --depth 2

And for a low-scoring prompt it goes one step further with a suggested rewrite — your vague opener, re-cast to name the target and pick the structural path. A Share button copies the whole review as plain text, so a quick "here's how to phrase these" lands in a teammate's DM without a screenshot.

The cheapest token is the file the agent never had to read because you told it where to look.

It won't lie to fill a column

One factor wants to know whether a turn went brute-force — and the honest signal for that is a tool's runtime, which Claude Code's transcript doesn't record. So we don't pretend to have it. The score uses what the transcript does carry — a tool's result size stands in for "this Read pulled a lot of weight" — and where a number can't be measured, the factor leans on the measurable proxy rather than inventing one. It's the same line the SpecShip Impact page draws between spend (measured) and saved (estimated, marked est.): show the real thing, flag the estimate, never fake the gap.

Why a score, not a lecture

Nobody changes how they prompt because a doc told them to. They change because they watched one prompt cost 60% of a session, saw the four-word reason, copied the rewrite, and felt the next one come back in three turns instead of thirty. The quality card is a feedback loop on the one habit that moves the needle — and it sits right next to the cost it's explaining, so the lesson and the bill are never more than a glance apart.

#claude-code#observability#prompting#honesty
SE
Selva E.
Building SpecShip · graph, specs & harness
Works on the parsing, storage, and spec layers of SpecShip. Spends an unreasonable amount of time thinking about edges.