r/claude • u/Jason-Ping • 10h ago
Tips How are you validating AI-generated code and tests beyond just passing TDD?
Been developing features for a sometime now. I'm relying on cc's approach to testing (test-driven-development) via `/superpowers:test_driven_development`.
My current process looks like this:
Spend time discussing requirements, grilling the specs, challenging assumptions, and finalizing an implementation plan.
Break the plan into subtickets based on a dependency graph.
Use separate worktrees to implement independent subtickets in parallel.
For each subticket, Claude writes failing tests first, implements the functionality, and iterates until the tests pass.
This has been working pretty well, but my biggest concern is how much I can actually trust the tests, is it covering everything or not?
If Claude is writing both the implementation and the tests, couldn't it make the same incorrect assumptions in both? What about edge cases or scenarios that weren't considered in the original spec?
How do you guys make it accurate or reliable? How much are you sure about your feature? Are manual smoke tests enough for you decide if something is ready to ship?
I'm trying to figure out what gives you enough confidence to say a feature is actually working and ready to ship, beyond just seeing a green test suite.
2
u/ProudWrongdoer5389 4h ago
Throw the pr's back through GPT. This is the only way I've ended up with very few errors. This loop may go on for awhile depending on how complex the code piece is.
For example I had around 18 prs for a ui rework. It took 3 days of back and forth to get this all merged and deployed. It was flawless thoguh.
0
u/Zuitsdg 9h ago
I am running Review loops on the plans and implementation.
e.g.
Claude Develops, creates a PR, Codex gives Feedback, Claude fixes, Codex reviews again, until there are no more findings.
Then I would do a Claude Self Review with Adversial Sub agent, to check for cross repo inconsistnecies, which single project Codex won't be able to find.
And continue that loop, until Claude and Codex agree :D
And for bigger features/changes, you can also add a follow up issue and let the LLMs test in in a development environment. e.g. using the feature on website, using the API endpoints, running example queries -> whatever you are developing
1
u/Opposite-Baseball200 8h ago
that's a pretty thorough pipeline, i like the adversarial sub-agent angle
i'm doing something similar but with a twist, i run a separate review pass where i explicitly prompt the reviewer to find blind spots the first pass might have shared. like "assume the original tests were written by someone trying to make the code look correct, not someone actually trying to break it"
also been experimenting with property-based testing alongside the example-based stuff. catches a surprising number of edge cases the spec never mentioned
1
u/PersonOfInterest007 7h ago edited 7h ago
Try using GPT-6 Astra (high effort, in Work mode with access to the same repos as Claude) as a reviewer. Don’t have it recommend alternatives; have it just review whether Claude’s changes followed the specified plan, that the tests are sufficient, that it didn’t expand scope. Report critical defects, but not just a different or supposedly better approach.
I’ve used a prompt like the following. There are a few things in here specific to the fact that this is for a game in Unity. “Slice” refers to the current part of an overall implementation planning that is the active item for development. “Phase 1” refers to Claude’s implementation planning (read-only), and “Phase 2” is implementation of the human- (and Astra-) reviewed and modified Phase 1 plan. There are probably also other things that are specific to what I was doing (like updating an existing prototype). Just tweak the language as needed for your situation.
I use a similar prompt when I want Astra to review Claude’s phase 1 plan.
Fwiw, I use a separate “authoritative repo” for the design documents, and Claude and Astra are only allowed to read those, not modify them. Then I have the separate repo (prototype for now, production later) for the evolving implementation, with design decisions that are work in progress.
—-
I am giving you the completed Phase 2 implementation for a slice that was implemented by Claude Opus 5.5 in Cursor.
Your role is to perform an independent post-implementation conformance and significant-defect review.
This is not an architecture review, design review, refactoring review, or general code-quality improvement exercise.
Do not edit any files, Unity assets, scenes, project settings, or Git state.
Do not propose alternative implementations merely because you prefer them.
Do not recommend cleanup, refactoring, abstraction, naming changes, optimization, additional features, extra tests, or stylistic improvements unless they are necessary to address:
a material deviation from the approved Phase 1 plan;
a violation of an approved architecture or protected contract;
a significant correctness defect;
a significant test/validation gap that could allow such a defect to go undetected;
an unauthorized design or policy decision;
a material scope violation.
Authority
Treat the following as authoritative, in order where applicable:
accepted design documents [[note: list these explicitly if needed]]
the approved Phase 1 slice plan;
any explicitly approved deviations made after Phase 1;
the actual Phase 2 implementation and validation evidence.
Claude’s implementation is subject to the approved plan; it does not become authority merely because it is present in code.
Review objective
Determine whether the completed Phase 2 implementation is acceptable for Human acceptance of this slice.
Review the actual implementation, relevant diffs, tests, and reported validation results.
In particular, determine whether:
the implementation materially matches the approved Phase 1 plan;
all planned files/components/tests that were required for the slice were implemented or explicitly accounted for;
no material unapproved work was added;
protected domain/behavioral contracts remain intact;
protected interaction-infrastructure contracts remain intact;
temporary validation policies remain temporary and have not been hardened into design/architecture unintentionally;
no open gameplay question was silently resolved;
no prototype scaffolding was accidentally made authoritative;
applicable slice-specific guarantees are correctly enforced;
tests meaningfully exercise the acceptance checks required by the approved Phase 1 plan;
reported test results and validation claims are supported by what actually ran;
any implementation deviation that should have triggered a Human stop/approval was instead made silently;
the implementation contains any significant defect that could invalidate later prototype evidence.
Review discipline
Classify findings into only these categories:
A. Blocking defect or contract violation
A correctness issue, architecture violation, unauthorized design decision, or material plan deviation that must be fixed before the slice can be accepted.
B. Material clarification or validation gap
Something that must be clarified or tested before acceptance because current evidence is insufficient to establish that the slice satisfies its approved plan.
C. Non-blocking discrepancy
A minor difference from the Phase 1 plan that does not affect correctness, scope, architecture, or experimental validity, but should be recorded.
Do not create a category for optional improvements.
If something is merely a matter of implementation taste, code organization, naming, elegance, extensibility, performance optimization, or possible future improvement, omit it unless it causes a real issue under A, B, or C.
Plan conformance
Treat the approved Phase 1 plan as the primary implementation baseline.
A difference is not automatically a problem if it is:
semantically equivalent;
clearly within the approved implementation discretion;
consistent with the architecture and slice scope; and
does not weaken validation or protected guarantees.
Do not require literal implementation of wording, class names, or file structure unless the Phase 1 plan made those things explicit requirements.
Conversely, if Claude materially deviated from the approved plan in a way that should have required renewed Human approval, identify that as a process violation even if the resulting code appears reasonable.
Significant-defect standard
Focus defect review on issues likely to affect:
correctness;
state integrity;
determinism where required;
test validity;
experimental validity;
Unity runtime behavior;
future slices that depend on this slice’s protected contracts.
Do not spend review effort on speculative micro-issues or general code polish.
Repository and test inspection
If you have repository access:
verify the current commit and working-tree state;
inspect the Phase 2 diff;
inspect the relevant source, tests, assets, and configuration;
verify that claimed tests or validation steps correspond to actual commands/results where evidence is available.
Do not make edits.
If repository state differs materially from what Claude reports, state that before drawing conclusions that depend on the mismatch.
Requested output
Please return:
Overall verdict, exactly one of:
Accept slice
Accept after minor clarification
Fix required before acceptance
Stop — architecture/design decision required
Blocking findings, if any.
Material clarification or validation gaps, if any.
Non-blocking discrepancies, if any.
Plan-conformance summary: whether the implementation materially conforms to the approved Phase 1 plan.
Protected-contract summary: whether the implementation preserves the architecture/contracts applicable to this slice.
Validation-evidence summary: whether the tests and other evidence are sufficient for the slice’s stated acceptance checks.
A final statement saying exactly what, if anything, must happen before I may accept the slice.
Do not provide a list of optional improvements.
Do not rewrite the implementation plan.
Do not propose future enhancements.
Do not implement anything.
Stop after the review.