Repository explainer · andygsu/The-Medtech-Flywheel
An agent pipeline that sources, scores and verifies medtech spin-out ideas — designed so that no machine-generated claim can ever reach a score, a shortlist, or a cheque without an independent check against a primary source, and so that the decisions that spend money or touch patients stay human.
The one rule that matters
No claim reaches a score without a Verifier pass, and no Verifier pass shares context with the Scorer that drafted the claim.
Every other convention in the repo exists to protect that sentence. The failure mode it is built against is specific and well documented: an agent chain where each step trusts the previous step's claim without re-checking it, drifting into fabricated predicate devices, invented CPT codes and phantom patents — each one fluent, plausible, and wrong.
Medtech idea evaluation is mostly a research problem: is there a regulatory pathway, does a billing code exist, does anyone pay enough for it, is the mechanism already patented, how big is the market really. Those questions are slow, and a language model will answer all of them instantly and confidently whether or not it knows. That is the whole problem this repository is organised around.
So the pipeline splits the work by how much it costs to be wrong. Generating ideas and drafting a first-pass score is cheap to get wrong — run it unattended, all day. Deciding that a predicate device exists is expensive to get wrong — that goes through a structurally separate agent that must produce an API record ID, not a paragraph. Deciding to spend money, sign an FTO opinion, or start an animal study is catastrophic to get wrong — no agent has a code path to it at all.
The result is a flywheel rather than a pipeline: outcomes feed back into the scoring model, killed ideas feed back into generation, and changed facts in the outside world wake dormant candidates back up.
The tiers are not a description of how careful each agent tries to be. They are a description of what each one is able to do. Phase 2 of the spec made that explicit: a convention an agent's prompt could violate under a bad escalation, a confused context, or a prompt injection from a scraped page is not a boundary — it has to be a capability limit.
The residual gap, stated plainly. SQLite has no real per-table permissions, so a raw
sqlite3 db/flywheel.db "INSERT INTO human_decisions ..." from any shell with file access still
bypasses the CLI. The repo says so in scripts/human_decision.py rather than pretending otherwise;
blocking it generically would also block legitimate read-only debugging. Moving to a real database server is
where GRANT-level enforcement would replace the application-layer boundary.
This is the whole system in one picture. The centre column is the path an idea takes; the left column is what gets written to disk at each step (agents hand work to each other through files, never through conversation); the right column is where ideas leave the path, and the two feedback lanes that bring information back.
Each agent is defined by a file in .claude/agents/ that states its job, its tool list, and —
at least as importantly — a What you never do section. The tool lists are the real separation: the
Scorer has no network access at all, so it physically cannot verify its own claims; the Verifier has no Write
tool for scores, only Edit on claim lines.
| Agent | Tier | Stage | Job | Structurally cannot |
|---|---|---|---|---|
| Scout | 1 | 0 | Watch for facts that changed — new clearances, patent expirations, code and payer-policy changes, rival activity, literature | Score, generate, or set a confidence tag. A promising find is logged as a “possible verification lead”, nothing more |
| Generator | 1 | 1 | Write new candidate files with device class, target, mechanism, predicate, intended use and jurisdiction | Verify its own claims, score them, or re-propose a shape already in the killed-idea log without a signal that fired its revive condition |
| Scorer | 1 | 2 & 4 | Apply the 9-factor rubric — once on unverified claims for triage, once on verified claims for a real composite | Change a confidence tag, treat unconfirmed as true, or fill in the gate check |
| Verifier | 2 | 3 | Check every claim against a typed primary-source connector and record the result with a record ID | Touch any score, run in the Scorer's context, soften a refuted finding, or answer a claim with a web search |
| Librarian | 1 | 0, 5, 6, 8 | Bookkeeping: killed-idea log, calibration log, flagging candidates for re-scoring, applying approved rubric changes | Decide anything. It executes decisions that already exist and never edits Claims, Scores or a composite |
| Diligence Lead | 1+2 | 7 | Per-candidate deep package: bench evidence, failure modes, draft FTO memo, reimbursement strategy | Approve spend, greenlight an animal study, or state that FTO is “clear”. Every deliverable is explicitly a draft for sign-off |
Why the Scorer and Verifier must not meet. If the Verifier can see the reasoning that produced a claim, it stops being an independent check and becomes a reviewer of an argument — and arguments are exactly what a language model is good at accepting. The repo enforces the split three ways: the Verifier runs as a fresh subagent invocation, it reads the candidate file cold off disk, and only the specific claim text — never the Scorer's trace — is in its context.
Tier 1 Scout · /scan · runs continuously
Watches the outside world for facts that should re-open a question already in the knowledge base. It is pointed at what is already logged: unresolved claims first, then the explicit “revive condition” attached to every killed idea.
Tier 1 Generator · /generate [n] [theme]
Produces candidates from the template, primed with the current rubric (so it knows what a strong idea needs) and the killed-idea log (so it stops re-proposing dead shapes).
Tier 1 Scorer · /score idea-XXX
A triage pass whose only purpose is deciding whether a candidate is worth the cost of verification. Every
number rests on an unverified claim and none of them is meant to be acted on.
The valuable output is not the score — it is the “What would settle it” column: for each factor, the specific primary source that would turn a guess into evidence. It costs nothing to fill in during scoring, it is what the Verifier works from, and it is what turns an unresolved factor into a work item instead of a permanent shrug. “More research” is not an allowed answer; a named source is.
Tier 2 Verifier · /verify idea-XXX · fresh context, always
Every unverified claim is routed to a typed connector and comes back with a status and a
source reference — an API endpoint plus record ID, not a search URL. See the deep dive
below for the routing rules and the two guards that run before any lookup.
Tier 1 Scorer · /rescore idea-XXX
The composite is recomputed over verified claims only. A factor with no verified claim is left
blank — not zeroed, not guessed, not carried over from the Stage 2 draft. A refuted claim
is active negative evidence and pulls its factor down.
Two counts are reported alongside the score: factors verified n/9, which governs whether the candidate advances, and core factors verified n/4 (Regulatory, Reimbursement, TAM, FTO/IP), which is a diagnostic that gates nothing — it exists because a candidate with all four gate-relevant factors resolved is in a very different position from one with the same total spread across factors the gate never reads.
Tier 1 · /gate-check idea-XXX · mechanical, no subagent
The kill conditions, applied to verified data only. If a Verified Composite is not present, the stage refuses to run rather than falling back on Stage 2 numbers.
Humans · /shortlist compiles, it does not judge
The autonomous cycle stops here, permanently and by design. /shortlist assembles a packet:
one paragraph per cleared candidate, with intended use, jurisdiction, composite, verified counts, the claims
behind the best and worst factors, and the gap analysis verbatim. It ranks by composite score and explicitly
does not recommend.
Two things are flagged because they look like strength and are not: a non-US launch jurisdiction, whose claims are unconfirmed for want of a connector rather than for want of an answer, and a Reimbursement factor resting on code existence alone.
Tier 1+Tier 2 Diligence Lead · /diligence idea-XXX · requires status: shortlisted
Tier 1 Librarian · /log-outcome idea-XXX "..."
Whatever happened — killed, funded, piloted, and why — is written back. Where the real approval timeline, real cost or real sales cycle diverges from what the model predicted, that gap is the entire input to recalibrating the rubric. The Librarian records the gap precisely and does not editorialise about which factor was “wrong”: a rubric change has to cite the row, and the row has to be worth citing.
Stage 3 is where this repository differs from “an agent that researches ideas”. The requirement is not that the Verifier looks things up carefully — it is that “checks the FDA database” means a typed API call with a parseable response, not a search query summarised by a model. A summarised search result is not a verified claim; it is an opinion about a search result.
Two guards run before any lookup happens at all. Both exist because of specific ways a check can come back confidently wrong.
unconfirmed rather than an instruction an agent could drift past.Eleven subcommands, one per claim shape, so the Verifier never falls through to a web search by omission.
Three of them can never return anything but unverifiable — and that is a feature, not a gap
waiting to be filled.
python3 -m connectors <subcommand>, printing one JSON object. A result carries a
source_ref — an endpoint plus a record ID — whenever it is verified or
refuted. The four CMS-backed connectors read locally cached files that a human supplies; when the
file is absent the answer is unconfirmed with instructions, never a substituted web search.Market size. No free, authoritative, machine-queryable source will tell you the addressable
market for a device. So market returns unverifiable permanently, and only a human
attaching a licensed report and editing the tag by hand can change that. But before settling for it, the claim
usually decomposes: a TAM claim is three claims wearing a trenchcoat — how often the procedure happens (which
is checkable), what each one is worth, and what share is addressable. Verifying one of the three and
leaving two unconfirmed is far more useful than one blanket shrug.
Adoption and stakeholder preference. “Surgeons would switch for a 20% time saving”, “payers would view this as cost-neutral”. A model asked this will produce a fluent, plausible, well-hedged answer, and that answer is a synthetic stakeholder opinion — a fabricated claim in better clothes, harder to spot than an invented patent number because nothing about it looks made up. Consulting firms answer these with fifty interviews and a hundred-respondent survey. The productive move is rescoping to the checkable proxy hiding inside: does a coverage policy exist for the comparator, and how often is the procedure actually done?
CPT codes. Blocked in code, for a licensing reason rather than an epistemic one. CPT is
AMA-licensed and cannot be scraped or reproduced verbatim, so check_cpt_code refuses to return a
verified or refuted result under any circumstance. A claim that depends on a CPT code either gets a licensed
source attached by a human or gets rescoped to HCPCS.
“Is there a reimbursement pathway” looks like one question and is three. The repo splits them because answering only the first and reporting it as a pathway is how a dead product gets a green badge.
| Question | Check | What a “yes” actually means |
|---|---|---|
| Can this be described on a claim form? | hcpcs A1234 | A code exists. Nothing more. Not that anyone pays. |
| Does it pay enough to be a business? | payment A1234 --min-expected 42.50 | The rate clears a stated cost basis. A code paying $8 for a $40 consumable is a kill condition wearing a green badge. |
| Has a payer actually agreed to cover it? | coverage A1234 | An NCD or LCD exists naming the code — not that it is positive. Many LCDs are non-coverage or impose limits, and that difference lives in the policy prose. |
Four tags, and the whole design rests on them being respected. A claim that never got a Verifier pass stays
unverified forever and never contributes to a composite, no matter how obviously true it looks.
“Helpfully” upgrading a tag to move a candidate along is the single most damaging thing an agent in this repo
could do, because everything downstream reads the tag rather than re-deriving the fact.
unconfirmed versus refuted. “We looked and could not confirm it” must never
collapse into “the answer is no” — one queues more research, the other can kill an idea.Nine factors, each scored 0–5 using only verified claims, weighted and scaled to 0–100. The
rubric file is explicit that v1's weights are a provisional starting guess rather than a derived result — they
are meant to change, but only when logged outcomes say they should.
| # | Factor | What it measures | Weight | Verification path |
|---|---|---|---|---|
| 1 | Regulatory pathway | Class, predicate or De Novo availability, review burden | 15% | connector openFDA |
| 2 | Reimbursement pathway | Code exists, pays enough, and is covered | 15% | connector CMS files |
| 3 | TAM | Addressable market from primary data, not aggregator blogs | 15% | partial volume only |
| 4 | FTO / IP | Blocking patents, expiry timelines, workarounds | 15% | connector PatentsView |
| 5 | Moat / defensibility | What stops a fast-follower once the idea is public | 10% | judgment no source |
| 6 | Mechanism & clinical risk | Plausibility; what bench or animal evidence must show | 10% | judgment no source |
| 7 | Capital intensity & time-to-revenue | Cost and calendar time to first dollar | 10% | judgment no source |
| 8 | Competitive intensity | Existing players, funding, launches, recalls | 5% | judgment no source |
| 9 | Strategic fit & portfolio balance | Hero bet versus cash-generative spoke | 5% | judgment no source |
queued-research. Whether the core count should replace the 6/9 threshold is a rubric
question, and rubric questions are decided by humans, not by the agent that would benefit from the
change.“Gate” here means something narrower than “step”: a point where a candidate, a claim, or a decision is tested and can be stopped. Some are decisions in a prompt; the important ones are functions that return a refusal. Below is the Stage 5 hard gate drawn out, then a catalogue of every gate and guard in the system.
| # | Gate | Fires when | Outcome | What it prevents |
|---|---|---|---|---|
| 1 | Verification triage Stage 2→3 | A candidate would fail a floor even on generous unverified claims | Parked, and named in the cycle report | Spending verification effort on ideas already dead on their own optimistic numbers |
| 2 | Jurisdiction tag present Stage 3 | A claim carries no jurisdiction: field | Stays unverified; omission flagged | Silently defaulting an unstated claim to US |
| 3 | Jurisdiction supported code | A claim is about the EU, UK or anywhere non-US | unconfirmed, with the reason recorded | Citing a real US 510(k) record as evidence for an EU pathway claim |
| 4 | Intended use stated Stage 3 | intended_use: is blank and a regulatory claim is being checked | Regulatory claims stay unverified; Factor 1 unscored | “Verifying” a device class in the abstract, which is meaningless rather than merely weak |
| 5 | Typed connector only Stage 3 | Any claim check | A parseable status plus a record ID, or an explicit “no connector for this” | A summarised web search being recorded as a primary-source check |
| 6 | CPT hard block code | Any CPT lookup, always | unverifiable, unconditionally | Scraping or reproducing AMA-licensed code descriptions |
| 7 | Structurally unverifiable code | Market-size or adoption/preference claims, always | unverifiable → treated as permanently unconfirmed | A model's plausible guess about market size or clinician behaviour entering the record as evidence |
| 8 | Missing cached file code | A CMS data file the connector needs is not in data/ | unconfirmed, naming the file and its source | Substituting a web search or a hand-typed table for a primary dataset |
| 9 | Only verified claims score Stage 4 | Composite computation | Unverified and unconfirmed claims contribute nothing; factors go blank | An idea looking more solid than its evidence |
| 10 | 6-of-9 threshold Stage 4/5 | Fewer than six factors carry a verified claim | queued-research — explicitly not a kill | Conflating “we don't know yet” with “the answer is no” |
| 11 | FTO blocked Stage 5 | A live, enforceable patent covers the mechanism with no workaround | Automatic kill, regardless of all other scores | Building on ground somebody else owns |
| 12 | TAM floor Stage 5 | TAM scores below 2/5 on verified data | Kill, logged with a revive condition | Pursuing a real device into a market too small to matter |
| 13 | Regulatory + reimbursement floor Stage 5 | Both score below 2/5 | Kill | A product that is both hard to clear and impossible to get paid for |
| 14 | Tier 3 boundary Stage 6 | A candidate clears the gate check | /flywheel-cycle stops; a packet is compiled, not a recommendation | An autonomous run walking a candidate into a shortlist, a budget or a study |
| 15 | Human decision write path code | Anything tries to record an approval | Only scripts/human_decision.py writes it — non-interactive stdin refused, confirmation phrase typed, Bash access denied in settings | Any agent code path reaching a Tier 3 outcome |
| 16 | Two-queue separation schema | Routing a question upward | escalation_queue for model-to-model synthesis; human_review_queue for anything a person must decide | A Tier 3 judgment being quietly answered as a Tier 1/2 one “for a second opinion” |
| 17 | Rubric edit convention + queue | Anyone wants to change a weight, floor or factor | Written proposal → queue → human decision → Librarian applies it citing a calibration row | An agent tuning the scoring function that judges its own output |
| 18 | Diligence precondition Stage 7 | /diligence is run on a candidate | Refuses unless status: shortlisted — i.e. a human already said yes | Deep work, and its implicit momentum, being spent ahead of the human decision |
| 19 | Append-only history Stage 8 | Any edit to the killed-idea or calibration logs | Append; a revived idea gets a new linked entry, never an erased one | Losing the record of what was believed and when — the only thing calibration can be computed from |
Two queues, deliberately not one. Conflating them is the exact failure the split exists to prevent: an escalation queue's worst case is wasted tokens and a hypothesis to redo, while a human review queue's worst case is a real capital, legal or clinical decision made by something that cannot be held responsible for it.
human_review_queue and never through escalation_queue — no code path routes a Stage 6
item through a model first “for a second opinion”, because that would quietly convert a Tier 3 judgment into a
Tier 1 one.The rubric decides what gets killed and what gets funded, which makes editing it adjacent to allocating capital. So it follows the same propose-then-approve path — and the standard for approval is a citation, not an argument.
knowledge-base/rubric-proposals/ and the calibration log has no rows in it, because no candidate
has been through Stage 8 yet. The repo's own reading: deferred is a well-founded answer to every one
of them — the log's emptiness is information, not an obstacle to route around.The most useful thing this repository does is state, in its own README, the reason it cannot currently finish a lap. Only three of the nine factors have any verification path at all. The other six are not under-resourced — five of them are judgments rather than facts, and no database anywhere will resolve them.
The Core factors verified: n/4 readout is the instrumentation for deciding this properly: it
distinguishes “we know nothing about this candidate” from “we know everything the kill conditions actually
read, and nothing else”. Whether that should replace the 6/9 rule is written up in
knowledge-base/rubric-proposals/2026-08-21-core-factor-threshold.md and parked for a human.
Agents do not pass results to each other through conversation. They read and write the same markdown file on disk, in sequence, across separate invocations. That is what makes the Verifier/Scorer separation real rather than a naming convention — and it means the file itself is the audit trail: who claimed what, who checked it, against which record, and on what date.
| Stage | The same claim, as it evolves |
|---|---|
| Stage 1 | - [ ] A 510(k)-cleared hydrocolloid dressing predicate exists — jurisdiction: US — confidence: unverified — source: Generator's best guess |
| Stage 3 | - [x] A 510(k)-cleared hydrocolloid dressing predicate exists — jurisdiction: US — confidence: verified — source: api.fda.gov/device/510k · K123456 (fda.check_predicate_510k) — matched on device_name, decision date on record |
| Stage 4 | Factor 1 may now carry a score. Until that middle line existed, it could not. |
From the spec, and worth reading because it shows the pipeline producing a modest, useful answer rather than a heroic one.
| Stage | What happens |
|---|---|
| 0 · Scout | No new signal needed; the existing rubric applies. |
| 1 · Generator | “Hydrocolloid wound dressing, repackaged for canine/feline post-surgical use, sourced from an expired-patent human product.” |
| 2 · Scorer | Regulatory 5/5 (veterinary device, no premarket review), moat 2/5 (commodity). Flags “patent expired” as unconfirmed. |
| 3 · Verifier | Checks the specific product's patent family — confirms expiry, upgrades the claim. Confirms no antimicrobial or drug component, so it stays device-class rather than drug-class. |
| 4 · Re-score | Composite lands in the high 60s. Clears the gates; low moat. |
| 6 · Humans | Flagged as a “cash-generative spoke, not a hero bet” — approved at low priority and low budget. |
| 7 · Diligence | Sourcing agreements and fur-adhesive reformulation logged as the one available moat lever. |
| 8 · Outcome | Becomes the template for “expired-patent human device + veterinary repackage” — future Generator runs can propose variants of the pattern faster, and the Scorer can pre-fill confidence on similar claims. |
That last row is the flywheel doing its actual job: the value is not the dressing, it is that the next twenty ideas of this shape cost a fraction as much to evaluate.
Every pipeline stage is a slash command in a Claude Code session opened on the repo. They map 1:1 to stages, so a stage can be run or debugged on its own.
| Command | Stage | What it does |
|---|---|---|
/flywheel-cycle [theme] | 0–5 | One full autonomous cycle: scan, generate, score, verify, re-score, gate-check. Stops hard at Stage 6. |
/scan | 0 | Scout pass; then routes any signals through the Librarian to flag affected candidates |
/generate [n] [theme] | 1 | New candidates, checked against the killed-idea log |
/score idea-XXX | 2 | First-pass triage scoring |
/verify idea-XXX | 3 | Verifier in a fresh context, connector per claim |
/rescore idea-XXX | 4 | Composite from verified claims only |
/gate-check idea-XXX | 5 | Kill conditions on verified data |
/shortlist | 6 | Compiles the human review packet. Does not recommend |
/diligence idea-XXX | 7 | Deep package — requires status: shortlisted |
/log-outcome idea-XXX "…" | 8 | Records predicted vs actual into the calibration log |
| Command | Purpose |
|---|---|
pip install -r requirements.txt | One dependency: requests |
python3 init_db.py | Creates db/flywheel.db from db/schema.sql. Safe to re-run |
python3 -m connectors 510k "device name" | Any connector call; prints one JSON object with status, source_ref, detail |
python3 -m pytest tests/ | 33 tests, mostly asserting that guards refuse rather than answer |
python3 scripts/queue_rubric_proposal.py list | Rubric proposals awaiting a human |
python3 scripts/human_decision.py list | humans only Run in your own terminal — never through an agent |
docker-compose up --build | Same image, runs init_db.py at build time |
Every connector call exits 0 even when the lookup fails — a failed or unconfirmed result is a normal
answer, not a CLI error, so callers read the status field rather than the exit code. Anything
jurisdiction-specific takes --jurisdiction and defaults to US.
The data/ directory is deliberately empty in git. Four connectors read cached
CMS releases from it; none are checked in, because a stale copy would look authoritative while answering last
year's question. Until a file is there, its connector returns unconfirmed and says exactly what is
missing. A fabricated cache file would be indistinguishable from a real one to every agent downstream, which
makes it the single most dangerous thing anyone could put in that directory.
SQLite is the intended source of truth; the markdown files are the working surface agents actually use day to day. The two are not synced yet — that is stated as an open item rather than implied to be done.
| Table | Holds | Written by |
|---|---|---|
candidates | Idea, class, target, mechanism, intended use, jurisdiction, stage, status, composite | Tier 1 |
claims | Claim text, type, jurisdiction, status, source_ref, who verified it, when | Tier 2 |
killed_ideas | Kill reason and tags, so the Generator can be steered off the pattern | Tier 1 |
calibration_log | Predicted value, actual value, and what the gap implies | Tier 1 |
escalation_queue | Tier 1/2 questions gated up to a synthesis pass. Never a route to a human decision | Tier 1 |
human_review_queue | Anything a person must decide, plus a detail pointer to the full argument | any tier |
human_decisions | approved / rejected / deferred, who decided, and their note | humans only |
perspectives, insights | Stage 7 diligence output, scoped per candidate | Tier 1 |
| Path | What lives there |
|---|---|
docs/spec.md | The original architecture spec — stages, tiers, agent roster, worked example |
docs/phase2-spec.md | Addendum: structural tier enforcement, typed connectors, the SQLite data model |
docs/research/ | Competitive review of a commercially-sold medtech assessment, benchmarked against the rubric |
CLAUDE.md | Day-to-day operating rules for anyone — human or agent — working in the repo |
knowledge-base/rubric.md | The live 9-factor rubric, weights, floors, changelog |
knowledge-base/rubric-proposals/ | Written arguments for proposed changes, pending human decision |
knowledge-base/killed-ideas.md | Append-only: every gated idea and its revive condition |
knowledge-base/calibration-log.md | Append-only: predicted versus actual, the input to every rubric change |
knowledge-base/signals/log.md | Stage 0 output — what changed, and which candidates it affects |
knowledge-base/candidates/ | One file per idea; _template.md is the format and carries the rules inline |
.claude/agents/ | The six agent definitions, each with its tool list and its “never” section |
.claude/commands/ | One slash command per stage, plus /flywheel-cycle |
.claude/settings.json | Denies Bash access to the human-decision script from any session |
connectors/ | The typed lookups, the jurisdiction guard, the shared cache loader, the result type |
db/, init_db.py | Schema and migrations for the structural boundary |
scripts/ | human_decision.py (human-only) and queue_rubric_proposal.py (agent-runnable) |
tests/ | pytest suite, largely asserting that guards refuse |
Read together, the constraints have a shape. Almost every one of them makes the system slower and less impressive in exchange for making its output believable.
| It refuses to… | Because |
|---|---|
| Let a scoring agent verify its own claim | A checker that has seen the argument is reviewing a case, not checking a fact |
| Accept a summarised search result as verification | That is an opinion about a search result, not a primary source |
| Answer a non-US claim from a US database | The citation would be real, and about a different regulatory regime |
| Score a device class without a stated indication | Class is a property of the device plus its claimed use — the question is otherwise malformed |
| Treat “could not confirm” as “no” | One queues research; the other kills an idea. Collapsing them destroys the difference between ignorance and evidence |
| Guess a market size or a clinician preference | The answer would be fluent, plausible, and fabricated — the hardest kind of error to catch |
| Report a reimbursement pathway from a code lookup | A code that exists, pays $8, and has no coverage policy is a code, not a pathway |
| Let an agent record a human decision | Not because it would decide badly, but because nobody could be accountable for the decision afterwards |
| Let an agent tune the rubric that scores its work | A model that can adjust its own grading function will, eventually, describe its own priors as findings |
| Lower its threshold to produce a shortlist | “Nothing cleared” is a true answer. A stream of under-evidenced shortlists is a false one |
The point of all of it
A pipeline like this is not trying to be right more often than a person. It is trying to be wrong in ways that are visible — every claim carrying a tag, every tag carrying a record ID, every gap named as a specific document somebody would have to go and get. An idea that arrives at a human with three verified factors, six honest blanks and a list of what each blank would cost to fill is worth more than a confident score with nothing underneath it.