Yeah
AI Builder + Marketer · Still in Love AI Studio · 2026
Rotten Tomatoes for YouTube reviews
4,516 YouTube Videos
26,056 Claims Extracted
2,576 Themes Synthesized
3,442 SKUs Catalogued
342 Verdicts Published
Run in the world
Human evaluation
Generative AI distribution
More reviews + opinions
Millions of takes. One verdict: Buy. Skip. Consider.
Can a world-coupled pipeline combine probabilistic inference with deterministic checks to turn thousands of hours of video into auditable, 30-second decisions?
What would help someone trust that decision?
InsideClose
The challenge extends beyond a single model response: building a system that keeps incorporating new reviews, combines probabilistic interpretation with deterministic checks, and makes its conclusions traceable. As people turn to creators for advice, I wanted to help them weigh those opinions without spending hours watching.
LLMs synthesize; code enforces contracts.
Asking a model to cite its sources is fragile. A deterministic Python harness assigns run-relative claim IDs and derives citations, so a buyer can follow a sentence back to its evidence.
DetailsClose: LLMs synthesize; code enforces contracts.
A paragraph
Buyer-facing writing points to the themes it draws from.
Its supporting theme
A theme identifies the particular opinions behind it.
The attributed opinion
The claim keeps the exact source identity used in synthesis.
The source moment
The evidence resolves back to the creator's original video and recorded timestamp quality.
Change the contract, not the model.
Adjudicating absolute truth failed in every trial. Aggregation with per-item failure containment made it possible to finish a run without one bad item taking down the pipeline.
- Adjudication
- 27 / 27 failed
- Aggregation
- 233 consecutive clean runs
- Cost per run
- ~$0.105
DetailsClose: Change the contract, not the model.
The system, end to end.
Two independent engines turn review evidence into a grade and an explanation. They meet in the runner’s product page.
Evidence
Shared source materialThe grade
Explicit grading rulesThe explanation
Grounded reasoningPublication
Editorial path onlyDelivery
Stored grade + current editorialCreator reviews
Independent running-shoe reviews enter through submitted videos, channels, playlists and scheduled discovery. Each voice keeps its identity.
- The queue distinguishes video, channel, playlist and scheduled acquisition jobs.
- Source records connect videos to creator metadata.
Source identity
Video and channel metadata establish who spoke, what the review covers and where the original can be found.
- The metadata client uses the official YouTube Data API.
- Channel, video and playlist lookups preserve their source identifiers.
Timestamped words
Transcript segments preserve the reviewer's words and their position in the video, giving later claims a place to point back to.
- A configurable acquisition chain accepts supplied files and enabled providers.
- A fetched document must match the requested source ID.
Evidence store
Source records and timestamped transcripts are stored before interpretation. Later stages can reuse evidence without acquiring it again.
- Source artifacts preserve the original transcript payload and acquisition provenance.
- The ingestion handler reuses existing artifacts before calling a provider.
Corpus search
Operators can search the stored transcript corpus without repeatedly parsing every original artifact or making another model call.
- A rebuildable PostgreSQL full-text projection uses TSVECTOR and a GIN index.
- This admin-only projection is separate from the original timestamped evidence.
Mention extraction
Structured extraction identifies product mentions, resolves which shoe they concern and removes duplicates before they reach the grading engine.
- Time-marked windows pass through extraction and product resolution.
- Accepted records are deduplicated and persisted as mentions.
Stored mentions
Product-linked review evidence is stored with source identity. This is the input to the deterministic verdict engine.
- Mention records connect the product, source and extracted review evidence.
- Recomputation reads those records rather than generated editorial prose.
Deterministic verdict
A separate grading engine computes score, tier, verdict and confidence from stored review evidence. The editorial model does not choose the grade.
- The pure consensus engine applies shared, explicit grading rules.
- Its result is stored as a product verdict snapshot.
Attributed opinions
The editorial pipeline begins by extracting what each reviewer said. An unsupported claim can be removed while faithful claims from the same source survive.
- Input
- Timestamped transcripts
- Output
- Attributed claims
- Stage A produces source-specific distillates with claim-level admission.
- The cached output records transcript content, model, prompt and product scope.
Grounded themes
Opinions become useful themes: agreement, disagreement, minority views and individual observations. Each theme cites the particular claims that support it.
- Input
- Attributed claims
- Output
- Evidence-backed themes
- Stage B resolves exact Stage A claim IDs through the harness.
- A durable product theme set carries the evidence identities used in synthesis.
Buyer editorial
Grounded themes become clear writing about who a shoe suits and what tradeoffs reviewers describe. The writing can interpret buyer relevance without inventing product facts.
- Input
- Evidence-backed themes
- Output
- Product graph
- Stage C rewrites accepted themes and audits for invention or reversal.
- Its durable draft remains separate from the versioned publication.
Fidelity checks
Checks protect against fabrication, distortion and broken traceability. They act on individual claims, themes and paragraphs, while recording what was actually verified.
- Deterministic validation works alongside configured model verification.
- Verification coverage and exclusions remain recorded rather than implying universal review.
Pinned provenance
A published claim stays connected to the exact evidence used to write it, even if another ingestion run later changes the available corpus.
- Each selected distillate is pinned by ID with its full cache identity.
- Theme sets retain those pins for stable publication readback.
Durable queue
Work is recorded as database jobs, so acquisition and reasoning can continue independently of a page request or an operator's session.
- Active jobs are deduplicated by kind and entity.
- Workers claim queued rows with FOR UPDATE SKIP LOCKED.
Bounded workers
Workers claim a job, process its units and record progress. A maintenance pass recovers stale work and keeps operational state visible.
- The worker dispatches registered handlers outside the short claim transaction.
- Heartbeats and a stale-job sweep support recovery.
Progress checkpoints
Completed units are recorded as work proceeds. A recovered job can continue from its progress instead of repeating every finished unit.
- The worker commits per-unit progress and refreshes heartbeats.
- Budget pauses preserve the job's recorded progress.
Bounded recovery
Recovery handles infrastructure and unusable provider responses without repeatedly regenerating sound evidence because a checker disliked the content.
- The reasoning harness bounds producer retries and distinguishes infrastructure failures from low yield.
- Stages reuse durable upstream outputs rather than discarding valid prior work.
Spend controls
An estimate precedes execution. Per-call usage is recorded and reconciled, while a daily spend check can stop workers from claiming more billable work.
- The ledger stores estimated and actual cost, and jobs can pause for budget.
- In-flight work can still overshoot; this is not an exactly-once spending guarantee.
Models by role
Extraction, synthesis, writing and verification have distinct model roles. A run records its chosen configuration instead of leaving the model identity implicit.
- Explicit request choices override saved prompt choices and curated defaults.
- The runtime supports configured transports and records routing violations.
Durable stage outputs
Opinions, themes and draft editorial have their own stored outputs. One stage's failure does not require purchasing the work of every earlier stage again.
- Distillates, product theme sets and editorial drafts are persisted independently.
- Model, prompt and contract identity distinguish reusable outputs.
Publication preview
The operator can inspect the exact buyer-facing package before publishing a version. Its prose, citations and lineage are captured together.
- The publication bridge exposes a free preview and idempotent version creation.
- Its frozen public record excludes private runtime and transcript payloads.
Explicit activation
A published candidate becomes the current editorial only through an explicit activation choice. Producing a draft and making it public are distinct actions.
- The bridge appends a version linked to its source run.
- Only an explicit make_current choice changes the active editorial.
Version rollback
A previous editorial version can be restored without asking a model to recreate what the reader saw before.
- The publication bridge retains version and previous-version identity.
- Its rollback operation restores the previously current editorial.
Stored-result API
The product endpoint reads stored verdicts, editorial and evidence. Opening a product page does not launch the reasoning pipeline.
- The product response combines separately stored outputs.
- Import-boundary tests keep model runtime and pipeline modules out of the public read path.
A useful answer
The buyer sees a verdict, an explanation and the sources behind it. Separate systems come together in one understandable product page.
- The frontend displays the backend's verdict rather than recomputing a grade.
- Evidence links expose source identity and timestamp quality.
Discovery surfaces
Published product and source context also supports discovery through feeds, creator pages and rankings. Evidence becomes part of finding the product, not only explaining it.
- The feed reuses the shared publication predicate.
- Frontend clients expose creator and contribution leaderboard surfaces.
Typed contracts
Explicit data contracts connect Python services and the TypeScript interface. Optional additions preserve older responses while the system evolves.
- Strict Pydantic models define consensus inputs and outputs.
- The frontend's additive reasoning contract preserves the existing editorial surface.
Pinned dependencies
The backend CI installs a frozen dependency set so validation runs against the versions recorded in the repository.
- CI installs Python through uv and uses uv sync --frozen.
- The checked-in lockfile controls dependency resolution.
Hermetic validation
Backend CI tests behavior without provider calls or production data. Focused contracts protect the separation between reasoning, evidence and public reading.
- GitHub Actions runs pytest and a blocking lint check over the transcript surface.
- Publication tests check that evidence readback uses pinned source records.
Review artifacts
A branch and pull request carry each implementation slice into review. Recorded changes and check output give the next reviewer a durable starting point.
- The operating model names GitHub PRs as the shared review artifact.
- It requires command output rather than an unsupported claim that checks passed.
Runtime and checks supervise the work. Dashed rules distinguish them from the data flow above.
Runtime
Safeguards
Toolchain
Dissent is context, not noise.
A review depends on weight, pace, terrain, and foot shape. Contrasting citations sit beside supporting ones so a runner can find the experience closest to their own.
DetailsClose: Dissent is context, not noise.
I paired model-based evaluation with category judgment and weekly run-club field testing.
The same structured evidence powers the product pages and the creative that helps people discover them, with advertising spend guided by performance signals.

A source-linked verdict from Yeah · September 2026 Leave the grade blank when the evidence is thin.
The Evidence Gate withholds tier grades below three distinct creators. An incomplete catalog is more useful than a full grid of grades the evidence cannot support.
170 of 340 published shoes · grades withheld by Evidence Gate, PR #142
DetailsClose: Leave the grade blank when the evidence is thin.
86× cheaper extraction. 45% fewer citations. - Cost per video
- $0.43$0.005
- Citations per video
- 11.96.6
I reverted the cheaper extraction model when citation coverage fell. Coverage became a daily release check alongside category judgment and field testing.



























