AI Agents That Critique Their Own Work

01 The Philosophy

AI producing content isn't interesting. AI honestly evaluating its own content — and killing what doesn't pass — is.

The standard AI content pipeline is: generate → publish → forget. Every piece ships because it exists, not because it's good. The result is the internet's flood of mediocre, interchangeable AI text.

We built the opposite system. Every piece goes through adversarial self-critique before it can publish. The same AI agents that write the content also serve as their own harshest critics — scoring on explicit rubrics, catching factual errors, identifying AI voice patterns, and rejecting work that doesn't meet the bar.

Three Principles

1. Honest scoring beats rubber-stamping. Self-scores are routinely marked down during independent audits. Our quality audit found games over-scoring themselves by 2-10 points on the old /30 scale. The audit catches it.
2. Minimum bars are real. Articles need 8+/10 after 3+ revision cycles. Games need 60/100 minimum to ship. Anything below B-tier gets two improvement cycles, then gets cut.
3. The process is the product. We don't publish our best first draft. We publish the result of a draft being attacked, defended, and rebuilt — sometimes 4-5 times.

02 Live Stats

Current counts across all three sites and the games/experiences catalog.

03 Article Evaluation

Draft → Evaluate → Criticize → Revise → Publish. Minimum 3 revision cycles and 8+/10 to ship.

What the Critics Actually Catch

These aren't gentle suggestions. Every revision cycle runs a genuinely adversarial critique that looks for specific failure modes:

14 Journalist Voices

Each site maintains distinct journalist personas with specific beats, voices, and editorial standards. LITF alone has 14 journalists — from Kai Nakamura (autonomous transport, engineering-precise) to Nadia Kovac (labor & AI policy, former labor reporter energy) to Maya Ramirez (education & learning, data-driven with teacher-interview texture). Each voice has banned patterns and required tones that the critique cycle enforces.

The critique doesn't just check facts. It checks whether the wrong journalist's voice leaked in. A revision caught Rex Driverton's noir voice ("Invisible at 2 a.m. in any Walmart parking lot in Ohio") appearing in Dale Impactor's sports-stats column. Fixed.

04 The 6-Critic System

For high-stakes articles, we run 6 parallel AI critics per revision round. Each critic has a distinct lens and independently scores the draft. The 6th critic — Research Rigor — holds articles to the standards of highly-cited scholarly papers.

The Six Critics

Research Rigor: What It Means

The 6th critic was added after we noticed that articles could score 8+/10 across five dimensions while still being fundamentally synthesis — well-written summaries of other people's work with no original contribution. Highly-cited scholarly papers share five traits that our articles lacked:

• Novel contribution. The paper contains at least one finding, calculation, or test that didn't exist before. Synthesis of existing work, no matter how well-organized, scores low. Our articles must contain at least one original analysis: a calculation nobody ran, a dataset nobody combined, a comparison nobody made.
• Limitations acknowledgment. Not inline hedging ("to be sure...") but a dedicated, honest accounting of what the article didn't prove, what data was missing, and where uncertainty remains. The best papers make their weaknesses explicit because it strengthens the parts they're confident about.
• Strongest counterargument. Not a strawman paragraph that gets knocked down in the next sentence. The best counterargument to the article's thesis must be stated at full strength, engaged with seriously, and either rebutted with evidence or acknowledged as a genuine limitation.
• Verifiability. Every factual claim should be traceable to a cited source that the reader can check. "According to researchers" scores 0. "According to Chen et al. (2024), Table 3" scores 5. If a reader can't verify a claim, the article is asking for trust it hasn't earned.
• Methodology transparency. When the article makes claims involving numbers — cost comparisons, statistical trends, market projections — the math must be shown. Not just "costs would increase 340%" but the actual calculation with inputs, assumptions, and the formula used.

How It Works — The Phased Pipeline

Inspired by gstack's explicit cognitive modes, each article advances through 5 phases. Each cron cycle (every 2h) wears one hat — never trying to research, write, critique, and publish in the same session.

Why Phases?

• No more timeouts. Previous crons tried to research + draft + critique + publish in 600 seconds. That's like running gstack's plan, review, and ship in one session.
• Better research. Phase 1 is dedicated thinking time — "is this the right story?" — not "what's the quickest topic I can draft?"
• QA catches broken deploys. We never checked if the live site actually worked. Now it's a mandatory phase.
• State persists across cycles. drafts/status.json tracks phase, round, scores, and journalist. Each cycle picks up where the last left off.

What We Learned

• Easy gains exhaust by round 3-4. Scores plateau around 8.0. Breaking through to 8.5+ requires something the critics can't give you: an act of genuine reporting or a human insight.
• Critics converge on the same issues. When 3+ critics flag the same problem (e.g., too many em dashes, self-congratulatory ending), that's high-confidence signal.
• Voice is the hardest dimension. Consistently scores lowest. AI recognizing its own voice patterns is inherently limited — you can't see biases you share with the thing you're evaluating.
• Em dash count is a reliable proxy for AI voice. 19 em dashes = obvious AI. 3 = human-passing. Now enforced as a hard metric.
• Research rigor is the newest and most differentiating critic. An article can have perfect voice, solid ethics, and great shareability while contributing nothing original. The rigor critic forces articles to discover something, not just organize what's known.
• Human input breaks the asymptotic ceiling. The best insights in our copyright article came from the human editor noticing things 35 AI critics missed across 7 rounds.

05 Games & Experiences Rubric

10 criteria × 5 points = 50 max, displayed as /100. Built specifically for smart glasses with bone conduction audio and a 5-button D-pad.

Tier System

• S-tier (90-100) — Would genuinely recommend to a stranger. Exceptional, polished, glasses-native.
• A-tier (76-89) — Ship proudly. Strong across most dimensions.
• B-tier (60-75) — Solid but has clear weaknesses.
• C-tier (40-59) — Cut candidate. 2 improvement cycles or remove.
• F-tier (<40) — Remove. Phone does it better.

Game Scores

Hover over any score to see the 10-dimension breakdown. Sorted by score, highest first.

Scores are Metacritic-calibrated honest evaluations. Each game scored against its genre benchmark (e.g., Dungeon Crawl vs NetHack, Stalk vs Metal Gear Solid). 100 is effectively unreachable.

The Audit Process

Every game and experience went through an independent quality audit where the evaluator reads the actual source code — not just the inventory description. The audit found:

• Self-scores inflated by 1-5 points on average (experiences were worse offenders than games)
• Dead code and abandoned features presented as working (e.g., Photon Dodge claimed mic-reactive bullets but had zero mic integration)
• False persistence claims (features described as "persisted in localStorage" with no actual save/load code)
• The #1 systemic weakness: zero localStorage persistence — settings and progress lost on every page reload
• The universal fix: rank progression based on cumulative achievement, confirmed working 12 times across the catalog

What Got Fixed

Two items started at B-tier and were improved to A through targeted interventions:

• Neck Stretch (44→88/100) — Added breathing guide as a core mechanic: mic detects breathing, good sync speeds up hold timer by 35%. The mic went from absent to gameplay-critical in 3 improvement cycles.
• Particle Life (44→70/100) — Added ecosystem audio drones (6 species oscillators), replaced destructive Enter-to-reset with a Shepherd Pulse ability (3.5× force burst on cooldown), and added rank progression.

06 Anti-AI Voice Rules

AI-generated text has recognizable tells. We actively hunt and remove them.

Banned Phrases

These phrases are instant red flags during critique. If any appear in a draft, the revision cycle kills them:

Structural Tells

Beyond individual phrases, AI text has structural patterns that trained readers spot instantly:

• The Setup-Pivot: "That's not a [mundane thing]. It's a [grandiose reframe]." — Every. Single. Time.
• Uniform paragraph length: AI defaults to 3-4 sentences per paragraph. Real writers vary from one word to a full page.
• The consulting-deck transition: "The logical next step is..." / "The trajectory points toward..." — Nobody talks like this.
• Reflexive hedging: Starting paragraphs with "To be sure," or "Of course," before making a point. Just make the point.
• Category-exhaustion lists: Listing exactly 3-5 items for every point, each the same length. Real arguments are messy.

What We Require Instead

• Vary sentence length. Fragment. Then a 40-word sentence that builds and builds. Then a question? Then back to short.
• Have opinions. "This is a bad product" is more honest than "This product presents certain challenges."
• Read-aloud test. If you wouldn't say it in conversation, don't write it.
• Hyperlinked sources only. Every claim links to a source. No "according to industry experts."
• Name names. Companies, researchers, dollar amounts. Not "leading players in the space."

07 EVALUATE.md

The complete evaluation playbook used for games and experiences. Copy the raw markdown to use in your own projects.

# EVALUATE.md — Games & Experiences Quality Playbook

## Scoring Rubric (10 dimensions × 5 = 50 raw, displayed as /100)

### Original 6 Dimensions
| Criterion      | 1 (Bad)                      | 3 (OK)                  | 5 (Great)                          |
|----------------|------------------------------|-------------------------|------------------------------------|
| Trigger moment | Can't think of one           | Vaguely useful sometimes| Specific: "I'm at X doing Y"      |
| 5-second hook  | Confusing, needs explanation  | Makes sense, mildly int.| Instantly delightful               |
| Glasses advant.| Phone is better              | About equal             | Clearly better hands-free          |
| Return visits  | Once and done                | Maybe weekly            | Daily habit potential              |
| D-pad fit      | Awkward, needs more inputs   | Works but clunky        | Natural, satisfying                |
| Audio/context  | Ignores mic and sensors      | Uses one sensor         | Deeply integrated with environment |

### 4 New Dimensions (added March 2026)
| Criterion          | 1 (Bad)                    | 3 (OK)              | 5 (Great)                                  |
|--------------------|----------------------------|----------------------|--------------------------------------------|
| Session Variance   | Identical every time       | Some randomization   | Deeply procedural with emergent gameplay   |
| Strategic Depth    | Pure reflexes, no decisions| Some tactical choices| Deep resource management and tradeoffs     |
| Surprise/Discovery | Fully known in 30 seconds  | Some unlockables     | Genuine emergent discoveries               |
| Craft              | Functional but generic     | Well-made            | Has a "wow, that's clever" moment          |

## Tier System (scores displayed as /100)

- **S-tier (90-100)**: Would genuinely recommend to a stranger. Exceptional.
- **A-tier (76-88)**: Ship proudly. Strong across most dimensions.
- **B-tier (60-74)**: Solid but has clear weaknesses.
- **C-tier (40-58)**: Cut candidate. 2 cycles or remove.
- **F-tier (<40)**: Remove. Phone does it better.

**Minimum score to ship: 60/100**
**Scores are Metacritic-calibrated. 100 is effectively unreachable.**

## Genre Benchmarks

Every game scored against its spiritual benchmark — the best game in its genre.
A 90/100 means it captures the core loop and adds something only glasses can do.
95+ would mean the game does something its benchmark CAN'T.

| Game            | Benchmark                          |
|-----------------|------------------------------------|
| Dungeon Crawl   | NetHack / Brogue                   |
| Sonar Sub       | Subnautica                         |
| Fisher          | Stardew Valley (fishing)           |
| Gravity Sling   | Angry Birds / Kerbal Space Program |
| Trader          | Offworld Trading Company           |
| Stalk           | Metal Gear Solid                   |
| Terraform       | SimCity / Dwarf Fortress           |
| Mine            | Motherload / SteamWorld Dig        |
| Signal          | Missile Command / Papers Please    |
| Hex Collapse    | Tetris / Hexic                     |
| Photon Dodge    | Ikaruga / Touhou                   |
| Rhythm Pulse    | Beat Saber / Crypt of NecroDancer  |
| Duel            | Quick Draw / WarioWare             |

## What Works on Glasses (Validated)

### Proven Patterns
- Ambient + glanceable: check for 5 seconds, not 5 minutes
- Audio-reactive: mic responds to sound = magical
- One-hand, zero-attention games: D-pad while walking
- Utility you'd open daily: compass, sound meter, metronome
- Breathing/meditation: always-on HUD, no device to hold

### What Makes Glasses Different From Phone
- Hands-free — you're doing something else
- Ambient awareness — real world + app simultaneously
- Mic always available — audio-reactive is the killer feature
- Contextual — compass while hiking, decibel at a concert
- Glanceable — information at a glance, not deep interaction

## What Fails on Glasses (Anti-Patterns)

- "Pretty animation" with no purpose — cool for 5 seconds, never opened again
- Complex interactions — D-pad has 5 buttons. If you need more, it's wrong.
- Things that need precision — pixel art, drawing, cursor work
- Deep reading — nobody reads paragraphs on a HUD
- Things phones do better — calculators, text input, keyboards
- Tech demos — "look, I can render particles!" is not a product

## Red Flags in Proposals

- "Interactive [noun]" where the interaction is just scrolling
- Anything that needs a tutorial longer than one sentence
- "Generative art" that's random pretty colors with no agency
- Experiences that only work sitting at a desk
- Games that require sustained focus for >2 minutes

## Key Lessons Learned

- If a game has idle D-pad during any phase, add player input
- "Environment IS the game variable" is the strongest pattern
- Rank progression is the universal Return:4→5 fix (confirmed 12×)
- If it has ZERO audio, it's broken on an audio-first platform
- Measure + feedback loop beats passive display every time
- Turn-based > real-time for glasses (user is multitasking IRL)
- If >50% of use is passive watching, add agency to the idle phase

08 AI Agent Assessments

We asked four AI agents with different perspectives to honestly evaluate this evaluation system. They were not prompted to be kind.

09 The Unified Scheduler

One dispatcher replaces ten independent crons. Inspired by OS process schedulers: priority classes, concurrency limits, and backpressure. Read the full story →

Architecture: Before & After

Priority Classes

Concurrency Limits

Backpressure rule: If the previous dispatch is still running (subagent hasn't returned), skip the entire cycle. Don't pile on.

Phase Priority (P2 Dispatch Order)

When the scheduler has multiple phases ready across sites, it picks by this priority:

1. SHIP — Cheapest, unblocks the pipeline. Publish what's ready.
2. QA — Verify what shipped. Quick live-site checks.
3. CRITIQUE — Expensive but blocking. The article can't advance without it.
4. DRAFT — Moderate. Build the article from research notes.
5. RESEARCH — Can wait. Cheapest but least urgent.

The 5-Phase Pipeline

Every article matures across 5-7 scheduler cycles (10-14 hours). One phase per dispatch. State tracked in drafts/status.json per site.

1. RESEARCH: Find the story, challenge the thesis ("Is this the right story?"), identify 3+ primary sources. Kill if sources don't hold up.
2. DRAFT: Write the full article with hero image, meta tags, anti-AI voice rules. Self-score to establish baseline.
3. CRITIQUE: 6 parallel critics score the draft. Revise and repeat until all 6 score 8.5+. Max 3 rounds. Park if stuck.
4. SHIP: 1/day gate check, run validation script, update index + sitemap, commit + push, newsletter send.
5. QA: Fetch live URL, verify 200 response, image loads, meta tags accessible, article in index and sitemap.

Maximum 1 article per day per site. If the SHIP phase fires and one was already published today, it waits for the next cycle.

P3 Idle Work Rotation

When no article phase is ready (all sites waiting or freshly published), the scheduler rotates through maintenance tasks:

State Machine

The scheduler reads from two state files:

• scheduler/state.json — Global: concurrency counters, dispatch history, P3 rotation index, daily stats, backpressure tracking
• {site}/drafts/status.json — Per-site: current article slug, phase, round, critic scores, journalist, publication history

Task definitions live in scheduler/tasks/ — one file per task type containing the full pipeline instructions, quality gates, and git setup. The scheduler reads the task file and passes it to the dispatched subagent.

Remaining Standalone Crons

Down from 10 crons to 3. The other 7 tasks are now dispatched by the scheduler based on priority, not timers.

10 Quality System Evolution

The quality system isn't static. It improves — but not as fast as it should. Here's an honest accounting.

What Self-Improves

• EVALUATE.md lessons log. Every time a game improvement reveals a pattern (e.g., "rank progression is the universal Return:4→5 fix, confirmed 12×"), it's documented. Future cycles apply these lessons automatically.
• QUALITY.md tier rankings. Games and experiences are re-evaluated every cycle. Tiers shift based on improvements or regressions. F-tier items get cut.
• Anti-AI voice rules. New banned patterns accumulate as critics identify them. The "structural tells" section grew from conversation, not from the cron prompts.
• This page. The weekly aipm-update cron re-evaluates whether methodology has evolved and updates stats, tier distributions, and assessments.

What Doesn't Self-Improve (Yet)

• Cron prompts are static. The markdown definitions in cron.d/ don't update themselves. When the publish threshold should be raised from 8/10 to 9/10, a human has to do it.
• Scoring rubric expanded from 6→10 dimensions (March 2026). New criteria: Session Variance, Strategic Depth, Surprise/Discovery, and Craft. Genre benchmarks added (NetHack, Metal Gear, etc.). Metacritic-calibrated — 100 is effectively unreachable. 3 games have earned S-tier (90+) honestly.
• Publish threshold raised from 20/30 to 60/100. 8/10 for articles was the initial bar. Games went from 20/30 (old system) to 60/100 (new system).
• The critique prompts don't learn from past critiques. Each cycle's adversarial review starts fresh. It doesn't know what the last 10 critiques found.

Case Study: The Copyright Article

The strongest example of quality evolution in action was the copyright reckoning article (published March 2026). It went through 7 rounds of critique with 35 subagent reviewers across 5 dimensions (general, social/shareability, ethics, voice, legal accuracy).

Key moments that leveled it up:

• v2: Training pipeline + displacement sections (5.9→7.1)
• v5: "$300/month vs $900K/year" stat, BLS data (7.8→8.0)
• v6: Fake bylines disclosure, journalist texture (8.0→8.2)
• v7: Common Crawl discovery — the article's only act of genuine reporting (8.2→8.5)
• Post-pub: "The layers don't bottom out at 'human'" — caught by the human editor, missed by 35 AI critics (8.5→8.7)

The last point is the most revealing: the article's best insight came from a human noticing something 35 subagent critics missed across 7 rounds. Self-critique has an asymptotic ceiling. The system needs human input to break through it.