PT-BREN
CTRL K

    Startups & Tech · 25 May 2026 · 11 min read

    Code review became the bottleneck. CodeRabbit won't save you alone

    A team went from 2 weeks of PR backlog to zero human review on staging. How CodeRabbit works, where it breaks, and why CLAUDE.md is the single source.

    Pablo Winter Pablo WinterPartner · Engineering
    Too lazy to read? Ask AI

    TL;DR

    AI sped up dev. The bottleneck moved to review. I saw a consulting team with 2 weeks of PR backlog waiting on the tech lead, and the team thinking the fix was hiring another senior. It isn’t. Dev cadence changed, review cadence didn’t. CodeRabbit can clear that queue and leave the PR pipeline to develop 100% autonomous after about a month of calibration. It works. But there’s a catch: the team starts trusting the pipeline so much they drop the reflex to test locally. And then the deploy breaks on staging because of a bug nobody saw running.

    This post is about both sides.

    AI didn’t eliminate the bottleneck. It pushed it to the tech lead.

    Look at the number: code co-authored with AI generates 1.7x more issues per PR than 100% human code. Source is CodeRabbit’s own State of AI Code Generation Report, analyzing 470 PRs from open source projects in December 2025. The finding is consistent with what any tech lead who adopted Cursor or Claude Code on the team is seeing in practice.

    It tracks: the dev produces more code, faster, and not always with the same context load they had when writing it all by hand. More code + less context = more stuff to review and less automatic confidence that the author knows what they’re doing.

    Look at the effect on the team:

    The tech lead becomes a funnel. I worked with a consulting team where the PR review backlog hit 2 weeks. The senior in charge was waking up at 6am to review before standup, staying after hours to review before bed, and the queue still grew. The team thought it was understaffing.

    It wasn’t. Code review (the step where another human validates the PR before merge) became the new bottleneck in the delivery pipeline. The individual dev got faster. The collective process didn’t.

    The bottleneck just keeps moving.

    The month the tech lead trained the bot

    The move was rolling out CodeRabbit (an AI code review bot that comments line by line on every PR) with the tech lead piloting it for a full month. It wasn’t “install it and let everyone loose”. It was:

    1. CodeRabbit comments on the PR
    2. Tech lead reviews on top: confirms what’s right, pushes back on what’s wrong
    3. When they push back, they go into .coderabbit.yaml and add a rule for next time
    4. When CodeRabbit misses something important, they go into .coderabbit.yaml and add a path instruction: a review instruction written in natural language paired with a file glob
    5. Repeat

    Within two weeks the number of rules the tech lead added per day dropped. Within three weeks CodeRabbit was getting more right than wrong. By the end of the first month the curve flattened: a new rule became the exception.

    The unlock was wiring up two things CodeRabbit doesn’t catch on its own:

    The second one changes the game more than it sounds.

    Why does requiring a JIRA ID in the PR description change the game?

    Because CodeRabbit stops reviewing just code and starts reviewing whether the PR delivers the story. Are the acceptance criteria in the US? Then the bot checks each AC against the diff and flags: “AC #3 mentions duplicate email validation, but I don’t see that check in the PR”. This isn’t opinion: it’s a checklist.

    Except there’s a prerequisite few teams want to face: the US has to be properly sized with decent AC. I see team after team failing exactly there. PO ships a giant, vague US, with AC like “validate form”. CodeRabbit reads that and can’t do anything with it. Then people decide the tool is useless. It isn’t. The refinement is.

    Without proper AC, CodeRabbit is just a ruler with no markings.

    Today, PRs to staging don’t go through a human anymore

    After that month of calibration, here’s what changed in the flow:

    “You let AI approve code on its own. This will go bad.”

    That’s the comment that shows up every time I tell this story. Usually from someone who’s never watched a tech lead burn 8 hours a day on PR review instead of doing architecture. Yes, we do. On staging. Where the worst case is the deploy breaks and we roll back. Not production. Staging.

    And the practical difference: the tech lead is back to doing architecture. The team ships more. The dev gets CodeRabbit feedback in minutes instead of days.

    CodeRabbit vs GitHub Copilot Code Review vs Greptile: which one?

    Short answer: depends on what hurts most.

    Small team already paying for Copilot: start with Copilot Code Review and see how far it gets you. Tech lead drowning in review backlog: CodeRabbit pays for itself in the first month.

    And honesty: an independent audit of 28 PRs reviewed by CodeRabbit found 15% of comments were “useless/noise” and 21% were nitpicking. It’s not a silver bullet. You have to tune it. You have to teach it. You have to use the learnings. Whoever installs and walks away will complain it’s bad. Because for that usage, it is.

    One file, three brains: CLAUDE.md as single source

    This is the trick few people have caught onto yet.

    CodeRabbit auto-detects CLAUDE.md, AGENTS.md, .cursor/rules/*.mdc, and .github/copilot-instructions.md as knowledge base. The rule you write once in CLAUDE.md applies to:

    1. Claude Code while coding: follows the rule when writing
    2. CodeRabbit while reviewing: checks the diff against the same rule
    3. Cursor on autocomplete: respects the convention

    One file, three brains reading. You stop maintaining the same rule duplicated across three systems. The PR that goes up is already almost approved because it was written under the same rules being checked at review.

    And there’s another piece that closes the loop: CodeRabbit’s CLI (coderabbit --prompt-only) spits the review feedback in a format consumable by an agent. You can build a slash command in Claude Code that resolves the comments in a loop and keeps pushing back until the bot approves.

    Save this as .claude/commands/coderabbit-loop.md in the repo and use /coderabbit-loop in Claude Code:

    Resolve CodeRabbit comments on the current PR until approval.

BEFORE accepting any suggestion, invoke the `receiving-code-review` skill
from the superpowers plugin. Without it, you become the bot's doormat.

Flow:
1. Run `coderabbit --prompt-only` and capture the comments
2. For each comment:
   - If it makes technical sense: apply the change and commit with a
     message tied to the comment ("addresses CodeRabbit: <summary>")
   - If it does NOT make sense: reply on the PR with technical
     justification and mark as wontfix via `@coderabbitai resolve`
3. `git push` on the branch
4. Wait for re-review (poll the PR via `gh pr view` every 60s, max 5min)
5. If there are still new unresolved comments, go back to step 2
6. Stop when CodeRabbit approves OR after 5 iterations
   (at that point, call the human: there's likely real disagreement)

Use `gh pr view --comments` for status. Use `gh pr comment` to reply.
Never `--force-push`: always incremental commits.

    The receiving-code-review line isn’t a detail. It’s the point.

    Without it, Claude Code accepts any CodeRabbit suggestion in “performative agreement” mode: agrees to look polite, refactors code that was fine, and the PR grows with changes that shouldn’t exist. The receiving-code-review skill from the superpowers plugin forces technical rigor: validate the suggestion before applying, push back when you disagree, demand evidence. It’s the filter that keeps the dev in charge, even when the dev is an AI.

    Where the pipeline breaks: the dev stopped testing locally

    This is the part nobody posts on LinkedIn.

    A team with the full stack (Claude Code + Superpowers + CodeRabbit) starts trusting the pipeline too much. The dev thinks if it passed CodeRabbit, it’s fine. The tech lead thinks if CodeRabbit approved, it was reviewed. The QA thinks if it reached staging, it was tested.

    Result: NOBODY runs anything locally before pushing. I’ve seen this happen in three different teams. Symptom always the same: PR merged to develop, deploy to staging, and then they discover the feature doesn’t work because nobody opened the browser to confirm the button actually clicks.

    AI reviews code. AI doesn’t test software.

    The fix I adopted as non-negotiable: mandatory workflow with an E2E validation command before the push. In my case it’s a /validar-e2e that spins up the project’s Docker stack, fires 3 agents in parallel (QA matrix, backend via curl/SQL, frontend via MCP Playwright in Claude Code) and only releases the push when every scenario passes. Re-runs everything after any fix, never partial validation.

    Here’s the skeleton to adapt to your project. Save as .claude/commands/validar-e2e.md:

    Orchestrated E2E validation before requesting human review.

Spin up the local stack, generate the scenario matrix, and ONLY THEN
fire backend + frontend in parallel with the matrix as input. DO NOT
stop on partial. After any fix, RE-RUN EVERYTHING, not just what changed.

QUALITY RULE: if an agent delivers a shallow result, with no concrete
evidence (no log/SQL/screenshot), with scenarios skipped without
justification, or clearly incomplete: RELAUNCH the agent with a more
explicit briefing about what was missing. Accepting bad output
contaminates the merge decision.

## Phase 1: Bring up / validate the stack

- `docker compose -f docker-compose.e2e.yml up -d`
- Wait for health checks to return 200 (5min timeout)
- If any service failed, report the container log and stop

## Phase 2: Agent A: QA matrix (BLOCKING, runs alone)

Launch ONE agent and WAIT for the full output before moving on.
Agents B and C depend on this matrix: without it, they test blind.

Agent A briefing:
  Produce a matrix with ≥20 scenarios based on the commits on this
  branch vs develop. Categories: happy path, regression, edge cases
  (null/empty/limits), error (DB unavailable, auth failure), migration
  (idempotence). For each: ID, description, severity (P0/P1/P2),
  steps, expected result. Save to
  `docs/specs/<feature>-qa-matrix.md`. Report count per category +
  the 3 priority P0 scenarios + the critical UI flows to be covered
  by frontend.

## Phase 3: Agents B and C in parallel (fed by the matrix)

RULE: ONE message with 2 simultaneous Agent tool calls. Paste the 3
P0 scenarios (Agent A output) into B's briefing and the critical UI
flows into C's briefing. Limit ≤80 tool calls per agent (above that
you get socket errors: relaunch with smaller scope if it crashes).

### Agent B: Backend E2E
Run the P0 scenarios below via curl against the local stack. Validate
the DB after each call (psql/mongosh/redis-cli depending on stack).
Also run unit tests for the changed branches. Report PASS/FAIL with
evidence (1-2 lines of log/SQL) in ≤600 words. Don't rebuild Docker,
don't touch production code.

  QA P0 scenarios: <paste Agent A output here>

### Agent C: Frontend MCP Playwright
Run the critical UI flows below in the browser via MCP Playwright.
For each: screenshot of the state, console inspection (JS errors),
network request validation. Report regressions in ≤700 words with
screenshots.

  QA critical flows: <paste Agent A output here>

## Phase 4: Consolidate

- B and C both PASS with evidence → release `git push` and open the PR
- Any FAIL → fix the code and GO BACK to Phase 3 (re-run B and C with
  the same matrix; only re-run Agent A if the fix changed scenarios)
- BLOCKED → diagnose infra/context before trying again
- Socket error on an agent → relaunch with reduced scope (≤50 tool calls)
- Shallow result / no evidence → RELAUNCH the agent with a reinforced
  briefing demanding exactly what was missing (logs, SQL queries,
  screenshots, specific assertions). Don't accept PASS without proof.

    And it’s not just bureaucratic friction: it’s how you keep the reflex. Whoever tests locally catches the bug in 30 seconds. Whoever waits for staging catches it in 30 minutes. Whoever waits for production pays much more.

    The pipeline is yours. The AI is just the engine.

    CodeRabbit + Claude Code + Superpowers is a stack. A good stack. Removes a real bottleneck. Gives tech lead time back for architecture, zeroes out the review backlog, and PRs come out rounder because the rule set is single.

    But it’s a stack. Not a process.

    Process is the discipline of well-scoped US, well-written AC, mandatory local testing, and the humility to accept that AI speeds up what’s right and speeds up what’s wrong right alongside it.

    Anyone who confuses stack with process is going to find out the hard way. Probably on a Friday deploy.