Catch the flaky test before it's flaky.
Paste your Playwright or Cypress test. Get the known race-condition patterns flagged before this test ever runs in CI, not after it's already failed three times in a row.
How FlakeCheck works
- 1
Pick your framework
Toggle between Playwright and Cypress. The two frameworks have different flake patterns and different fixes (networkidle and strict mode are Playwright-specific; cy.intercept ordering is Cypress-specific), so the scanner switches its rule set to match.
- 2
Paste your test source
Drop in a single test file or a block of test code. Nothing is uploaded, the parse happens in your browser. FlakeCheck strips comments so a commented-out cy.wait won't be flagged as a real one.
- 3
The scanner walks the source line by line
It matches each line against the known anti-pattern rules and, for the ordering-sensitive patterns, tracks where navigation happens inside each test block so it can tell a route registered before goto from one registered after.
- 4
Findings are grouped by severity
Each finding shows the line number, which named pattern it matched, a plain-English explanation of the race condition, and the minimal code fix. High-severity findings (the ones that cause outright flakiness) sort to the top.
- 5
Cross-check against the cheat sheet
The panel beside the results lists every pattern the scanner checks for the selected framework, so you can eyeball anything the static scan can't prove and learn the taxonomy as you go.
What each finding means
FlakeCheck reports against a named taxonomy of flaky-test patterns. Each one is a documented race condition, not a style nitpick. Here is what triggers each, ordered from the ones most likely to flake to the ones that only bite under specific conditions.
waitForLoadState('networkidle') or a { waitUntil: 'networkidle' } option. It waits for 500ms of network silence, which never arrives on pages with websockets, polling, or analytics, so it times out or resolves at a random moment.
await page.goto('/chat');
await page.waitForLoadState('networkidle'); // websocket never idlespage.route() or cy.intercept() registered after the goto/visit that fires the request it is meant to catch. On a fast network the request completes before the handler attaches and the interception silently misses.
await page.goto('/orders');
await page.route('**/api/orders', ...); // too lateawait page.waitForResponse() on its own line, after the action that triggers the response. A fast response arrives before the wait is set up, so the wait hangs until timeout. The subscription has to start before the action.
await page.click('#save');
await page.waitForResponse('**/api/save'); // may already be donepage.waitForTimeout(ms) or cy.wait(number). A fixed sleep bets the app is always ready by exactly that time: too short on a slow CI runner (flake), wasted seconds on a fast one. Wait for a condition instead.
await page.waitForTimeout(2000);
cy.wait(2000);A locator matching a bare tag or a single class (button, .item), or getByRole without a name. The moment a second match renders, Playwright throws a strict-mode violation and Cypress clicks the wrong element.
page.locator('button'); // strict-mode risk
cy.get('.item'); // acts on first match
page.getByRole('button'); // no { name }A shared storageState or cy.session without validation. State from one test leaks into the next, so the suite passes or fails depending on run order and how the tests are sharded across CI workers.
test.use({ storageState: 'auth.json' });
cy.session('user'); // no validate().first(), .nth(n), or .eq(n) selects an element by position. Any change in render order, or an injected banner or async list item, silently shifts which element you act on.
page.locator('.row').first();
cy.get('.row').eq(0);if (await el.isVisible()) branching, or conditional should('be.visible') logic. It reads the DOM at one instant; if the element is mid-animation the branch is taken inconsistently between runs.
if (await page.locator('.toast').isVisible()) { ... }The flaky pattern, and the fix, side by side
Every finding maps to a mechanical rewrite. The fix syntax differs between the two frameworks, so here is the before/after for the most common patterns in each.
Playwright
// networkidle → wait for a real element
- await page.waitForLoadState('networkidle');
+ await expect(page.getByRole('heading', { name: 'Home' })).toBeVisible();
// late route → register before navigation
- await page.goto('/x'); await page.route('**/api', h);
+ await page.route('**/api', h); await page.goto('/x');
// wait after action → Promise.all
- await page.click('#save'); await page.waitForResponse('**/api/save');
+ const [res] = await Promise.all([
+ page.waitForResponse('**/api/save'),
+ page.click('#save'),
+ ]);
// hard sleep → assertion
- await page.waitForTimeout(2000);
+ await expect(locator).toBeVisible();
// broad locator → named role or testid
- page.locator('button')
+ page.getByRole('button', { name: 'Save' })Cypress
// late intercept → intercept, then visit, then wait alias
- cy.visit('/orders'); cy.intercept('GET', '/api/orders').as('o');
+ cy.intercept('GET', '/api/orders').as('o'); cy.visit('/orders'); cy.wait('@o');
// hard sleep → wait on the aliased request
- cy.wait(2000);
+ cy.wait('@o');
// broad locator → data-cy attribute
- cy.get('button')
+ cy.get('[data-cy=save-button]')
// unvalidated session → add validate()
- cy.session('user', setup);
+ cy.session('user', setup, { validate() { cy.getCookie('sid').should('exist'); } });When to use FlakeCheck
| Situation |
|---|
| Self-check before opening a PR that adds E2E tests |
| Code review of a teammate's test file |
| First-pass audit of a suite that's already flaky |
| Onboarding onto an inherited test suite |
| Deciding whether a re-run is a real failure or a flake |
| Teaching juniors the flaky-test taxonomy |
Frequently Asked Questions
What does FlakeCheck do?
FlakeCheck is a static scanner for Playwright and Cypress test files. You paste your test source, pick the framework, and it flags the well-documented patterns that cause flaky tests: networkidle waits, route handlers registered after navigation, hard-coded sleeps, broad locators that trip Playwright's strict mode, and shared state that leaks between tests.
It does this without running your test. Every finding names the pattern it matched, explains the race condition in plain English, and shows the minimal fix. The goal is to catch these at write time, before the test has ever burned a red CI run.
Does this replace CI-based flake detection like Mergify or BuildPulse?
No, and it is not meant to. They solve different halves of the problem at different moments.
| FlakeCheck (this tool) | CI history trackers | |
|---|---|---|
| When it runs | At write time, before the first run | After dozens of CI runs |
| Needs CI history | No | Yes (often 10+ runs) |
| How it finds flakes | Static pattern match on the source | Pass/fail statistics over time |
| Catches brand-new tests | Yes | No, needs run history first |
| Cost | Free, no account | Usually a paid CI plan |
FlakeCheck is the pre-commit sanity check. History-based tools are the ongoing production monitor. A healthy workflow uses both: FlakeCheck before the PR, a history tracker once the test is living in CI.
Does FlakeCheck support Selenium, WebdriverIO, or Puppeteer?
Not yet. FlakeCheck currently understands Playwright and Cypress syntax specifically, because the anti-patterns and their fixes differ meaningfully between frameworks (there is no networkidle load state or strict-mode violation in Selenium, for example).
That said, several of the underlying causes are universal: hard-coded sleeps, positional element selection, and shared session state flake in any framework. If you write Selenium or WebdriverIO, the taxonomy on this page is still worth reading even though the scanner will not parse the exact calls.
Is my test code sent to a server?
No. FlakeCheck is fully client-side. The scan runs in JavaScript inside your browser, the test source never leaves your machine, and there is no backend, no upload, and no logging of what you paste. Once the page has loaded you can even run it offline.
How do I fix a networkidle wait that FlakeCheck flagged?
waitForLoadState('networkidle') waits for 500ms of zero network activity, which never reliably happens on a page with a websocket, polling, or an analytics beacon. Replace it with a wait for the specific element or state you actually care about:
// Flaky: resolves at an unpredictable moment
await page.goto('/dashboard');
await page.waitForLoadState('networkidle');
// Deterministic: wait for the thing you're about to use
await page.goto('/dashboard');
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();Why is registering a route after navigation flaky if my test still passes?
It passes when the network is slow enough that the request is still in flight by the time the handler attaches. On a fast CI runner, or with a warm cache, the request completes before the handler is registered, so the interception silently misses and your assertion about the stubbed response fails.
This is the defining trait of a flaky test: nothing about the code is wrong on the run where it passes, so people re-run the job until it goes green instead of fixing the ordering. Register the route (Playwright) or intercept (Cypress) before the navigation that triggers the request.
// Flaky: handler races the request the navigation fires
await page.goto('/orders');
await page.route('**/api/orders', route => route.fulfill({ body: '[]' }));
// Correct: handler is in place before the request exists
await page.route('**/api/orders', route => route.fulfill({ body: '[]' }));
await page.goto('/orders');