SEO significance calculator
You changed a page and its clicks moved. Did the change do it, or did every page move? Type the changed page’s clicks before and after, and the same two figures for three to eight pages you left alone. You get the lift versus what those pages predict, with its 95% interval and p-value, or a plain statement that the numbers cannot tell.
Nothing you type leaves your browser. There is no upload and no account; this page's security policy blocks fetch, XHR, WebSocket, EventSource and beacon calls and form posts. The estimator is the one Penlid’s paid product runs, not a copy of it.
The same estimator the paid ledger uses
One copy of the estimator
The estimate is estimateLift from stats.js, the module Penlid’s server imports, and the sentence is describeEffect. There is no second implementation on this page, and a test runs the did-it-work tool and this calculator on the same numbers and requires the same stored result from both.
Choose the comparison pages first
Write down which pages you will compare against before you look at any of their clicks after the change. If you pick the pages that make the answer look the way you hoped, the interval and p-value are for a question nobody asked.
“Cannot tell” is an answer
Too few usable comparison pages, no baseline traffic, or a gap too small for the numbers to separate from chance each come back as a sentence saying so, with the smallest lift the numbers could have detected. A small number invented to fill the space is what this product exists to avoid.
How to fill it in
The windows are 28 days. Call the day the change went live day 0. Before is days −28 to −1. After is days 1 to 28, so the day of the change itself is left out. Earlier window is days −56 to −29 and is optional. Use the same calendar days for the changed page and for every comparison page: that is the whole idea, because anything that moved every page (a season, an update, a holiday) moves them in the same days. Search Console dates are Pacific Time, and Google says the newest data can be preliminary, so use windows that have fully ended.
Where the clicks come from. In Search Console’s Performance report (Search results), set a custom date range for one window and read each page’s clicks from the Pages tab, or filter to the page and read the Clicks total. Take every figure the same way, for every page and both windows. Type whole numbers: Search Console reports whole clicks, and a figure with a decimal point is refused rather than rounded.
Which comparison pages. Pages you did not change in either window, that earn clicks from the same kind of query and are not tiny. Three is the least the estimator accepts and eight the most it uses. Choose them and write them down before you read their numbers. If you have more than eight candidates, decide which eight by something other than their results, such as the closest in size.
One rule this page applies. A comparison page needs at least 10 clicks across the two windows together. A page with a handful of clicks has a before-to-after ratio that chance alone moves a long way, and the did-it-work tool leaves such pages out of a Compare export for the same reason. A page left out is named in the result.
Have a Search Console file instead of figures? Did it work? reads it and chooses comparison pages from your own data. Deciding whether a test is worth running before you change anything? Use the SEO test calculator.
What the answer cannot tell you
- It is timing evidence, not proof. The comparison pages remove what hit every page. They do not remove what hit only the page you changed: a link, a ranking move, a news event, a change to the page you forgot.
- It depends on the comparison pages. A different set of untouched pages can give a different answer. The estimator allows for the drift pages show on their own, so the interval widens when your comparison pages disagree with each other.
- A small count gets a direction, not a size. Under 10 clicks before the change the answer is improved, declined or too close to call, with no percentage: going from 3 clicks to 58 is clearly up, and any percentage attached to it is mostly the luck of the 3. With no clicks before, there is nothing to measure a lift from.
- The calculator trusts what you type. It has no file to check, so it cannot see a window that is the wrong length, a figure taken a different way for one page, or a comparison page that was changed.
- Nothing is remembered. The numbers are not stored or put in the address, so reload clears the form.
What happens to what you type
Nothing leaves your browser. The numbers are read by JavaScript running in this tab and used in memory; they are not uploaded, there is no copy on a server, and nothing from them is written to a cookie, to local storage or to the address bar (the theme switch remembers light or dark and nothing else). The page loads no analytics and no third-party scripts.
You do not have to take our word for it. Open your browser's developer tools, choose the Network tab, run the estimator, and
watch it make no request. Two things back that up. This page carries a content security policy, readable in its source,
that blocks fetch, XHR, WebSocket, EventSource and beacon calls (connect-src 'none') and form posts
(form-action 'none'), and allows images only as inline data: URIs. A content security policy
does not cover every browser feature (WebRTC, for one), so the second check matters: the project's tests read this page's
own script and every module it imports, and fail on any of the browser's network APIs and on storage.
The site-wide script penlid.js, which every page loads, contains network calls for the signed-in app's
account switcher. This page never calls it, and a test checks that those calls sit only inside the switcher.
Methods: the rules and thresholds
Each number below is a constant in public/assets/lib/stats.js, diagnose.js or page-tools.js, which this page reads and does not copy. The project's tests compare the numbers quoted here with those constants; that is a check on the numbers, and not on every sentence. They are judgement calls, and they are the owner's to change.
- The estimator
- Version lr-re-2. For each page the log of its after clicks over its before clicks, each with half a click added, with the counting variance that implies; the changed page’s log ratio is compared with the comparison pages’ through a random-effects model with a Paule-Mandel between-page variance, and the p-value and the 95% interval come from that model’s reference distribution, tested at 0.05. The lift is the changed page’s clicks versus what the untouched pages predict. This page calls
estimateLift,effectRowanddescribeEffectand does no arithmetic of its own on the estimate. - Windows
- 28 days before, 28 days after (from the day after the change), and, optionally, the 28 days before the before window.
- Comparison pages
- At least 3 and at most 8 are accepted; with more than 8 the page refuses, because choosing the best 8 after seeing the result would choose the answer. A page with fewer than 10 clicks across its two windows is left out and named, which is the rule the did-it-work tool applies to a Compare export; if that leaves fewer than 3, the estimator’s own “not measurable” sentence is shown.
- Size or direction
- With 10 or more clicks before the change the answer has a size and an interval; with 1 to 9 it has a direction only (improved, declined or too close to call); with none there is no baseline and no measurement.
- The earlier window
- If it is typed for the changed page and every comparison page used, the estimator also runs the same comparison one window earlier, as a check on whether the page was already moving differently before the change. It is a flag, not a gate. With any earlier window missing, the check does not run and the result says so.
- What is typed
- Whole numbers only, with a comma, space or apostrophe allowed between groups of three digits. Anything else is refused with the reason and nothing is rounded. A count above 10,000,000,000 is refused as a probable typing error, because a count near the largest whole number a computer holds gives percentages with no meaning.
- Two sentences that are this page’s own
- The estimator words two refusals for a Search Console file, and both would be false of numbers you typed: “New page” (the page may be years old; it had no clicks in the window typed) and “other pages have traffic on this site”. For those two, the result says what was typed instead, as the did-it-work tool does for a Compare export. The stored estimate is unchanged, and every other sentence is the estimator’s own.