One load took three seconds. Is the site faster?
One three-second disaster wrecks a t-test built on means. Mann-Whitney compares ranks: the outlier counts as "the slowest load" and nothing more, so the verdict is about your typical visitor.
SaaS Starter Statistics free
After you install, this is the model to open.
Faster Site, Proven Without the Bell Curve
- In your spreadsheet, click the Sortia icon in the strip of icons down the right-hand edge. No strip? Click the arrow at the bottom-right to open it. You can also use Extensions, then Sortia, then Open Sortia.
- Click Start from a template and put that name in the search box.
- Pick the card with that name and click Load this template. It arrives on a new tab with real numbers already in it.
The answer
Ranks see through both outliers:
- U statistic
- 8
- p (two-tail)
- 0.0104
- Typical speedup
- ~2x at the median
The typical visitor loads in 217.5 ms against 450 before, roughly twice as fast, and the p-value of 0.0104, counted exactly rather than approximated, says eight-vs-eight samples this separated are unlikely to be luck. For skewed data, report exactly that median pair, never the means the outliers have already bent. Response times, revenue per user, ticket sizes: same tool.
The model
Eight page loads on the old CDN, eight on the new, each column deliberately carrying one multi-second disaster (2,900 ms and 3,100 ms), because real latency data always has one.
| Old CDN | median 450 ms, one 2,900 ms outlier |
| New CDN | median 217.5 ms, one 3,100 ms outlier |
| U statistic | calculated |
| p-value | calculated |
Once it is in your sheet
- The model arrives with real numbers in it and runs as it stands, so you can press the button first and understand it second.
- Change the numbers to yours. The sheet marks which cells are inputs and which hold formulas, and most labels carry a note explaining the row.
- Press the run button at the bottom of the panel. It is labeled for the tool you are in, and the result lands on its own tab, with a written reading of it beside the figures.
Never used Google Sheets? Start here goes the whole way, in seven steps, and assumes nothing.
Next question
- Every day improved. Could that just be luck?Did the New Process Cut Ticket Times?
- Your churn is a band. What does that do to LTV?What Is a Customer Really Worth?
- The lift is real. Is it real for every segment?Did the New Onboarding Move Activation?
- What does hiring now cost you next January?What Three Hiring Plans Cost
- Your traffic is fixed. What lift can it even see?What Lift Could We Even Detect?
- Do the service credits or the churn cost you more?What Would an Outage Cost Us?
Every model like this one, and the method behind them: Statistics in Google Sheets.