A/B Test Sample Size Calculator MCP
MCP server that computes A/B test sample sizes, durations and significance tests for AI
Answered our latest tool-list check 2 hours ago with 5 tools. It will be marked verified after 3 successful checks in 7 days.
- Pricing
- Open source
- Platforms
- CLI, API, self-hosted
- Runs
- Remotely
- Version
- 1.0.0
- Listing updated
- today
Similar MCP servers
About A/B Test Sample Size Calculator MCP
A/B Test Sample Size Calculator MCP is an open-source MCP server for analytics and data. It runs as a hosted remote server, so there is nothing to install locally.
The server gives an AI assistant exact statistical routines for planning and reading conversion experiments. Given a baseline rate, a relative or absolute minimum detectable effect, alpha, power and sidedness, it returns users per variant, and it can convert that into days and weeks from daily traffic and the number of variants.
Beyond planning, it runs post-test checks: a two-proportion z-test with confidence interval and relative lift, chi-square with and without Yates correction, Fisher exact tests, and a sample ratio mismatch check. A separate tool covers the Fisher-Freeman-Halton exact test for r x c tables and reports the Pearson chi-square alongside it. A peeking-risk tool estimates the false positive rate when a test is checked several times and returns Pocock and O'Brien-Fleming boundaries for up to 10 looks.
The README says it needs no API key and makes no network calls when run locally over stdio, and a hosted Streamable HTTP endpoint is also offered. It is aimed at analysts, growth teams and developers who want an assistant to produce auditable numbers rather than estimate them. Its stated limits include conversion-rate metrics only, fixed-horizon assumptions, and a work limit on very large exact tables.
Install
Run in a terminal
claude mcp add --transport http ab-test-sample-size-calculator-mcp https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcpAdd to Settings → Connectors → Add custom connector
https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcpAdd to ~/.cursor/mcp.json
{
"mcpServers": {
"ab-test-sample-size-calculator-mcp": {
"url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp"
}
}
}Add to .vscode/mcp.json
{
"servers": {
"ab-test-sample-size-calculator-mcp": {
"type": "http",
"url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp"
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"ab-test-sample-size-calculator-mcp": {
"serverUrl": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp"
}
}
}Add to Grok Bot desktop app → Connect apps → Add a custom connector
https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcpEnter a Name (for example "A/B Test Sample Size Calculator MCP"), choose Type: Remote HTTPS, paste this as the Server URL and choose Add plugin.
Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"ab-test-sample-size-calculator-mcp": {
"type": "http",
"url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp"
}
}
}Using several servers? Build one config file with the MCP config generator.
Tools this server provides (5)
Read directly from the server.
All 5 tools say they're read-only.
From the server's own tool annotations; not verified by us.
- calculate_sample_sizeSays read-only
- Exact sample size per variant for an A/B test on a conversion rate (two independent proportions): baseline rate, minimum detectable effect (relative or absolute), alpha, power, one or two-sided, several variants with optional Bonferroni, and unequal allocation. Uses the pooled/unpooled normal formula (same as R power.prop.test) and also reports the figure Evan Miller's calculator gives, so the two
- estimate_test_durationSays read-only
- How many days an A/B test must run: from a known sample size per variant, or from baseline rate and MDE (computed as in calculate_sample_size), plus daily visitors and the share of traffic in the test. Returns exact days and the recommended whole weeks (to cover weekday effects).
- test_ab_significanceSays read-only
- Is an A/B test result statistically significant? From visitors and conversions per group: two-proportion z-test (pooled), one or two-sided p-value, Wald confidence interval for the difference, relative lift, Pearson chi-square with and without Yates correction (R prop.test), Fisher exact p for the same 2x2 table, and a sample ratio mismatch (SRM) check against the planned split.
- fisher_exact_testSays read-only
- Exact p-value for a contingency table of counts. 2x2: Fisher exact test (two-sided, greater or less, as R fisher.test). Larger r x c tables (for example 3 variants x converted/not, or 4 x 4 survey tables): Fisher-Freeman-Halton exact test by a network algorithm. Also returns Pearson chi-square and how many cells have expected counts below 5, so you can see when the chi-square approximation is unsa
- check_peeking_riskSays read-only
- What peeking does to an A/B test: the real false positive rate if you check a fixed-horizon test K times and stop at the first p < alpha (for example about 14% at 5 looks with alpha 0.05), and the Pocock and O'Brien-Fleming group sequential boundaries (z and nominal p per look) that keep the overall rate at alpha, for up to 10 equally spaced looks.
What it does
- Sample size planning
- Calculates users per variant from baseline rate, MDE, alpha, power and sidedness, including multi-variant Bonferroni and unequal splits.
- Test duration estimates
- Turns the required sample into days and whole weeks using daily visitors and traffic share.
- Significance testing
- Runs a two-proportion z-test, chi-square, Fisher exact and a sample ratio mismatch check on finished experiments.
- Exact r x c tests
- Applies the Fisher-Freeman-Halton exact test to larger contingency tables alongside Pearson chi-square.
- Peeking risk analysis
- Estimates inflated false positive rates for repeated looks and provides sequential boundaries for up to 10 looks.
- Structured results
- Each result includes JSON output, a short summary, method references and a link to the matching calculator.
Good for
- Working out how many visitors a checkout or signup experiment needs before launch
- Estimating how many weeks a multi-variant test will take at current traffic
- Checking whether a finished A/B result is statistically significant
- Running an exact test on a small survey or contingency table
- Quantifying the false positive cost of checking a dashboard daily
Pricing
Open source
MIT-licensed and free; the hosted endpoint is listed as free with no sign-up.
Prices change. Check abwex.com before you buy.
A/B Test Sample Size Calculator MCP FAQ
Does the server need an API key?
The README says no API key is needed and the server makes no network calls when run locally over stdio. The hosted endpoint is also described as requiring no token or sign-up.
Which metrics are supported for sample size?
Sample size covers conversion rates, meaning two proportions. The README says continuous metrics such as revenue per user need a standard deviation and are not covered.
What happens if an exact test is too large to compute?
The README says the Fisher-Freeman-Halton test can exceed the work limit, more so on the hosted server, and that the tool then returns the chi-square result with a note instead of an exact p-value.
Can it handle repeated checks of a running test?
Yes, the check_peeking_risk tool gives the inflated false positive rate for K looks and Pocock and O'Brien-Fleming boundaries for up to 10 looks, assuming equally spaced looks and two-sided tests.
Is A/B Test Sample Size Calculator MCP free?
Yes. A/B Test Sample Size Calculator MCP is open source, so you can use and self-host it for free.
How do I add A/B Test Sample Size Calculator MCP to Claude Code?
Run this command in your terminal: claude mcp add --transport http ab-test-sample-size-calculator-mcp https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp
What are the best alternatives to A/B Test Sample Size Calculator MCP?
The closest alternatives to A/B Test Sample Size Calculator MCP are effectsize, Hive A/B Test, Stats Explorer, Data Compute Stats Mcp and Aetherya.
Where this comes from
- Description: written from the project's README, the project's website and its MCP Registry entry.
- Status and tools: checked automatically by AIPoweredBot on 19 November 2026.
- Install steps: from the official MCP Registry entry.