A/B tests with consent: stable assignment, conversion goals and results with significance, no extra cookies
Install
genpm add @core/experimentsWhat you get
- Source in src/lib/experiments/, 8 files. (21.4 kB)
- AI rules in src/lib/experiments/AGENTS.md, plus IDE rule files.
- Env vars added to .env.example: EXPERIMENTS_SECRET.
- Resolves @core/consent, @core/contracts, @core/db, @core/jobs for you.
README
This package has no README.
This is exactly what your AI reads when it works in src/lib/experiments. Nothing else is added to its context.
@core/experiments — rules for AI agents
Purpose
A/B tests that respect consent. Visitors who accepted analytics cookies get a stable variant (deterministic hash of
the pseudonymous consent id, no extra cookie); everyone else always sees the control and nothing is stored.
Conversions are recorded once per visitor and goal; results show rates, lift, p-value and a 95 % confidence interval
(two-proportion z-test) and are frozen when the experiment stops. Tables: experiments, experiment_participants.
Map
index.ts—getVariant,recordConversion,upsertExperiment,setExperimentStatus,experimentResults,experimentsAdminResource,pruneExperimentsJob,compare.react.ts—<Experiment name cookieHeader variants={{ a, b }} />(server component).stats.ts— statistics without dependencies.schema.ts— tables.
Integration
- Requires @core/consent (the banner) and its cookie. Env:
EXPERIMENTS_SECRET(≥ 32 random chars). - Migrations as in
src/lib/db/AGENTS.md. AddexperimentsAdminResourcetosrc/genpm/admin.ts(with kit-landing: its "A/B tests (optional)" lines). - Create the experiment in the admin (key, goal, variants: the first is the control), then press "Start".
- Render:
const { variant } = await getVariant('hero-test', cookieHeader)(cookie header of the request) and show the matching content; or<Experiment name="hero-test" cookieHeader={…} variants={{ a: <A />, b: <B /> }} />. - Conversion: where the goal happens on the server,
await recordConversion('lead', req.headers.get('cookie')). - Schedule
experiments.prunedaily (@core/jobsschedule) to delete participants 30 days after an experiment stops (with kit-cms, itssetup-schedules.tsdoes it). - Verify: with analytics accepted the same browser always sees the same variant; with cookies rejected, always the control.
Conventions
- Decide only when
significantis true and the test ran at least one or two full weeks (weekday effects). - Change one thing per experiment; don't edit variants or weights while it runs (blocked; stop and create a new one).
Don't
- Don't assign or record visitors without analytics consent, and don't store emails or ids in experiments.
- Don't stop a test early because one variant "looks better" (peeking inflates false positives).
- Don't use experiments to show different prices to different people.
# @core/experiments — rules for AI agents
## Purpose
A/B tests that respect consent. Visitors who accepted analytics cookies get a stable variant (deterministic hash of
the pseudonymous consent id, no extra cookie); everyone else always sees the control and nothing is stored.
Conversions are recorded once per visitor and goal; results show rates, lift, p-value and a 95 % confidence interval
(two-proportion z-test) and are frozen when the experiment stops. Tables: `experiments`, `experiment_participants`.
## Map
- `index.ts` — `getVariant`, `recordConversion`, `upsertExperiment`, `setExperimentStatus`, `experimentResults`, `experimentsAdminResource`, `pruneExperimentsJob`, `compare`.
- `react.ts` — `<Experiment name cookieHeader variants={{ a, b }} />` (server component).
- `stats.ts` — statistics without dependencies. `schema.ts` — tables.
## Integration
1. Requires @core/consent (the banner) and its cookie. Env: `EXPERIMENTS_SECRET` (≥ 32 random chars).
2. Migrations as in `src/lib/db/AGENTS.md`. Add `experimentsAdminResource` to `src/genpm/admin.ts` (with kit-landing:
its "A/B tests (optional)" lines).
3. Create the experiment in the admin (key, goal, variants: the first is the control), then press "Start".
4. Render: `const { variant } = await getVariant('hero-test', cookieHeader)` (cookie header of the request) and show
the matching content; or `<Experiment name="hero-test" cookieHeader={…} variants={{ a: <A />, b: <B /> }} />`.
5. Conversion: where the goal happens on the server, `await recordConversion('lead', req.headers.get('cookie'))`.
6. Schedule `experiments.prune` daily (@core/jobs `schedule`) to delete participants 30 days after an experiment stops
(with kit-cms, its `setup-schedules.ts` does it).
7. Verify: with analytics accepted the same browser always sees the same variant; with cookies rejected, always the control.
## Conventions
- Decide only when `significant` is true and the test ran at least one or two full weeks (weekday effects).
- Change one thing per experiment; don't edit variants or weights while it runs (blocked; stop and create a new one).
## Don't
- Don't assign or record visitors without analytics consent, and don't store emails or ids in experiments.
- Don't stop a test early because one variant "looks better" (peeking inflates false positives).
- Don't use experiments to show different prices to different people.
The exact tree that will be injected, after .genpmignore. Pinned to
// Recurso "Experimentos" para @core/admin: crear, empezar, parar y ver resultados con significación.
import { count, desc, eq } from 'drizzle-orm';
import type { AdminContext, AdminResource } from '../contracts/index.ts';
import { getDb } from '../db/index.ts';
import { ExperimentError, ExperimentInput, experimentResults, setExperimentStatus, upsertExperiment, type VariantResult } from './experiments.ts';
import { type Experiment, experiments } from './schema.ts';
export type ExperimentAdminRow = Experiment & { summary: string; current: VariantResult[] };
async function need(ctx: AdminContext, p: string) {
if (!(await ctx.can(p))) throw new ExperimentError('forbidden', `forbidden: ${p}`);
}
const pct = (x: number) => `${(x * 100).toFixed(1)}%`;
/** Resumen legible: tasa por variante y, frente al control, mejora, p-valor y si es significativo. */
export function summarize(results: VariantResult[]): string {
return results
.map((r) =>
r.vsControl
? `${r.variant}: ${pct(r.rate)} (${r.conversions}/${r.participants}) · ${r.vsControl.lift === null ? '—' : `${r.vsControl.lift >= 0 ? '+' : ''}${pct(r.vsControl.lift)}`} · p=${r.vsControl.pValue.toFixed(3)}${r.vsControl.significant ? ' · significant' : ' · not significant yet'}`
: `${r.variant} (control): ${pct(r.rate)} (${r.conversions}/${r.participants})`,
)
.join('\n');
}
async function row(e: Experiment): Promise<ExperimentAdminRow> {
const current = e.status === 'draft' ? [] : await experimentResults(e.id);
return { ...e, current, summary: current.length ? summarize(current) : 'Not started' };
}
export const experimentsAdminResource: AdminResource<ExperimentAdminRow> = {
name: 'experiments',
label: { singular: 'Experiment', plural: 'Experiments' },
group: 'Marketing',
fields: [
{ name: 'name', label: 'Name', type: 'text', required: true, list: true },
{ name: 'key', label: 'Key (used in code)', type: 'slug', required: true, list: true },
{ name: 'goal', label: 'Goal (e.g. lead, purchase)', type: 'text', required: true, list: true },
{
name: 'variants',
label: 'Variants (the first one is the control)',
type: 'list',
required: true,
fields: [
{ name: 'key', label: 'Key (a, b…)', type: 'text', required: true },
{ name: 'weight', label: 'Traffic weight', type: 'number' },
],
},
{ name: 'status', label: 'Status', type: 'text', readOnly: true, list: true },
{ name: 'summary', label: 'Results', type: 'textarea', readOnly: true, help: 'Decide only when the result is significant and the test ran full weeks.' },
],
input: ExperimentInput,
title: (e) => e.name,
async list(q, ctx) {
await need(ctx, 'experiments:read');
const [total] = await getDb().select({ n: count() }).from(experiments);
const rows = await getDb().select().from(experiments).orderBy(desc(experiments.createdAt)).limit(q.pageSize).offset((Math.max(q.page, 1) - 1) * q.pageSize);
return { rows: await Promise.all(rows.map(row)), total: total?.n ?? 0 };
},
async get(id, ctx) {
await need(ctx, 'experiments:read');
const [e] = await getDb().select().from(experiments).where(eq(experiments.id, id));
return e ? row(e) : null;
},
async create(input, ctx) {
await need(ctx, 'experiments:create');
return row(await upsertExperiment(input as never));
},
async update(id, input, ctx) {
await need(ctx, 'experiments:update');
return row(await upsertExperiment(input as never, id));
},
actions: [
{
name: 'start',
label: 'Start',
permission: 'experiments:update',
confirm: true,
available: (e) => e.status === 'draft',
async run(id, _i, ctx) {
await need(ctx, 'experiments:update');
return row(await setExperimentStatus(id, 'running'));
},
},
{
name: 'stop',
label: 'Stop and keep results',
permission: 'experiments:update',
confirm: true,
available: (e) => e.status === 'running',
async run(id, _i, ctx) {
await need(ctx, 'experiments:update');
return row(await setExperimentStatus(id, 'stopped'));
},
},
],
};
This package declares no MCP servers.
| Version | Commit | Published | Scan |
|---|---|---|---|
| 1.0.1 | c83c2b1 | 2 hours ago | scan passed |
- proposed
- GenPM proposes the npm command and runs it only if you say yes.
- Used by (0)
- No public package depends on it yet.
- scan
- scan passed · 0 findings
- commit
- v1.0.1 → c83c2b1012acc5483bb03c0706dba0781378102b · verified after fetch
- scripts
- None. GenPM never runs package code.
- license
- MIT
- Quality
- 100/100
- Recognized licensepassed
- AGENTS.md explains its purposepassed
- AGENTS.md has integration stepspassed
- AGENTS.md lists conventions or don'tspassed
- Includes testspassed
- Security scan passedpassed
- Released in the last 6 monthspassed
- Verified publisherpassed
- Summary and keywordspassed
- report
- See something wrong?