Testes A/B com consentimento: atribuição estável, metas de conversão e resultados com significância
Instalar
genpm add @core/experimentsO que você recebe
- Código em src/lib/experiments/, 8 arquivos. (21,4 kB)
- Regras de IA em src/lib/experiments/AGENTS.md, mais arquivos de regras da IDE.
- Variáveis de ambiente adicionadas ao .env.example: EXPERIMENTS_SECRET.
- Resolve @core/consent, @core/contracts, @core/db, @core/jobs para você.
README
Este pacote não tem README.
Isto é exatamente o que sua IA lê quando trabalha em src/lib/experiments. Nada mais é adicionado ao contexto dela.
@core/experiments — rules for AI agents
Purpose
A/B tests that respect consent. Visitors who accepted analytics cookies get a stable variant (deterministic hash of
the pseudonymous consent id, no extra cookie); everyone else always sees the control and nothing is stored.
Conversions are recorded once per visitor and goal; results show rates, lift, p-value and a 95 % confidence interval
(two-proportion z-test) and are frozen when the experiment stops. Tables: experiments, experiment_participants.
Map
index.ts—getVariant,recordConversion,upsertExperiment,setExperimentStatus,experimentResults,experimentsAdminResource,pruneExperimentsJob,compare.react.ts—<Experiment name cookieHeader variants={{ a, b }} />(server component).stats.ts— statistics without dependencies.schema.ts— tables.
Integration
- Requires @core/consent (the banner) and its cookie. Env:
EXPERIMENTS_SECRET(≥ 32 random chars). - Migrations as in
src/lib/db/AGENTS.md. AddexperimentsAdminResourcetosrc/genpm/admin.ts(with kit-landing: its "A/B tests (optional)" lines). - Create the experiment in the admin (key, goal, variants: the first is the control), then press "Start".
- Render:
const { variant } = await getVariant('hero-test', cookieHeader)(cookie header of the request) and show the matching content; or<Experiment name="hero-test" cookieHeader={…} variants={{ a: <A />, b: <B /> }} />. - Conversion: where the goal happens on the server,
await recordConversion('lead', req.headers.get('cookie')). - Schedule
experiments.prunedaily (@core/jobsschedule) to delete participants 30 days after an experiment stops (with kit-cms, itssetup-schedules.tsdoes it). - Verify: with analytics accepted the same browser always sees the same variant; with cookies rejected, always the control.
Conventions
- Decide only when
significantis true and the test ran at least one or two full weeks (weekday effects). - Change one thing per experiment; don't edit variants or weights while it runs (blocked; stop and create a new one).
Don't
- Don't assign or record visitors without analytics consent, and don't store emails or ids in experiments.
- Don't stop a test early because one variant "looks better" (peeking inflates false positives).
- Don't use experiments to show different prices to different people.
# @core/experiments — rules for AI agents
## Purpose
A/B tests that respect consent. Visitors who accepted analytics cookies get a stable variant (deterministic hash of
the pseudonymous consent id, no extra cookie); everyone else always sees the control and nothing is stored.
Conversions are recorded once per visitor and goal; results show rates, lift, p-value and a 95 % confidence interval
(two-proportion z-test) and are frozen when the experiment stops. Tables: `experiments`, `experiment_participants`.
## Map
- `index.ts` — `getVariant`, `recordConversion`, `upsertExperiment`, `setExperimentStatus`, `experimentResults`, `experimentsAdminResource`, `pruneExperimentsJob`, `compare`.
- `react.ts` — `<Experiment name cookieHeader variants={{ a, b }} />` (server component).
- `stats.ts` — statistics without dependencies. `schema.ts` — tables.
## Integration
1. Requires @core/consent (the banner) and its cookie. Env: `EXPERIMENTS_SECRET` (≥ 32 random chars).
2. Migrations as in `src/lib/db/AGENTS.md`. Add `experimentsAdminResource` to `src/genpm/admin.ts` (with kit-landing:
its "A/B tests (optional)" lines).
3. Create the experiment in the admin (key, goal, variants: the first is the control), then press "Start".
4. Render: `const { variant } = await getVariant('hero-test', cookieHeader)` (cookie header of the request) and show
the matching content; or `<Experiment name="hero-test" cookieHeader={…} variants={{ a: <A />, b: <B /> }} />`.
5. Conversion: where the goal happens on the server, `await recordConversion('lead', req.headers.get('cookie'))`.
6. Schedule `experiments.prune` daily (@core/jobs `schedule`) to delete participants 30 days after an experiment stops
(with kit-cms, its `setup-schedules.ts` does it).
7. Verify: with analytics accepted the same browser always sees the same variant; with cookies rejected, always the control.
## Conventions
- Decide only when `significant` is true and the test ran at least one or two full weeks (weekday effects).
- Change one thing per experiment; don't edit variants or weights while it runs (blocked; stop and create a new one).
## Don't
- Don't assign or record visitors without analytics consent, and don't store emails or ids in experiments.
- Don't stop a test early because one variant "looks better" (peeking inflates false positives).
- Don't use experiments to show different prices to different people.
A árvore exata que será injetada, após o .genpmignore. Fixada em
// Estadística de los resultados: test z de dos proporciones (bilateral) e intervalo de confianza de la diferencia.
// Sin dependencias; validado con ejemplos de referencia en los tests.
/** Función de distribución normal estándar (Abramowitz–Stegun 7.1.26, error < 1,5e-7). */
export function normalCdf(z: number): number {
const t = 1 / (1 + 0.3275911 * (Math.abs(z) / Math.SQRT2));
const erf = 1 - ((((1.061405429 * t - 1.453152027) * t + 1.421413741) * t - 0.284496736) * t + 0.254829592) * t * Math.exp(-(z * z) / 2);
return z >= 0 ? (1 + erf) / 2 : (1 - erf) / 2;
}
export type Comparison = {
/** Tasa de conversión de control y variante (0–1). */
controlRate: number;
variantRate: number;
/** Mejora relativa sobre el control (0,25 = +25 %); null si el control no tiene conversiones. */
lift: number | null;
z: number;
pValue: number;
/** Intervalo de confianza del 95 % de la diferencia absoluta de tasas. */
ci95: [number, number];
/** p < 0,05 y muestra mínima (≥ 100 participantes y ≥ 10 conversiones por grupo). */
significant: boolean;
};
export function compare(control: { n: number; conversions: number }, variant: { n: number; conversions: number }): Comparison {
const p1 = control.n ? control.conversions / control.n : 0;
const p2 = variant.n ? variant.conversions / variant.n : 0;
const pooled = control.n + variant.n ? (control.conversions + variant.conversions) / (control.n + variant.n) : 0;
const se = control.n && variant.n ? Math.sqrt(pooled * (1 - pooled) * (1 / control.n + 1 / variant.n)) : 0;
const z = se ? (p2 - p1) / se : 0;
const pValue = se ? 2 * (1 - normalCdf(Math.abs(z))) : 1;
const seDiff = control.n && variant.n ? Math.sqrt((p1 * (1 - p1)) / control.n + (p2 * (1 - p2)) / variant.n) : 0;
const enough = Math.min(control.n, variant.n) >= 100 && Math.min(control.conversions, variant.conversions) >= 10;
return {
controlRate: p1,
variantRate: p2,
lift: p1 ? (p2 - p1) / p1 : null,
z,
pValue,
ci95: [p2 - p1 - 1.959964 * seDiff, p2 - p1 + 1.959964 * seDiff],
significant: enough && pValue < 0.05,
};
}
Este pacote não declara servidores MCP.
| Versão | Commit | Publicado | Análise |
|---|---|---|---|
| 1.0.1 | c83c2b1 | há 5 horas | análise aprovada |
- proposto
- O GenPM propõe o comando npm e só o executa se você disser sim.
- Usado por (1)
- @core/kit-landing ^1.0.0
- análise
- análise aprovada · 0 achados
- commit
- v1.0.1 → c83c2b1012acc5483bb03c0706dba0781378102b · verificado após o download
- scripts
- Nenhum. O GenPM nunca executa código de pacotes.
- licença
- MIT
- Qualidade
- 100/100
- Licença reconhecidacumprido
- AGENTS.md explica o propósitocumprido
- AGENTS.md tem passos de integraçãocumprido
- AGENTS.md lista convenções ou proibiçõescumprido
- Inclui testescumprido
- Escaneamento de segurança aprovadocumprido
- Publicado nos últimos 6 mesescumprido
- Publicador verificadocumprido
- Resumo e palavras-chavecumprido
- denúncia
- Viu algo errado?