@core / experiments

1.0.1 ▾

1.0.1

인증됨MIT

GitHub

동의 기반 A/B 테스트: 안정적 배정, 전환 목표, 유의성 포함 결과

코드파일 8개컨텍스트약 601토큰검사 통과

약 601토큰→ src/lib/experiments/AGENTS.md→ .cursor/rules/genpm-core-experiments.mdc

이것이 AI가 src/lib/experiments에서 작업할 때 읽는 내용 그대로입니다. 그 외에는 컨텍스트에 아무것도 추가되지 않습니다.

@core/experiments — rules for AI agents

Purpose

A/B tests that respect consent. Visitors who accepted analytics cookies get a stable variant (deterministic hash of the pseudonymous consent id, no extra cookie); everyone else always sees the control and nothing is stored. Conversions are recorded once per visitor and goal; results show rates, lift, p-value and a 95 % confidence interval (two-proportion z-test) and are frozen when the experiment stops. Tables: experiments, experiment_participants.

Map

index.ts — getVariant, recordConversion, upsertExperiment, setExperimentStatus, experimentResults, experimentsAdminResource, pruneExperimentsJob, compare.
react.ts — <Experiment name cookieHeader variants={{ a, b }} /> (server component).
stats.ts — statistics without dependencies. schema.ts — tables.

Integration

Requires @core/consent (the banner) and its cookie. Env: EXPERIMENTS_SECRET (≥ 32 random chars).
Migrations as in src/lib/db/AGENTS.md. Add experimentsAdminResource to src/genpm/admin.ts (with kit-landing: its "A/B tests (optional)" lines).
Create the experiment in the admin (key, goal, variants: the first is the control), then press "Start".
Render: const { variant } = await getVariant('hero-test', cookieHeader) (cookie header of the request) and show the matching content; or <Experiment name="hero-test" cookieHeader={…} variants={{ a: <A />, b: <B /> }} />.
Conversion: where the goal happens on the server, await recordConversion('lead', req.headers.get('cookie')).
Schedule experiments.prune daily (@core/jobs schedule) to delete participants 30 days after an experiment stops (with kit-cms, its setup-schedules.ts does it).
Verify: with analytics accepted the same browser always sees the same variant; with cookies rejected, always the control.

Conventions

Decide only when significant is true and the test ran at least one or two full weeks (weekday effects).
Change one thing per experiment; don't edit variants or weights while it runs (blocked; stop and create a new one).

Don't

Don't assign or record visitors without analytics consent, and don't store emails or ids in experiments.
Don't stop a test early because one variant "looks better" (peeking inflates false positives).
Don't use experiments to show different prices to different people.

# @core/experiments — rules for AI agents

## Purpose
A/B tests that respect consent. Visitors who accepted analytics cookies get a stable variant (deterministic hash of
the pseudonymous consent id, no extra cookie); everyone else always sees the control and nothing is stored.
Conversions are recorded once per visitor and goal; results show rates, lift, p-value and a 95 % confidence interval
(two-proportion z-test) and are frozen when the experiment stops. Tables: `experiments`, `experiment_participants`.

## Map
- `index.ts` — `getVariant`, `recordConversion`, `upsertExperiment`, `setExperimentStatus`, `experimentResults`, `experimentsAdminResource`, `pruneExperimentsJob`, `compare`.
- `react.ts` — `<Experiment name cookieHeader variants={{ a, b }} />` (server component).
- `stats.ts` — statistics without dependencies. `schema.ts` — tables.

## Integration
1. Requires @core/consent (the banner) and its cookie. Env: `EXPERIMENTS_SECRET` (≥ 32 random chars).
2. Migrations as in `src/lib/db/AGENTS.md`. Add `experimentsAdminResource` to `src/genpm/admin.ts` (with kit-landing:
   its "A/B tests (optional)" lines).
3. Create the experiment in the admin (key, goal, variants: the first is the control), then press "Start".
4. Render: `const { variant } = await getVariant('hero-test', cookieHeader)` (cookie header of the request) and show
   the matching content; or `<Experiment name="hero-test" cookieHeader={…} variants={{ a: <A />, b: <B /> }} />`.
5. Conversion: where the goal happens on the server, `await recordConversion('lead', req.headers.get('cookie'))`.
6. Schedule `experiments.prune` daily (@core/jobs `schedule`) to delete participants 30 days after an experiment stops
   (with kit-cms, its `setup-schedules.ts` does it).
7. Verify: with analytics accepted the same browser always sees the same variant; with cookies rejected, always the control.

## Conventions
- Decide only when `significant` is true and the test ran at least one or two full weeks (weekday effects).
- Change one thing per experiment; don't edit variants or weights while it runs (blocked; stop and create a new one).

## Don't
- Don't assign or record visitors without analytics consent, and don't store emails or ids in experiments.
- Don't stop a test early because one variant "looks better" (peeking inflates false positives).
- Don't use experiments to show different prices to different people.

.genpmignore 적용 후 주입될 정확한 트리입니다. 고정 대상:

src/lib/experiments/schema.ts읽기 전용 · c83c2b1

// Tablas de @core/experiments. Las recoge drizzle-kit vía src/lib/db/drizzle.config.ts.
import { index, jsonb, pgTable, primaryKey, text, timestamp } from 'drizzle-orm/pg-core';
import { primaryId, timestamps } from '../db/index.ts';

export type ExperimentVariant = { key: string; weight: number };

export const experiments = pgTable('experiments', {
  id: primaryId('exp'),
  /** Clave estable usada en el código (`getVariant('hero-test')`). */
  key: text('key').notNull().unique(),
  name: text('name').notNull(),
  status: text('status', { enum: ['draft', 'running', 'stopped'] }).notNull().default('draft'),
  /** La primera variante es el control. */
  variants: jsonb('variants').$type<ExperimentVariant[]>().notNull(),
  /** Objetivo de conversión (`lead`, `purchase`, `click`… lo decide quien llama a `recordConversion`). */
  goal: text('goal').notNull(),
  startedAt: timestamp('started_at', { withTimezone: true, mode: 'date' }),
  stoppedAt: timestamp('stopped_at', { withTimezone: true, mode: 'date' }),
  /** Resultados congelados al parar (los participantes se borran después). */
  results: jsonb('results').$type<unknown[]>(),
  ...timestamps,
});

/**
 * Participantes con consentimiento de analítica: seudónimo (HMAC del id de consentimiento), variante y si convirtió.
 * Sin datos personales; se borran 30 días después de parar el experimento (`pruneExperimentsJob`).
 */
export const experimentParticipants = pgTable(
  'experiment_participants',
  {
    experimentId: text('experiment_id')
      .notNull()
      .references(() => experiments.id, { onDelete: 'cascade' }),
    visitor: text('visitor').notNull(),
    variant: text('variant').notNull(),
    exposedAt: timestamp('exposed_at', { withTimezone: true, mode: 'date' }).notNull().defaultNow(),
    convertedAt: timestamp('converted_at', { withTimezone: true, mode: 'date' }),
  },
  (t) => [primaryKey({ columns: [t.experimentId, t.visitor] }), index('experiment_participants_variant_idx').on(t.experimentId, t.variant)],
);

export type Experiment = typeof experiments.$inferSelect;