Detecting Token Drift Between Design and Production

Part of Visual Regression & Token Drift. Every other check in the pipeline compares two things inside the repository. This one compares the design source against what a browser actually resolves on the live site — the one edge nothing else covers.

Collecting both sides of the drift comparison The design tool's variable API on one side and a headless browser reading computed custom properties from production on the other, both normalised into the same shape before comparison. design variables API names, values, modes read-only, scheduled production page getComputedStyle on :root per theme, per brand normalise name, colour space, units classify four categories Normalisation is most of the work: the two sides express the same colour in different spaces and the same name in different cases.
Reading production rather than the repository is what makes this check meaningful — it measures what shipped, including anything a deploy step changed.

Prerequisites

  • Read access to the design tool’s variables API, with a token scoped to read-only.
  • A published mapping from design variable names to CSS custom property names — usually the same adapter the sync pipeline uses.
  • A public or authenticated URL that renders the production stylesheet.
  • A scheduled CI job; this check does not belong on a pull request.

Step-by-Step Implementation

Step 1 — Read the design side

Intent: fetch the variables and flatten them into name → value pairs per mode.

// scripts/drift/design.mjs
export async function readDesignVariables({ fileKey, token }) {
  const response = await fetch(
    `https://api.figma.com/v1/files/${fileKey}/variables/local`,
    { headers: { 'X-Figma-Token': token } });
  if (!response.ok) throw new Error(`Design API ${response.status}`);

  const { meta } = await response.json();
  const out = {};                       // mode name → { variable name → value }

  for (const variable of Object.values(meta.variables)) {
    const collection = meta.variableCollections[variable.variableCollectionId];
    for (const [modeId, value] of Object.entries(variable.valuesByMode)) {
      const mode = collection.modes.find((m) => m.modeId === modeId)?.name ?? 'default';
      (out[mode] ??= {})[variable.name] = value;
    }
  }
  return out;
}

Why this works: grouping by mode rather than flattening everything into one map preserves the light and dark distinction, which is exactly where drift concentrates — a designer updating one mode and forgetting the other is the single most common source of it.

Step 2 — Read the production side from a real browser

Intent: capture the resolved values a visitor receives, not the values in the repository.

// scripts/drift/production.mjs
import { chromium } from 'playwright';

export async function readProductionTokens(url, names) {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  await page.goto(url, { waitUntil: 'networkidle' });

  const out = {};
  for (const theme of ['light', 'dark']) {
    await page.evaluate((t) => document.documentElement.setAttribute('data-theme', t), theme);
    out[theme] = await page.evaluate((list) => {
      const styles = getComputedStyle(document.documentElement);
      return Object.fromEntries(list.map((n) => [n, styles.getPropertyValue(n).trim()]));
    }, names);
  }

  await browser.close();
  return out;
}

Why this works: getComputedStyle returns the value after the whole cascade has resolved — including anything a CDN transform, a tenant injection or a stale cached stylesheet contributed. Reading the repository would miss all of those, and those are precisely the failures a drift check exists to find.

Step 3 — Normalise before comparing

Intent: the two sides express the same value differently, and comparing raw strings produces noise instead of signal.

// scripts/drift/normalise.mjs
export const toCssName = (designName) =>
  '--ds-' + designName.toLowerCase().replace(/[\/\s]+/g, '-');

export function normaliseColor(value) {
  if (typeof value === 'object' && 'r' in value) {          // design API returns 0–1 floats
    const ch = (x) => Math.round(x * 255).toString(16).padStart(2, '0');
    return `#${ch(value.r)}${ch(value.g)}${ch(value.b)}`;
  }
  const rgb = String(value).match(/rgba?\(([^)]+)\)/);      // browser returns rgb()
  if (rgb) {
    const [r, g, b] = rgb[1].split(',').map((n) => Number(n.trim()));
    const ch = (x) => x.toString(16).padStart(2, '0');
    return `#${ch(r)}${ch(g)}${ch(b)}`;
  }
  return String(value).trim().toLowerCase();
}

Why this works: collapsing both sides to a lowercase hex string removes four categories of false difference at once — float versus integer channels, rgb() versus hex notation, uppercase versus lowercase, and incidental whitespace. Without this step a first run reports every single token as drifted, which is how these checks get abandoned on day one.

Four categories of drift A two-by-two grid classifying differences: value drift, missing in production, missing in design, and name mismatch, each with its typical cause. Value drift same name, different value cause: a change landed on one side only Missing in production designed but never shipped cause: a sync that has not run or was rejected Missing in design shipped but not in the design source cause: an engineering-only token, often legitimate Name mismatch same value, different name cause: a rename applied on one side only
Only the first category is unambiguously a defect. The other three need a human decision, which is why this check reports rather than blocks.

Step 4 — Classify rather than count

Intent: a single drift number is not actionable; four categories are.

// scripts/drift/compare.mjs
export function compare(design, production) {
  const result = { valueDrift: [], missingInProduction: [], missingInDesign: [], nameMismatch: [] };
  const prodByValue = new Map(
    Object.entries(production).map(([name, value]) => [value, name]));

  for (const [name, value] of Object.entries(design)) {
    if (!(name in production)) {
      const sameValue = prodByValue.get(value);
      if (sameValue) result.nameMismatch.push({ design: name, production: sameValue, value });
      else result.missingInProduction.push({ name, value });
      continue;
    }
    if (production[name] !== value) {
      result.valueDrift.push({ name, design: value, production: production[name] });
    }
  }

  for (const name of Object.keys(production)) {
    if (!(name in design)) result.missingInDesign.push({ name, value: production[name] });
  }
  return result;
}

Why this works: detecting the name-mismatch case by looking up the value is what stops a rename from appearing as one missing token plus one unexpected token — two entries that look like problems and are really one bookkeeping difference.

Step 5 — Report as a trend, not a gate

Intent: drift accumulates from legitimate activity on both sides, so a blocking check would fire constantly.

# .github/workflows/drift.yml
name: Design/production drift
on:
  schedule: [{ cron: '0 6 * * 1' }]      # Monday morning, before the week's planning
  workflow_dispatch:
jobs:
  drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20', cache: 'npm' }
      - run: npm ci
      - name: Measure drift
        env:
          FIGMA_TOKEN: ${{ secrets.FIGMA_READ_TOKEN }}
        run: node scripts/drift/run.mjs --url https://www.example.com --out drift.json
      - name: Open or update the tracking issue
        uses: actions/github-script@v7
        with:
          script: |
            const drift = require('./drift.json');
            const body = [
              `**Design/production drift — ${new Date().toISOString().slice(0, 10)}**`,
              ``,
              `- value drift: ${drift.valueDrift.length}`,
              `- missing in production: ${drift.missingInProduction.length}`,
              `- missing in design: ${drift.missingInDesign.length}`,
              `- name mismatch: ${drift.nameMismatch.length}`,
            ].join('\n');
            // …find the existing issue by title and update it, else create it.

Why this works: a weekly issue update creates a visible series without interrupting anyone’s work. Teams that make drift blocking discover within a fortnight that the design source is edited by people who are not in the pull request, so the block lands on whoever happens to open the next unrelated change.

Verification

Prove each category can be produced. In a scratch design file and a preview deployment: change one value on the design side only and confirm it appears as value drift; add a variable that has not been synced and confirm it appears as missing in production; add a CSS-only token and confirm it appears as missing in design; rename a variable and confirm it appears as a name mismatch rather than as two separate entries.

Then confirm the normalisation is doing its job by running the comparison with the two sides deliberately in agreement. The result should be four empty arrays. A first run that reports hundreds of differences almost always means a normalisation gap rather than genuine drift, and the fix is in step 3.

A drift series read over eight weeks A bar series showing value drift counts across eight weekly runs, rising steadily and then dropping after a sync pipeline fix, illustrating why the trend matters more than any single reading. w1 w2 w3 w4 w5 w6 w7 w8 Value drift count, weekly The rise across weeks 3 to 6 was a broken sync job; nobody noticed until the series made it obvious. A single reading of "18 tokens drifted" means nothing. The shape of the series is the finding.
Drift is a rate, not a state. The value of the check is almost entirely in the second derivative — whether it is getting worse.

Troubleshooting

Symptom Likely cause Fix
Every token reports as drifted on the first run Colour space or name normalisation is incomplete Print ten sample pairs side by side; the pattern is usually obvious immediately
Drift appears only in the dark theme The design file’s dark mode was updated without a sync Check the sync job’s mode mapping, then re-run it
Production values differ from the repository A cached stylesheet or a CDN transform Compare the deployed asset’s hash against the built one before investigating tokens
The design API returns 403 on a schedule but works locally The scheduled job uses a different token or lacks file access Use a dedicated read-only token stored as a secret, and test it in the scheduled context
Name mismatches appear after every rename The adapter’s naming function changed on one side Version the naming function alongside the tokens so both sides use the same one

Why Read Production Rather Than the Repository

It would be simpler to compare the design source against the token JSON in the repository, and most teams try that first. It is worth understanding what that version misses, because the gap is larger than it appears.

Between the token file and the pixels a visitor sees there are several stages that can change a value: the compiler and its transforms, any post-processing such as autoprefixing or minification, a bundler that may inline or reorder declarations, a CDN that may serve a stale artefact, and — in a multi-tenant product — a runtime injection that overrides tokens per request. Comparing against the repository validates the first link of that chain and asserts nothing about the rest.

Reading production also catches the failure that no repository comparison can: a deploy that did not happen. A token change merged three weeks ago and never released looks perfectly consistent in the repository and is invisibly absent from the site. Drift measured against production reports it, which is often the first time anyone notices the release pipeline stalled.

The cost is that the check needs a real URL and a browser, which makes it slower and gives it a dependency on the site being up. That is acceptable for a weekly scheduled job and unacceptable for a pull request check — which is another reason this particular comparison belongs on a schedule rather than in the merge path.

Migration Note

The first run of a drift check on a mature system is uncomfortable, and it is worth setting expectations before running it. A system that has been shipping for two years with a manual sync typically shows drift in the tens of tokens, and most of it turns out to be legitimate: engineering-only tokens, design explorations that were never intended to ship, and renames applied on one side.

Spend the first week triaging into two lists — differences that should be reconciled, and differences that are expected and should be suppressed. Suppress the second list explicitly, with a reason per entry, in a file next to the drift script. That file becomes the record of what the two sources are allowed to disagree about, which is a genuinely useful artefact quite apart from the check itself.