---
title: "Visualization Engine — Living Build Guide & Field Manual"
type: engine-playbook
status: shipped                       # MVP is built, deployed, and live
version: 1.0.0                        # xx.xx.xx · 1.x.x = MVP-ready (built + deployed). See Changelog at the end.
last_updated: 2026-06-28
reference_implementation: cm92-studio-os
reference_deploy: https://studio.tentaclepress.com
applies_to: any project with a file-based corpus of code + knowledge (markdown, scripts, configs)
ai_directive: >
  On read in a project: first determine whether this visualization already exists here
  (look for an interactive file map / graph + a generator script + a deploy).
  • If it does NOT exist → SPEC it against §3 Quickstart, confirm scope + scrub policy with the owner
    (one decision at a time), then BUILD. Do not build silently — the owner blesses scope first.
  • If it DOES exist → treat this as the maintenance + iteration manual; re-generate to keep it living,
    and extend per §5 Roadmap.
maintainers: [owner (the human router), build co-pilot (AI)]
tags: [visualization, infographic, d3, force-graph, generator, vercel, reusable-tooling, data-viz]
harmonizes_with: [README.md, scripts/build-graph.mjs, index.html, svg-1-overview.svg, svg-2-howitworks.svg, svg-3-files.svg]
---

# 📊 Visualization Engine — Living Build Guide & Field Manual

*How we turned a sprawling, two-repo studio into a single navigable picture — and how to do it again, anywhere.*

> **Read this if** you want to (a) stand up the same "living file map" in another project, (b) understand
> what's already built and why, or (c) build the *next, more advanced* visualization on top of the same data.
> It is three documents in one: a **build guide** (§3), a **field manual + retro** (§1, §4), and a **roadmap +
> changelog** (§5, §7). It is meant to be **executed**, not just read — see the `ai_directive` in the frontmatter.

---

## 0. The one idea

**Every file in a project already knows about other files — it just doesn't show you.** A markdown doc cites
another doc; a script reads two repos; a decision log points at a charter. Those references form a graph that
already exists, invisibly, in the text. The engine's whole job is to **extract that latent graph and make it
something you can see, filter, and walk** — generated straight from the repos, so it never drifts from the truth.

The reference build is live: **[studio.tentaclepress.com](https://studio.tentaclepress.com)** — *129 files · 723
references*, color-coded by department, sized by how connected each file is, click any node to trace what it
references and what references it.

---

## 1. What's built (the reference implementation)

Four artifacts, each independently useful, that together make up the engine. All live in this repo
(`cm92-studio-os`), are self-contained (no build step), and deploy as static files.

| # | Artifact | File | What it is |
|---|---|---|---|
| 1 | **The living generator** | `scripts/build-graph.mjs` | A Node script that walks the project's repos, extracts file-to-file references, and emits the graph data. Re-run anytime to refresh — this is what keeps the map "living." |
| 2 | **The interactive map** | `index.html` + `graph-data.js` | A single-file D3 force-directed graph. The data file is generated; the HTML is the viewer. This is the "Data is Beautiful" centerpiece. |
| 3 | **The static trio** | `svg-1-overview.svg`, `svg-2-howitworks.svg`, `svg-3-files.svg` | Three square (1:1, mobile-friendly) hand-built diagrams: a high-level overview, the session loop, and the file index. For docs/README/social where an interactive graph can't go. |
| 4 | **The deploy rails** | git → Vercel (git-connected) | Push to GitHub → Vercel auto-rebuilds → live on a custom subdomain. Zero-config because the repo serves from its root. |

### The data contract (what the generator emits)

`graph-data.js` defines one global: `window.GRAPH_DATA = { meta, nodes, edges }`.

```js
meta:  { generated:"YYYY-MM-DD", nodeCount, edgeCount, byGroup:{ <group>:count, … } }
nodes: [ { id, label, path, half, group, deg } , … ]   // id = "<half>/<path>"; deg = total references touching it
edges: [ { source, target } , … ]                       // a directed "source references target"
```

- **`group`** is the visual layer — for this studio: the six departments (`art · design · engine · narrative ·
  audio · production`) plus `studio` (Codex), `mailbox`, `code`, `scripts`, `charter`. Each gets a color.
- **`half`** is which repo the file lives in (`code` vs `docs`) — the two-repo split, preserved in the data.
- **`deg`** (degree) drives node size and label priority: the more connected a file, the bigger and the more
  likely it's labelled at rest.

> This contract is the seam. **Any** visualization in §5 consumes this same `{meta, nodes, edges}` — build the
> generator once, and every future infographic is just a new lens on the same data.

---

## 2. How each piece works (the deeper cut)

### 2a. The generator — `scripts/build-graph.mjs`
- **Inputs:** two repo roots via env vars (`CM92_CODE_ROOT`, `CM92_DOCS_ROOT`), with neutral defaults. It walks
  each tree, including `.md .sh .html .mjs .js .json .webmanifest`, skipping `.git`, `node_modules`, `assets`,
  `knowledge-backup`, `_archive`, `.venv`, `vault`, `.vercel`, `experiments`, and anything matching `vault|secret|.env|imagegen|credential`.
- **Edge detection:** for each file, it reads the content and looks for *other files' names/paths* appearing in
  the text. A hit becomes an edge. Filenames shorter than 5 chars and ambiguous basenames (same name in two
  places) are handled conservatively to avoid false links.
- **Output:** `graph-data.js` at the repo root — **paths and edges only, never file contents.**
- **Three safety gates** run after every build and print ✓/✗: (1) no absolute paths leaked, (2) no forbidden
  tokens, (3) no "spoiler" tokens survived. See §4 for why these exist.

### 2b. The viewer — `index.html` (D3 v7, vanilla JS, ~230 lines, one file)
- **Encoding:** color = `group`; radius = `4 + √degree × 2.3`; hubs (deg > 14) get a colored glow.
- **Physics:** a `forceSimulation` with link/charge/center/collide forces; link distance grows with the degree
  of its endpoints so hubs get breathing room.
- **Interactions:**
  - **Drag** a node to rearrange; **scroll/pinch** to zoom (0.25×–4×); **pan** the canvas.
  - **Click a node → trace:** highlights its neighborhood, dims everything else, colors its links, and opens a
    **detail panel** listing *references →* and *← referenced by*, each row clickable to hop to that file.
  - **Search box:** dims non-matches, outlines matches in white.
  - **Legend = filter:** tap any layer to toggle it on/off.
  - **Smart labels (`refreshLabels`):** at rest, only major hubs (deg ≥ 18) are labelled to avoid clutter;
    while tracing a *small* neighborhood, everything in it is labelled; for a hub, just the selected node;
    search labels all matches.
- **Look:** cosmic-dark, radial-gradient background with a faint honeycomb hex pattern — the studio's motif.
- **Responsive:** `viewBox` tracks the window; a mobile media query shrinks the chrome. Works on a phone.

### 2c. The static trio — the `svg-*.svg` files
Hand-authored, all `viewBox="0 0 1000 1000"` (square, reads well on desktop **and** phone), same cosmic-dark
honeycomb language as the viewer. Use them where an interactive canvas can't go (a README, a slide, a tweet).
- `svg-1-overview.svg` — the high-level "flower" honeycomb + the stated goal.
- `svg-2-howitworks.svg` — the **Kickoff → Work → Handoff** session loop, with drawn "RULE" badges.
- `svg-3-files.svg` — the file index / table of contents, code-half vs docs-half.

---

## 3. Quickstart — stand this up in a NEW project

Assumes the project is one or more repos of text files. Adjust names to taste.

1. **Copy the two engine files** into the new project's *web* repo (the one that deploys):
   `scripts/build-graph.mjs` and `index.html`.
2. **Point the generator at the repos.** Edit the `ROOTS` array (or set `CM92_CODE_ROOT` / `CM92_DOCS_ROOT`
   env vars) to the project's code + docs roots. Adjust `DEPTS` / the `group` logic to *that* project's layers
   (departments, packages, modules — whatever the meaningful color buckets are).
3. **Generate:** `node scripts/build-graph.mjs` → writes `graph-data.js`. Confirm the three safety gates print
   ✓ and the node/edge counts look sane.
4. **Preview locally:** serve the repo root (`python3 -m http.server 8131`) and open `index.html`. Click a few
   nodes; confirm tracing + the detail panel work.
5. *(Optional)* **Author the static trio** if you want shareable stills — copy the `svg-*` files and re-theme.
6. **Scrub if public** (§4): decide what must not leave the building, add a gitignored scrub map + denylist,
   re-run, confirm gates green, and **sweep the whole output directory** for stray secrets/paths before pushing.
7. **Deploy (Vercel, git-connected):**
   - Push the web repo to GitHub. In Vercel, import it (or `vercel link` + `vercel git connect`) → it
     auto-deploys on every push. Serve from repo root = zero config.
   - **Custom subdomain:** add the subdomain to the Vercel project, then add **one** DNS record at your DNS
     provider. See the DNS lesson in §4 — get the **exact per-domain CNAME** from Vercel's API, not the generic
     one, and **do not** switch your nameservers.

> **AI execution note — spec before you build.** If you're an agent standing this up in a fresh project, run
> this 5-question spec past the owner first (one at a time), then build:
> 1. **Repos** — which code/docs roots feed the graph?
> 2. **Groups** — what are the meaningful color buckets (departments · packages · modules)?
> 3. **Public or private** — does anything in the corpus need scrubbing before it ships?
> 4. **Scrub list** — if public, which names/paths must be generalized (→ a *gitignored* scrub config + denylist)?
> 5. **Home + deploy** — which repo hosts it, and on what URL/subdomain?
>
> Only after the owner blesses these do you build.

---

## 4. Lessons learned (the retro — what worked, what bit us)

Tagged in the studio's Codex style: 🔑 key (it works) · 🧱 wall (don't) · ⚠️ danger (works, but bites) · 📜 decision.

- 🔑 **Generate, never hand-maintain.** The map is built *from* the repos, so it can't lie about them. A
  hand-drawn diagram rots the day after you draw it; a generated one is true every time you re-run it.
- 🔑 **One data contract, many lenses.** Emitting plain `{meta, nodes, edges}` means the force graph, a future
  timeline, a heatmap, and a chord diagram are all *the same data, drawn differently.* Don't couple the
  extractor to one chart.
- 🔑 **Square + dark + responsive by default.** `1:1` SVGs and a `viewBox`-tracking canvas read well on a phone
  with no extra work. The owner reviews on a phone — design for the small screen first.
- 🔑 **Labels are a feature, not a default.** Labelling every node turns the graph into noise. Cap labels to
  hubs at rest and reveal more only on trace/search — this single rule was the difference between "mess" and
  "map."
- ⚠️ **Filename-matching produces false edges.** Short or duplicated basenames create spurious links; we guard
  with a min-length and an ambiguous-basename check. Tune these per project — too loose = hairball, too tight =
  disconnected dust.
- 🧱 **Don't switch nameservers to the host's.** Vercel's dashboard nudges you to delegate your whole domain to
  its nameservers. For a single subdomain on an existing domain, that's the wrong path — it would move *all*
  your DNS (other sites, email) to the host. Keep your third-party DNS and add **one** record.
- 🔑 **Get the *exact* per-domain CNAME from the host's API, not the generic one.** The generic `cname.<host>`
  target works but won't match your other subdomains' records and looks "off." The host assigns a **per-domain**
  CNAME (a hashed target); pull it from the API
  (`GET /v6/domains/<domain>/config` → `recommendedCNAME` rank 1) so the record is exact and consistent.
  *(This one cost a round-trip of confusion before we went to the API.)*
- 📜 **The generator's own source must not disclose what it scrubs.** If you rename `secret-name.md` →
  `neutral.md` *inside* the public script, the script itself leaks the secret name. Move the rename map +
  denylist into a **gitignored** local config the script reads at runtime, and keep a **denylist tripwire** that
  fails the build if a forbidden token survives into the output. Scrub the *map*, not just the output.
- ⚠️ **A public showcase needs a full-directory sweep, not just gate-passing.** The automated gates catch the
  output file; a human/agent still sweeps the whole deploy dir for stray absolute paths, emails, tokens, and
  private names before flipping anything public. Belt *and* suspenders.

---

## 5. Roadmap — the next visualizations (more lenses on the same data)

The force graph is v1. Every idea below reuses the **same generated data** (or a tiny extension of it), so each
is mostly a new viewer, not a new pipeline. Rough order by value-to-effort:

| Lens | What it shows | Extra data needed | Effort |
|---|---|---|---|
| 🕒 **History timeline / swimlane** | Every decision ticket + handoff as a dated event, one lane per department — the studio's story over time. | dates from decision logs / frontmatter `last_updated` | low |
| 🌡️ **Knowledge-freshness heatmap** | Each file colored by how long since it changed — stale docs glow red. Operationalizes "make silent failures loud." | `last_updated` per file (already in frontmatter) | low |
| 🧭 **Orphan / dead-end finder** | Files with zero inbound or zero outbound references — cleanup + "what's disconnected" targets. | none (derive from existing edges) | low |
| 🍩 **Department sunburst / treemap** | File counts + size per department, drill into folders — where the weight sits. | none (group + path already present) | low |
| 🎻 **Cross-department chord diagram** | Reference density between layers (who leans on whom) — a compact alternative to the force graph. | none (aggregate existing edges) | medium |
| 🌊 **Decision → finding Sankey** | How a question (deli ticket) flowed into a locked decision and then a Codex finding — the audit chain, visualized. | linkage between tickets, decisions, findings | medium |
| 🔖 **Build / version ribbon** | Git tags + build numbers as a release timeline, annotated with what each shipped. | git tag history | medium |
| 🗂️ **Codex knowledge graph** | The findings themselves as nodes (walls/keys/decisions), linked by their `[[wikilinks]]`. | parse Codex wikilinks | medium |
| 🖼️ **Asset contact sheet** | A visual index of generated art/sprites/meshes per creature/zone — the art pipeline made browsable. | image manifest from the art repo | medium |
| 🎬 **Session-replay arc** | Kickoff→work→handoff loops animated over time — momentum + cadence of the whole studio. | session timestamps | high |

> When picking the next one: prefer the lenses that **need no new data** (top of the table) — they're nearly
> free given the generator already exists. Each new lens is a candidate for its own minor version bump below.
>
> **Recommended next two:** the **History timeline** (low effort, high narrative payoff) and the
> **Knowledge-freshness heatmap** (operationalizes *make silent failures loud*) — propose them as **1.1** and
> **1.2** once 1.0 has settled.

---

## 6. Operating notes (keeping it living)

- **Re-deploy on demand:** the project is git-connected, so a normal `git push` rebuilds the live site. For an
  out-of-band redeploy, the Vercel CLI with a token from your secrets vault can ship the current build directly.
- **Refresh the data:** re-run `node scripts/build-graph.mjs` whenever the repos change, commit the regenerated
  `graph-data.js`, push. (A future nicety: run the generator as a Vercel build step so the data refreshes on
  every deploy automatically.)
- **Scrub discipline:** the scrub map lives **only** in a gitignored local config — never commit it. If you add
  a new sensitive name to hide, add it to that config + the denylist, re-run, and confirm the gates stay green.
- **Leave it green:** after any change, load the page and confirm it renders + traces before moving on.

---

## 7. Changelog

*Versioning: `MAJOR.MINOR.PATCH`. **`1.x.x` = MVP-ready** (built, deployed, and working). `0.x.x` = pre-MVP
iterations. Each release below corresponds to a real step in the build; consider git-tagging future releases so
this list and the version-control history stay in lock-step.*

| Version | Date | What changed |
|---|---|---|
| **1.0.1** | 2026-06-28 | **Field manual authored** (`VISUALIZATION-ENGINE.md`) + **adversarially verified** (3-critic pass: accuracy · completeness · safety). Accuracy fixes (full excluded-dirs list + `imagegen` in the regex); README status flipped *in progress → live*; §3 spec checklist + §5 next-candidates added. Roadmap of 10 next-lens visualizations captured (§5). |
| **1.0.0** | 2026-06-28 | **🚀 MVP SHIPPED.** Deployed live at **studio.tentaclepress.com** (Vercel, git-connected; per-domain CNAME at third-party DNS). 129 files · 723 references rendering, traceable, filterable, on desktop + mobile. |
| **0.4.0** | 2026-06-28 | Pre-publish hardening: 4-agent leak audit; gitignored `scrub.local.json` rename-map + denylist tripwire; three safety gates green + full-dir sweep clean; repo restructured to serve from root (zero-config Vercel); pushed to a private GitHub repo. |
| **0.3.0** | 2026-06-28 | **The interactive engine.** Living generator (`scripts/build-graph.mjs`, parses the two repos → `{meta,nodes,edges}`) + the D3 force-graph viewer (`index.html`): color-by-layer, size-by-degree, click-to-trace + detail panel, search, legend filter, smart label-capping, mobile-responsive. |
| **0.2.0** | 2026-06-28 | **The static trio.** `svg-1-overview` (flower hero + goal line), `svg-2-howitworks` (session loop, drawn RULE badges), `svg-3-files` (file index). All square 1:1, mobile-friendly, cosmic-dark honeycomb language. Fixed badge legibility + title/subtitle overlap + box overflow. |
| **0.1.0** | 2026-06-28 | First static diagram (`architecture.svg`) — the "0.5 version" that proved the visual language before the trio. |

---

*Part of the Critical Mass '92 Studio Operating System. The system is the part meant to travel — this manual is
how this piece of it travels.*