product-vamp/TECHNICAL_GUIDE.md

# Product Vamp Technical Guide

## Repository Shape

Product Vamp follows the Modulus product layout used by the khodo repos:

- `include/vamp_zero` and `src/vamp_zero`: Pi Zero C++23 application.
- `include/vamp_pico` and `src/vamp_pico`: Pi Pico 2 W C++17 firmware.
- `include/vamp_protocol` and `src/vamp_protocol`: shared route and page-command contracts used by both devices.
- `cmake/pico.cmake` and `cmake/zero.cmake`: device-specific build definitions selected by the root `CMakeLists.txt`.
- `include/vamp_style.hpp` and `src/vamp_style.cpp`: shared visual palette. `include/vamp_pico/artwork.hpp` defines the Pico coverage-image and sprite interfaces; fixed artwork is generated in the build directory.
- `data`: original songs, paired training solos, the frozen solo model, and harmonic/accompaniment policies. Expected-output fixtures live under `tests/`.
- `generator`: build-time Python rhythm planning, part generation, guitar placement, engraving, audio compilation, Pico artwork, and catalog construction.
- `package`: generic bundle inputs, scripts, config, static font assets, and device install files.

## Zero Runtime

The Zero process can run headless for local/API quick checks or with Luma for the Waveshare Zero-DISP-7A display.

`runtime.headless` controls display startup and `audio.enabled` controls audio startup. The unused `runtime.hardware` setting has been removed; older copies of that field are ignored like other unknown configuration fields.

The API is plain HTTP on the private AP:

- `GET /api/health`
- `GET /api/remote/display`
- `POST /api/remote/title` (registered title image ID)
- `POST /api/remote/heartbeat`
- `POST /api/remote/next`
- `POST /api/remote/previous`
- `POST /api/remote/select`

There is no browser remote, HTTP font route, or full-catalog status dump. `control` owns serialized device actions; direct menu actions carry the displayed input revision and catalog identity so intervening remote navigation invalidates them. Pico display responses include only focused-item/settings values and the selected title artwork ID.

Page commands use `/api/remote/next` and `/api/remote/previous`; the old `/api/page/*` routes are removed.

The AP setup pins NetworkManager to 2.4 GHz channel 6 with WPA2/RSN and CCMP ciphers so the Pico W can join using `CYW43_AUTH_WPA2_AES_PSK`.

On Raspberry Pi OS Lite, the installed display service owns `tty1`, conflicts with `getty@tty1.service`, and starts the Zero container through `product_vamp_start_zero`. The wrapper switches to the display VT when `chvt` is available and runs `vamp_zero` inside the container through `setsid --wait --ctty` with `/dev/tty1` as stdin, giving raylib's DRM backend a real virtual terminal while keeping Docker attached to the long-running display process. SSH remains the normal admin path and `tty2` is enabled as the fallback local console.

At startup the Zero validates the packaged `catalog.db` application identifier and required table structure, opens it read-only through Noto, and loads menu metadata. It never creates or migrates a database, scans score JSON, or repairs metadata. The main menu contains Performance and Settings. Settings holds Playback, Metronome, and Tempo; its values apply for the current session. Song -> Part -> Score navigation offers Acoustic, Electric, and Vocal without key selection. Musical interpretation, spacing, beaming, ties, accidentals, and pagination happen on the build host. The Zero displays engraved PNGs and retains at most two page textures.

Runtime catalogs contain immutable metadata shared by navigation, the HTTP service, and the UI. Song groups reference canonical records by index. SQLite retains complete bundles, while cached measure/page counts let menu queries avoid reading musical JSON. Python constructs the complete database in one transaction before deployment; metadata and embedded JSON are published together after an integrity check. The single `songs` table uses strict column types, primary-key and path uniqueness constraints, and JSON validity checks. Menu rows are ordered by title, composer, part, and ID with binary collation, using the matching `songs_menu` index; C++ groups the ordered rows in one pass without sorting them again. Noto provides read-only connections, bound score lookups, application-ID checks, and quick integrity checks. Product-specific schema and asset-path validation remain in the application. There are no custom rows, runtime imports, metadata-repair triggers, or persisted page-state tables. Selecting a score loads only that bundle asynchronously, discarding stale results after navigation; the view keeps at most two page textures.

`--once` (also `runtime.exit_after_startup`) is a headless readiness check: it checks database integrity and validates the catalog and referenced page assets, checks runtime-directory writes and the configured server bind, and exits without opening audio/display devices or publishing a readiness heartbeat. Page checks include containment, checksums, CPU-only PNG decoding, and exact 976x484 dimensions. The configured heartbeat destination must be absent or a regular file, never a directory, symlink, or special file. Runtime writes and readiness probes use exclusive sibling temporary files; readiness leaves an existing heartbeat unchanged and removes its probes. `--health-check` instead probes the running service's read-only `/api/health` endpoint using the configured timeout. A stopped/unresponsive service or an invalid response fails the check; it does not create or repair files. `--check-config` remains configuration-only.

Pico networking progresses without blocking touch sampling. Navigation commands use a bounded 16-entry FIFO ahead of background status polls; a full queue reports busy. Commands are not automatically retried because a timed-out request may already have executed. Background polls wait one second after completion before trying again.

The Zero menus use direct taps: release on a row to select it, tap switches to toggle them, and use the tempo minus/plus buttons to adjust BPM. Drags, canceled touches, multiple fingers, and stale menu actions do not activate items. Back clears tempo editing; returning to Settings restores the last setting's focus. Long song lists have previous/next focus buttons.

In score view, Zero and Pico share one gesture state machine: a second release within 220 ms of the first release produces previous, a hold of at least 700 ms produces select once, and other taps produce next. A late second release preserves both single taps; holding consumes a pending first tap. Zero schedules pending-tap deadlines without polling and discards canceled or obsolete timer callbacks.

## Score Generation

The build runs `./modulus.py python -m generator` against `data/` and writes build-only assets under the CMake binary tree. Every song produces acoustic, electric, and vocal parts in its authored musical key. Source counts are discovered, not fixed in code. Generator-only optional chord markers are consumed before publishing.

Songs live directly in `data/songs/`. Each source produces acoustic, electric, and vocal parts under `songs/`, with matching page directories. Adding, renaming, or removing a source requires no registry edit. Regeneration replaces the output tree and catalog. The two current songs generate six parts.

The pipeline is Soundslice/MuseScore -> uncompressed `.xml` -> musical model and rhythm plans -> part generation -> semantic JSON, audio, engraved pages, and Pico title artwork -> packaged SQLite catalog and assets. Export MusicXML 3.1 or 4.0, `score-partwise`, with one part and one staff. The musical model retains independent voices and per-tone durations/ties. Packaged music describes each generated part, not an archival copy of the XML.

### Musical Data

`data/songs/` contains authoritative source scores. `data/training/manifest.json` pairs each authored solo with a source and a slot (1 or 2), and declares its pitch convention. Both slots train **one pooled model**. Slot 1 maps to acoustic playback and slot 2 to electric playback; slot and instrument names are not model features. The four XML files are preserved unchanged, including the Autumn Leaves pickup and All of Me's full-bar opening.

`data/models/solo.json` is an explicitly trained, versioned statistical model. Its short-context counts, rhythm and register statistics are learned from the paired examples. Provenance records input hashes and the training version. This is supervised use of the supplied solos, not independence from them and not a neural network pretrained on an external corpus.

`data/composition/` contains shared guitar/difficulty policy, sparse accompaniment figures and explicit harmonic-treatment validation rules. The exhaustive rhythm/interval catalogs, 21-pattern scale catalog, fixed maps, old solo grammar and handwritten phrase scorer have been removed. The renderer can still display existing scale annotations as part of its compatible score schema; new solos do not create them.

### MusicXML Contract

| Area | Accepted notation |
| --- | --- |
| Pitch | Exact rational letter/alteration/octave spelling, including microtones, double/triple accidentals, and courtesy accidentals. B-sharp and C-flat retain the correct octave. |
| Rhythm | All fourteen MusicXML types, maxima through 1024th; up to sixteen augmentation dots; exact positive rational metric durations and separately stored written values. Time modifications must reconcile the two. |
| Chords and rests | Independent voice lanes, simultaneous chord tones, shorter additional chord tones, rests, full-measure rests, hidden events, forward spacers, and rhythm slashes. |
| Ties and tuplets | Per-pitch ties, including partial chord ties; nested and cross-bar tuplet spans. Nested tuplets require explicit `tuplet-actual` and `tuplet-normal` numbers. |
| Small notes | Grace notes with zero metric duration, cue notes, and measured single/paired tremolos. |
| Staff state | G/F/C, percussion, and no-clef staves; octave clefs and octave-shift spans; major/minor/modal keys or custom seven-letter alterations; timed key, clef, and meter changes. Authored TAB and technical fingering instructions are rejected; TAB is generated from pitched guitar parts. |
| Meter | Simple, compound, additive, mixed-denominator, and unmetered timelines. A partial measure must be marked `implicit="yes"`; an initial implicit measure is the arrangement pickup, including a full measure padded with leading rests. |
| Harmony | Structured roots, kinds, inversions, slash basses, and degree additions/alterations/subtractions. Harmony events have their own offsets, including inside held notes. Electric melody generation intentionally reattacks eligible long notes at changed harmonies; import preserves the authored notes. |
| Text | Title, composer, part name, multiple lyric verses with syllabification, elisions, and extenders, rehearsal labels at measure starts, and Vamp directives. |
| Tempo | Metronome directions and positive quarter-note BPM from `<sound tempo="...">`, either within a direction or directly in a measure. Playback/navigation attributes on `sound` are rejected. |
| Reading order | All measures are written out. Repeat bars, numbered endings, navigation jumps, and pass-specific ties are rejected. Ordinary barlines and contiguous ties remain supported. |

Multiple staves/parts, D.C./D.S./coda/fine navigation, slurs, dynamics, articulations, and expressive ornaments are outside the product contract and rejected. Unknown musical elements also fail. Serialized times and alterations use reduced signed-64-bit numerators and positive denominators; pitch octaves are -4 through 16, clef/octave shifts span at most three octaves, and meter counts are 1-1024 with denominator units 1-4096. These resource bounds produce diagnostics, not clamping or truncation.

Layout/exporter information such as page defaults, credits, print layout, font/color/position attributes, and source pagination does not determine the small-screen layout. Engraving is regenerated. Beam groups and written note values are retained; unrelated engraving metadata does not authorize ignoring musical descendants. `defusedxml==0.7.1` forbids XML entities and external access. Ordinary MusicXML DOCTYPE declarations are allowed without fetching their DTDs.

Chord labels are generated from structured harmony using `M`, `m`, `aug`, `dim`, and `sus`: for example `CM`, `Cm7`, `CM9`, `Caug`, `Cdim7`, `Csus`, `Csus2`, and `Cm7b5`. `CmM7` is a minor triad with a major seventh, corresponding to MusicXML's `major-minor` kind. Ninths, elevenths, and thirteenths include their underlying seventh; `Cmadd9` does not. Slash basses, degree alterations, added tones, and omissions remain semantic, not display-only. Older recognized spellings such as `-7`, `maj7`, and `+` are accepted but rendered canonically.

Standard MusicXML kinds and explicit degrees remain authoritative. For Soundslice compatibility, a text-only `<kind text="mM7">other</kind>` (also `m7b5`, `7b9`, and other recognized chord spellings) is normalized into a standard kind and degrees. Complete symbols such as `(Fm7)` or `(Bb7)` supply their own root; an exporter placeholder C is accepted only for text-only `other` harmonies. A conflicting non-placeholder root, conflicting bass, contradictory structured chord, or unknown spelling is an error with source/measure context. Raw exporter text is not retained as an override that could become stale during transposition. This compatibility path does not require editing the XML or weakening structured-harmony validation.

### Vamp Directives

Musical genres and player-style presets are removed. There is no default genre, alias table, generated `style` metadata, or genre selection. The strict importer rejects every obsolete `vamp:style` miscellaneous-field hint, including an empty value, instead of discarding it. The XML writer does not emit that field. Keys, modes, keyless notation, and local changes remain preserved; there is no transposition reference or `vamp:tonic` property. Visual styling, font attributes, engraving `Style`, presentation fingerprints, and ordinary `<bar-style>` notation are unrelated and remain supported.

Parenthesized chord symbols mark optional turnarounds, including Soundslice's text-only `(Fm7)` and `(Bb7)` exports. Alternatively, place staff text `vamp:optional-chord` at the same onset as exactly one harmony; this text may be hidden. Earlier arrangement sections retain those chords and their parentheses; the final ending omits them. The normalized harmony stores `parenthesized` display metadata separately from the generator-only `chord_optional` flag. Orphaned/duplicate markers and unknown `vamp:` properties are errors; consumed directives never enter runtime JSON.

### Linear Reading Order

There is no repeat analyzer, expansion pass, or repeat/ending engraver. Performance retains the seven-section arranger, writing every section explicitly onto successive pages; neither display nor playback jumps backward.

Repeat signs, numbered endings, D.C./D.S./coda navigation, and `time-only` ties fail import rather than being ignored or expanded. Write the intended notes and rests explicitly, including any alternate ending. Legacy repeat fields in generated musical documents are rejected; regenerate assets instead of retaining a compatibility interpreter. Full leading-rest measures remain full measures, not inferred pickups. Missing titles fall back to filenames and missing composer credits remain empty, as with the current Soundslice exports.

An explicitly marked opening pickup is excluded from the repeated song body. At each vocal or electric melody entry, it is right-aligned within the preceding arrangement bar. Outgoing music continues until the pickup's first sounding note, then releases as the incoming melody takes over; pickup rests do not erase earlier accompaniment. Lyrics, ties into the following downbeat, tuplets, and octave spans retain their timing. Pickup harmony changes also apply to the accompanying parts at their relocated offsets. All parts share the same bar count, and a pickup longer than its destination bar fails generation. Autumn Leaves uses a four-beat padded pickup and a 32-bar body, yielding 176 arrangement bars rather than 181.

Ordinary ties still link adjacent matching tones across measures and systems. Tuplets, independent voices, local staff changes, and manual page navigation are unchanged. Audio sample sustain loops remain an implementation detail for holding a single note, not a score-repeat feature.

### Runtime Schema

A newly generated bundle has three top-level fields:

- `music`: metadata and a single staff containing measures, independent voice lanes, timed harmonies/attributes/tempos, instruments, tuplets, and octave spans.
- `presentation`: engraver name, style checksum, fixed width/height, ordered relative PNG paths with SHA-256 checksums, and the `title_image` content identifier for Pico artwork.
- `playback`: availability, exact performed beat duration/events, shared PCM sample descriptors, and a generated release envelope. Unavailable interpretations carry a reason instead of an approximate timeline.

There are no authored-key/selected-key metadata pairs or musical genre/style fields. The score's key metadata describes its actual music, and menus display that key alongside its part name. Instrument normalization transforms explicit key cancellations with the same interval as the notes. Presentation's `style_sha256` identifies visual engraving settings and fonts, not a musical genre.

Measure durations, event onsets/durations, harmony offsets, span times, and pitch alterations use `{ "numerator": N, "denominator": D }`. Times are in quarter-note units and alterations in semitones. There is no tick grid, display-duration surrogate, or runtime flattening into one voice. Written note types, dots, and time modifications remain distinct from metric duration. Every event owns a voice; every chord tone has its own ID, written duration, duration, and reciprocal tie links. Tuplets reference events and belong to one voice.

### Shared Part Generation

`production.generate_role` accepts immutable canonical source analysis, a role, production data and settings. Canonical musical identity excludes display titles and filenames. Accompaniment, source melody, vocals and each solo have their own caches and can run in independent worker processes. No role reads another generated role. Assembly applies source-defined participation windows, including guitar-role exclusions and returning pickups, without changing another role's decisions.

`parts.py` retains exact source notes, rests, lyrics and ties. `melody.py` harmonizes sustained electric source-melody notes using their actual chord, without changing the head's pitch or rhythm. Guitar notation applies its octave transposition once. `comping.Canonical` independently chooses compact chord voicings and rhythm from the source form and shared policy, then reuses that part at the same source-form position throughout the arrangement. There is no generated-solo response or reuse of a generated melody's grips. Sparse bars and deliberate releases are legal.

Monophonic solos use strings 1–6 and frets 1–12, with no open strings or acoustic/electric string restrictions. Placement compares complete paths through legal positions, accounting for hand-region changes, travel time and string crossings. Ties retain their position. Chord placement retains its separate policy: frets 1–20 with a sounding anchor at fret 12 or below and the existing inclusive three-fret span. These costs approximate physical feasibility; they do not assign fingers or guarantee comfortable playing.

### Playback Boundary

All musical compilation and tone synthesis run in Python (`generator/playback.py`). Each transformed score receives a sorted timeline of independent tone events with rational quarter-note `onset` and `end` positions. Ties become one continuous attack. Rests and spacers remain silence, including trailing silence. Harmony labels are not auto-accompaniment: only pitches actually present in the selected arrangement sound. Cue notes are silent, hidden pitched notes sound, and pitchless rhythm slashes are silent.

The `soft_struck` profile generates shared mono 48 kHz PCM16 WAVs with a short attack/decay and a seamless sustain loop, plus a shared 50 ms release envelope. Exact sounding pitches, including supported microtones, determine samples. Instrument transposition is applied once from the generated musical model, without role- or filename-dependent octave heuristics. Samples are shared across scores and voices by content identity. There are no SoundFonts, bundled instrument samples, oscillators, or MusicXML parsing at runtime.

Grace groups honor explicit steal-time-previous/following or make-time; unspecified groups share 25 percent of the following event. Conflicting grace modes, trailing grace groups, unsupported unpitched sound, tied measured tremolo, and incompatible paired tremolo make playback explicitly unavailable while retaining the notation. Measured tremolo uses written subdivision and tuplet ratios. The tone range is 20-18,000 Hz. Asset validation checks containment, SHA-256, PCM format/length, loop endpoints, envelope shape, event ordering, references, and rational timing. Generation remains transactional if validation fails.

When grace groups steal from both ends of one donor, their adjustments accumulate against its original duration. Consuming the entire donor or any chord tone makes playback unavailable rather than overlapping the grace group and principal note.

At runtime the shared on-demand score loader serves both display and audio. A background worker validates and loads only the selected score's samples, converts absolute beat positions to rounded sample frames at the menu BPM, and prepares a Sonus sequence. Pitch does not change with BPM, and rounding absolute positions avoids cumulative timing drift. The I/O loop publishes the prepared sequence and polls completion without doing disk I/O or musical interpretation. Sonus mixes preallocated voices from immutable PCM and retires them on control threads; the callback never synthesizes tones or loads files. Limits are one million events, 512 distinct samples, 64 MiB decoded samples per selection, 64 simultaneous voices including releases, and 24 hours at the selected BPM. Exceeding them reports an error rather than dropping notes.

`audio.playback_enabled` defaults to false. The obsolete `track_enabled` setting is rejected with a diagnostic naming its replacement. Playback suppresses the metronome without changing its stored toggle. Authored tempo data remains in the musical document but is not printed; menu BPM is constant throughout a session. Selecting a part starts a new session. Manual page changes never restart or seek. Natural completion, including trailing silence/release, returns to the corresponding part menu with its selection preserved. Leaving the score cancels playback; stale results cannot close a newer score. Errors remain visible rather than being mistaken for completion.

`--once` additionally validates referenced audio without opening display/audio hardware. Runtime packages contain WAVs alongside pages and the SQLite catalog; the source XML and Python dependencies remain build-only.

### Authored Repertoire and Learned Solos

The production decision is explicit:

1. Look up the canonical source music and solo slot in the training manifest. A registered pair uses its authored solo **exactly**, with deterministic guitar placement and normal notation/export. This is library reuse, not a claim that a model reproduced it. An incompatible arrangement window, invalid tie or unplayable note fails rather than clipping, transposing or replacing it. Authored chromatic/color notes are preserved; generated-harmony checks are diagnostic for authored music.
2. For an unregistered pair, load the frozen pooled model and call `solo.composer.compose → decoder.Decoder → placement → validate → notation/export`. This path does not read authored notes or train during an ordinary build. `generate_role(..., settings={'predict': True})` explicitly exercises prediction even for a registered song.

The small conditional model backs off from up to four recent joint pitch/rhythm events through source-melody features, current/future chord quality, metrical phase and global evidence. Pitch relationships are relative; there are no song-name conditions or per-song prediction tables. All examples contribute to the same distribution. Source intervals, sounding state and chord-relative source pitches influence prediction. Register comes from pooled training evidence, not an acoustic/electric octave formula.

Prediction uses constrained autoregressive sampling with a stable seed derived from canonical musical source content, slot/role and relevant configuration. The two slots draw from the same learned distribution; the seed permits different interpretations without teaching separate instrument styles. It scores valid joint continuations by their learned likelihood, then draws deterministically from the normalized probabilities. This avoids the mostly-silent maximum-likelihood mode observed during testing. At most four deterministic attempts are allowed if a sample cannot finish or contains no sounding notes; this is not a beam search or a claim of global phrase optimization.

Binary durations and complete triplet groups remain exact. Contiguous silence is one event, matching training tokenization; the decoder cannot repeatedly concatenate rest tokens to invent a long silent passage. Selected holds survive barline notation splits. Generic chromatic approaches/enclosures include their exact target as complete validated actions. Their event score is identical to emitting the constituent events separately, and equivalent actions are deduplicated before sampling. The report's negative log likelihood is diagnostic, not a ranking of complete solos or a model of listener preference.

Fixed scale maps no longer constrain prediction. Brief ordinary movement uses the source key and written chord tones; chord quality, alterations, omissions and slash-bass semantics remain authoritative. Long nonchord events require a supported color or resolution contract. Approaches resolve to their specified pitch and octave; anticipation validates the upcoming harmony. Missing harmony does not authorize arbitrary pitches. Search failure is explicit and never invokes the old composer. General harmonic validators support more contracts than this first decoder actively proposes; no claim is made that every validator has a learned production construction path.

The model has only four examples from two songs. Its four-event context does not establish long-range thematic memory, nor does training prove transfer to another standard. Phrase report boundaries describe audible breaths after selection; they are not another composer or an assertion of phrase-wide theme planning. This initial implementation can still repeat local patterns or choose weak continuations on unfamiliar harmony. Exact authored playback and novel model output are reported separately.

Train explicitly:

```sh
./modulus.py python -m generator.train --leave-one-song-out
```

Register an additional pair and retrain without changing Python:

```sh
./modulus.py python -m generator.train --add-source /path/to/new-song.xml --add-solo /path/to/new-song-solo1.xml --slot 1
./modulus.py python -m generator.train --add-source data/songs/new-song.xml --add-solo /path/to/new-song-solo2.xml --slot 2
```

Files are copied into `data/songs` and `data/training` without overwriting different contents. Validation checks the exact measure/harmony correspondence, guitar-written convention and fret range before publishing the manifest. An existing source without a registered slot uses model prediction for that slot. Use `--pitch-convention encoded` only for files whose own transposition describes sounding pitch. Updating existing examples means editing their XML/manifest deliberately and rerunning training. The report under `build/training/report.json` records teacher-forced likelihood and optional leave-one-song-out likelihood; neither is a listening assessment or unassisted-generation accuracy.

Production input traces distinguish authored reuse from learned prediction. Caches include source, role, relevant data and code; authored cache keys include the selected XML's content and convention. Build and package inputs now include the model, manifest and authored repertoire. Changes to another solo do not change an already registered part's notes. Updating the pooled model can legitimately change predictions for any unregistered part. Build-time composition defaults to two independent processes; rendering and runtime remain unchanged.

The generated `solo_xml/*-solo1.xml` and `*-solo2.xml` exports use local measure numbers, preserve pickups, and store whole-arrangement positions in `manifest.solo`. Exact reuse means complete sounding pitches, onsets, releases, repeated attacks and silence agree. XML whitespace, note splitting, measure labels and exporter-specific annotation containers need not be identical. Optional `python -m tests.solo_compare` compares these completed exports; it never feeds a production decision. Training and authored reuse are the explicitly permitted dependencies on the supplied music.

Migration inventory:

| Kept or added | Consumer and reason |
| --- | --- |
| `data/songs/` (two songs) | Normal source importer and immutable arrangement analysis. |
| `data/training/` (four unchanged XMLs + manifest) | Explicit trainer and registered authored playback. All slots are pooled for learning. |
| `data/models/solo.json` | Frozen prediction model; reproducible by `generator.train`. |
| `data/composition/{roles,harmony,lead_ins}.json` | Physical/difficulty settings and generic harmonic validity. Fixed-map permissions removed. |
| `data/composition/accompaniment.json` | Live sparse comping figures, migrated from the former mixed rhythm file. |
| `generator/harmonic_time.py` | Exact chord/key spans retained from the removed scale subsystem. |
| Solo event, expression, placement, notation and export helpers | Retained physical/timing safeguards shared by authored and predicted music. |

Removed: `data/{scales,sequences,rhythms}`, old `composition/solo.json` and mixed `rhythms.json`, catalog import/cache code, old solo construction/phrase/selection/scoring/sequence modules, unused accompaniment catalog planner, three evaluation-song excerpts, and tests specific to those retired algorithms. Shared importer, timing, guitar, playback, engraving, arrangement and independence tests remain, with new pooled-learning, registration and exact-repertoire tests. No `docs/` or `audition/` directory is required. Disposable prior comparison outputs are replaced by one current comparison under `build/solo_comparison`.

Verification on October 5, 2026: `unittest discover -s tests -p 'test_*.py'` passed 347 tests in 112.99 seconds. The normal six-part asset build passed `--check`, including playback, engraving, TAB and catalog validation. On an Intel Core i7-1355U, a build with an empty composition cache, warm engraving cache and two composition workers took 24.31 seconds; performance-part generation occupied 5.65 seconds. Each registered solo took 0.59–0.68 seconds. A subsequent CMake build reused all role and engraving results and finished in approximately nine seconds. These measurements describe registered playback, not novel prediction. Detailed timings are in `build/training/build-profile.json`; isolated, cache-free prediction timings and input fingerprints are in `build/training/prediction-check.json`.

The optional exact comparison confirmed all four exports: 137/141 attacks for All of Me and 121/115 for Autumn Leaves, with no pitch, onset, release or rest differences. This verifies authored reuse rather than learned reproduction. The commands were the ordinary asset generator with `--export-solo-xml --check`, `cmake --build build/build_x86_debug --target vamp_generated_assets`, and `python -m tests.solo_compare` with four explicit `--reference` pairs, `--reference-pitch guitar-written` and `--require-exact`. No ear-based audition or claim of transfer quality was made.

Forced prediction, without reading the training XMLs, completed all four source/role combinations in 21.81–27.16 seconds each with one worker and no concrete-result cache. These were composition-only runs on the training-source songs, not held-out transfer tests. Their notes intentionally differ from registered playback and were not published as replacements. Training the pooled 2,860-context model took 0.89 seconds. Prediction remains the slower path, chiefly candidate harmonic validation and event-likelihood evaluation.

### Engraving Boundary

`generator/engraving/` is a custom Python engraver, not a wrapper around MuseScore or another notation engine. `generator.engraving.layout.layout_score(document, style, fonts)` produces immutable layout pages with exact source intervals, notation anchors and drawing primitives. `generator.engraving.raster.rasterize_page(page, fonts)` draws one page at four-times resolution and downsamples it. `generator.engraving.pipeline.engrave_tree(root, fonts)` is the asset-pipeline entry point. `fonts.Fonts`, `types.Style`, and `types.EngravingError` live in their respective engraving modules; page constants and `check_page` live in `validation`. Package initializers do not re-export these APIs. `generator/musicxml_writer.py` remains a round-trip/export utility and is not involved in engraving.

Engraving supports a bounded process pool (`VAMP_ENGRAVING_JOBS`, default 2) and a build-only content-addressed page cache (`VAMP_ENGRAVING_CACHE_DIR`, default `build/build_common/engraving_cache`). Keys include the complete musical document, engraving implementation, fonts/metadata, style, Python and rendering dependency versions. Playback is deliberately excluded from page keys and is always rebuilt independently. Cache hits are copied into the current staged output and checked for checksums, PNG decoding, dimensions, blank pages and clipped ink. Missing or corrupt entries are regenerated. All semantic and asset validation, database construction and transactional publication remain in place. The cache is not installed and can be deleted at any time when builds are idle. CI saves it after the native build and reuses it for ARM packaging; generation prints stage timing and per-worker progress instead of remaining silent during engraving.

Reuse has two levels. A full-score hit restores complete page images for the same musical document and metadata. A notation-body group shares system layout and body pixels among documents with identical staff content, while retaining each document's title, subtitle, pagination, footer, and complete semantic bundle. Body keys include the rendering context and actual page-band geometry; incompatible placement cannot reuse those pixels. Each header/footer is composed separately, and pages whose drawing primitives cannot be separated safely use full-page rasterization. This reuse never merges score IDs, catalog records, or authored music.

`notation.py` interprets staff state, exact pitch positions and shared accidental state; `notes.py` and `attributes.py` construct symbols; `spans.py` handles beams, ties, tuplets, octave shifts and lyric continuations; `layout.py` handles spacing and pagination. `fonts.py` reads bundled OTF outlines and SMuFL metadata, caching metrics and outlines for one generation run. `raster.py` uses FontTools, aggdraw and Pillow only as low-level font and drawing libraries. Modulus owns the pinned optional `assets` dependency group. Configure prepares its build-only environment; CMake and `./modulus.py python` use the exported runner to import shared build utilities without shadowing the Modulus package. Standard glyph names and their upstream notice are vendored alongside the fonts; no system music fonts are consulted.

Voices retain separate stems and collision offsets; chord tones retain their own written durations and per-tone ties. Pitched and percussion material use their own staff positions. Unknown exact microtonal alterations use a rational semitone annotation rather than a rounded accidental; an unavailable explicit SMuFL glyph is an error. The broader model/engraver supports this notation, while automatic guitar generation enforces the source contracts above.

Generated guitar tones carry `tablature: {string, fret}`; their staff is marked `tablature: true`. `engraving/tablature.py` draws a six-line TAB view below standard notation, sharing onset columns, barlines, and page breaks. Both views belong to one system and never duplicate the semantic timeline or playback events. Standard notation carries rhythm, rests, and tuplets; TAB carries positions, solo slashes, and tied continuations without rest glyphs. Strings and frets are validated against sounding pitch, and the layout audit checks each visible TAB numeral's position and alignment.

Every part uses two music systems per page; only an odd final system occupies a row alone. A compact title/composer/part header reserves the same height on every page, wrapping credits if necessary. Pagination fits the two systems using their individual heights and distributes extra whitespace around them while retaining minimum system and header/footer clearances. One uniform scale applies throughout the entire part, reducing the ordinary notation size only when required by the tallest pair. Notation, TAB, text, and anchors scale together without changing musical content. Guitar layout starts with 10-pixel notation spacing, 10.5-pixel TAB spacing, 12-pixel fret labels, and a 14-pixel minimum gap between systems. The unscaled TAB spacing accommodates Petaluma numerals up to 9.75 pixels tall. Tempo markings consume neither ink nor spacing; their semantic data is preserved. Existing `.pages/*.png` preview files are the exact packaged page textures.

Line breaks occur only at measure boundaries. Each line fits as many complete measures as readable spacing allows; it never fills remaining space with a partial measure. A complete measure exceeding the available width or height fails with source and measure context instead of being split or squeezed to fit. Authored ties, beams, tuplets, and lyric spans can continue across whole-measure line breaks. Column measurement and fit trials reuse a score index and bounded column cache. Page placement then re-spaces columns for the final height-fitting scale, keeping every notation/TAB staff at the same physical left and right margins without stretching glyphs. Non-final lines are justified; a short final line retains natural note spacing while staff lines extend to the same right margin. Layout audits enforce these margins as well as musical coverage.

`audit.py` checks the final layout against the musical model: timeline and fragment coverage, exact fragment durations, actual glyph bounds, pitch positions, visible note primitives, required accidentals, ties, lyrics, harmonies and spans. It enforces consistent notation size throughout a part and header/footer clearance, and rejects out-of-page geometry and collisions among unrelated noteheads, accidentals and same-lane text. Staff, ledger, stem, beam and tie contacts are intentional. This is not a proof of publication-quality engraving for every possible MusicXML document; notation changes need targeted semantic, geometry and reviewed image tests. The writer's XML round-trip audit is retained separately in the test suite.

Bundles preserve the complete `music` object. Presentation identifies the `Vamp Python` engraver; its style fingerprint includes the style settings, both font files, font-specific metadata and the glyph-name mapping. Scores, playback manifests, and the catalog have no numbered schema labels. Required fields, types, relationships, dimensions, and checksums determine whether generated data is usable. Incompatible data is rejected, never migrated or guessed.

Pages are 976x484 pixels with 10-pixel guitar or 12-pixel vocal staff spacing, 28-pixel horizontal margins and 18-pixel vertical margins. The two acoustic rest lanes sit closer to the staff, with collision offsets preserving their separation. Dense measures continue across systems/pages; sustained notes become exact, tied presentation fragments and rests become exact rest fragments. These fragments never change the semantic JSON. Irreducible simultaneous content that cannot fit a fixed page fails contextually rather than being clipped or shrunk. Blank pages, incorrect dimensions, and ink reaching the page edge are rejected. Runtime letterboxing preserves aspect ratio. Luma's `load_texture_uncached()` lets Product Vamp retain at most two textures without a hidden strong asset-cache reference. The UI reports missing/corrupt assets instead of falling back to the removed renderer. Runtime validates bundle metadata, paths, dimensions, and page checksums; the build validates musical semantics.

### Validation And Publication

Generation validates sources and complete staged output before replacing the previous asset directory. Replacement is always complete; deleted sources cannot leave stale JSON, pages, audio, or Pico title assets. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalog. Runtime packages contain `catalog.db` with embedded semantic JSON and title-image identifiers, PNG pages, generated PCM WAVs, Pico title images, and the OTF UI text font, not MusicXML, MuseScore, or Python dependencies. Python test discovery runs model/import/export tests; the normal asset build additionally qualifies and audits every catalog score.

The page cache uses `<renderer fingerprint>/fullscore/<score fingerprint>/` and `<renderer fingerprint>/notation/<band fingerprint>/` directories, without a manual cache revision. `./modulus.py python -m generator --renderer-fingerprint` reports the same identity used by rendering and CI: source bytes, fonts, visual style, Python implementation/version, and rendering-library versions. Python compiler/build timestamps are excluded. CI keys append all generator and active XML inputs, including rhythms, and restore earlier snapshots only within the same renderer directory; only that directory is saved after a successful build.

`cold namespace` means that renderer's persistent directory does not yet exist. A renderer or environment change starts a new namespace; deleting a namespace has the same effect. It does not disable reuse within the current run: shared notation is still laid out once per group and its body pixels can be reused as other titles are composed, even without a persistent cache. Conversely, a warm directory does not guarantee that a particular score is cached. Progress reports full-score hits, body hits, newly rasterized body bands, composed pages, missing/rejected entries, and phase timings. Worker phase timings are summed and may exceed wall time. Obsolete namespaces are not migrated and can be removed when builds are idle.

Diagnostics identify the file, part, measure, and unsupported element or invalid relationship. Examples include `part P1, measure 4: unsupported musical element <slur>`, `tie stop has no contiguous matching start`, and `optional-chord marker must identify exactly one harmony`.

The six compact source files were replaced, not retained as a second authoring format. Installation replaces the entire catalog, rather than retaining or migrating rows from older releases. Legacy data must be authored as supported MusicXML and regenerated; the runtime never upgrades old databases.

References: [MusicXML durations](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/duration/), [tuplets](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/tuplet/), [degree alterations](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/degree-alter/), and [SMuFL accidentals](https://smufl.formats.music/latest/tables/standard-accidentals-12-edo.html).

Generator checks compare the generated performance summary against `tests/fixtures/performance_summary.json`. The summary stores compatibility invariants such as section layout, chord-label placement, absence of uncomposed solo slashes, and vocal timing without committing the generated score files themselves. Refresh that file only when an intentional arrangement-logic change should become the new baseline.

## Pico Runtime

The Pico firmware uses the Pico SDK C++ stack and the `pico2_w` board definition. It initializes the ST7789T3 display over SPI, reads the CST816D touch controller over I2C, joins the Zero AP, polls `/api/remote/display`, and sends HTTP POST commands for gestures.

The display is rotated 90 degrees clockwise into a 320x240 landscape UI. The top status bar uses the shared palette:

- Blue: connecting.
- Yellow: retrying or remote request failed.
- Green: connected to the Zero API.

The body shows the focused menu item, song title, or setting and its current value. In score view it shows the active title, part, and page count. Both devices use a light, score-like palette. Touch controls are tap for next, double tap for previous, and tap-and-hold for select. All visible text uses Petaluma Script, matching score-page text and the Zero UI.

`generator.pico_artwork` shapes text on the build host with the bundled `PetalumaScript.otf` and Pillow's Raqm support. Fixed menu, part, connection/status, and settings labels, digits, and `/` become four-bit coverage sprites. The Pico CMake rule writes `generated/pico/static.p4` and embeds it through `generated/pico/artwork.S`. The Zero does not link this sprite atlas; Luma renders its dynamic UI text from the packaged OTF.

Score generation renders titles as 296x64 four-bit coverage images under `pico/titles/<sha256>.mask`. Each file is 9,480 bytes: an eight-byte `VPTI` header with little-endian width/height, followed by packed coverage. Identical image bytes share an asset without merging score identities. The generated `presentation.title_image` identifier is also stored in catalog metadata; no separate lookup manifest is required. These assets are generated and validated before staged output is published. Remote status includes `title_image=<sha256>`; `POST /api/remote/title` with `{"id":"<sha256>"}` serves validated bytes only for catalog-registered titles. Pico fetches a changed title, validates its fixed format and bounds, rejects stale responses, and retains the selected image. Part labels and page counts use fixed sprites without reshaping or refetching title text. Commands take priority over image requests, and failed title requests display `UNAVAILABLE` until a retry succeeds.

Pico performs coverage blending and sprite placement only: no OTF parsing, font shaping, glyph rasterization, or general text renderer runs in firmware. Missing glyphs and invalid source text are build errors. Font or generator changes rebuild both static firmware sprites and title artwork. Direct generation uses `./modulus.py python -m generator.pico_artwork --static-output <path>` for firmware sprites and `./modulus.py python -m generator.pico_artwork --assets <generated_assets>` for title assets. Python, Pillow/Raqm, and FontTools remain build-only dependencies.

The root `CMakeLists.txt` is also the Pico firmware entrypoint when configured with `-DVAMP_PICO_FIRMWARE_PROJECT=ON`. That mode returns before the Zero/Linux targets are declared, so the Pico build can use the embedded ARM toolchain cleanly.

Build defaults can be overridden at package time with CMake cache variables:

- `VAMP_PICO_WIFI_SSID`
- `VAMP_PICO_WIFI_PASSPHRASE`
- `VAMP_ZERO_HOST`
- `VAMP_ZERO_PORT`

## Packaging

Modulus packages the CMake install stage. Product Vamp uses `install_prefix` `.` so the registry artifact is a top-level multi-device bundle rather than a root filesystem overlay.

The ARM64 package target builds:

- `zero/product_vamp_zero_linux_arm64.docker.tar`
- `pico/product_vamp_pico2w.uf2`
- `manifest.json`

Building the full package requires Docker/buildx, the ARM embedded GCC toolchain, and the Pico SDK. CI passes `PICO_SDK_PATH` directly. The generated Modulus Docker image also carries the pinned Pico SDK at `/opt/pico-sdk` for local package builds. The Zero Docker runtime targets Debian trixie to match current Raspberry Pi OS and the ARM64 dependency binaries, and includes `util-linux` for the `setsid --wait --ctty` launch path.

Generated score assets are created during CMake build/package and copied into the Zero image context. Static font assets remain tracked under `package/assets/fonts`.

`generator/catalog.py` uses Python's standard-library SQLite module to construct `catalog.db` after musical, playback, and engraving validation. The database carries application ID `0x56414D50`, complete score bundles, and menu metadata. Runtime validates both required strict tables, their column names/types, non-null requirements, and primary keys directly. Sorted insertion and the absence of wall-clock timestamps make repeated builds reproducible with the same toolchain. It is closed and integrity-checked before publication, without journal/WAL sidecars. Individual JSON files remain available in the build tree for inspection, but both CMake installation and Docker image assembly exclude them.

The default `database.path` is `/usr/share/product_vamp/catalog.db`; development configuration points at the native generated_assets database. Stored source paths are logical, asset-relative locations, not build-host filenames. Runtime resolves them against `runtime.assets_path` and rejects traversal and escaping symlinks. The JSON file itself need not exist: its parent identifies the score's page directory. Normal startup checks catalog compatibility; `--once` additionally runs SQLite's integrity probe and checks all page/audio files. Missing, corrupt, or incompatible catalogs fail rather than creating an empty library. `--ensure-database` is no longer a supported command.

Every run of `zero/install_zero.sh` is a clean installation. It stages configuration/scripts, verifies the manifest's Docker tag against the incoming archive's image digest, and probes that image before stopping Product Vamp. It then clears the owned `/etc/product_vamp`, `/var/lib/product_vamp`, `/run/product_vamp`, and obsolete host `/usr/share/product_vamp` trees, including old SQLite sidecars. The installed container uses assets from its image and no longer mounts a writable state database. Startup reads the verified image ID from `/etc/product_vamp/image_id`, rather than depending on a mutable Docker tag. The installer also removes the fixed legacy hyphenated Product Vamp paths, units, containers, and network profile after preflight, while rejecting symlinked destinations. It replaces its scripts and systemd units, removes their old overrides, recreates only the `product_vamp_control` NetworkManager profile, and requires a successful service health check before announcing success.

Configuration, AP credentials, and all prior catalog data are intentionally reset to bundle defaults. There is no automatic backup or migration. Installation through the control AP can disconnect the session; use the local console or independent networking. Preflight failure leaves existing application files untouched and restores the previous image tag; replacement failure stops Product Vamp and reports the error. Rerunning the installer is the recovery path. Ordinary boots/restarts do not run cleanup. Unrelated networks, OS data, containers, and system logs remain untouched. Cleanup rejects symlink/mount targets, never follows paths from old configuration, and removes only identified obsolete Product Vamp images not used by another container.

CI builds stable target-specific archives with `python3 modulus.py package`, then `package/scripts/publish_latest_generic_package.py` uploads exactly the configured targets and their checksums. Gitea requires a registry slot, so publishing replaces `latest` and removes obsolete remote entries. There is no application release number, artifact version argument, or versioned install directory. The manifest, heartbeat, status endpoint, and `--build-info` retain the Git commit for diagnostics. Conan references, compiler/dependency pins, and MusicXML protocol identifiers remain external tool requirements.

## Shared Protocol

The Pico firmware does not link Zero-side Linux application libraries such as Ligo, Luma, or Noto. Those remain Zero-side dependencies: Ligo serves the HTTP API, Luma renders the Zero display, and Noto owns SQLite persistence. Cross-device code should stay in `vamp_protocol` and `vamp_style`: plain C++17 with no OS, allocator-heavy, or Zero-only dependencies.

Luma owns the application shell, UI control tree, texture loading, and UI-thread texture lifetime. Product Vamp's page cache uses the uncached loading API; there is no score-specific Raylib font cache or handwritten notation renderer.

## LSP

`tooling.editor.clangd` in `modulus.json` generates the ignored `.clangd`, pointing native Zero and shared protocol code at `build/build_x86_debug`, and Pico code at `build/build_pico_debug`. Refresh both compile databases with:

```sh
python3 modulus.py lsp
```

The `tooling.lsp` section in `modulus.json` owns the Pico CMake configure. It unsets host compiler variables before configuring the Pico SDK project so clangd sees the embedded ARM toolchain. When running directly outside the generated Modulus Docker image, set `PICO_SDK_PATH` before refreshing the Pico database.