product-vamp/TECHNICAL_GUIDE.md

# Product Vamp Technical Guide

## Repository Shape

Product Vamp follows the Modulus product layout used by the khodo repos:

- `include/vamp_zero` and `src/vamp_zero`: Pi Zero C++23 application.
- `include/vamp_pico` and `src/vamp_pico`: Pi Pico 2 W C++17 firmware.
- `include/vamp_protocol` and `src/vamp_protocol`: shared route and page-command contracts used by both devices.
- `cmake/pico.cmake` and `cmake/zero.cmake`: device-specific build definitions selected by the root `CMakeLists.txt`.
- `include/vamp_style.hpp`, `include/vamp_style/font.hpp`, and `src/vamp_style.cpp`: shared color palette and Petaluma font interfaces. Bitmap font data is generated in the build directory.
- `data`: authored MusicXML for performance songs and practice exercises, plus the generator regression summary.
- `generator`: build-time Python preprocessing for performance arrangements and practice variants.
- `package`: generic bundle inputs, scripts, config, static font assets, and device install files.

## Zero Runtime

The Zero process can run headless for local/API quick checks or with Luma for the Waveshare Zero-DISP-7A display. The API is plain HTTP on the private AP:

- `GET /api/status`
- `GET /api/remote/display`
- `POST /api/remote/heartbeat`
- `POST /api/remote/next`
- `POST /api/remote/previous`
- `POST /api/remote/select`

Legacy `/api/page/next` and `/api/page/previous` routes are still accepted for simple manual checks.

The AP setup pins NetworkManager to 2.4 GHz channel 6 with WPA2/RSN and CCMP ciphers so the Pico W can join using `CYW43_AUTH_WPA2_AES_PSK`.

On Raspberry Pi OS Lite, the installed display service owns `tty1`, conflicts with `getty@tty1.service`, and starts the Zero container through `product-vamp-start-zero`. The wrapper switches to the display VT when `chvt` is available and runs `vamp-zero` inside the container through `setsid --wait --ctty` with `/dev/tty1` as stdin, giving raylib's DRM backend a real virtual terminal while keeping Docker attached to the long-running display process. SSH remains the normal admin path and `tty2` is enabled as the fallback local console.

At startup the Zero scans `share/product-vamp/songs/*.json` and `share/product-vamp/practice/<type>/<lesson_group>/<exercise>/<key>.json`, ensures the current SQLite schema with Noto, imports both catalogs, then reloads runtime menu state from the database. Runtime navigation and rendering use that loaded Noto-backed state instead of rescanning asset folders. The UI supports a top-level mode menu, Performance song/part/score menus, and Practice type/lesson/exercise/key/score menus. Melody practice offers key selection by choosing a pre-generated score variant; chord practice skips key selection and renders the authored score key directly. The score renderer draws title/composer/part metadata, systems, measures, clef/key/time preambles, enlarged chord symbols and section labels, Petaluma SMuFL noteheads/rests/flags/dots/accidentals, simple vector stems/ties, and lyrics. Slash notation is represented explicitly with `notehead: "slash"` in generated JSON.
Petaluma music symbols are placed from SMuFL bounding boxes and stem/flag anchors instead of rough font-size offsets, which keeps clefs, noteheads, rests, dots, accidentals, and eighth-note flags aligned to staff-space coordinates. Printed accidentals are derived from the active key signature and per-measure accidental state, so natural signs are shown only when they are musically required.

## Score Generation

The build runs `python3 -m generator` against `data/` and writes build-only assets under the CMake binary tree. Performance source songs become acoustic, electric, and vocal arrangements. Melody practice sources are emitted in all 12 keys at build time. Chord practice sources are emitted once in their authored key. Generated JSON keeps notation metadata while removing generator-only optional chord markers; guitar playability logic is used only inside the arrangement generator.

The source pipeline is MuseScore -> uncompressed `.musicxml` -> `generator/musicxml.py` -> arranger/transposer -> schema-v2 JSON. Export MusicXML 3.1 or 4.0, `score-partwise`, with one part, one staff, and one rhythmic voice. Chord tones share an onset but retain their own spelling, written duration, and tie links. Separate voices/staves must not be flattened into the source voice. The schema keeps staff/voice ownership explicit for a future polyphonic renderer.

### MusicXML Contract

| Area | Accepted notation |
| --- | --- |
| Pitch | Twelve-tone letter/alteration/octave spelling, including double/triple accidentals and courtesy accidentals. B-sharp and C-flat retain the correct octave. |
| Rhythm | All fourteen MusicXML types, maxima through 1024th; up to sixteen augmentation dots; exact positive rational metric durations and separately stored written values. Time modifications must reconcile the two. |
| Chords and rests | Simultaneous chord tones, shorter additional chord tones, rests, full-measure rests, hidden events, forward spacers, and rhythm slashes. |
| Ties and tuplets | Per-pitch ties, including partial chord ties; nested and cross-bar tuplet spans. Nested tuplets require explicit `tuplet-actual` and `tuplet-normal` numbers. |
| Small notes | Grace notes with zero metric duration, cue notes, and measured single/paired tremolos. |
| Staff state | G/F/C clefs on lines 1-5, octave clefs and octave-shift spans; standard major/minor/modal keys or custom seven-letter alterations; key, clef, and meter changes. |
| Meter | Simple, compound, additive, mixed-denominator, and unmetered timelines. A partial measure must be marked `implicit="yes"`; an initial implicit measure is the arrangement pickup. Leading rests are retained. |
| Harmony | Structured roots, kinds, inversions, slash basses, and degree additions/alterations/subtractions. Harmony events have their own offsets, including inside held notes. Their positions never split or shorten notes. |
| Text | Title, composer, part name, one lyric verse with syllabification and extenders, rehearsal labels at measure starts, and Vamp directives. |

Microtones, independent voices, multiple staves/parts, percussion/tab staves, repeats/navigation, slurs, dynamics, articulations, and expressive ornaments are rejected. Unknown musical elements also fail, rather than disappearing. Custom key-octave placement is not supported. Resource limits are explicit: serialized times use reduced signed-64-bit numerators/positive denominators; pitches use octaves -4 through 16 and alterations -3 through 3; clef/octave shifts span at most three octaves; meter counts are 1-1024 with denominator units 1-4096. These limits produce diagnostics, not clamping or truncation.

Layout/exporter information such as page defaults, credits, print layout, font/color/position attributes, and source pagination does not determine the small-screen layout. Engraving is regenerated. Beam groups and written note values are retained; unrelated engraving metadata does not authorize ignoring musical descendants. `defusedxml==0.7.1` forbids XML entities and external access. Ordinary MusicXML DOCTYPE declarations are allowed without fetching their DTDs.

### Vamp Metadata

In MuseScore's score properties, set `vamp:style` to a supported player style. It exports as standard MusicXML miscellaneous metadata:

```xml
<miscellaneous>
  <miscellaneous-field name="vamp:style">Medium Swing</miscellaneous-field>
</miscellaneous>
```

The default is Medium Swing; `Med.`, `Med. Swing`, and `Medium` remain aliases. A keyless melody exercise needs `vamp:tonic` (for example, `C`) as its transposition reference. Modes and local key changes are preserved when generating practice variants.

Place staff text `vamp:optional-chord` at the same onset as exactly one harmony to mark a turnaround omitted from the final arrangement section. The text may be hidden. Parenthesizing an ordinary chord does not make it optional. Orphaned/duplicate markers and unknown `vamp:` properties are errors; consumed directives never enter runtime JSON.

### Runtime Schema

`schema_version: 2` is mandatory. `metadata` and one `staves[]` entry contain measures and an explicit voice. Measure `duration`, event `onset`/`duration`, harmony offsets, and span endpoints use `{ "numerator": N, "denominator": D }` in quarter-note units. Fractions are reduced; there is no fixed tick grid and no remainder distribution. The native loader uses exact arithmetic for validation, converting only layout coordinates to floating point.

Events have `kind`, `written: { "type", "dots", "ratio"? }`, and independently identified pitches. Each pitch carries integer `alter`, `octave`, its own `duration` and `written`, and optional reciprocal `tie_from`/`tie_to` pitch IDs. Staff-level tuplets reference event IDs; octave spans reference absolute musical time. Attributes and harmonies are timestamped independently. The normalized harmony degree alteration is relative to the natural diatonic interval; the importer converts MusicXML's kind-relative alterations and dominant-relative additions.

The arranger normalizes transposing instruments to concert pitch before generating parts. Guitar parts explicitly use written pitches one octave above sounding pitch and a -1 octave instrument transposition. Chord pitch definitions are separate from playable voicing choices; voicings may omit a natural fifth or lower unaltered extensions, but never change a chord kind or replace an unsupported chord with an unrelated major chord. Unplayable combinations produce a labelled rhythmic slash and a build warning.

The renderer uses written values for glyphs, meter-aware/multilevel beams, per-pitch ties, timestamped harmony, and OTF-derived glyph bounds. Content-aware systems split dense measures into continuations. A canonical 976x484 layout makes page navigation stable across viewport sizes; vertical extent affects pagination instead of clamping pitches. The active page geometry is cached between redraws.

### Validation And Migration

Generation validates sources and complete staged output before replacing the previous asset directory. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalogue. Runtime packages contain JSON and OTF fonts, not MusicXML or Python importer dependencies. Python test discovery runs all generator/importer tests; `tests/fixtures/notation.musicxml` exercises the broader renderer contract. The optional `vamp-render-score` test target captures a page using a desktop-capable Raylib build.

Diagnostics identify the file, part, measure, and unsupported element or invalid relationship. Examples include `part P1, measure 4: unsupported musical element <backup>`, `tie stop has no contiguous matching start`, and `optional-chord marker must identify exactly one harmony`.

The six compact source files were replaced, not retained as a second authoring format. Existing catalogue rows refresh from regenerated JSON at startup. Custom database rows (`catalog_managed = 0`) are retained, including ID conflicts, and are never automatically deleted to resolve a schema mismatch. Custom legacy JSON must be reimported as schema v2; the loader reports a regeneration error instead of interpreting old ticks heuristically.

References: [MusicXML durations](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/duration/), [tuplets](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/tuplet/), [degree alterations](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/degree-alter/), and [SMuFL accidentals](https://smufl.formats.music/latest/tables/standard-accidentals-12-edo.html).

Generator checks compare the generated performance summary against `data/expected/performance_summary.json`. The summary stores compatibility invariants such as section layout, chord-label placement, slash-section placement, and vocal timing without committing the generated score files themselves. Refresh that file only when an intentional arrangement-logic change should become the new baseline.

## Pico Runtime

The Pico firmware uses the Pico SDK C++ stack and the `pico2_w` board definition. It initializes the ST7789T3 display over SPI, reads the CST816D touch controller over I2C, joins the Zero AP, polls `/api/remote/display`, and sends HTTP POST commands for gestures.

The display is rotated 90 degrees clockwise into a 320x240 landscape UI. The top status bar uses the shared palette:

- Blue: connecting.
- Yellow: retrying or remote request failed.
- Green: connected to the Zero API.

The body stays blank until a performance or practice score is active, then it shows the active title, selected part or active practice key, and page count. Touch controls are tap for next, double tap for previous, and tap-and-hold for select.
The Pico text renderer uses an antialiased bitmap atlas generated from `package/assets/fonts/petaluma/PetalumaScript.otf`. The shared font CMake rule generates `generated/vamp_style/font.cpp` separately in each native/Pico build directory, using `tools/generate_pico_font.py --output <path>`. Font and generator changes trigger regeneration. The generated atlas includes ASCII kerning pairs so display text such as `VAMP` uses the same spacing as the source font. The pinned Pillow build dependency is declared in `tools/requirements.txt`.

The root `CMakeLists.txt` is also the Pico firmware entrypoint when configured with `-DVAMP_PICO_FIRMWARE_PROJECT=ON`. That mode returns before the Zero/Linux targets are declared, so the Pico build can use the embedded ARM toolchain cleanly.

Build defaults can be overridden at package time with CMake cache variables:

- `VAMP_PICO_WIFI_SSID`
- `VAMP_PICO_WIFI_PASSPHRASE`
- `VAMP_ZERO_HOST`
- `VAMP_ZERO_PORT`

## Packaging

Modulus packages the CMake install stage. Product Vamp uses `install_prefix` `.` so the registry artifact is a top-level multi-device bundle rather than a root filesystem overlay.

The ARM64 package target builds:

- `zero/product-vamp-zero-linux-arm64.docker.tar`
- `pico/product-vamp-pico2w.uf2`
- `manifest.json`

Building the full package requires Docker/buildx, the ARM embedded GCC toolchain, and the Pico SDK. CI passes `PICO_SDK_PATH` directly. The generated Modulus Docker image also carries the pinned Pico SDK at `/opt/pico-sdk` for local package builds. The Zero Docker runtime targets Debian trixie to match current Raspberry Pi OS and the ARM64 dependency binaries, and includes `util-linux` for the `setsid --wait --ctty` launch path.

Generated score assets are created during CMake build/package and copied into the Zero image context. Static font assets remain tracked under `package/assets/fonts`.

CI builds the generic artifact with `python3 modulus.py package --version latest`, then `package/scripts/publish_latest_generic_package.py` replaces the Gitea generic package version `latest` and deletes older Product Vamp generic package versions.

## Shared Protocol

The Pico firmware does not link Zero-side Linux application libraries such as Ligo, Luma, or Noto. Those remain Zero-side dependencies: Ligo serves the HTTP API, Luma renders the Zero display, and Noto owns SQLite persistence. Cross-device code should stay in `vamp_protocol` and `vamp_style`: plain C++17 with no OS, allocator-heavy, or Zero-only dependencies.

Luma owns the application shell and UI control tree. The score renderer uses Luma primitives for layout, staff lines, and simple vector strokes, and uses a small score-local Raylib font cache for Petaluma SMuFL glyphs until Luma grows first-class codepoint loading.

## LSP

`tooling.editor.clangd` in `modulus.json` generates the ignored `.clangd`, pointing native Zero and shared protocol code at `build/build_x86_debug`, and Pico code at `build/build_pico_debug`. Refresh both compile databases with:

```sh
python3 modulus.py lsp
```

The `tooling.lsp` section in `modulus.json` owns the Pico CMake configure. It unsets host compiler variables before configuring the Pico SDK project so clangd sees the embedded ARM toolchain. When running directly outside the generated Modulus Docker image, set `PICO_SDK_PATH` before refreshing the Pico database.