Package

TECHNICAL_GUIDE.md

@@ -35,7 +35,7 @@ Page commands use `/api/remote/next` and `/api/remote/previous`; the old `/api/p
 
 The AP setup pins NetworkManager to 2.4 GHz channel 6 with WPA2/RSN and CCMP ciphers so the Pico W can join using `CYW43_AUTH_WPA2_AES_PSK`.
 
-On Raspberry Pi OS Lite, the installed display service owns `tty1`, conflicts with `getty@tty1.service`, and starts the Zero container through `product-vamp-start-zero`. The wrapper switches to the display VT when `chvt` is available and runs `vamp-zero` inside the container through `setsid --wait --ctty` with `/dev/tty1` as stdin, giving raylib's DRM backend a real virtual terminal while keeping Docker attached to the long-running display process. SSH remains the normal admin path and `tty2` is enabled as the fallback local console.
+On Raspberry Pi OS Lite, the installed display service owns `tty1`, conflicts with `getty@tty1.service`, and starts the Zero container through `product_vamp_start_zero`. The wrapper switches to the display VT when `chvt` is available and runs `vamp_zero` inside the container through `setsid --wait --ctty` with `/dev/tty1` as stdin, giving raylib's DRM backend a real virtual terminal while keeping Docker attached to the long-running display process. SSH remains the normal admin path and `tty2` is enabled as the fallback local console.
 
 At startup the Zero validates the packaged `catalog.db` application identifier and required table structure, opens it read-only through Noto, and loads menu metadata. It never creates or migrates a database, scans score JSON, or repairs metadata. The main menu contains Performance and Settings. Settings holds Playback, Metronome, and Tempo; its values apply for the current session. Song -> Part -> Score navigation offers Acoustic, Electric, and Vocal without key selection. Musical interpretation, spacing, beaming, ties, accidentals, and pagination happen on the build host. The Zero displays engraved PNGs and retains at most two page textures.
 
@@ -51,7 +51,7 @@ In score view, Zero and Pico share one gesture state machine: a second release w
 
 ## Score Generation
 
-The build runs `python3 -m generator` against `data/` and writes build-only assets under the CMake binary tree. Every song produces acoustic, electric, and vocal parts in its authored musical key. Source counts are discovered, not fixed in code. Generator-only optional chord markers are consumed before publishing.
+The build runs `./modulus.py python -m generator` against `data/` and writes build-only assets under the CMake binary tree. Every song produces acoustic, electric, and vocal parts in its authored musical key. Source counts are discovered, not fixed in code. Generator-only optional chord markers are consumed before publishing.
 
 Songs live directly in `data/songs/`. Each source produces acoustic, electric, and vocal parts under `songs/`, with matching page directories. Adding, renaming, or removing a source requires no registry edit. Regeneration replaces the output tree and catalog. The two current songs generate six parts.
 
@@ -132,7 +132,7 @@ Generated accompaniment has no ties. Acoustic attacks release at the earliest of
 
 Acoustic accompaniment uses the planned foundation. Electric accompaniment sections in songs use the response plan; song electric melody sections are raised one octave before harmonization. Song sections remain Rhythm Lead-in, Vocal Melody, Guitar Melody, Acoustic Solo, Electric Solo, Vocal Melody, and Ending, with each part retaining its melody, accompaniment, rest, or solo-slash role. Optional turnarounds are removed before planning the final ending. The generated phrase cycles contain explicit events in reading order, not runtime loops or repeat signs.
 
-Each source is parsed, validated, and normalized once for all three parts. The batch generator computes shared accompaniment plans before building those parts. Public `arrange_document` and `generate_performance_scores` calls require a keyword `rhythms` argument holding a `generator.rhythm.Library`; internal part builders receive precomputed plans. Builders treat the prepared source and plans as read-only and own their generated notes and spans. Source-path checks reject colliding song slugs before writing parts: for example, `one-two.xml` and `one_two.xml` would both produce `one-two-acoustic.json` and cannot coexist.
+Each source is parsed, validated, and normalized once for all three parts. The batch generator computes shared accompaniment plans before building those parts. Public `arrange_document` and `generate_performance_scores` calls require a keyword `rhythms` argument holding a `generator.rhythm.Library`; internal part builders receive precomputed plans. Builders treat the prepared source and plans as read-only and own their generated notes and spans. Source-path checks reject colliding song slugs before writing parts: for example, `one-two.xml` and `one_two.xml` would both produce `one_two_acoustic.json` and cannot coexist.
 
 All three parts use octave-transposing notation: written notes sound one octave lower. This includes the vocal part for the singer's range. Unmarked source notation uses that convention by default. Vocal melodies retain their written pitches and octave numbers. Song electric melody sections add twelve semitones to both written and sounding pitch; the guitar's octave-transposing convention still applies. Written C4 sounds C3 (A-string fret 3), and written C#6 sounds C#5, within the 1-20 fret range. Generated accompaniment is notated using the same convention.
 
@@ -164,7 +164,7 @@ There is no finger assignment, barre logic, or predefined chord-shape dictionary
 
 All musical compilation and tone synthesis run in Python (`generator/playback.py`). Each transformed score receives a sorted timeline of independent tone events with rational quarter-note `onset` and `end` positions. Ties become one continuous attack. Rests and spacers remain silence, including trailing silence. Harmony labels are not auto-accompaniment: only pitches actually present in the selected arrangement sound. Cue notes are silent, hidden pitched notes sound, and pitchless rhythm slashes are silent.
 
-The `soft-struck` profile generates shared mono 48 kHz PCM16 WAVs with a short attack/decay and a seamless sustain loop, plus a shared 50 ms release envelope. Exact sounding pitches, including supported microtones, determine samples. Instrument transposition is applied once from the generated musical model, without role- or filename-dependent octave heuristics. Samples are shared across scores and voices by content identity. There are no SoundFonts, bundled instrument samples, oscillators, or MusicXML parsing at runtime.
+The `soft_struck` profile generates shared mono 48 kHz PCM16 WAVs with a short attack/decay and a seamless sustain loop, plus a shared 50 ms release envelope. Exact sounding pitches, including supported microtones, determine samples. Instrument transposition is applied once from the generated musical model, without role- or filename-dependent octave heuristics. Samples are shared across scores and voices by content identity. There are no SoundFonts, bundled instrument samples, oscillators, or MusicXML parsing at runtime.
 
 Grace groups honor explicit steal-time-previous/following or make-time; unspecified groups share 25 percent of the following event. Conflicting grace modes, trailing grace groups, unsupported unpitched sound, tied measured tremolo, and incompatible paired tremolo make playback explicitly unavailable while retaining the notation. Measured tremolo uses written subdivision and tuplet ratios. The tone range is 20-18,000 Hz. Asset validation checks containment, SHA-256, PCM format/length, loop endpoints, envelope shape, event ordering, references, and rational timing. Generation remains transactional if validation fails.
 
@@ -186,17 +186,17 @@ Selection first preserves every required pitch. Recognized ii-V pairs, resolving
 
 Only the soloing part receives compact `scale_spans`, with absolute rational start/end offsets, labels, and matched status. `N.C.` and solo boundaries stop annotations. Accompaniment, melody, pitches, TAB, and playback are unchanged; no solo notes are synthesized. Original melody notes are not constraints on these slash-only solo passages. Unmatched chords print `No exact scale match` and emit a source/measure/beat warning without changing the harmony. Invalid scale data or missing tonal metadata instead fail generation before publication.
 
-Scale labels appear above chord symbols at collection changes and system starts. Detailed parent collections, source and arranged positions, chord-root modes, degrees, chord-tone targets, and selection reasons are written to `generated-assets/analysis/<song>.txt`. These reports are published transactionally with the build output but excluded from both installation and Docker runtime assets. The C++ runtime only displays the generated pages; it does not load the scale library or run harmonic analysis.
+Scale labels appear above chord symbols at collection changes and system starts. Detailed parent collections, source and arranged positions, chord-root modes, degrees, chord-tone targets, and selection reasons are written to `generated_assets/analysis/<song>.txt`. These reports are published transactionally with the build output but excluded from both installation and Docker runtime assets. The C++ runtime only displays the generated pages; it does not load the scale library or run harmonic analysis.
 
 ### Engraving Boundary
 
 `generator/engraving/` is a custom Python engraver, not a wrapper around MuseScore or another notation engine. `generator.engraving.layout.layout_score(document, style, fonts)` produces immutable layout pages with exact source intervals, notation anchors and drawing primitives. `generator.engraving.raster.rasterize_page(page, fonts)` draws one page at four-times resolution and downsamples it. `generator.engraving.pipeline.engrave_tree(root, fonts)` is the asset-pipeline entry point. `fonts.Fonts`, `types.Style`, and `types.EngravingError` live in their respective engraving modules; page constants and `check_page` live in `validation`. Package initializers do not re-export these APIs. `generator/musicxml_writer.py` remains a round-trip/export utility and is not involved in engraving.
 
-Engraving supports a bounded process pool (`VAMP_ENGRAVING_JOBS`, default 2) and a build-only content-addressed page cache (`VAMP_ENGRAVING_CACHE_DIR`, default `build/build_common/engraving-cache`). Keys include the complete musical document, engraving implementation, fonts/metadata, style, Python and rendering dependency versions. Playback is deliberately excluded from page keys and is always rebuilt independently. Cache hits are copied into the current staged output and checked for checksums, PNG decoding, dimensions, blank pages and clipped ink. Missing or corrupt entries are regenerated. All semantic and asset validation, database construction and transactional publication remain in place. The cache is not installed and can be deleted at any time when builds are idle. CI saves it after the native build and reuses it for ARM packaging; generation prints stage timing and per-worker progress instead of remaining silent during engraving.
+Engraving supports a bounded process pool (`VAMP_ENGRAVING_JOBS`, default 2) and a build-only content-addressed page cache (`VAMP_ENGRAVING_CACHE_DIR`, default `build/build_common/engraving_cache`). Keys include the complete musical document, engraving implementation, fonts/metadata, style, Python and rendering dependency versions. Playback is deliberately excluded from page keys and is always rebuilt independently. Cache hits are copied into the current staged output and checked for checksums, PNG decoding, dimensions, blank pages and clipped ink. Missing or corrupt entries are regenerated. All semantic and asset validation, database construction and transactional publication remain in place. The cache is not installed and can be deleted at any time when builds are idle. CI saves it after the native build and reuses it for ARM packaging; generation prints stage timing and per-worker progress instead of remaining silent during engraving.
 
 Reuse has two levels. A full-score hit restores complete page images for the same musical document and metadata. A notation-body group shares system layout and body pixels among documents with identical staff content, while retaining each document's title, subtitle, pagination, footer, and complete semantic bundle. Body keys include the rendering context and actual page-band geometry; incompatible placement cannot reuse those pixels. Each header/footer is composed separately, and pages whose drawing primitives cannot be separated safely use full-page rasterization. This reuse never merges score IDs, catalog records, or authored music.
 
-`notation.py` interprets staff state, exact pitch positions and shared accidental state; `notes.py` and `attributes.py` construct symbols; `spans.py` handles beams, ties, tuplets, octave shifts and lyric continuations; `layout.py` handles spacing and pagination. `fonts.py` reads bundled OTF outlines and SMuFL metadata, caching metrics and outlines for one generation run. `raster.py` uses FontTools, aggdraw and Pillow only as low-level font and drawing libraries. All dependencies are pinned in `generator/requirements.txt` and remain build-only. Standard glyph names and their upstream notice are vendored alongside the fonts; no system music fonts are consulted.
+`notation.py` interprets staff state, exact pitch positions and shared accidental state; `notes.py` and `attributes.py` construct symbols; `spans.py` handles beams, ties, tuplets, octave shifts and lyric continuations; `layout.py` handles spacing and pagination. `fonts.py` reads bundled OTF outlines and SMuFL metadata, caching metrics and outlines for one generation run. `raster.py` uses FontTools, aggdraw and Pillow only as low-level font and drawing libraries. Modulus owns the pinned optional `assets` dependency group. Configure prepares its build-only environment; CMake and `./modulus.py python` use the exported runner to import shared build utilities without shadowing the Modulus package. Standard glyph names and their upstream notice are vendored alongside the fonts; no system music fonts are consulted.
 
 Voices retain separate stems and collision offsets; chord tones retain their own written durations and per-tone ties. Pitched and percussion material use their own staff positions. Unknown exact microtonal alterations use a rational semitone annotation rather than a rounded accidental; an unavailable explicit SMuFL glyph is an error. The broader model/engraver supports this notation, while automatic guitar generation enforces the source contracts above.
 
@@ -216,7 +216,7 @@ Pages are 976x484 pixels with 10-pixel guitar or 12-pixel vocal staff spacing, 2
 
 Generation validates sources and complete staged output before replacing the previous asset directory. Replacement is always complete; deleted sources cannot leave stale JSON, pages, audio, or Pico title assets. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalog. Runtime packages contain `catalog.db` with embedded semantic JSON and title-image identifiers, PNG pages, generated PCM WAVs, Pico title images, and the OTF UI text font, not MusicXML, MuseScore, or Python dependencies. Python test discovery runs model/import/export tests; the normal asset build additionally qualifies and audits every catalog score.
 
-The page cache uses `<renderer fingerprint>/fullscore/<score fingerprint>/` and `<renderer fingerprint>/notation/<band fingerprint>/` directories, without a manual cache revision. `python3 -m generator --renderer-fingerprint` reports the same identity used by rendering and CI: source bytes, fonts, visual style, Python implementation/version, and rendering-library versions. Python compiler/build timestamps are excluded. CI keys append all generator and active XML inputs, including rhythms, and restore earlier snapshots only within the same renderer directory; only that directory is saved after a successful build.
+The page cache uses `<renderer fingerprint>/fullscore/<score fingerprint>/` and `<renderer fingerprint>/notation/<band fingerprint>/` directories, without a manual cache revision. `./modulus.py python -m generator --renderer-fingerprint` reports the same identity used by rendering and CI: source bytes, fonts, visual style, Python implementation/version, and rendering-library versions. Python compiler/build timestamps are excluded. CI keys append all generator and active XML inputs, including rhythms, and restore earlier snapshots only within the same renderer directory; only that directory is saved after a successful build.
 
 `cold namespace` means that renderer's persistent directory does not yet exist. A renderer or environment change starts a new namespace; deleting a namespace has the same effect. It does not disable reuse within the current run: shared notation is still laid out once per group and its body pixels can be reused as other titles are composed, even without a persistent cache. Conversely, a warm directory does not guarantee that a particular score is cached. Progress reports full-score hits, body hits, newly rasterized body bands, composed pages, missing/rejected entries, and phase timings. Worker phase timings are summed and may exceed wall time. Obsolete namespaces are not migrated and can be removed when builds are idle.
 
@@ -244,7 +244,7 @@ The body shows the focused menu item, song title, or setting and its current val
 
 Score generation renders titles as 296x64 four-bit coverage images under `pico/titles/<sha256>.mask`. Each file is 9,480 bytes: an eight-byte `VPTI` header with little-endian width/height, followed by packed coverage. Identical image bytes share an asset without merging score identities. The generated `presentation.title_image` identifier is also stored in catalog metadata; no separate lookup manifest is required. These assets are generated and validated before staged output is published. Remote status includes `title_image=<sha256>`; `POST /api/remote/title` with `{"id":"<sha256>"}` serves validated bytes only for catalog-registered titles. Pico fetches a changed title, validates its fixed format and bounds, rejects stale responses, and retains the selected image. Part labels and page counts use fixed sprites without reshaping or refetching title text. Commands take priority over image requests, and failed title requests display `UNAVAILABLE` until a retry succeeds.
 
-Pico performs coverage blending and sprite placement only: no OTF parsing, font shaping, glyph rasterization, or general text renderer runs in firmware. Missing glyphs and invalid source text are build errors. Font or generator changes rebuild both static firmware sprites and title artwork. Direct generation uses `python3 -m generator.pico_artwork --static-output <path>` for firmware sprites and `python3 -m generator.pico_artwork --assets <generated-assets>` for title assets. Python, Pillow/Raqm, and FontTools remain build-only dependencies.
+Pico performs coverage blending and sprite placement only: no OTF parsing, font shaping, glyph rasterization, or general text renderer runs in firmware. Missing glyphs and invalid source text are build errors. Font or generator changes rebuild both static firmware sprites and title artwork. Direct generation uses `./modulus.py python -m generator.pico_artwork --static-output <path>` for firmware sprites and `./modulus.py python -m generator.pico_artwork --assets <generated_assets>` for title assets. Python, Pillow/Raqm, and FontTools remain build-only dependencies.
 
 The root `CMakeLists.txt` is also the Pico firmware entrypoint when configured with `-DVAMP_PICO_FIRMWARE_PROJECT=ON`. That mode returns before the Zero/Linux targets are declared, so the Pico build can use the embedded ARM toolchain cleanly.
 
@@ -261,8 +261,8 @@ Modulus packages the CMake install stage. Product Vamp uses `install_prefix` `.`
 
 The ARM64 package target builds:
 
-- `zero/product-vamp-zero-linux-arm64.docker.tar`
-- `pico/product-vamp-pico2w.uf2`
+- `zero/product_vamp_zero_linux_arm64.docker.tar`
+- `pico/product_vamp_pico2w.uf2`
 - `manifest.json`
 
 Building the full package requires Docker/buildx, the ARM embedded GCC toolchain, and the Pico SDK. CI passes `PICO_SDK_PATH` directly. The generated Modulus Docker image also carries the pinned Pico SDK at `/opt/pico-sdk` for local package builds. The Zero Docker runtime targets Debian trixie to match current Raspberry Pi OS and the ARM64 dependency binaries, and includes `util-linux` for the `setsid --wait --ctty` launch path.
@@ -271,9 +271,9 @@ Generated score assets are created during CMake build/package and copied into th
 
 `generator/catalog.py` uses Python's standard-library SQLite module to construct `catalog.db` after musical, playback, and engraving validation. The database carries application ID `0x56414D50`, complete score bundles, and menu metadata. Runtime validates both required strict tables, their column names/types, non-null requirements, and primary keys directly. Sorted insertion and the absence of wall-clock timestamps make repeated builds reproducible with the same toolchain. It is closed and integrity-checked before publication, without journal/WAL sidecars. Individual JSON files remain available in the build tree for inspection, but both CMake installation and Docker image assembly exclude them.
 
-The default `database.path` is `/usr/share/product-vamp/catalog.db`; development configuration points at the native generated-assets database. Stored source paths are logical, asset-relative locations, not build-host filenames. Runtime resolves them against `runtime.assets_path` and rejects traversal and escaping symlinks. The JSON file itself need not exist: its parent identifies the score's page directory. Normal startup checks catalog compatibility; `--once` additionally runs SQLite's integrity probe and checks all page/audio files. Missing, corrupt, or incompatible catalogs fail rather than creating an empty library. `--ensure-database` is no longer a supported command.
+The default `database.path` is `/usr/share/product_vamp/catalog.db`; development configuration points at the native generated_assets database. Stored source paths are logical, asset-relative locations, not build-host filenames. Runtime resolves them against `runtime.assets_path` and rejects traversal and escaping symlinks. The JSON file itself need not exist: its parent identifies the score's page directory. Normal startup checks catalog compatibility; `--once` additionally runs SQLite's integrity probe and checks all page/audio files. Missing, corrupt, or incompatible catalogs fail rather than creating an empty library. `--ensure-database` is no longer a supported command.
 
-Every run of `zero/install-zero.sh` is a clean installation. It stages configuration/scripts, verifies the manifest's Docker tag against the incoming archive's image digest, and probes that image before stopping Product Vamp. It then clears the owned `/etc/product-vamp`, `/var/lib/product-vamp`, `/run/product-vamp`, and obsolete host `/usr/share/product-vamp` trees, including old SQLite sidecars. The installed container uses assets from its image and no longer mounts a writable state database. Startup reads the verified image ID from `/etc/product-vamp/image-id`, rather than depending on a mutable Docker tag. The installer replaces its scripts and systemd units, removes their old overrides, recreates only the `product-vamp-control` NetworkManager profile, and requires a successful service health check before announcing success.
+Every run of `zero/install_zero.sh` is a clean installation. It stages configuration/scripts, verifies the manifest's Docker tag against the incoming archive's image digest, and probes that image before stopping Product Vamp. It then clears the owned `/etc/product_vamp`, `/var/lib/product_vamp`, `/run/product_vamp`, and obsolete host `/usr/share/product_vamp` trees, including old SQLite sidecars. The installed container uses assets from its image and no longer mounts a writable state database. Startup reads the verified image ID from `/etc/product_vamp/image_id`, rather than depending on a mutable Docker tag. The installer also removes the fixed legacy hyphenated Product Vamp paths, units, containers, and network profile after preflight, while rejecting symlinked destinations. It replaces its scripts and systemd units, removes their old overrides, recreates only the `product_vamp_control` NetworkManager profile, and requires a successful service health check before announcing success.
 
 Configuration, AP credentials, and all prior catalog data are intentionally reset to bundle defaults. There is no automatic backup or migration. Installation through the control AP can disconnect the session; use the local console or independent networking. Preflight failure leaves existing application files untouched and restores the previous image tag; replacement failure stops Product Vamp and reports the error. Rerunning the installer is the recovery path. Ordinary boots/restarts do not run cleanup. Unrelated networks, OS data, containers, and system logs remain untouched. Cleanup rejects symlink/mount targets, never follows paths from old configuration, and removes only identified obsolete Product Vamp images not used by another container.