Music Notation

TECHNICAL_GUIDE.md

@@ -30,31 +30,30 @@ The AP setup pins NetworkManager to 2.4 GHz channel 6 with WPA2/RSN and CCMP cip
 
 On Raspberry Pi OS Lite, the installed display service owns `tty1`, conflicts with `getty@tty1.service`, and starts the Zero container through `product-vamp-start-zero`. The wrapper switches to the display VT when `chvt` is available and runs `vamp-zero` inside the container through `setsid --wait --ctty` with `/dev/tty1` as stdin, giving raylib's DRM backend a real virtual terminal while keeping Docker attached to the long-running display process. SSH remains the normal admin path and `tty2` is enabled as the fallback local console.
 
-At startup the Zero scans `share/product-vamp/songs/*.json` and `share/product-vamp/practice/<type>/<lesson_group>/<exercise>/<key>.json`, ensures the current SQLite schema with Noto, imports both catalogs, then reloads runtime menu state from the database. Runtime navigation and rendering use that loaded Noto-backed state instead of rescanning asset folders. The UI supports a top-level mode menu, Performance song/part/score menus, and Practice type/lesson/exercise/key/score menus. Melody practice offers key selection by choosing a pre-generated score variant; chord practice skips key selection and renders the authored score key directly. The score renderer draws title/composer/part metadata, systems, measures, clef/key/time preambles, enlarged chord symbols and section labels, Petaluma SMuFL noteheads/rests/flags/dots/accidentals, simple vector stems/ties, and lyrics. Slash notation is represented explicitly with `notehead: "slash"` in generated JSON.
-Petaluma music symbols are placed from SMuFL bounding boxes and stem/flag anchors instead of rough font-size offsets, which keeps clefs, noteheads, rests, dots, accidentals, and eighth-note flags aligned to staff-space coordinates. Printed accidentals are derived from the active key signature and per-measure accidental state, so natural signs are shown only when they are musically required.
+At startup the Zero scans `share/product-vamp/songs/*.json` and `share/product-vamp/practice/<type>/<lesson_group>/<exercise>/<key>.json`, ensures the current SQLite schema with Noto, imports both catalogs, then reloads runtime menu state from the database. Runtime navigation and rendering use that loaded Noto-backed state instead of rescanning asset folders. The UI supports a top-level mode menu, Performance song/part/score menus, and Practice type/lesson/exercise/key/score menus. Melody practice offers key selection by choosing a pre-generated score variant; chord practice skips key selection and renders the authored score key directly. The score view displays build-time engraved PNG pages. Musical interpretation, spacing, beaming, ties, accidentals, and pagination happen on the build host, not on the Pi. The view retains at most two page textures.
 
 ## Score Generation
 
 The build runs `python3 -m generator` against `data/` and writes build-only assets under the CMake binary tree. Performance source songs become acoustic, electric, and vocal arrangements. Melody practice sources are emitted in all 12 keys at build time. Chord practice sources are emitted once in their authored key. Generated JSON keeps notation metadata while removing generator-only optional chord markers; guitar playability logic is used only inside the arrangement generator.
 
-The source pipeline is MuseScore -> uncompressed `.xml` -> `generator/musicxml.py` -> arranger/transposer -> schema-v2 JSON. Export MusicXML 3.1 or 4.0, `score-partwise`, with one part, one staff, and one rhythmic voice. Chord tones share an onset but retain their own spelling, written duration, and tie links. Separate voices/staves must not be flattened into the source voice. The schema keeps staff/voice ownership explicit for a future polyphonic renderer.
+The pipeline is MuseScore -> uncompressed `.xml` -> musical model -> automatic arranger/transposer -> semantic JSON plus engraved pages. Export MusicXML 3.1 or 4.0, `score-partwise`, with one part and one staff. Independent rhythmic voices remain separate; simultaneous chord tones retain their own spelling, written duration, and tie links. The packaged music describes each transformed part or practice variant, not an archival copy of the source XML.
 
 ### MusicXML Contract
 
 | Area | Accepted notation |
 | --- | --- |
-| Pitch | Twelve-tone letter/alteration/octave spelling, including double/triple accidentals and courtesy accidentals. B-sharp and C-flat retain the correct octave. |
+| Pitch | Exact rational letter/alteration/octave spelling, including microtones, double/triple accidentals, and courtesy accidentals. B-sharp and C-flat retain the correct octave. |
 | Rhythm | All fourteen MusicXML types, maxima through 1024th; up to sixteen augmentation dots; exact positive rational metric durations and separately stored written values. Time modifications must reconcile the two. |
-| Chords and rests | Simultaneous chord tones, shorter additional chord tones, rests, full-measure rests, hidden events, forward spacers, and rhythm slashes. |
+| Chords and rests | Independent voice lanes, simultaneous chord tones, shorter additional chord tones, rests, full-measure rests, hidden events, forward spacers, and rhythm slashes. |
 | Ties and tuplets | Per-pitch ties, including partial chord ties; nested and cross-bar tuplet spans. Nested tuplets require explicit `tuplet-actual` and `tuplet-normal` numbers. |
 | Small notes | Grace notes with zero metric duration, cue notes, and measured single/paired tremolos. |
-| Staff state | G/F/C clefs on lines 1-5, octave clefs and octave-shift spans; standard major/minor/modal keys or custom seven-letter alterations; key, clef, and meter changes. |
+| Staff state | G/F/C, percussion, TAB, and no-clef staves; tablature string/fret/tunings; octave clefs and octave-shift spans; standard major/minor/modal keys or custom seven-letter alterations; key, clef, and meter changes. |
 | Meter | Simple, compound, additive, mixed-denominator, and unmetered timelines. A partial measure must be marked `implicit="yes"`; an initial implicit measure is the arrangement pickup. Leading rests are retained. |
 | Harmony | Structured roots, kinds, inversions, slash basses, and degree additions/alterations/subtractions. Harmony events have their own offsets, including inside held notes. Their positions never split or shorten notes. |
-| Text | Title, composer, part name, one lyric verse with syllabification and extenders, rehearsal labels at measure starts, and Vamp directives. |
+| Text | Title, composer, part name, multiple lyric verses with syllabification, elisions, and extenders, rehearsal labels at measure starts, and Vamp directives. |
 | Tempo | Metronome directions and positive quarter-note BPM from `<sound tempo="...">`, either within a direction or directly in a measure. Playback/navigation attributes on `sound` are rejected. |
 
-Microtones, independent voices, multiple staves/parts, percussion/tab staves, repeats/navigation, slurs, dynamics, articulations, and expressive ornaments are rejected. Unknown musical elements also fail, rather than disappearing. Custom key-octave placement is not supported. Resource limits are explicit: serialized times use reduced signed-64-bit numerators/positive denominators; pitches use octaves -4 through 16 and alterations -3 through 3; clef/octave shifts span at most three octaves; meter counts are 1-1024 with denominator units 1-4096. These limits produce diagnostics, not clamping or truncation.
+Multiple staves/parts, repeats/navigation, slurs, dynamics, articulations, and expressive ornaments are outside the product contract and rejected. Unknown musical elements also fail. Serialized times and alterations use reduced signed-64-bit numerators and positive denominators; pitch octaves are -4 through 16, clef/octave shifts span at most three octaves, and meter counts are 1-1024 with denominator units 1-4096. These resource bounds produce diagnostics, not clamping or truncation.
 
 Layout/exporter information such as page defaults, credits, print layout, font/color/position attributes, and source pagination does not determine the small-screen layout. Engraving is regenerated. Beam groups and written note values are retained; unrelated engraving metadata does not authorize ignoring musical descendants. `defusedxml==0.7.1` forbids XML entities and external access. Ordinary MusicXML DOCTYPE declarations are allowed without fetching their DTDs.
 
@@ -74,21 +73,35 @@ Place staff text `vamp:optional-chord` at the same onset as exactly one harmony
 
 ### Runtime Schema
 
-`schema_version: 2` is mandatory. `metadata` and one `staves[]` entry contain measures and an explicit voice. Measure `duration`, event `onset`/`duration`, harmony offsets, and span endpoints use `{ "numerator": N, "denominator": D }` in quarter-note units. Fractions are reduced; there is no fixed tick grid and no remainder distribution. The native loader uses exact arithmetic for validation, converting only layout coordinates to floating point.
+A bundle has three top-level fields:
 
-Events have `kind`, `written: { "type", "dots", "ratio"? }`, and independently identified pitches. Each pitch carries integer `alter`, `octave`, its own `duration` and `written`, and optional reciprocal `tie_from`/`tie_to` pitch IDs. Staff-level tuplets reference event IDs; octave spans reference absolute musical time. Attributes and harmonies are timestamped independently. The normalized harmony degree alteration is relative to the natural diatonic interval; the importer converts MusicXML's kind-relative alterations and dominant-relative additions.
+- `schema_version: 3`.
+- `music`: metadata and a single staff containing measures, independent voice lanes, timed harmonies/attributes/tempos, instruments, tuplets, and octave spans.
+- `presentation`: engraver/version, style checksum, fixed width/height, and ordered relative PNG paths with SHA-256 checksums.
 
-The arranger normalizes transposing instruments to concert pitch before generating parts. Guitar parts explicitly use written pitches one octave above sounding pitch and a -1 octave instrument transposition. Chord pitch definitions are separate from playable voicing choices; voicings may omit a natural fifth or lower unaltered extensions, but never change a chord kind or replace an unsupported chord with an unrelated major chord. Unplayable combinations produce a labelled rhythmic slash and a build warning.
+Measure durations, event onsets/durations, harmony offsets, span times, and pitch alterations use `{ "numerator": N, "denominator": D }`. Times are in quarter-note units and alterations in semitones. There is no tick grid, display-duration surrogate, or runtime flattening into one voice. Written note types, dots, and time modifications remain distinct from metric duration. Every event owns a voice; every chord tone has its own ID, written duration, duration, and reciprocal tie links. Tuplets reference events and belong to one voice.
 
-The renderer uses written values for glyphs, meter-aware/multilevel beams, per-pitch ties, timestamped harmony, and OTF-derived glyph bounds. Content-aware systems split dense measures into continuations. A canonical 976x484 layout makes page navigation stable across viewport sizes; vertical extent affects pagination instead of clamping pitches. The active page geometry is cached between redraws.
+The arranger normalizes instrument transposition to concert pitch and preserves all source voices in source-copy sections. Generated guitar rhythms occupy one lane. The seven-section form, optional turnaround handling, twelve melody-practice keys, and authored-key chord exercises remain automatic. Unplayable guitar harmonies produce a warned rhythmic-slash fallback without changing the structured chord meaning or quantizing microtones.
+
+Automatic performance arrangements require a pitched, non-TAB lead. Percussion and tablature remain available in the model and practice pipeline, but conversion into guitar/vocal parts fails explicitly rather than reinterpreting percussion positions or fret numbers as ordinary treble-staff notation. Practice transposition retains an authored string only when the resulting fret is an exact nonnegative integer; it does not invent a new fingering.
+
+### Engraving Boundary
+
+`generator/musicxml_writer.py` emits MusicXML from the transformed model. `generator/engraving/` runs pinned MuseScore Studio 4.7.4 with the checked-in Petaluma house style, isolated home/config/font caches, and the repository OTF fonts. The checksum-verified AppImage is installed only on the build host; it is never packaged for the Pi.
+
+Model support is deliberately broader than engine qualification. More than four source voice identities, maxima, more than four dots, independently sustained tones within one chord, cross-bar tuplets, double stems, custom accidental glyphs, and unqualified microtonal alterations produce explicit engraving errors. Unequal chord durations can be authored as separate voices for this engine. No model limitation or silent approximation is introduced to conceal these failures.
+
+Each export is read back as MusicXML and audited for pitch spelling, exact timing, voice/event counts, written values, ties, explicit beams/stems, lyrics, harmony degrees, staff changes, tempos, and spans. Engraver changes fail generation. This is not a claim that MuseScore can engrave every MusicXML construct. Newly supported constructs must pass importer/writer tests, engine auditing, and visual qualification.
+
+Pages are 976x484 pixels with 12-pixel staff spacing. Pagination is done by the engraver, not by shrinking the page or clipping notation. Blank pages, incorrect dimensions, and ink reaching the page edge are rejected. Runtime letterboxing preserves aspect ratio. Luma's `load_texture_uncached()` lets Product Vamp retain at most two textures without a hidden strong asset-cache reference. The UI reports missing/corrupt assets instead of falling back to the removed renderer. Runtime validates bundle metadata, paths, dimensions, and page checksums; the build validates musical semantics.
 
 ### Validation And Migration
 
-Generation validates sources and complete staged output before replacing the previous asset directory. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalogue. Runtime packages contain JSON and OTF fonts, not MusicXML or Python importer dependencies. Python test discovery runs all generator/importer tests; `tests/fixtures/notation.xml` exercises the broader renderer contract. The optional `vamp-render-score` test target captures a page using a desktop-capable Raylib build.
+Generation validates sources and complete staged output before replacing the previous asset directory. Replacement is always complete, including when the compatibility `--clean` flag is omitted; deleted sources cannot leave stale JSON or pages. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalogue. Runtime packages contain semantic JSON, PNG pages, and the OTF UI text font, not MusicXML, MuseScore, or Python dependencies. Python test discovery runs model/import/export tests; the normal asset build additionally qualifies and audits every catalogue score.
 
-Diagnostics identify the file, part, measure, and unsupported element or invalid relationship. Examples include `part P1, measure 4: unsupported musical element <backup>`, `tie stop has no contiguous matching start`, and `optional-chord marker must identify exactly one harmony`.
+Diagnostics identify the file, part, measure, and unsupported element or invalid relationship. Examples include `part P1, measure 4: unsupported musical element <slur>`, `tie stop has no contiguous matching start`, and `optional-chord marker must identify exactly one harmony`.
 
-The six compact source files were replaced, not retained as a second authoring format. Existing catalogue rows refresh from regenerated JSON at startup. Custom database rows (`catalog_managed = 0`) are retained, including ID conflicts, and are never automatically deleted to resolve a schema mismatch. Custom legacy JSON must be reimported as schema v2; the loader reports a regeneration error instead of interpreting old ticks heuristically.
+The six compact source files were replaced, not retained as a second authoring format. Existing catalogue rows refresh from regenerated JSON at startup. Custom database rows (`catalog_managed = 0`) are retained, including ID conflicts, and are never automatically deleted to resolve a schema mismatch. Custom legacy JSON must be regenerated as a schema-v3 bundle with page assets; the loader reports a regeneration error instead of interpreting old ticks heuristically.
 
 References: [MusicXML durations](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/duration/), [tuplets](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/tuplet/), [degree alterations](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/degree-alter/), and [SMuFL accidentals](https://smufl.formats.music/latest/tables/standard-accidentals-12-edo.html).
 
@@ -136,7 +149,7 @@ CI builds the generic artifact with `python3 modulus.py package --version latest
 
 The Pico firmware does not link Zero-side Linux application libraries such as Ligo, Luma, or Noto. Those remain Zero-side dependencies: Ligo serves the HTTP API, Luma renders the Zero display, and Noto owns SQLite persistence. Cross-device code should stay in `vamp_protocol` and `vamp_style`: plain C++17 with no OS, allocator-heavy, or Zero-only dependencies.
 
-Luma owns the application shell and UI control tree. The score renderer uses Luma primitives for layout, staff lines, and simple vector strokes, and uses a small score-local Raylib font cache for Petaluma SMuFL glyphs until Luma grows first-class codepoint loading.
+Luma owns the application shell, UI control tree, texture loading, and UI-thread texture lifetime. Product Vamp's page cache uses the uncached loading API; there is no score-specific Raylib font cache or handwritten notation renderer.
 
 ## LSP