Music XML

TECHNICAL_GUIDE.md

@@ -9,7 +9,7 @@ Product Vamp follows the Modulus product layout used by the khodo repos:
 - `include/vamp_protocol` and `src/vamp_protocol`: shared route and page-command contracts used by both devices.
 - `cmake/pico.cmake` and `cmake/zero.cmake`: device-specific build definitions selected by the root `CMakeLists.txt`.
 - `include/vamp_style.hpp`, `include/vamp_style/font.hpp`, and `src/vamp_style.cpp`: shared color palette and Petaluma font interfaces. Bitmap font data is generated in the build directory.
-- `data`: authored source score JSON for performance songs and practice exercises.
+- `data`: authored MusicXML for performance songs and practice exercises, plus the generator regression summary.
 - `generator`: build-time Python preprocessing for performance arrangements and practice variants.
 - `package`: generic bundle inputs, scripts, config, static font assets, and device install files.
 
@@ -37,7 +37,59 @@ Petaluma music symbols are placed from SMuFL bounding boxes and stem/flag anchor
 
 The build runs `python3 -m generator` against `data/` and writes build-only assets under the CMake binary tree. Performance source songs become acoustic, electric, and vocal arrangements. Melody practice sources are emitted in all 12 keys at build time. Chord practice sources are emitted once in their authored key. Generated JSON keeps notation metadata while removing generator-only optional chord markers; guitar playability logic is used only inside the arrangement generator.
 
-Authored source files use compact JSON and are normalized by the generator before arrangement. A root-level `measures` array means compact source; staff/voice JSON with `staves` is still accepted. Measure notation supports `PITCH:DURATION`, `r:DURATION`, parenthesized chord tones like `(G4 B4 F5):4`, `[Chord]` markers, optional turnaround markers as `[(Chord)]`, quoted lyrics, `~` tie starts/stops, and tuplets as `{N:M ...}`. Measure objects may also carry notation metadata such as attributes, directions, harmony, and barlines so the data shape can grow toward MusicXML-style concepts without changing the compact note syntax. Tuplet normalization keeps the written duration in `display_ticks` and distributes rounded timeline ticks across the tuplet group so the measure total remains stable on the existing 16,384 ticks-per-whole-note grid.
+The source pipeline is MuseScore -> uncompressed `.musicxml` -> `generator/musicxml.py` -> arranger/transposer -> schema-v2 JSON. Export MusicXML 3.1 or 4.0, `score-partwise`, with one part, one staff, and one rhythmic voice. Chord tones share an onset but retain their own spelling, written duration, and tie links. Separate voices/staves must not be flattened into the source voice. The schema keeps staff/voice ownership explicit for a future polyphonic renderer.
+
+### MusicXML Contract
+
+| Area | Accepted notation |
+| --- | --- |
+| Pitch | Twelve-tone letter/alteration/octave spelling, including double/triple accidentals and courtesy accidentals. B-sharp and C-flat retain the correct octave. |
+| Rhythm | All fourteen MusicXML types, maxima through 1024th; up to sixteen augmentation dots; exact positive rational metric durations and separately stored written values. Time modifications must reconcile the two. |
+| Chords and rests | Simultaneous chord tones, shorter additional chord tones, rests, full-measure rests, hidden events, forward spacers, and rhythm slashes. |
+| Ties and tuplets | Per-pitch ties, including partial chord ties; nested and cross-bar tuplet spans. Nested tuplets require explicit `tuplet-actual` and `tuplet-normal` numbers. |
+| Small notes | Grace notes with zero metric duration, cue notes, and measured single/paired tremolos. |
+| Staff state | G/F/C clefs on lines 1-5, octave clefs and octave-shift spans; standard major/minor/modal keys or custom seven-letter alterations; key, clef, and meter changes. |
+| Meter | Simple, compound, additive, mixed-denominator, and unmetered timelines. A partial measure must be marked `implicit="yes"`; an initial implicit measure is the arrangement pickup. Leading rests are retained. |
+| Harmony | Structured roots, kinds, inversions, slash basses, and degree additions/alterations/subtractions. Harmony events have their own offsets, including inside held notes. Their positions never split or shorten notes. |
+| Text | Title, composer, part name, one lyric verse with syllabification and extenders, rehearsal labels at measure starts, and Vamp directives. |
+
+Microtones, independent voices, multiple staves/parts, percussion/tab staves, repeats/navigation, slurs, dynamics, articulations, and expressive ornaments are rejected. Unknown musical elements also fail, rather than disappearing. Custom key-octave placement is not supported. Resource limits are explicit: serialized times use reduced signed-64-bit numerators/positive denominators; pitches use octaves -4 through 16 and alterations -3 through 3; clef/octave shifts span at most three octaves; meter counts are 1-1024 with denominator units 1-4096. These limits produce diagnostics, not clamping or truncation.
+
+Layout/exporter information such as page defaults, credits, print layout, font/color/position attributes, and source pagination does not determine the small-screen layout. Engraving is regenerated. Beam groups and written note values are retained; unrelated engraving metadata does not authorize ignoring musical descendants. `defusedxml==0.7.1` forbids XML entities and external access. Ordinary MusicXML DOCTYPE declarations are allowed without fetching their DTDs.
+
+### Vamp Metadata
+
+In MuseScore's score properties, set `vamp:style` to a supported player style. It exports as standard MusicXML miscellaneous metadata:
+
+```xml
+<miscellaneous>
+  <miscellaneous-field name="vamp:style">Medium Swing</miscellaneous-field>
+</miscellaneous>
+```
+
+The default is Medium Swing; `Med.`, `Med. Swing`, and `Medium` remain aliases. A keyless melody exercise needs `vamp:tonic` (for example, `C`) as its transposition reference. Modes and local key changes are preserved when generating practice variants.
+
+Place staff text `vamp:optional-chord` at the same onset as exactly one harmony to mark a turnaround omitted from the final arrangement section. The text may be hidden. Parenthesizing an ordinary chord does not make it optional. Orphaned/duplicate markers and unknown `vamp:` properties are errors; consumed directives never enter runtime JSON.
+
+### Runtime Schema
+
+`schema_version: 2` is mandatory. `metadata` and one `staves[]` entry contain measures and an explicit voice. Measure `duration`, event `onset`/`duration`, harmony offsets, and span endpoints use `{ "numerator": N, "denominator": D }` in quarter-note units. Fractions are reduced; there is no fixed tick grid and no remainder distribution. The native loader uses exact arithmetic for validation, converting only layout coordinates to floating point.
+
+Events have `kind`, `written: { "type", "dots", "ratio"? }`, and independently identified pitches. Each pitch carries integer `alter`, `octave`, its own `duration` and `written`, and optional reciprocal `tie_from`/`tie_to` pitch IDs. Staff-level tuplets reference event IDs; octave spans reference absolute musical time. Attributes and harmonies are timestamped independently. The normalized harmony degree alteration is relative to the natural diatonic interval; the importer converts MusicXML's kind-relative alterations and dominant-relative additions.
+
+The arranger normalizes transposing instruments to concert pitch before generating parts. Guitar parts explicitly use written pitches one octave above sounding pitch and a -1 octave instrument transposition. Chord pitch definitions are separate from playable voicing choices; voicings may omit a natural fifth or lower unaltered extensions, but never change a chord kind or replace an unsupported chord with an unrelated major chord. Unplayable combinations produce a labelled rhythmic slash and a build warning.
+
+The renderer uses written values for glyphs, meter-aware/multilevel beams, per-pitch ties, timestamped harmony, and OTF-derived glyph bounds. Content-aware systems split dense measures into continuations. A canonical 976x484 layout makes page navigation stable across viewport sizes; vertical extent affects pagination instead of clamping pitches. The active page geometry is cached between redraws.
+
+### Validation And Migration
+
+Generation validates sources and complete staged output before replacing the previous asset directory. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalogue. Runtime packages contain JSON and OTF fonts, not MusicXML or Python importer dependencies. Python test discovery runs all generator/importer tests; `tests/fixtures/notation.musicxml` exercises the broader renderer contract. The optional `vamp-render-score` test target captures a page using a desktop-capable Raylib build.
+
+Diagnostics identify the file, part, measure, and unsupported element or invalid relationship. Examples include `part P1, measure 4: unsupported musical element <backup>`, `tie stop has no contiguous matching start`, and `optional-chord marker must identify exactly one harmony`.
+
+The six compact source files were replaced, not retained as a second authoring format. Existing catalogue rows refresh from regenerated JSON at startup. Custom database rows (`catalog_managed = 0`) are retained, including ID conflicts, and are never automatically deleted to resolve a schema mismatch. Custom legacy JSON must be reimported as schema v2; the loader reports a regeneration error instead of interpreting old ticks heuristically.
+
+References: [MusicXML durations](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/duration/), [tuplets](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/tuplet/), [degree alterations](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/degree-alter/), and [SMuFL accidentals](https://smufl.formats.music/latest/tables/standard-accidentals-12-edo.html).
 
 Generator checks compare the generated performance summary against `data/expected/performance_summary.json`. The summary stores compatibility invariants such as section layout, chord-label placement, slash-section placement, and vocal timing without committing the generated score files themselves. Refresh that file only when an intentional arrangement-logic change should become the new baseline.