Toggle Solos

README.md

@@ -5,15 +5,16 @@ Product Vamp is a two-device sheet-music appliance.
 - Raspberry Pi Zero 2 W hosts the control Wi-Fi AP, runs the display/control service in Docker, and drives a Waveshare Zero-DISP-7A 1024x600 touch display.
 - Raspberry Pi Pico 2 W runs UF2 firmware for a Waveshare 2-inch capacitive touch LCD and sends menu/page commands over the Zero-hosted AP.
 
-Python builds the complete SQLite catalog with the score assets. The Zero opens that packaged catalog read-only through Noto, without creating a database, migrating schemas, or importing JSON at startup. Runtime navigation and rendering use Ligo for the HTTP API, Luma for touch UI rendering, and Modulus for shared build/runtime utilities.
+Python builds the complete SQLite catalog with the score assets. The Zero opens that packaged catalog read-only through Noto, without creating a database, migrating schemas, or importing JSON at startup. Runtime services use Ligo for the HTTP API, Luma for touch UI rendering, Sonus for audio playback, and Modulus for shared build/runtime utilities.
 
 The catalog stores shared, immutable menu metadata. Complete score bundles stay in SQLite and are loaded individually when selected; snapshots and Pico heartbeats do not copy the catalog's musical JSON.
 
 Device controls:
 
-On the Zero, tap menu rows and switches directly; Tempo has minus/plus buttons. The main menu offers **Performance** and **Settings**.
+On the Zero, tap menu rows and switches directly; Tempo has minus/plus buttons. The main menu offers **Performance** and **Settings**. **Settings → Solos** switches both guitar parts between written solos (On) and beat slashes for improvisation (Off), including any solo pickup. Chord symbols and the rest of the arrangement stay in place. Solos starts On and remembers your choice until the app restarts; it applies when you next open a part on either device.
 
 On the Pico (and on the Zero while viewing a score):
+
 - Tap: next menu item or next score page.
 - Double tap: previous menu item or previous score page.
 - Tap and hold: select the focused item, enter/leave tempo editing, or return from a score to its part menu.
@@ -34,6 +35,8 @@ Release verification:
 ```sh
 python3 modulus.py configure --target x86_64_gcc12_release
 python3 modulus.py build --target x86_64_gcc12_release
+python3 modulus.py test --target x86_64_gcc12_release
+python3 modulus.py python -m unittest discover -s tests -p 'test_*.py'
 ```
 
 `python3 modulus.py run` uses `package/config/vamp.development.json` for a headless readiness check against native build assets. `vamp_zero --once` checks the packaged database and referenced page/audio assets, checks runtime writes and the server bind, then exits without opening audio or display hardware. Missing or incompatible catalogs require rebuilding or reinstalling; they are never repaired at runtime. `--check-config` only validates configuration; `--health-check` probes the running service's `/api/health` endpoint without modifying files.
@@ -60,7 +63,7 @@ Modulus also generates `.nvim.lua` for Neovim 0.11+ with `exrc` enabled. Review
 
 ## Package
 
-Product packaging targets Raspberry Pi OS/Debian trixie on ARM64 and optionally builds Pico firmware. Set `PICO_SDK_PATH` when building the firmware artifact:
+Product packaging targets Raspberry Pi OS/Debian trixie on ARM64. The configured package builds both the Zero Docker image and Pico firmware. The build container includes the Pico SDK; when building directly outside that container, set `PICO_SDK_PATH`:
 
 ```sh
 PICO_SDK_PATH=/path/to/pico-sdk python3 modulus.py package --update
@@ -91,19 +94,31 @@ Artifacts are published to the khodo generic package registry's required `latest
 
 ## Score Assets
 
-Uncompressed MusicXML 3.1/4.0 scores exported from Soundslice or MuseScore live in `data/`. Generated score JSON is not committed.
+Uncompressed MusicXML 3.1/4.0 source scores exported from Soundslice or MuseScore live in `data/songs/`. Generated score JSON is not committed.
 
-`data/songs` supplies the original melody and harmony. The 21 scale patterns, 2,954 relative interval paths, meter-specific rhythm catalogs, and general rules in `data/composition` supply musical vocabulary. Production never reads reference solos, fitted parameters, or previous music.
+`data/songs/` supplies the original melody and harmony. `data/composition/` holds guitar/difficulty settings, sparse accompaniment figures, and harmonic-validation policies. `data/training/manifest.json` registers authored solos and their source/slot pairings; `data/models/solo.json` holds the frozen pooled prediction model. The former scale, interval, and rhythm catalogs and handwritten phrase composer have been removed.
 
-Six roles generate independently: acoustic accompaniment/solo; electric accompaniment/melody/solo; vocal melody. A process pool and per-role caches support parallel and incremental builds. Canonical accompaniment repeats at corresponding source-form positions and does not react to generated solos. Original vocal and electric head content remain unchanged.
+Six roles generate independently: acoustic accompaniment/solo; electric accompaniment/melody/solo; vocal melody. A process pool and per-role caches support parallel and incremental builds. Each role uses the source and its own inputs without reading another generated role. Canonical accompaniment repeats at corresponding source-form positions and does not react to generated solos. Vocal melody preserves the authored notes and lyrics. Electric melody preserves source pitches and harmonizes eligible sustained notes, reattacking them at chord changes.
 
-The solo composer creates complete source-related melodic phrases before selection. A realized theme develops through returns, answers, register changes, arpeggio motion and complete scale routes. Phrase destinations, rhythm, rests and holds are coordinated. Both guitars use strings 1–6 and frets 1–12 without open strings. Colors, lead-ins, anticipations and suspensions need explicit harmonic evidence. No dynamics, accents or humanization are added.
+Electric accompaniment and electric melody share compact chord families on strings 1–4. Melody harmonization adapts these to keep the original melody on the lowest-numbered feasible string, preferring it above the added chord tones. The separate Electric Solo stays monophonic. Acoustic accompaniment uses one bass note on string 5 or 6 and three upper notes on strings 2, 3 and 4. Transposable degree families cover triads, sevenths/sixths, suspensions, diminished/augmented and extended/altered chords, with root/fifth bass alternatives on both bass strings. Plain major/minor chords also allow third basses; explicit slash basses and inversions take priority. Unplayable variants are discarded. Each instrument selects a connected voicing path across the source form to reduce hand movement, then reuses it through the arrangement.
 
-`--export-solo-xml` (CMake `VAMP_EXPORT_SOLO_XML=ON`) writes `generated_assets/solo_xml/*-solo1.xml` for acoustic and `*-solo2.xml` for electric, with actual notes and TAB. These exports do not recompose music. `python -m generator.compose --song ... --role acoustic_solo --data-dir data --output ... --solo-xml ...` builds one independent role.
+Accompaniment uses independently authored two-bar jazz pattern families, chosen against each song's original melody and harmonic rhythm. Acoustic repeats two patterns (A–B); electric repeats four (A–B–C–D). Occasional source-phrase-ending fills recur at the same positions in every iteration; both instruments allow at most one per eight full measures, including the repeat join. Acoustic normally has 1–3 attack positions per full bar with quarter/eighth bass and upper-chord notes played together or in sequential pairs. Electric has distinct patterns with normally 0–2 attacks per bar, stabs, sustains and space. Cross-bar holds use ties; repeated attacks remain attacks. Pattern phase follows source-form position, including partial entries and odd-length forms. Meter changes use compatible cycles, with explicit diagnostics if fewer distinct patterns are available. Tempo changes do not reshuffle the accompaniment.
 
-Optional comparison lives in `python -m tests.solo_compare`, outside production. Supply `--generated-dir`, explicit `--reference ROLE=PATH`, `--form-start ROLE=MEASURE`, `--reference-pitch guitar-written`, and `--output build/solo_comparison/review`; add `--assets` and `--baseline-dir` for consistent audio comparisons. References are analysis-only. The supplied targets sound one octave below encoded pitches; All of Me aligns at measure 1 and Autumn Leaves at measure 2, preserving the lead-in. Closeness is reported using actual notes and timing; matching statistics or technical validity do not establish musical preference.
+Each instrument has 24 distinct two-bar patterns in 4/4, 24 in 3/4, and 24 for each of the 3+2 and 2+3 groupings in 5/4. Written additive grouping takes precedence. Otherwise, the original chord changes and melody choose one consistent 5/4 grouping per continuous meter section, shared as source analysis by the independently generated parts. Explicit meter matching prevents treating 6/8 as 3/4. The learned solo model and playback count-in are unchanged by this accompaniment expansion.
 
-See [Independent Musical Generation](TECHNICAL_GUIDE.md#independent-musical-generation) for the active pipeline and limitations. Keep just one compact baseline and the latest comparison under `build/solo_comparison/`.
+Both accompaniment parts balance movement with sounding register and a soft cost for mean fret positions above nine; comfortable shifts can win over staying high. Acoustic uses a register center of MIDI 53 and electric uses 60. Compact spans and preferred chord families remain higher priorities, including for the final sustained chord. Electric melody's supporting chords use the same balance after the existing melody-string and foreground priorities; the original melody pitches and rhythm remain fixed. Voicing preferences do not reseed accompaniment rhythms.
+
+Acoustic attacks normally play bass and upper chord together. Consecutive attacks exactly an eighth note apart split into bass–upper pairs within the same harmony; an odd leftover attack plays both together. Pairing continues across barlines and two-bar pattern boundaries, but stops at chord changes, chorus boundaries and source-defined entries/handoffs. The first attack for each new chord therefore includes bass, including a qualified anticipated entrance. Delayed entrances remain delayed. Every new acoustic attack releases the previous sound: a bass eighth followed by an upper-chord quarter has no bass underneath the quarter. Complete-cycle selection, fills and final realization use this same release rule. Bass and upper notes share one engraved part; uninterrupted holds may still tie across barlines.
+
+Each song has six canonical roles: electric melody, electric accompaniment, electric solo, acoustic accompaniment, acoustic solo, and vocal melody. Matching source-form positions reuse the same musical events and TAB whenever a role returns. Final-turnaround omission changes only affected harmony; source-defined pickups/handoffs and the three-bar instrumental closing sustain remain explicit exceptions.
+
+**Registered Solo 1 is notated exactly in the acoustic part and Solo 2 exactly in the electric part.** Unregistered song/slot pairs use deterministic prediction from one pooled statistical model trained on both slots. There is no learned acoustic/electric style distinction. Prediction uses up to four recent pitch/rhythm events, source-melody features, harmony, and metrical position; it does not plan complete themes or guarantee long-range phrase development. The current corpus is four authored solos from Autumn Leaves and All of Me.
+
+Training, authored score reuse, and prediction share contextual harmony validation based on actual notes, timing, and chords. Explanatory labels and justification tags are not required. Unsupported relationships fail with a diagnostic rather than silently changing notes. Both solo roles use strings 1–6 and frets 1–12 without open strings. No dynamics, accents, velocity shaping, or random timing are added.
+
+`./modulus.py python -m generator.compose --song ... --role acoustic_solo --data-dir data --output ...` builds one independent role as JSON. To exercise prediction for a registered song, pass `--settings` with a JSON file containing `{"predict": true}`. Each song produces five score bundles: the normal acoustic, electric and vocal parts, plus acoustic and electric `_improv` variants. The variants reuse the same accompaniment and melody, replacing the full solo allocation (including pickup rests) with meter-aware slashes in notation and TAB. Their subtitles identify them as Improvisation. Normal scores retain written solos; separate solo MusicXML files are not generated. Authored XMLs remain under `data/training`, and regression tests compare their sounding events directly with generated score data.
+
+See [Shared Part Generation](TECHNICAL_GUIDE.md#shared-part-generation) and [Authored Repertoire and Learned Solos](TECHNICAL_GUIDE.md#authored-repertoire-and-learned-solos) for the active pipeline and limitations. Keep temporary comparisons and one-off diagnostics under `/tmp/`; `build/` retains product outputs, tools and dependencies.
 
 To inspect generated assets without building the full package, run:
 
@@ -114,36 +129,38 @@ To inspect generated assets without building the full package, run:
 The `--check` path compares generation invariants against `tests/fixtures/performance_summary.json`.
 Modulus owns the pinned build-only dependencies in its optional `assets` group, selected by `tooling.python.groups` in `modulus.json`. Configure prepares `build/build_common/python`; `./modulus.py python` and CMake use that environment and the exported Modulus modules. Builds never install Python dependencies. After dependency pins change, run configure again. C++-only libraries do not opt in, and Python is not packaged for the Pi.
 
-CMake builds engrave with two Python workers and share a disposable page cache under `build/build_common/engraving_cache`. Native and ARM packaging reuse unchanged pages; CI also retains this cache between runs. Cached musical roles are reused independently; music, playback, and catalog validation still run for every generation. Set `VAMP_ENGRAVING_JOBS` (1-16) to adjust concurrency, or set `VAMP_ENGRAVING_CACHE_DIR` to an empty CMake value to disable caching. Direct generator runs can opt in with `--jobs 2 --engraving-cache build/build_common/engraving_cache --progress`.
+CMake builds use two composition workers and two engraving workers, with separate disposable caches under `build/build_common/composition_cache` and `build/build_common/engraving_cache`. Native and ARM packaging reuse unchanged roles and pages; CI also retains the page cache between runs. Music, recording, and catalog validation still run for every generation. Set `VAMP_COMPOSITION_JOBS` or `VAMP_ENGRAVING_JOBS` (1-16) to adjust concurrency, or set `VAMP_ENGRAVING_CACHE_DIR` to an empty CMake value to disable persistent page caching. Direct generator runs default to two composition workers and one engraving worker, without persistent caches; use `--composition-cache`, `--engraving-cache`, `--composition-jobs`, `--jobs`, and `--progress` as needed.
 
 Identical notation bodies share layout and body rasterization even when titles or score identities differ; each score retains its own header, pages, music, and catalog identity. A `cold namespace` means no cache directory exists for the current renderer fingerprint. It does not mean that every score must be rendered independently: repeated bodies can still be reused within that build, including with persistent caching disabled. Progress separates full-score hits, body hits, newly rasterized bodies, and composed pages.
 
-Solos use one pooled model trained on the paired scores in `data/training`. Both solo slots contribute equally; there is no learned acoustic/electric style distinction. **Registered Solo 1 is played exactly by acoustic and Solo 2 exactly by electric.** Unregistered song/slot pairs use deterministic model prediction. Exact registered playback is explicit authored reuse, not a claim of model memorization. Only Autumn Leaves and All of Me are included initially.
+Add a source and its authored solo, then retrain the pooled model, without changing generator code:
-
-Training and playback use the same contextual harmony validator as predictions. It recognizes supported colors, passing notes, approaches and anticipations from the actual notes, durations and chords. No explanatory score labels or hand-authored justification tags are required. The four approved solos pass this validation; their explanatory labels have been removed without changing their music. Unsupported relationships still produce a diagnostic rather than silently changing a note.
-
-Add a source and its authored solo without changing generator code:
 
 ```sh
 ./modulus.py python -m generator.train --add-source /path/to/song.xml --add-solo /path/to/solo1.xml --slot 1
 ./modulus.py python -m generator.train --add-source data/songs/song.xml --add-solo /path/to/solo2.xml --slot 2
 ```
 
-To retrain existing pairs, run `./modulus.py python -m generator.train --leave-one-song-out`. Ordinary builds use the saved model; they never retrain automatically. The four-example model is a starting point and does not establish musical quality on unfamiliar songs. [Training and playback details](TECHNICAL_GUIDE.md#authored-repertoire-and-learned-solos) cover pairing, timing, pitch convention, data migration and limitations.
+To retrain existing pairs, run `./modulus.py python -m generator.train`; add `--leave-one-song-out` for held-out likelihood diagnostics. Training writes `data/models/solo.json` and `build/training/report.json`. Ordinary builds use registered authored music or the saved model and never retrain automatically. Exact registered playback verifies authored reuse; it does not demonstrate model memorization or musical quality on unfamiliar songs. [Training and playback details](TECHNICAL_GUIDE.md#authored-repertoire-and-learned-solos) cover pairing, timing, pitch convention, data migration, and limitations.
 
-The build-time pipeline is Soundslice/MuseScore -> `.xml` -> musical model and independent parts -> part generation -> JSON, engraved PNG pages, generated_tone audio, Pico title artwork, and a read-only catalog database. Sources in `data/songs/` retain the seven-section performance form. Songs generate Acoustic, Electric, and Vocal parts in their authored key, without key variants or a key-selection menu. Performance measure numbers run continuously across the complete arrangement, starting at 1 in the lead-in; they do not restart with a new section or chorus. Pickups occupy the preceding measure. Standalone comparison XMLs use local measure numbers and retain the original arrangement numbers in `manifest.solo`.
+The build-time pipeline is MusicXML -> source analysis -> independent roles -> assembled parts -> semantic JSON, supplied recordings, engraved PNG pages, Pico title artwork, and a read-only catalog database. Each source produces the six main performance sections plus the held Ending as Acoustic, Electric, and Vocal parts in its authored key, without key variants or a key-selection menu. Performance measure numbers run continuously across the complete arrangement, starting at 1 in the lead-in; they do not restart with a new section or chorus. Pickups occupy the preceding measure.
 
-All generated monophonic guitar material uses frets 1–12 without open strings. Accompaniment uses all legal compact positions; a chord can extend to fret 20 while retaining at least one sounding anchor at fret 12 or below and the existing three-fret span restriction. Electric source-melody harmonization retains its existing placement policy. Exact source pitches and ties are preserved; unplayable or unsupported input fails explicitly rather than being silently octave-folded.
+All generated monophonic guitar material uses frets 1–12 without open strings. Chords use two span tiers: **one to three consecutive fret positions share the primary tier; exactly four positions are a fallback** (e.g. 7–9 is primary, 7–10 is fallback). Five or more positions are forbidden. Selection minimizes fallback use before family, melody-string, movement or register preferences. A chord can extend to fret 20 while retaining at least one sounding anchor on frets 1–12 throughout its duration. Acoustic bass remains within frets 1–12 and upper chords need their own anchor when played separately. Some chord/key combinations still have no legal inversion; generation reports them instead of removing required tones. Electric source-melody harmonization retains its chord-change reattack policy. Import preserves exact source pitches and ties; unplayable or unsupported input fails explicitly rather than being silently octave-folded.
 
 There are no genre presets or runtime composition. The complete arranged score is played in reading order. The source-defined form and continuous whole-arrangement measure numbering remain unchanged.
 
-All parts use two music systems per page, with a single system only when the score ends on an odd system. Lines break only between measures and share consistent staff/TAB widths. A measure that cannot fit at readable spacing fails with a diagnostic instead of being split. Tall notation is scaled uniformly to fit the two rows; guitar notation and TAB stay together and aligned. TAB displays frets and ties without duplicate rests; standard notation retains the complete rhythm. The generated `.pages/*.png` files are the same page images displayed by the product.
+Normal browsing uses two music systems per page, with a single system only when the score ends on an odd system. Playback uses overlapping pairs (1–2, 2–3, 3–4): each new active line moves to the top, revealing the next below. The final line moves to the top with an empty lower half. Measure numbers sit above their starting barlines in a clear lane; playback outlines the active measure number in the accent color. Lines break only between measures and share consistent staff/TAB widths. A measure that cannot fit at readable spacing fails with a diagnostic instead of being split. Tall notation is scaled uniformly to fit the two rows; guitar notation and TAB stay together and aligned. TAB displays frets and ties without duplicate rests; standard notation retains the complete rhythm. The generated `.pages/*.png` files are the same page images displayed by the product.
 
-Electric source-melody harmonization retains its existing chord and exact-pitch behavior. Accompaniment uses compact source-derived voicings and sparse rhythm figures, reused at corresponding form positions. Neither accompaniment reads a generated solo or the other guitar's realization. Solos use all six strings; fixed scale maps and the former handwritten solo search/catalogs have been removed. New predictions retain chord, color and resolution checks. No dynamics, accents, velocity shaping or random timing are added.
+Write all music in reading order: repeat bars, numbered endings, navigation jumps, and pass-specific ties are rejected. The performance arrangements, including the held Ending, remain fully written out. Chord labels use `M`, `m`, `aug`, `dim`, `sus`, and `mM7`, including explicit major triads such as `CM`. Recognized Soundslice text-only harmonies are normalized into structured chords; parenthesized chords mark optional turnarounds omitted from the final vocal pass.
 
-Write all music in reading order: repeat bars, numbered endings, navigation jumps, and pass-specific ties are rejected. The seven-section performance arrangements remain fully written out. Chord labels use `M`, `m`, `aug`, `dim`, `sus`, and `mM7`, including explicit major triads such as `CM`. Recognized Soundslice text-only harmonies are normalized into structured chords; parenthesized chords mark optional turnarounds omitted from the final arrangement ending.
+Settings provides independent **Metronome** and **Backing** toggles, both off by default, plus **Tempo**. Song -> Part -> Score navigation stays the same. With both toggles off, pages turn manually. With either enabled, opening a part starts a one-bar count-in and an automatic performance; tap/double-tap page commands are ignored, and hold exits to the part menu. Natural completion also returns there.
 
-The Settings menu's **Playback** toggle plays the selected part once at the menu BPM, suppresses the metronome, and returns to the part menu after natural completion. Manual page turns do not seek or restart audio. Playback is off by default. Python generates shared PCM tones and exact beat timelines; C++/Sonus only loads, schedules, and mixes them. Audio sample sustain loops hold individual notes for their written duration; they never repeat musical passages. No SoundFonts or runtime synthesizer are required. The main menu contains Performance and Settings. Settings holds Playback, Metronome, and Tempo; its values apply for the current session. Navigation is Song -> Part -> Score.
+A performance is an eight-bar intro followed by five complete passes: Vocal Melody, Guitar Melody, Acoustic Solo, Electric Solo, and Vocal Melody. A final Ending section adds one guitar chord attack sustained across three full measures, with vocal rests. The current 32-bar songs contain 171 numbered bars. The ending holds the final performed harmony (GM6 for All of Me, Dm for Autumn Leaves). Optional turnaround chords are omitted from the final vocal pass.
+
+Audio comes exclusively from supplied WAV files: `package/assets/sounds/beat1.wav` accents each measure's first beat, and `beat2.wav` plays its other beats. Backing tracks belong in `package/assets/backing/<song_id>_<bpm>.wav`, for example `all_of_me_120.wav`. The normalized song ID comes from the source filename. Files are copied unchanged; the app never synthesizes notes or stretches recordings.
+
+Tempo choices are 60, 70, 80, 90, 100, 110, and 120 BPM. With Backing on, choices are the tempos available for every catalog song; currently that is 120 BPM. Enabling Backing selects the closest available tempo, preferring the slower choice on a tie. If no tempo is shared, Backing is unavailable. Add recordings at the other tempos and rebuild to enable them.
+
+Backing recordings include one bar of count-in, the complete arrangement including its three-bar held ending, and decay. The existing WAVs are copied unchanged. The recorded count-in plays alone. Metronome clicks stop at the start of the written Ending section; those three measures and the decay remain unclicked. Metronome-only mode supplies its own count-in, waits silently through the ending, and then stops. Playback line turns and measure highlighting follow the rendered audio clock through all 171 measures, including rests and ties; the final line stays visible through the recording decay. Preparation and count-in have no highlighted measure.
 
 Code style is managed by Modulus, using compact two-space formatting. Python imports modules by their full names and qualifies imported symbols; only `from __future__ import annotations` is exempt, and imports have no aliases. Run `./modulus.py format` to format C++ and Python, or `./modulus.py format --check` to check formatting and prohibited aliases. The project-local Neovim integration routes Conform formatting through the same container tools, including format on save and existing LazyVim toggles. Run configure first when the image or exported tooling needs updating. Use `tooling.editor.clang_format` for C++ overrides and `tooling.format.exclude` for project-relative exclusion globs; do not edit generated `.clang-format` files.
 
@@ -153,7 +170,9 @@ See [Score Generation](TECHNICAL_GUIDE.md#score-generation) for supported MusicX
 
 Build-time engraving is implemented in Python in `generator/engraving`, using the bundled Petaluma OTF fonts and SMuFL metadata. MuseScore is an optional authoring tool, not a build dependency or rendering backend. Configure installs the optional Modulus asset dependencies. Exact musical timing is retained through layout, with line breaks between complete measures. Missing glyphs, unrepresentable page geometry, and failed notation audits stop generation before replacing existing assets.
 
-`generator/palette.json` supplies the dark gig theme for Zero, Pico, score pages, and previews. CMake generates the shared C++ colors from the same palette used by Python. Palette changes invalidate the engraving cache without changing music, spacing, or pagination.
+Every normal build also writes printable PDFs to `build/<target>/generated_assets/print/`: written-solo and improvisation versions of each acoustic and electric part, plus one vocal part per song, including every arranged iteration and the ending. Print at actual size on portrait US Letter paper. Pages have white backgrounds, black notation, at least half-inch printer margins, and multiple systems per sheet; guitar parts include TAB. These are fresh 300 DPI renders, without the duplicated lines used by playback transition pages. PDFs use the existing Modulus-managed Pillow dependency, are cached separately, and remain build-only artifacts rather than being installed on the device.
+
+`generator/palette.json` preserves the sheet music’s existing dark palette across Zero, Pico, score pages, and previews. Menus use the same Petaluma Script lettering, quiet rules, tune/composer hierarchy and restrained accent colors for a jazz Real Book feel. Performance status shows Count-in, Playing or Sustain, BPM, page progress and a hold-to-exit hint. CMake generates the shared C++ colors from the same palette used by Python. Palette changes invalidate the engraving cache without changing music, spacing, or pagination.
 
 Petaluma is from Steinberg under the SIL Open Font License. Petaluma supplies music glyphs, and Petaluma Script supplies all visible text on score pages and both device interfaces. The Zero uses packaged `PetalumaScript.otf` for dynamic UI text; the Pico displays prerendered Petaluma Script artwork. Runtime packages retain `OFL.txt`; the music font and engraver stay build-only.
 
@@ -170,3 +189,4 @@ To inspect the fixed artwork independently, run `./modulus.py python -m generato
 ## License
 
 Product Vamp is released under the MIT License. Bundled third-party fonts keep their own licenses next to the font files.
+

SETUP.md

@@ -16,7 +16,7 @@ Copy the tarball to the Zero, verify its `.sha256`, unpack it anywhere, and run:
 sudo zero/install_zero.sh
 ```
 
-The installer loads `zero/product_vamp_zero_linux_arm64.docker.tar`, installs `/etc/product_vamp/vamp.json` if missing, configures NetworkManager AP mode, and enables the systemd units.
+The installer loads and probes `zero/product_vamp_zero_linux_arm64.docker.tar`, replaces Product Vamp configuration with the bundle defaults in `/etc/product_vamp/vamp.json`, recreates the control Wi-Fi profile, and enables the systemd units. Every installation resets Product Vamp data and settings; there is no automatic backup or migration. Use a local console or independent network connection because replacing the control Wi-Fi profile can disconnect the session.
 
 Project-owned install paths, services, executables, and archives use underscores. The checkout and Git URL remain `product-vamp`. On upgrades, the installer validates and probes the incoming image before removing the old hyphenated Product Vamp paths, services, and network profile. Unrelated services, networks, and data are preserved; symlinked installation paths are rejected.
 
@@ -31,9 +31,13 @@ Defaults:
 - Zero IP: `192.168.42.1`
 - API: `http://192.168.42.1:8080`
 
-For API-only quick checks, set `runtime.headless` to `true` or run `vamp_zero --once` from a development build.
+To run the service without its display, set `runtime.headless` to `true`; audio is controlled separately by `audio.enabled`. For a readiness check that validates configuration, catalog, page/audio assets, runtime writes, and server binding without opening hardware, run `python3 modulus.py run` from the configured development checkout. This invokes `vamp_zero --once` with the development configuration and exits after the checks. On an installed Zero, probe the running service with:
 
-Score inputs are tracked under `data/songs/`. The build/package step generates acoustic, electric, and vocal parts, engraved pages, audio, and the read-only catalog database. Installation replaces these assets together; no database migration or runtime score import is needed.
+```sh
+sudo docker exec product_vamp_zero /usr/bin/vamp_zero --config /etc/product_vamp/vamp.json --health-check
+```
+
+Score inputs are tracked under `data/songs/`. The build/package step generates acoustic, electric, and vocal parts, engraved pages, supplied WAV recordings, and the read-only catalog database. Installation replaces these assets together; no database migration or runtime score import is needed.
 
 ## Raspberry Pi Pico 2 W
 
@@ -47,10 +51,14 @@ to the mounted `RPI-RP2` drive.
 
 The default firmware connects to SSID `Vamp Control` with passphrase `vamp-control`, then talks to `192.168.42.1:8080`.
 
-The Pico screen is landscape. Its top bar is blue while connecting, yellow while retrying or when the Zero API is unavailable, and green when the Zero responds. The area below the bar stays blank until a score is active, then it shows the title, selected part, and page count.
+The Pico screen is landscape and uses the shared dark theme. The header's connection indicator is green when connected, amber while connecting or offline, and red on a Wi-Fi initialization error; status text distinguishes connecting, offline, error, and busy states. The body shows the focused menu item, song title, or setting and its value. In score view it shows the title, selected part, performance phase, BPM, page count, and hold-to-exit hint. The palette and Petaluma Script lettering match the engraved sheet music.
 
 Controls:
 
-- Tap: next menu item or next page.
+- Tap: next menu item, or next page when both audio toggles are off.
-- Double tap: previous menu item or previous page.
+- Double tap: previous menu item, or previous page when both audio toggles are off.
-- Tap and hold: select the highlighted menu row, or return from score view to the previous menu.
+- Tap and hold: select the highlighted menu row, enter/leave tempo editing, or return from score view to the part menu. While editing tempo, tap increases BPM and double tap decreases it.
+
+Settings has independent Metronome and Backing toggles. Either starts automatic playback when a part opens; both off enables manual reading. Backing plays the recorded count-in and held ending. Metronome clicks stop at the written three-bar Ending section; page timing continues through the hold. Hold exits at any time.
+
+Keep the two metronome samples under `package/assets/sounds/`. Put song recordings under `package/assets/backing/`, named `<normalized_song_id>_<bpm>.wav`. Supply separate files at 60, 70, 80, 90, 100, 110 and 120 BPM, each with one bar of count-in, the complete eight-bar intro plus five body passes, and its three-bar held ending plus decay (171 written bars for the current songs). Rebuild/package after adding recordings. Backing tempo choices include only tempos supplied for every song; the current package has 120 BPM for both songs.

TECHNICAL_GUIDE.md

@@ -10,8 +10,8 @@ Product Vamp follows the Modulus product layout used by the khodo repos:
 - `cmake/pico.cmake` and `cmake/zero.cmake`: device-specific build definitions selected by the root `CMakeLists.txt`.
 - `include/vamp_style.hpp` and `src/vamp_style.cpp`: shared visual palette. `include/vamp_pico/artwork.hpp` defines the Pico coverage-image and sprite interfaces; fixed artwork is generated in the build directory.
 - `data`: original songs, paired training solos, the frozen solo model, and harmonic/accompaniment policies. Expected-output fixtures live under `tests/`.
-- `generator`: build-time Python rhythm planning, part generation, guitar placement, engraving, audio compilation, Pico artwork, and catalog construction.
+- `generator`: build-time Python source analysis, independent role generation, authored solo reuse, learned prediction, guitar placement, engraving, recording validation, Pico artwork, and catalog construction.
-- `package`: generic bundle inputs, scripts, config, static font assets, and device install files.
+- `package`: generic bundle inputs, scripts, config, fonts, metronome samples and backing recordings, and device install files.
 
 ## Zero Runtime
 
@@ -37,9 +37,9 @@ The AP setup pins NetworkManager to 2.4 GHz channel 6 with WPA2/RSN and CCMP cip
 
 On Raspberry Pi OS Lite, the installed display service owns `tty1`, conflicts with `getty@tty1.service`, and starts the Zero container through `product_vamp_start_zero`. The wrapper switches to the display VT when `chvt` is available and runs `vamp_zero` inside the container through `setsid --wait --ctty` with `/dev/tty1` as stdin, giving raylib's DRM backend a real virtual terminal while keeping Docker attached to the long-running display process. SSH remains the normal admin path and `tty2` is enabled as the fallback local console.
 
-At startup the Zero validates the packaged `catalog.db` application identifier and required table structure, opens it read-only through Noto, and loads menu metadata. It never creates or migrates a database, scans score JSON, or repairs metadata. The main menu contains Performance and Settings. Settings holds Playback, Metronome, and Tempo; its values apply for the current session. Song -> Part -> Score navigation offers Acoustic, Electric, and Vocal without key selection. Musical interpretation, spacing, beaming, ties, accidentals, and pagination happen on the build host. The Zero displays engraved PNGs and retains at most two page textures.
+At startup the Zero validates the packaged `catalog.db` application identifier and required table structure, opens it read-only through Noto, and loads menu metadata. It never creates or migrates a database, scans score JSON, or repairs metadata. The main menu contains Performance and Settings. Settings holds Backing, Metronome, Solos, and Tempo; its values apply for the current session. Song -> Part -> Score navigation offers Acoustic, Electric, and Vocal without key selection. Musical interpretation, spacing, beaming, ties, accidentals, and pagination happen on the build host. The Zero displays engraved PNGs and retains at most two page textures.
 
-Runtime catalogs contain immutable metadata shared by navigation, the HTTP service, and the UI. Song groups reference canonical records by index. SQLite retains complete bundles, while cached measure/page counts let menu queries avoid reading musical JSON. Python constructs the complete database in one transaction before deployment; metadata and embedded JSON are published together after an integrity check. The single `songs` table uses strict column types, primary-key and path uniqueness constraints, and JSON validity checks. Menu rows are ordered by title, composer, part, and ID with binary collation, using the matching `songs_menu` index; C++ groups the ordered rows in one pass without sorting them again. Noto provides read-only connections, bound score lookups, application-ID checks, and quick integrity checks. Product-specific schema and asset-path validation remain in the application. There are no custom rows, runtime imports, metadata-repair triggers, or persisted page-state tables. Selecting a score loads only that bundle asynchronously, discarding stale results after navigation; the view keeps at most two page textures.
+Runtime catalogs contain immutable metadata shared by navigation, the HTTP service, and the UI. Song groups reference canonical records by index. SQLite retains complete bundles, while cached measure/page counts let menu queries avoid reading musical JSON. Python constructs the complete database in one transaction before deployment; metadata and embedded JSON are published together after an integrity check. The `songs` table includes a required integer `solos_enabled` flag (0 or 1), matching the boolean in music metadata. Catalog generation and startup validate paired variants for arranged guitar parts and a single shared Vocal. This schema requires rebuilt assets and updated Zero/Pico software; runtime migration is not supported. The `songs` table uses strict column types, primary-key and path uniqueness constraints, and JSON validity checks. Menu rows are ordered by title, composer, part, and ID with binary collation, using the matching `songs_menu` index; C++ groups the ordered rows in one pass without sorting them again. Noto provides read-only connections, bound score lookups, application-ID checks, and quick integrity checks. Product-specific schema and asset-path validation remain in the application. There are no custom rows, runtime imports, metadata-repair triggers, or persisted page-state tables. Selecting a score loads only that bundle asynchronously, discarding stale results after navigation; the view keeps at most two page textures.
 
 `--once` (also `runtime.exit_after_startup`) is a headless readiness check: it checks database integrity and validates the catalog and referenced page assets, checks runtime-directory writes and the configured server bind, and exits without opening audio/display devices or publishing a readiness heartbeat. Page checks include containment, checksums, CPU-only PNG decoding, and exact 976x484 dimensions. The configured heartbeat destination must be absent or a regular file, never a directory, symlink, or special file. Runtime writes and readiness probes use exclusive sibling temporary files; readiness leaves an existing heartbeat unchanged and removes its probes. `--health-check` instead probes the running service's read-only `/api/health` endpoint using the configured timeout. A stopped/unresponsive service or an invalid response fails the check; it does not create or repair files. `--check-config` remains configuration-only.
 
@@ -49,16 +49,26 @@ The Zero menus use direct taps: release on a row to select it, tap switches to t
 
 In score view, Zero and Pico share one gesture state machine: a second release within 220 ms of the first release produces previous, a hold of at least 700 ms produces select once, and other taps produce next. A late second release preserves both single taps; holding consumes a pending first tap. Zero schedules pending-tap deadlines without polling and discards canceled or obsolete timer callbacks.
 
+Score loading uses a dedicated single-worker executor for SQLite reads and JSON parsing. Completion returns to the control executor. The loader retains one in-flight request, coalesces pending selections, and discards stale results, keeping database work off HTTP and playback polling.
+
 ## Score Generation
 
 The build runs `./modulus.py python -m generator` against `data/` and writes build-only assets under the CMake binary tree. Every song produces acoustic, electric, and vocal parts in its authored musical key. Source counts are discovered, not fixed in code. Generator-only optional chord markers are consumed before publishing.
 
-Songs live directly in `data/songs/`. Each source produces acoustic, electric, and vocal parts under `songs/`, with matching page directories. Adding, renaming, or removing a source requires no registry edit. Regeneration replaces the output tree and catalog. The two current songs generate six parts.
+Songs live directly in `data/songs/`. Each source produces acoustic, electric, and vocal parts under `songs/`, with matching page directories. Adding, renaming, or removing a source requires no registry edit. Regeneration replaces the output tree and catalog. Each song generates five bundles: the usual three parts and acoustic/electric `_improv` alternatives. The two current songs generate ten bundles. Solos defaults to On at each app start; Off selects the improvisation guitar variants while Vocal is shared. Part menus retain three rows, using the selected variant’s normal and playback page counts. Pico status includes `solos=0/1`.
 
-The pipeline is Soundslice/MuseScore -> uncompressed `.xml` -> musical model and rhythm plans -> part generation -> semantic JSON, audio, engraved pages, and Pico title artwork -> packaged SQLite catalog and assets. Export MusicXML 3.1 or 4.0, `score-partwise`, with one part and one staff. The musical model retains independent voices and per-tone durations/ties. Packaged music describes each generated part, not an archival copy of the XML.
+The pipeline is Soundslice/MuseScore -> uncompressed `.xml` -> musical model and rhythm plans -> part generation -> semantic JSON, supplied WAVs, engraved pages, and Pico title artwork -> packaged SQLite catalog and assets. Export MusicXML 3.1 or 4.0, `score-partwise`, with one part and one staff. The musical model retains independent voices and per-tone durations/ties. Packaged music describes each generated part, not an archival copy of the XML.
 
 ### Musical Data
 
+Generator dependencies run from pure musical values and source analysis toward composition, assembly, and publication. `model` contains primitive values; `validation` checks score documents; `score` contains pure score operations; and `score_io` owns bundle/file validation. `source` owns canonical source identity and immutable source events. `arrange.form.FormPlan` supplies source-derived sections, ownership windows, and pickup stops to both accompaniment and role assembly.
+
+`composition` generates one role from an explicit immutable `vocabulary.CompositionContext`; `production` schedules typed worker requests and publishes the assembled parts. Lower musical functions receive that context explicitly. The scoped vocabulary API remains available for standalone callers and tests. Cached vocabulary loads return owned copies so one caller cannot alter another caller's resources.
+
+Role cache identities include the transitive generator imports of the composition entrypoint. Engraving cache identities similarly include renderer dependencies. Cache identities never seed musical choices. `cache` provides shared content identity and staged-file publication mechanics, while role, PNG, and PDF adapters retain their own validation. `engraving.worker` owns rendering and its bounded image cache; `engraving.pipeline` groups and schedules work. `performance` contains pure timing contracts, and `audio_assets` validates and copies supplied recordings.
+
+The reviewed output contract is checked by `tests.test_output_contract` for complete synthetic role scores, including learned solos, and by `python -m tests.asset_contract <generated-assets-directory>` for every non-diagnostic asset. CTest runs the asset check against the normal build output. Musical output, JSON/catalog serialization, PNGs, and PDFs must remain unchanged during structural refactors; diagnostic text and cache identities may change.
+
 `data/songs/` contains authoritative source scores. `data/training/manifest.json` pairs each authored solo with a source and a slot (1 or 2), and declares its pitch convention. Both slots train **one pooled model**. Slot 1 maps to acoustic playback and slot 2 to electric playback; slot and instrument names are not model features. The four XML files retain their notes, harmony, timing, ties, rests, Autumn Leaves pickup and All of Me's full-bar opening. Explanatory labels such as `Lead in`, `b9` and `6` were removed; they were not model features. Written chord extensions remain intact.
 
 `data/models/solo.json` is an explicitly trained, versioned statistical model. Its short-context counts, rhythm and register statistics are learned from the paired examples. Provenance records input hashes and the training version. This is supervised use of the supplied solos, not independence from them and not a neural network pretrained on an external corpus.
@@ -93,17 +103,19 @@ Standard MusicXML kinds and explicit degrees remain authoritative. For Soundslic
 
 Musical genres and player-style presets are removed. There is no default genre, alias table, generated `style` metadata, or genre selection. The strict importer rejects every obsolete `vamp:style` miscellaneous-field hint, including an empty value, instead of discarding it. The XML writer does not emit that field. Keys, modes, keyless notation, and local changes remain preserved; there is no transposition reference or `vamp:tonic` property. Visual styling, font attributes, engraving `Style`, presentation fingerprints, and ordinary `<bar-style>` notation are unrelated and remain supported.
 
-Parenthesized chord symbols mark optional turnarounds, including Soundslice's text-only `(Fm7)` and `(Bb7)` exports. Alternatively, place staff text `vamp:optional-chord` at the same onset as exactly one harmony; this text may be hidden. Earlier arrangement sections retain those chords and their parentheses; the final ending omits them. The normalized harmony stores `parenthesized` display metadata separately from the generator-only `chord_optional` flag. Orphaned/duplicate markers and unknown `vamp:` properties are errors; consumed directives never enter runtime JSON.
+Parenthesized chord symbols mark optional turnarounds, including Soundslice's text-only `(Fm7)` and `(Bb7)` exports. Alternatively, place staff text `vamp:optional-chord` at the same onset as exactly one harmony; this text may be hidden. Earlier arrangement sections retain those chords and their parentheses; the final vocal pass omits them. The normalized harmony stores `parenthesized` display metadata separately from the generator-only `chord_optional` flag. Orphaned/duplicate markers and unknown `vamp:` properties are errors; consumed directives never enter runtime JSON.
 
 ### Linear Reading Order
 
-There is no repeat analyzer, expansion pass, or repeat/ending engraver. Performance retains the seven-section arranger, writing every section explicitly onto successive pages; neither display nor playback jumps backward.
+There is no repeat analyzer, expansion pass, or repeat/ending engraver. Performance writes an eight-bar intro and five full passes (vocal, guitar melody, acoustic solo, electric solo, vocal), followed by three held ending measures, explicitly onto successive pages; neither display nor playback jumps backward.
 
-Repeat signs, numbered endings, D.C./D.S./coda navigation, and `time-only` ties fail import rather than being ignored or expanded. Write the intended notes and rests explicitly, including any alternate ending. Legacy repeat fields in generated musical documents are rejected; regenerate assets instead of retaining a compatibility interpreter. Full leading-rest measures remain full measures, not inferred pickups. Missing titles fall back to filenames and missing composer credits remain empty, as with the current Soundslice exports.
+Repeat signs, numbered endings, D.C./D.S./coda navigation, and `time-only` ties fail import rather than being ignored or expanded. Write the intended notes and rests explicitly, including any alternate ending. Legacy repeat fields in generated musical documents are rejected; regenerate assets instead of retaining a compatibility interpreter. Full leading-rest measures remain full measures, not inferred pickups. Missing titles fall back to readable, spaced filename titles and missing composer credits remain empty, as with the current Soundslice exports.
 
-An explicitly marked opening pickup is excluded from the repeated song body. At each vocal or electric melody entry, it is right-aligned within the preceding arrangement bar. Outgoing music continues until the pickup's first sounding note, then releases as the incoming melody takes over; pickup rests do not erase earlier accompaniment. Lyrics, ties into the following downbeat, tuplets, and octave spans retain their timing. Pickup harmony changes also apply to the accompanying parts at their relocated offsets. All parts share the same bar count, and a pickup longer than its destination bar fails generation. Autumn Leaves uses a four-beat padded pickup and a 32-bar body, yielding 176 arrangement bars rather than 181.
+An explicitly marked opening pickup is excluded from the repeated song body. At each vocal or electric melody entry, it is right-aligned within the preceding arrangement bar. Outgoing music continues until the pickup's first sounding note, then releases as the incoming melody takes over; pickup rests do not erase earlier accompaniment. Lyrics, ties into the following downbeat, tuplets, and octave spans retain their timing. Pickup harmony changes also apply to the accompanying parts at their relocated offsets. All parts share the same bar count, and a pickup longer than its destination bar fails generation. Autumn Leaves uses a four-beat padded pickup and a 32-bar body, yielding 171 arrangement bars: the eight-bar intro, five 32-bar passes, and three written held-ending measures.
 
-Ordinary ties still link adjacent matching tones across measures and systems. Tuplets, independent voices, local staff changes, and manual page navigation are unchanged. Audio sample sustain loops remain an implementation detail for holding a single note, not a score-repeat feature.
+Improvisation assembly reuses the independent accompaniment and melody roles and fills each source-defined solo window with regular beat slashes. The complete pickup allocation also becomes slashes, including rests within that allocation; music before a partial pickup and subsequent melody handoffs are preserved. Slash events carry no pitches or fret positions, and render in both the staff and TAB. This does not rerun solo composition or change backing recordings, measure numbering, harmony, or performance duration. Both variants receive normal/playback pages and build-only printable PDFs.
+
+Ordinary ties still link adjacent matching tones across measures and systems. Tuplets, independent voices, local staff changes, and manual page navigation are unchanged. The recording’s three-bar sustain tail has no added score section or metronome clicks.
 
 ### Runtime Schema
 
@@ -111,7 +123,7 @@ A newly generated bundle has three top-level fields:
 
 - `music`: metadata and a single staff containing measures, independent voice lanes, timed harmonies/attributes/tempos, instruments, tuplets, and octave spans.
 - `presentation`: engraver name, style checksum, fixed width/height, ordered relative PNG paths with SHA-256 checksums, and the `title_image` content identifier for Pico artwork.
-- `playback`: availability, exact performed beat duration/events, shared PCM sample descriptors, and a generated release envelope. Unavailable interpretations carry a reason instead of an approximate timeline.
+- `performance`: normalized source song ID, exact score duration, count-in length and metronome beat/accent positions. Each presentation page additionally stores `first_measure`, `measure_count`, `start_beat` and `end_beat`, derived from its actual engraved systems. These ranges must cover every measure once, without gaps.
 
 There are no authored-key/selected-key metadata pairs or musical genre/style fields. The score's key metadata describes its actual music, and menus display that key alongside its part name. Instrument normalization transforms explicit key cancellations with the same interval as the notes. Presentation's `style_sha256` identifies visual engraving settings and fonts, not a musical genre.
 
@@ -121,25 +133,98 @@ Measure durations, event onsets/durations, harmony offsets, span times, and pitc
 
 `production.generate_role` accepts immutable canonical source analysis, a role, production data and settings. Canonical musical identity excludes display titles and filenames. Accompaniment, source melody, vocals and each solo have their own caches and can run in independent worker processes. No role reads another generated role. Assembly applies source-defined participation windows, including guitar-role exclusions and returning pickups, without changing another role's decisions.
 
-`parts.py` retains exact source notes, rests, lyrics and ties. `melody.py` harmonizes sustained electric source-melody notes using their actual chord, without changing the head's pitch or rhythm. Guitar notation applies its octave transposition once. `comping.Canonical` independently chooses compact chord voicings and rhythm from the source form and shared policy, then reuses that part at the same source-form position throughout the arrangement. There is no generated-solo response or reuse of a generated melody's grips. Sparse bars and deliberate releases are legal.
+`parts.py` retains exact source notes, rests, lyrics and ties. `melody.py` harmonizes eligible sustained electric source-melody notes using their actual chord, preserving source pitches while splitting and reattacking held notes at harmony changes. Guitar notation applies its octave transposition once. `comping.Canonical` independently chooses compact chord voicings and rhythm from the source form and shared policy, then reuses that part at the same source-form position throughout the arrangement. There is no generated-solo response or reuse of a generated melody's grips. Sparse bars and deliberate releases are legal.
 
-Monophonic solos use strings 1–6 and frets 1–12, with no open strings or acoustic/electric string restrictions. Placement compares complete paths through legal positions, accounting for hand-region changes, travel time and string crossings. Ties retain their position. Chord placement retains its separate policy: frets 1–20 with a sounding anchor at fret 12 or below and the existing inclusive three-fret span. These costs approximate physical feasibility; they do not assign fingers or guarantee comfortable playing.
+Electric accompaniment shares the electric melody's `guitar.chord_positions(..., profile='electric')` families: strings 1–3 or 1–4, with a preference for four-note grips. `chord_families.py` classifies actual shapes as drop-two, close-position, compact-open, compact-doubled or three-note shells, ranked by versioned policy. Electric melody chords must also use strings 1–3 or 1–4, always including string 1. The exact source pitch is retained even when a legal supporting chord tone must sound above it. Within the selected span tier, the melody's lowest feasible string number and foreground position precede chord size. A primary grip with the melody on string 2 beats a four-fret grip with it on string 1. Short monophonic melody notes and solos use the shared hand-window placement objective. A harmonization with no legal chord retains the existing reported single-note fallback. These electric roles use the same written chord requirements without taking responsibility for the slash bass. Acoustic accompaniment uses `acoustic_voicing.candidates`: exactly one bass on string 5/6 and one upper note each on strings 4/3/2. Without an explicit slash bass or inversion, its bass is a root or written fifth (including an altered fifth); plain major/minor triads also permit the third. Plain triads retain all three chord members. Explicit bass/inversion requirements override this choice.
 
-### Playback Boundary
+Acoustic families are transposable degree orders for triads, tetrads and extended chords, with root/fifth alternatives on both bass strings. Slots resolve to the written third/suspension, sixth/seventh, fifth and extension, including alterations and omissions. They are not per-song grip tables. Only harmonically and physically valid realizations enter selection; unavailable bass/inversion variants are discarded. Coverage checks exercise 35 common qualities in all 12 keys. Four-fret fallbacks support previously unavailable acoustic Bb9, BbM9, Bb7#9 and Bb6add9; for example, Bb9 uses fifth-bass frets 8/10/7/9 on strings 5/4/3/2. Acoustic Bbm9 and Bm9 still have no grip meeting all constraints. Unsupported chords fail with source measure, beat and chord rather than silently omitting defining tones.
 
-All musical compilation and tone synthesis run in Python (`generator/playback.py`). Each transformed score receives a sorted timeline of independent tone events with rational quarter-note `onset` and `end` positions. Ties become one continuous attack. Rests and spacers remain silence, including trailing silence. Harmony labels are not auto-accompaniment: only pitches actually present in the selected arrangement sound. Cue notes are silent, hidden pitched notes sound, and pitchless rhythm slashes are silent.
+The active voicing selector compares complete paths with dynamic programming over concrete grips. The leading cost minimizes four-fret grip selections: every span of one, two or three positions has zero cost in this tier. Existing criteria then apply within the tier. Electric accompaniment prioritizes chord size and shared family rank; acoustic prefers declared degree-order families on either bass string. Both then minimize hand-window travel, the passage maximum hand position, secondary finger/grip effort, missing optional tones and sounding-register distance, in that order. Stable role-specific keys resolve equal costs. Since accompaniment transitions have travel costs but no hard travel cutoff, four-fret candidates are safely removed from a span when any primary grip exists; this exact dominance reduction keeps the repeat search small. Electric placement retains distinct alternatives where future held/tied notes can constrain feasibility, and does not charge tied continuations as fresh grip choices. The remaining acoustic placement helper uses the same leading fallback tier.
 
-The `soft_struck` profile generates shared mono 48 kHz PCM16 WAVs with a short attack/decay and a seamless sustain loop, plus a shared 50 ms release envelope. Exact sounding pitches, including supported microtones, determine samples. Instrument transposition is applied once from the generated musical model, without role- or filename-dependent octave heuristics. Samples are shared across scores and voices by content identity. There are no SoundFonts, bundled instrument samples, oscillators, or MusicXML parsing at runtime.
+Silent spans retain the preceding hand position. A continuous accompaniment repeat includes the last-to-first transition. Source-harmony variants, such as relocated pickup harmony, are solved between fixed canonical neighbors without mutating a shared previous-grip state; request order cannot change them. Per-span composition reports include candidate counts, selected family, bass/string family availability, selected positions, bass degree and movement, plus `fret_span`, `span_group` and `span_candidates` (primary/fallback counts before dominance reduction). These are diagnostic sidecar fields; the score/runtime format is unchanged.
 
-Grace groups honor explicit steal-time-previous/following or make-time; unspecified groups share 25 percent of the following event. Conflicting grace modes, trailing grace groups, unsupported unpitched sound, tied measured tremolo, and incompatible paired tremolo make playback explicitly unavailable while retaining the notation. Measured tremolo uses written subdivision and tuplet ratios. The tone range is 20-18,000 Hz. Asset validation checks containment, SHA-256, PCM format/length, loop endpoints, envelope shape, event ordering, references, and rational timing. Generation remains transactional if validation fails.
+`comping_rhythm.py` selects recurring source-only figures from version 6 of `composition/accompaniment.json`. The normal acoustic cycle has two two-bar slots; electric has four. Each template and anticipation mode is evaluated at every occurrence of its slot before retrieval limits apply. Up to eight diverse candidates per slot enter bounded complete-cycle comparison. Distinct slots differ in attack/duration relationships, not just IDs, bass/upper assignments or anticipation flags. Repetition is not penalized. The source melody and harmonic rhythm affect selection; title, iteration, generated solo and tempo do not. `groove_cycles` reports slot identities, anticipation modes, aggregate cost, compared cycles and any reduction caused by vocabulary or form length.
 
-When grace groups steal from both ends of one donor, their adjustments accumulate against its original duration. Consuming the entire donor or any chord tone makes playback unavailable rather than overlapping the grace group and principal note.
+Meter-specific full patterns declare `meter` and, for 5/4, `grouping: [3, 2]` or `[2, 3]`; `meter: partial` marks the bounded one-bar pickup vocabulary. The loader checks nominal lengths, grouping, quarter/eighth arithmetic, electric lane overlap and distinct acoustic/electric onset patterns. Acoustic lane overlap is checked after automatic assignment. Each role has 24 distinct attack/duration figures per selectable pool: 4/4 (11 families), 3/4 (12 families), and each 5/4 grouping (12 families). The 4/4 pool removes one duplicate acoustic figure, develops another previously duplicated answer, and adds three electric variants. Patterns are independently authored from general principles, with research URLs and provenance in the resource, rather than transcriptions. The counts describe available choices, not required musical activity.
 
-At runtime the shared on-demand score loader serves both display and audio. A background worker validates and loads only the selected score's samples, converts absolute beat positions to rounded sample frames at the menu BPM, and prepares a Sonus sequence. Pitch does not change with BPM, and rounding absolute positions avoids cumulative timing drift. The I/O loop publishes the prepared sequence and polls completion without doing disk I/O or musical interpretation. Sonus mixes preallocated voices from immutable PCM and retires them on control threads; the callback never synthesizes tones or loads files. Limits are one million events, 512 distinct samples, 64 MiB decoded samples per selection, 64 simultaneous voices including releases, and 24 hours at the selected BPM. Exceeding them reports an error rather than dropping notes.
+`grouping_for` is a pure source-only decision. It honors written 3+2 or 2+3, otherwise compares actual chord changes at the secondary boundary, then melody entrances following at least an eighth-note rest, then other melody attacks; equal evidence defaults to 3+2. An unchanged chord marking is not a change. The selection stays fixed through a continuous meter section, and each role derives it independently from the same source. Fills cannot borrow another grouping. No accents or playback swing changes are inferred. `groove_cycles` additionally reports grouping, its evidence and the eligible pattern count. Equal-duration compound meters (such as 6/8) do not borrow the 3/4 vocabulary; unsupported comping meters fail explicitly.
 
-`audio.playback_enabled` defaults to false. The obsolete `track_enabled` setting is rejected with a diagnostic naming its replacement. Playback suppresses the metronome without changing its stored toggle. Authored tempo data remains in the musical document but is not printed; menu BPM is constant throughout a session. Selecting a part starts a new session. Manual page changes never restart or seek. Natural completion, including trailing silence/release, returns to the corresponding part menu with its selection preserved. Leaving the score cancels playback; stale results cannot close a newer score. Errors remain visible rather than being mistaken for completion.
+Accompaniment dependency keys and choice streams include only full-pattern meters and partial lengths present in that source, plus the applicable instrument policy. Adding unused-meter patterns therefore does not reseed an existing song; unrelated role policies remain excluded. Research prose is excluded from musical seeds. This schema migration changes the accompaniment choice-key domain once; it does not promise identical previous 4/4 tie-break winners. Source melodies, registered solos, notation format and playback count-in are unchanged.
 
-`--once` additionally validates referenced audio without opening display/audio hardware. Runtime packages contain WAVs alongside pages and the SQLite catalog; the source XML and Python dependencies remain build-only.
+The cycle begins at the first full source-body measure. A partial-form entry first fits the incoming harmony, then slices the sounded figure instead of restarting it; this prevents a previously clipped chord hold from reappearing as a new attack. An odd final bar uses the appropriate half-pattern. Meter changes restart a compatible cycle. Unsupported durations fail explicitly, and limited vocabularies report a shorter cycle. Optional final-harmony changes rebind the same selected pattern. Canonical grips are fixed wherever the chord and interval start remain unchanged, so a new turnaround cannot move neighboring unaffected voicings.
+
+Fills replace only a two-bar figure's ending bar, using a same-family variation that improves the existing source-fit objective. Eligible boundaries are source section endings, form endings, or a melody breath of at least one quarter-note beat at the block end. Carried events from the opening bar are protected. The largest improvements are retained with an eight-measure minimum separation around the whole cyclic form, equally for both roles; no fill quota applies. Fills do not change cycle phase and recur at identical source positions. Reports record the base slot, selected fill, source index and reason.
+
+Normal acoustic patterns use 1–3 attack positions per full bar. Their event layer is `acoustic`; the instrument's versioned `attack_assignment` policy is `combined_except_eighth_pairs`. Authored bass/upper decisions are replaced by chronological assignment across the complete canonical source form: ordinary attacks are combined, while adjacent half-beat-separated attacks in the same effective harmony become bass then upper. Pairs do not overlap; an odd leftover attack is combined. Bar and two-bar pattern boundaries do not reset pairing. Source-defined entries, handoffs and chorus boundaries do; the same boundary set applies at repeated form positions. Chord changes break pairs, but repeated identical chord labels do not. An anticipation uses its upcoming harmonic region for this decision. First attacks of new harmonic spans still include bass (`first_attack: bass_per_harmony`).
+
+Quarter/eighth articulations and explicitly authored sustains release at the earliest of their intended end, the next acoustic attack in either lane, or a required harmonic/handoff boundary. Bass cannot ring beneath a later upper attack, nor can an upper chord ring beneath a later bass attack. Combined attacks have one shared release. Shortening that would strand an anticipation first rebinds it to current harmony and recomputes assignment. `acoustic_context` and `realize_acoustic_form` implement the shared source-form contract used by cycle comparison, fill selection and final source-harmony variants, before grip placement or notation splitting. Raw candidates retain authored releases so a changed neighboring attack can be evaluated afresh. Fills must improve the resolved whole-form cost without changing events outside their editable second bar. Concrete block caches include their assigned event signature: a neighboring chord change can break a pair even when this block's own harmony is unchanged. The renderer verifies actual sounding lane events against the selected assignment. The completed validator follows sounding tie chains and rejects any overlap between distinct acoustic attacks, even across different lanes. Placement validates all simultaneously sounding strings, fret spans and anchors. Bass and upper events merge into one notated part. Uninterrupted cross-bar holds and the combined three-bar closing sustain remain intact.
+
+Electric normally uses 0–2 attack positions per bar, with stabs, sustains and space. Direct events release at changed harmony. An eligible acoustic/electric template can anticipate a chord by half a beat when it continues into that chord; the grip comes from the upcoming harmony and releases by its following change. Acoustic anticipations also include bass on the new chord's first attack. Missing harmony remains silent. Protected participation limits retain the chosen figure, shortening only its affected releases and recording exceptions. Cross-bar sounding notes become ties, never automatic merging of repeated attacks.
+
+All six production roles receive reuse audits. Musical comparisons include rests, pitch durations, ties and TAB while ignoring regenerated event IDs and global measure numbers. Source-defined pickup/participation changes and effective final-turnaround harmony are reported separately. The final three-bar hold remains instrumental.
+
+Named activity, melody-support, harmonic-support and articulation contributions evaluate complete figures. Cycle selection aggregates them over all occurrences, retaining family diversity before the final bounded comparison. `groove_cycles.retrieval_cost` is the isolated-figure estimate used for shortlisting. `cost` is the actual selected base cycle's cost before optional fills; acoustic cycles are assigned and released together before this comparison. Acoustic `final_cost` reports the complete source-form result after fills and any meter-section joins. The selected score is never overwritten by a post-selection report. Reports identify the base slot, realized pattern, projected events, continuations, anticipation target times and source-context exceptions. Tempo affects metadata/cache identity but never musical choice. This release correction leaves musical data and role choice seeds unchanged; code fingerprints invalidate stale concrete caches.
+
+Acoustic events use temporary bass/upper notation lanes for projection; then `acoustic.combine` converts their sounding intervals to one rhythmic voice before assembly. It splits notes at attack/release boundaries and ties only continuing pitches; repeated attacks stay separate. Staff and TAB share the result, with no competing rests or opposing layer stems. This preserves exact sounding durations instead of truncating overlaps. The older truncating `prepare` helper serves only the noncanonical legacy/test path. The product build audits canonical accompaniment reuse across arrangement iterations, allowing only source-derived changes in harmony, pickups and role handoffs. The `.roles` sidecars report pattern IDs, exact projected layer events, continuations, anticipation targets and exceptions. Their `selection_cost` records the complete figure’s local contributions, repeated on each projected bar for context; the cycle selector aggregates these by the number of measures in each occurrence, without an anti-repetition transition cost. Accompaniment dependency keys contain only that instrument's policy; electric melody consumes the shared electric voicing families. Solos retain their existing learned source-melody/harmony conditioning and registered exact performances.
+
+Acoustic sequential-release verification on October 7, 2026: `python3 modulus.py python -m unittest discover -s tests -p 'test_*.py'` passed 451 tests in 162.318 seconds. Coverage includes release in both lane directions, cross-bar pairing in 3/4, 4/4 and both 5/4 groupings, expired and surviving anticipations, complete-cycle ranking versus isolated retrieval, fills protecting neighboring assignments, malformed overlap before/after notation merging, canonical reuse, cold/warm caches and parallel roles. `python3 modulus.py format --check` passed. `python3 modulus.py build --target x86_64_gcc12_debug --target arm_64_gcc12_release` passed both generator/asset checks; 394 score JSON/PNG artifacts match across targets. Acoustic composition took 2.1 seconds for All of Me and 2.2 seconds for Autumn Leaves; native assets finished in 31 seconds and the cached ARM build in 11 seconds. Compared with the saved pre-change output, all attack pitches, onsets and TAB positions are identical: only 67 Autumn Leaves bass releases are shorter. Its arrangement measure 3 now contains bass [0, 1/2), upper chord [1/2, 3/2), and combined chord [2, 3), without ties. All of Me and the ten other-role documents are unchanged. Canonical reuse audits pass, and both three-bar instrumental endings retain one twelve-beat attack. Both acoustic opening pages were visually inspected; no audio audition was performed. The comparison baseline remains outside the repository under `/tmp/vamp-sequential-acoustic-yt_irkfd`.
+
+Monophonic solos use strings 1–6 and frets 1–12, with no open strings or acoustic/electric string restrictions. Placement compares complete paths through legal positions, accounting for hand-region changes, travel time and string crossings. Ties retain their position. Chord placement retains its separate policy: frets 1–20, a sounding anchor on frets 1–12, primary spans of up to three consecutive positions and fallback spans of exactly four. The span is inclusive: 7–9 is three positions, 7–10 is four. Five positions are always invalid. The anchor must remain sounding, including after shorter notes release. Acoustic accompaniment additionally keeps its bass within frets 1–12 and requires an independent anchor in its upper three-note cluster, preserving the option of separate bass/upper attacks. These costs approximate physical feasibility; they do not assign fingers or guarantee comfortable playing.
+
+Guitar placement shares the `hand-window-2` model in `generator/hand_position.py`. **Both chords and single notes** may occupy any legal four-consecutive-fret window. Position 7 covers frets 7–10; a chord occupying only frets 7–8 can use positions 5, 6 or 7, according to its neighbors. Position is the bottom of the selected window, not necessarily an occupied fret. This is an approximate positioning model, not an anatomical finger assignment. Sounded-note limits remain binding: a solo note cannot exceed fret 12 merely because its hand window extends farther.
+
+After the existing compact-span, chord-family and electric-melody foreground priorities, selection minimizes **total absolute hand-position travel**, then the **maximum hand position across the complete path**. Finger changes and string crossings within the same window contribute zero travel. Only after those two criteria tie do relative finger adjustment, string crossings, grip complexity and time pressure decide between paths. Acoustic bass is one voice even across strings 5 and 6; upper voices follow their strings. Added/removed chord voices contribute one secondary adjustment each. Solo secondary effort uses elapsed time from the preceding attack, not a tied notation fragment. A genuine continuation retains pitch, TAB and hand position without another movement charge.
+
+A low fret is not rewarded independently at every event. All three voicing policies retain only `register_weight` (default 1), weighting distance from their sounding-register center after travel, peak position, secondary effort and optional-tone coverage. Centers remain 53 for acoustic accompaniment and 60 for electric accompaniment/melody. Exact melody and solo pitches are never transposed to meet these preferences. Canonical accompaniment selection includes source-form repeat joins and solves source-derived variations between actual selected neighboring grips **and hand positions**. Unchanged intervals retain both. The instrumental ending receives the last accompaniment grip and selected window; it does not infer the hand from the lowest occupied fret.
+
+The dynamic programs retain concrete grips/held strings, selected hand position and distinct earlier peaks. A later high position can equalize two earlier peaks and expose a subordinate tie-break, so retaining only one lowest-peak history is unsafe. Strictly worse primary travel at an identical endpoint/hand can still be discarded. Equal fret averages do not imply equal placement states. Stable ordering resolves equal complete costs.
+
+Role reports expose `total_travel`, `shift_count`, `largest_shift`, `peak_position` and actual selected hand positions. Accompaniment costs separate travel from secondary `effort`; individual spans separate hand shift, finger adjustment, string changes, grip effort and register tie-breaking. Repeat travel is assigned to the first canonical span. Solo reports call the time-weighted subordinate value `secondary_effort`. Cache fingerprints include placement code while accompaniment rhythm seeds remain unchanged. Every role remains independently composed; the solo model, pitches and rhythms are unchanged.
+
+The pre-window baseline is preserved outside production inputs under `/tmp/vamp-hand-window-baseline-dwh4anzh`. Verification compares sounded content and rhythms separately from TAB, and checks all electric melody chords for strings 1–3 or 1–4. Independent exhaustive test oracles cover complete grip/window paths, fixed neighbors, cyclic repeats and single-note paths; tests also cover a greedy-position trap, future peak equalization, zero-cost within-window string changes and tied-event invariance.
+
+Hand-window verification on October 8, 2026: `python3 modulus.py python -m unittest discover -s tests -p 'test_*.py'` passed 472 tests in 100.928 seconds. The subsequent asset build exposed a print-footer overlap: Letter pagination had reserved a nominal font height instead of the actual page-number outline. The new tight-page regression reproduced that failure, and `python3 modulus.py python -m unittest tests.test_printing` passed all eight tests after measured footer bounds replaced the estimate. Content and collision audits remain enabled.
+
+`python3 modulus.py build --target x86_64_gcc12_debug --target arm_64_gcc12_release` then passed both asset checks and produced six complete PDFs (42 pages). Native regeneration took 33 seconds with cached roles and a cold engraving namespace; ARM reused the assets in six seconds. Before the footer repair, uncached composition completed in five seconds for all twelve roles, with solos taking about 0.5 seconds each. `python3 modulus.py test --target x86_64_gcc12_debug` passed in 13.27 seconds. All 400 score JSON, normal/playback PNG, report and PDF files match between native and ARM.
+
+Against the saved pre-window baseline, all four solos retain exactly the same pitches, timing and ties; vocal documents and accompaniment rhythm reports remain identical. Electric melody retains its exact source notes and timing; supporting chord tones may change to enforce string 1. All 46 All of Me and 33 Autumn Leaves electric melody chord fragments now use strings 1–3 or 1–4; previously 13 and three fragments respectively omitted string 1. Short monophonic melody remains permissible.
+
+| Solo | Total hand travel, before → after | Number of shifts, before → after | Peak hand position, before → after |
+|---|---:|---:|---:|
+| All of Me acoustic | 131 → 12 | 87 → 10 | 9 → 7 |
+| All of Me electric | 128 → 9 | 87 → 7 | 9 → 7 |
+| Autumn Leaves acoustic | 141 → 2 | 85 → 2 | 9 → 3 |
+| Autumn Leaves electric | 106 → 6 | 77 → 6 | 9 → 5 |
+
+Travel is the sum of absolute window-position changes, measured in frets. An independent dynamic program over every legal string/fret/window confirmed that each new solo achieves the minimum raw travel for its unchanged sounding notes. This verifies the placement objective, not anatomical comfort or musical preference. The canonical accompaniment cycles now travel 10/14 frets for All of Me/Autumn Leaves acoustic, and 0/6 for electric; they retain source-based repeat consistency. No listening assessment was performed.
+
+The sibling Modulus Conan helper was repaired to call `modulus.build_context.build_context`, replacing the removed `modulus.cli.build_context` API. Its Python suite passed 132 tests with four existing skips. A locally exported and built Modulus package passed `python3 modulus.py configure --target x86_64_gcc12_release`, followed by the release build and runtime test above. The offline attempt first identified missing cached Ligo sources; allowing ordinary Conan source retrieval completed configuration. An unwritable generated engraving cache was also restored to the build user's ownership. The Modulus source fix must be published through that repository's normal release process before remote CI using `modulus/latest` receives it; no remote package was published during this repair.
+
+Recurring-groove verification on October 6, 2026: the full Python regression command passed 423 tests in 149.85 seconds. After the final partial-entry grip fix and one additional real-evaluator fill test, the focused accompaniment/arrangement command passed 59 tests in 21.64 seconds. The native test target passed in 30.58 seconds. The initial native rebuild took 42 seconds, including 7 seconds for independently generated roles and 33.62 seconds of engraving; accompaniment roles took 0.8–1.6 seconds each. The following ARM build reused all role/score caches and took 11 seconds. All four solo documents, both electric melodies and both vocals exactly matched the prechange baseline; native/ARM score assets matched byte for byte. The chosen base cycles differed by song and retained all two/four requested slots. Autumn Leaves selected the authored overlapping-quarter acoustic pattern. No optional fill improved these production winners, so no fills were forced; independently authored tests verify real source-fit selection of a fill. An acoustic score/TAB page was visually checked for overlap ties in a single part. Recordings were unchanged and no listening assessment was performed. Baseline snapshots stayed under `/tmp/`.
+
+Shared-preference and bass-first verification on October 6, 2026: `python3 modulus.py python -m unittest discover -s tests -p 'test_*.py'` passed 408 tests in 61.35 seconds, and the native test target passed all 124 cases. Both product builds passed asset checks; 139 native/ARM score and page files matched byte for byte. All 273 acoustic measures containing new attacks started with bass, including 69 delayed entrances. Tied carries did not trigger new attacks. Both solos, vocals, electric accompaniment rhythms, and electric foreground melody pitches/timing/TAB matched the saved baseline exactly. Some All of Me melody support chords changed; Autumn Leaves' complete electric melody document did not. All of Me's canonical electric accompaniment mean fret dropped from 7.26 to 4.13 and movement cost from 57 to 48; Autumn Leaves stayed at 2.95 and 46. Recordings were unchanged. These are score checks and page inspections, not a listening assessment.
+
+With no concurrent build, isolated cold role generation measured 0.56–0.57 seconds for acoustic accompaniment, 0.35–0.37 for electric accompaniment, and 0.26–0.29 for electric melody; cache hits took 0.02–0.04 seconds and matched clean regeneration for both songs. The normal two-worker native asset stage took about 14 seconds (4 composing, 9 engraving, 1 other); ARM reused the caches in about 4 seconds. Application compilation is separate from these asset-stage timings.
+
+Lower-position verification on October 6, 2026: all 400 Python tests and 124 native test cases passed. Native and ARM builds passed asset checks; all 139 score/page files matched byte for byte. Against the saved production output, acoustic rhythms and all other five role documents were identical for both songs. Mean fret across canonical active acoustic grips changed from 8.90 to 4.50 in Autumn Leaves and 5.19 to 4.29 in All of Me; movement costs rose from 144 to 190 and 128 to 140 respectively. In the requested six-chord passage, five suggested grips won; Em7b5 instead chose its valid fifth-bass shape on strings 6/4/3/2 at frets 6/5/7/5. Tests also verify that the complete suggested sequence beats the previous higher sequence with fixed neighbors, and that sufficiently costly travel can still retain a higher grip. These are score/selection facts, not an ear-based audition.
+
+Isolated uncached acoustic role generation took 0.61–0.62 seconds per song, versus 0.56–0.58 seconds before this change; cache hits took about 0.048 seconds and matched clean output. With two composition and engraving workers, the native asset rebuild took approximately 11 seconds (4 composing, 6 engraving, 1 other); ARM reused caches and took approximately 6 seconds. These are individual local measurements, not a statistical benchmark. Supplied recordings remain byte-identical.
+
+### Recording Playback and Page Timing
+
+Vamp uses only supplied PCM16 WAVs. `package/assets/sounds/beat1.wav` and `beat2.wav` provide measure accents and ordinary beats. Backing files are `package/assets/backing/<normalized_source_id>_<bpm>.wav`. `generator/performance.py` checks containment, format, decoded length and checksums, then copies the recordings byte for byte into the staged output. A catalog `resources` row stores their paths, checksums, sample rates, channels, frame counts, song IDs and tempos. No note-event audio compiler, pitch synthesizer, release envelope or generated audition audio remains. Pitch mathematics remains for notation, TAB, composition and exact MusicXML comparisons.
+
+Settings exposes `audio.backing_enabled`, `audio.metronome_enabled` and `audio.tempo_bpm`; both toggles default off and tempo defaults to 120. The fixed tempo set is 60 through 120 in steps of 10. Backing restricts it to the intersection available for all catalog songs. Turning it on snaps to the nearest available tempo, with ties going slower; an empty intersection makes it unavailable. Obsolete playback and configurable tempo-range settings are rejected. Recordings are never stretched or trimmed.
+
+The existing recordings contain one bar of count-in, 168 main-form bars, three held bars and decay. `arrange/ending.py` now writes that existing tail as measures 169–171, without extending or transforming the recordings. Both accompaniment roles choose a valid closing voicing using their own outgoing position and hold it with ties; melody/solo roles are silent there and the vocal has three full rests. The chord is the final effective harmony after omitting optional turnarounds, including carried harmony. Missing/N.C. final harmony fails accompaniment generation explicitly. The original final bar is untouched.
+
+Score timing includes `sustain_start`, an exact measure boundary, as well as the total written duration. Ending measures carry `ending_sustain`; Python and C++ validate a contiguous tail and consistent timing. Older scores without the new timing field retain their previous full-duration click behavior. Both toggles play the recorded count-in alone, then backing plus sample clicks until `sustain_start`. No clicks sound over the ending. Metronome-only supplies its own count-in and completes after the three silent held bars. Backing-only plays the complete file and decay. Both off enables manual navigation; hold exits and natural completion returns to the part menu. The final page remains visible through decay.
+
+Preparation runs on a worker: checksums and samples are loaded before publishing a Sonus recording transport. Sonus streams backing audio through miniaudio’s background resource manager and mixes preloaded stereo clicks on the same rendered-frame clock. Callback work uses preallocated buffers, and resource retirement occurs on control threads. Stream failures stop progress and surface an error. Absolute rational beat positions are rounded to frames at 48 kHz once per position, avoiding accumulated rounding drift. The UI polls that audio clock for system and measure boundaries and Count-in/Playing/Sustain status; wall-clock timers never determine musical progress.
+
+Page metadata comes from actual system pagination and is preserved and revalidated in engraving caches. `presentation.pages` partitions the music into normal two-system pages. `presentation.playback_pages` holds one view per system: that system and its successor, or a final single top system. Both reuse the same system breaks, horizontal geometry, and part-wide scale, fitted against every adjacent pair. Each descriptor carries system measure ranges, exact beat boundaries, and page-coordinate rectangles for its measure labels. Normal systems partition the score; overlapping playback views must match that canonical partition. Bundles missing playback metadata require regeneration.
+
+Measure numbers are centered over their starting barlines (the staff’s left edge at system start), in a reserved lane above notation and below harmony/section labels. The display outlines the current label using the page texture’s letterbox transform; count-in, preparation, exit, and playback error clear it. Measure updates publish even when the line stays unchanged. Normal catalog page counts remain unchanged; the loader reads the separate playback count from bundle metadata. During automatic playback, navigation bounds, the header’s Line counter, and Pico’s page/pages fields refer to overlapping views. Manual browsing uses the normal sequence. The UI prepares the current and next page before playback starts and retains a two-texture cache. Session and catalog identities reject stale preparation, page updates and completion after cancellation. Missing or corrupt pages/audio leave a visible error and permit hold/back. `--once` validates packaged recording checksums and both normal and playback score pages without opening audio/display hardware.
 
 ### Authored Repertoire and Learned Solos
 
@@ -160,7 +245,7 @@ Fixed scale maps no longer constrain prediction. Brief ordinary movement uses th
 
 Prediction proposes stable and resolving colors, anticipated colors, delayed approaches with real space or intervening valid motion, and chromatic passing notes between valid endpoints, alongside the existing complete enclosures and connections. These are joint actions containing their required destinations. Disabling expressions removes these options. Prepared suspensions are recognized and validated but do not yet have a dedicated prediction action. The model still selects from its small learned distribution; validation establishes supported musical relationships, not aesthetic quality.
 
-Training checks the complete source-form harmonic context. In the repeated performance form, an unlabelled pickup inherits the final source-form chord; an explicit pickup chord or `N.C.` takes precedence. This is the same arrangement context used for playback, not an invented first chord or a rule tied to either song name.
+Training checks the complete source-form harmonic context. In the repeated performance form, an unlabeled pickup inherits the final source-form chord; an explicit pickup chord or `N.C.` takes precedence. This is the same arrangement context used for playback, not an invented first chord or a rule tied to either song name.
 
 The model has only four examples from two songs. Its four-event context does not establish long-range thematic memory, nor does training prove transfer to another standard. Phrase report boundaries describe audible breaths after selection; they are not another composer or an assertion of phrase-wide theme planning. This initial implementation can still repeat local patterns or choose weak continuations on unfamiliar harmony. Exact authored playback and novel model output are reported separately.
 
@@ -170,6 +255,8 @@ Train explicitly:
 ./modulus.py python -m generator.train --leave-one-song-out
 ```
 
+This writes the pooled model to `data/models/solo.json` and diagnostics to `build/training/report.json`. Omit `--leave-one-song-out` to train without held-out likelihood diagnostics. Ordinary asset builds never run training.
+
 Register an additional pair and retrain without changing Python:
 
 ```sh
@@ -179,9 +266,9 @@ Register an additional pair and retrain without changing Python:
 
 Files are copied into `data/songs` and `data/training` without overwriting different contents. Validation checks the exact measure/harmony correspondence, contextual harmonic validity, guitar-written convention and fret range before publishing the manifest. An existing source without a registered slot uses model prediction for that slot. Use `--pitch-convention encoded` only for files whose own transposition describes sounding pitch. Updating existing examples means editing their XML/manifest deliberately and rerunning training. The report under `build/training/report.json` records automatically recognized harmonic treatments, teacher-forced likelihood and optional leave-one-song-out likelihood; none is a listening assessment or unassisted-generation accuracy.
 
-Production input traces distinguish authored reuse from learned prediction. Caches include source, role, relevant data and code; authored cache keys include the selected XML's content and convention. Build and package inputs now include the model, manifest and authored repertoire. Changes to another solo do not change an already registered part's notes. Updating the pooled model can legitimately change predictions for any unregistered part. Build-time composition defaults to two independent processes; rendering and runtime remain unchanged.
+Production input traces distinguish authored reuse from learned prediction. Caches include source, role, relevant data and code; authored cache keys include the selected XML's content and convention. Build and package inputs include the model, manifest and authored repertoire. Changes to another solo do not change an already registered part's notes. Updating the pooled model can legitimately change predictions for any unregistered part. CMake defaults to two composition processes (`VAMP_COMPOSITION_JOBS`, 1–16) and caches each role under `build/build_common/composition_cache`. Direct generator runs use `--composition-jobs` and opt into persistent caching with `--composition-cache`; single-role builds use `--cache-dir`.
 
-The generated `solo_xml/*-solo1.xml` and `*-solo2.xml` exports use local measure numbers, preserve pickups, and store whole-arrangement positions in `manifest.solo`. Exact reuse means complete sounding pitches, onsets, releases, repeated attacks and silence agree. XML whitespace, note splitting, measure labels and exporter-specific annotation containers need not be identical. Optional `python -m tests.solo_compare` compares these completed exports; it never feeds a production decision. Training and authored reuse are the explicitly permitted dependencies on the supplied music.
+Solos are published inside the arranged score JSON and engraved pages, without separate solo MusicXML exports or comparison manifests. Exact-repertoire tests compare generated sounding pitches, onsets, releases, repeated attacks, measure timing and silence directly with the authored XMLs. Tied notation fragments are merged into sounding events for this check, and generated TAB is checked against the guitar constraints. Training and authored reuse remain the explicitly permitted dependencies on the supplied music.
 
 Migration inventory:
 
@@ -191,15 +278,29 @@ Migration inventory:
 | `data/training/` (four XMLs + manifest) | Explicit trainer and registered authored playback. All slots are pooled for learning. Annotation-only cleanup preserves the music and learned counts. |
 | `data/models/solo.json` | Frozen prediction model; reproducible by `generator.train`. |
 | `data/composition/{roles,harmony,lead_ins}.json` | Physical/difficulty settings and generic harmonic validity. Fixed-map permissions removed. |
-| `data/composition/accompaniment.json` | Live sparse comping figures, migrated from the former mixed rhythm file. |
+| `data/composition/accompaniment.json` | Version 3: independently authored per-instrument duration families, layer ranges, bounded retrieval and generic chord-degree/geometry families. Consumers: `comping_rhythm.py`, `comping.py`, `chord_families.py` and `guitar.py`. Migrates the v2 shared-duration schema; no parallel legacy data or reference-derived material. |
 | `generator/harmonic_time.py` | Exact chord/key spans retained from the removed scale subsystem. |
 | Solo event, expression, placement, notation and export helpers | Retained physical/timing safeguards shared by authored and predicted music. |
 
-Removed: `data/{scales,sequences,rhythms}`, old `composition/solo.json` and mixed `rhythms.json`, catalog import/cache code, old solo construction/phrase/selection/scoring/sequence modules, unused accompaniment catalog planner, three evaluation-song excerpts, and tests specific to those retired algorithms. Shared importer, timing, guitar, playback, engraving, arrangement and independence tests remain, with new pooled-learning, registration and exact-repertoire tests. No `docs/` or `audition/` directory is required. Disposable prior comparison outputs are replaced by one current comparison under `build/solo_comparison`.
+Removed: `data/{scales,sequences,rhythms}`, old `composition/solo.json` and mixed `rhythms.json`, catalog import/cache code, old solo construction/phrase/selection/scoring/sequence modules, unused accompaniment catalog planner, three evaluation-song excerpts, and tests specific to those retired algorithms. Shared importer, timing, guitar, playback, engraving, arrangement and independence tests remain, with new pooled-learning, registration and exact-repertoire tests. No `docs/` or `audition/` directory is required. Temporary comparisons and diagnostics belong under `/tmp/`; old comparison pages, diagnostic reports, formatter-debug scripts and the copied verification bundle have been removed from `build/`. Product builds, caches and toolchains remain. Desktop and ARM builds resolve Sonus through Conan; separate local Sonus installations and temporary CMake dependency overrides are unnecessary. Refresh both with `python3 modulus.py configure --update --target x86_64_gcc12_debug --target arm_64_gcc12_release` when dependency packages change. The obsolete playback-test assets are superseded by the current performance-test fixtures.
 
-Verification on October 5, 2026: `unittest discover -s tests -p 'test_*.py'` passed 361 tests in 78.62 seconds. These include all four approved solos without annotations or authored-role privileges, source-form pickup context, invalid resolutions, passing endpoints, anticipated colors, tied-event partition invariance and active prediction construction. The normal six-part asset build passed `--check`, including playback, engraving, TAB and catalog validation. With two composition workers, newly generated solo results, cached accompaniment/melody and a warm engraving cache, it took 5.10 seconds; performance-part generation occupied 2.37 seconds. Each registered solo took 0.36–0.40 seconds. The subsequent `cmake --build build/build_x86_debug --target vamp_generated_assets` also passed. These measurements describe registered playback, not novel prediction. Detailed timings are in `build/training/build-profile.json`; isolated, cache-free prediction timings and input fingerprints are in `build/training/prediction-check.json`.
+Verification on October 5, 2026: `unittest discover -s tests -p 'test_*.py'` passed 361 tests in 78.62 seconds. These include all four approved solos without annotations or authored-role privileges, source-form pickup context, invalid resolutions, passing endpoints, anticipated colors, tied-event partition invariance and active prediction construction. The normal six-part asset build passed `--check`, including playback, engraving, TAB and catalog validation. With two composition workers, newly generated solo results, cached accompaniment/melody and a warm engraving cache, it took 5.10 seconds; performance-part generation occupied 2.37 seconds. Each registered solo took 0.36–0.40 seconds. The subsequent `cmake --build build/build_x86_debug --target vamp_generated_assets` also passed. These measurements describe registered playback, not novel prediction; the disposable diagnostic reports have since been removed.
 
-The optional exact comparison confirmed all four exports: 137/141 attacks for All of Me and 121/115 for Autumn Leaves, with no pitch, onset, release or rest differences. All four now pass contextual harmony validation as well. Removing 69 explanatory annotations left the complete event streams and learned musical count tables identical to the pre-edit baseline; only model provenance hashes changed. This verifies authored reuse rather than learned reproduction. Commands used the ordinary asset generator with `--export-solo-xml --check` and `python -m tests.solo_compare` with four explicit `--reference` pairs, `--reference-pitch guitar-written` and `--require-exact`. No ear-based audition or claim of transfer quality was made.
+Voicing-only verification on October 6, 2026, before the rhythm update: the suite passed 363 tests in 120.66 seconds, including 11 new voicing tests. Exhaustive small cases verify that the movement selector beats greedy selection and includes repeat/neighbor boundaries. That change retained the prior accompaniment onset/duration/rest sequences; source melody, vocal and both solo role documents were unchanged. Its six-score production rebuild passed `--check` in 11.54 seconds with two composition and two engraving workers: 2.95 seconds generating parts and 8.19 seconds engraving, with two unchanged score cache hits. This checks timing, harmonic/physical constraints and rendering, not a claim of ergonomic comfort from listening.
+
+Rhythm verification on October 6, 2026: `unittest discover -s tests -p 'test_*.py'` passed all 375 tests in 83.57 seconds. Coverage includes attack ranges, bass downbeats, mixed durations, harmonic-change budgets, overlapping voices, tied joins, 3/4 and 5/4, melody-conditioned rhythm costs, tempo invariance, canonical reuse, and serial/parallel role identity. A synthetic learned-model test also confirms that changing original melody rhythm or upcoming harmony changes solo duration probabilities with a fixed seed. Both songs' solo, electric melody and vocal role documents remain exactly equal to the saved pre-rhythm baseline; no model retraining was performed.
+
+The ordinary six-score rebuild with `--check --composition-jobs 2 --jobs 2` passed in 13.39 seconds: 4.44 seconds generating parts and 8.48 seconds engraving. All role composition results were newly generated; the warm engraving namespace reused two unchanged vocal scores. The subsequent `cmake --build build/build_x86_debug --target vamp_generated_assets` passed with all role/score caches reused. Actual selected acoustic upper rhythms use one or two attacks in these songs; three-attack alternatives are available and tested but are not forced. All of Me electric has three zero-attack source/harmony contexts; Autumn Leaves electric uses one or two attacks. Neither song required a density exception. Reuse audits checked 103/62 acoustic/electric repeated bars for Autumn Leaves and 102/62 for All of Me, with only source-defined pickups and final-harmony exceptions. Sample acoustic notation/TAB pages were visually inspected. Supplied audio recordings remain unchanged; no ear-based audition of the new accompaniment was performed. Disposable logs, profiles and comparisons are under `/tmp/`, outside product assets.
+
+Chord-family and unified-acoustic verification on October 6, 2026: `unittest discover -s tests -p 'test_*.py'` passed 386 tests in 81.67 seconds. New cases cover 35 chord qualities in three keys for both instruments, discarded bass variants and explicit unsupported combinations, the six suggested acoustic grips, highest feasible electric melody strings, scoped policy dependencies, and one-voice tie preservation including repeated attacks and cross-bar ties. Quarter/eighth acoustic attack durations and available consecutive-eighth patterns are tested. The acoustic sounding events in both complete arrangements contain only quarter/eighth durations, even where notation splits them into ties.
+
+The full checked rebuild with two composition and two engraving workers took 13.76 seconds: 4.68 seconds for all parts and 8.63 seconds engraving. All role caches missed after the code change; two unchanged vocal engravings were reused. Serial composition-only generation into a disposable directory also passed in 5.58 seconds. The subsequent CMake asset build passed and reproduced every role document from warm caches. Both registered solos and vocal documents are exactly unchanged from the saved prechange baseline; the electric melody's original pitches/onsets/durations also match. Reuse audits retain the same 103/62 and 102/62 repeated-bar counts. Acoustic upper rhythms now select one to three attacks in Autumn Leaves and one or two in All of Me. Acoustic and electric melody pages were visually inspected; this is notation/constraint verification, not an ear-based audition. Only product artifacts and normal caches were written under `build/`; diagnostic logs and baseline comparisons remain under `/tmp/`.
+
+Four-fret fallback verification on October 6, 2026: `unittest discover -s tests -p 'test_*.py'` passed 392 tests in 85.65 seconds. This includes primary-tier precedence over family, arbitrary movement savings and melody-string height; a complete held-note path that needs an earlier four-fret option; tied-fragment cost invariance; and anchor release checks. The first full run exposed one obsolete C5 rejection expectation, now replaced by a test verifying its legal four-fret voicings contain only C/G. Coverage tests check 35 qualities in 12 keys for each instrument: 418/420 acoustic contexts and 420/420 electric contexts have a legal grip. Acoustic Bbm9/Bm9 remain explicitly unsupported; this is measured coverage, not a guarantee for every written extension or slash chord.
+
+The checked six-score rebuild with `--composition-jobs 2 --jobs 2` passed in 9.21 seconds (4.58 seconds composing all 12 roles, 3.90 seconds processing six cached engravings). All composition caches missed after the code change. Every resulting role document and accompaniment rhythm report exactly matches the saved baseline: current production chords already have primary grips, so none needed fallback substitutions. New sidecar span diagnostics identify the tiers and available candidates. The subsequent `cmake --build build/build_x86_debug --target vamp_generated_assets` passed with all role caches reused and identical per-role musical hashes. Acoustic and electric score/TAB pages were visually checked; supplied recordings are unchanged. Formatter and diff checks passed. Temporary verification files remain under `/tmp/`.
+
+Earlier exact-repertoire verification confirmed all four authored solos: 137/141 attacks for All of Me and 121/115 for Autumn Leaves, with no pitch, onset, release or rest differences. All four also passed contextual harmony validation. Removing 69 explanatory annotations left the complete event streams and learned musical count tables identical to the pre-edit baseline; only model provenance hashes changed. This verifies authored reuse rather than learned reproduction. These checks now compare generated score data directly with the training inputs instead of producing standalone XML exports. No ear-based audition or claim of transfer quality was made.
 
 Forced prediction, without reading the training XMLs, completed all four source/role combinations with valid harmony. These were composition-only runs on the training-source songs, not held-out transfer tests. Their notes intentionally differ from registered playback and were not published as replacements. The expanded treatment candidates initially took 99.03–120.33 seconds per role with one worker and no concrete-result cache. Removing repeated validation of identical resolution endpoints reduced the isolated All of Me acoustic check from 104.30 to 60.13 seconds, with an identical complete output hash. Six additional contexts retained all 32,566 ordered action candidates exactly. These expanded prediction choices remain slower than the earlier 21.81–27.16-second decoder; registered playback is unaffected. Training the pooled 2,860-context model took 0.57 seconds. Prediction remains the slower path, chiefly candidate construction and event-likelihood evaluation; further profiling is needed before expanding its choices again.
 
@@ -207,7 +308,7 @@ Forced prediction, without reading the training XMLs, completed all four source/
 
 `generator/engraving/` is a custom Python engraver, not a wrapper around MuseScore or another notation engine. `generator.engraving.layout.layout_score(document, style, fonts)` produces immutable layout pages with exact source intervals, notation anchors and drawing primitives. `generator.engraving.raster.rasterize_page(page, fonts)` draws one page at four-times resolution and downsamples it. `generator.engraving.pipeline.engrave_tree(root, fonts)` is the asset-pipeline entry point. `fonts.Fonts`, `types.Style`, and `types.EngravingError` live in their respective engraving modules; page constants and `check_page` live in `validation`. Package initializers do not re-export these APIs. `generator/musicxml_writer.py` remains a round-trip/export utility and is not involved in engraving.
 
-Engraving supports a bounded process pool (`VAMP_ENGRAVING_JOBS`, default 2) and a build-only content-addressed page cache (`VAMP_ENGRAVING_CACHE_DIR`, default `build/build_common/engraving_cache`). Keys include the complete musical document, engraving implementation, fonts/metadata, style, Python and rendering dependency versions. Playback is deliberately excluded from page keys and is always rebuilt independently. Cache hits are copied into the current staged output and checked for checksums, PNG decoding, dimensions, blank pages and clipped ink. Missing or corrupt entries are regenerated. All semantic and asset validation, database construction and transactional publication remain in place. The cache is not installed and can be deleted at any time when builds are idle. CI saves it after the native build and reuses it for ARM packaging; generation prints stage timing and per-worker progress instead of remaining silent during engraving.
+Engraving supports a bounded process pool (`VAMP_ENGRAVING_JOBS`, default 2) and a build-only content-addressed page cache (`VAMP_ENGRAVING_CACHE_DIR`, default `build/build_common/engraving_cache`). Keys include the complete musical document, engraving implementation, fonts/metadata, style, Python and rendering dependency versions. Page timing is included in cached manifests and checked against exact measure boundaries; corrupt coverage is rebuilt. Recording files are copied independently of the page cache. Cache hits are copied into the current staged output and checked for checksums, PNG decoding, dimensions, blank pages and clipped ink. Missing or corrupt entries are regenerated. All semantic and asset validation, database construction and transactional publication remain in place. The cache is not installed and can be deleted at any time when builds are idle. CI saves it after the native build and reuses it for ARM packaging; generation prints stage timing and per-worker progress instead of remaining silent during engraving.
 
 Reuse has two levels. A full-score hit restores complete page images for the same musical document and metadata. A notation-body group shares system layout and body pixels among documents with identical staff content, while retaining each document's title, subtitle, pagination, footer, and complete semantic bundle. Body keys include the rendering context and actual page-band geometry; incompatible placement cannot reuse those pixels. Each header/footer is composed separately, and pages whose drawing primitives cannot be separated safely use full-page rasterization. This reuse never merges score IDs, catalog records, or authored music.
 
@@ -217,21 +318,33 @@ Voices retain separate stems and collision offsets; chord tones retain their own
 
 Generated guitar tones carry `tablature: {string, fret}`; their staff is marked `tablature: true`. `engraving/tablature.py` draws a six-line TAB view below standard notation, sharing onset columns, barlines, and page breaks. Both views belong to one system and never duplicate the semantic timeline or playback events. Standard notation carries rhythm, rests, and tuplets; TAB carries positions, solo slashes, and tied continuations without rest glyphs. Strings and frets are validated against sounding pitch, and the layout audit checks each visible TAB numeral's position and alignment.
 
-Every part uses two music systems per page; only an odd final system occupies a row alone. A compact title/composer/part header reserves the same height on every page, wrapping credits if necessary. Pagination fits the two systems using their individual heights and distributes extra whitespace around them while retaining minimum system and header/footer clearances. One uniform scale applies throughout the entire part, reducing the ordinary notation size only when required by the tallest pair. Notation, TAB, text, and anchors scale together without changing musical content. Guitar layout starts with 10-pixel notation spacing, 10.5-pixel TAB spacing, 12-pixel fret labels, and a 14-pixel minimum gap between systems. The unscaled TAB spacing accommodates Petaluma numerals up to 9.75 pixels tall. Tempo markings consume neither ink nor spacing; their semantic data is preserved. Existing `.pages/*.png` preview files are the exact packaged page textures.
+Normal pages use two music systems; only an odd final system occupies a row alone. Playback pages use overlapping adjacent pairs and always finish with the last system alone on top. A compact title/composer/part header reserves the same height on every page, wrapping credits if necessary. Pagination fits the two systems using their individual heights and distributes extra whitespace around them while retaining minimum system and header/footer clearances. One uniform scale applies throughout the entire part, reducing the ordinary notation size only when required by the tallest adjacent pair, including transition pages. Notation, TAB, text, and anchors scale together without changing musical content. Guitar layout starts with 10-pixel notation spacing, 10.5-pixel TAB spacing, 12-pixel fret labels, and a 14-pixel minimum gap between systems. The unscaled TAB spacing accommodates Petaluma numerals up to 9.75 pixels tall. Tempo markings consume neither ink nor spacing; their semantic data is preserved. Existing `.pages/*.png` preview files are the exact packaged page textures.
 
 Line breaks occur only at measure boundaries. Each line fits as many complete measures as readable spacing allows; it never fills remaining space with a partial measure. A complete measure exceeding the available width or height fails with source and measure context instead of being split or squeezed to fit. Authored ties, beams, tuplets, and lyric spans can continue across whole-measure line breaks. Column measurement and fit trials reuse a score index and bounded column cache. Page placement then re-spaces columns for the final height-fitting scale, keeping every notation/TAB staff at the same physical left and right margins without stretching glyphs. Non-final lines are justified; a short final line retains natural note spacing while staff lines extend to the same right margin. Layout audits enforce these margins as well as musical coverage.
 
 `audit.py` checks the final layout against the musical model: timeline and fragment coverage, exact fragment durations, actual glyph bounds, pitch positions, visible note primitives, required accidentals, ties, lyrics, harmonies and spans. It enforces consistent notation size throughout a part and header/footer clearance, and rejects out-of-page geometry and collisions among unrelated noteheads, accidentals and same-lane text. Staff, ledger, stem, beam and tie contacts are intentional. This is not a proof of publication-quality engraving for every possible MusicXML document; notation changes need targeted semantic, geometry and reviewed image tests. The writer's XML round-trip audit is retained separately in the test suite.
 
-Bundles preserve the complete `music` object. Presentation identifies the `Vamp Python` engraver; its style fingerprint includes the style settings, both font files, font-specific metadata and the glyph-name mapping. Scores, playback manifests, and the catalog have no numbered schema labels. Required fields, types, relationships, dimensions, and checksums determine whether generated data is usable. Incompatible data is rejected, never migrated or guessed.
+Bundles preserve the complete `music` object. Presentation identifies the `Vamp Python` engraver; its style fingerprint includes the style settings, both font files, font-specific metadata and the glyph-name mapping. Scores, performance metadata, and the catalog have no numbered schema labels. Required fields, types, relationships, dimensions, and checksums determine whether generated data is usable. Incompatible data is rejected, never migrated or guessed.
 
-Pages are 976x484 pixels with 10-pixel guitar or 12-pixel vocal staff spacing, 28-pixel horizontal margins and 18-pixel vertical margins. The two acoustic rest lanes sit closer to the staff, with collision offsets preserving their separation. Dense measures continue across systems/pages; sustained notes become exact, tied presentation fragments and rests become exact rest fragments. These fragments never change the semantic JSON. Irreducible simultaneous content that cannot fit a fixed page fails contextually rather than being clipped or shrunk. Blank pages, incorrect dimensions, and ink reaching the page edge are rejected. Runtime letterboxing preserves aspect ratio. Luma's `load_texture_uncached()` lets Product Vamp retain at most two textures without a hidden strong asset-cache reference. The UI reports missing/corrupt assets instead of falling back to the removed renderer. Runtime validates bundle metadata, paths, dimensions, and page checksums; the build validates musical semantics.
+Pages are 976x484 pixels with nominal 10-pixel guitar or 12-pixel vocal staff spacing, 28-pixel horizontal margins and 18-pixel vertical margins. The two acoustic rest lanes sit closer to the staff, with collision offsets preserving their separation. Systems and pages contain complete measures; tied notes and notation spans may continue across their boundaries. Presentation fragments preserve exact timing and never change the semantic JSON. A measure that cannot fit at readable spacing fails with source and measure context; the uniform part-wide scaling described above fits pairs of complete systems vertically. Blank pages, incorrect dimensions, and ink reaching the page edge are rejected. Runtime letterboxing preserves aspect ratio. Luma's `load_texture_uncached()` lets Product Vamp retain at most two textures without a hidden strong asset-cache reference. The UI reports missing/corrupt assets instead of falling back to the removed renderer. Runtime validates bundle metadata, paths, dimensions, and page checksums; the build validates musical semantics.
+
+### Printable Parts
+
+`generator.printing.generate_tree` runs after screen engraving inside the normal asset staging transaction. It reads each completed song/part's musical document, not its screen PNGs or overlapping playback views, and writes `print/<score-stem>.pdf`. Every arranged measure and the instrumental ending appear once in order. Acoustic/electric parts retain standard notation and TAB; vocal parts retain lyrics and rests. No music, runtime bundle fields or catalog rows are changed. Both CMake installation and runtime image copying exclude the `print` directory.
+
+Print layout uses portrait US Letter (612 × 792 PDF points), a minimum half-inch clear margin, and fixed five-point guitar/six-point vocal staff spacing. It reserves the actual overhang of centered measure numbers inside the margin. Complete systems fill the available page height with consistent notation size; the final page keeps natural vertical spacing. Headers identify the tune, composer and instrument; printed page numbers are independent of device pagination. The existing musical/fragment/TAB audits remain active, with variable systems per page allowed only by the print caller. Screen audits still require two systems except the odd final system.
+
+The shared primitive renderer accepts explicit colors and output scale while retaining unchanged screen defaults. Print draws black on white directly at 2550 × 3300 pixels (300 DPI), then writes grayscale image pages through Pillow's PDF writer at quality 95. This is a raster PDF, not searchable/vector notation. One page is rendered and appended at a time to bound image memory. The pinned Pillow parser checks page count, Letter media boxes and image resolution/color; tests decode the embedded JPEGs and verify clear margins. No new Python package is required: Modulus's existing `assets` group owns Pillow, fonttools and aggdraw. Timestamps and output-path metadata are omitted for byte-reproducible PDFs.
+
+With `--engraving-cache`, print results use a separate `print/<print-renderer>/<score-key>/` namespace. Keys cover score content, print settings, fonts, renderer source and dependency versions. Cached PDFs have checked hashes and page metadata; missing or corrupt entries regenerate. Native and ARM builds share the cache locally. This print namespace is separate from the screen-only CI cache snapshot. Output PDFs are validated before the build publishes; failure preserves the previous complete asset directory. Stale PDFs disappear when source parts disappear. Temporary print files live only in build/cache staging.
+
+Print rollout verification: `python3 modulus.py python -m unittest discover -s tests -p 'test_*.py'` passed 458 tests; the seven `tests.test_printing` tests were rerun after the final footer-margin adjustment. `python3 modulus.py build --target x86_64_gcc12_debug --target arm_64_gcc12_release` passed with byte-identical PDFs on both targets. All of Me has 10/8/5 acoustic/electric/vocal pages and Autumn Leaves has 9/7/4, totaling 43. All pages were checked for Letter size, 300 DPI grayscale images and pure-white half-inch margins; representative opening, solo, lyric and ending pages were visually inspected. All 394 preexisting score JSON/screen PNG files stayed byte-identical. PDF generation took 9.7 seconds uncached and 0.1 seconds cached on this development machine; this is the print stage only, not a clean whole-product build benchmark. Physical printing was not tested.
 
 ### Validation And Publication
 
-Generation validates sources and complete staged output before replacing the previous asset directory. Replacement is always complete; deleted sources cannot leave stale JSON, pages, audio, or Pico title assets. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalog. Runtime packages contain `catalog.db` with embedded semantic JSON and title-image identifiers, PNG pages, generated PCM WAVs, Pico title images, and the OTF UI text font, not MusicXML, MuseScore, or Python dependencies. Python test discovery runs model/import/export tests; the normal asset build additionally qualifies and audits every catalog score.
+Generation validates sources and complete staged output before replacing the previous asset directory. Replacement is always complete; deleted sources cannot leave stale JSON, pages, audio, or Pico title assets. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalog. Runtime packages contain `catalog.db` with embedded semantic JSON and title-image identifiers, PNG pages, supplied WAV recordings, Pico title images, and the OTF UI text font, not MusicXML, MuseScore, or Python dependencies. Python test discovery runs model/import/export tests; the normal asset build additionally qualifies and audits every catalog score.
 
-The page cache uses `<renderer fingerprint>/fullscore/<score fingerprint>/` and `<renderer fingerprint>/notation/<band fingerprint>/` directories, without a manual cache revision. `./modulus.py python -m generator --renderer-fingerprint` reports the same identity used by rendering and CI: source bytes, fonts, visual style, Python implementation/version, and rendering-library versions. Python compiler/build timestamps are excluded. CI keys append all generator and active XML inputs, including rhythms, and restore earlier snapshots only within the same renderer directory; only that directory is saved after a successful build.
+The page cache uses `<renderer fingerprint>/fullscore/<score fingerprint>/` and `<renderer fingerprint>/notation/<band fingerprint>/` directories, without a manual cache revision. `./modulus.py python -m generator --renderer-fingerprint` reports the same identity used by rendering and CI: source bytes, fonts, visual style, Python implementation/version, and rendering-library versions. Python compiler/build timestamps are excluded. CI keys append generator code, the palette, source songs, composition policies, the frozen model, the training manifest, and authored solo XMLs. Earlier snapshots are restored only within the same renderer directory; only that directory is saved after a successful build.
 
 `cold namespace` means that renderer's persistent directory does not yet exist. A renderer or environment change starts a new namespace; deleting a namespace has the same effect. It does not disable reuse within the current run: shared notation is still laid out once per group and its body pixels can be reused as other titles are composed, even without a persistent cache. Conversely, a warm directory does not guarantee that a particular score is cached. Progress reports full-score hits, body hits, newly rasterized body bands, composed pages, missing/rejected entries, and phase timings. Worker phase timings are summed and may exceed wall time. Obsolete namespaces are not migrated and can be removed when builds are idle.
 
@@ -247,13 +360,13 @@ Generator checks compare the generated performance summary against `tests/fixtur
 
 The Pico firmware uses the Pico SDK C++ stack and the `pico2_w` board definition. It initializes the ST7789T3 display over SPI, reads the CST816D touch controller over I2C, joins the Zero AP, polls `/api/remote/display`, and sends HTTP POST commands for gestures.
 
-The display is rotated 90 degrees clockwise into a 320x240 landscape UI. The top status bar uses the shared palette:
+The display is rotated 90 degrees clockwise into a 320x240 landscape UI. A small connection indicator in the header uses the shared palette:
 
-- Blue: connecting.
+- Amber (`retrying`): connecting, offline, or a remote request failed.
-- Yellow: retrying or remote request failed.
+- Green (`connected`): connected to the Zero API.
-- Green: connected to the Zero API.
+- Red (`danger`): Wi-Fi initialization failed.
 
-The body shows the focused menu item, song title, or setting and its current value. In score view it shows the active title, part, and page count. Both devices use a light, score-like palette. Touch controls are tap for next, double tap for previous, and tap-and-hold for select. All visible text uses Petaluma Script, matching score-page text and the Zero UI.
+The body shows the focused menu item, song title, or setting and its current value. In score view it shows the active title, part, and page count. A separate status label displays connecting, offline, error, or busy when applicable. Both devices and score pages use the dark palette in `generator/palette.json`. Touch controls are tap for next, double tap for previous, and tap-and-hold for select. Holding Tempo enters or leaves editing; taps then increase BPM and double taps decrease it. All visible text uses Petaluma Script, matching score-page text and the Zero UI.
 
 `generator.pico_artwork` shapes text on the build host with the bundled `PetalumaScript.otf` and Pillow's Raqm support. Fixed menu, part, connection/status, and settings labels, digits, and `/` become four-bit coverage sprites. The Pico CMake rule writes `generated/pico/static.p4` and embeds it through `generated/pico/artwork.S`. The Zero does not link this sprite atlas; Luma renders its dynamic UI text from the packaged OTF.
 
@@ -284,7 +397,7 @@ Building the full package requires Docker/buildx, the ARM embedded GCC toolchain
 
 Generated score assets are created during CMake build/package and copied into the Zero image context. Static font assets remain tracked under `package/assets/fonts`.
 
-`generator/catalog.py` uses Python's standard-library SQLite module to construct `catalog.db` after musical, playback, and engraving validation. The database carries application ID `0x56414D50`, complete score bundles, and menu metadata. Runtime validates both required strict tables, their column names/types, non-null requirements, and primary keys directly. Sorted insertion and the absence of wall-clock timestamps make repeated builds reproducible with the same toolchain. It is closed and integrity-checked before publication, without journal/WAL sidecars. Individual JSON files remain available in the build tree for inspection, but both CMake installation and Docker image assembly exclude them.
+`generator/catalog.py` uses Python's standard-library SQLite module to construct `catalog.db` after musical, recording, and engraving validation. The database carries application ID `0x56414D50`, complete score bundles, and menu metadata in a strict `songs` table with a `songs_menu` index; a strict `resources` table contains the supplied-audio manifest. Runtime validates the table's column names/types, non-null requirements, and primary key directly. Sorted insertion and the absence of wall-clock timestamps make repeated builds reproducible with the same toolchain. It is closed and integrity-checked before publication, without journal/WAL sidecars. Individual JSON files remain available in the build tree for inspection, but both CMake installation and Docker image assembly exclude them.
 
 The default `database.path` is `/usr/share/product_vamp/catalog.db`; development configuration points at the native generated_assets database. Stored source paths are logical, asset-relative locations, not build-host filenames. Runtime resolves them against `runtime.assets_path` and rejects traversal and escaping symlinks. The JSON file itself need not exist: its parent identifies the score's page directory. Normal startup checks catalog compatibility; `--once` additionally runs SQLite's integrity probe and checks all page/audio files. Missing, corrupt, or incompatible catalogs fail rather than creating an empty library. `--ensure-database` is no longer a supported command.
 
@@ -292,7 +405,7 @@ Every run of `zero/install_zero.sh` is a clean installation. It stages configura
 
 Configuration, AP credentials, and all prior catalog data are intentionally reset to bundle defaults. There is no automatic backup or migration. Installation through the control AP can disconnect the session; use the local console or independent networking. Preflight failure leaves existing application files untouched and restores the previous image tag; replacement failure stops Product Vamp and reports the error. Rerunning the installer is the recovery path. Ordinary boots/restarts do not run cleanup. Unrelated networks, OS data, containers, and system logs remain untouched. Cleanup rejects symlink/mount targets, never follows paths from old configuration, and removes only identified obsolete Product Vamp images not used by another container.
 
-CI builds stable target-specific archives with `python3 modulus.py package`, then `package/scripts/publish_latest_generic_package.py` uploads exactly the configured targets and their checksums. Gitea requires a registry slot, so publishing replaces `latest` and removes obsolete remote entries. There is no application release number, artifact version argument, or versioned install directory. The manifest, heartbeat, status endpoint, and `--build-info` retain the Git commit for diagnostics. Conan references, compiler/dependency pins, and MusicXML protocol identifiers remain external tool requirements.
+CI builds stable target-specific archives with `python3 modulus.py package`, then `python3 modulus.py publish` uses Modulus's artifact publisher to upload exactly the configured targets and their checksums. Gitea requires a registry slot, so publishing replaces `latest` and removes obsolete remote entries. There is no application release number, artifact version argument, or versioned install directory. The manifest, runtime heartbeat file, and `--build-info` retain the Git commit for diagnostics. The health endpoint reports readiness; remote display responses contain device navigation state. Conan references, compiler/dependency pins, and MusicXML protocol identifiers remain external tool requirements.
 
 ## Shared Protocol
 

cmake/runtime_assets.cmake

@@ -1,4 +1,4 @@
 if(NOT EXISTS "${SOURCE}/catalog.db")
   message(FATAL_ERROR "Missing generated catalog: ${SOURCE}/catalog.db")
 endif()
-file(COPY "${SOURCE}/" DESTINATION "${DESTINATION}" PATTERN "*.json" EXCLUDE PATTERN "analysis" EXCLUDE)
+file(COPY "${SOURCE}/" DESTINATION "${DESTINATION}" PATTERN "*.json" EXCLUDE PATTERN "analysis" EXCLUDE PATTERN "print" EXCLUDE)

cmake/zero.cmake

@@ -24,8 +24,8 @@ check_cxx_source_compiles("
   #include <sonus.hpp>
   #include <luma.hpp>
   #include <utility>
-  static_assert(requires(sonus::engine& engine, sonus::prepared_sequence sequence) {
+  static_assert(requires(sonus::engine& engine, sonus::prepared_transport transport) {
-    engine.play_sequence(std::move(sequence));
+    engine.play_transport(std::move(transport));
   });
   static_assert(requires(luma::application& application, std::string path) {
     application.load_texture_uncached(path);
@@ -34,7 +34,7 @@ check_cxx_source_compiles("
 " VAMP_DEPENDENCY_APIS_AVAILABLE)
 cmake_pop_check_state()
 if(NOT VAMP_DEPENDENCY_APIS_AVAILABLE)
-  message(FATAL_ERROR "Product Vamp requires current Sonus sequence and Luma uncached-texture APIs. Run ./modulus.py configure --update after publishing the updated dependencies, or create their Conan packages locally before configuring --no-remote. See CMake's configure log for the compiler diagnostic.")
+  message(FATAL_ERROR "Product Vamp requires current Sonus recording transport and Luma uncached-texture APIs. Run ./modulus.py configure --update after publishing the updated dependencies, or create their Conan packages locally before configuring --no-remote. See CMake's configure log for the compiler diagnostic.")
 endif()
 
 execute_process(
@@ -90,7 +90,7 @@ set(VAMP_CORE_SOURCES
   src/vamp_zero/navigation.cpp
   src/vamp_zero/notation.cpp
   src/vamp_zero/page_control.cpp
-  src/vamp_zero/playback.cpp
+  src/vamp_zero/performance.cpp
   src/vamp_zero/control.cpp
   src/vamp_zero/score.cpp
   src/vamp_zero/score_loader.cpp
@@ -148,6 +148,8 @@ file(GLOB_RECURSE VAMP_GENERATOR_INPUTS CONFIGURE_DEPENDS
   ${CMAKE_CURRENT_SOURCE_DIR}/generator/theme.py
   ${CMAKE_CURRENT_SOURCE_DIR}/cmake/build_python.cmake
   ${CMAKE_CURRENT_SOURCE_DIR}/cmake/theme.cmake
+  ${CMAKE_CURRENT_SOURCE_DIR}/package/assets/sounds/*.wav
+  ${CMAKE_CURRENT_SOURCE_DIR}/package/assets/backing/*.wav
   ${CMAKE_CURRENT_SOURCE_DIR}/package/assets/fonts/*
   ${CMAKE_CURRENT_SOURCE_DIR}/package/assets/fonts/**/*
 )
@@ -155,11 +157,6 @@ set(VAMP_GENERATED_ASSETS_DIR ${CMAKE_CURRENT_BINARY_DIR}/generated_assets)
 set(VAMP_GENERATED_ASSETS_STAMP ${CMAKE_CURRENT_BINARY_DIR}/generated_assets.stamp)
 set(VAMP_ENGRAVING_JOBS 2 CACHE STRING "Maximum parallel Python engraving workers (1-16)")
 set(VAMP_COMPOSITION_JOBS 2 CACHE STRING "Maximum independent musical role workers (1-16)")
-option(VAMP_EXPORT_SOLO_XML "Export completed solo roles as comparison MusicXML" OFF)
-set(VAMP_SOLO_EXPORT_ARGS)
-if(VAMP_EXPORT_SOLO_XML)
-  list(APPEND VAMP_SOLO_EXPORT_ARGS --export-solo-xml)
-endif()
 set(VAMP_ENGRAVING_CACHE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/build/build_common/engraving_cache" CACHE PATH "Shared build-only page cache; empty disables caching")
 set(VAMP_ENGRAVING_CACHE_ARGS)
 if(VAMP_ENGRAVING_CACHE_DIR)
@@ -167,7 +164,7 @@ if(VAMP_ENGRAVING_CACHE_DIR)
 endif()
 string(JOIN "\n" VAMP_GENERATOR_INPUT_MANIFEST ${VAMP_GENERATOR_INPUTS})
 file(CONFIGURE OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/generated_assets.inputs
-  CONTENT "@VAMP_GENERATOR_INPUT_MANIFEST@\nsolo_xml=@VAMP_EXPORT_SOLO_XML@\n" @ONLY)
+  CONTENT "@VAMP_GENERATOR_INPUT_MANIFEST@\n" @ONLY)
 add_custom_command(
   OUTPUT ${VAMP_GENERATED_ASSETS_STAMP}
   BYPRODUCTS ${VAMP_GENERATED_ASSETS_DIR}/catalog.db
@@ -182,7 +179,6 @@ add_custom_command(
     --composition-cache ${CMAKE_CURRENT_SOURCE_DIR}/build/build_common/composition_cache
     --progress
     ${VAMP_ENGRAVING_CACHE_ARGS}
-    ${VAMP_SOLO_EXPORT_ARGS}
   COMMAND ${CMAKE_COMMAND} -E touch ${VAMP_GENERATED_ASSETS_STAMP}
   WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
   DEPENDS ${VAMP_GENERATOR_INPUTS} ${CMAKE_CURRENT_BINARY_DIR}/generated_assets.inputs
@@ -199,7 +195,7 @@ install(TARGETS vamp_zero
 )
 install(FILES package/config/vamp.example.json DESTINATION zero/config)
 install(DIRECTORY ${VAMP_GENERATED_ASSETS_DIR}/ DESTINATION zero/share/product_vamp
-  PATTERN "*.json" EXCLUDE PATTERN "analysis" EXCLUDE)
+  PATTERN "*.json" EXCLUDE PATTERN "analysis" EXCLUDE PATTERN "print" EXCLUDE)
 install(PROGRAMS
   package/zero/install_zero.sh
   package/zero/start_ap.sh

data/composition/roles.json

@@ -8,15 +8,25 @@
     "acoustic_accompaniment": {
       "seed": "acoustic_comp",
       "bass": "primary",
-      "center": 53
+      "center": 53,
+      "voicing": {
+        "register_weight": 1
+      }
     },
     "electric_accompaniment": {
       "seed": "electric_comp",
       "bass": "essential",
-      "center": 60
+      "center": 60,
+      "voicing": {
+        "register_weight": 1
+      }
     },
     "electric_melody": {
-      "seed": "head"
+      "seed": "head",
+      "center": 60,
+      "voicing": {
+        "register_weight": 1
+      }
     },
     "vocal_melody": {
       "seed": "voice"

generator/__main__.py

@@ -2,14 +2,14 @@ from __future__ import annotations
 
 import argparse
 import json
+import os
 import pathlib
+import platform
 import shutil
 import tempfile
 import time
 
-import modulus.filesystem
+import generator.audio_assets
-
-import generator.arrange.generate
 import generator.catalog
 import generator.engraving.cache
 import generator.engraving.fonts
@@ -17,9 +17,12 @@ import generator.engraving.pipeline
 import generator.engraving.types
 import generator.parts
 import generator.pico_artwork
-import generator.playback
+import generator.printing
+import generator.production
 import generator.score
+import generator.score_io
 import generator.summary
+import modulus.filesystem
 
 RUNTIME_FONT_FILES = ("PetalumaScript.otf", "OFL.txt")
 
@@ -41,9 +44,8 @@ def parse_args() -> argparse.Namespace:
   parser.add_argument("--jobs", type=int, default=1, choices=range(1, 17), metavar="1-16")
   parser.add_argument("--composition-jobs", type=int, default=2, choices=range(1, 17), metavar="1-16")
   parser.add_argument("--composition-cache", type=pathlib.Path)
-  parser.add_argument("--export-solo-xml", action="store_true", help="Export completed solo roles as comparison MusicXML, without recomposing.")
+  parser.add_argument("--trace-level", choices=("summary", "full"), default="summary", help="Diagnostic detail only; does not seed composition.")
-  parser.add_argument("--trace-level",choices=("summary","full"),default="summary",help="Diagnostic detail only; does not seed composition.")
+  parser.add_argument("--profile-output", type=pathlib.Path, help="Write build and per-role elapsed timings (not a search timeout).")
-  parser.add_argument("--profile-output",type=pathlib.Path,help="Write build and per-role elapsed timings (not a search timeout).")
   parser.add_argument("--progress", action="store_true")
   args = parser.parse_args()
   if not args.renderer_fingerprint and (args.data_dir is None or args.output_dir is None):
@@ -58,19 +60,22 @@ def generate_assets(args: argparse.Namespace) -> None:
   cache_dir = cache_dir.resolve() if cache_dir else None
   if cache_dir and (cache_dir.is_relative_to(output_dir) or output_dir.is_relative_to(cache_dir)):
     raise ValueError('engraving cache and generated assets must not overlap')
-  composition_cache = getattr(args,'composition_cache',None)
+  composition_cache = getattr(args, 'composition_cache', None)
   if composition_cache:
-    composition_cache=composition_cache.resolve()
+    composition_cache = composition_cache.resolve()
     if composition_cache.is_relative_to(output_dir) or output_dir.is_relative_to(composition_cache):
       raise ValueError('composition cache and generated assets must not overlap')
   progress = getattr(args, 'progress', False)
   started = time.monotonic()
-  timings=[];stages=[];last_stage=started
+  timings = []
+  stages = []
+  last_stage = started
 
   def report(message):
     nonlocal last_stage
-    now=time.monotonic()
+    now = time.monotonic()
-    stages.append({'completed_before':message,'seconds':now-last_stage});last_stage=now
+    stages.append({'completed_before': message, 'seconds': now - last_stage})
+    last_stage = now
     if progress:
       print(f'Assets [{time.monotonic() - started:.0f}s]: {message}', flush=True)
 
@@ -82,40 +87,39 @@ def generate_assets(args: argparse.Namespace) -> None:
     staged.mkdir(parents=True, exist_ok=True)
     copy_runtime_fonts(static_assets_dir, staged)
     report('generating performance parts')
-    performance_count = generator.arrange.generate.generate_performance_scores(data_dir / 'songs', staged / 'songs',
+    performance_count = generator.production.generate_tree(data_dir / 'songs', staged / 'songs', jobs=getattr(args, 'composition_jobs', 2), cache_dir=getattr(args, 'composition_cache', None), progress=progress, timings=timings)
-      composition_jobs=getattr(args,'composition_jobs',2), cache_dir=getattr(args,'composition_cache',None), progress=progress,timings=timings)
+    if getattr(args, 'trace_level', 'summary') == 'summary':
-    if getattr(args,'trace_level','summary')=='summary':
+      for path in (staged / 'analysis').glob('*.roles'):
-      for path in (staged/'analysis').glob('*.roles'):
+        roles = json.loads(path.read_text())
-        roles=json.loads(path.read_text())
         for role in roles.values():
-          trace=role.get('details',{}).get('selection_trace',{})
+          trace = role.get('details', {}).get('selection_trace', {})
-          trace.pop('decisions',None)
+          trace.pop('decisions', None)
-        path.write_text(json.dumps(roles,sort_keys=True,indent=2))
+        path.write_text(json.dumps(roles, sort_keys=True, indent=2))
-    if getattr(args, 'export_solo_xml', False):
+    generator.score_io.validate_generated_tree(staged)
-      from generator.solo.export import export_tree
-      export_tree(staged)
-    generator.score.validate_generated_tree(staged)
     if args.check:
       generator.summary.validate_expected_performance_summary(pathlib.Path(__file__).resolve().parents[1] / 'tests' / 'fixtures' / 'performance_summary.json', staged / 'songs')
-    report('compiling playback')
+    report('copying supplied recordings')
-    generator.playback.compile_tree(staged)
+    song_ids = {generator.score.slugify(path.stem) for path in (data_dir / 'songs').glob('*.xml')}
+    audio_assets = generator.audio_assets.copy_audio(static_assets_dir, staged, song_ids)
+    generator.audio_assets.validate_recording_lengths(staged, audio_assets)
     report('engraving pages')
     validated_pages = generator.engraving.pipeline.engrave_tree(staged, static_assets_dir / 'fonts' / 'petaluma', cache_dir=cache_dir, jobs=getattr(args, 'jobs', 1), progress=progress)
+    report('generating printable parts')
+    generator.printing.generate_tree(staged, static_assets_dir / 'fonts' / 'petaluma', cache_dir=cache_dir, progress=progress)
     report('rendering Pico titles')
     generator.pico_artwork.generate_titles(staged, static_assets_dir / 'fonts' / 'petaluma')
     report('validating score, page, and audio assets')
-    generator.score.validate_generated_tree(staged, validated_pages)
+    generator.score_io.validate_generated_tree(staged, validated_pages)
     report('building catalog database')
-    generator.catalog.build_catalog(staged)
+    generator.catalog.build_catalog(staged, audio_assets=audio_assets)
     with modulus.filesystem.publish_directories([(staged, output_dir)]):
       pass
     report('publishing complete')
-    profile_path=getattr(args,'profile_output',None)
+    profile_path = getattr(args, 'profile_output', None)
     if profile_path:
-      import platform,os
+      profile_path.parent.mkdir(parents=True, exist_ok=True)
-      profile_path.parent.mkdir(parents=True,exist_ok=True)
+      profile_path.write_text(json.dumps({'total_seconds': time.monotonic() - started, 'stages': stages, 'roles': timings,
-      profile_path.write_text(json.dumps({'total_seconds':time.monotonic()-started,'stages':stages,'roles':timings,
+        'platform': platform.platform(), 'cpus': os.cpu_count(), 'workers': getattr(args, 'composition_jobs', 2)}, indent=2))
-        'platform':platform.platform(),'cpus':os.cpu_count(),'workers':getattr(args,'composition_jobs',2)},indent=2))
     print(f'Generated Product Vamp assets: {performance_count} performance scores.')
     if args.check:
       print('Generator check passed.')

generator/acoustic.py

@@ -6,6 +6,8 @@ import itertools
 
 import generator.accompaniment
 import generator.model
+import generator.score
+import generator.harmonic_time
 
 @dataclasses.dataclass
 class HarmonySpan:
@@ -76,27 +78,107 @@ def prepare(document: dict) -> None:
       generator.accompaniment.normalize_beams(notes)
 
 def combine(document: dict) -> None:
-  """Write prepared bass/upper attacks as one voice with shared stems and rests."""
+  """One rhythmic voice; split held pitches into ties rather than release them."""
+  originals = {p['id']: p for m in document['staves'][0]['measures'] for v in m['voices'] for n in v['notes'] for p in n['pitches']}
+  edges = [(p['id'], p['tie_to']) for p in originals.values() if p.get('tie_to')]
+  fragments = {}
   for measure in measures(document):
     length = generator.model.fraction(measure['duration'])
-    attacks = sorted((n for v in measure['voices'] for n in v['notes'] if n['pitches']), key=lambda n: generator.model.fraction(n['onset']))
+    sounding = [(generator.model.fraction(n['onset']), generator.model.fraction(n['onset']) + generator.model.fraction(p['duration']), p)
-    notes, cursor = [], fractions.Fraction()
+      for v in measure['voices'] for n in v['notes'] for p in n['pitches']]
-
+    boundaries = sorted({fractions.Fraction(), length, *(t for a, b, _ in sounding for t in (a, b))})
-    def rest(end):
+    notes = []
-      nonlocal cursor
+    for a, b in zip(boundaries, boundaries[1:]):
-      for duration, written in generator.accompaniment.written_durations(end - cursor):
+      active = sorted((p for start, end, p in sounding if start <= a < end), key=lambda p: (generator.model.pitch_number(p), p['tablature']['string']))
-        notes.append(generator.model.make_event(f'{measure["id"]}_acoustic_{len(notes) + 1}', cursor, duration, 'rest', written=written) | {'voice': 'voice_arranged'})
+      for duration, written in generator.accompaniment.written_durations(b - a):
-        cursor += duration
+        pitches = [{k: v for k, v in p.items() if k not in ('tie_from', 'tie_to', 'stem')} for p in active]
-
+        note = generator.model.make_event(f'{measure["id"]}_acoustic_{len(notes) + 1}', a, duration, 'note' if pitches else 'rest', pitches, written)
-    for start, group in itertools.groupby(attacks, key=lambda n: generator.model.fraction(n['onset'])):
+        note['voice'] = 'voice_arranged'
-      rest(start)
+        for old, new in zip(active, note['pitches']):
-      group = list(group)
+          fragments.setdefault(old['id'], []).append(new)
-      pitches = sorted((p for n in group for p in n['pitches']), key=lambda p: (generator.model.pitch_number(p), p['tablature']['string']))
+        notes.append(note)
-      pitches = [{k: v for k, v in p.items() if k != 'stem'} for p in pitches]
+        a += duration
-      duration = generator.model.fraction(group[0]['duration'])
+    if not sounding and length == generator.model.meter_length(measure['meter']):
-      notes.append(generator.model.make_event(f'{measure["id"]}_acoustic_{len(notes) + 1}', start, duration, 'note', pitches, group[0]['written']) | {'voice': 'voice_arranged'})
-      cursor = start + duration
-    rest(length)
-    if not attacks and length == generator.model.meter_length(measure['meter']):
       notes = [generator.model.make_event(f'{measure["id"]}_acoustic_1', fractions.Fraction(), length, 'rest', written={'type': 'whole', 'dots': 0}) | {'voice': 'voice_arranged', 'measure_rest': True}]
     measure['voices'] = [{'id': 'voice_arranged', 'notes': notes}]
+  for pieces in fragments.values():
+    for a, b in zip(pieces, pieces[1:]):
+      a['tie_to'], b['tie_from'] = b['id'], a['id']
+  for start, end in edges:
+    if start in fragments or end in fragments:
+      a = fragments[start][-1] if start in fragments else originals[start]
+      b = fragments[end][0] if end in fragments else originals[end]
+      a['tie_to'], b['tie_from'] = b['id'], a['id']
+
+def sounding_lanes(measures):
+  """Recover complete string-lane events, preserving ties versus reattacks."""
+  tones, starts, cursor = {}, {}, fractions.Fraction()
+  for measure in measures:
+    for note in generator.score.measure_notes(measure):
+      for pitch in note['pitches']:
+        tones[pitch['id']] = pitch
+        starts[pitch['id']] = cursor + generator.model.fraction(note['onset'])
+    cursor += generator.model.fraction(measure['duration'])
+  groups = {}
+  for identifier, pitch in tones.items():
+    if pitch.get('tie_from') in tones:
+      if tones[pitch['tie_from']].get('tie_to') != identifier:
+        raise ValueError('broken acoustic sustain link')
+      continue
+    start, tail = starts[identifier], pitch
+    seen = {identifier}
+    while tail.get('tie_to') in tones:
+      target = tail['tie_to']
+      if target in seen:
+        raise ValueError('cyclic acoustic sustain')
+      continuation = tones[target]
+      if continuation.get('tie_from') != tail['id'] or starts[target] != starts[tail['id']] + generator.model.fraction(tail['duration']):
+        raise ValueError('broken acoustic sustain link or timing')
+      if generator.model.pitch_number(continuation) != generator.model.pitch_number(tail) or continuation.get('tablature') != tail.get('tablature'):
+        raise ValueError('acoustic sustain changes pitch or position')
+      seen.add(target)
+      tail = continuation
+    end = starts[tail['id']] + generator.model.fraction(tail['duration'])
+    string = pitch.get('tablature', {}).get('string')
+    if string is None:
+      raise ValueError('acoustic comping requires placed notes')
+    groups.setdefault((start, end), set()).add('bass' if string >= 5 else 'upper')
+  return groups
+
+def validate_assignment(measures, events):
+  expected = {}
+  for e in events:
+    lanes = ['bass', 'upper'] if e.lane == 'combined' else [e.lane]
+    expected.setdefault((e.start, e.end), set()).update(lanes)
+  if sounding_lanes(measures) != expected:
+    raise ValueError('rendered acoustic attacks disagree with the selected bass/upper assignment')
+
+def validate_comping(document: dict) -> None:
+  """Check complete sounding events, not barline or notation fragments."""
+  groups = sounding_lanes(document['staves'][0]['measures'])
+  events = []
+  for (start, end), lanes in groups.items():
+    if end <= start or (end - start < 1 and end - start != fractions.Fraction(1, 2)):
+      raise ValueError('acoustic comping requires quarter/eighth notes or authored sustains')
+    events.append((start, end))
+  events.sort()
+  # A combined attack is one interval with both lanes. Separate attacks must
+  # be sequential, including bass/upper alternation and cross-bar ties.
+  for i, a in enumerate(events):
+    for b in events[i + 1:]:
+      if a[1] > b[0]:
+        raise ValueError('acoustic sound must release before the next attack')
+  spans = []
+  for span in generator.harmonic_time.read(document['staves'][0]):
+    if spans and generator.accompaniment.same_chord(spans[-1].chord, span.chord):
+      spans[-1] = dataclasses.replace(spans[-1], end=span.end)
+    else:
+      spans.append(span)
+  for span in spans:
+    if span.chord is None:
+      continue
+    first = next((event for event in events if span.start <= event[0] < span.end), None)
+    # A bounded anticipated bass can already be sounding into this harmony.
+    anticipated = any(span.start - fractions.Fraction(1, 2) <= a < span.start < b and 'bass' in groups[a, b] for a, b in events)
+    if first and not anticipated and not any('bass' in groups[e] for e in events if e[0] == first[0]):
+      raise ValueError(f'measure {span.source_measure}, beat {span.beat}: first acoustic attack for a chord must include bass')

generator/acoustic_voicing.py

@@ -8,6 +8,7 @@ import typing
 import generator.acoustic
 import generator.fretboard
 import generator.harmony
+import generator.hand_position
 
 class Voicing(typing.NamedTuple):
   bass: generator.fretboard.TabPosition
@@ -35,11 +36,14 @@ def candidates(requirements: generator.fretboard.ChordRequirements, complete_tri
   return tuple(sorted(result))
 
 @functools.lru_cache(maxsize=16384)
-def movement(previous: Voicing, current: Voicing) -> int:
+def movement(previous: Voicing, current: Voicing, old_hand: int, hand: int) -> int:
-  return sum(abs(a.fret - b.fret) for a, b in zip(previous, current)) + abs(previous.bass.string_number - current.bass.string_number)
+  return generator.hand_position.motion(previous, current, old_hand, hand, True).shift
 
 class Cost(typing.NamedTuple):
+  extended_grips: int = 0
   travel: int = 0
+  peak_position: int = 0
+  effort: int = 0
   missing_tones: int = 0
 
 @dataclasses.dataclass
@@ -47,14 +51,20 @@ class Placement:
   cost: Cost
   voicing: Voicing | None = None
   previous: Placement | None = None
+  hand: int = 0
 
 def advance(options: tuple[Voicing, ...], requirements: generator.fretboard.ChordRequirements, states: list[Placement]) -> list[Placement]:
-  result = []
+  result = {}
   for voicing in options:
     missing = len(set(requirements.allowed) - {p.midi % 12 for p in voicing})
-    cost, index = min((Cost(p.cost.travel + (movement(p.voicing, voicing) if p.voicing else 0), p.cost.missing_tones + missing), i) for i, p in enumerate(states))
+    for hand in generator.hand_position.regions(voicing):
-    result.append(Placement(cost, voicing, states[index]))
+      for previous in states:
-  return result
+        motion = generator.hand_position.motion(previous.voicing, voicing, previous.hand, hand, True)
+        cost = Cost(previous.cost.extended_grips + generator.fretboard.span_penalty(voicing), previous.cost.travel + motion.shift, max(previous.cost.peak_position, hand), previous.cost.effort + motion.ergonomics, previous.cost.missing_tones + missing)
+        key = (voicing, hand, cost.peak_position)
+        if key not in result or cost < result[key].cost:
+          result[key] = Placement(cost, voicing, previous, hand)
+  return list(result.values())
 
 def select(spans: list[generator.acoustic.HarmonySpan]) -> list[Voicing]:
   states = [Placement(Cost())]

generator/arrange/accompaniment.py

@@ -1,60 +1,70 @@
 """One learned accompaniment per instrument, reused at authored form positions."""
 from __future__ import annotations
 
-import copy
+import fractions
-from fractions import Fraction as F
 
 import generator.accompaniment
-import generator.acoustic
+import generator.arrange.form
-import generator.acoustic_voicing
-import generator.electric
-import generator.guitar
-import generator.fretboard
-import generator.model
-import generator.parts
-import generator.score
-from generator.arrange import form
 import generator.arrange.pickup
-
+import generator.model
+import generator.score
 
 def harmonic_key(measure, previous):
+
   def key(chord):
     return repr({k: chord.get(k) for k in ('root', 'kind', 'degrees', 'bass', 'inversion')}) if chord else ''
-  active = generator.accompaniment.active_chord(measure, F(), previous)
+
+  active = generator.accompaniment.active_chord(measure, fractions.Fraction(), previous)
   return (key(active), tuple((generator.model.fraction(h['onset']), key(h)) for h in measure['harmonies']
     if generator.model.fraction(h['onset']) > 0), measure['duration'], measure['time_signature'])
 
-
+def audit(source, documents, *, role_documents=False):
-def audit(source, documents):
   """Compare music, not generated IDs, at every reused form position."""
   authored = generator.score.source_staff(source)['measures']
   result = {}
-  for part, document in documents.items():
+  for name, document in documents.items():
-    if part == 'vocal': continue
+    part = name.split('_')[0] if role_documents else name
-    sections = form.full_arrangement_sections(authored, part)
+    wanted = {'solo': 'slash', 'melody': 'vocal' if part == 'vocal' else 'melody'}.get(name.split('_')[-1], 'accompaniment') if role_documents else 'accompaniment'
+    if part == 'vocal' and not role_documents:
+      continue
+    sections = generator.arrange.form.full_arrangement_sections(authored, part)
+    entry_positions = {s.source_measure_index for s in sections if s.role == wanted}
     measures = document['staves'][0]['measures']
     carried = generator.accompaniment.carried_chords(measures)
     seen, exceptions, offset, reused = {}, [], 0, 0
     for si, section in enumerate(sections):
       following = sections[si + 1] if si + 1 < len(sections) else None
       for j in range(section.measure_count):
-        if section.role != 'accompaniment': continue
+        if section.role != wanted:
-        m = measures[offset+j]
+          continue
-        if j == section.measure_count-1 and following and following.role == 'melody' and generator.arrange.pickup.enters(following, authored):
+        m = measures[offset + j]
+        if wanted == 'accompaniment' and j == section.measure_count - 1 and following and following.role == 'melody' and generator.arrange.pickup.enters(following, authored):
           exceptions.append({'measure': m['number'], 'reason': 'protected melody pickup'})
           continue
-        key = (section.source_measure_index+j, repr(harmonic_key(m, carried[offset+j])))
+        if wanted == 'accompaniment' and j == section.measure_count - 1 and following and following.role == 'slash' and generator.score.body_start_measure_index(authored):
-        signature = tuple((generator.model.fraction(n['onset']), generator.model.fraction(n['duration']),
+          exceptions.append({'measure': m['number'], 'reason': 'source-defined solo pickup allocation'})
-          tuple((generator.model.pitch_number(p),p['tablature']['string'],p['tablature']['fret']) for p in n['pitches']))
+          continue
-          for n in generator.score.measure_notes(m))
+        position = section.source_measure_index + j
+        # A complete figure depends on both measures' effective harmony and
+        # source-defined participation. Changed final turnarounds/pickups are
+        # explicit contexts, not incidental changes between iterations.
+        first = generator.score.body_start_measure_index(authored)
+        block_start = first + 2 * ((position - first) // 2)
+        neighbors = range(max(section.source_measure_index, block_start), min(section.source_measure_index + section.measure_count, block_start + 2))
+        context = tuple(repr(harmonic_key(measures[offset + k - section.source_measure_index], carried[offset + k - section.source_measure_index])) for k in neighbors)
+        boundary = following.role if following and block_start + 2 >= section.source_measure_index + section.measure_count and generator.score.body_start_measure_index(authored) and following.role in ('slash', 'melody') else ''
+        key = (position, context, boundary)
+        signature = tuple((v['id'], generator.model.fraction(n['onset']), generator.model.fraction(n['duration']),
+          tuple((generator.model.pitch_number(p), generator.model.fraction(p['duration']), p.get('tablature', {}).get('string'), p.get('tablature', {}).get('fret'), bool(p.get('tie_from')) if position not in entry_positions or generator.model.fraction(n['onset']) else False, bool(p.get('tie_to'))) for p in n['pitches']))
+          for v in m['voices'] for n in v['notes'])
         if key in seen:
           if signature != seen[key]:
-            raise ValueError(f'{part}, arrangement measure {m["number"]}: canonical accompaniment changed')
+            raise ValueError(f'{name}, arrangement measure {m["number"]}: canonical {wanted} changed')
           reused += 1
         else:
           if any(k[0] == key[0] for k in seen):
-            exceptions.append({'measure': m['number'], 'reason': 'changed effective harmony or duration'})
+            exceptions.append({'measure': m['number'], 'reason': 'changed two-bar harmony, duration, or protected participation'})
           seen[key] = signature
       offset += section.measure_count
-    result[part] = {'reused_bars': reused, 'distinct_form_bars': len(seen), 'exceptions': exceptions}
+    result[name] = {'reused_bars': reused, 'distinct_form_bars': len(seen), 'exceptions': exceptions}
   return result

generator/arrange/assembly.py

@@ -0,0 +1,119 @@
+"""Assemble source-defined role windows without composing or respelling."""
+from __future__ import annotations
+import copy
+import fractions
+
+import generator.accompaniment
+import generator.arrange.form
+import generator.arrange.pickup
+import generator.composition_types
+import generator.fretboard
+import generator.model
+import generator.parts
+import generator.score
+import generator.validation
+
+def mask(document, allowed):
+  """Apply source-defined role exclusions, without observing another part."""
+  absolute = fractions.Fraction()
+  for measure in document['staves'][0]['measures']:
+    length = generator.model.fraction(measure['duration'])
+    for voice in measure['voices']:
+      selected = []
+      for n in voice['notes']:
+        start = absolute + generator.model.fraction(n['onset'])
+        end = start + generator.model.fraction(n['duration'])
+        intervals = [(max(start, a), min(end, b)) for a, b in allowed if a < end and b > start]
+        for a, b in intervals:
+          note = copy.deepcopy(n)
+          if a != start:
+            note['onset'] = generator.model.rational(a - absolute)
+            for pitch in note['pitches']:
+              pitch.pop('tie_from', None)
+          generator.arrange.pickup.truncate(note, b - absolute)
+          if a < b:
+            selected.append(note)
+      voice['notes'] = fill(selected, length, measure['id'] + '_' + voice['id'], voice['id'])
+    absolute += length
+  staff = document['staves'][0]
+  staff['tuplets'] = generator.arrange.pickup.retain_links(staff['measures'], staff.get('tuplets', []))
+
+def fill(notes, length, prefix, voice='voice_arranged'):
+  notes = sorted(notes, key=lambda n: generator.model.fraction(n['onset']))
+  result = []
+  cursor = fractions.Fraction()
+
+  def rests(end):
+    nonlocal cursor
+    for duration, written in generator.accompaniment.written_durations(end - cursor):
+      result.append(generator.model.make_event(prefix + '_rest_' + str(len(result)), cursor, duration, 'rest', written=written) | {'voice': voice})
+      cursor += duration
+
+  for n in notes:
+    start = generator.model.fraction(n['onset'])
+    if start < cursor:
+      raise ValueError(prefix + ': overlapping independently generated roles')
+    rests(start)
+    n['voice'] = voice
+    result.append(n)
+    cursor = start + generator.model.fraction(n['duration'])
+  rests(length)
+  return result
+
+def improvisation(source, template, part):
+  """Fill the source's solo allocation, including pickups, with beat slashes."""
+  document = copy.deepcopy(template)
+  staff = document['staves'][0]
+  for field in ('tuplets', 'octave_shifts', 'scale_spans'):
+    staff[field] = []
+  for measure in staff['measures']:
+    measure['id'] = part + '_improv_' + measure['id']
+    measure['voices'] = [{'id': 'voice_arranged', 'notes': generator.parts.generated_voice(measure, 'slash')}]
+  generator.score.reidentify(staff['measures'], [])
+  plan = generator.arrange.form.FormPlan.read(source.document(), part)
+  mask(document, [(a, b) for a, b, role in plan.windows if role == part + '_solo'])
+  return document
+
+def assemble(source, results, *, solos_enabled=True):
+  """Union prevalidated role windows; no selection or pitch/fingering rewrite."""
+  documents = {}
+  for part in generator.parts.PART_TYPES:
+    selected = [results[r]['document'] for r in generator.composition_types.ROLES
+      if r.startswith(part + '_') and (solos_enabled or not r.endswith('_solo'))]
+    if not solos_enabled and part != 'vocal':
+      selected.append(improvisation(source, selected[0], part))
+    document = copy.deepcopy(selected[0])
+    staff = document['staves'][0]
+    if part == 'vocal':
+      documents[part] = document
+      continue
+    staff['tuplets'] = []
+    staff['octave_shifts'] = []
+    staff['scale_spans'] = []
+    for other in selected:
+      for field in ('tuplets', 'octave_shifts', 'scale_spans'):
+        staff[field].extend(copy.deepcopy(other['staves'][0].get(field, [])))
+    for i, m in enumerate(staff['measures']):
+      # Tuplet rests belong to the source/role rhythm and must survive union.
+      lanes = {}
+      for d in selected:
+        for voice in d['staves'][0]['measures'][i]['voices']:
+          lane = voice['id'] if part == 'acoustic' and voice['id'] in ('upper', 'bass') else 'voice_arranged'
+          sounding = [copy.deepcopy(n) for n in voice['notes'] if n['pitches'] or n.get('tuplet_ids') or n.get('notehead') == 'slash']
+          if sounding:
+            lanes.setdefault(lane, []).extend(sounding)
+      lanes = lanes or {'voice_arranged': []}
+      m['voices'] = [{'id': lane, 'notes': fill(notes, generator.model.fraction(m['duration']), m['id'] + '_' + lane, lane)} for lane, notes in lanes.items()]
+      # Attribute changes belong to source melody pickups and are assembly metadata.
+      for d in selected:
+        other = d['staves'][0]['measures'][i]
+        if other.get('attributes'):
+          m['attributes'] = copy.deepcopy(other['attributes'])
+    staff['tuplets'] = generator.arrange.pickup.retain_links(staff['measures'], staff['tuplets'])
+    owners = {n['id']: v['id'] for m in staff['measures'] for v in m['voices'] for n in v['notes']}
+    for span in staff['tuplets']:
+      span['voice'] = owners[span['start']]
+    generator.validation.validate_document(document)
+    generator.fretboard.validate_document(document)
+    documents[part] = document
+  return documents

generator/arrange/ending.py

@@ -0,0 +1,72 @@
+"""The recorded three-bar hold, expressed once as source-derived score time."""
+import copy
+import fractions
+
+import generator.accompaniment
+import generator.acoustic_voicing
+import generator.arrange.ending_plan
+import generator.comping
+import generator.fretboard
+import generator.model
+import generator.parts
+import generator.score
+import generator.vocabulary
+
+def chord(source):
+  effective = [{**m, 'harmonies': [h for h in m['harmonies'] if not h.get('chord_optional')]} for m in source]
+  prior = generator.accompaniment.carried_chords(effective)[-1]
+  return generator.accompaniment.active_chord(effective[-1], generator.model.fraction(effective[-1]['duration']), prior)
+
+def append(source, output, part, *, sounding=False, center=55, composition=None, incoming=None):
+  composition = composition or generator.vocabulary.current()
+  if not source or not output:
+    return
+  model = generator.arrange.ending_plan.template(source)
+  harmony = chord(source)
+  grip = ()
+  if sounding:
+    if harmony is None:
+      raise ValueError('Ending: final performed harmony is missing or N.C.; cannot invent a closing chord')
+    options = generator.comping.voicings(generator.comping.semantic_key(harmony), part, composition=composition)
+    recent = {}
+    for m in reversed(output):
+      for n in sorted(generator.score.measure_notes(m), key=lambda n: generator.model.fraction(n['onset']), reverse=True):
+        for p in n['pitches']:
+          if 'tablature' in p:
+            tab = p['tablature']
+            recent.setdefault(tab['string'], generator.fretboard.TabPosition(tab['string'], tab['fret'], int(generator.model.pitch_number(p)) - 12))
+      if len(recent) >= 4:
+        break
+    previous = tuple(recent[s] for s in sorted(recent, reverse=True))
+    if part == 'acoustic':
+      bass = next((recent[s] for s in (5, 6) if s in recent), None)
+      previous = generator.acoustic_voicing.Voicing(bass, *(recent[s] for s in (4, 3, 2))) if bass and all(s in recent for s in (4, 3, 2)) else ()
+    if incoming is not None:
+      previous, hand = incoming
+    else:
+      hand = None
+    costs = {v: generator.comping.local_cost(part, generator.comping.semantic_key(harmony), v, center, composition=composition) for v in options}
+    path, _, _ = generator.comping.select_grips((options,), (costs,), part, 'ending-position', left=previous, left_hand=hand)
+    grip = path[0]
+  length = generator.model.fraction(model['duration'])
+  previous = None
+  for index in range(generator.arrange.ending_plan.BARS):
+    measure = generator.parts.build_measure(model, f'measure_{len(output) + 1}', 'rest', part, None, 'Ending' if index == 0 else '')
+    measure['ending_sustain'] = True
+    if index == 0 and harmony:
+      measure['harmonies'] = [{**copy.deepcopy(harmony), 'onset': generator.model.rational(0)}]
+      measure['harmonies'][0].pop('chord_optional', None)
+    if grip:
+      notes, cursor = [], fractions.Fraction()
+      for duration, written in generator.accompaniment.written_durations(length):
+        n = generator.model.make_event(f'ending_{part}_{index}_{len(notes)}', cursor, duration, 'note', generator.fretboard.chord_pitches(harmony, grip), written)
+        n['voice'] = 'voice_arranged'
+        if previous:
+          for a, b in zip(previous['pitches'], n['pitches']):
+            a['tie_to'], b['tie_from'] = b['id'], a['id']
+        notes.append(n)
+        previous = n
+        cursor += duration
+      measure['voices'] = [{'id': 'voice_arranged', 'notes': notes}]
+    output.append(measure)
+  generator.score.reidentify(output[-generator.arrange.ending_plan.BARS:], [])

generator/arrange/ending_plan.py

@@ -0,0 +1,22 @@
+"""Source-derived ending geometry, independent of instrument composition."""
+import copy
+import fractions
+
+import generator.model
+
+BARS = 3
+
+def template(source):
+  result = copy.deepcopy(source[-1])
+  # An ending is three full bars in the final active meter, never a pickup.
+  for change in result.get('attributes', []):
+    for key in ('meter', 'key', 'clef', 'transpose', 'staff_details'):
+      if key in change:
+        result[key] = copy.deepcopy(change[key])
+  result.update(duration=generator.model.rational(generator.model.meter_length(result['meter'])), implicit=False,
+    attributes=[], harmonies=[], voices=[], tempos=[], section_label='')
+  result['time_signature'] = generator.model.meter_label(result['meter'])
+  return result
+
+def duration(source):
+  return BARS * generator.model.fraction(template(source)['duration']) if source else fractions.Fraction()

generator/arrange/form.py

@@ -1,8 +1,12 @@
 from __future__ import annotations
 
 import dataclasses
+import fractions
 import typing
 
+import generator.arrange.ending_plan
+import generator.arrange.pickup
+import generator.model
 import generator.score
 
 @dataclasses.dataclass(frozen=True)
@@ -28,10 +32,10 @@ def section_role(part_type: str, section_index: int) -> typing.Literal["accompan
       return "slash"
     if section_index == 2:
       return "melody"
-    if section_index in {1, 3, 5, 6}:
+    if section_index in {1, 3, 5}:
       return "accompaniment"
     return "rest"
-  return "vocal" if section_index in {1, 5, 6} else "rest"
+  return "vocal" if section_index in {1, 5} else "rest"
 
 def full_arrangement_sections(measures: list[dict], part_type: str) -> list[ArrangementSection]:
   if not measures:
@@ -49,10 +53,63 @@ def full_arrangement_sections(measures: list[dict], part_type: str) -> list[Arra
     ("Acoustic Solo", body_start, body_measure_count, 3),
     ("Electric Solo", body_start, body_measure_count, 4),
     ("Vocal Melody", body_start, body_measure_count, 5),
-    ("Ending", edge_source_measure, edge_measure_count, 6),
   )
   return [
     ArrangementSection(label=label, source_measure_index=start, measure_count=count, section_index=index, role=section_role(part_type, index))
     for label, start, count, index in specs
     if count > 0
   ]
+
+@dataclasses.dataclass(frozen=True)
+class FormPlan:
+  sections: tuple[ArrangementSection, ...]
+  windows: tuple[tuple[fractions.Fraction, fractions.Fraction, str], ...]
+  stops: tuple[tuple[int, fractions.Fraction] | None, ...]
+  @classmethod
+  def read(cls, document, part):
+    source = generator.score.source_staff(document)['measures']
+    sections = tuple(full_arrangement_sections(source, part))
+    stops = tuple(section_stop(source, section, sections[i + 1] if i + 1 < len(sections) else None) for i, section in enumerate(sections))
+    return cls(sections, _windows(document, part, sections), stops)
+
+def section_stop(source, section, following):
+  if following is None or not generator.score.body_start_measure_index(source):
+    return None
+  last = section.source_measure_index + section.measure_count - 1
+  pad = generator.arrange.pickup.offset(source[last], source[0])
+  if following.role == 'slash':
+    return last, pad
+  if following.role == 'melody' and generator.arrange.pickup.enters(following, source):
+    attacks = [generator.model.fraction(note['onset']) for note in generator.score.measure_notes(source[0]) if note['pitches']]
+    if attacks:
+      return last, pad + min(attacks)
+  return None
+
+def windows(document, part):
+  return FormPlan.read(document, part).windows
+
+def _windows(document, part, sections):
+  source = generator.score.source_staff(document)['measures']
+  result, cursor = [], fractions.Fraction()
+  for section in sections:
+    duration = sum((generator.model.fraction(m['duration']) for m in source[section.source_measure_index:section.source_measure_index + section.measure_count]), fractions.Fraction())
+    role = {'slash': 'solo', 'vocal': 'melody'}.get(section.role, section.role)
+    if section.role in ('vocal', 'melody') and generator.arrange.pickup.enters(section, source) and result:
+      attacks = [generator.model.fraction(n['onset']) for n in generator.score.measure_notes(source[0]) if n['pitches']]
+      if attacks:
+        start = cursor - generator.model.fraction(source[0]['duration']) + min(attacks)
+        a, _, old = result[-1]
+        result[-1] = (a, start, old)
+        result.append((start, cursor, part + '_' + role))
+    end = cursor + duration
+    start = cursor
+    if section.role == 'slash' and generator.score.body_start_measure_index(source) and result:
+      # Allocate the source's complete pickup bar before composition. This
+      # does not inspect either generated solo; a role may leave it silent.
+      start = cursor - generator.model.fraction(source[0]['duration'])
+      a, _, old = result[-1]
+      result[-1] = (a, start, old)
+    result.append((start, end, part + '_' + role))
+    cursor = end
+  result.append((cursor, cursor + generator.arrange.ending_plan.duration(source), part + ('_accompaniment' if part != 'vocal' else '_rest')))
+  return tuple(result)

generator/arrange/generate.py

@@ -2,33 +2,31 @@ from __future__ import annotations
 
 import copy
 import fractions
-import pathlib
 
 import generator.accompaniment
-import generator.arrange.accompaniment
 import generator.acoustic
 import generator.acoustic_voicing
+import generator.arrange.ending
 import generator.arrange.form
 import generator.arrange.pickup
+import generator.comping
 import generator.electric
 import generator.fretboard
 import generator.guitar
 import generator.melody
 import generator.model
-import generator.musicxml
 import generator.parts
 import generator.score
-import generator.solo.arrangement
+import generator.vocabulary
 
-FINAL_SECTION_INDEX = 6
+FINAL_SECTION_INDEX = 5
 
-def arrange_document(document: dict, part_type: str) -> dict:
+def arrange_document(document: dict, part_type: str, *, composition=None) -> dict:
   """Canonical source-only template; solos are inserted by production roles."""
-  from generator import comping, vocabulary
+  composition = composition or generator.vocabulary.CompositionContext.create(generator.vocabulary.load(), part_type + '_accompaniment')
-  document=generator.parts.prepare_document(document)
+  document = generator.parts.prepare_document(document)
-  with vocabulary.using(vocabulary.load(),part_type+'_accompaniment'):
+  backing = generator.comping.Canonical(document, part_type, 'canonical', composition=composition) if part_type != 'vocal' else None
-    backing=comping.Canonical(document,part_type,'canonical') if part_type!='vocal' else None
+  return _arrange_document(document, part_type, accompaniment=backing, composition=composition)
-    return _arrange_document(document,part_type,accompaniment=backing)
 
 def section_source(source: list[dict], section, next_section=None) -> list[dict]:
   result = copy.deepcopy(source)
@@ -41,12 +39,14 @@ def section_source(source: list[dict], section, next_section=None) -> list[dict]
     generator.arrange.pickup.harmonies(destination, result[0])
   return result
 
-def _arrange_document(document: dict, part_type: str, plans=None, *, owned=None, accompaniment=None) -> dict:
+def _arrange_document(document: dict, part_type: str, plan=None, *, owned=None, accompaniment=None, composition=None, placement_report=None) -> dict:
+  composition = composition or generator.vocabulary.current()
   if part_type not in generator.parts.PART_TYPES:
     raise ValueError(f'unknown arrangement part: {part_type}')
   staff = generator.score.source_staff(document)
   source = staff['measures']
-  sections = generator.arrange.form.full_arrangement_sections(source, part_type)
+  plan = plan or generator.arrange.form.FormPlan.read(document, part_type)
+  sections = plan.sections
   output, tuplets, octave_shifts, voicing_origins = [], [], [], []
   absolute = fractions.Fraction()
   source_starts = []
@@ -59,10 +59,12 @@ def _arrange_document(document: dict, part_type: str, plans=None, *, owned=None,
     section_measures = section_source(source, section, sections[index + 1] if index + 1 < len(sections) else None)
     previous = generator.accompaniment.carried_chords(section_measures)
     segment = []
+    stop = plan.stops[index]
+    next_section = sections[index + 1] if index + 1 < len(sections) else None
     for si in range(section.source_measure_index, section.source_measure_index + section.measure_count):
       m = section_measures[si]
       if role == 'accompaniment' and accompaniment is not None:
-        result = accompaniment(m, previous[si], si)
+        result = accompaniment(m, previous[si], si, sources=section_measures, bounds=(section.source_measure_index, section.source_measure_index + section.measure_count), stop=stop)
         result['id'] = f'measure_{len(output) + len(segment) + 1}'
         result['section_label'] = section.label if not segment else ''
       else:
@@ -95,25 +97,29 @@ def _arrange_document(document: dict, part_type: str, plans=None, *, owned=None,
           octave_shifts.append({**shift, 'id': f'pickup_{section.section_index}_{shift["id"]}', 'start': generator.model.rational(offset + low), 'end': generator.model.rational(offset + high)})
     output.extend(segment)
     absolute += segment_length
+  if sections and sections[-1].section_index == FINAL_SECTION_INDEX:
+    generator.arrange.ending.append(source, output, part_type,
+      sounding=part_type != 'vocal' and (owned is None or 'accompaniment' in owned),
+      center=composition.role(part_type + '_accompaniment').get('center', 55), composition=composition, incoming=accompaniment.last_position if accompaniment is not None else None)
   tuplets = generator.arrange.pickup.retain_links(output, tuplets)
   result = generator.score.make_arranged_document(document, staff, staff_id='staff_arranged_' + part_type, staff_name=part_type.capitalize(), role='vocal' if part_type == 'vocal' else 'guitar', arrangement_part_type=part_type, measures=output, tuplets=tuplets, octave_shifts=octave_shifts)
   result['metadata'].setdefault('tempo', 124)
-  if part_type == 'acoustic' and accompaniment is None and (owned is None or 'accompaniment' in owned):
+  if part_type == 'acoustic':
-    generator.acoustic.prepare(result)
+    if accompaniment is None and (owned is None or 'accompaniment' in owned):
-    generator.acoustic_voicing.place(result)
+      generator.acoustic.prepare(result)
+      generator.acoustic_voicing.place(result)
+    generator.acoustic.validate_comping(result)
     generator.acoustic.combine(result)
   elif part_type == 'electric' and (owned is None or 'melody' in owned):
     generator.melody.harmonize(result)
-    generator.guitar.place_guitar(result)
+    report = generator.guitar.place_guitar(result, composition=composition)
     if generator.accompaniment.connect_continuations(result):
-      generator.guitar.place_guitar(result)
+      report = generator.guitar.place_guitar(result, composition=composition)
+    if placement_report is not None:
+      placement_report.update(report)
   # Source labels identify the head while arranging; published labels identify
   # the complete performance, including its lead-in and repeated choruses.
   for number, measure in enumerate(output, 1):
     measure['number'] = str(number)
   generator.fretboard.validate_document(result)
   return result
-
-def generate_performance_scores(source_dir: pathlib.Path, output_dir: pathlib.Path, *, composition_jobs=2, cache_dir=None, progress=False, timings=None) -> int:
-  from generator.production import generate_tree
-  return generate_tree(source_dir, output_dir, jobs=composition_jobs, cache_dir=cache_dir, progress=progress,timings=timings)

generator/audio_assets.py

@@ -0,0 +1,68 @@
+"""Validate and copy supplied recordings without synthesis or transformation."""
+from __future__ import annotations
+
+import fractions
+import hashlib
+import json
+import re
+import shutil
+import wave
+
+import generator.model
+import generator.performance
+
+TEMPOS = tuple(range(60, 121, 10))
+
+def describe_wav(path, root, *, song_id='', bpm=0):
+  if path.is_symlink() or not path.is_file() or not path.resolve().is_relative_to(root.resolve()):
+    raise ValueError(f'{path}: expected a contained regular WAV')
+  try:
+    with wave.open(str(path), 'rb') as audio:
+      channels, rate, frames = audio.getnchannels(), audio.getframerate(), audio.getnframes()
+      if audio.getcomptype() != 'NONE' or audio.getsampwidth() != 2 or channels not in (1, 2) or not 8000 <= rate <= 192000 or frames <= 0:
+        raise ValueError('expected nonempty mono/stereo PCM16 WAV')
+      remaining = frames
+      while remaining:
+        count = min(remaining, 65536)
+        if len(audio.readframes(count)) != count * channels * 2:
+          raise ValueError('truncated WAV data')
+        remaining -= count
+  except (wave.Error, EOFError) as error:
+    raise ValueError(f'{path}: invalid WAV: {error}') from error
+  with path.open('rb') as source:
+    digest = hashlib.file_digest(source, 'sha256').hexdigest()
+  return dict(path=path.relative_to(root).as_posix(), sha256=digest, channels=channels, sample_rate=rate, frames=frames, song_id=song_id, bpm=bpm)
+
+def copy_audio(static, destination, song_ids):
+  entries = []
+  inputs = [(static / 'sounds' / name, '', 0) for name in ('beat1.wav', 'beat2.wav')]
+  for path in sorted((static / 'backing').glob('*.wav')):
+    match = re.fullmatch(r'([a-z0-9]+(?:_[a-z0-9]+)*)_(60|70|80|90|100|110|120)\.wav', path.name)
+    if not match:
+      raise ValueError(f'{path}: unsupported backing filename or tempo')
+    if match[1] not in song_ids:
+      continue
+    inputs.append((path, match[1], int(match[2])))
+  for source, song_id, bpm in inputs:
+    description = describe_wav(source, static, song_id=song_id, bpm=bpm)
+    target = destination / description['path']
+    target.parent.mkdir(parents=True, exist_ok=True)
+    shutil.copyfile(source, target)
+    entries.append(description)
+  return entries
+
+def validate_recording_lengths(root, entries):
+  durations = {}
+  for path in sorted((root / 'songs').glob('*.json')):
+    document = json.loads(path.read_text())
+    performance = generator.performance.timing(document)
+    song_id = performance['song_id']
+    duration = generator.model.fraction(performance['duration']) + performance['count_in_beats']
+    if song_id in durations and durations[song_id] != duration:
+      raise ValueError(f'{song_id}: parts disagree about performance duration')
+    durations[song_id] = duration
+  for entry in entries:
+    if entry['song_id']:
+      seconds = fractions.Fraction(entry['frames'], entry['sample_rate'])
+      if seconds < durations[entry['song_id']] * 60 / entry['bpm']:
+        raise ValueError(f"{entry['path']}: backing recording is shorter than count-in and score")

generator/cache.py

@@ -0,0 +1,82 @@
+"""Content identities and atomic storage shared by generator cache adapters."""
+from __future__ import annotations
+import ast
+import contextlib
+import functools
+import hashlib
+import json
+import os
+import pathlib
+import shutil
+import tempfile
+
+ROOT = pathlib.Path(__file__).resolve().parent
+
+@functools.lru_cache(maxsize=256)
+def imports(content):
+  modules = set()
+  for node in ast.walk(ast.parse(content)):
+    if isinstance(node, ast.Import):
+      modules.update(alias.name for alias in node.names if alias.name.startswith('generator.'))
+    elif isinstance(node, ast.ImportFrom) and node.module and node.module.startswith('generator'):
+      modules.add(node.module)
+      modules.update(node.module + '.' + alias.name for alias in node.names)
+  return tuple(sorted(modules))
+
+def source_identity(entrypoints):
+  """Conservatively include the transitive imports of explicit stage roots.
+
+  Read bytes on every call, so edits in a long-lived build process invalidate
+  both code content and newly introduced dependencies. Only parsing is cached.
+  """
+  pending, sources = list(entrypoints), {}
+  while pending:
+    module = pending.pop()
+    relative = module.split('.')[1:]
+    path = ROOT.joinpath(*relative).with_suffix('.py') if relative else ROOT / '__init__.py'
+    if not path.is_file():
+      path = ROOT.joinpath(*relative, '__init__.py')
+    if path in sources or not path.is_file():
+      continue
+    content = path.read_bytes()
+    sources[path] = hashlib.sha256(content).hexdigest()
+    pending.extend(imports(content))
+    pending.extend('.'.join(module.split('.')[:i]) for i in range(1, len(module.split('.'))))
+  return {path.relative_to(ROOT).as_posix(): sources[path] for path in sorted(sources)}
+
+def fingerprint(value):
+  return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(',', ':')).encode()).hexdigest()
+
+def write_json(path, value):
+  path.parent.mkdir(parents=True, exist_ok=True)
+  temporary = None
+  try:
+    with tempfile.NamedTemporaryFile('w', dir=path.parent, delete=False) as output:
+      temporary = pathlib.Path(output.name)
+      json.dump(value, output, sort_keys=True)
+    os.replace(temporary, path)
+  finally:
+    if temporary is not None:
+      temporary.unlink(missing_ok=True)
+
+def has_symlink(path):
+  return any(parent.is_symlink() for parent in (path, *path.parents))
+
+@contextlib.contextmanager
+def stage_entry(entry, prefix):
+  if has_symlink(entry.parent):
+    raise ValueError('cache directories must not be symlinks')
+  entry.parent.mkdir(parents=True, exist_ok=True)
+  if entry.is_symlink() or entry.is_file():
+    entry.unlink(missing_ok=True)
+  entry.mkdir(exist_ok=True)
+  with tempfile.TemporaryDirectory(prefix=prefix, dir=entry.parent) as temporary:
+    yield pathlib.Path(temporary)
+
+def publish_files(entry, staged, names):
+  """Publish complete files in order, with the artifact manifest last."""
+  for name in names:
+    target = entry / name
+    if target.is_dir() and not target.is_symlink():
+      shutil.rmtree(target)
+    (staged / name).replace(target)

generator/catalog.py

@@ -6,6 +6,11 @@ import pathlib
 import re
 import sqlite3
 import tempfile
+import sys
+if __name__ == '__main__' and __package__ is None:
+  sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
+
+import generator.audio_assets
 
 APPLICATION_ID = 0x56414D50
 SONG_COLUMNS = """
@@ -40,22 +45,46 @@ def metadata(path: pathlib.Path, root: pathlib.Path) -> dict:
     raise ValueError('presentation requires a generated Pico title image; regenerate assets')
   if not isinstance(staff['name'], str):
     raise ValueError('staff name must be a string')
+  solos_enabled = meta.get('solos_enabled', True)
+  if type(solos_enabled) is not bool:
+    raise ValueError('solos_enabled must be a boolean')
   relative = path.relative_to(root)
   return dict(id=path.stem, title=meta['title'], composer=meta['composer'], key_signature=key,
     source_path=relative.as_posix(), source_json=text,
-    measure_count=len(staff['measures']), total_pages=len(presentation['pages']), title_image=title_image, part=staff['name'])
+    measure_count=len(staff['measures']), total_pages=len(presentation['pages']), title_image=title_image, part=staff['name'], solos_enabled=int(solos_enabled))
 
-def populate(connection: sqlite3.Connection, root: pathlib.Path) -> None:
+VARIANT_ERRORS = """
+  SELECT json_extract(source_json, '$.music.metadata.song_id'), part FROM songs
+  WHERE json_extract(source_json, '$.music.staves[0].arranged') = 1
+  GROUP BY json_extract(source_json, '$.music.metadata.song_id'), part
+  HAVING (part IN ('Acoustic', 'Electric') AND (count(*) != 2 OR sum(solos_enabled) != 1))
+    OR (part = 'Vocal' AND (count(*) != 1 OR sum(solos_enabled) != 1))
+"""
+
+def populate(connection: sqlite3.Connection, root: pathlib.Path, audio_assets=None) -> None:
   if not root.is_dir():
     raise ValueError(f'{root}: score asset directory not found')
+  if audio_assets is None:
+    audio_assets = []
+    for path in sorted(root.glob('sounds/*.wav')):
+      if path.name not in ('beat1.wav', 'beat2.wav'):
+        raise ValueError(f'{path}: unknown metronome sample')
+      audio_assets.append(generator.audio_assets.describe_wav(path, root))
+    for path in sorted(root.glob('backing/*.wav')):
+      match = re.fullmatch(r'([a-z0-9]+(?:_[a-z0-9]+)*)_(60|70|80|90|100|110|120)\.wav', path.name)
+      if not match:
+        raise ValueError(f'{path}: invalid backing filename')
+      audio_assets.append(generator.audio_assets.describe_wav(path, root, song_id=match[1], bpm=int(match[2])))
   folder = root / 'songs'
   if (folder.exists() or folder.is_symlink()) and not folder.is_dir():
     raise ValueError(f'{folder}: score asset directory not found')
   with connection:
     connection.execute('BEGIN')
     connection.execute(f'PRAGMA application_id = {APPLICATION_ID}')
-    connection.execute(f'CREATE TABLE songs ({SONG_COLUMNS}, part TEXT NOT NULL) STRICT')
+    connection.execute(f'CREATE TABLE songs ({SONG_COLUMNS}, part TEXT NOT NULL, solos_enabled INTEGER NOT NULL CHECK(solos_enabled IN (0, 1))) STRICT')
     connection.execute('CREATE INDEX songs_menu ON songs(title, composer, part, id)')
+    connection.execute('CREATE TABLE resources (id TEXT PRIMARY KEY, value TEXT NOT NULL CHECK(json_valid(value))) STRICT')
+    connection.execute('INSERT INTO resources VALUES (?, ?)', ('audio', json.dumps(audio_assets)))
     for path in sorted(folder.rglob('*.json')):
       try:
         if path.parent != folder:
@@ -64,10 +93,12 @@ def populate(connection: sqlite3.Connection, root: pathlib.Path) -> None:
         connection.execute(f'INSERT INTO songs ({", ".join(row)}) VALUES ({", ".join("?" for _ in row)})', tuple(row.values()))
       except (ValueError, KeyError, TypeError, sqlite3.Error) as error:
         raise ValueError(f'{path}: {error}') from error
+    if connection.execute(VARIANT_ERRORS).fetchall():
+      raise ValueError('missing or duplicate solo variants; regenerate score assets')
     if connection.execute('PRAGMA integrity_check').fetchall() != [('ok',)]:
       raise ValueError('catalog integrity check failed')
 
-def build_catalog(root: pathlib.Path, destination: pathlib.Path | None = None) -> pathlib.Path:
+def build_catalog(root: pathlib.Path, destination: pathlib.Path | None = None, *, audio_assets=None) -> pathlib.Path:
   if not root.is_dir():
     raise ValueError(f'{root}: score asset directory not found')
   destination = destination or root / 'catalog.db'
@@ -76,7 +107,7 @@ def build_catalog(root: pathlib.Path, destination: pathlib.Path | None = None) -
     staged = pathlib.Path(temporary) / 'catalog.db'
     connection = sqlite3.connect(staged)
     try:
-      populate(connection, root)
+      populate(connection, root, audio_assets)
     finally:
       connection.close()
     staged.replace(destination)

generator/chord_families.py

@@ -0,0 +1,56 @@
+"""Transposable guitar families, derived from the written chord and legal grips."""
+import generator.fretboard
+import generator.harmony
+import generator.model
+import generator.vocabulary
+
+def acoustic(chord, grip, *, composition=None):
+  composition = composition or generator.vocabulary.current()
+  degrees = generator.harmony.degrees(chord)
+  root = int(generator.model.NATURAL[chord['root']['letter']] + generator.model.alteration(chord['root']['alter']))
+  classes = {d: (root + generator.harmony.degree_semitones(d, a)) % 12 for d, a in degrees.items()}
+  guide = next((d for d in (3, 4, 2) if d in degrees), None)
+  terminal = next((d for d in (7, 6) if d in degrees), None)
+  color = max((d for d in degrees if d > 7), default=None)
+  kind = 'extended' if terminal and color else 'tetrad' if terminal else 'triad'
+  bass = next((d for d in (1, 5, 3) if classes.get(d) == grip[0].midi % 12), None)
+  slots = {'root': 1, 'fifth': 5, 'guide': guide, 'terminal': terminal, 'color': color}
+  policy = composition.accompaniment('acoustic')['voicing_families']
+  orders = policy[kind].get(f'{grip[0].string_number}/{bass}', ())
+  upper = tuple(p.midi % 12 for p in grip[1:])
+  preferred = any(all(slots[s] in classes for s in order) and tuple(classes[slots[s]] for s in order) == upper for order in orders)
+  # The physical/harmonic enumerator remains authoritative: an explicit bass
+  # or an unrepresented extension can use another valid compact voicing.
+  return {'id': f'{kind}/s{grip[0].string_number}/bass{bass or "written"}/' + '-'.join(str(next((d for d in sorted(classes) if pc == classes[d]), 'bass')) for pc in upper),
+    'preferred': preferred, 'bass_degree': bass, 'bass_string': grip[0].string_number,
+    'relative_frets': [p.fret - grip[0].fret for p in grip]}
+
+def electric(grip):
+  """Classify actual voicing geometry, without inventing chord permissions."""
+  pitches = sorted(p.midi for p in grip)
+  if len(grip) == 3:
+    return 'three-note-shell'
+  if len(set(p % 12 for p in pitches)) < len(pitches):
+    return 'compact-doubled'
+  if len(pitches) == 4:
+    lifted = sorted([pitches[0] + 12, *pitches[1:]])
+    if lifted[-1] - lifted[0] < 12 and lifted[2] == pitches[0] + 12:
+      return 'drop-two'
+  return 'close-position' if pitches and pitches[-1] - pitches[0] < 12 else 'compact-open'
+
+def coverage(chord, options, part, *, composition=None):
+  composition = composition or generator.vocabulary.current()
+  if part == 'electric':
+    return {name: sum(electric(v) == name for v in options) for name in composition.accompaniment('electric')['voicing_families']['rank']}
+  families = [acoustic(chord, v, composition=composition) for v in options]
+  result = []
+  for string in (5, 6):
+    for degree in (1, 5):
+      found = [f for f in families if f['bass_degree'] == degree and f['bass_string'] == string]
+      result.append({'bass_string': string, 'bass_degree': degree, 'candidates': len(found), 'preferred': sum(f['preferred'] for f in found),
+        'reason': '' if found else 'no legal grip for this bass under written harmony and current fret/span limits'})
+  return result
+
+def span_coverage(options):
+  return {'primary': sum(not generator.fretboard.span_penalty(v) for v in options),
+    'fallback': sum(generator.fretboard.span_penalty(v) for v in options)}

generator/comping.py

@@ -1,123 +1,497 @@
-"""Source-only sparse accompaniment, frozen once at each form position."""
+"""Source-only accompaniment: joint layered rhythms and connected grips."""
 from __future__ import annotations
 import copy
+import dataclasses
+import fractions
 import functools
 import json
-from fractions import Fraction as F
+import typing
-
-from generator import accompaniment, fretboard, guitar, harmony, model, parts, score, vocabulary
-from generator.solo.policy import stable_rank
 
+import generator.accompaniment
+import generator.acoustic
+import generator.acoustic_voicing
+import generator.arrange.form
+import generator.chord_families
+import generator.comping_rhythm
+import generator.fretboard
+import generator.guitar
+import generator.harmony
+import generator.hand_position
+import generator.model
+import generator.parts
+import generator.score
+import generator.solo.policy
+import generator.vocabulary
 
 def semantic_key(chord):
   return json.dumps({k: chord[k] for k in ('root', 'kind', 'degrees', 'bass', 'inversion') if k in chord}, sort_keys=True)
 
+def requirements(chord, part):
+  result = generator.fretboard.chord_requirements(chord, respect_bass=part != 'electric')
+  if part == 'electric' or result.basses:
+    return result
+  degrees = generator.harmony.degrees(chord)
+  plain = degrees in ({1: 0, 3: 0, 5: 0}, {1: 0, 3: -1, 5: 0})
+  root = int(generator.model.NATURAL[chord['root']['letter']] + generator.model.alteration(chord['root']['alter']))
+  basses = tuple(sorted({(root + generator.harmony.degree_semitones(d, degrees[d])) % 12 for d in ((1, 3, 5) if plain else (1, 5)) if d in degrees}))
+  if not basses:
+    raise ValueError('acoustic accompaniment requires a written bass, root or fifth')
+  return dataclasses.replace(result, basses=basses)
+
+def voicings(key, part, *, composition=None):
+  return _voicings(key, part, composition or generator.vocabulary.current())
 
 @functools.lru_cache(maxsize=2048)
-def voicings(key, bass):
+def _voicings(key, part, composition):
   chord = json.loads(key)
-  degrees = harmony.degrees(chord)
+  required = requirements(chord, part)
-  root = int(model.NATURAL[chord['root']['letter']] + model.alteration(chord['root']['alter']))
+  if part == 'electric':
-  classes = {d: (root + harmony.degree_semitones(d, a)) % 12 for d, a in degrees.items()}
+    options = generator.guitar.chord_positions(required, prune=False, profile='electric', composition=composition)
-  required = {p for d, p in classes.items() if d in (2, 3, 4, 6, 7) or d == 5 and degrees[d] or d > 7}
+  elif part == 'acoustic':
-  full = fretboard.chord_requirements(chord)
+    degrees = generator.harmony.degrees(chord)
-  required.update(full.basses)
+    plain = degrees in ({1: 0, 3: 0, 5: 0}, {1: 0, 3: -1, 5: 0})
-  if bass and 1 in classes:
+    options = tuple(v for v in generator.acoustic_voicing.candidates(required, plain)
-    required.add(classes[1])
+      if v.bass.fret <= generator.fretboard.MAX_SINGLE_FRET
-  if len(required) < 2:
+      and generator.fretboard.has_fret_anchor(v[1:])
-    required.update(classes[d] for d in (1, 5) if d in classes)
+      and (not plain or set(required.allowed) <= {p.midi % 12 for p in v}))
-  allowed = set(full.allowed)
+  else:
-  requirements = fretboard.ChordRequirements(tuple(sorted(required)), tuple(sorted(allowed)), tuple(sorted(allowed)), full.basses)
+    raise ValueError('Unknown accompaniment instrument: ' + part)
-  options = guitar.chord_positions(requirements, prune=False)
+  options = tuple(v for v in options if generator.fretboard.playable_attack((p, fractions.Fraction(1)) for p in v))
-  size = max(2, len(required))
-  options = [v for v in options if len(v) == size and fretboard.playable_attack((p, F(1)) for p in v)]
   if not options:
-    raise ValueError(harmony.symbol(chord) + ': no legal sparse voicing preserving essential written degrees')
+    layout = 'bass on 5/6 plus upper strings 2/3/4' if part == 'acoustic' else 'melody chord families on strings 1–4'
-  # Retain distinct hand regions and inversions, independent of enumeration.
+    raise ValueError(f'{generator.harmony.symbol(chord)}: no legal {part} voicing ({layout}); required degrees/bass, no open strings, fret anchor 1–12 and at most {generator.fretboard.MAX_FRET_SPAN} consecutive frets remain binding')
-  best = {}
+  return tuple(sorted(options))
-  for v in options:
-    family = (min(p.fret for p in v), min(p.midi for p in v) % 12)
-    rank = (guitar.compactness(v), stable_rank('comp-voicing', v))
-    if family not in best or rank < best[family][0]:
-      best[family] = rank, v
-  return tuple(v for _, v in sorted(best.values()))
 
+def movement(part, previous, current, old_hand, hand):
+  return generator.hand_position.motion(previous, current, old_hand, hand, part == 'acoustic').shift
+
+class VoicingCost(typing.NamedTuple):
+  extended_grips: int = 0
+  family: int = 0
+  travel: int = 0
+  peak_position: int = 0
+  effort: fractions.Fraction = fractions.Fraction()
+  missing_tones: int = 0
+  register: fractions.Fraction = fractions.Fraction()
+  def plus(self, other):
+    return VoicingCost(*(max(a, b) if field == 'peak_position' else a + b for field, a, b in zip(self._fields, self, other)))
+  def moved(self, distance, ergonomics=0):
+    return self._replace(travel=self.travel + distance, effort=self.effort + ergonomics)
+
+def local_cost(part, key, grip, center, *, composition=None):
+  composition = composition or generator.vocabulary.current()
+  required = requirements(json.loads(key), part)
+  register = generator.guitar.voicing_preferences(grip, generator.guitar.voicing_preference(part + '_accompaniment', center, composition=composition))
+  if part == 'electric':
+    value = generator.guitar.voicing_cost(grip, 'electric', required.allowed, composition=composition)
+    return VoicingCost(extended_grips=value.extended_grips, family=3 * value.string_shortfall + value.family_penalty, effort=value.effort, missing_tones=value.missing_tones, register=register)
+  # Keep the familiar families, then compare physical effort before position.
+  # Sounding register only breaks otherwise equal placement decisions.
+  family = generator.chord_families.acoustic(json.loads(key), grip, composition=composition)
+  return VoicingCost(extended_grips=generator.fretboard.span_penalty(grip), family=int(not family['preferred']), missing_tones=len(set(required.allowed) - {p.midi % 12 for p in grip}), register=register)
+
+def select_grips(layers, costs, part, seed, *, left=(), right=(), left_hand=None, right_hand=None, close=False, fixed_hands=None):
+  """Return complete grips, cost and hand windows, including repeat travel.
+
+  A concrete grip may occupy several windows. Preserve distinct earlier peaks
+  at a common grip/hand, and fix both starting values for a cyclic path.
+  """
+  if not layers:
+    return (), VoicingCost(), ()
+
+  def hands(grip, fixed=None):
+    options = generator.hand_position.regions(grip)
+    if fixed is not None:
+      return tuple(h for h in options if h == fixed)
+    return options
+
+  retained = []
+  for i, layer in enumerate(layers):
+    tier = min(costs[i][v].extended_grips for v in layer)
+    fixed = fixed_hands[i] if fixed_hands is not None else None
+    options = tuple((v, h) for v in layer if costs[i][v].extended_grips == tier for h in hands(v, fixed))
+    if not options:
+      raise ValueError(f'accompaniment span {i}: no grip fits the committed hand position')
+    retained.append(options)
+  ranks = [{v: generator.solo.policy.stable_rank(seed, (i, v)) for v, _ in layer} for i, layer in enumerate(retained)]
+  winners = []
+  for first in retained[0] if close else [None]:
+    states = {(left, h, h): (VoicingCost(peak_position=h), (), (), ()) for h in hands(left, left_hand)}
+    for i, layer in enumerate(retained):
+      following = {}
+      for grip, hand in [first] if i == 0 and first is not None else layer:
+        local = costs[i][grip]._replace(peak_position=hand)
+        for (previous, old_hand, _), (cost, ties, path, positions) in states.items():
+          motion = generator.hand_position.motion(previous, grip, old_hand, hand, part == 'acoustic')
+          total = cost.plus(local).moved(motion.shift, motion.ergonomics)
+          candidate = (total, (*ties, (ranks[i][grip], hand)), (*path, grip), (*positions, hand))
+          key = (grip, hand, total.peak_position)
+          if key not in following or candidate < following[key]:
+            following[key] = candidate
+      # Future travel is identical at a common concrete grip and hand.
+      best = {}
+      for (grip, hand, _), value in following.items():
+        primary = value[0][:3]
+        best[grip, hand] = min(best.get((grip, hand), primary), primary)
+      states = {key: value for key, value in following.items() if value[0][:3] == best[key[:2]]}
+    endpoints = [first] if close else [(right, h) for h in hands(right, right_hand)]
+    for (grip, hand, _), (cost, ties, path, positions) in states.items():
+      for endpoint, end_hand in endpoints:
+        motion = generator.hand_position.motion(grip, endpoint, hand, end_hand, part == 'acoustic')
+        total = cost.plus(VoicingCost(peak_position=end_hand)).moved(motion.shift, motion.ergonomics)
+        winners.append((total, ties, path, positions))
+  if not winners:
+    raise ValueError('accompaniment boundary grip does not fit the committed hand position')
+  cost, _, path, positions = min(winners)
+  return path, cost, positions
+
+@dataclasses.dataclass(frozen=True)
+class HarmonyChoice:
+  index: int
+  start: fractions.Fraction
+  end: fractions.Fraction
+  chord: dict
+  events: tuple
+  figure: dict
+  options: tuple
+  hand: int | None = None
+  @property
+  def key(self):
+    return (self.index, self.start, self.end, semantic_key(self.chord), self.events)
+
+def split_measures(flat, inputs, part):
+  """Project a selected sounding figure into notation; barlines create ties only."""
+  output, fragments, original = [], {}, {}
+  cursor = fractions.Fraction()
+  for m, prior, index in inputs:
+    length = generator.model.fraction(m['duration'])
+    result = generator.parts.build_measure(m, f'comp_{part}_{index}', 'rest', part, prior)
+    voices = []
+    for voice in flat['voices']:
+      notes = []
+      for n in voice['notes']:
+        start = generator.model.fraction(n['onset'])
+        end = start + generator.model.fraction(n['duration'])
+        a, b = max(cursor, start), min(cursor + length, end)
+        if a >= b:
+          continue
+        for duration, written in generator.accompaniment.written_durations(b - a):
+          pitches = [{k: v for k, v in p.items() if k not in ('tie_from', 'tie_to')} for p in n['pitches']]
+          event = generator.model.make_event(f'comp_{part}_{index}_{voice["id"]}_{len(notes)}', a - cursor, duration, 'note' if pitches else 'rest', pitches, written)
+          event['voice'] = voice['id']
+          for old, new in zip(n['pitches'], event['pitches']):
+            fragments.setdefault(old['id'], []).append(new)
+            original[old['id']] = old
+          notes.append(event)
+          a += duration
+      voices.append({'id': voice['id'], 'notes': notes})
+    result['voices'] = voices
+    for chord in result['harmonies']:
+      chord.pop('chord_optional', None)
+    output.append(result)
+    cursor += length
+  for pieces in fragments.values():
+    for a, b in zip(pieces, pieces[1:]):
+      a['tie_to'], b['tie_from'] = b['id'], a['id']
+  for identifier, old in original.items():
+    if old.get('tie_to') in fragments:
+      a, b = fragments[identifier][-1], fragments[old['tie_to']][0]
+      a['tie_to'], b['tie_from'] = b['id'], a['id']
+  return tuple(output)
 
 class Canonical:
-  def __init__(self, document, part, seed):
+  def __init__(self, document, part, seed, *, composition=None, plan=None):
+    self.composition = composition or generator.vocabulary.current()
     self.part, self.seed = part, seed
-    self.policy = vocabulary.role(part + '_accompaniment')
+    self.policy = self.composition.role(part + '_accompaniment')
-    self.cache, self.previous, self.report = {}, (), []
+    self.cache, self._reports, self._rhythm_reports = {}, {}, {}
-    measures = score.source_staff(document)['measures']
+    self.cache_positions = {}
-    self.first_index = score.body_start_measure_index(measures)
+    self.last_position = ((), 0)
-    self.last_index = len(measures)-1
+    self.measures = generator.score.source_staff(document)['measures']
-    carried = accompaniment.carried_chords(measures)
+    # A held accompaniment chord may continue into the bar containing the
-    for index in range(score.body_start_measure_index(measures), len(measures)):
+    # source melody pickup. Keep its voice identity through that splice.
-      source = copy.deepcopy(measures[index])
+    self.voice_id = self.measures[0]['voices'][0]['id']
-      for h in source['harmonies']:
+    self.melody = generator.comping_rhythm.melody_contexts(document)
-        h.pop('chord_optional', None)
+    self.first = generator.score.body_start_measure_index(self.measures)
-      self(source, carried[index], index)
+    plan = plan or generator.arrange.form.FormPlan.read(document, part)
+    sections = plan.sections
+    self.closed = any(a.role == b.role == 'accompaniment' and a.source_measure_index + a.measure_count == len(self.measures)
+      and b.source_measure_index == self.first for a, b in zip(sections, sections[1:]))
+    carried = generator.accompaniment.carried_chords(self.measures)
+    self.groups = []
+    index = self.first
+    while index < len(self.measures):
+      size = 1
+      if index + 1 < len(self.measures):
+        lengths = [generator.model.fraction(m['duration']) for m in self.measures[index:index + 2]]
+        if self.measures[index]['time_signature'] == self.measures[index + 1]['time_signature'] and any(tuple(map(fractions.Fraction, p['lengths'])) == tuple(lengths) for p in self.composition.accompaniment(part)['patterns']):
+          size = 2
+      self.groups.append(tuple((self.measures[i], carried[i], i) for i in range(index, index + size)))
+      index += size
+    self.acoustic_cache = {}
+    self.pair_breaks = {self.first, len(self.measures)}
+    self.pair_stops = []
+    for section in sections:
+      if section.role == 'accompaniment':
+        self.pair_breaks.update((section.source_measure_index, section.source_measure_index + section.measure_count))
+    self.pair_stops = [stop for section, stop in zip(sections, plan.stops) if section.role == 'accompaniment' and stop is not None]
+    breaks = [(i, fractions.Fraction()) for i in self.pair_breaks] + self.pair_stops
+    choices, locations, self.groove_report = generator.comping_rhythm.choose_cycles(self.groups, part, self.melody, seed, breaks=breaks, composition=self.composition)
+    self.patterns = {g[0][2]: (p, a) for g, (p, a, _) in zip(self.groups, choices)}
+    self.locations = {g[0][2]: location for g, location in zip(self.groups, locations)}
+    selected = [r for _, _, r in choices]
+    self.block_rhythms = {group[0][2]: rhythm for group, rhythm in zip(self.groups, selected)}
+    prepared = [self.prepare(group, rhythm) for group, rhythm in zip(self.groups, selected)]
+    active = tuple(span for spans in prepared for span in spans if span.events)
+    self.canonical_spans = {group[0][2]: spans for group, spans in zip(self.groups, prepared)}
+    path, self.path_cost, self.hand_order = self.select(active, close=self.closed)
+    self.base = dict(zip((s.key for s in active), path))
+    self.hands = dict(zip((s.key for s in active), self.hand_order))
+    self.order = tuple((s.index, s.start, v) for s, v in zip(active, path))
+    self.rhythms = {}
+    previous = (path[-1], self.hand_order[-1]) if self.closed and path else ((), 0)
+    for group, rhythm, spans in zip(self.groups, selected, prepared):
+      grips = tuple((self.base[s.key], self.hands[s.key]) for s in spans if s.events)
+      key = (self.key(group), rhythm.signature) if part == 'acoustic' else self.key(group)
+      self.cache[key], self.cache_positions[key] = self.render(group, rhythm, spans, grips, previous, False)
+      cursor = fractions.Fraction()
+      for m, _, i in group:
+        length = generator.model.fraction(m['duration'])
+        self.rhythms[i] = dataclasses.replace(rhythm, events=tuple(dataclasses.replace(e, start=max(e.start, cursor) - cursor, end=min(e.end, cursor + length) - cursor, harmonic_time=e.harmonic_time - cursor if e.harmonic_time is not None else None) for e in rhythm.events if e.start < cursor + length and e.end > cursor))
+        cursor += length
+      if grips:
+        previous = grips[-1]
+  @property
+  def report(self):
+    return [row for key in sorted(self._reports, key=repr) for row in self._reports[key]]
+  @property
+  def placement_report(self):
+    previous = self.hand_order[-1] if self.closed and self.hand_order else None
+    shifts = []
+    for hand in self.hand_order:
+      shifts.append(abs(hand - previous) if previous is not None else 0)
+      previous = hand
+    return {'version': generator.hand_position.VERSION, 'priority': 'hand travel, maximum hand position, ergonomics, coverage/register, stable tie',
+      **generator.hand_position.summary(shifts), 'peak_position': self.path_cost.peak_position,
+      'cost': {k: str(v) for k, v in self.path_cost._asdict().items()},
+      'positions': [{'source_index': i, 'onset': str(t), 'hand_position': h} for (i, t, _), h in zip(self.order, self.hand_order)]}
+  @property
+  def rhythm_report(self):
+    return [row for key in sorted(self._rhythm_reports, key=repr) for row in self._rhythm_reports[key]]
+  @staticmethod
+  def key(inputs, limit=None):
+    return (tuple((i, str(generator.model.fraction(m['duration'])), tuple((str(a), str(b), semantic_key(c) if c else '') for a, b, c in generator.comping_rhythm.spans(m, p))) for m, p, i in inputs), str(limit))
+  def candidates(self, inputs, limit=None):
+    flat = generator.comping_rhythm.flatten(inputs)
+    melody, cursor, lengths = [], fractions.Fraction(), []
+    for m, _, i in inputs:
+      melody.extend((a + cursor, b + cursor) for a, b in self.melody[i])
+      lengths.append(generator.model.fraction(m['duration']))
+      cursor += lengths[-1]
+    return generator.comping_rhythm.candidates(flat, None, self.part, tuple(melody), self.seed + '/' + str(inputs[0][2]), lengths=tuple(lengths), limit=limit, composition=self.composition)
+  def acoustic_realizations(self, sources, bounds=None, stop=None):
+    """Assign the canonical attack stream once, before splitting it into blocks.
 
-  def __call__(self, source, prior, index):
+    A variant uses the same patterns with the source's actual harmony and
-    key = (index, tuple((model.fraction(h['onset']), semantic_key(h)) for h in source['harmonies']), semantic_key(prior) if prior else '', source['duration'].__repr__())
+    participation boundaries. No selected notes from another role enter here.
-    if key in self.cache:
+    """
-      return copy.deepcopy(self.cache[key])
+    carried = generator.accompaniment.carried_chords(sources)
-    length = model.fraction(source['duration'])
+    full = tuple((m, carried[i], i) for i, m in enumerate(sources))
-    boundaries = sorted({F(), length, *(model.fraction(h['onset']) for h in source['harmonies'])})
+    key = (self.key(full), bounds, stop)
-    attacks = []
+    if key in self.acoustic_cache:
-    source_attacks = {model.fraction(n['onset']) for n in score.measure_notes(source) if n['pitches'] and not all('tie_from' in p for p in n['pitches'])}
+      return self.acoustic_cache[key]
-    for start, end in zip(boundaries, boundaries[1:]):
+    starts = [fractions.Fraction()]
-      chord = accompaniment.active_chord(source, start, prior)
+    for m in sources:
+      starts.append(starts[-1] + generator.model.fraction(m['duration']))
+    breaks = [(i, fractions.Fraction()) for i in self.pair_breaks | set(bounds or ())] + self.pair_stops
+    if stop:
+      breaks.append(stop)
+    groups, raw = [], []
+    for original in self.groups:
+      group = tuple(full[i] for _, _, i in original)
+      index = group[0][2]
+      pattern, anticipate = self.patterns[index]
+      limit = starts[stop[0]] + stop[1] - starts[index] if stop and any(i == stop[0] for _, _, i in group) else None
+      rhythm = generator.comping_rhythm.realize(pattern, group, 'acoustic', self.melody, anticipate=anticipate, limit=limit, boundary=True, assign=False)
+      if rhythm is None:
+        raise ValueError(f'source measure {sources[index]["number"]}: selected acoustic rhythm cannot fit written harmony')
+      raw.append(rhythm)
+      groups.append(group)
+    context = generator.comping_rhythm.acoustic_context(groups, self.melody, breaks)
+    realized = generator.comping_rhythm.realize_acoustic_form(raw, context)
+    result = {g[0][2]: r for g, r in zip(groups, realized)}
+    self.acoustic_cache[key] = result
+    return result
+  def prepare(self, inputs, rhythm):
+    flat = generator.comping_rhythm.flatten(inputs)
+    spans = []
+    offsets, cursor = [], fractions.Fraction()
+    for m, _, i in inputs:
+      offsets.append((cursor, i))
+      cursor += generator.model.fraction(m['duration'])
+    for start, end, chord in generator.comping_rhythm.spans(flat, None):
       if not chord:
         continue
-      duration = end - start
+      offset, index = next((a, i) for a, i in reversed(offsets) if a <= start)
-      # Essential bass is a source-derived form responsibility, independent
+      events = tuple(e for e in rhythm.events if start <= (e.harmonic_time if e.harmonic_time is not None else e.start) < end)
-      # of which other guitar happens to be playing or what it generated.
+      try:
-      bass = self.policy.get('bass') == 'primary' or bool(chord.get('bass')) or index in (self.first_index,self.last_index) or start > 0
+        options = voicings(semantic_key(chord), self.part, composition=self.composition)
-      options = voicings(semantic_key(chord), bass)
+      except ValueError as error:
-      center = self.policy.get('center', 55)
+        raise ValueError(f'source measure {inputs[index - inputs[0][2]][0]["number"]}, beat {start - offset + 1}: {error}') from error
-      selected = min(options, key=lambda v: (
+      spans.append(HarmonyChoice(index, start, end, chord, events, dict(rhythm.families), options))
-        .08 * abs(sum(p.midi for p in v) / len(v) - center)
+    return tuple(spans)
-        + (.15 * guitar.movement(self.previous, v) if self.previous else 0)
+  def select(self, spans, *, left=(), right=(), left_hand=None, right_hand=None, close=False):
-        + .1 * guitar.compactness(v), stable_rank(self.seed, (index, start, v))))
+    layers = tuple(s.options for s in spans)
-      self.previous = selected
+    costs = tuple({v: local_cost(self.part, semantic_key(s.chord), v, self.policy.get('center', 55), composition=self.composition) for v in layer} for s, layer in zip(spans, layers))
-      patterns = []
+    return select_grips(layers, costs, self.part, self.seed, left=left, right=right, left_hand=left_hand, right_hand=right_hand, close=close, fixed_hands=tuple(s.hand for s in spans))
-      for figure in vocabulary.data('accompaniment')['comping']:
+  def __call__(self, source, prior, index, *, sources=None, bounds=None, stop=None):
-        events = tuple((start+F(a), min(end, start+F(a)+F(d))) for a, d in figure['attacks'] if F(a) < duration)
+    group = next((g for g in self.groups if any(i == index for _, _, i in g)), ((source, prior, index),))
-        # Space is available on held harmony, never an accidental empty winner.
+    original_group = group
-        if not events and (not prior or not accompaniment.same_chord(chord, prior) or start):
+    sources = sources or self.measures
-          continue
+    if self.part == 'acoustic' and source != sources[index]:
-        conflicts = sum(a in source_attacks for a, _ in events)
+      sources = [source if i == index else m for i, m in enumerate(sources)]
-        target = 1 if duration <= 3 else 1.5
+    carried = generator.accompaniment.carried_chords(sources)
-        cost = .2*abs(len(events)-target) + .08*conflicts
+    low, high = bounds or (self.first, len(sources))
-        if figure['id'] == 'sustain' and duration < 3:
+    group = tuple((source if i == index else sources[i], prior if i == index else carried[i], i) for _, _, i in group if low <= i < high)
-          cost += .2
+    limit = None
-        patterns.append((cost, stable_rank(self.seed, (index, start, figure['id'])), events, figure['id']))
+    if stop and any(i == stop[0] for _, _, i in group):
-      # Source-specific variation within a broad acceptability band, not a loop.
+      limit = sum((generator.model.fraction(m['duration']) for m, _, i in group if i < stop[0]), fractions.Fraction()) + stop[1]
-      low = min(p[0] for p in patterns)
+    key = self.key(group, limit)
-      _, _, events, label = min((p for p in patterns if p[0] <= low+.25), key=lambda p:p[1])
+    complete = None
-      for a, b in events:
+    if self.part == 'acoustic' and original_group[0][2] in self.patterns:
-        attacks.append((a, b, selected, chord))
+      complete = self.acoustic_realizations(sources, (low, high), stop)[original_group[0][2]]
-      self.report.append({'source_index': index, 'onset': str(start), 'chord': harmony.symbol(chord), 'figure': label, 'tones': len(selected), 'bass': bass,
+      key = (key, complete.signature)
-        'fuller_voicing_reason': 'explicit written degrees/bass' if len(selected)>3 else ''})
+    if key not in self.cache:
-    notes, cursor = [], F()
+      start, end = group[0][2], group[-1][2]
-    def emit(a, b, positions=(), chord=None):
+      offset = sum((generator.model.fraction(m['duration']) for m, _, i in original_group if i < start), fractions.Fraction())
-      nonlocal cursor
+      if original_group[0][2] in self.patterns:
-      previous = None
+        pattern, anticipate = self.patterns[original_group[0][2]]
-      for duration, written in accompaniment.written_durations(b-a):
+        if self.part == 'acoustic':
-        pitches = fretboard.chord_pitches(chord, positions) if positions else []
+          end_time = offset + (limit if limit is not None else sum(generator.model.fraction(m['duration']) for m, _, _ in group))
-        n = model.make_event(f'comp_{self.part}_{index}_{len(notes)}', a, duration, 'note' if pitches else 'rest', pitches, written)
+          rhythm = dataclasses.replace(complete, events=tuple(dataclasses.replace(e, start=max(e.start, offset) - offset, end=min(e.end, end_time) - offset, harmonic_time=e.harmonic_time - offset if e.harmonic_time is not None and e.harmonic_time >= offset else None)
-        if previous:
+            for e in complete.events if e.start < end_time and e.end > offset))
-          for old, new in zip(previous['pitches'], n['pitches']):
+        elif offset:
-            old['tie_to'], new['tie_from'] = new['id'], old['id']
+          # Fit the incoming context before slicing. A raw template sustain
-        notes.append(n); previous = n; a += duration
+          # may already have ended at a chord change; projecting it first
-      cursor = b
+          # would resurrect that note as an unintended new downbeat attack.
-    for start, end, positions, chord in attacks:
+          context = tuple((source if i == index else sources[i], prior if i == index else carried[i], i)
-      if cursor < start: emit(cursor, start)
+            for _, _, i in original_group if i <= end)
-      emit(start, end, positions, chord)
+          incoming = generator.comping_rhythm.realize(pattern, context, self.part, self.melody, anticipate=anticipate, limit=offset + limit if limit is not None else None, boundary=True)
-    if cursor < length: emit(cursor, length)
+          if incoming is None:
-    result = parts.build_measure(source, 'canonical', 'rest', self.part, prior)
+            raise ValueError(f'source measure {source["number"]}: selected groove cannot fit incoming harmony')
-    result['voices'] = [{'id': 'voice_arranged', 'notes': notes}]
+          pattern = {**pattern, 'events': [{'onset': str(e.start), 'duration': str(e.end - e.start),
-    for note in notes: note['voice'] = 'voice_arranged'
+            'layer': e.lane, 'anticipate': e.harmonic_time is not None and e.lane in ('upper', 'electric')} for e in incoming.events]}
-    self.cache[key] = copy.deepcopy(result)
+        if self.part != 'acoustic':
-    return result
+          pattern = generator.comping_rhythm.project(pattern, offset, tuple(generator.model.fraction(m['duration']) for m, _, _ in group))
+          rhythm = generator.comping_rhythm.realize(pattern, group, self.part, self.melody, anticipate=anticipate, limit=limit, boundary=True)
+        if rhythm is None:
+          raise ValueError(f'source measure {source["number"]}: selected groove cannot fit protected boundary')
+      else:
+        # A source pickup outside the repeating body uses the existing partial
+        # vocabulary. It never selects or changes the body groove.
+        rhythm = min(self.candidates(group, limit), key=lambda r: (r.cost, generator.solo.policy.stable_rank(self.seed, r.signature)))
+      spans = self.prepare(group, rhythm)
+      # Keep every unchanged harmonic interval's canonical grip, even when a
+      # neighbor loses an optional turnaround or a section starts mid-pattern.
+      fixed = []
+      for span in spans:
+        original = next((s for s in self.canonical_spans.get(original_group[0][2], ()) if s.key in self.base
+          and s.start <= span.start + offset < s.end and semantic_key(s.chord) == semantic_key(span.chord)), None)
+        fixed.append(dataclasses.replace(span, options=(self.base[original.key],), hand=self.hands[original.key]) if original else span)
+      spans = tuple(fixed)
+      before = [(v, h) for (i, _, v), h in zip(self.order, self.hand_order) if i < start]
+      after = [(v, h) for (i, _, v), h in zip(self.order, self.hand_order) if i > end]
+      left = before[-1] if before else (self.order[-1][2], self.hand_order[-1]) if self.closed and self.order else ((), 0)
+      right = after[0] if after else (self.order[0][2], self.hand_order[0]) if self.closed and self.order else ((), 0)
+      selected, _, hands = self.select(tuple(s for s in spans if s.events), left=left[0], right=right[0], left_hand=left[1], right_hand=right[1])
+      self.cache[key], self.cache_positions[key] = self.render(group, rhythm, spans, tuple(zip(selected, hands)), left, True, limit)
+    placement = self.cache_positions[key][index - group[0][2]]
+    if placement is not None:
+      self.last_position = placement
+    return copy.deepcopy(self.cache[key][index - group[0][2]])
+  def render(self, inputs, rhythm, spans, selected, previous, variant, limit=None):
+    flat = generator.comping_rhythm.flatten(inputs)
+    length = generator.model.fraction(flat['duration'])
+    index = inputs[0][2]
+    attacks, rows, placed_events = [], [], []
+    choices = iter(selected)
+    for span in spans:
+      grip, hand = next(choices) if span.events else ((), 0)
+      motion = generator.hand_position.motion(previous[0], grip, previous[1], hand, self.part == 'acoustic')
+      bass = grip[0] if grip and self.part == 'acoustic' else None
+      degrees = generator.harmony.degrees(span.chord)
+      root = int(generator.model.NATURAL[span.chord['root']['letter']] + generator.model.alteration(span.chord['root']['alter']))
+      rows.append({'source_index': span.index, 'onset': str(span.start), 'chord': generator.harmony.symbol(span.chord),
+        'figure': span.figure, 'pattern_id': rhythm.pattern_id, 'tones': len(grip), 'bass': bool(bass), 'variant': variant,
+        'family': (generator.chord_families.acoustic(span.chord, grip, composition=self.composition) if self.part == 'acoustic' else generator.chord_families.electric(grip)) if grip else None,
+        'family_coverage': generator.chord_families.coverage(span.chord, span.options, self.part, composition=self.composition),
+        'fret_span': generator.fretboard.fret_span(grip),
+        'span_group': ('fallback' if generator.fretboard.span_penalty(grip) else 'primary') if grip else None,
+        'span_candidates': generator.chord_families.span_coverage(span.options),
+        'positions': [{'string': p.string_number, 'fret': p.fret, 'sounding_midi': p.midi} for p in grip],
+        'bass_degree': next((d for d, a in degrees.items() if bass and (root + generator.harmony.degree_semitones(d, a)) % 12 == bass.midi % 12), None),
+        'bass_string': bass.string_number if bass else None,
+        'movement': motion.shift, 'voicing_candidates': len(span.options)})
+      if grip:
+        local = local_cost(self.part, semantic_key(span.chord), grip, self.policy.get('center', 55), composition=self.composition)
+        rows[-1]['hand_position'] = hand
+        rows[-1]['voicing_cost'] = {'movement': motion.shift, 'hand_shift': motion.shift, 'finger_adjustment': motion.fingers,
+          'string_changes': motion.strings, 'grip_effort': str(local.effort), 'register_tiebreak': str(local.register),
+          'secondary_effort': str(motion.ergonomics + local.effort)}
+      for event in span.events:
+        placed_events.append((event.start, event.end, (grip, hand)))
+        lanes = ['bass', 'upper'] if event.lane == 'combined' else [event.lane]
+        for lane in lanes:
+          positions = (grip[0],) if lane == 'bass' else grip[1:] if lane == 'upper' else grip
+          attacks.append((lane, event.start, event.end, positions, span.chord))
+      if grip:
+        previous = (grip, hand)
+    key = self.key(inputs, limit)
+    if self.part == 'acoustic':
+      key = (key, rhythm.signature)
+    measure_hands, cursor = [], fractions.Fraction()
+    for m, _, _ in inputs:
+      end = cursor + generator.model.fraction(m['duration'])
+      played = sorted((a, b, p) for a, b, p in placed_events if a < end and b > cursor)
+      measure_hands.append(played[-1][2] if played else None)
+      cursor = end
+    self._reports[key] = rows
+    voices = []
+    for lane in ['upper', 'bass'] if self.part == 'acoustic' else ['electric']:
+      voice_id = lane if self.part == 'acoustic' else self.voice_id
+      notes, cursor = [], fractions.Fraction()
+      def emit(a, b, positions=(), chord=None):
+        previous = None
+        for duration, written in generator.accompaniment.written_durations(b - a):
+          pitches = generator.fretboard.chord_pitches(chord, positions) if positions else []
+          n = generator.model.make_event(f'comp_{self.part}_{index}_{lane}_{len(notes)}', a, duration, 'note' if pitches else 'rest', pitches, written)
+          n['voice'] = voice_id
+          if previous and pitches:
+            for old, new in zip(previous['pitches'], n['pitches']):
+              old['tie_to'], new['tie_from'] = new['id'], old['id']
+          notes.append(n)
+          previous = n
+          a += duration
+      for _, start, end, positions, chord in sorted((e for e in attacks if e[0] == lane), key=lambda e: e[1]):
+        if cursor < start:
+          emit(cursor, start)
+        emit(start, end, positions, chord)
+        cursor = end
+      if cursor < length:
+        emit(cursor, length)
+      voices.append({'id': voice_id, 'notes': notes})
+    result = split_measures({'voices': voices}, inputs, self.part)
+    if self.part == 'acoustic':
+      generator.acoustic.validate_assignment(result, rhythm.events)
+    reports, cursor = [], fractions.Fraction()
+    for m, _, i in inputs:
+      duration = generator.model.fraction(m['duration'])
+      reports.append({'source_index': i, 'pattern_id': rhythm.pattern_id, 'pattern_start': index,
+        **next((self.locations[g[0][2]] for g in self.groups if any(j == i for _, _, j in g)), {}),
+        'families': dict(rhythm.families), 'contributions': dict(rhythm.contributions), 'selection_cost': rhythm.cost,
+        'exceptions': rhythm.exceptions, 'source_melody': [(str(a), str(b)) for a, b in self.melody[i]],
+        'events': [{'layer': e.lane, 'onset': str(max(e.start, cursor) - cursor), 'duration': str(min(e.end, cursor + duration) - max(e.start, cursor)),
+          'continuation': e.start < cursor, 'harmonic_target': str(e.harmonic_time - cursor) if e.harmonic_time is not None else None} for e in rhythm.events if e.start < cursor + duration and e.end > cursor]})
+      cursor += duration
+    self._rhythm_reports[key] = reports
+    return result, tuple(measure_hands)

generator/comping_rhythm.py

@@ -0,0 +1,547 @@
+"""Complete, source-only comping figures. Time is in rational quarter notes."""
+import dataclasses
+import fractions
+import itertools
+
+import generator.accompaniment
+import generator.model
+import generator.solo.policy
+import generator.source
+import generator.vocabulary
+
+def meter_context(measure):
+  """Nominal meter and explicit additive grouping, independent of bar length.
+
+  In particular, a three-quarter-note pickup or a 6/8 bar is not 3/4.
+  """
+  groups = measure['meter']['groups']
+  if not groups or any(g['unit'] != 4 for g in groups):
+    raise ValueError(f"source measure {measure['number']}: no comping vocabulary for {measure['time_signature']}")
+  beats = tuple(b for g in groups for b in g['beats'])
+  total = sum(beats)
+  if total not in (3, 4, 5) or (len(beats) > 1 and beats not in ((3, 2), (2, 3))):
+    raise ValueError(f"source measure {measure['number']}: unsupported comping grouping {measure['time_signature']}")
+  return f'{total}/4', beats if len(beats) > 1 else ()
+
+def grouping_for(run, melody):
+  """Choose one source-only quintuple feel, shared by independent roles.
+
+  Written grouping wins. Otherwise compare actual chord changes, entrances
+  after an eighth-note breath, then all melodic attacks at the group boundary.
+  Counts are evidence, not inferred accents or performance dynamics.
+  """
+  meter, explicit = meter_context(run[0][0][0])
+  if meter != '5/4':
+    return (), {'reason': 'simple_meter'}
+  if explicit:
+    return explicit, {'reason': 'written'}
+  evidence = {}
+  for boundary in (3, 2):
+    changes, entrances, attacks = 0, 0, 0
+    for group in run:
+      for m, prior, i in group:
+        changes += sum(a == boundary and c is not None for a, _, c in spans(m, prior))
+        context = melody[i]
+        for a, _ in context:
+          if a != boundary:
+            continue
+          attacks += 1
+          previous = max((b for x, b in context if x < a), default=fractions.Fraction())
+          entrances += a - previous >= fractions.Fraction(1, 2)
+    evidence[boundary] = (changes, entrances, attacks)
+  boundary = 2 if evidence[2] > evidence[3] else 3
+  return (boundary, 5 - boundary), {'reason': 'source' if evidence[2] != evidence[3] else 'default_3+2',
+    'boundary_evidence': {str(k): dict(zip(('chord_changes', 'melody_entrances', 'melody_attacks'), v)) for k, v in evidence.items()}}
+
+def patterns_for(policy, measure, lengths, grouping=()):
+  meter, explicit = meter_context(measure)
+  grouping = explicit or grouping or ((3, 2) if meter == '5/4' else ())
+  # Untagged in-memory figures are allowed for independently authored unit
+  # tests; the resource loader requires metadata for every production entry.
+  return [p for p in policy['patterns'] if tuple(map(fractions.Fraction, p['lengths'])) == tuple(lengths)
+    and (p.get('meter') == 'partial' or (p.get('meter', meter) == meter and tuple(p.get('grouping', grouping)) == grouping))]
+
+def relevant_policy(policy, measures):
+  """An unused meter's vocabulary must not influence this role's decisions."""
+  meters = {meter_context(m)[0] for m in measures}
+  lengths = {generator.model.fraction(m['duration']) for m in measures}
+  return {**policy, 'patterns': [p for p in policy['patterns'] if p.get('meter') in meters
+    or (p.get('meter') == 'partial' and fractions.Fraction(p['lengths'][0]) in lengths)]}
+
+@dataclasses.dataclass(frozen=True, order=True)
+class Attack:
+  lane: str
+  start: fractions.Fraction
+  end: fractions.Fraction
+  harmonic_time: fractions.Fraction | None = None
+
+@dataclasses.dataclass(frozen=True)
+class Rhythm:
+  events: tuple
+  families: tuple
+  contributions: tuple
+  exceptions: tuple = ()
+  pattern_id: str = ''
+  @property
+  def cost(self):
+    return sum(value for _, value in self.contributions)
+  @property
+  def signature(self):
+    return tuple((e.lane, str(e.start), str(e.end - e.start), str(e.harmonic_time)) for e in self.events)
+
+def melody_contexts(document):
+  melody = generator.source.attacks(document)
+  result, cursor = [], fractions.Fraction()
+  for m in document['staves'][0]['measures']:
+    length = generator.model.fraction(m['duration'])
+    result.append(tuple((n.start - cursor, n.end - cursor) for n in melody if n.start < cursor + length and n.end > cursor))
+    cursor += length
+  return tuple(result)
+
+def spans(measure, prior):
+  length = generator.model.fraction(measure['duration'])
+  boundaries = sorted({fractions.Fraction(), length, *(generator.model.fraction(h['onset']) for h in measure['harmonies'])})
+  result = []
+  for a, b in zip(boundaries, boundaries[1:]):
+    chord = generator.accompaniment.active_chord(measure, a, prior)
+    if result and generator.accompaniment.same_chord(result[-1][2], chord):
+      result[-1] = (result[-1][0], b, chord)
+    else:
+      result.append((a, b, chord))
+  return tuple(result)
+
+def flatten(inputs):
+  """Harmony/timing adapter only; never concatenate separately chosen rhythms."""
+  first = inputs[0][0]
+  result = {**first, 'harmonies': []}
+  cursor = fractions.Fraction()
+  for m, prior, _ in inputs:
+    chord = generator.accompaniment.active_chord(m, fractions.Fraction(), prior)
+    initial = chord or {'kind': 'none'}
+    result['harmonies'].append({**initial, 'onset': generator.model.rational(cursor)})
+    result['harmonies'].extend({**h, 'onset': generator.model.rational(cursor + generator.model.fraction(h['onset']))} for h in m['harmonies'] if generator.model.fraction(h['onset']))
+    cursor += generator.model.fraction(m['duration'])
+  result['duration'] = generator.model.rational(cursor)
+  return result
+
+def validate_events(events, part, lengths=None):
+  ordered = sorted(events, key=lambda e: e.start)
+  for e in ordered:
+    duration = e.end - e.start
+    if duration <= 0:
+      raise ValueError('comping event must have positive sounding duration')
+    if part == 'acoustic':
+      if e.lane not in ('bass', 'upper', 'combined') or (duration < 1 and duration != fractions.Fraction(1, 2)):
+        raise ValueError('acoustic attacks require quarter/eighth notes or authored sustains')
+    elif e.lane != 'electric':
+      raise ValueError('electric comping requires chord events')
+  for i, a in enumerate(ordered):
+    for b in ordered[i + 1:]:
+      if a.end > b.start:
+        raise ValueError('comping sound must release before the next attack')
+
+def assign_acoustic(events, harmony, breaks=()):
+  """Assign attacks, not notation fragments, across a complete source span.
+
+  Pair left to right within an effective harmony. An anticipated attack is
+  already playing its target harmony. Form/participation boundaries break
+  pairs; ordinary barlines and computational blocks do not.
+  """
+  ordered = sorted(events, key=lambda e: e.start)
+  if len({e.start for e in ordered}) != len(ordered):
+    raise ValueError('acoustic rhythm must have one event per attack position')
+
+  def region(event):
+    time = event.harmonic_time if event.harmonic_time is not None else event.start
+    return next(i for i, (a, b, _) in enumerate(harmony) if a <= time < b)
+
+  lanes, i = ['combined'] * len(ordered), 0
+  while i + 1 < len(ordered):
+    a, b = ordered[i:i + 2]
+    if b.start - a.start == fractions.Fraction(1, 2) and region(a) == region(b) and not any(a.start < t <= b.start for t in breaks):
+      lanes[i:i + 2] = ['bass', 'upper']
+      i += 2
+    else:
+      i += 1
+  result = [dataclasses.replace(e, lane=lane) for e, lane in zip(ordered, lanes)]
+  # One rhythm-guitar stream: every new attack releases the previous sound,
+  # including a change of lane. A combined attack has one shared release.
+  for i, a in enumerate(result):
+    following = result[i + 1].start if i + 1 < len(result) else a.end
+    result[i] = dataclasses.replace(a, end=min(a.end, following))
+  # Shortening must not strand an anticipation before its promised arrival.
+  # Rebind that event to the current chord, then reconsider its pair once.
+  expired = {e.start for e in result if e.harmonic_time is not None and e.end <= e.harmonic_time}
+  if expired:
+    rebound = []
+    for e in ordered:
+      if e.start in expired:
+        boundary = next(b for a, b, _ in harmony if a <= e.start < b)
+        e = dataclasses.replace(e, harmonic_time=None, end=min(e.end, boundary))
+      rebound.append(e)
+    return assign_acoustic(rebound, harmony, breaks)
+  validate_events(result, 'acoustic')
+  return tuple(result)
+
+def fit(pattern, measure, prior, part, *, anticipate=False, limit=None, assign=True):
+  """Bind a complete figure to written harmony, with bounded early arrivals."""
+  length = generator.model.fraction(measure['duration'])
+  limit = length if limit is None else min(length, limit)
+  harmony = spans(measure, prior)
+  events, exceptions, available = [], [], []
+  for row in pattern['events']:
+    start = fractions.Fraction(row['onset'])
+    end = min(start + fractions.Fraction(row['duration']), limit)
+    if start >= limit:
+      continue
+    active = next(((a, b, c) for a, b, c in harmony if a <= start < b), None)
+    if active is None or active[2] is None:
+      exceptions.append(('space', 'no written harmony at ' + str(start)))
+      continue
+    available.append((dict(row), start, end, active))
+  lengths = tuple(map(fractions.Fraction, pattern.get('lengths', (length,))))
+  for row, start, end, active in available:
+    _, boundary, chord = active
+    target = None
+    if anticipate and row.get('anticipate'):
+      future = next(((a, b, c) for a, b, c in harmony if a == start + fractions.Fraction(1, 2) and c), None)
+      if future and future[0] < end and not generator.accompaniment.same_chord(chord, future[2]):
+        target, boundary = future[0], future[1]
+    end = min(end, boundary)
+    events.append(Attack('acoustic' if part == 'acoustic' else row['layer'], start, end, target))
+  try:
+    if part == 'acoustic' and assign:
+      events = assign_acoustic(events, harmony)
+      exceptions.append(('layer_assignment', 'combined attacks; same-harmony eighth pairs use bass then upper'))
+    elif part == 'acoustic' and any(e.end <= e.start or (e.end - e.start < 1 and e.end - e.start != fractions.Fraction(1, 2)) for e in events):
+      raise ValueError('acoustic articulation cannot fit this boundary')
+    if part != 'acoustic' or assign:
+      validate_events(events, part, lengths)
+  except ValueError:
+    return None
+  return tuple(events), tuple(exceptions)
+
+def evaluate(events, part, melody, harmony, lengths):
+  activity, support, coverage, articulation = 0., 0., 0., 0.
+  cursor = fractions.Fraction()
+  for length in lengths:
+    local = [e for e in events if cursor <= e.start < cursor + length]
+    attacks = [a for a, b in melody if cursor <= a < cursor + length]
+    busy = len(attacks) >= length
+    sounding = any(e.start < cursor + length and e.end > cursor for e in events)
+    target = (1 if busy else 1.5) if part == 'electric' else (2 if busy else 2.5)
+    # Sustained occupancy and planned space are not missing attacks.
+    activity += .22 * max(0, abs(len({e.start for e in local}) - target) - .75) if not (sounding and not local) else 0
+    # A bass paired with the upper chord is one rhythmic attack, regardless
+    # of whether unequal releases require two internal lane events.
+    attacks_by_time = {e.start: max((v for v in local if v.start == e.start), key=lambda v: v.end) for e in local}
+    for e in attacks_by_time.values():
+      gap = not any(a <= e.start < b for a, b in melody)
+      entrance = any(a == e.start and (i == 0 or a - melody[i - 1][1] >= fractions.Fraction(1, 2)) for i, (a, _) in enumerate(melody))
+      support += -.2 if entrance else -.12 if gap else .18 if busy and e.start in attacks else 0
+      articulation += .08 if busy and e.end - e.start > 2 else 0
+    cursor += length
+  for a, b, chord in harmony:
+    if chord and b - a >= 1 and not any(e.start < b and e.end > a for e in events):
+      coverage += .18  # A preference, never a chord-attack quota.
+  divisor = max(1, len(lengths))
+  return tuple((k, v / divisor) for k, v in (('activity', activity), ('melody_support', support), ('harmonic_support', coverage), ('articulation', articulation)))
+
+def candidates(measure, prior, part, melody, seed, *, lengths=None, limit=None, grouping=(), composition=None):
+  composition = composition or generator.vocabulary.current()
+  lengths = lengths or (generator.model.fraction(measure['duration']),)
+  policy = composition.accompaniment(part)
+  result = []
+  for pattern in patterns_for(policy, measure, lengths, grouping):
+    for anticipate in (False, True):
+      fitted = fit(pattern, measure, prior, part, anticipate=anticipate, limit=limit)
+      if fitted is None:
+        continue
+      events, exceptions = fitted
+      if anticipate and not any(e.harmonic_time is not None for e in events):
+        continue
+      contributions = evaluate(events, part, melody, spans(measure, prior), lengths)
+      result.append(Rhythm(events, (('pattern', pattern['family']),), contributions, exceptions, pattern['id'] + ('.anticipated' if anticipate else '')))
+  if not result:
+    raise ValueError(f'measure {measure["number"]}: no {part} pattern for lengths {lengths} and written harmonic/release constraints')
+  return representatives(result, policy['pattern_candidates'], seed)
+
+def representatives(candidates, limit, seed):
+  unique = {}
+  for c in candidates:
+    rank = (c.cost, generator.solo.policy.stable_rank(seed, (c.pattern_id, c.signature)))
+    if c.signature not in unique or rank < unique[c.signature][0]:
+      unique[c.signature] = rank, c
+  ordered = [c for _, c in sorted(unique.values(), key=lambda item: item[0])]
+  retained, families = [], set()
+  for c in ordered:
+    if c.families not in families:
+      retained.append(c)
+      families.add(c.families)
+  retained.extend(c for c in ordered if c not in retained)
+  return tuple(retained[:limit])
+
+def project(pattern, offset, lengths):
+  """Take a form-position slice, rather than restarting a pattern at an entry."""
+  end = offset + sum(lengths)
+  rows = []
+  for row in pattern['events']:
+    a = fractions.Fraction(row['onset'])
+    b = a + fractions.Fraction(row['duration'])
+    if a < end and b > offset:
+      rows.append({**row, 'onset': str(max(a, offset) - offset), 'duration': str(min(b, end) - max(a, offset))})
+  return {**pattern, 'lengths': list(map(str, lengths)), 'events': rows}
+
+def realize(pattern, inputs, part, melody, *, anticipate=False, limit=None, boundary=False, assign=True):
+  flat = flatten(inputs)
+  lengths = tuple(generator.model.fraction(m['duration']) for m, _, _ in inputs)
+  fitted = fit(pattern, flat, None, part, anticipate=anticipate, limit=limit, assign=assign)
+  if fitted is None and boundary:
+    # Keep the selected onset/layers at a protected handoff, using the longest
+    # permitted short articulation that fits rather than another groove.
+    clipped = project(pattern, fractions.Fraction(), (min(sum(lengths), limit) if limit is not None else sum(lengths),))
+    rows = []
+    for row in clipped['events']:
+      a, duration = fractions.Fraction(row['onset']), fractions.Fraction(row['duration'])
+      _, b, _ = next(s for s in spans(flat, None) if s[0] <= a < s[1])
+      duration = min(duration, b - a)
+      if part == 'acoustic' and duration <= 1:
+        duration = next((d for d in (fractions.Fraction(1), fractions.Fraction(1, 2)) if d <= duration), fractions.Fraction())
+      if duration:
+        rows.append({**row, 'duration': str(duration)})
+    fitted = fit({**pattern, 'events': rows}, flat, None, part, anticipate=False, limit=limit, assign=assign)
+    if fitted is not None:
+      fitted = fitted[0], (*fitted[1], ('release', 'shortened articulation at source-defined boundary'))
+  if fitted is None:
+    return None
+  events, exceptions = fitted
+  context, cursor = [], fractions.Fraction()
+  for m, _, i in inputs:
+    context.extend((a + cursor, b + cursor) for a, b in melody[i])
+    cursor += generator.model.fraction(m['duration'])
+  return Rhythm(events, (('pattern', pattern['family']),), evaluate(events, part, context, spans(flat, None), lengths), exceptions,
+    pattern['id'] + ('.anticipated' if any(e.harmonic_time is not None for e in events) else ''))
+
+def identity(pattern):
+  """Layer swaps and anticipation permissions alone are not different rhythms."""
+  return tuple((fractions.Fraction(r['onset']), fractions.Fraction(r['duration'])) for r in pattern['events'])
+
+def acoustic_context(groups, melody, breaks=()):
+  """Immutable source timing for assigning and evaluating a complete stream.
+
+  Breaks are (source measure index, offset) pairs. Ordinary storage boundaries
+  are absent: adjacent figures must see each other's attacks before scoring.
+  """
+  blocks, starts, cursor = [], {}, fractions.Fraction()
+  for group in groups:
+    start, context, lengths = cursor, [], []
+    for m, _, i in group:
+      starts[i] = cursor
+      context.extend((a + cursor - start, b + cursor - start) for a, b in melody[i])
+      lengths.append(generator.model.fraction(m['duration']))
+      cursor += lengths[-1]
+      starts[i + 1] = cursor
+    blocks.append((start, cursor, tuple(context), spans(flatten(group), None), tuple(lengths)))
+  full = tuple(row for group in groups for row in group)
+  return {'blocks': tuple(blocks), 'harmony': spans(flatten(full), None),
+    'breaks': tuple(sorted(starts[i] + t for i, t in breaks if i in starts))}
+
+def realize_acoustic_form(rhythms, context):
+  """The shared release/lane contract for selection, fills and final output.
+
+  Inputs retain authored releases. Re-evaluate those releases whenever an
+  edited neighboring figure changes the next attack; never restore duration
+  from previously shortened notation.
+  """
+  if len(rhythms) != len(context['blocks']):
+    raise ValueError('acoustic realization requires one rhythm per source block')
+  events = []
+  for rhythm, (start, _, _, _, _) in zip(rhythms, context['blocks']):
+    events.extend(dataclasses.replace(e, start=e.start + start, end=e.end + start, harmonic_time=e.harmonic_time + start if e.harmonic_time is not None else None) for e in rhythm.events)
+  assigned = assign_acoustic(events, context['harmony'], context['breaks'])
+  result = []
+  for rhythm, (start, end, melody, harmony, lengths) in zip(rhythms, context['blocks']):
+    local = tuple(dataclasses.replace(e, start=e.start - start, end=e.end - start, harmonic_time=e.harmonic_time - start if e.harmonic_time is not None else None)
+      for e in assigned if start <= e.start < end)
+    exceptions = tuple(e for e in rhythm.exceptions if e[0] != 'layer_assignment')
+    result.append(dataclasses.replace(rhythm, events=local, contributions=evaluate(local, 'acoustic', melody, harmony, lengths),
+      pattern_id=rhythm.pattern_id.removesuffix('.anticipated') + ('.anticipated' if any(e.harmonic_time is not None for e in local) else ''),
+      exceptions=(*exceptions, ('layer_assignment', 'combined attacks or bass/upper eighth pairs; release at every next attack'))))
+  return tuple(result)
+
+def form_cost(rhythms, groups):
+  return sum(r.cost * len(g) for r, g in zip(rhythms, groups))
+
+def choose_cycles(groups, part, melody, seed, *, breaks=(), composition=None):
+  """Select a small recurring vocabulary against every occurrence in the form.
+
+  No local shortlist precedes whole-form evaluation. A pattern/mode is retained
+  as one decision even where a chord boundary changes its legal release.
+  """
+  composition = composition or generator.vocabulary.current()
+  policy = composition.accompaniment(part)
+  settings = policy['groove']
+  selected, locations, reports = [], [], []
+  runs = []
+  for group in groups:
+    m = group[0][0]
+    key = (meter_context(m), generator.model.fraction(m['duration']))
+    if not runs or runs[-1][0] != key:
+      runs.append((key, []))
+    runs[-1][1].append(group)
+  for ((meter, _), length), run in runs:
+    grouping, grouping_evidence = grouping_for(run, melody)
+    patterns = patterns_for(policy, run[0][0][0], (length, length), grouping)
+    if not patterns:
+      patterns = patterns_for(policy, run[0][0][0], (length,), grouping)
+    requested = settings['cycle_patterns']
+    identities = {p['id']: identity(p) for p in patterns}
+    available = len(set(identities.values()))
+    cache, estimates = {}, {}
+    context = acoustic_context(run, melody, breaks) if part == 'acoustic' else None
+    for i, group in enumerate(run):
+      lengths = tuple(generator.model.fraction(m['duration']) for m, _, _ in group)
+      local_context = acoustic_context((group,), melody, breaks) if context else None
+      for p in patterns:
+        for anticipate in (False, True):
+          value = realize(project(p, fractions.Fraction(), lengths), group, part, melody, anticipate=anticipate, assign=part != 'acoustic')
+          cache[i, p['id'], anticipate] = value
+          try:
+            estimates[i, p['id'], anticipate] = realize_acoustic_form((value,), local_context)[0] if context and value is not None else value
+          except ValueError:
+            estimates[i, p['id'], anticipate] = None
+    winner, examined = None, 0
+    for width in range(min(requested, len(run), available), 0, -1):
+      layers = []
+      for slot in range(width):
+        unique = {}
+        for p in patterns:
+          for anticipate in (False, True):
+            values = [estimates[i, p['id'], anticipate] for i in range(slot, len(run), width)]
+            if any(v is None for v in values):
+              continue
+            cost = sum(v.cost * len(run[i]) for i, v in zip(range(slot, len(run), width), values))
+            rank = (cost, generator.solo.policy.stable_rank(seed, (meter, slot, p['id'], anticipate)))
+            option = (rank, p, anticipate)
+            key = identities[p['id']]
+            if key not in unique or rank < unique[key][0]:
+              unique[key] = option
+        ordered = sorted(unique.values(), key=lambda o: o[0])
+        diverse, families = [], set()
+        for option in ordered:
+          if option[1]['family'] not in families:
+            diverse.append(option)
+            families.add(option[1]['family'])
+        diverse.extend(o for o in ordered if o not in diverse)
+        layers.append(diverse[:settings['slot_candidates']])
+      for cycle in itertools.product(*layers):
+        if len({identities[o[1]['id']] for o in cycle}) != width:
+          continue
+        examined += 1
+        cost = sum(o[0][0] for o in cycle)
+        if context:
+          values = [cache[i, cycle[i % width][1]['id'], cycle[i % width][2]] for i in range(len(run))]
+          try:
+            cost = form_cost(realize_acoustic_form(values, context), run)
+          except ValueError:
+            continue
+        rank = (cost, tuple(o[0][1] for o in cycle))
+        if winner is None or rank < winner[0]:
+          winner = rank, cycle
+      if winner is not None:
+        break
+    if winner is None:
+      raise ValueError(f'source measure {run[0][0][0]["number"]}: no legal recurring {part} groove for {meter}; check harmonic releases and pattern vocabulary')
+    cycle = winner[1]
+    report = {'source_start': run[0][0][2], 'source_end': run[-1][-1][2], 'meter': meter,
+      'grouping': list(grouping), 'grouping_evidence': grouping_evidence, 'eligible_patterns': len(patterns),
+      'requested_patterns': requested, 'cycle_patterns': len(cycle), 'patterns': [o[1]['id'] for o in cycle],
+      'anticipation': [o[2] for o in cycle], 'cost': winner[0][0], 'complete_cycles_compared': examined,
+      'retrieval_cost': sum(o[0][0] for o in cycle),
+      'reduction': 'available distinct, legal patterns or source length' if len(cycle) < requested else None}
+    reports.append(report)
+    for i, group in enumerate(run):
+      _, p, anticipate = cycle[i % len(cycle)]
+      selected.append((p, anticipate, cache[i, p['id'], anticipate]))
+      locations.append({'cycle_start': run[0][0][2], 'cycle_slot': i % len(cycle), 'base_pattern': p['id'], 'fill': None})
+  context = acoustic_context(groups, melody, breaks) if part == 'acoustic' else None
+  add_fills(groups, selected, locations, part, melody, seed, context=context, composition=composition)
+  if context:
+    realized = realize_acoustic_form([r for _, _, r in selected], context)
+    selected = [(p, a, r) for (p, a, _), r in zip(selected, realized)]
+    for report in reports:
+      report['final_cost'] = sum(r.cost * len(g) for g, r in zip(groups, realized) if report['source_start'] <= g[0][2] <= report['source_end'])
+      report['attack_assignment'] = 'sequential_combined_except_eighth_pairs'
+  return selected, locations, reports
+
+def add_fills(groups, selected, locations, part, melody, seed, *, context=None, composition=None):
+  """Fixed source-position answers; no iteration seed or generated lead input."""
+  composition = composition or generator.vocabulary.current()
+  policy = composition.accompaniment(part)
+  spacing = policy['groove']['fill_spacing_measures']
+  if not groups:
+    return
+  body_start, body_end = groups[0][0][2], groups[-1][-1][2] + 1
+  body_length = body_end - body_start
+  options = []
+  if body_length < spacing:
+    return
+  original = realize_acoustic_form([r for _, _, r in selected], context) if context else tuple(r for _, _, r in selected)
+
+  def proposal(i, candidate, baseline):
+    if not context:
+      return candidate.cost < baseline[i].cost - 1e-9, baseline[i].cost - candidate.cost
+    raw = [r for _, _, r in selected]
+    raw[i] = candidate
+    try:
+      revised = realize_acoustic_form(raw, context)
+    except ValueError:
+      return False, 0
+    middle = generator.model.fraction(groups[i][0][0]['duration'])
+    # A second-bar fill must not alter the preceding/following block's lane
+    # assignment or release, nor any part of its own first bar.
+    if any(a.events != b.events for j, (a, b) in enumerate(zip(baseline, revised)) if j != i):
+      return False, 0
+    if tuple(e for e in baseline[i].events if e.start < middle) != tuple(e for e in revised[i].events if e.start < middle):
+      return False, 0
+    gain = form_cost(baseline, groups) - form_cost(revised, groups)
+    return gain > 1e-9, gain
+
+  for i, (group, (base, anticipate, _)) in enumerate(zip(groups, selected)):
+    current = original[i]
+    if len(group) != 2:
+      continue
+    last, _, index = group[-1]
+    length = generator.model.fraction(last['duration'])
+    reason = 'form_end' if i == len(groups) - 1 else 'section_end' if groups[i + 1][0][0].get('section_label') else None
+    if reason is None and max((min(b, length) for a, b in melody[index]), default=fractions.Fraction()) <= length - 1:
+      reason = 'melody_breath'
+    if reason is None:
+      continue
+    middle = generator.model.fraction(group[0][0]['duration'])
+    if any(e.start < middle < e.end for e in current.events):
+      continue
+    for variation in policy['patterns']:
+      if variation['family'] != base['family'] or tuple(variation['lengths']) != tuple(base['lengths']) or variation.get('meter') != base.get('meter') or tuple(variation.get('grouping', ())) != tuple(base.get('grouping', ())):
+        continue
+      hybrid = {**base, 'id': base['id'] + '.fill.' + variation['id'], 'events':
+        [r for r in base['events'] if fractions.Fraction(r['onset']) < middle] +
+        [r for r in variation['events'] if fractions.Fraction(r['onset']) >= middle]}
+      candidate = realize(hybrid, group, part, melody, anticipate=anticipate, assign=part != 'acoustic')
+      if candidate is None:
+        continue
+      if not context and tuple(e for e in candidate.events if e.start < middle) != tuple(e for e in current.events if e.start < middle):
+        continue
+      valid, gain = proposal(i, candidate, original)
+      if not valid:
+        continue
+      options.append((-gain, generator.solo.policy.stable_rank(seed, (index, hybrid['id'])), i, index, hybrid, anticipate, candidate, reason))
+  chosen = []
+  for _, _, i, index, pattern, anticipate, candidate, reason in sorted(options):
+    if any(min(abs(index - other), body_length - abs(index - other)) < spacing for other in chosen):
+      continue
+    if context and not proposal(i, candidate, realize_acoustic_form([r for _, _, r in selected], context))[0]:
+      continue
+    selected[i] = pattern, anticipate, candidate
+    locations[i]['fill'] = {'pattern': pattern['id'], 'source_index': index, 'reason': reason}
+    chosen.append(index)

generator/compose.py

@@ -1,31 +1,30 @@
 """Explicit single-role build. Does not publish or consume an existing ensemble."""
 import argparse
 import json
+import pathlib
 import time
-from pathlib import Path
+
-from generator import musicxml, parts, production
+import generator.composition
+import generator.composition_types
+import generator.musicxml
+import generator.parts
+import generator.source
 
 def main():
-  parser=argparse.ArgumentParser(description=__doc__)
+  parser = argparse.ArgumentParser(description=__doc__)
-  parser.add_argument('--song',type=Path,required=True)
+  parser.add_argument('--song', type=pathlib.Path, required=True)
-  parser.add_argument('--role',choices=production.ROLES,required=True)
+  parser.add_argument('--role', choices=generator.composition_types.ROLES, required=True)
-  parser.add_argument('--data-dir',type=Path,default=Path('data'))
+  parser.add_argument('--data-dir', type=pathlib.Path, default=pathlib.Path('data'))
-  parser.add_argument('--output',type=Path,required=True)
+  parser.add_argument('--output', type=pathlib.Path, required=True)
-  parser.add_argument('--cache-dir',type=Path)
+  parser.add_argument('--cache-dir', type=pathlib.Path)
-  parser.add_argument('--solo-xml',type=Path,help='Also export a completed solo role as MusicXML.')
+  parser.add_argument('--settings', type=pathlib.Path)
-  parser.add_argument('--settings',type=Path)
+  args = parser.parse_args()
-  args=parser.parse_args()
+  source = generator.source.SourceAnalysis.read(generator.parts.prepare_document(generator.musicxml.load_musicxml(args.song)))
-  if args.solo_xml and not args.role.endswith('_solo'):
+  start = time.monotonic()
-    parser.error('--solo-xml requires a solo role')
+  result = generator.composition.generate_role(source, args.role, args.data_dir, json.loads(args.settings.read_text()) if args.settings else {}, args.cache_dir)
-  source=production.SourceAnalysis.read(parts.prepare_document(musicxml.load_musicxml(args.song)))
+  args.output.parent.mkdir(parents=True, exist_ok=True)
-  start=time.monotonic()
+  args.output.write_text(json.dumps(result, indent=2))
-  result=production.generate_role(source,args.role,args.data_dir,json.loads(args.settings.read_text()) if args.settings else {},args.cache_dir)
+  print(f'{args.role}: {time.monotonic() - start:.2f}s wall; cached={result["cache_hit"]}; role generation/cache lookup {result["seconds"]:.2f}s')
-  args.output.parent.mkdir(parents=True,exist_ok=True)
-  args.output.write_text(json.dumps(result,indent=2))
-  if args.solo_xml:
-    from generator.solo.export import write_role
-    manifest=write_role(result,args.solo_xml)
-    args.solo_xml.with_suffix('.manifest.solo').write_text(json.dumps(manifest,indent=2)+'\n')
-  print(f'{args.role}: {time.monotonic()-start:.2f}s wall; cached={result["cache_hit"]}; role generation/cache lookup {result["seconds"]:.2f}s')
 
-if __name__=='__main__':main()
+if __name__ == '__main__':
+  main()

generator/composition.py

@@ -0,0 +1,161 @@
+"""Independent, reproducible role generation from source and policy inputs.
+
+Only source documents and production resources enter generate_role. Completed
+roles and published assets are downstream values, never composition inputs.
+"""
+from __future__ import annotations
+import copy
+import dataclasses
+import functools
+import hashlib
+import json
+import pathlib
+import time
+
+import generator.accompaniment
+import generator.acoustic
+import generator.arrange.assembly
+import generator.arrange.form
+import generator.arrange.generate
+import generator.comping
+import generator.comping_rhythm
+import generator.composition_types
+import generator.fretboard
+import generator.input_audit
+import generator.role_cache
+import generator.score
+import generator.solo.arrangement
+import generator.solo.authored
+import generator.solo.composer
+import generator.solo.learned
+import generator.solo.notation
+import generator.solo.report
+import generator.source
+import generator.validation
+import generator.vocabulary
+
+VERSION = 'independent-roles-1'
+
+def dependencies(role, resources, measures=None):
+  names = ['harmony', 'lead_ins'] if role.endswith('_solo') else []
+  accompaniment = {}
+  if role.endswith('_accompaniment'):
+    policy = generator.vocabulary.accompaniment(role.split('_')[0], resources)
+    accompaniment = {'accompaniment': generator.comping_rhythm.relevant_policy(policy, measures) if measures is not None else policy}
+  return {'roles': {'role': resources['roles']['roles'][role], 'difficulty': resources['roles']['difficulty']},
+    **({'voicing_families': generator.vocabulary.accompaniment('electric', resources)['voicing_families']} if role == 'electric_melody' else {}),
+    **accompaniment,
+    **{name: resources[name] for name in names}}
+
+def _resource_key(directory, role):
+  return {'model': hashlib.sha256((directory / 'models' / 'solo.json').read_bytes()).hexdigest()} if role.endswith('_solo') else {}
+
+@functools.lru_cache(maxsize=4)
+def _solo_model(directory, fingerprint):
+  return generator.solo.learned.Model.load(pathlib.Path(directory) / 'models' / 'solo.json')
+
+def generate_role(source, role, data_dir, settings=None, cache_dir=None) -> generator.composition_types.RoleResult:
+  started = time.monotonic()
+  with generator.input_audit.composition(data_dir, role, cache_dir) as reads:
+    result = _generate_role(source, role, data_dir, settings, cache_dir)
+  # Observed reads are diagnostics, never cache identity or musical seeds.
+  result['trace'] = {**result['trace'], 'input_boundary': 'production data (including explicit authored repertoire), generator code, Python runtime, own role cache'}
+  result['observed_reads'] = sorted(reads)
+  result['seconds'] = time.monotonic() - started
+  return result
+
+def _generate_role(source, role, data_dir, settings=None, cache_dir=None):
+  if role not in generator.composition_types.ROLES:
+    raise ValueError('Unknown generation role: ' + role)
+  data_dir = pathlib.Path(data_dir)
+  resources = generator.vocabulary.load(data_dir / 'composition')
+  settings = settings or {}
+  registered = generator.solo.authored.lookup(source, role.removesuffix('_solo'), data_dir) if role.endswith('_solo') and not settings.get('predict', False) else None
+  algorithm = VERSION + '/' + generator.solo.learned.VERSION if role.endswith('_solo') else VERSION
+  inputs = {'version': algorithm, 'source': source.musical_key, 'role': role, 'settings': settings,
+    'vocabulary': dependencies(role, resources, generator.score.source_staff(source.document())['measures']), 'catalogs': ({'authored': registered['sha256'], 'id': registered['id'], 'pitch_convention': registered['pitch_convention']} if registered else _resource_key(data_dir, role)), 'code': generator.role_cache.code_key()}
+  # Source notation/lyrics also invalidate cached score representation, never the musical seed.
+  key = generator.vocabulary.fingerprint({'inputs': inputs, 'notation': source.document_json})
+  path = pathlib.Path(cache_dir) / role / (key + '.json') if cache_dir else None
+  cached = generator.role_cache.restore(path, key)
+  if cached is not None:
+    return cached
+  started = time.monotonic()
+  document = source.document()
+  part = role.split('_')[0]
+  plan = generator.arrange.form.FormPlan.read(document, part)
+  trace = {'source': source.musical_key, 'role': role, 'production_resources': inputs['catalogs'],
+    'vocabulary': generator.vocabulary.fingerprint(inputs['vocabulary']), 'settings': settings, 'algorithm': algorithm, 'code': inputs['code'], 'cache_key': key}
+  choice_source = source.accompaniment_key() if role.endswith('_accompaniment') else source.musical_key
+  choice_data, choice_settings = inputs['vocabulary'], settings
+  if role.endswith('_accompaniment'):
+    # Voicing weights invalidate the concrete cache, but must not reshuffle
+    # rhythm patterns. Removing this new field preserves the pre-policy seed.
+    choice_data = copy.deepcopy(choice_data)
+    choice_data['roles']['role'].pop('voicing', None)
+    for field in ('provenance', 'license', 'research'):
+      choice_data['accompaniment'].pop(field, None)
+    choice_settings = {k: v for k, v in settings.items() if k != 'voicing'}
+  seed = generator.vocabulary.fingerprint({'source': choice_source, 'role': role, 'settings': choice_settings, 'data': choice_data, 'version': algorithm})
+  details = {}
+  solo_report = ''
+  composition_timings = {}
+  composition = generator.vocabulary.CompositionContext.create(resources, role, settings | {'choice_seed': seed})
+  if role.endswith('_solo'):
+    result = generator.arrange.generate._arrange_document(document, part, plan, owned={'slash'}, composition=composition)
+    si = 3 if part == 'acoustic' else 4
+    first, count, offset, length = generator.solo.arrangement.regions(document)[si]
+    # The template/harmony comes directly from source form, not another generated part.
+    authored = generator.score.source_staff(document)['measures']
+    section = plan.sections[si]
+    effective = generator.arrange.generate.section_source(authored, section, plan.sections[si + 1])
+    template = copy.deepcopy(effective[section.source_measure_index:section.source_measure_index + count])
+    initial = generator.accompaniment.carried_chords(effective)[section.source_measure_index]
+    start, end = next((a - offset, b - offset) for a, b, r in plan.windows if r == role)
+    notation_first = first
+    if start < 0:
+      # The preceding arrangement harmony comes from the source form. Do
+      # not invent harmony from a comparison score's unlabelled lead-in.
+      template.insert(0, copy.deepcopy(result['staves'][0]['measures'][first - 1]))
+      notation_first -= 1
+      initial = generator.accompaniment.carried_chords(result['staves'][0]['measures'])[first - 1]
+    if registered:
+      solo = generator.solo.authored.realize(registered, generator.source.analyze(document), template, part, start, end, initial, composition=composition)
+    else:
+      solo_model = _solo_model(str(data_dir), inputs['catalogs']['model'])
+      solo = generator.solo.composer.compose(generator.source.analyze(document), template, part, end, initial=initial, learned_model=solo_model, expressions_enabled=composition.role().get('expressions', True), start=start, composition=composition)
+    composition_timings = {k: v for k, v in solo.diagnostics.items() if k.endswith('_seconds')}
+    relative = tuple(dataclasses.replace(n, start=n.start - start, end=n.end - start) for n in solo.notes())
+    output = generator.solo.notation.measures(result['staves'][0]['measures'][notation_first:first + count], relative, 'solo_' + part)
+    generator.solo.notation.replace_region(result, notation_first, output)
+    details = generator.solo.report.structure(solo, composition=composition)
+    details['allocation'] = {'start': str(start), 'end': str(end), 'lead_in_measures': first - notation_first}
+    solo_report = generator.solo.report.describe(solo, first, offset, detail=details, composition=composition)
+  elif role.endswith('_accompaniment'):
+    canonical = generator.comping.Canonical(document, part, seed, composition=composition, plan=plan)
+    result = generator.arrange.generate._arrange_document(document, part, plan, owned={'accompaniment'}, accompaniment=canonical, composition=composition)
+    details = {'canonical_comping': canonical.report, 'guitar_placement': canonical.placement_report, 'accompaniment_rhythm': canonical.rhythm_report, 'groove_cycles': canonical.groove_report, 'rhythm_source': choice_source}
+  else:
+    placement_report = {}
+    result = generator.arrange.generate._arrange_document(document, part, plan, owned={'vocal' if part == 'vocal' else 'melody'}, composition=composition, placement_report=placement_report)
+    if placement_report:
+      details['guitar_placement'] = placement_report
+  allowed = tuple((a, b) for a, b, r in plan.windows if r == role)
+  generator.arrange.assembly.mask(result, allowed)
+  if role == "acoustic_accompaniment":
+    generator.acoustic.validate_comping(result)
+  staff = result['staves'][0]
+  for measure in staff['measures']:
+    measure['id'] = role + '_' + measure['id']
+  staff['tuplets'] = generator.score.reidentify(staff['measures'], staff.get('tuplets', []))
+  if part != 'vocal':
+    result['staves'][0]['tablature'] = True
+  generator.validation.validate_document(result)
+  generator.fretboard.validate_document(result)
+  value = {'role': role, 'document': result, 'trace': trace, 'details': details, 'report': solo_report, 'seconds': time.monotonic() - started, 'composition_timings': composition_timings, 'cache_hit': False}
+  # Sidecar diagnostics use the same rational-safe representation as reports.
+  # Identical insertion order after cold generation and cache loading also
+  # keeps serialized scores and the catalog byte-for-byte reproducible.
+  value = json.loads(json.dumps(value, default=str, sort_keys=True))
+  generator.role_cache.store(path, key, value)
+  return value

generator/composition_types.py

@@ -0,0 +1,28 @@
+"""Worker requests and role results; score serialization stays unchanged."""
+from __future__ import annotations
+import dataclasses
+import pathlib
+import typing
+
+import generator.source
+
+ROLES = ('acoustic_accompaniment', 'electric_accompaniment', 'electric_melody', 'vocal_melody', 'acoustic_solo', 'electric_solo')
+
+@dataclasses.dataclass(frozen=True)
+class RoleRequest:
+  source: generator.source.SourceAnalysis
+  role: str
+  directory: pathlib.Path
+  settings_json: str = '{}'
+  cache: pathlib.Path | None = None
+
+class RoleResult(typing.TypedDict):
+  role: str
+  document: dict
+  trace: dict
+  details: dict
+  report: str
+  seconds: float
+  composition_timings: dict
+  cache_hit: bool
+  observed_reads: typing.NotRequired[list[str]]

generator/engraving/audit.py

@@ -19,16 +19,17 @@ def audit_primitives(primitives, width, height, fonts):
       if any(abs(a - b) > 0.001 for a, b in zip((actual.left, actual.top, actual.right, actual.bottom), (item.box.left, item.box.top, item.box.right, item.box.bottom))):
         raise generator.engraving.types.EngravingError(f'{item.sources}: glyph bounds disagree with its actual outline')
 
-def audit_layout(document: dict, layout: generator.engraving.types.LayoutDocument, fonts: generator.engraving.fonts.Fonts) -> None:
+def audit_layout(document: dict, layout: generator.engraving.types.LayoutDocument, fonts: generator.engraving.fonts.Fonts, *, systems_per_page: int | None = 2) -> None:
   index = generator.engraving.notation.ScoreIndex(document)
   fragments, primitives = {}, {}
   previous_end = fractions.Fraction()
   scale = layout.pages[0].systems[0].scale
   for page in layout.pages:
-    if len(page.systems) != 2 and (page is not layout.pages[-1] or len(page.systems) != 1):
+    if not page.systems or (systems_per_page is not None and len(page.systems) != systems_per_page and (page is not layout.pages[-1] or not 1 <= len(page.systems) < systems_per_page)):
       raise generator.engraving.types.EngravingError('pages must contain two music systems, except an odd final system')
-    if len(page.systems) == 2 and page.systems[0].box.bottom + layout.style.system_gap > page.systems[1].box.top + 0.01:
+    for first, second in zip(page.systems, page.systems[1:]):
-      raise generator.engraving.types.EngravingError('music systems overlap their page rows')
+      if first.box.bottom + layout.style.system_gap > second.box.top + 0.01:
+        raise generator.engraving.types.EngravingError('music systems overlap their page rows')
     audit_primitives(page.primitives, page.width, page.height, fonts)
     header = generator.engraving.types.bounds([p for p in page.primitives if p.role in ('title', 'subtitle')])
     footer = generator.engraving.types.bounds([p for p in page.primitives if p.role == 'page_number'])
@@ -79,6 +80,9 @@ def audit_layout(document: dict, layout: generator.engraving.types.LayoutDocumen
       # Staff, ledger, stem, beam and tie contacts are intentional. Distinct
       # noteheads, accidentals and text in the same lane must remain separate.
       checked = [p for p in system.primitives if p.role in ('head', 'tab', 'accidental', 'harmony', 'lyric', 'scale', 'section')]
+      for label in (p for p in system.primitives if p.role == 'measure_number'):
+        if any(label.box.overlaps(p.box, 3 * system.scale) for p in system.primitives if p is not label):
+          raise generator.engraving.types.EngravingError(f'{label.sources}: measure number overlaps notation')
       for i, item in enumerate(checked):
         for other in checked[i + 1:]:
           if set(item.sources) & set(other.sources) and (item.role != 'tab' or other.role != 'tab'):
@@ -141,3 +145,29 @@ def audit_layout(document: dict, layout: generator.engraving.types.LayoutDocumen
   for span in index.staff.get('octave_shifts', []):
     if not any(p.role == 'octave-shift' for p in primitives.get(span['id'], [])):
       raise generator.engraving.types.EngravingError(f'{span["id"]}: octave shift was omitted')
+
+def audit_playback(pages, playback, style, fonts):
+  canonical = [system for page in pages for system in page.systems]
+  if len(playback) != len(canonical):
+    raise generator.engraving.types.EngravingError('playback must have one view per system')
+  for i, page in enumerate(playback):
+    audit_primitives(page.primitives, page.width, page.height, fonts)
+    expected = canonical[i:i + 2]
+    if len(page.systems) != len(expected):
+      raise generator.engraving.types.EngravingError('invalid playback system pairing')
+    header = generator.engraving.types.bounds([p for p in page.primitives if p.role in ('title', 'subtitle')])
+    footer = generator.engraving.types.bounds([p for p in page.primitives if p.role == 'page_number'])
+    if page.systems[0].box.top < header.bottom + style.page_clearance - 0.01 or page.systems[-1].box.bottom > footer.top - style.page_clearance + 0.01:
+      raise generator.engraving.types.EngravingError('playback overlaps header or footer')
+    for a, b in zip(page.systems, page.systems[1:]):
+      if a.box.bottom + style.system_gap > b.box.top + 0.01:
+        raise generator.engraving.types.EngravingError('playback systems overlap')
+    for source, actual in zip(expected, page.systems):
+      if (source.start, source.end, source.scale, source.fragments) != (actual.start, actual.end, actual.scale, actual.fragments):
+        raise generator.engraving.types.EngravingError('playback changes system content')
+      dy = actual.box.top - source.box.top
+      if len(source.primitives) != len(actual.primitives):
+        raise generator.engraving.types.EngravingError('playback changes primitives')
+      for a, b in zip(source.primitives, actual.primitives):
+        if (a.kind, a.role, a.sources, a.value, a.size) != (b.kind, b.role, b.sources, b.value, b.size) or len(a.points) != len(b.points) or any(abs(x - u) > .001 or abs(y + dy - v) > .001 for (x, y), (u, v) in zip(a.points, b.points)):
+          raise generator.engraving.types.EngravingError('playback changes notation geometry')

generator/engraving/cache.py

@@ -9,12 +9,12 @@ import pathlib
 import platform
 import re
 import shutil
-import tempfile
 
 import PIL.Image
 import PIL.ImageChops
 import PIL.features
 
+import generator.cache
 import generator.engraving.validation
 import generator.theme
 
@@ -22,9 +22,10 @@ def renderer_identity(fonts, style) -> dict:
   root = pathlib.Path(__file__).resolve().parents[1]
   versions = {name: importlib.metadata.version(name) for name in ('Pillow', 'fonttools', 'aggdraw')}
   versions.update({name: PIL.features.version(name) for name in PIL.features.get_supported()})
-  sources = [*root.joinpath('engraving').glob('*.py'), root / 'model.py', root / 'harmony.py', root / 'theme.py', root / 'palette.json']
+  sources = generator.cache.source_identity(('generator.engraving.pipeline', 'generator.printing'))
+  sources['palette.json'] = hashlib.sha256((root / 'palette.json').read_bytes()).hexdigest()
   return {'fonts_style': fonts.fingerprint(style), 'python': {'implementation': platform.python_implementation(), 'version': platform.python_version()}, 'dependencies': versions,
-    'sources': {path.relative_to(root).as_posix(): hashlib.sha256(path.read_bytes()).hexdigest() for path in sorted(sources)}}
+    'sources': sources}
 
 def renderer_fingerprint(fonts, style) -> str:
   return hashlib.sha256(json.dumps(renderer_identity(fonts, style), sort_keys=True).encode()).hexdigest()
@@ -36,7 +37,7 @@ def renderer_namespace(cache: pathlib.Path, renderer: str) -> pathlib.Path:
   return namespace
 
 def score_fingerprint(document: dict, renderer: str) -> str:
-  music = {key: value for key, value in document.items() if key != 'playback'}
+  music = document
   return hashlib.sha256(json.dumps([renderer, music], sort_keys=True, ensure_ascii=False).encode()).hexdigest()
 
 def notation_fingerprint(document: dict, renderer: str) -> str:
@@ -46,12 +47,9 @@ def band_fingerprint(notation: str, page, band) -> str:
   context = [notation, page.width, page.height, page.supersampling, dataclasses.asdict(band)]
   return hashlib.sha256(json.dumps(context, sort_keys=True, ensure_ascii=False).encode()).hexdigest()
 
-def has_symlink(path: pathlib.Path) -> bool:
-  return any(parent.is_symlink() for parent in (path, *path.parents))
-
 def restore_pages(entry: pathlib.Path, destination: pathlib.Path, validated_pages: set[str] | None = None) -> list[dict] | None:
   try:
-    if has_symlink(entry) or has_symlink(destination):
+    if generator.cache.has_symlink(entry) or generator.cache.has_symlink(destination):
       return None
     manifest = entry / 'pages.json'
     if manifest.is_symlink():
@@ -61,7 +59,7 @@ def restore_pages(entry: pathlib.Path, destination: pathlib.Path, validated_page
       return None
     destination.mkdir(parents=True, exist_ok=True)
     for index, page in enumerate(pages, 1):
-      if not isinstance(page, dict) or set(page) != {'path', 'sha256'} or page['path'] != f'{index:04}.png' or not isinstance(page['sha256'], str) or not re.fullmatch('[0-9a-f]{64}', page['sha256']):
+      if not isinstance(page, dict) or set(page) != {'path', 'sha256', 'first_measure', 'measure_count', 'start_beat', 'end_beat', 'systems', 'playback'} or type(page['playback']) is not bool or not isinstance(page['path'], str) or not re.fullmatch(r'(?:playback-)?[0-9]{4,5}\.png', page['path']) or not isinstance(page['sha256'], str) or not re.fullmatch('[0-9a-f]{64}', page['sha256']):
         return None
       path = entry / page['path']
       if path.is_symlink() or not path.is_file():
@@ -76,28 +74,17 @@ def restore_pages(entry: pathlib.Path, destination: pathlib.Path, validated_page
     return None
 
 def store_pages(entry: pathlib.Path, destination: pathlib.Path, pages: list[dict]) -> None:
-  if has_symlink(entry.parent):
+  with generator.cache.stage_entry(entry, '.engraving_') as staged:
-    raise ValueError('engraving cache directories must not be symlinks')
-  entry.parent.mkdir(parents=True, exist_ok=True)
-  if entry.is_symlink() or entry.is_file():
-    entry.unlink(missing_ok=True)
-  entry.mkdir(exist_ok=True)
-  with tempfile.TemporaryDirectory(prefix='.engraving_', dir=entry.parent) as temporary:
-    staged = pathlib.Path(temporary)
     for page in pages:
       shutil.copyfile(destination / page['path'], staged / page['path'])
     (staged / 'pages.json').write_text(json.dumps(pages), encoding='utf-8')
     # Identical keys have identical pages; publish complete files, then the manifest.
-    for name in [page['path'] for page in pages] + ['pages.json']:
+    generator.cache.publish_files(entry, staged, list(dict.fromkeys(page['path'] for page in pages)) + ['pages.json'])
-      target = entry / name
-      if target.is_dir() and not target.is_symlink():
-        shutil.rmtree(target, ignore_errors=True)
-      (staged / name).replace(target)
 
 def restore_band(entry: pathlib.Path, key: str, width: int, height: int) -> PIL.Image.Image | None:
   try:
     manifest, asset = entry / 'band.json', entry / 'band.png'
-    if has_symlink(entry) or manifest.is_symlink() or asset.is_symlink():
+    if generator.cache.has_symlink(entry) or manifest.is_symlink() or asset.is_symlink():
       return None
     metadata = json.loads(manifest.read_text(encoding='utf-8'))
     if not isinstance(metadata, dict) or set(metadata) != {'context', 'width', 'height', 'sha256'}:
@@ -122,19 +109,8 @@ def restore_band(entry: pathlib.Path, key: str, width: int, height: int) -> PIL.
     return None
 
 def store_band(entry: pathlib.Path, key: str, image: PIL.Image.Image) -> None:
-  if has_symlink(entry.parent):
+  with generator.cache.stage_entry(entry, '.engraving_') as staged:
-    raise ValueError('engraving cache directories must not be symlinks')
-  entry.parent.mkdir(parents=True, exist_ok=True)
-  if entry.is_symlink() or entry.is_file():
-    entry.unlink()
-  entry.mkdir(exist_ok=True)
-  with tempfile.TemporaryDirectory(prefix='.engraving_', dir=entry.parent) as temporary:
-    staged = pathlib.Path(temporary)
     image.save(staged / 'band.png', format='PNG')
     metadata = {'context': key, 'width': image.width, 'height': image.height, 'sha256': hashlib.sha256((staged / 'band.png').read_bytes()).hexdigest()}
     (staged / 'band.json').write_text(json.dumps(metadata), encoding='utf-8')
-    for name in ('band.png', 'band.json'):
+    generator.cache.publish_files(entry, staged, ('band.png', 'band.json'))
-      target = entry / name
-      if target.is_dir() and not target.is_symlink():
-        shutil.rmtree(target)
-      (staged / name).replace(target)

generator/engraving/layout.py

@@ -16,7 +16,7 @@ import generator.engraving.spans
 import generator.engraving.tablature
 import generator.engraving.types
 import generator.harmony
-import generator.model
+import generator.validation
 
 def lyric_text(verse: dict) -> str:
   return ''.join(s['text'] for s in verse['syllables'] if s['kind'] in ('text', 'elision'))
@@ -57,7 +57,6 @@ def column(index: generator.engraving.notation.ScoreIndex, time: fractions.Fract
       items.append(fonts.text(generator.harmony.symbol(item), 0, 0, 18 if index.staff.get('tablature') else 21, 'harmony', (item['id'],)))
   items.extend(generator.engraving.scales.column(index, time, start, fonts))
   if time == measure_start:
-    items.append(fonts.text(str(measure['number']), 0, -style.space * (index.state(time)['staff_details']['lines'] + 0.5), 12, 'measure_number', (measure['id'],)))
     if label := measure.get('section_label'):
       items.append(fonts.text(label, 0, 0, 16 if index.staff.get('tablature') else 19, 'section', (measure['id'] + ':section',)))
   return items, anchors, fragments
@@ -103,6 +102,10 @@ def make_system(index: generator.engraving.notation.ScoreIndex, start: fractions
       for owner, ci, change in index.attributes_by_onset.get(time, []):
         extra = generator.engraving.attributes.staff_attributes(index, index.state(time), time, fonts, style.space, owner['id'] + f':attributes:{ci}', tuple(change))
         leading.extend(p.moved(max((p.box.right for p in leading), default=0) + 10, 0) for p in extra)
+    if time == measure_start:
+      # Anchor labels to the barline, not to the first note's time column.
+      bar_x = cursor - generator.engraving.types.bounds(leading).left if leading else 0
+      items.append(fonts.text(str(m['number']), bar_x, 0, 12, 'measure_number', (m['id'],), True))
     if leading:
       width = generator.engraving.types.bounds(leading).right - generator.engraving.types.bounds(leading).left + 14
       items.extend(p.moved(cursor - generator.engraving.types.bounds(leading).left, 0) for p in leading)
@@ -162,7 +165,7 @@ def fit_part_systems(systems, index, available_height, fonts, style):
 
   def fitting_scale(items):
     scale = 1
-    for first in range(0, len(items), 2):
+    for first in range(len(items)):
       pair = items[first:first + 2]
       height = sum(s.box.bottom - s.box.top for s in pair)
       scale = min(scale, (available_height - style.system_gap * (len(pair) - 1)) / height)
@@ -191,6 +194,8 @@ def part_header(document, fonts, style):
     raise generator.engraving.types.EngravingError('score title cannot fit the page without clipping')
   result = [item]
   subtitle = ' - '.join(str(value) for value in (document['metadata'].get('composer'), document['staves'][0].get('name')) if value)
+  if document['metadata'].get('solos_enabled') is False:
+    subtitle += ' - Improvisation'
   if not subtitle:
     return result
   credit = fonts.text(subtitle, 0, style.margin_y + size, 12, 'subtitle', ())
@@ -214,34 +219,40 @@ def part_header(document, fonts, style):
   result.append(fonts.text(line, style.margin_x, baseline, 12, 'subtitle', ()))
   return result
 
-def paginate(systems: list[generator.engraving.types.System], document: dict, fonts: generator.engraving.fonts.Fonts, style: generator.engraving.types.Style, *, index=None) -> tuple[generator.engraving.types.Page, ...]:
+def page_views(systems: list[generator.engraving.types.System], document: dict, fonts: generator.engraving.fonts.Fonts, style: generator.engraving.types.Style, *, index=None) -> tuple[tuple[generator.engraving.types.Page, ...], tuple[generator.engraving.types.Page, ...]]:
-  pages = []
   index = index or generator.engraving.notation.ScoreIndex(document)
   header = part_header(document, fonts, style)
   top = generator.engraving.types.bounds(header).bottom + style.page_clearance
-  footers = [fonts.text(str(number), style.width / 2, style.height - style.margin_y, 12, 'page_number', (), True) for number in range(1, (len(systems) + 1) // 2 + 1)]
+  footers = [fonts.text(str(number), style.width / 2, style.height - style.margin_y, 12, 'page_number', (), True) for number in range(1, len(systems) + 1)]
   bottom = min(p.box.top for p in footers) - style.page_clearance
   if bottom - top <= style.system_gap:
     raise generator.engraving.types.EngravingError('page header leaves no room for two music systems')
   fitted = fit_part_systems(systems, index, bottom - top, fonts, style)
-  for first in range(0, len(systems), 2):
+
-    number = len(pages) + 1
+  def place(stride):
-    pair = fitted[first:first + 2]
+    pages = []
-    slack = max(0, bottom - top - sum(s.box.bottom - s.box.top for s in pair) - style.system_gap * (len(pair) - 1))
+    for first in range(0, len(fitted), stride):
-    spacing = slack / 3 if len(pair) == 2 else 0
+      number = len(pages) + 1
-    cursor = top + spacing
+      pair = fitted[first:first + 2]
-    placed = []
+      slack = max(0, bottom - top - sum(s.box.bottom - s.box.top for s in pair) - style.system_gap * (len(pair) - 1))
-    for system in pair:
+      spacing = slack / 3 if len(pair) == 2 else 0
-      dx, dy = style.margin_x + 1, cursor - system.box.top
+      cursor = top + spacing
-      placed.append(dataclasses.replace(system, primitives=tuple(p.moved(dx, dy) for p in system.primitives), anchors=tuple(a.moved(dx, dy) for a in system.anchors), box=system.box.moved(dx, dy)))
+      placed = []
-      cursor = placed[-1].box.bottom + style.system_gap + spacing
+      for system in pair:
-    footer = footers[first // 2]
+        dx, dy = style.margin_x + 1, cursor - system.box.top
-    pages.append(generator.engraving.types.Page(number, style.width, style.height, style.supersampling, tuple(placed), tuple(header + [p for s in placed for p in s.primitives] + [footer])))
+        placed.append(dataclasses.replace(system, primitives=tuple(p.moved(dx, dy) for p in system.primitives), anchors=tuple(a.moved(dx, dy) for a in system.anchors), box=system.box.moved(dx, dy)))
-  return tuple(pages)
+        cursor = placed[-1].box.bottom + style.system_gap + spacing
+      pages.append(generator.engraving.types.Page(number, style.width, style.height, style.supersampling, tuple(placed), tuple(header + [p for s in placed for p in s.primitives] + [footers[number - 1]])))
+    return tuple(pages)
+
+  return place(2), place(1)
+
+def paginate(systems, document, fonts, style, *, index=None):
+  return page_views(systems, document, fonts, style, index=index)[0]
 
 def layout_systems(document: dict, style: generator.engraving.types.Style, fonts: generator.engraving.fonts.Fonts, *, index=None) -> tuple[generator.engraving.types.System, ...]:
   if index is None:
-    generator.model.validate_document(document)
+    generator.validation.validate_document(document)
     index = generator.engraving.notation.ScoreIndex(document)
   if not index.measures:
     raise generator.engraving.types.EngravingError('score has no measures')
@@ -267,7 +278,7 @@ def layout_systems(document: dict, style: generator.engraving.types.Style, fonts
   return tuple(systems)
 
 def layout_score(document: dict, style: generator.engraving.types.Style, fonts: generator.engraving.fonts.Fonts) -> generator.engraving.types.LayoutDocument:
-  generator.model.validate_document(document)
+  generator.validation.validate_document(document)
   style = generator.engraving.types.score_style(document, style)
   index = generator.engraving.notation.ScoreIndex(document)
   systems = layout_systems(document, style, fonts, index=index)

generator/engraving/pipeline.py

@@ -1,181 +1,17 @@
 from __future__ import annotations
 
-import collections
 import concurrent.futures
-import dataclasses
-import hashlib
 import json
 import multiprocessing
 import pathlib
-import shutil
-import tempfile
 import time
 
-import PIL.Image
-
-import generator.engraving.audit
 import generator.engraving.cache
 import generator.engraving.fonts
-import generator.engraving.layout
-import generator.engraving.notation
-import generator.engraving.raster
 import generator.engraving.types
 import generator.engraving.validation
-import generator.model
+import generator.engraving.worker
-
+import generator.validation
-@dataclasses.dataclass
-class GroupResult:
-  scores: list = dataclasses.field(default_factory=list)
-  fullscore: int = 0
-  bodyhits: int = 0
-  bodies: int = 0
-  renderedpages: int = 0
-  rejected: int = 0
-  timings: dict = dataclasses.field(default_factory=lambda: dict.fromkeys(('layout', 'pagination', 'raster', 'cache'), 0.0))
-
-class BodyBands:
-  def __init__(self, root, persistent, result, limit=16 * 1024 * 1024):
-    self.root, self.persistent, self.result, self.limit = root, persistent, result, limit
-    self.images = collections.OrderedDict()
-    self.size = 0
-    self.known = {}
-    self.root.mkdir(parents=True, exist_ok=True)
-  def get(self, notation, page, band, fonts):
-    key = generator.engraving.cache.band_fingerprint(notation, page, band)
-    if key in self.images:
-      self.images.move_to_end(key)
-      self.result.bodyhits += 1
-      return self.images[key]
-    started = time.monotonic()
-    entry = self.root / (key + '.rgb')
-    image = None
-    if key in self.known:
-      if entry.is_symlink() or entry.stat().st_size != page.width * (band.bottom - band.top) * 3:
-        raise generator.engraving.types.EngravingError('temporary notation band was corrupted')
-      data = entry.read_bytes()
-      if hashlib.sha256(data).hexdigest() != self.known[key]:
-        raise generator.engraving.types.EngravingError('temporary notation band was corrupted')
-      image = PIL.Image.frombytes('RGB', (page.width, band.bottom - band.top), data)
-    elif self.persistent:
-      cached = self.persistent / 'notation' / key
-      image = generator.engraving.cache.restore_band(cached, key, page.width, band.bottom - band.top)
-      self.result.rejected += int(image is None and (cached.exists() or cached.is_symlink()))
-    self.result.timings['cache'] += time.monotonic() - started
-    if image is None:
-      started = time.monotonic()
-      image = generator.engraving.raster.rasterize_band(page, band.primitives, band.top, band.bottom, fonts)
-      self.result.timings['raster'] += time.monotonic() - started
-      self.result.bodies += 1
-      if self.persistent:
-        started = time.monotonic()
-        generator.engraving.cache.store_band(self.persistent / 'notation' / key, key, image)
-        self.result.timings['cache'] += time.monotonic() - started
-    else:
-      self.result.bodyhits += 1
-    if key not in self.known:
-      started = time.monotonic()
-      # Spill finalized pixels, so eviction never repeats rasterization or PNG decoding.
-      data = image.tobytes()
-      entry.write_bytes(data)
-      self.result.timings['cache'] += time.monotonic() - started
-      self.known[key] = hashlib.sha256(data).hexdigest()
-    size = image.width * image.height * 3
-    while self.images and self.size + size > self.limit:
-      _, old = self.images.popitem(last=False)
-      self.size -= old.width * old.height * 3
-      old.close()
-    self.images[key] = image
-    self.size += size
-    return image
-  def close(self):
-    for image in self.images.values():
-      image.close()
-    self.images.clear()
-    self.size = 0
-
-def clear_pages(destination):
-  if destination.is_symlink() or destination.is_file():
-    destination.unlink()
-  elif destination.exists():
-    shutil.rmtree(destination)
-
-def write_bundle(path, document, pages, fingerprint):
-  entries = [{**page, 'path': path.with_suffix('.pages').name + '/' + page['path']} for page in pages]
-  music = {key: value for key, value in document.items() if key != 'playback'}
-  bundle = {'music': music, 'presentation': {'engine': generator.engraving.validation.ENGINE_NAME, 'style_sha256': fingerprint,
-    'width': generator.engraving.validation.PAGE_WIDTH, 'height': generator.engraving.validation.PAGE_HEIGHT, 'pages': entries}}
-  if 'playback' in document:
-    bundle['playback'] = document['playback']
-  path.write_text(json.dumps(bundle, indent=2, ensure_ascii=False) + '\n', encoding='utf-8')
-
-def engrave_group(notation, paths, root, font_path, cache_dir, fingerprint, progress=False):
-  family = generator.engraving.fonts.Fonts(font_path)
-  style = generator.engraving.types.Style()
-  systems = None
-  index = None
-  scores = {}
-  audited = set()
-  result = GroupResult()
-  with tempfile.TemporaryDirectory(prefix='vamp_notation_') as temporary:
-    bodies = BodyBands(pathlib.Path(temporary), cache_dir, result)
-    try:
-      for path, score_key in paths:
-        try:
-          document = json.loads(path.read_text(encoding='utf-8'))
-          style = generator.engraving.types.score_style(document, generator.engraving.types.Style())
-          destination = path.with_suffix('.pages')
-          clear_pages(destination)
-          destination.mkdir()
-          if score_key in scores:
-            source, pages = scores[score_key]
-            for page in pages:
-              shutil.copyfile(source / page['path'], destination / page['path'])
-            result.fullscore += 1
-          else:
-            if systems is None:
-              started = time.monotonic()
-              generator.model.validate_document(document)
-              index = generator.engraving.notation.ScoreIndex(document)
-              systems = generator.engraving.layout.layout_systems(document, style, family, index=index)
-              result.timings['layout'] += time.monotonic() - started
-            started = time.monotonic()
-            pages_layout = generator.engraving.layout.paginate(systems, document, family, style, index=index)
-            placement = tuple((page.width, page.height, tuple((system.start, system.end, system.box) for system in page.systems)) for page in pages_layout)
-            if placement not in audited:
-              layout = generator.engraving.types.LayoutDocument(pages_layout, style, fingerprint)
-              generator.engraving.audit.audit_layout(document, layout, family)
-              audited.add(placement)
-            else:
-              for page in pages_layout:
-                decoration = tuple(p for p in page.primitives if p.role in ('title', 'subtitle', 'page_number'))
-                generator.engraving.audit.audit_primitives(decoration, page.width, page.height, family)
-            result.timings['pagination'] += time.monotonic() - started
-            pages = []
-            for page in pages_layout:
-              bands = generator.engraving.raster.page_bands(page)
-              if bands is None:
-                started = time.monotonic()
-                image = generator.engraving.raster.rasterize_page(page, family)
-              else:
-                body = bodies.get(notation, page, bands[1], family)
-                started = time.monotonic()
-                image = generator.engraving.raster.compose_page(page, family, bands, body)
-              target = destination / f'{page.number:04}.png'
-              with image:
-                image.save(target, format='PNG')
-              result.timings['raster'] += time.monotonic() - started
-              result.renderedpages += 1
-              pages.append({'path': target.name, 'sha256': hashlib.sha256(target.read_bytes()).hexdigest()})
-            scores[score_key] = (destination, pages)
-          write_bundle(path, document, pages, fingerprint)
-          result.scores.append((path, score_key, pages))
-          if progress and len(result.scores) % 50 == 0:
-            print(f'Engraving body {notation[:12]}: {len(result.scores)}/{len(paths)} scores, fullscore {result.fullscore}, bodyhits {result.bodyhits}, bodies {result.bodies}, renderedpages {result.renderedpages}', flush=True)
-        except ValueError as error:
-          raise generator.engraving.types.EngravingError(f'{path.relative_to(root)}: {error}') from error
-    finally:
-      bodies.close()
-  return result
 
 def engrave_tree(root: pathlib.Path, fonts: pathlib.Path, *, cache_dir: pathlib.Path | None = None, jobs: int = 1, progress: bool = False) -> set[str]:
   if type(jobs) is not int or not 1 <= jobs <= 16:
@@ -197,26 +33,31 @@ def engrave_tree(root: pathlib.Path, fonts: pathlib.Path, *, cache_dir: pathlib.
       print(f'Engraving cache enabled: {cache_dir} ({state}; renderer {renderer})', flush=True)
   elif progress:
     print('Engraving cache disabled', flush=True)
-  totals = GroupResult()
+  totals = generator.engraving.worker.GroupResult()
   missing, rejected, completed = 0, 0, 0
   groups = {}
   for path in paths:
     try:
       document = json.loads(path.read_text(encoding='utf-8'))
-      generator.model.validate_document(document)
+      generator.validation.validate_document(document)
       score_key = generator.engraving.cache.score_fingerprint(document, renderer)
       destination = path.with_suffix('.pages')
-      clear_pages(destination)
+      generator.engraving.worker.clear_pages(destination)
       entry = cache_dir / 'fullscore' / score_key if cache_dir else None
       present = entry is not None and (entry.exists() or entry.is_symlink())
       pages = generator.engraving.cache.restore_pages(entry, destination, validated_pages) if entry else None
+      if pages is not None:
+        try:
+          generator.engraving.worker.validate_cached_views(document, pages)
+        except (ValueError, TypeError, KeyError):
+          pages = None
       if pages is None:
         rejected += int(present)
         missing += int(entry is not None and not present)
         notation = generator.engraving.cache.notation_fingerprint(document, renderer)
         groups.setdefault(notation, []).append((path, score_key))
       else:
-        write_bundle(path, document, pages, fingerprint)
+        generator.engraving.worker.write_bundle(path, document, pages, fingerprint)
         totals.fullscore += 1
         completed += 1
     except ValueError as error:
@@ -253,10 +94,10 @@ def engrave_tree(root: pathlib.Path, fonts: pathlib.Path, *, cache_dir: pathlib.
   jobs = min(jobs, len(groups))
   if jobs <= 1:
     for notation, group in groups.items():
-      finish(engrave_group(notation, group, root, fonts, cache_dir, fingerprint, progress))
+      finish(generator.engraving.worker.engrave_group(notation, group, root, fonts, cache_dir, fingerprint, progress))
   else:
     with concurrent.futures.ProcessPoolExecutor(max_workers=jobs, mp_context=multiprocessing.get_context('spawn')) as pool:
-      futures = [pool.submit(engrave_group, notation, group, root, fonts, cache_dir, fingerprint, progress) for notation, group in groups.items()]
+      futures = [pool.submit(generator.engraving.worker.engrave_group, notation, group, root, fonts, cache_dir, fingerprint, progress) for notation, group in groups.items()]
       for future in concurrent.futures.as_completed(futures):
         finish(future.result())
   if progress:

generator/engraving/raster.py

@@ -30,12 +30,14 @@ class OutlinePen(fontTools.pens.basePen.BasePen):
   def _endPath(self):
     pass
 
-def rasterize_page(page: generator.engraving.types.Page, fonts: generator.engraving.fonts.Fonts) -> PIL.Image.Image:
+def rasterize_page(page: generator.engraving.types.Page, fonts: generator.engraving.fonts.Fonts, *, ink=None, background=None, output_scale=1) -> PIL.Image.Image:
-  return rasterize_band(page, page.primitives, 0, page.height, fonts)
+  return rasterize_band(page, page.primitives, 0, page.height, fonts, ink=ink, background=background, output_scale=output_scale)
 
-def rasterize_band(page: generator.engraving.types.Page, primitives: tuple[generator.engraving.types.Primitive, ...], top: int, bottom: int, fonts: generator.engraving.fonts.Fonts) -> PIL.Image.Image:
+def rasterize_band(page: generator.engraving.types.Page, primitives: tuple[generator.engraving.types.Primitive, ...], top: int, bottom: int, fonts: generator.engraving.fonts.Fonts, *, ink=None, background=None, output_scale=1) -> PIL.Image.Image:
-  scale = page.supersampling
+  ink = generator.theme.rgb('ink') if ink is None else ink
-  image = PIL.Image.new('RGB', (page.width * scale, (bottom - top) * scale), generator.theme.rgb('background'))
+  background = generator.theme.rgb('background') if background is None else background
+  scale = page.supersampling * output_scale
+  image = PIL.Image.new('RGB', (round(page.width * scale), round((bottom - top) * scale)), background)
   context = aggdraw.Draw(image)
   for item in primitives:
     points = [(x * scale, y * scale - top * scale) for x, y in item.points]
@@ -56,16 +58,18 @@ def rasterize_band(page: generator.engraving.types.Page, primitives: tuple[gener
       if item.kind == 'polygon':
         path.close()
     if item.kind in ('glyph', 'polygon'):
-      context.path(path, None, aggdraw.Brush(generator.theme.rgb('ink')))
+      context.path(path, None, aggdraw.Brush(ink))
     else:
-      context.path(path, aggdraw.Pen(generator.theme.rgb('ink'), item.thickness * scale))
+      context.path(path, aggdraw.Pen(ink, item.thickness * scale))
   context.flush()
   drawing = PIL.ImageDraw.Draw(image)
   for item in primitives:
     if item.kind == 'text':
       x, y = item.points[0]
-      drawing.text((x * scale, y * scale - top * scale), item.value, font=fonts.text_font(item.size * scale), anchor='ls', fill=generator.theme.rgb('ink'))
+      drawing.text((x * scale, y * scale - top * scale), item.value, font=fonts.text_font(item.size * scale), anchor='ls', fill=ink)
-  return image.resize((page.width, bottom - top), PIL.Image.Resampling.LANCZOS)
+  result = image.resize((round(page.width * output_scale), round((bottom - top) * output_scale)), PIL.Image.Resampling.LANCZOS)
+  image.close()
+  return result
 
 @dataclasses.dataclass(frozen=True)
 class Band:

generator/engraving/spans.py

@@ -123,7 +123,7 @@ def draw_spans(index: generator.engraving.notation.ScoreIndex, anchors: list[gen
       y = anchor.y - anchor.direction * 8
       if anchor.x - 2 > left:
         result.append(generator.engraving.types.curve('tie', (identifier,), ((left, y), (left + 5, y - anchor.direction * 9), (anchor.x - 7, y - anchor.direction * 9), (anchor.x - 2, y)), 1.6))
-  top = min(-style.space * 5, min((p.box.top for p in items if p.role not in ('harmony', 'lyric', 'section', 'scale', 'tempo')), default=0))
+  top = min(-style.space * 5, min((p.box.top for p in items if p.role not in ('harmony', 'lyric', 'section', 'scale', 'tempo', 'measure_number')), default=0))
   tuplet_lanes = []
   for span in index.staff.get('tuplets', []):
     a, b = index.by_id[span['start']], index.by_id[span['end']]
@@ -203,11 +203,11 @@ def draw_spans(index: generator.engraving.notation.ScoreIndex, anchors: list[gen
   return items + result
 
 def place_text_lanes(items: list[generator.engraving.types.Primitive], fonts: generator.engraving.fonts.Fonts, style: generator.engraving.types.Style) -> list[generator.engraving.types.Primitive]:
-  music = [p for p in items if p.role not in ('harmony', 'lyric', 'section', 'scale')]
+  music = [p for p in items if p.role not in ('harmony', 'lyric', 'section', 'scale', 'measure_number')]
   box = generator.engraving.types.bounds(music)
   result = list(music)
   top = box.top - 12
-  for role in ('harmony', 'scale', 'section'):
+  for role in ('measure_number', 'harmony', 'scale', 'section'):
     lane = [p for p in items if p.role == role]
     if lane:
       extent = generator.engraving.types.bounds(lane)

generator/engraving/worker.py

@@ -0,0 +1,190 @@
+from __future__ import annotations
+
+import collections
+import dataclasses
+import hashlib
+import json
+import pathlib
+import shutil
+import tempfile
+import time
+
+import PIL.Image
+
+import generator.engraving.audit
+import generator.engraving.cache
+import generator.engraving.fonts
+import generator.engraving.layout
+import generator.engraving.notation
+import generator.engraving.raster
+import generator.engraving.types
+import generator.engraving.validation
+import generator.performance
+import generator.validation
+
+@dataclasses.dataclass
+class GroupResult:
+  scores: list = dataclasses.field(default_factory=list)
+  fullscore: int = 0
+  bodyhits: int = 0
+  bodies: int = 0
+  renderedpages: int = 0
+  rejected: int = 0
+  timings: dict = dataclasses.field(default_factory=lambda: dict.fromkeys(('layout', 'pagination', 'raster', 'cache'), 0.0))
+
+class BodyBands:
+  def __init__(self, root, persistent, result, limit=16 * 1024 * 1024):
+    self.root, self.persistent, self.result, self.limit = root, persistent, result, limit
+    self.images = collections.OrderedDict()
+    self.size = 0
+    self.known = {}
+    self.root.mkdir(parents=True, exist_ok=True)
+  def get(self, notation, page, band, fonts):
+    key = generator.engraving.cache.band_fingerprint(notation, page, band)
+    if key in self.images:
+      self.images.move_to_end(key)
+      self.result.bodyhits += 1
+      return self.images[key]
+    started = time.monotonic()
+    entry = self.root / (key + '.rgb')
+    image = None
+    if key in self.known:
+      if entry.is_symlink() or entry.stat().st_size != page.width * (band.bottom - band.top) * 3:
+        raise generator.engraving.types.EngravingError('temporary notation band was corrupted')
+      data = entry.read_bytes()
+      if hashlib.sha256(data).hexdigest() != self.known[key]:
+        raise generator.engraving.types.EngravingError('temporary notation band was corrupted')
+      image = PIL.Image.frombytes('RGB', (page.width, band.bottom - band.top), data)
+    elif self.persistent:
+      cached = self.persistent / 'notation' / key
+      image = generator.engraving.cache.restore_band(cached, key, page.width, band.bottom - band.top)
+      self.result.rejected += int(image is None and (cached.exists() or cached.is_symlink()))
+    self.result.timings['cache'] += time.monotonic() - started
+    if image is None:
+      started = time.monotonic()
+      image = generator.engraving.raster.rasterize_band(page, band.primitives, band.top, band.bottom, fonts)
+      self.result.timings['raster'] += time.monotonic() - started
+      self.result.bodies += 1
+      if self.persistent:
+        started = time.monotonic()
+        generator.engraving.cache.store_band(self.persistent / 'notation' / key, key, image)
+        self.result.timings['cache'] += time.monotonic() - started
+    else:
+      self.result.bodyhits += 1
+    if key not in self.known:
+      started = time.monotonic()
+      # Spill finalized pixels, so eviction never repeats rasterization or PNG decoding.
+      data = image.tobytes()
+      entry.write_bytes(data)
+      self.result.timings['cache'] += time.monotonic() - started
+      self.known[key] = hashlib.sha256(data).hexdigest()
+    size = image.width * image.height * 3
+    while self.images and self.size + size > self.limit:
+      _, old = self.images.popitem(last=False)
+      self.size -= old.width * old.height * 3
+      old.close()
+    self.images[key] = image
+    self.size += size
+    return image
+  def close(self):
+    for image in self.images.values():
+      image.close()
+    self.images.clear()
+    self.size = 0
+
+def clear_pages(destination):
+  if destination.is_symlink() or destination.is_file():
+    destination.unlink()
+  elif destination.exists():
+    shutil.rmtree(destination)
+
+def write_bundle(path, document, pages, fingerprint):
+  entries = [{k: v for k, v in {**page, 'path': path.with_suffix('.pages').name + '/' + page['path']}.items() if k != 'playback'} for page in pages]
+  normal = [entry for entry, page in zip(entries, pages) if not page['playback']]
+  playback = [entry for entry, page in zip(entries, pages) if page['playback']]
+  music = document
+  bundle = {'music': music, 'presentation': {'engine': generator.engraving.validation.ENGINE_NAME, 'style_sha256': fingerprint,
+    'width': generator.engraving.validation.PAGE_WIDTH, 'height': generator.engraving.validation.PAGE_HEIGHT, 'pages': normal, 'playback_pages': playback}}
+  validate_cached_views(document, pages)
+  bundle['performance'] = generator.performance.timing(document)
+  path.write_text(json.dumps(bundle, indent=2, ensure_ascii=False) + '\n', encoding='utf-8')
+
+def validate_cached_views(document, pages):
+  generator.performance.validate_views(document, [p for p in pages if not p['playback']], [p for p in pages if p['playback']])
+
+def engrave_group(notation, paths, root, font_path, cache_dir, fingerprint, progress=False):
+  family = generator.engraving.fonts.Fonts(font_path)
+  style = generator.engraving.types.Style()
+  systems = None
+  index = None
+  scores = {}
+  audited = set()
+  result = GroupResult()
+  with tempfile.TemporaryDirectory(prefix='vamp_notation_') as temporary:
+    bodies = BodyBands(pathlib.Path(temporary), cache_dir, result)
+    try:
+      for path, score_key in paths:
+        try:
+          document = json.loads(path.read_text(encoding='utf-8'))
+          style = generator.engraving.types.score_style(document, generator.engraving.types.Style())
+          destination = path.with_suffix('.pages')
+          clear_pages(destination)
+          destination.mkdir()
+          if score_key in scores:
+            source, pages = scores[score_key]
+            for page in pages:
+              shutil.copyfile(source / page['path'], destination / page['path'])
+            result.fullscore += 1
+          else:
+            if systems is None:
+              started = time.monotonic()
+              generator.validation.validate_document(document)
+              index = generator.engraving.notation.ScoreIndex(document)
+              systems = generator.engraving.layout.layout_systems(document, style, family, index=index)
+              result.timings['layout'] += time.monotonic() - started
+            started = time.monotonic()
+            pages_layout, playback_layout = generator.engraving.layout.page_views(systems, document, family, style, index=index)
+            placement = tuple((page.width, page.height, tuple((system.start, system.end, system.box) for system in page.systems)) for page in pages_layout)
+            if placement not in audited:
+              layout = generator.engraving.types.LayoutDocument(pages_layout, style, fingerprint)
+              generator.engraving.audit.audit_layout(document, layout, family)
+              generator.engraving.audit.audit_playback(pages_layout, playback_layout, style, family)
+              audited.add(placement)
+            else:
+              for page in pages_layout + playback_layout:
+                decoration = tuple(p for p in page.primitives if p.role in ('title', 'subtitle', 'page_number'))
+                generator.engraving.audit.audit_primitives(decoration, page.width, page.height, family)
+            result.timings['pagination'] += time.monotonic() - started
+            pages = []
+            rendered = {}
+            for is_playback, page in [(False, p) for p in pages_layout] + [(True, p) for p in playback_layout]:
+              if page.primitives in rendered:
+                name, digest = rendered[page.primitives]
+                pages.append(dict(path=name, sha256=digest, playback=is_playback, **generator.performance.page_timing(document, page)))
+                continue
+              bands = generator.engraving.raster.page_bands(page)
+              if bands is None:
+                started = time.monotonic()
+                image = generator.engraving.raster.rasterize_page(page, family)
+              else:
+                body = bodies.get(notation, page, bands[1], family)
+                started = time.monotonic()
+                image = generator.engraving.raster.compose_page(page, family, bands, body)
+              target = destination / (('playback-' if is_playback else '') + f'{page.number:04}.png')
+              with image:
+                image.save(target, format='PNG')
+              result.timings['raster'] += time.monotonic() - started
+              result.renderedpages += 1
+              digest = hashlib.sha256(target.read_bytes()).hexdigest()
+              rendered[page.primitives] = target.name, digest
+              pages.append(dict(path=target.name, sha256=digest, playback=is_playback, **generator.performance.page_timing(document, page)))
+            scores[score_key] = (destination, pages)
+          write_bundle(path, document, pages, fingerprint)
+          result.scores.append((path, score_key, pages))
+          if progress and len(result.scores) % 50 == 0:
+            print(f'Engraving body {notation[:12]}: {len(result.scores)}/{len(paths)} scores, fullscore {result.fullscore}, bodyhits {result.bodyhits}, bodies {result.bodies}, renderedpages {result.renderedpages}', flush=True)
+        except ValueError as error:
+          raise generator.engraving.types.EngravingError(f'{path.relative_to(root)}: {error}') from error
+    finally:
+      bodies.close()
+  return result

generator/fretboard.py

@@ -12,7 +12,8 @@ STANDARD_TUNING = ((6, 40), (5, 45), (4, 50), (3, 55), (2, 59), (1, 64))
 MIN_FRET = 1
 MAX_FRET = 20
 MAX_SINGLE_FRET = 12
-MAX_FRET_SPAN = 3
+PREFERRED_FRET_SPAN = 3
+MAX_FRET_SPAN = 4
 MAX_ANCHOR_FRET = 12
 WRITTEN_GUITAR_OCTAVE_OFFSET = 12
 
@@ -21,10 +22,17 @@ class TabPosition(typing.NamedTuple):
   fret: int
   midi: int
 
-def within_fret_span(positions) -> bool:
+def fret_span(positions) -> int:
-  """Count occupied fret positions inclusively: frets 1-3 fit, 1-4 do not."""
+  """Inclusive occupied positions: frets 7-9 span three, 7-10 span four."""
   frets = [position.fret for position in positions]
-  return not frets or max(frets) - min(frets) + 1 <= MAX_FRET_SPAN
+  return max(frets) - min(frets) + 1 if frets else 0
+
+def span_penalty(positions) -> int:
+  """Two tiers for legal grips; all one-to-three-fret shapes share tier zero."""
+  return int(fret_span(positions) > PREFERRED_FRET_SPAN)
+
+def within_fret_span(positions) -> bool:
+  return fret_span(positions) <= MAX_FRET_SPAN
 
 def has_fret_anchor(positions) -> bool:
   return any(MIN_FRET <= position.fret <= MAX_ANCHOR_FRET for position in positions)
@@ -66,6 +74,7 @@ def validate_document(document: dict) -> None:
     if not staff.get('tablature'):
       continue
     for measure in staff['measures']:
+      sounding = []
       for voice in measure['voices']:
         for note in voice['notes']:
           tones = []
@@ -73,8 +82,15 @@ def validate_document(document: dict) -> None:
             tab = pitch['tablature']
             position = TabPosition(tab['string'], tab['fret'], dict(STANDARD_TUNING)[tab['string']] + tab['fret'])
             tones.append((position, generator.model.fraction(pitch['duration'])))
+            start = generator.model.fraction(note['onset'])
+            sounding.append((start, start + generator.model.fraction(pitch['duration']), position))
           if not playable_attack(tones):
             raise ValueError(f'{measure["id"]}, event {note["id"]}: single notes require frets {MIN_FRET}-{MAX_SINGLE_FRET}; each sounding chord requires an anchor in {MIN_FRET}-{MAX_ANCHOR_FRET} and at most {MAX_FRET_SPAN} consecutive fret positions, including held/tied tails')
+      if len(measure['voices']) > 1:
+        for time in sorted({t for a, b, _ in sounding for t in (a, b)}):
+          held = [(p, b - time) for a, b, p in sounding if a <= time < b]
+          if not playable_attack(held):
+            raise ValueError(f'{measure["id"]}, beat {time + 1}: overlapping guitar voices conflict in string, fret span or sounding anchor')
 
 def validate_tuning(state: dict, location: str) -> None:
   details = state['staff_details']

generator/guitar.py

@@ -7,27 +7,62 @@ import itertools
 import logging
 import typing
 
+import generator.chord_families
 import generator.fretboard
 import generator.model
+import generator.hand_position
 import generator.pitch
+import generator.vocabulary
 
-PREFERRED_MAX_FRET = 12
 ELECTRIC_STRINGS = ((3, 2, 1), (4, 3, 2, 1))
 
 class VoicingCost(typing.NamedTuple):
-  string_shortfall: int = 0
+  extended_grips: int = 0
+  melody_string: int = 0
   covered_melody: int = 0
   covering_tones: int = 0
-  effort: int = 0
+  string_shortfall: int = 0
+  family_penalty: int = 0
+  travel: int = 0
+  peak_position: int = 0
+  effort: fractions.Fraction = fractions.Fraction()
   missing_tones: int = 0
+  register: fractions.Fraction = fractions.Fraction()
   def plus(self, other: VoicingCost) -> VoicingCost:
-    return VoicingCost(*(a + b for a, b in zip(self, other)))
+    return VoicingCost(*(max(a, b) if field == "peak_position" else a + b for field, a, b in zip(self._fields, self, other)))
   @functools.lru_cache(maxsize=16384)
-  def moved(self, distance: int) -> VoicingCost:
+  def moved(self, distance: int, ergonomics: int = 0) -> VoicingCost:
-    return self._replace(effort=self.effort + distance)
+    return self._replace(travel=self.travel + distance, effort=self.effort + ergonomics)
+
+def voicing_preference(role, center=None, *, composition=None):
+  """Immutable role settings for both selection and scored-retrieval caches."""
+  composition = composition or generator.vocabulary.current()
+  settings = composition.role(role)
+  policy = settings.get('voicing', {})
+  center = settings.get('center', 53 if role.startswith('acoustic') else 60) if center is None else center
+  if set(policy) - {'register_weight'}:
+    raise ValueError('unknown voicing preference; hand position is ranked across the passage')
+  result = tuple(fractions.Fraction(str(v)) for v in (center, policy.get('register_weight', 1)))
+  if result[1] < 0:
+    raise ValueError('voicing preference requires a nonnegative register tie weight')
+  return result
 
 @functools.lru_cache(maxsize=8192)
-def voicing_cost(positions: tuple[generator.fretboard.TabPosition, ...], profile: str = '', allowed: tuple[int, ...] = (), melody: tuple[int, ...] = ()) -> VoicingCost:
+def voicing_preferences(positions, preference):
+  if not positions or not preference:
+    return fractions.Fraction()
+  center, register_weight = preference
+  register = abs(fractions.Fraction(sum(p.midi for p in positions), len(positions)) - center)
+  return register_weight * register
+
+def voicing_cost(positions: tuple[generator.fretboard.TabPosition, ...], profile: str = '', allowed: tuple[int, ...] = (), melody: tuple[int, ...] = (), *, composition=None) -> VoicingCost:
+  composition = composition or generator.vocabulary.current()
+  families = tuple(composition.accompaniment('electric')['voicing_families']['rank'].items()) if profile == 'electric' else ()
+  preference = voicing_preference('electric_melody', composition=composition) if profile == 'electric' and melody else ()
+  return _voicing_cost(positions, profile, allowed, melody, families, preference)
+
+@functools.lru_cache(maxsize=8192)
+def _voicing_cost(positions, profile, allowed, melody, families, preference):
   frets = {p.fret for p in positions}
   classes = {p.midi % 12 for p in positions}
   above = sum(p.midi > max(melody) for p in positions) if melody else 0
@@ -35,11 +70,19 @@ def voicing_cost(positions: tuple[generator.fretboard.TabPosition, ...], profile
   if profile == 'electric':
     effort = 2 * max(0, len(frets) - 1) + 3 * span
   else:
-    effort = 2 * max(0, max(frets, default=0) - PREFERRED_MAX_FRET) + span
+    effort = span
-  return VoicingCost(max(0, 4 - len(positions)) if profile == 'electric' else 0, int(bool(above)), above, effort, len(set(allowed) - classes))
+  family_penalty = dict(families).get(generator.chord_families.electric(positions), 0) if positions and families else 0
+  melody_string = min((p.string_number - 1 for p in positions if melody and p.midi == max(melody)), default=0)
+  return VoicingCost(extended_grips=generator.fretboard.span_penalty(positions), melody_string=melody_string, string_shortfall=max(0, 4 - len(positions)) if profile == 'electric' else 0, covered_melody=int(bool(above)), covering_tones=above, family_penalty=family_penalty, effort=effort, missing_tones=len(set(allowed) - classes), register=voicing_preferences(positions, preference))
+
+def chord_positions(requirements: generator.fretboard.ChordRequirements, prune: bool = True, profile: str = '', melody: tuple[int, ...] = (), *, composition=None) -> tuple[tuple[generator.fretboard.TabPosition, ...], ...]:
+  composition = composition or generator.vocabulary.current()
+  families = tuple(composition.accompaniment('electric')['voicing_families']['rank'].items()) if profile == 'electric' else ()
+  preference = voicing_preference('electric_melody', composition=composition) if profile == 'electric' and melody else ()
+  return _chord_positions(requirements, prune, profile, melody, families, preference)
 
 @functools.lru_cache(maxsize=256)
-def chord_positions(requirements: generator.fretboard.ChordRequirements, prune: bool = True, profile: str = '', melody: tuple[int, ...] = ()) -> tuple[tuple[generator.fretboard.TabPosition, ...], ...]:
+def _chord_positions(requirements, prune, profile, melody, families, preference):
   result = []
   options = [tuple(generator.fretboard.TabPosition(string, fret, open_midi + fret) for fret in range(generator.fretboard.MIN_FRET, generator.fretboard.MAX_FRET + 1) if (open_midi + fret) % 12 in requirements.allowed or open_midi + fret in melody) for string, open_midi in generator.fretboard.STANDARD_TUNING]
 
@@ -57,6 +100,7 @@ def chord_positions(requirements: generator.fretboard.ChordRequirements, prune:
 
   if profile:
     by_string = {string: options[i] for i, (string, _) in enumerate(generator.fretboard.STANDARD_TUNING)}
+    # Every electric chord includes string 1, even with a lower melody inside.
     for strings in ELECTRIC_STRINGS:
       for selected in itertools.product(*(by_string[s] for s in strings)):
         if not generator.fretboard.within_fret_span(selected) or not generator.fretboard.has_fret_anchor(selected) or not requirements.covers({p.midi % 12 for p in selected}) or not set(melody) <= {p.midi for p in selected}:
@@ -66,29 +110,13 @@ def chord_positions(requirements: generator.fretboard.ChordRequirements, prune:
         result.append(selected)
   else:
     visit(0, (), set())
-  if not prune:
+  # Equal centers no longer imply equal future movement. Keep concrete grips;
-    return tuple(sorted(result))
+  # only identical positions can be merged before contextual placement.
-  # Independent attacks with equal centers have identical movement costs.
+  return tuple(sorted(set(result)))
-  # Held strings require the full candidate set instead of this reduction.
-  best = {}
-  for positions in result:
-    center = position_center(positions)
-    rank = (voicing_cost(positions, profile, requirements.allowed, melody), positions)
-    if center not in best or rank < best[center][0]:
-      best[center] = (rank, positions)
-  return tuple(sorted(item[1] for item in best.values()))
 
 @functools.lru_cache(maxsize=16384)
-def movement(previous: tuple[generator.fretboard.TabPosition, ...], current: tuple[generator.fretboard.TabPosition, ...]) -> int:
+def movement(previous, current, old_hand, hand) -> int:
-  if not previous or not current:
+  return generator.hand_position.motion(previous, current, old_hand, hand).shift
-    return 0
-  return sum(abs(a - b) for a, b in zip(position_center(previous), position_center(current)))
-
-@functools.lru_cache(maxsize=4096)
-def position_center(positions: tuple[generator.fretboard.TabPosition, ...]) -> tuple[int, int]:
-  frets = [p.fret for p in positions] or [0]
-  strings = [p.string_number for p in positions]
-  return min(frets) + max(frets), min(strings) + max(strings) - 2
 
 def compactness(positions) -> int:
   frets = [p.fret for p in positions]
@@ -110,38 +138,7 @@ class Placement:
   active: tuple
   previous: Placement | None
   choices: tuple
-
+  hand: int = 0
-def nearest_positions(states: dict) -> list[list[tuple | None]]:
-  """Exact Manhattan distance transform on the small fret/string position grid."""
-  initial = min(((p.cost, i, p) for i, p in enumerate(states.values()) if not p.active), key=lambda item: item[:2], default=None)
-  grid = [[initial for _ in range(11)] for _ in range(2 * generator.fretboard.MAX_FRET + 1)]
-  for index, previous in enumerate(states.values()):
-    if not previous.active:
-      continue
-    x, y = position_center(tuple(p for _, p in previous.active))
-    candidate = (previous.cost, index, previous)
-    if grid[x][y] is None or candidate[:2] < grid[x][y][:2]:
-      grid[x][y] = candidate
-
-  def relax(x, y, a, b):
-    source = grid[a][b]
-    if source is not None:
-      cost, index, previous = source
-      candidate = (cost.moved(1), index, previous)
-      if grid[x][y] is None or candidate[:2] < grid[x][y][:2]:
-        grid[x][y] = candidate
-
-  for y in range(11):
-    for x in range(1, len(grid)):
-      relax(x, y, x - 1, y)
-    for x in range(len(grid) - 2, -1, -1):
-      relax(x, y, x + 1, y)
-  for x in range(len(grid)):
-    for y in range(1, 11):
-      relax(x, y, x, y - 1)
-    for y in range(9, -1, -1):
-      relax(x, y, x, y + 1)
-  return grid
 
 @dataclasses.dataclass(frozen=True)
 class GuitarTone:
@@ -173,7 +170,8 @@ def continued_sources(notes):
 def has_attack(notes: list[dict]) -> bool:
   return any('_harmony' in n and '_continue_from' not in n or any('tie_from' not in p for p in n['pitches']) for n in notes)
 
-def attack_candidates(notes: list[dict], tones: dict[str, GuitarTone], active: dict, held: dict, preserve_strings: bool, fallback: bool = False):
+def attack_candidates(notes: list[dict], tones: dict[str, GuitarTone], active: dict, held: dict, preserve_strings: bool, fallback: bool = False, *, composition=None):
+  composition = composition or generator.vocabulary.current()
   pending = []
   chords = [n for n in notes if '_harmony' in n]
   chord = chords[0] if chords else None
@@ -191,7 +189,7 @@ def attack_candidates(notes: list[dict], tones: dict[str, GuitarTone], active: d
   requirements = generator.fretboard.chord_requirements(chord['_harmony'], respect_bass=profile != 'electric') if chord else generator.fretboard.ChordRequirements()
   melody = tuple(sorted({p.midi for _, options in pending for p in options})) if profile == 'electric' else ()
   fresh = has_attack(notes)
-  candidates = chord_positions(requirements, prune=not (preserve_strings or held or pending or profile), profile=profile, melody=melody) if chord else [()]
+  candidates = chord_positions(requirements, prune=not (preserve_strings or held or pending or profile), profile=profile, melody=melody, composition=composition) if chord else [()]
   for exact in extend_positions(held, sorted(pending, key=lambda p: len(p[1]))):
     exact_positions = dict(exact)
     for positions in candidates:
@@ -215,7 +213,7 @@ def attack_candidates(notes: list[dict], tones: dict[str, GuitarTone], active: d
           choices.append(EventPlacement(note, tuple((p, exact_positions[p['id']]) for p in note['pitches']), added))
         if not compatible or not playable_choices(choices):
           continue
-        cost = voicing_cost(positions, profile, requirements.allowed, melody) if fresh else VoicingCost()
+        cost = voicing_cost(positions, profile, requirements.allowed, melody, composition=composition) if fresh else VoicingCost()
         yield AttackCandidate(tuple(sorted(assignment.items())), tuple(choices), positions, cost)
       else:
         shape = tuple(p for _, p in exact) + positions
@@ -225,7 +223,7 @@ def attack_candidates(notes: list[dict], tones: dict[str, GuitarTone], active: d
         choices = tuple(EventPlacement(n, tuple((p, exact_positions[p['id']]) for p in n['pitches']), positions if n is chord else ()) for n in notes)
         if not playable_choices(choices):
           continue
-        cost = voicing_cost(shape, profile, requirements.allowed, melody) if fresh else VoicingCost()
+        cost = voicing_cost(shape, profile, requirements.allowed, melody, composition=composition) if fresh else VoicingCost()
         yield AttackCandidate(assignment, choices, shape, cost)
 
 def guitar_events(document: dict) -> tuple[list, dict[str, GuitarTone]]:
@@ -276,55 +274,78 @@ def guitar_events(document: dict) -> tuple[list, dict[str, GuitarTone]]:
 def retain_tone(tone: GuitarTone, onset: fractions.Fraction) -> bool:
   return tone.end > onset or tone.end == onset and bool(tone.pitch.get('tie_to'))
 
+def keep_position(result, candidate, previous, fresh=True):
+  old_shape = tuple(p for _, p in previous.active)
+  hands = [previous.hand]
+  if fresh:
+    hands = generator.hand_position.regions(candidate.shape)
+  for hand in hands:
+    if candidate.shape and not all(hand <= p.fret <= hand + 3 for p in candidate.shape):
+      continue
+    motion = generator.hand_position.motion(old_shape, candidate.shape, previous.hand, hand) if fresh else generator.hand_position.Motion()
+    local = candidate.cost._replace(peak_position=hand)
+    cost = previous.cost.plus(local).moved(motion.shift, motion.ergonomics)
+    key = (candidate.active, hand, cost.peak_position)
+    value = Placement(cost, candidate.active, previous, candidate.choices, hand)
+    if key not in result or cost < result[key].cost:
+      result[key] = value
+
+def reduce_positions(result):
+  # Future effort depends on the concrete held strings and hand, not on the
+  # earlier route. A strictly worse primary cost is dominated even if lower.
+  best = {}
+  for (active, hand, _), state in result.items():
+    key = (active, hand)
+    best[key] = min(best.get(key, state.cost[:7]), state.cost[:7])
+  return {key: state for key, state in result.items() if state.cost[:7] == best[key[:2]]}
+
 def advance_attacks(candidates, states: dict) -> dict:
-  nearest = nearest_positions(states)
   result = {}
   for candidate in candidates:
-    x, y = position_center(candidate.shape)
+    for previous in states.values():
-    cost, _, previous = nearest[x][y]
+      keep_position(result, candidate, previous)
-    cost = cost.plus(candidate.cost)
+  return reduce_positions(result)
-    if candidate.active not in result or cost < result[candidate.active].cost:
-      result[candidate.active] = Placement(cost, candidate.active, previous, candidate.choices)
-  return result
 
-def advance_positions(notes: list[dict], onset: fractions.Fraction, tones: dict[str, GuitarTone], states: dict, preserve_strings: bool, fallback: bool = False) -> dict:
+def advance_positions(notes: list[dict], onset: fractions.Fraction, tones: dict[str, GuitarTone], states: dict, preserve_strings: bool, fallback: bool = False, *, composition=None) -> dict:
+  composition = composition or generator.vocabulary.current()
   continued = continued_sources(notes)
   fresh = has_attack(notes)
   held_ids = {i for previous in states.values() for i, _ in previous.active if retain_tone(tones[i], onset)}
   result = {}
-
-  def keep(candidate, previous, cost):
-    cost = cost.plus(candidate.cost)
-    if candidate.active not in result or cost < result[candidate.active].cost:
-      result[candidate.active] = Placement(cost, candidate.active, previous, candidate.choices)
-
   if not held_ids and not continued:
-    result = advance_attacks(attack_candidates(notes, tones, {}, {}, preserve_strings, fallback), states)
+    result = advance_attacks(attack_candidates(notes, tones, {}, {}, preserve_strings, fallback, composition=composition), states)
   else:
     for previous in states.values():
       active = dict(previous.active)
       held = {i: p for i, p in previous.active if i not in continued and i in held_ids}
-      for candidate in attack_candidates(notes, tones, active, held, preserve_strings, fallback):
+      for candidate in attack_candidates(notes, tones, active, held, preserve_strings, fallback, composition=composition):
-        distance = movement(tuple(p for _, p in previous.active), candidate.shape) if fresh else 0
+        keep_position(result, candidate, previous, fresh)
-        keep(candidate, previous, previous.cost.moved(distance))
   if not result:
     if not fallback and any(n.get('_harmonize') for n in notes):
-      return advance_positions(notes, onset, tones, states, preserve_strings, True)
+      return advance_positions(notes, onset, tones, states, preserve_strings, True, composition=composition)
     raise ValueError(f'event {notes[0]["id"]}, onset {onset}: no guitar placement; single notes need frets {generator.fretboard.MIN_FRET}-{generator.fretboard.MAX_SINGLE_FRET}; chords need frets {generator.fretboard.MIN_FRET}-{generator.fretboard.MAX_FRET}, an anchor <= {generator.fretboard.MAX_ANCHOR_FRET}, and at most {generator.fretboard.MAX_FRET_SPAN} consecutive fret positions; held/tied strings must remain fixed')
-  return result
+  return reduce_positions(result)
 
-def place_guitar(document: dict) -> None:
+def place_guitar(document: dict, *, composition=None) -> dict:
   """Choose electric voicings or exact/held melody placements."""
+  composition = composition or generator.vocabulary.current()
   events, tones = guitar_events(document)
   states = {(): Placement(VoicingCost(), (), None, ())}
   groups = [(key[0], [item[3] for item in values]) for key, values in itertools.groupby(events, key=lambda e: e[:3])]
   for index, (onset, notes) in enumerate(groups):
     next_onset = groups[index + 1][0] if index + 1 < len(groups) else None
     preserve_strings = next_onset is not None and any('_harmony' in n and onset + generator.model.fraction(n['duration']) > next_onset for n in notes)
-    states = advance_positions(notes, onset, tones, states, preserve_strings)
+    states = advance_positions(notes, onset, tones, states, preserve_strings, composition=composition)
   selected = min(states.values(), key=lambda s: (s.cost, s.active))
+  trace = []
+  final_cost = selected.cost
   skipped = []
   while selected.previous is not None:
+    fresh = has_attack([c.note for c in selected.choices])
+    motion = generator.hand_position.motion(tuple(p for _, p in selected.previous.active), tuple(p for _, p in selected.active), selected.previous.hand, selected.hand) if fresh else generator.hand_position.Motion()
+    trace.append({'hand_position': selected.hand, 'events': [c.note['id'] for c in selected.choices],
+      'hand_shift': motion.shift, 'finger_adjustment': motion.fingers, 'string_changes': motion.strings,
+      'effort': str(selected.cost.effort - selected.previous.cost.effort)})
     for choice in selected.choices:
       note = choice.note
       for tone, position in choice.exact:
@@ -344,3 +365,4 @@ def place_guitar(document: dict) -> None:
     logging.getLogger(__name__).warning('%s: %s melody harmonizations kept as single notes; first event %s', document['metadata']['title'], len(skipped), skipped[-1])
   document['staves'][0]['tablature'] = True
   generator.fretboard.validate_document(document)
+  return {'version': generator.hand_position.VERSION, **generator.hand_position.summary(r['hand_shift'] for r in trace), 'peak_position': final_cost.peak_position, 'cost': {k: str(v) for k, v in final_cost._asdict().items()}, 'positions': list(reversed(trace))}

generator/hand_position.py

@@ -0,0 +1,52 @@
+"""Four-fret hand windows shared by chord and single-note placement.
+
+This describes playable regions, not an anatomical fingering solver. Chord
+and single tones may occupy any frets within the window. Only shifting the
+window is movement. Actual sounded-note limits remain in fretboard.py.
+"""
+import functools
+import typing
+
+import generator.fretboard
+
+VERSION = 'hand-window-2'
+
+class Motion(typing.NamedTuple):
+  shift: int = 0
+  fingers: int = 0
+  strings: int = 0
+  @property
+  def ergonomics(self):
+    return self.fingers + self.strings
+
+def base(shape):
+  return min((p.fret for p in shape), default=0)
+
+@functools.lru_cache(maxsize=4096)
+def regions(shape):
+  if not shape:
+    return tuple([0])
+  return tuple(range(max(1, max(p.fret for p in shape) - 3), min(base(shape), generator.fretboard.MAX_ANCHOR_FRET) + 1))
+
+@functools.lru_cache(maxsize=32768)
+def motion(previous, current, old_hand, hand, acoustic=False):
+  if not previous or not current:
+    return Motion()
+  if len(previous) == len(current) == 1:
+    pairs = ((previous[0], current[0]),)
+    unmatched = 0
+  else:
+
+    def voices(shape):
+      return {('bass' if acoustic and p.string_number in (5, 6) else p.string_number): p for p in shape}
+
+    a, b = voices(previous), voices(current)
+    pairs = tuple((a[s], b[s]) for s in a.keys() & b.keys())
+    unmatched = len(a.keys() ^ b.keys())
+  return Motion(abs(hand - old_hand),
+    unmatched + sum(abs((a.fret - old_hand) - (b.fret - hand)) for a, b in pairs),
+    sum(abs(a.string_number - b.string_number) for a, b in pairs))
+
+def summary(shifts):
+  shifts = tuple(shifts)
+  return {'total_travel': sum(shifts), 'shift_count': sum(s > 0 for s in shifts), 'largest_shift': max(shifts, default=0)}

generator/harmonic_time.py

@@ -4,7 +4,7 @@ import dataclasses
 import fractions
 
 import generator.model
-
+import generator.score
 
 def solo_source(staff):
   """Source-form pickup context, shared by training and arranged choruses.
@@ -14,11 +14,11 @@ def solo_source(staff):
   retain their ordinary initial-harmony checks. Explicit pickup harmony
   (including N.C.) still takes precedence in read().
   """
-  from generator.score import body_start_measure_index
+  measures = staff['measures']
-  measures=staff['measures'];initial=None
+  initial = None
-  if body_start_measure_index(measures):
+  if generator.score.body_start_measure_index(measures):
-    initial=next((h for m in reversed(measures) for h in reversed(m['harmonies'])),None)
+    initial = next((h for m in reversed(measures) for h in reversed(m['harmonies'])), None)
-  return read(staff,initial=initial)
+  return read(staff, initial=initial)
 
 @dataclasses.dataclass(frozen=True)
 class ChordSpan:

generator/input_audit.py

@@ -3,35 +3,39 @@ from __future__ import annotations
 import contextlib
 import contextvars
 import os
-from pathlib import Path
+import pathlib
 import sys
 
-ACTIVE=contextvars.ContextVar('composition_reads',default=None)
+ACTIVE = contextvars.ContextVar('composition_reads', default=None)
 
-
+def _read(event, args):
-def _read(event,args):
+  scope = ACTIVE.get()
-  scope=ACTIVE.get()
+  if scope is None or event != 'open':
-  if scope is None or event!='open':return
+    return
-  name,mode,flags=args
+  name, mode, flags = args
-  if not isinstance(name,(str,bytes,os.PathLike)):return
+  if not isinstance(name, (str, bytes, os.PathLike)):
+    return
   # r+ and w+ can read; they must obey the same boundary as a plain read.
-  if flags & os.O_WRONLY:return
+  if flags & os.O_WRONLY:
-  path=Path(os.fsdecode(name)).resolve()
+    return
-  data,code,cache,reads=scope
+  path = pathlib.Path(os.fsdecode(name)).resolve()
-  python_runtime=any(path.is_relative_to(Path(root).resolve()) for root in (sys.base_prefix,sys.prefix))
+  data, code, cache, reads = scope
-  allowed=path.is_relative_to(data) or path.is_relative_to(code) or python_runtime
+  python_runtime = any(path.is_relative_to(pathlib.Path(root).resolve()) for root in (sys.base_prefix, sys.prefix))
-  if cache and path.is_relative_to(cache):allowed=True
+  allowed = path.is_relative_to(data) or path.is_relative_to(code) or python_runtime
-  if not allowed:raise ValueError('Forbidden composition input: '+str(path))
+  if cache and path.is_relative_to(cache):
+    allowed = True
+  if not allowed:
+    raise ValueError('Forbidden composition input: ' + str(path))
   reads.add(str(path))
 
-
 sys.addaudithook(_read)
 
-
 @contextlib.contextmanager
-def composition(data,role,cache=None):
+def composition(data, role, cache=None):
-  reads=set()
+  reads = set()
-  code=Path(__file__).resolve().parent
+  code = pathlib.Path(__file__).resolve().parent
-  token=ACTIVE.set((Path(data).resolve(),code,Path(cache).resolve()/role if cache else None,reads))
+  token = ACTIVE.set((pathlib.Path(data).resolve(), code, pathlib.Path(cache).resolve() / role if cache else None, reads))
-  try:yield reads
+  try:
-  finally:ACTIVE.reset(token)
+    yield reads
+  finally:
+    ACTIVE.reset(token)

generator/model.py

@@ -138,159 +138,3 @@ def make_event(identifier: str, onset: fractions.Fraction, duration: fractions.F
     if 'alter' in event['pitches'][-1]:
       event['pitches'][-1]['alter'] = rational(alteration(p['alter']))
   return event
-
-def validate_document(document: dict) -> None:
-  if not isinstance(document, dict) or not isinstance(document.get('metadata'), dict):
-    raise ValueError('score requires a metadata object')
-  if any(not isinstance(document['metadata'].get(name), str) for name in ('title', 'composer', 'key_signature', 'time_signature')):
-    raise ValueError('score metadata requires title, composer, key_signature and time_signature strings')
-  staves = document.get('staves', [])
-  if not isinstance(staves, list) or len(staves) != 1 or not isinstance(staves[0], dict):
-    raise ValueError('exactly one staff is currently supported')
-  if not isinstance(staves[0].get('measures'), list) or not staves[0]['measures']:
-    raise ValueError('score requires measures')
-  instruments = staves[0].get('instruments', [])
-  instrument_ids = {i['id'] for i in instruments}
-  if len(instruments) != len(instrument_ids) or any(not isinstance(i, str) or not i for i in instrument_ids):
-    raise ValueError('score instruments require unique nonempty IDs')
-  ids: set[str] = set()
-  events: dict[str, dict] = {}
-  tones: dict[str, tuple[dict, fractions.Fraction, fractions.Fraction, str]] = {}
-  ownership: dict[str, str] = {}
-  absolute = fractions.Fraction()
-  for measure in staves[0]['measures']:
-    if {'repeats', 'endings', 'initial_attributes'} & measure.keys():
-      raise ValueError('repeat notation is unsupported; regenerate scores from music written out in reading order')
-    length = fraction(measure['duration'])
-    if length <= 0:
-      raise ValueError('measure duration must be positive')
-    voices = measure.get('voices', [])
-    if not voices or len({v['id'] for v in voices}) != len(voices) or any(not v['id'] for v in voices):
-      raise ValueError('measure requires uniquely identified voice lanes')
-    previous_ends = {v['id']: fractions.Fraction() for v in voices}
-    for voice, note in ((v, n) for v in voices for n in v['notes']):
-      if any({'sound_ties', 'printed_ties'} & pitch.keys() for pitch in note['pitches']):
-        raise ValueError('pass-specific tie metadata is unsupported; regenerate scores')
-      if not note['id'] or note['id'] in ids:
-        raise ValueError(f'duplicate event ID: {note["id"]}')
-      ids.add(note['id'])
-      events[note['id']] = note
-      ownership[note['id']] = voice['id']
-      if note.get('voice', voice['id']) != voice['id']:
-        raise ValueError('event voice ownership disagrees with its lane')
-      onset, duration = fraction(note['onset']), fraction(note['duration'])
-      if onset < 0 or duration < 0 or onset + duration > length:
-        raise ValueError(f'event outside measure: {note["id"]}')
-      if note['kind'] not in ('note', 'rest', 'spacer'):
-        raise ValueError('invalid event kind')
-      if not note.get('grace'):
-        if duration == 0 or onset < previous_ends[voice['id']]:
-          raise ValueError('overlapping events in one rhythmic voice')
-        previous_ends[voice['id']] = onset + duration
-      elif duration != 0:
-        raise ValueError('grace notes must not advance metric time')
-      expected = metric_length(note['written'])
-      if not note.get('grace') and not note.get('measure_rest') and duration != expected:
-        raise ValueError('metric and written event durations disagree')
-      if note['kind'] != 'note' and note.get('pitches'):
-        raise ValueError('rests and spacers cannot contain pitches')
-      if note['kind'] == 'note' and not note.get('pitches') and note.get('notehead') != 'slash':
-        raise ValueError('pitched event requires at least one pitch')
-      for p in note.get('pitches', []):
-        if p.get('instrument') and p['instrument'] not in instrument_ids:
-          raise ValueError('note refers to an unknown score instrument')
-        if 'unpitched' in p:
-          position = p['unpitched']
-          if position['letter'] not in LETTERS or type(position['octave']) is not int:
-            raise ValueError('invalid unpitched display position')
-        else:
-          pitch_number(p)
-        if staves[0].get('tablature'):
-          import generator.fretboard
-          import generator.pitch
-          tab = p.get('tablature', {})
-          string, fret = tab.get('string'), tab.get('fret')
-          if type(string) is not int or type(fret) is not int or string not in dict(generator.fretboard.STANDARD_TUNING) or not generator.fretboard.MIN_FRET <= fret <= generator.fretboard.MAX_FRET:
-            raise ValueError(f'{p["id"]}: invalid generated tablature position')
-          state = dict(measure)
-          for change in measure.get('attributes', []):
-            if fraction(change['onset']) <= onset:
-              state.update(change)
-          if dict(generator.fretboard.STANDARD_TUNING)[string] + fret != generator.pitch.sounding_pitch(p, state):
-            raise ValueError(f'{p["id"]}: tablature disagrees with sounding pitch')
-        pd = fraction(p['duration'])
-        if pd < 0 or (pd == 0 and not note.get('grace')) or pd > duration or not p['id'] or p['id'] in tones:
-          raise ValueError('invalid chord-tone duration or duplicate ID')
-        if not note.get('grace') and pd != metric_length(p['written']):
-          raise ValueError('metric and written chord-tone durations disagree')
-        tones[p['id']] = (p, absolute + onset, pd, voice['id'])
-    for harmony in measure.get('harmonies', []):
-      if not 0 <= fraction(harmony['onset']) < length:
-        raise ValueError('harmony outside measure')
-    for change in measure.get('attributes', []):
-      if not 0 <= fraction(change['onset']) < length:
-        raise ValueError('attribute change outside measure')
-    for state in [measure, *measure.get('attributes', [])]:
-      if 'meter' in state:
-        validate_meter(state['meter'])
-      if 'key' in state:
-        if len(state['key']['alterations']) != 7:
-          raise ValueError('key alterations must contain seven exact alterations')
-        for value in state['key']['alterations']:
-          alteration(value)
-      if 'clef' in state and (state['clef']['sign'] not in ('G', 'F', 'C', 'percussion', 'none') or not 1 <= state['clef']['line'] <= 6 or not -3 <= state['clef']['octave_change'] <= 3):
-        raise ValueError('invalid clef')
-    absolute += length
-  for identifier, (p, start, duration, voice) in tones.items():
-    if target := p.get('tie_to'):
-      if target not in tones:
-        raise ValueError(f'dangling tie: {identifier} -> {target}')
-      other, next_start, _, next_voice = tones[target]
-      same_pitch = all(p.get(k) == other.get(k) for k in ('unpitched', 'instrument')) if 'unpitched' in p else all(p.get(k) == other.get(k) for k in ('letter', 'alter', 'octave'))
-      if other.get('tie_from') != identifier or not same_pitch or next_voice != voice or next_start != start + duration:
-        raise ValueError(f'invalid tie: {identifier} -> {target}')
-      if staves[0].get('tablature') and p['tablature'] != other['tablature']:
-        raise ValueError(f'{identifier}: tied guitar tones must retain their string and fret')
-    if source := p.get('tie_from'):
-      if source not in tones or tones[source][0].get('tie_to') != identifier:
-        raise ValueError(f'dangling incoming tie: {identifier}')
-  span_ids = set()
-  event_order = {identifier: index for index, identifier in enumerate(events)}
-  for span in staves[0].get('tuplets', []):
-    if not span['id'] or span['id'] in span_ids or span['start'] not in ids or span['end'] not in ids or not 0 < span['note_count'] <= 1000000 or not 0 < span['space_count'] <= 1000000 or not 0 <= span['depth'] <= 16:
-      raise ValueError('invalid tuplet span')
-    members = [n['id'] for n in events.values() if span['id'] in n.get('tuplet_ids', [])]
-    if not members or members[0] != span['start'] or members[-1] != span['end'] or event_order[span['start']] > event_order[span['end']]:
-      raise ValueError('tuplet endpoints disagree with member events')
-    if len({ownership[n] for n in members}) != 1 or span.get('voice', ownership[members[0]]) != ownership[members[0]]:
-      raise ValueError('tuplet crosses independent voice lanes')
-    if span.get('bracket', 'auto') not in ('yes', 'no', 'auto') or span.get('show_number', 'actual') not in ('none', 'actual', 'both'):
-      raise ValueError('invalid tuplet appearance')
-    span_ids.add(span['id'])
-  for note in events.values():
-    for verse in note.get('lyrics', []):
-      start = tones[note['pitches'][0]['id']][1] if note['pitches'] else fractions.Fraction()
-      if 'extend_to' in verse and not start < fraction(verse['extend_to']) <= absolute:
-        raise ValueError('lyric extension endpoint is outside the score')
-    if any(t not in span_ids for t in note.get('tuplet_ids', [])):
-      raise ValueError('event refers to an unknown tuplet')
-    if tremolo := note.get('tremolo'):
-      if tremolo['type'] not in ('single', 'start', 'stop') or type(tremolo['marks']) is not int or not 0 <= tremolo['marks'] <= 8:
-        raise ValueError('invalid measured tremolo')
-      if tremolo['type'] in ('start', 'stop'):
-        other = events.get(tremolo.get('partner'))
-        expected_type = 'stop' if tremolo['type'] == 'start' else 'start'
-        if not other or other.get('tremolo', {}).get('partner') != note['id'] or other['tremolo']['type'] != expected_type or other['tremolo']['marks'] != tremolo['marks']:
-          raise ValueError('invalid paired tremolo')
-  for span in staves[0].get('octave_shifts', []):
-    if not 0 <= fraction(span['start']) < fraction(span['end']) <= absolute or not -3 <= span['octaves'] <= 3:
-      raise ValueError('invalid octave-shift span')
-  previous_end = fractions.Fraction()
-  for span in staves[0].get('scale_spans', []):
-    start, end = fraction(span['start']), fraction(span['end'])
-    if not previous_end <= start < end <= absolute or not isinstance(span.get('label'), str) or not span['label'].strip() or type(span.get('matched')) is not bool:
-      raise ValueError('invalid solo scale span')
-    if not isinstance(span.get('id'), str) or not span['id'] or span['id'] in ids:
-      raise ValueError('solo scale spans require unique IDs')
-    ids.add(span['id'])
-    previous_end = end

generator/musicxml.py

@@ -8,6 +8,7 @@ import defusedxml.ElementTree
 
 import generator.harmony
 import generator.model
+import generator.validation
 
 class MusicXMLError(ValueError):
   pass
@@ -521,7 +522,7 @@ class Importer:
       name = item.get('name', '')
       if name.startswith('vamp:'):
         self.fail(f'unsupported Vamp property {name!r}')
-    metadata = {'title': root.findtext('work/work-title') or root.findtext('movement-title') or self.path.stem, 'composer': ' / '.join(c.text for c in root.findall('identification/creator') if c.get('type') == 'composer' and c.text)}
+    metadata = {'title': root.findtext('work/work-title') or root.findtext('movement-title') or self.path.stem.replace('_', ' ').replace('-', ' ').title(), 'composer': ' / '.join(c.text for c in root.findall('identification/creator') if c.get('type') == 'composer' and c.text)}
     measures = []
     self.children(parts[0], {'measure'})
     for index, bar in enumerate(parts[0]):
@@ -621,7 +622,7 @@ class Importer:
     metadata.update({'key_signature': measures[0]['key_signature'], 'time_signature': measures[0]['time_signature'], 'key': copy.deepcopy(measures[0]['key'])})
     result = {'metadata': metadata, 'staves': [{'id': 'staff_source', 'name': definitions[0].findtext('part-name', 'Source'), 'role': 'vocal', 'clef': copy.deepcopy(measures[0]['clef']), 'tuplets': self.tuplets, 'octave_shifts': self.octaves, 'measures': measures}]}
     result['staves'][0]['instruments'] = instruments
-    generator.model.validate_document(result)
+    generator.validation.validate_document(result)
     return result
 
 def load_musicxml(path: pathlib.Path) -> dict:

generator/musicxml_writer.py

@@ -7,6 +7,7 @@ import xml.etree.ElementTree
 
 import generator.harmony
 import generator.model
+import generator.validation
 
 def element(parent, tag, content=None, **attributes):
   node = xml.etree.ElementTree.SubElement(parent, tag, {k.replace('_', '-'): str(v) for k, v in attributes.items() if v is not None})
@@ -209,7 +210,7 @@ def write_barline(bar, measure, location):
   element(node, 'bar-style', measure.get('barlines', {}).get(location, 'regular'))
 
 def musicxml_tree(document: dict, *, tablature=False) -> xml.etree.ElementTree.ElementTree:
-  generator.model.validate_document(document)
+  generator.validation.validate_document(document)
   root = xml.etree.ElementTree.Element('score-partwise', {'version': '4.0'})
   metadata, staff = document['metadata'], document['staves'][0]
   element(element(root, 'work'), 'work-title', metadata['title'])

generator/parts.py

@@ -3,7 +3,6 @@ from __future__ import annotations
 import fractions
 import pathlib
 
-import generator.accompaniment
 import generator.fretboard
 import generator.model
 import generator.pitch

generator/performance.py

@@ -0,0 +1,113 @@
+"""Pure page coverage and performance timing."""
+from __future__ import annotations
+
+import fractions
+import math
+
+import generator.model
+
+TEMPOS = tuple(range(60, 121, 10))
+
+def boundaries(document):
+  result = [fractions.Fraction()]
+  for measure in document['staves'][0]['measures']:
+    result.append(result[-1] + generator.model.fraction(measure['duration']))
+  return result
+
+def page_timing(document, page):
+  times = boundaries(document)
+  start, end = page.systems[0].start, page.systems[-1].end
+  first, last = times.index(start), times.index(end)
+  measures = document['staves'][0]['measures']
+  systems = []
+  for system in page.systems:
+    a, b = times.index(system.start), times.index(system.end)
+    labels = {p.sources[0]: p for p in system.primitives if p.role == 'measure_number'}
+    boxes = []
+    for i in range(a, b):
+      box = labels[measures[i]['id']].box
+      boxes.append(dict(measure=i + 1, bounds=[box.left, box.top, box.right, box.bottom]))
+    systems.append(dict(first_measure=a + 1, measure_count=b - a, start_beat=generator.model.rational(system.start), end_beat=generator.model.rational(system.end), labels=boxes))
+  return dict(first_measure=first + 1, measure_count=last - first, start_beat=generator.model.rational(start), end_beat=generator.model.rational(end), systems=systems)
+
+def validate_views(document, pages, playback_pages):
+  validate_pages(document, pages)
+  times = boundaries(document)
+  canonical = []
+
+  def system_range(system):
+    a, count = system['first_measure'], system['measure_count']
+    if type(a) is not int or type(count) is not int or a < 1 or count < 1 or a + count > len(times):
+      raise ValueError('invalid system measure coverage')
+    if generator.model.fraction(system['start_beat']) != times[a - 1] or generator.model.fraction(system['end_beat']) != times[a + count - 1]:
+      raise ValueError('invalid system timing')
+    if [label['measure'] for label in system['labels']] != list(range(a, a + count)):
+      raise ValueError('invalid measure label coverage')
+    for label in system['labels']:
+      box = label['bounds']
+      if not isinstance(box, list) or len(box) != 4 or any(type(v) not in (int, float) or not math.isfinite(v) for v in box) or not (4 <= box[0] < box[2] <= 972 and 4 <= box[1] < box[3] <= 480):
+        raise ValueError('invalid measure label bounds')
+    return a, count
+
+  for page in pages:
+    systems = page.get('systems', [])
+    if len(systems) != 2 and (page is not pages[-1] or len(systems) != 1):
+      raise ValueError('invalid page systems')
+    cursor = page['first_measure']
+    for system in systems:
+      a, count = system_range(system)
+      if a != cursor:
+        raise ValueError('system gap or overlap')
+      cursor += count
+      canonical.append(system)
+    if cursor != page['first_measure'] + page['measure_count']:
+      raise ValueError('systems disagree with page coverage')
+  if not isinstance(playback_pages, list) or len(playback_pages) != len(canonical):
+    raise ValueError('missing playback pages; regenerate score assets')
+  for i, page in enumerate(playback_pages):
+    expected = canonical[i:i + 2]
+    systems = page.get('systems', [])
+    if len(systems) != len(expected):
+      raise ValueError('invalid playback systems')
+    for actual, source in zip(systems, expected):
+      system_range(actual)
+      if any(actual[k] != source[k] for k in ('first_measure', 'measure_count', 'start_beat', 'end_beat')):
+        raise ValueError('playback changes system boundaries')
+    if any(page[k] != expected[0][k] for k in ('first_measure', 'start_beat')) or page['end_beat'] != expected[-1]['end_beat'] or page['measure_count'] != sum(s['measure_count'] for s in expected):
+      raise ValueError('invalid playback page timing')
+
+def validate_pages(document, pages):
+  times = boundaries(document)
+  cursor = 0
+  for page in pages:
+    if not isinstance(page, dict):
+      raise ValueError('invalid page descriptor')
+    count = page.get('measure_count')
+    if type(count) is not int or count <= 0 or page.get('first_measure') != cursor + 1 or cursor + count >= len(times):
+      raise ValueError('invalid page measure coverage')
+    if generator.model.fraction(page.get('start_beat')) != times[cursor] or generator.model.fraction(page.get('end_beat')) != times[cursor + count]:
+      raise ValueError('page timing does not match its measures')
+    cursor += count
+  if cursor != len(times) - 1:
+    raise ValueError('incomplete page measure coverage')
+
+def timing(document):
+  times = boundaries(document)
+  measures = document['staves'][0]['measures']
+  ending = [i for i, m in enumerate(measures) if m.get('ending_sustain')]
+  if ending and ending != list(range(ending[0], len(measures))):
+    raise ValueError('held ending must be a contiguous terminal section')
+  sustain_start = times[ending[0]] if ending else times[-1]
+  # The written held ending participates in page time, not metronome time.
+  clicks = []
+  for start, measure in zip(times, measures):
+    if start >= sustain_start:
+      break
+    duration = generator.model.fraction(measure['duration'])
+    beat = fractions.Fraction()
+    while beat < duration:
+      clicks.append(dict(beat=generator.model.rational(start + beat), downbeat=beat == 0))
+      beat += 1
+  return dict(song_id=document['metadata'].get('song_id', ''),
+    duration=generator.model.rational(times[-1]),
+    sustain_start=generator.model.rational(sustain_start), count_in_beats=4, clicks=clicks)

generator/pico_artwork.py

@@ -20,7 +20,7 @@ TITLE_HEADER = struct.pack('<4sHH', b'VPTI', TITLE_WIDTH, TITLE_HEIGHT)
 TITLE_BYTES = len(TITLE_HEADER) + TITLE_WIDTH * TITLE_HEIGHT // 2
 MAX_TITLES = 8192
 LABELS = ('VAMP', 'CONNECTING', 'OFFLINE', 'BUSY', 'READY', 'LOADING', 'WIFI', 'ERROR', 'PAGE', 'Acoustic', 'Electric', 'Vocal', 'UNAVAILABLE',
-  'Main Menu', 'Performance', 'Settings', 'Playback', 'Metronome', 'Tempo', 'Back', 'On', 'Off', 'BPM')
+  'Main Menu', 'Performance', 'Settings', 'Backing', 'Metronome', 'Solos', 'Tempo', 'Back', 'On', 'Off', 'BPM', 'Manual', 'Preparing', 'Count-in', 'Playing', 'Sustain', 'Hold to exit')
 SPRITES = LABELS + tuple('0123456789/')
 DIGIT_WIDTH = 22
 DEFAULT_FONT_ROOT = pathlib.Path(__file__).resolve().parents[1] / 'package/assets/fonts/petaluma'

generator/pitch.py

@@ -89,6 +89,6 @@ def concert_document(document: dict, *, octave_transposing: bool = False) -> dic
 
 def sounding_pitch(pitch: dict, state: dict):
   if 'unpitched' in pitch:
-    raise ValueError('unpitched notes have no fretted or generated_tone interpretation')
+    raise ValueError('unpitched notes have no fretted pitch interpretation')
   transpose = state['transpose']
   return generator.model.pitch_number(pitch) + transpose['chromatic'] + 12 * transpose['octave_change']

generator/playback.py

@@ -1,260 +0,0 @@
-from __future__ import annotations
-
-import array
-import fractions
-import hashlib
-import json
-import math
-import pathlib
-import sys
-import wave
-
-import generator.model
-import generator.pitch
-
-SAMPLE_RATE = 48000
-PROFILE = 'soft_struck'
-MAX_EVENTS = 1000000
-
-class PlaybackUnavailable(ValueError):
-  pass
-
-def sounding_pitch(tone: dict, state: dict) -> fractions.Fraction:
-  if 'unpitched' in tone:
-    raise PlaybackUnavailable('unpitched notation has no generated_tone interpretation')
-  value = generator.pitch.sounding_pitch(tone, state)
-  frequency = 440 * 2 ** (float(value - 69) / 12)
-  if not 20 <= frequency <= 18000:
-    raise PlaybackUnavailable(f'sounding pitch {value} is outside the generated_tone range (20-18000 Hz)')
-  return value
-
-def compile_timeline(document: dict) -> dict:
-  events, notes, linked = [], {}, {}
-  absolute = fractions.Fraction()
-  for measure in document['staves'][0]['measures']:
-    for voice in measure['voices']:
-      state = {key: measure[key] for key in ('transpose', 'clef', 'staff_details')}
-      changes = iter(sorted(measure.get('attributes', []), key=lambda a: generator.model.fraction(a['onset'])))
-      change = next(changes, None)
-      for note in voice['notes']:
-        onset = generator.model.fraction(note['onset'])
-        while change is not None and generator.model.fraction(change['onset']) <= onset:
-          state.update({key: change[key] for key in state if key in change})
-          change = next(changes, None)
-        notes[note['id']] = (note, absolute + onset, dict(state), voice['id'])
-    absolute += generator.model.fraction(measure['duration'])
-  adjustments = {}
-  insertions = {}
-  lanes = {}
-  for item in notes.values():
-    lanes.setdefault(item[3], []).append(item)
-  for lane in lanes.values():
-    pending = []
-    previous = None
-    for note, onset, state, voice in lane:
-      if note.get('grace'):
-        pending.append((note, onset, state, voice))
-        continue
-      if pending:
-        cursor = onset
-        specified = [n['grace'] for n, _, _, _ in pending]
-        modes = {key for g in specified for key in ('make-time', 'steal-time-previous', 'steal-time-following') if key in g}
-        if len(modes) > 1:
-          raise PlaybackUnavailable('grace group mixes conflicting timing modes')
-        mode = next(iter(modes), 'steal-time-following')
-        if mode == 'make-time':
-          lengths = [generator.model.fraction(g.get(mode, generator.model.rational(0))) for g in specified]
-          total = sum(lengths, fractions.Fraction())
-          insertions[onset] = max(insertions.get(onset, fractions.Fraction()), total)
-        else:
-          donor = previous[0] if mode == 'steal-time-previous' and previous else note if mode == 'steal-time-following' else None
-          if donor is None:
-            raise PlaybackUnavailable('grace notes have no preceding duration to steal')
-          percentages = [generator.model.fraction(g[mode]) if mode in g else fractions.Fraction(25, len(pending)) for g in specified]
-          if not 0 < sum(percentages) < 100:
-            raise PlaybackUnavailable('grace group must steal between zero and 100 percent')
-          lengths = [generator.model.fraction(donor['duration']) * amount / 100 for amount in percentages]
-          total = sum(lengths, fractions.Fraction())
-          delay, extension = adjustments.get(donor['id'], (fractions.Fraction(), fractions.Fraction()))
-          if mode == 'steal-time-previous':
-            cursor -= total
-          else:
-            delay += total
-          extension -= total
-          if generator.model.fraction(donor['duration']) + extension <= 0 or any(generator.model.fraction(p['duration']) + extension <= 0 for p in donor['pitches']):
-            raise PlaybackUnavailable(f'{donor["id"]}: grace timing consumes a donor or chord tone')
-          adjustments[donor['id']] = (delay, extension)
-        for item, length in zip(pending, lengths):
-          value, _, pitch_state, lane_id = item
-          adjustments[value['id']] = (cursor - item[1], length)
-          cursor += length
-        pending = []
-      previous = (note, onset)
-    if pending:
-      raise PlaybackUnavailable('trailing grace group has no following anchor')
-  for identifier, (note, onset, state, voice) in notes.items():
-    if note.get('cue'):
-      continue
-    delay, extension = adjustments.get(identifier, (fractions.Fraction(), fractions.Fraction()))
-    performed = onset + delay + sum((length for time, length in insertions.items() if time < onset or time == onset and not note.get('grace')), fractions.Fraction())
-    tremolo = note.get('tremolo')
-    if tremolo and tremolo['type'] == 'stop':
-      continue
-    for pitch in note['pitches']:
-      if pitch.get('cue'):
-        continue
-      length = generator.model.fraction(pitch['duration']) + extension
-      if not note.get('grace'):
-        length += sum((amount for time, amount in insertions.items() if onset < time < onset + generator.model.fraction(pitch['duration'])), fractions.Fraction())
-      if length <= 0:
-        raise PlaybackUnavailable(f'{identifier}: grace timing consumes a chord tone')
-      value = sounding_pitch(pitch, state)
-      if pitch.get('tie_from'):
-        previous = linked.get(pitch['tie_from'])
-        if previous is None or generator.model.fraction(previous['onset']) + generator.model.fraction(previous['duration']) != performed or generator.model.fraction(previous['pitch']) != value:
-          raise PlaybackUnavailable(f'{identifier}: tie has inconsistent performed timing or pitch')
-        previous['duration'] = generator.model.rational(generator.model.fraction(previous['duration']) + length)
-        linked[pitch['id']] = previous
-        continue
-      if tremolo and tremolo['marks']:
-        if pitch.get('tie_to'):
-          raise PlaybackUnavailable('tied measured tremolo is not yet qualified for playback')
-        written = pitch['written']
-        step = min(fractions.Fraction(1), generator.model.TYPE_LENGTHS[written['type']]) * generator.model.fraction(written.get('ratio', generator.model.rational(1))) / 2 ** tremolo['marks']
-        if tremolo['type'] == 'start':
-          partner = notes[tremolo['partner']]
-          if partner[1] != onset + generator.model.fraction(note['duration']) or len(partner[0]['pitches']) != len(note['pitches']):
-            raise PlaybackUnavailable('paired tremolo requires contiguous equal-size chords')
-          step *= 2
-          length += generator.model.fraction(partner[0]['pitches'][note['pitches'].index(pitch)]['duration'])
-        cursor, attack = fractions.Fraction(), 0
-        while cursor < length:
-          amount = value
-          if tremolo['type'] == 'start' and attack % 2:
-            other = partner[0]['pitches'][note['pitches'].index(pitch)]
-            amount = sounding_pitch(other, partner[2])
-          events.append({'id': f'{pitch["id"]}_{attack}', 'pitch': generator.model.rational(amount), 'onset': generator.model.rational(performed + cursor), 'duration': generator.model.rational(min(step, length - cursor))})
-          cursor += step
-          attack += 1
-          if len(events) > MAX_EVENTS:
-            raise PlaybackUnavailable('playback exceeds one million tone events')
-      else:
-        event = {'id': pitch['id'], 'pitch': generator.model.rational(value), 'onset': generator.model.rational(performed), 'duration': generator.model.rational(length)}
-        events.append(event)
-        linked[pitch['id']] = event
-  if len(events) > MAX_EVENTS:
-    raise PlaybackUnavailable('playback exceeds one million tone events')
-  events.sort(key=lambda e: (generator.model.fraction(e['onset']), e['id']))
-  return {'available': True, 'duration': generator.model.rational(absolute + sum(insertions.values(), fractions.Fraction())), 'events': events}
-
-def write_pcm(path: pathlib.Path, values) -> str:
-  samples = array.array('h', (round(max(-1, min(1, sample)) * 32767) for sample in values))
-  if sys.byteorder != 'little':
-    samples.byteswap()
-  path.parent.mkdir(parents=True, exist_ok=True)
-  with wave.open(str(path), 'wb') as output:
-    output.setparams((1, 2, SAMPLE_RATE, 0, 'NONE', 'not compressed'))
-    output.writeframes(samples.tobytes())
-  return hashlib.sha256(path.read_bytes()).hexdigest()
-
-def generate_tone(root: pathlib.Path, pitch: fractions.Fraction) -> dict:
-  frequency = 440 * 2 ** (float(pitch - 69) / 12)
-  period = fractions.Fraction(frequency / SAMPLE_RATE).limit_denominator(SAMPLE_RATE)
-  loop_length, cycles = period.denominator, period.numerator
-  attack = 12000
-  name = hashlib.sha256(f'{PROFILE}:{pitch}'.encode('ascii')).hexdigest()[:24]
-  relative = f'audio/{name}.wav'
-  harmonics = [h for h in range(1, 7) if frequency * h < SAMPLE_RATE * 0.45]
-
-  def sample(frame):
-    phase = 2 * math.pi * cycles * frame / loop_length
-    envelope = min(1, frame / 240) * (0.18 + 0.82 * (1 - min(1, frame / attack)) ** 3)
-    return 0.3 * envelope * sum(math.sin(phase * h) / (h * h) for h in harmonics)
-
-  checksum = write_pcm(root / relative, (sample(i) for i in range(attack + loop_length)))
-  return {'id': name, 'path': relative, 'sha256': checksum, 'sample_rate': SAMPLE_RATE, 'frames': attack + loop_length, 'loop_start': attack, 'loop_end': attack + loop_length}
-
-def compile_tree(root: pathlib.Path) -> None:
-  tones = {}
-  release = {'path': 'audio/release.wav', 'sample_rate': SAMPLE_RATE, 'frames': 2400}
-  release['sha256'] = write_pcm(root / release['path'], ((1 - i / 2399) ** 2 for i in range(2400)))
-  for path in sorted(root.rglob('*.json')):
-    document = json.loads(path.read_text(encoding='utf-8'))
-    try:
-      playback = compile_timeline(document)
-      used = {}
-      for event in playback['events']:
-        event['end'] = generator.model.rational(generator.model.fraction(event['onset']) + generator.model.fraction(event.pop('duration')))
-        pitch = generator.model.fraction(event.pop('pitch'))
-        if pitch not in tones:
-          tones[pitch] = generate_tone(root, pitch)
-        tone = tones[pitch]
-        event['sample'] = tone['id']
-        used[tone['id']] = tone
-      playback.update({'samples': list(used.values()), 'release': release, 'profile': PROFILE})
-    except PlaybackUnavailable as error:
-      playback = {'available': False, 'reason': str(error)}
-    document['playback'] = playback
-    path.write_text(json.dumps(document, indent=2, ensure_ascii=False) + '\n', encoding='utf-8')
-
-def validate_assets(playback: dict, root: pathlib.Path, cache: dict | None = None) -> None:
-  if not isinstance(playback, dict) or type(playback.get('available')) is not bool:
-    raise ValueError('playback manifest requires boolean availability')
-  if not playback['available']:
-    if not isinstance(playback.get('reason'), str) or not playback['reason']:
-      raise ValueError('unavailable playback must explain why')
-    return
-  cache = {} if cache is None else cache
-
-  def check_sample(value):
-    name = value['path']
-    relative = pathlib.Path(name)
-    if not name.startswith('audio/') or relative.is_absolute() or relative.suffix != '.wav' or any(p in ('', '.', '..') for p in name.split('/')) or '\\' in name or '\x00' in name:
-      raise ValueError('invalid generated audio path')
-    path = (root / relative).resolve()
-    if not path.is_relative_to(root.resolve()):
-      raise ValueError('generated audio escapes asset root')
-    key = (str(path), value['sha256'], value['frames'], value['sample_rate'])
-    if key in cache:
-      return cache[key]
-    if value['sample_rate'] != SAMPLE_RATE or not 0 < value['frames'] <= 96000:
-      raise ValueError('invalid generated audio rate/length')
-    if path.stat().st_size != 44 + value['frames'] * 2 or hashlib.sha256(path.read_bytes()).hexdigest() != value['sha256']:
-      raise ValueError('generated audio checksum mismatch')
-    with wave.open(str(path)) as audio:
-      pcm = audio.readframes(value['frames'])
-      if (audio.getnchannels(), audio.getsampwidth(), audio.getframerate(), audio.getnframes()) != (1, 2, SAMPLE_RATE, value['frames']) or len(pcm) != value['frames'] * 2:
-        raise ValueError('invalid or truncated generated PCM WAV')
-    samples = array.array('h')
-    samples.frombytes(pcm)
-    if sys.byteorder != 'little':
-      samples.byteswap()
-    if min(samples) < -32767:
-      raise ValueError('generated PCM exceeds unit range')
-    cache[key] = samples
-    return samples
-
-  duration = generator.model.fraction(playback['duration'])
-  if duration <= 0 or len(playback['events']) > MAX_EVENTS:
-    raise ValueError('invalid playback timeline length')
-  release = check_sample(playback['release'])
-  if not 1 < playback['release']['frames'] <= SAMPLE_RATE // 2:
-    raise ValueError('invalid release envelope length')
-  if release[0] != 32767 or release[-1] != 0 or any(a < b for a, b in zip(release, release[1:])):
-    raise ValueError('release envelope must decrease from one to zero')
-  if len(playback['samples']) > 512 or sum(s['frames'] for s in playback['samples']) > 16 * 1024 * 1024:
-    raise ValueError('generated playback exceeds sample memory budget')
-  samples = {}
-  for sample in playback['samples']:
-    if not sample['id'] or sample['id'] in samples or not 0 <= sample['loop_start'] < sample['loop_end'] == sample['frames']:
-      raise ValueError('invalid sample identity or loop')
-    check_sample(sample)
-    samples[sample['id']] = sample
-  previous = fractions.Fraction()
-  identities = set()
-  for event in playback['events']:
-    onset, end = generator.model.fraction(event['onset']), generator.model.fraction(event['end'])
-    if event['id'] in identities or event['sample'] not in samples or not previous <= onset < end <= duration:
-      raise ValueError('invalid playback event identity, reference or timing')
-    identities.add(event['id'])
-    previous = onset

generator/printing.py

@@ -0,0 +1,180 @@
+"""Build-only Letter PDFs from the complete arranged parts, never screen PNGs."""
+from __future__ import annotations
+
+import dataclasses
+import hashlib
+import json
+import math
+import pathlib
+import shutil
+import time
+
+import PIL.PdfParser
+
+import generator.engraving.audit
+import generator.engraving.cache
+import generator.engraving.fonts
+import generator.engraving.layout
+import generator.engraving.notation
+import generator.engraving.raster
+import generator.engraving.types
+import generator.validation
+
+# Two layout units per PDF point: guitar staff spacing is five points.
+# Raster resolution is independent of layout and the device's display theme.
+STYLE = generator.engraving.types.Style(width=1224, height=1584, margin_x=72, margin_y=72, page_clearance=28, system_gap=32, supersampling=1)
+DPI = 300
+PIXELS = (2550, 3300)
+MEDIA_BOX = [0, 0, 612, 792]
+VERSION = 'letter-parts-v1'
+
+def layout_part(document, fonts):
+  generator.validation.validate_document(document)
+  style = dataclasses.replace(generator.engraving.types.score_style(document, STYLE), system_gap=STYLE.system_gap)
+  # Measure numbers are centered over barlines and can extend left of the
+  # staff. Reserve their actual overhang inside the half-inch paper margin.
+  overhang = max(-fonts.text(str(m['number']), 0, 0, 12, 'measure_number', (), True).box.left for m in document['staves'][0]['measures'])
+  style = dataclasses.replace(style, margin_x=style.margin_x + math.ceil(max(0, overhang)) + 4)
+  index = generator.engraving.notation.ScoreIndex(document)
+  systems = generator.engraving.layout.layout_systems(document, style, fonts, index=index)
+  # The shared header wraps long credits; print it at twice its display size.
+  header_style = dataclasses.replace(style, width=style.width // 2, margin_x=style.margin_x / 2, margin_y=style.margin_y / 2)
+  header = tuple(p.scaled(2) for p in generator.engraving.layout.part_header(document, fonts, header_style))
+  top = generator.engraving.types.bounds(header).bottom + style.page_clearance
+  # Reserve the measured outline for every possible page number, just as
+  # screen pagination does. A nominal font size is not its actual ascent.
+  footers = tuple(fonts.text(str(number), style.width / 2, style.height - style.margin_y - 8, 20, 'page_number', (), True)
+    for number in range(1, len(systems) + 1))
+  bottom = min(p.box.top for p in footers) - style.page_clearance
+  pages, placed, cursor = [], [], top
+
+  def finish():
+    number = len(pages) + 1
+    pages.append(generator.engraving.types.Page(number, style.width, style.height, style.supersampling, tuple(placed),
+      (*header, *(p for system in placed for p in system.primitives), footers[number - 1])))
+
+  for system in systems:
+    system = generator.engraving.layout.make_system(index, system.start, system.end, fonts, style, justify=system.end < index.end, staff_width=style.width - 2 * style.margin_x - 2)
+    height = system.box.bottom - system.box.top
+    if height > bottom - top:
+      raise generator.engraving.types.EngravingError(f'at musical offset {system.start}: system cannot fit a Letter page at readable size')
+    if placed and cursor + height > bottom:
+      finish()
+      placed, cursor = [], top
+    dx, dy = style.margin_x + 1, cursor - system.box.top
+    placed.append(dataclasses.replace(system, primitives=tuple(p.moved(dx, dy) for p in system.primitives),
+      anchors=tuple(a.moved(dx, dy) for a in system.anchors), box=system.box.moved(dx, dy)))
+    cursor += height + style.system_gap
+  finish()
+  result = generator.engraving.types.LayoutDocument(tuple(pages), style, fonts.fingerprint(style))
+  generator.engraving.audit.audit_layout(document, result, fonts, systems_per_page=None)
+  if any(p.box.left < STYLE.margin_x or p.box.right > style.width - STYLE.margin_x or
+    p.box.top < STYLE.margin_y or p.box.bottom > style.height - STYLE.margin_y for page in pages for p in page.primitives):
+    raise generator.engraving.types.EngravingError('print notation exceeds the half-inch printer margins')
+  return result
+
+def rasterize(page, fonts):
+  return generator.engraving.raster.rasterize_page(page, fonts, ink=(0, 0, 0), background=(255, 255, 255), output_scale=DPI / 144)
+
+def validate_pdf(path, expected_pages=None):
+  """Check the narrow PDF format written by the pinned Pillow asset dependency.
+
+  This adapter deliberately uses Pillow's parser, not a new general PDF stack.
+  Tests also decode the embedded page images to check print color/resolution.
+  """
+  try:
+    if path.is_symlink() or not path.is_file():
+      raise ValueError('expected a regular print PDF')
+    pdf = PIL.PdfParser.PdfParser(filename=str(path))
+    try:
+      count = len(pdf.pages)
+      if not 0 < count <= 10000 or (expected_pages is not None and count != expected_pages):
+        raise ValueError('invalid print PDF page count')
+      for reference in pdf.pages:
+        page = pdf.read_indirect(reference)
+        if page[b'MediaBox'] != MEDIA_BOX:
+          raise ValueError('print PDF must use portrait US Letter')
+        image = pdf.read_indirect(page[b'Resources'][b'XObject'][b'image'])
+        if (image.dictionary[b'Width'], image.dictionary[b'Height']) != PIXELS or image.dictionary[b'ColorSpace'] != b'DeviceGray':
+          raise ValueError('print PDF must contain 300 DPI grayscale pages')
+      return count
+    finally:
+      pdf.close()
+  except (OSError, KeyError, TypeError, IndexError, PIL.PdfParser.PdfFormatError) as error:
+    raise ValueError(f'{path.name}: invalid print PDF: {error}') from error
+
+def write_pdf(path, document, fonts):
+  layout = layout_part(document, fonts)
+  # Append one page at a time so a long arrangement never keeps a chorus of
+  # full-resolution raster images in memory. The enclosing build is staged.
+  for page in layout.pages:
+    with rasterize(page, fonts) as rgb, rgb.convert('L') as image:
+      if image.size != PIXELS or image.getextrema()[0] == 255:
+        raise ValueError('print renderer produced an empty or incorrectly sized page')
+      image.save(path, format='PDF', append=page.number > 1, resolution=DPI, quality=95,
+        title=document['metadata']['title'] + ' - ' + document['staves'][0].get('name', 'Part'),
+        creator='Product Vamp', creationDate=None, modDate=None)
+  validate_pdf(path, len(layout.pages))
+  return len(layout.pages)
+
+def renderer_key(fonts):
+  identity = generator.engraving.cache.renderer_identity(fonts, STYLE)
+  identity.update(print_version=VERSION, print_code=hashlib.sha256(pathlib.Path(__file__).read_bytes()).hexdigest(), dpi=DPI, ink='black', background='white', quality=95)
+  return hashlib.sha256(json.dumps(identity, sort_keys=True).encode()).hexdigest()
+
+def restore(entry, key, target):
+  try:
+    if generator.cache.has_symlink(entry):
+      return None
+    pdf, manifest = entry / 'part.pdf', entry / 'print.json'
+    if pdf.is_symlink() or manifest.is_symlink():
+      return None
+    info = json.loads(manifest.read_text())
+    if info['key'] != key or info['sha256'] != hashlib.sha256(pdf.read_bytes()).hexdigest():
+      return None
+    pages = validate_pdf(pdf, info['pages'])
+    shutil.copyfile(pdf, target)
+    return pages
+  except (OSError, ValueError, KeyError, TypeError):
+    return None
+
+def store(entry, key, target, pages):
+  with generator.cache.stage_entry(entry, '.print_') as staged:
+    shutil.copyfile(target, staged / 'part.pdf')
+    (staged / 'print.json').write_text(json.dumps({'key': key, 'sha256': hashlib.sha256(target.read_bytes()).hexdigest(), 'pages': pages}))
+    generator.cache.publish_files(entry, staged, ('part.pdf', 'print.json'))
+
+def generate_tree(root, font_path, *, cache_dir=None, progress=False):
+  """Called inside the normal build's atomic staging directory."""
+  if cache_dir and (cache_dir.resolve().is_relative_to(root.resolve()) or root.resolve().is_relative_to(cache_dir.resolve())):
+    raise ValueError('print cache and generated assets must not overlap')
+  destination = root / 'print'
+  destination.mkdir(exist_ok=True)
+  paths = sorted((root / 'songs').glob('*.json'))
+  fonts = generator.engraving.fonts.Fonts(font_path)
+  renderer = renderer_key(fonts)
+  expected, pages_total, reused = set(), 0, 0
+  started = time.monotonic()
+  for path in paths:
+    bundle = json.loads(path.read_text())
+    document = bundle.get('music', bundle)
+    target = destination / path.with_suffix('.pdf').name
+    expected.add(target)
+    key = generator.engraving.cache.score_fingerprint(document, renderer)
+    entry = cache_dir / 'print' / renderer / key if cache_dir else None
+    pages = restore(entry, key, target) if entry else None
+    if pages is None:
+      pages = write_pdf(target, document, fonts)
+      if entry:
+        store(entry, key, target, pages)
+    else:
+      reused += 1
+    pages_total += pages
+    if progress:
+      print(f'Print: {target.name}: {pages} pages', flush=True)
+  for stale in destination.glob('*.pdf'):
+    if stale not in expected:
+      stale.unlink()
+  if progress:
+    print(f'Print complete: {len(paths)} PDFs, {pages_total} pages, {reused} cached; {time.monotonic() - started:.1f}s', flush=True)
+  return len(paths)

generator/production.py

@@ -1,313 +1,75 @@
-"""Independent, reproducible role generation and non-composing assembly.
+"""File discovery, process scheduling and publication of independent roles."""
-
-Only source documents and production resources enter generate_role. Completed
-roles and published assets are downstream values, never composition inputs.
-"""
 from __future__ import annotations
 import concurrent.futures
 import copy
-import dataclasses
-import hashlib
-import functools
 import json
 import multiprocessing
-import os
+import pathlib
-from pathlib import Path
-import tempfile
-import time
-from fractions import Fraction as F
-
-from generator import accompaniment, comping, fretboard, model, musicxml, parts, score, vocabulary
-from generator.arrange import form, generate, pickup
-from generator.solo import analysis, composer, notation, report, learned
-from generator.solo import authored as repertoire
-
-VERSION = 'independent-roles-1'
-ROLES = ('acoustic_accompaniment', 'electric_accompaniment', 'electric_melody', 'vocal_melody', 'acoustic_solo', 'electric_solo')
-
-
-@dataclasses.dataclass(frozen=True)
-class SourceAnalysis:
-  document_json: str
-  musical_key: str
-
-  @classmethod
-  def read(cls, document):
-    prepared = copy.deepcopy(document)
-    staff = score.source_staff(prepared)
-    for i, m in enumerate(staff['measures']):
-      m['id'], m['number'] = f'source_{i}', str(i+1)
-    staff['tuplets'] = score.reidentify(staff['measures'], staff.get('tuplets', []))
-    prepared['metadata'] = {k:v for k,v in prepared['metadata'].items() if k not in ('title', 'composer', 'copyright', 'software')}
-    prepared['metadata'].update(title='Source', composer='')
-    staff.update(id='source', name='Source')
-    melodic = analysis.analyze(prepared)
-    content = {'melody': [(str(n.start), str(n.end), dataclasses.asdict(n.pitch)) for n in melodic.melody],
-      'measures': [{k: m.get(k) for k in ('duration', 'time_signature', 'key', 'harmonies', 'tempos', 'section_label', 'implicit')} for m in staff['measures']]}
-    # Fraction/string spelling is canonical and never includes filesystem IDs.
-    key = hashlib.sha256(json.dumps(content, sort_keys=True, default=str).encode()).hexdigest()
-    return cls(json.dumps(prepared, sort_keys=True), key)
-
-  def document(self):
-    return json.loads(self.document_json)
-
-
-def windows(document, part):
-  source = score.source_staff(document)['measures']
-  sections = form.full_arrangement_sections(source, part)
-  result, cursor = [], F()
-  for section in sections:
-    duration = sum((model.fraction(m['duration']) for m in source[section.source_measure_index:section.source_measure_index+section.measure_count]), F())
-    role = {'slash': 'solo', 'vocal': 'melody'}.get(section.role, section.role)
-    if section.role in ('vocal', 'melody') and pickup.enters(section, source) and result:
-      attacks = [model.fraction(n['onset']) for n in score.measure_notes(source[0]) if n['pitches']]
-      if attacks:
-        start = cursor-model.fraction(source[0]['duration'])+min(attacks)
-        a, _, old = result[-1]; result[-1] = (a, start, old)
-        result.append((start, cursor, part+'_'+role))
-    end = cursor+duration
-    start=cursor
-    if section.role == 'slash' and score.body_start_measure_index(source) and result:
-      # Allocate the source's complete pickup bar before composition. This
-      # does not inspect either generated solo; a role may leave it silent.
-      start=cursor-model.fraction(source[0]['duration'])
-      a,_,old=result[-1];result[-1]=(a,start,old)
-    result.append((start, end, part+'_'+role))
-    cursor = end
-  return tuple(result)
-
-
-def dependencies(role, resources):
-  names = ('harmony', 'lead_ins') if role.endswith('_solo') else ('accompaniment',) if role.endswith('_accompaniment') else ()
-  return {'roles': {'role': resources['roles']['roles'][role], 'difficulty': resources['roles']['difficulty']},
-    **{name: resources[name] for name in names}}
-
-
-def _code_key(role):
-  root = Path(__file__).parent
-  paths = [root / n for n in ('production.py', 'model.py', 'harmony.py', 'pitch.py', 'fretboard.py', 'parts.py', 'score.py', 'vocabulary.py')]
-  paths += list((root/'arrange').glob('*.py'))
-  if role.endswith('_solo'):
-    paths += list((root/'solo').glob('*.py'))
-    paths += [root/'harmonic_time.py', root/'musicxml.py',root/'training_io.py']
-  elif role.endswith('_accompaniment'):
-    paths += [root/n for n in ('comping.py', 'accompaniment.py', 'guitar.py')]
-  elif role == 'electric_melody':
-    paths += [root/'guitar.py', root/'melody.py']
-  return vocabulary.fingerprint([(p.relative_to(root).as_posix(), hashlib.sha256(p.read_bytes()).hexdigest()) for p in sorted(set(paths))])
-
-
-def _resource_key(directory, role):
-  return {'model': hashlib.sha256((directory/'models'/'solo.json').read_bytes()).hexdigest()} if role.endswith('_solo') else {}
-
-@functools.lru_cache(maxsize=4)
-def _solo_model(directory, fingerprint):
-  return learned.Model.load(Path(directory)/'models'/'solo.json')
-
-
-def _mask(document, allowed):
-  """Apply source-defined role exclusions, without observing another part."""
-  absolute = F()
-  for measure in document['staves'][0]['measures']:
-    length = model.fraction(measure['duration'])
-    for voice in measure['voices']:
-      selected = []
-      for n in voice['notes']:
-        start = absolute+model.fraction(n['onset'])
-        end = start+model.fraction(n['duration'])
-        intervals = [(max(start,a),min(end,b)) for a,b in allowed if a < end and b > start]
-        for a,b in intervals:
-          note = copy.deepcopy(n)
-          if a != start:
-            note['onset'] = model.rational(a-absolute)
-            for pitch in note['pitches']: pitch.pop('tie_from',None)
-          pickup.truncate(note,b-absolute)
-          if a < b: selected.append(note)
-      voice['notes'] = _fill(selected, length, measure['id']+'_'+voice['id'], voice['id'])
-    absolute += length
-  staff=document['staves'][0]
-  staff['tuplets']=pickup.retain_links(staff['measures'],staff.get('tuplets',[]))
-
-
-def _fill(notes, length, prefix, voice='voice_arranged'):
-  notes=sorted(notes,key=lambda n:model.fraction(n['onset']))
-  result=[]; cursor=F()
-  def rests(end):
-    nonlocal cursor
-    for duration,written in accompaniment.written_durations(end-cursor):
-      result.append(model.make_event(prefix+'_rest_'+str(len(result)),cursor,duration,'rest',written=written)|{'voice':voice})
-      cursor+=duration
-  for n in notes:
-    start=model.fraction(n['onset'])
-    if start<cursor: raise ValueError(prefix+': overlapping independently generated roles')
-    rests(start); n['voice']=voice;result.append(n)
-    cursor=start+model.fraction(n['duration'])
-  rests(length)
-  return result
-
-
-def generate_role(source, role, data_dir, settings=None, cache_dir=None):
-  from generator.input_audit import composition
-  started=time.monotonic()
-  with composition(data_dir,role,cache_dir) as reads:
-    result=_generate_role(source,role,data_dir,settings,cache_dir)
-  # Observed reads are diagnostics, never cache identity or musical seeds.
-  result['trace']={**result['trace'],'input_boundary':'production data (including explicit authored repertoire), generator code, Python runtime, own role cache'}
-  result['observed_reads']=sorted(reads)
-  result['seconds']=time.monotonic()-started
-  return result
-
-
-def _generate_role(source, role, data_dir, settings=None, cache_dir=None):
-  if role not in ROLES: raise ValueError('Unknown generation role: '+role)
-  data_dir=Path(data_dir); resources=vocabulary.load(data_dir/'composition')
-  settings=settings or {}
-  registered=repertoire.lookup(source,role.removesuffix('_solo'),data_dir) if role.endswith('_solo') and not settings.get('predict',False) else None
-  algorithm=VERSION+'/'+learned.VERSION if role.endswith('_solo') else VERSION
-  inputs={'version':algorithm,'source':source.musical_key,'role':role,'settings':settings,
-    'vocabulary':dependencies(role,resources),'catalogs':({'authored':registered['sha256'],'id':registered['id'],'pitch_convention':registered['pitch_convention']} if registered else _resource_key(data_dir,role)),'code':_code_key(role)}
-  # Source notation/lyrics also invalidate cached score representation, never the musical seed.
-  key=vocabulary.fingerprint({'inputs':inputs,'notation':source.document_json})
-  path=Path(cache_dir)/role/(key+'.json') if cache_dir else None
-  if path and path.is_file() and not path.is_symlink():
-    try:
-      entry=json.loads(path.read_text()); value=entry['result']
-      if entry['key']==key and entry['digest']==vocabulary.fingerprint(value):
-        model.validate_document(value['document']); fretboard.validate_document(value['document'])
-        return {**value,'cache_hit':True}
-    except (ValueError,KeyError,TypeError): pass
-  started=time.monotonic(); document=source.document(); part=role.split('_')[0]
-  trace={'source':source.musical_key,'role':role,'production_resources':inputs['catalogs'],
-    'vocabulary':vocabulary.fingerprint(inputs['vocabulary']),'settings':settings,'algorithm':algorithm,'code':inputs['code'],'cache_key':key}
-  seed=vocabulary.fingerprint({'source':source.musical_key,'role':role,'settings':settings,'data':inputs['vocabulary'],'version':algorithm})
-  details={}; solo_report=''; composition_timings={}
-  with vocabulary.using(resources,role,settings|{'choice_seed':seed}):
-    if role.endswith('_solo'):
-      result=generate._arrange_document(document,part,{},owned={'slash'})
-      from generator.solo.arrangement import regions
-      si=3 if part=='acoustic' else 4
-      first,count,offset,length=regions(document)[si]
-      # The template/harmony comes directly from source form, not another generated part.
-      authored=score.source_staff(document)['measures']
-      section=form.full_arrangement_sections(authored,part)[si]
-      effective=generate.section_source(authored,section,form.full_arrangement_sections(authored,part)[si+1])
-      template=copy.deepcopy(effective[section.source_measure_index:section.source_measure_index+count])
-      initial=accompaniment.carried_chords(effective)[section.source_measure_index]
-      start,end=next((a-offset,b-offset) for a,b,r in windows(document,part) if r==role)
-      notation_first=first
-      if start<0:
-        # The preceding arrangement harmony comes from the source form. Do
-        # not invent harmony from a comparison score's unlabelled lead-in.
-        template.insert(0,copy.deepcopy(result['staves'][0]['measures'][first-1]))
-        notation_first-=1
-        initial=accompaniment.carried_chords(result['staves'][0]['measures'])[first-1]
-      if registered:
-        solo=repertoire.realize(registered,analysis.analyze(document),template,part,start,end,initial)
-      else:
-        solo_model=_solo_model(str(data_dir),inputs['catalogs']['model'])
-        solo=composer.compose(analysis.analyze(document),template,part,end,
-          initial=initial,learned_model=solo_model,
-          expressions_enabled=vocabulary.role().get('expressions',True),start=start)
-      composition_timings={k:v for k,v in solo.diagnostics.items() if k.endswith('_seconds')}
-      relative=tuple(dataclasses.replace(n,start=n.start-start,end=n.end-start) for n in solo.notes())
-      output=notation.measures(result['staves'][0]['measures'][notation_first:first+count],relative,'solo_'+part)
-      notation.replace_region(result,notation_first,output)
-      details=report.structure(solo)
-      details['allocation']={'start':str(start),'end':str(end),'lead_in_measures':first-notation_first}
-      solo_report=report.describe(solo,first,offset,detail=details)
-    elif role.endswith('_accompaniment'):
-      canonical=comping.Canonical(document,part,seed)
-      result=generate._arrange_document(document,part,{},owned={'accompaniment'},accompaniment=canonical)
-      details={'canonical_comping':canonical.report}
-    else:
-      result=generate._arrange_document(document,part,{},owned={'vocal' if part=='vocal' else 'melody'})
-  allowed=tuple((a,b) for a,b,r in windows(document,part) if r==role)
-  _mask(result,allowed)
-  staff=result['staves'][0]
-  for measure in staff['measures']:
-    measure['id']=role+'_'+measure['id']
-  staff['tuplets']=score.reidentify(staff['measures'],staff.get('tuplets',[]))
-  if part!='vocal':result['staves'][0]['tablature']=True
-  model.validate_document(result);fretboard.validate_document(result)
-  value={'role':role,'document':result,'trace':trace,'details':details,'report':solo_report,'seconds':time.monotonic()-started,'composition_timings':composition_timings,'cache_hit':False}
-  # Sidecar diagnostics use the same rational-safe representation as reports.
-  # Identical insertion order after cold generation and cache loading also
-  # keeps serialized scores and the catalog byte-for-byte reproducible.
-  value=json.loads(json.dumps(value,default=str,sort_keys=True))
-  if path:
-    path.parent.mkdir(parents=True,exist_ok=True)
-    payload={'key':key,'digest':vocabulary.fingerprint(value),'result':value}
-    with tempfile.NamedTemporaryFile('w',dir=path.parent,delete=False) as out:
-      json.dump(payload,out,sort_keys=True);temporary=out.name
-    os.replace(temporary,path)
-  return value
-
-
-def assemble(source, results):
-  """Union prevalidated role windows; no selection or pitch/fingering rewrite."""
-  documents={}
-  for part in parts.PART_TYPES:
-    selected=[results[r]['document'] for r in ROLES if r.startswith(part+'_')]
-    document=copy.deepcopy(selected[0]); staff=document['staves'][0]
-    if part=='vocal':
-      documents[part]=document;continue
-    staff['tuplets']=[];staff['octave_shifts']=[];staff['scale_spans']=[]
-    for other in selected:
-      for field in ('tuplets','octave_shifts','scale_spans'):
-        staff[field].extend(copy.deepcopy(other['staves'][0].get(field,[])))
-    for i,m in enumerate(staff['measures']):
-      # Tuplet rests belong to the source/role rhythm and must survive union.
-      sounding=[copy.deepcopy(n) for d in selected for n in score.measure_notes(d['staves'][0]['measures'][i]) if n['pitches'] or n.get('tuplet_ids')]
-      m['voices']=[{'id':'voice_arranged','notes':_fill(sounding,model.fraction(m['duration']),m['id'])}]
-      # Attribute changes belong to source melody pickups and are assembly metadata.
-      for d in selected:
-        other=d['staves'][0]['measures'][i]
-        if other.get('attributes'):m['attributes']=copy.deepcopy(other['attributes'])
-    staff['tuplets']=pickup.retain_links(staff['measures'],staff['tuplets'])
-    for span in staff['tuplets']:span['voice']='voice_arranged'
-    model.validate_document(document);fretboard.validate_document(document)
-    documents[part]=document
-  return documents
 
+import generator.arrange.accompaniment
+import generator.arrange.assembly
+import generator.composition
+import generator.composition_types
+import generator.musicxml
+import generator.parts
+import generator.score
+import generator.score_io
+import generator.source
 
 def _job(request):
-  source,role,directory,settings,cache=request
+  return generator.composition.generate_role(request.source, request.role, request.directory, json.loads(request.settings_json), request.cache)
-  return generate_role(source,role,directory,settings,cache)
-
 
 def generate_tree(source_dir, output_dir, *, jobs=2, cache_dir=None, progress=False, timings=None):
-  if type(jobs) is not int or not 1<=jobs<=16:
+  if type(jobs) is not int or not 1 <= jobs <= 16:
     raise ValueError('composition workers must be between 1 and 16')
-  source_dir,output_dir=Path(source_dir),Path(output_dir)
+  source_dir, output_dir = pathlib.Path(source_dir), pathlib.Path(output_dir)
-  sources={}
+  sources = {}
-  for path in parts.source_paths(source_dir):
+  for path in generator.parts.source_paths(source_dir):
-    document=parts.prepare_document(musicxml.load_musicxml(path))
+    document = generator.parts.prepare_document(generator.musicxml.load_musicxml(path))
-    sources[path]=(document,SourceAnalysis.read(document))
+    sources[path] = (document, generator.source.SourceAnalysis.read(document))
-  requests=[(path,role,(s,role,source_dir.parent,{},cache_dir)) for path,(_,s) in sources.items() for role in ROLES]
+  requests = [(path, role, generator.composition_types.RoleRequest(s, role, source_dir.parent, cache=cache_dir)) for path, (_, s) in sources.items() for role in generator.composition_types.ROLES]
-  results={p:{} for p in sources}
+  results = {p: {} for p in sources}
-  def accept(path,role,value):
+
-    results[path][role]=value
+  def accept(path, role, value):
-    if timings is not None:timings.append({'source':str(path),'role':role,'seconds':value['seconds'],'cache_hit':value['cache_hit'],'composition_timings':value.get('composition_timings',{}),'observed_reads':value.get('observed_reads',[])})
+    results[path][role] = value
-    if progress:print(f'Composition: {path.stem}/{role}: '+('cached' if value['cache_hit'] else f'{value["seconds"]:.1f}s'),flush=True)
+    if timings is not None:
-  if jobs==1:
+      timings.append({'source': str(path), 'role': role, 'seconds': value['seconds'], 'cache_hit': value['cache_hit'], 'composition_timings': value.get('composition_timings', {}), 'observed_reads': value.get('observed_reads', [])})
-    for path,role,args in requests:
+    if progress:
-      try:accept(path,role,_job(args))
+      print(f'Composition: {path.stem}/{role}: ' + ('cached' if value['cache_hit'] else f'{value["seconds"]:.1f}s'), flush=True)
-      except Exception as error:raise ValueError(f'{path.name}/{role}: {error}') from error
+
+  if jobs == 1:
+    for path, role, args in requests:
+      try:
+        accept(path, role, _job(args))
+      except Exception as error:
+        raise ValueError(f'{path.name}/{role}: {error}') from error
   else:
-    with concurrent.futures.ProcessPoolExecutor(max_workers=jobs,mp_context=multiprocessing.get_context('spawn')) as pool:
+    with concurrent.futures.ProcessPoolExecutor(max_workers=jobs, mp_context=multiprocessing.get_context('spawn')) as pool:
-      futures={pool.submit(_job,args):(path,role) for path,role,args in requests}
+      futures = {pool.submit(_job, args): (path, role) for path, role, args in requests}
       for future in concurrent.futures.as_completed(futures):
-        path,role=futures[future]
+        path, role = futures[future]
-        try:accept(path,role,future.result())
+        try:
-        except Exception as error:raise ValueError(f'{path.name}/{role}: {error}') from error
+          accept(path, role, future.result())
-  for path,(original,source) in sources.items():
+        except Exception as error:
-    assembled=assemble(source,results[path])
+          raise ValueError(f'{path.name}/{role}: {error}') from error
-    for part,document in assembled.items():
+  for path, (original, source) in sources.items():
-      document['metadata']=copy.deepcopy(original['metadata'])
+    reuse = generator.arrange.accompaniment.audit(original, {role: value['document'] for role, value in results[path].items()}, role_documents=True)
-      score.write_json(output_dir/(score.slugify(path.stem)+'_'+part+'.json'),document)
+    for solos_enabled in (True, False):
-    analysis_dir=output_dir.parent/'analysis';analysis_dir.mkdir(parents=True,exist_ok=True)
+      assembled = generator.arrange.assembly.assemble(source, results[path], solos_enabled=solos_enabled)
-    (analysis_dir/(score.slugify(path.stem)+'.txt')).write_text('\n'.join(results[path][r]['report'] for r in ROLES))
+      generator.arrange.accompaniment.audit(original, assembled)
+      for part, document in assembled.items():
+        if part == 'vocal' and not solos_enabled:
+          continue
+        document['metadata'] = copy.deepcopy(original['metadata'])
+        document['metadata'].update(song_id=generator.score.slugify(path.stem), solos_enabled=solos_enabled)
+        suffix = '' if solos_enabled else '_improv'
+        generator.score_io.write_json(output_dir / (generator.score.slugify(path.stem) + '_' + part + suffix + '.json'), document)
+    analysis_dir = output_dir.parent / 'analysis'
+    analysis_dir.mkdir(parents=True, exist_ok=True)
+    (analysis_dir / (generator.score.slugify(path.stem) + '.txt')).write_text('\n'.join(results[path][r]['report'] for r in generator.composition_types.ROLES))
     # Not a score file: diagnostics are deliberately outside the songs schema.
-    stable={role:{k:v for k,v in value.items() if k not in ('seconds','composition_timings','cache_hit','observed_reads')} for role,value in results[path].items()}
+    stable = {role: {k: v for k, v in value.items() if k not in ('seconds', 'composition_timings', 'cache_hit', 'observed_reads')} for role, value in results[path].items()}
-    (analysis_dir/(score.slugify(path.stem)+'.roles')).write_text(json.dumps(stable,sort_keys=True,indent=2))
+    for role, audit in reuse.items():
-  return len(sources)*3
+      stable[role]['details']['reuse_audit'] = audit
+    (analysis_dir / (generator.score.slugify(path.stem) + '.roles')).write_text(json.dumps(stable, sort_keys=True, indent=2))
+  return len(sources) * 5

generator/role_cache.py

@@ -0,0 +1,29 @@
+"""Role cache persistence; cache identity never supplies musical randomness."""
+import json
+
+import generator.cache
+import generator.fretboard
+import generator.validation
+
+ENTRYPOINTS = ('generator.composition',)
+
+def code_key():
+  return generator.cache.fingerprint(generator.cache.source_identity(ENTRYPOINTS))
+
+def restore(path, key):
+  if path is None or not path.is_file() or path.is_symlink():
+    return None
+  try:
+    entry = json.loads(path.read_text())
+    value = entry['result']
+    if entry['key'] == key and entry['digest'] == generator.cache.fingerprint(value):
+      generator.validation.validate_document(value['document'])
+      generator.fretboard.validate_document(value['document'])
+      return {**value, 'cache_hit': True}
+  except (OSError, ValueError, KeyError, TypeError):
+    pass
+  return None
+
+def store(path, key, value):
+  if path is not None:
+    generator.cache.write_json(path, {'key': key, 'digest': generator.cache.fingerprint(value), 'result': value})

generator/score.py

@@ -1,50 +1,8 @@
+"""Pure score navigation, cloning and identity operations."""
 from __future__ import annotations
-
 import copy
-import json
-import pathlib
 import re
 
-import generator.engraving.validation
-import generator.fretboard
-import generator.model
-import generator.pico_artwork
-import generator.playback
-
-def load_json(path: pathlib.Path) -> dict:
-  document = json.loads(path.read_text(encoding='utf-8'))
-  return validate_json(document, path)
-
-def validate_json(document: dict, path: pathlib.Path, validated_pages: set[str] | None = None) -> dict:
-  if not isinstance(document, dict):
-    raise ValueError('expected a score object')
-  if 'music' in document:
-    presentation = document.get('presentation', {})
-    if not isinstance(document['music'], dict) or not isinstance(presentation, dict) or not isinstance(presentation.get('pages'), list) or not presentation['pages']:
-      raise ValueError('invalid score bundle; regenerate score assets')
-    if (presentation.get('engine'), presentation.get('width'), presentation.get('height')) != (generator.engraving.validation.ENGINE_NAME, generator.engraving.validation.PAGE_WIDTH, generator.engraving.validation.PAGE_HEIGHT):
-      raise ValueError('unsupported score presentation; regenerate score assets')
-    for page in presentation['pages']:
-      if not isinstance(page, dict) or not isinstance(page.get('path'), str) or not isinstance(page.get('sha256'), str):
-        raise ValueError('invalid score page manifest')
-      name = page['path']
-      relative = pathlib.Path(name)
-      if '\x00' in name or '\\' in name or relative.is_absolute() or any(part in ('', '.', '..') for part in name.split('/')) or relative.suffix != '.png':
-        raise ValueError('invalid score page path')
-      resolved = (path.parent / relative).resolve()
-      if not resolved.is_relative_to(path.parent.resolve()):
-        raise ValueError('score page escapes its asset directory')
-      generator.engraving.validation.check_page(resolved, validated_pages, expected_sha256=page['sha256'])
-    document = document['music']
-  generator.model.validate_document(document)
-  return document
-
-def write_json(path: pathlib.Path, document: dict) -> None:
-  generator.model.validate_document(document)
-  generator.fretboard.validate_document(document)
-  path.parent.mkdir(parents=True, exist_ok=True)
-  path.write_text(json.dumps(document, indent=2, ensure_ascii=False) + '\n', encoding='utf-8')
-
 def slugify(value: str) -> str:
   return re.sub(r'[^a-z0-9]+', '_', value.strip().lower()).strip('_') or 'score'
 
@@ -121,18 +79,3 @@ def reidentify(measures: list[dict], source_tuplets: list[dict]) -> list[dict]:
 
 def make_arranged_document(source_document: dict, staff: dict, *, staff_id: str, staff_name: str, role: str, arrangement_part_type: str, measures: list[dict], tuplets=None, octave_shifts=None) -> dict:
   return {'metadata': copy.deepcopy(source_document['metadata']), 'staves': [{'id': staff_id, 'name': staff_name, 'clef': copy.deepcopy(staff['clef']), 'role': role, 'arranged': True, 'arrangement_part_type': arrangement_part_type, 'measures': measures, 'tuplets': tuplets or [], 'octave_shifts': copy.deepcopy(octave_shifts or []), 'instruments': copy.deepcopy(staff.get('instruments', []))}]}
-
-def count_json_files(root: pathlib.Path) -> int:
-  return sum(1 for p in root.rglob('*.json') if p.is_file())
-
-def validate_generated_tree(root: pathlib.Path, validated_pages: set[str] | None = None) -> None:
-  audio_cache = {}
-  validated_pages = set() if validated_pages is None else validated_pages
-  for path in root.rglob('*.json'):
-    document = json.loads(path.read_text(encoding='utf-8'))
-    music = validate_json(document, path, validated_pages)
-    generator.fretboard.validate_document(music)
-    if 'presentation' in document:
-      generator.pico_artwork.validate_title(root, document['presentation'].get('title_image'))
-    if 'playback' in document:
-      generator.playback.validate_assets(document['playback'], root, audio_cache)

generator/score_io.py

@@ -0,0 +1,59 @@
+from __future__ import annotations
+
+import json
+import pathlib
+
+import generator.engraving.validation
+import generator.fretboard
+import generator.performance
+import generator.pico_artwork
+import generator.validation
+
+def load_json(path: pathlib.Path) -> dict:
+  document = json.loads(path.read_text(encoding='utf-8'))
+  return validate_json(document, path)
+
+def validate_json(document: dict, path: pathlib.Path, validated_pages: set[str] | None = None) -> dict:
+  if not isinstance(document, dict):
+    raise ValueError('expected a score object')
+  if 'music' in document:
+    presentation = document.get('presentation', {})
+    if not isinstance(document['music'], dict) or not isinstance(presentation, dict) or not isinstance(presentation.get('pages'), list) or not presentation['pages']:
+      raise ValueError('invalid score bundle; regenerate score assets')
+    if (presentation.get('engine'), presentation.get('width'), presentation.get('height')) != (generator.engraving.validation.ENGINE_NAME, generator.engraving.validation.PAGE_WIDTH, generator.engraving.validation.PAGE_HEIGHT):
+      raise ValueError('unsupported score presentation; regenerate score assets')
+    generator.performance.validate_views(document['music'], presentation['pages'], presentation.get('playback_pages'))
+    if document.get('performance') != generator.performance.timing(document['music']):
+      raise ValueError('invalid performance timing')
+    for page in presentation['pages'] + presentation['playback_pages']:
+      if not isinstance(page, dict) or not isinstance(page.get('path'), str) or not isinstance(page.get('sha256'), str):
+        raise ValueError('invalid score page manifest')
+      name = page['path']
+      relative = pathlib.Path(name)
+      if '\x00' in name or '\\' in name or relative.is_absolute() or any(part in ('', '.', '..') for part in name.split('/')) or relative.suffix != '.png':
+        raise ValueError('invalid score page path')
+      resolved = (path.parent / relative).resolve()
+      if not resolved.is_relative_to(path.parent.resolve()):
+        raise ValueError('score page escapes its asset directory')
+      generator.engraving.validation.check_page(resolved, validated_pages, expected_sha256=page['sha256'])
+    document = document['music']
+  generator.validation.validate_document(document)
+  return document
+
+def write_json(path: pathlib.Path, document: dict) -> None:
+  generator.validation.validate_document(document)
+  generator.fretboard.validate_document(document)
+  path.parent.mkdir(parents=True, exist_ok=True)
+  path.write_text(json.dumps(document, indent=2, ensure_ascii=False) + '\n', encoding='utf-8')
+
+def count_json_files(root: pathlib.Path) -> int:
+  return sum(1 for p in root.rglob('*.json') if p.is_file())
+
+def validate_generated_tree(root: pathlib.Path, validated_pages: set[str] | None = None) -> None:
+  validated_pages = set() if validated_pages is None else validated_pages
+  for path in root.rglob('*.json'):
+    document = json.loads(path.read_text(encoding='utf-8'))
+    music = validate_json(document, path, validated_pages)
+    generator.fretboard.validate_document(music)
+    if 'presentation' in document:
+      generator.pico_artwork.validate_title(root, document['presentation'].get('title_image'))

generator/solo/analysis.py

@@ -1,44 +0,0 @@
-"""Immutable source events, meter boundaries and tempo. No solo construction."""
-from __future__ import annotations
-import dataclasses, fractions
-import generator.model, generator.score, generator.solo.types
-
-@dataclasses.dataclass(frozen=True)
-class Analysis:
-  boundaries: tuple
-  tempo: fractions.Fraction
-  melody: tuple
-  body_offset: fractions.Fraction = fractions.Fraction()
-
-def attacks(document: dict) -> tuple[generator.solo.types.Note, ...]:
-  """Collapse tied tones before extracting monophonic melody fragments."""
-  result, pending = [], {}
-  absolute = fractions.Fraction()
-  for measure in generator.score.source_staff(document)['measures']:
-    for event in generator.score.measure_notes(measure):
-      if event.get('grace') or event.get('cue'):
-        continue
-      for tone in event['pitches']:
-        number = generator.model.pitch_number(tone)
-        if number.denominator != 1:
-          continue
-        start = absolute + generator.model.fraction(event['onset'])
-        end = start + generator.model.fraction(tone['duration'])
-        pitch = generator.solo.types.Pitch(tone['letter'], generator.model.alteration(tone['alter']), tone['octave'])
-        source = pending.pop(tone.get('tie_from'), None)
-        if source is not None:
-          result[source] = dataclasses.replace(result[source], end=end, sources=(*result[source].sources, event['id']))
-        else:
-          source = len(result)
-          result.append(generator.solo.types.Note(start, end, pitch, (event['id'],)))
-        if 'tie_to' in tone:
-          pending[tone['id']] = source
-    absolute += generator.model.fraction(measure['duration'])
-  return tuple(sorted(result, key=lambda n: (n.start, -n.pitch.number(), n.sources)))
-
-def analyze(document):
-  measures=generator.score.source_staff(document)['measures']
-  boundaries=[fractions.Fraction()]
-  for m in measures:boundaries.append(boundaries[-1]+generator.model.fraction(m['duration']))
-  tempo=next((generator.model.fraction(t['bpm'])*generator.model.written_length(t['written']) for m in measures for t in m.get('tempos',[]) if 'bpm' in t),fractions.Fraction(120))
-  return Analysis(tuple(boundaries),tempo,attacks(document),boundaries[generator.score.body_start_measure_index(measures)])

generator/solo/authored.py

@@ -6,73 +6,95 @@ policy. Unknown song/role pairs go to the learned composer.
 """
 from __future__ import annotations
 import dataclasses
+import fractions
 import hashlib
 import json
-from pathlib import Path
+import pathlib
-from fractions import Fraction as F
-from generator import musicxml,parts,training_io,harmonic_time
-from generator.solo import analysis,types,placement,profiles,harmonic
 
+import generator.harmonic_time
+import generator.musicxml
+import generator.parts
+import generator.solo.harmonic
+import generator.solo.placement
+import generator.solo.profiles
+import generator.solo.types
+import generator.source
+import generator.training_io
+import generator.vocabulary
 
-def sounding(evidence,offset=F()):
+def sounding(evidence, offset=fractions.Fraction()):
-  result=[]
+  result = []
-  for i,event in enumerate(evidence['sounding_events']):
+  for i, event in enumerate(evidence['sounding_events']):
-    p=event['pitch'];number=F(event['sounding_midi'])
+    p = event['pitch']
-    if number.denominator!=1:raise ValueError('Microtonal guitar solo is unsupported')
+    number = fractions.Fraction(event['sounding_midi'])
-    pitch=types.Pitch(p['step'].lower(),F(p['alter']),p['octave']+(int(number)-int(F(event['encoded_midi'])))//12)
+    if number.denominator != 1:
-    result.append(types.Note(F(event['start'])-offset,F(event['end'])-offset,pitch,role='authored',event_id=f'authored_{i}'))
+      raise ValueError('Microtonal guitar solo is unsupported')
+    pitch = generator.solo.types.Pitch(p['step'].lower(), fractions.Fraction(p['alter']), p['octave'] + (int(number) - int(fractions.Fraction(event['encoded_midi']))) // 12)
+    result.append(generator.solo.types.Note(fractions.Fraction(event['start']) - offset, fractions.Fraction(event['end']) - offset, pitch, role='authored', event_id=f'authored_{i}'))
   return tuple(result)
 
-
+def validate_evidence(document, evidence, *, composition=None):
-def validate_evidence(document,evidence):
   """Training uses the same source-form harmonic context as solo playback."""
-  h=harmonic.Harmony(harmonic_time.solo_source(document['staves'][0]))
+  composition = composition or generator.vocabulary.current()
-  checked=h.evaluate(sounding(evidence),complete=True)
+  h = generator.solo.harmonic.Harmony(generator.harmonic_time.solo_source(document['staves'][0]), composition=composition)
-  if not checked.valid:raise ValueError('Unsupported harmonic relationship in authored solo: '+checked.error)
+  checked = h.evaluate(sounding(evidence), complete=True)
+  if not checked.valid:
+    raise ValueError('Unsupported harmonic relationship in authored solo: ' + checked.error)
   return checked
 
-
+def lookup(source, role, data_dir):
-def lookup(source,role,data_dir):
+  root = pathlib.Path(data_dir)
-  from generator.production import SourceAnalysis
+  manifest = root / 'training/manifest.json'
-  root=Path(data_dir);manifest=root/'training/manifest.json'
+  if not manifest.exists():
-  if not manifest.exists():return None
+    return None
-  value=json.loads(manifest.read_text())
+  value = json.loads(manifest.read_text())
-  if value.get('version')!=1:raise ValueError('Unsupported training manifest')
+  if value.get('version') != 1:
-  matches=[]
+    raise ValueError('Unsupported training manifest')
+  matches = []
   for entry in value['examples']:
-    if entry['slot']!=(1 if role=='acoustic' else 2):continue
+    if entry['slot'] != (1 if role == 'acoustic' else 2):
-    path=(manifest.parent/entry['source']).resolve()
+      continue
-    if not path.is_relative_to((root/'songs').resolve()):raise ValueError('Registered sources must be in data/songs')
+    path = (manifest.parent / entry['source']).resolve()
-    doc=parts.prepare_document(musicxml.load_musicxml(path))
+    if not path.is_relative_to((root / 'songs').resolve()):
-    if SourceAnalysis.read(doc).musical_key==source.musical_key:
+      raise ValueError('Registered sources must be in data/songs')
-      target=(manifest.parent/entry['solo']).resolve()
+    doc = generator.parts.prepare_document(generator.musicxml.load_musicxml(path))
-      if not target.is_relative_to(manifest.parent.resolve()):raise ValueError('Registered solo must be inside data/training')
+    if generator.source.SourceAnalysis.read(doc).musical_key == source.musical_key:
-      evidence=training_io.read(target,role,pitch_convention=entry['pitch_convention'])
+      target = (manifest.parent / entry['solo']).resolve()
-      training_io.validate_harmony(doc,evidence,entry['id'])
+      if not target.is_relative_to(manifest.parent.resolve()):
-      matches.append({**entry,'path':target,'sha256':hashlib.sha256(target.read_bytes()).hexdigest()})
+        raise ValueError('Registered solo must be inside data/training')
-  if len(matches)>1:raise ValueError('Multiple authored solos registered for the same source music and role')
+      evidence = generator.training_io.read(target, role, pitch_convention=entry['pitch_convention'])
+      generator.training_io.validate_harmony(doc, evidence, entry['id'])
+      matches.append({**entry, 'path': target, 'sha256': hashlib.sha256(target.read_bytes()).hexdigest()})
+  if len(matches) > 1:
+    raise ValueError('Multiple authored solos registered for the same source music and role')
   return matches[0] if matches else None
 
-
+def realize(entry, source_analysis, measures, part, start, end, initial=None, *, composition=None):
-def realize(entry,source_analysis,measures,part,start,end,initial=None):
+  composition = composition or generator.vocabulary.current()
-  evidence=training_io.read(entry['path'],part,pitch_convention=entry['pitch_convention'])
+  evidence = generator.training_io.read(entry['path'], part, pitch_convention=entry['pitch_convention'])
-  if evidence['discrepancies']:raise ValueError(entry['id']+': invalid authored timing/ties: '+str(evidence['discrepancies']))
+  if evidence['discrepancies']:
-  expected=[(a,b-a) for a,b in zip(source_analysis.boundaries,source_analysis.boundaries[1:])]
+    raise ValueError(entry['id'] + ': invalid authored timing/ties: ' + str(evidence['discrepancies']))
-  if [(F(m['start']),F(m['duration'])) for m in evidence['measures']]!=expected:raise ValueError(entry['id']+': source/solo forms differ; register and retrain the correct pair')
+  expected = [(a, b - a) for a, b in zip(source_analysis.boundaries, source_analysis.boundaries[1:])]
-  notes=sounding(evidence,source_analysis.body_offset)
+  if [(fractions.Fraction(m['start']), fractions.Fraction(m['duration'])) for m in evidence['measures']] != expected:
+    raise ValueError(entry['id'] + ': source/solo forms differ; register and retrain the correct pair')
+  notes = sounding(evidence, source_analysis.body_offset)
   for n in notes:
-    if n.start<start or n.end>end:raise ValueError(f'{entry["id"]}: authored event {n.start}–{n.end} conflicts with arrangement window {start}–{end}; no clipping applied')
+    if n.start < start or n.end > end:
-  profile=profiles.for_part(part,source_analysis)
+      raise ValueError(f'{entry["id"]}: authored event {n.start}–{n.end} conflicts with arrangement window {start}–{end}; no clipping applied')
-  illegal=next((n for n in notes if not placement.positions(n.pitch.number(),profile)),None)
+  profile = generator.solo.profiles.for_part(part, source_analysis)
-  if illegal:raise ValueError(f'{entry["id"]}: {illegal.pitch.label()} at {illegal.start} is outside strings 1–6/frets 1–12; authored solo left unchanged')
+  illegal = next((n for n in notes if not generator.solo.placement.positions(n.pitch.number(), profile)), None)
-  placed=placement.place(tuple(notes),source_analysis.tempo,profile=profile)
+  if illegal:
-  if placed is None:raise ValueError(entry['id']+': no legal complete guitar placement')
+    raise ValueError(f'{entry["id"]}: {illegal.pitch.label()} at {illegal.start} is outside strings 1–6/frets 1–12; authored solo left unchanged')
-  spans=tuple(dataclasses.replace(s,start=s.start+start,end=s.end+start) for s in harmonic_time.read({'measures':measures},initial=initial))
+  placed = generator.solo.placement.place(tuple(notes), source_analysis.tempo, profile=profile)
-  checked=harmonic.Harmony(spans).evaluate(placed[1],complete=True)
+  if placed is None:
-  if not checked.valid:raise ValueError(entry['id']+': '+checked.error)
+    raise ValueError(entry['id'] + ': no legal complete guitar placement')
-  phrase=types.Phrase('authored',start,end,placed[1],'authored','authored','handoff',theme=entry['id'],expressions=checked.contracts)
+  spans = tuple(dataclasses.replace(s, start=s.start + start, end=s.end + start) for s in generator.harmonic_time.read({'measures': measures}, initial=initial))
-  return types.Solo(part,spans,(phrase,),source_analysis.tempo,end,
+  checked = generator.solo.harmonic.Harmony(spans, composition=composition).evaluate(placed[1], complete=True)
-    {'construction':'authored_repertoire','authored_id':entry['id'],'authored_sha256':entry['sha256'],
+  if not checked.valid:
-      'pitch_convention':entry['pitch_convention'],'exact_sounding_events':True,
+    raise ValueError(entry['id'] + ': ' + checked.error)
-      'note':'Authored music is preserved and passes automatic contextual harmonic validation.',
+  phrase = generator.solo.types.Phrase('authored', start, end, placed[1], 'authored', 'authored', 'handoff', theme=entry['id'], expressions=checked.contracts)
-      'annotations':evidence['annotations']},profile,start)
+  return generator.solo.types.Solo(part, spans, (phrase,), source_analysis.tempo, end,
+    {'construction': 'authored_repertoire', 'authored_id': entry['id'], 'authored_sha256': entry['sha256'],
+      'pitch_convention': entry['pitch_convention'], 'exact_sounding_events': True,
+      'note': 'Authored music is preserved and passes automatic contextual harmonic validation.',
+      'annotations': evidence['annotations']}, profile, start)

generator/solo/composer.py

@@ -1,57 +1,75 @@
 """The sole production solo composer: frozen learned model plus music validity."""
 import dataclasses
+import fractions
+import pathlib
 import time
-from fractions import Fraction as F
-from pathlib import Path
-from generator import harmonic_time, vocabulary
-from generator.solo import types, profiles, harmonic, placement, learned, decoder, groups
 
+import generator.harmonic_time
+import generator.solo.decoder
+import generator.solo.groups
+import generator.solo.harmonic
+import generator.solo.learned
+import generator.solo.placement
+import generator.solo.profiles
+import generator.solo.types
+import generator.vocabulary
 
-def compose(analysis,measures,part,end,*,learned_model=None,initial=None,expressions_enabled=True,start=F()):
+def compose(analysis, measures, part, end, *, learned_model=None, initial=None, expressions_enabled=True, start=fractions.Fraction(), composition=None):
-  began=time.monotonic()
+  composition = composition or generator.vocabulary.current()
-  spans=tuple(dataclasses.replace(s,start=s.start+start,end=s.end+start) for s in harmonic_time.read({'measures':measures},initial=initial))
+  began = time.monotonic()
-  profile=profiles.for_part(part,analysis);harmony=harmonic.Harmony(spans,profile)
+  spans = tuple(dataclasses.replace(s, start=s.start + start, end=s.end + start) for s in generator.harmonic_time.read({'measures': measures}, initial=initial))
-  model=learned_model or learned.Model.load(Path(__file__).resolve().parents[2]/'data/models/solo.json')
+  profile = generator.solo.profiles.for_part(part, analysis)
-  settings=vocabulary.role(part+'_solo')
+  harmony = generator.solo.harmonic.Harmony(spans, profile, composition=composition)
-  builder=decoder.Decoder(analysis,harmony,model,part,start,end,expressions_enabled=expressions_enabled,
+  model = learned_model or generator.solo.learned.Model.load(pathlib.Path(__file__).resolve().parents[2] / 'data/models/solo.json')
-    attempts=settings.get('attempts',4),seed=settings.get('choice_seed',''))
+  settings = composition.role(part + '_solo')
-  selected=builder.compose();decoded=time.monotonic()
+  builder = generator.solo.decoder.Decoder(analysis, harmony, model, part, start, end, expressions_enabled=expressions_enabled, attempts=settings.get('attempts', 4), seed=settings.get('choice_seed', ''), composition=composition)
-  placed=placement.place(selected.notes,analysis.tempo,profile=profile)
+  selected = builder.compose()
-  if placed is None:raise decoder.SearchExhausted('learned solo has no complete legal guitar path')
+  decoded = time.monotonic()
+  placed = generator.solo.placement.place(selected.notes, analysis.tempo, profile=profile)
+  if placed is None:
+    raise generator.solo.decoder.SearchExhausted('learned solo has no complete legal guitar path')
   # Phrase labels describe heard rests/arrivals. They do not impose a new
   # endpoint or reset the learned history at barlines or arbitrary windows.
-  notes=placed[1];ends=[n.end for i,n in enumerate(notes[:-1]) if notes[i+1].start-n.end>=1]
+  notes = placed[1]
-  boundaries=[start,*[t for t in ends if start<t<end],end];phrases=[]
+  ends = [n.end for i, n in enumerate(notes[:-1]) if notes[i + 1].start - n.end >= 1]
-  home=tuple(n for n in notes if n.start<(ends[0] if ends else min(end,start+8)))
+  boundaries = [start, *[t for t in ends if start < t < end], end]
-  for i,(a,b) in enumerate(zip(boundaries,boundaries[1:])):
+  phrases = []
-    ns=tuple(n for n in notes if a<=n.start<b)
+  home = tuple(n for n in notes if n.start < (ends[0] if ends else min(end, start + 8)))
-    cs=tuple(c for c in selected.contracts if a<=c.target.start<b)
+  for i, (a, b) in enumerate(zip(boundaries, boundaries[1:])):
-    ending='handoff' if b==end else 'settle'
+    ns = tuple(n for n in notes if a <= n.start < b)
-    phrases.append(types.Phrase(f'learned_{i+1}',a,b,ns,'learned_sequence','learned',ending,
+    cs = tuple(c for c in selected.contracts if a <= c.target.start < b)
-      expressions=cs,groups=groups.describe(tuple((n.start,n.end) for n in ns)),
+    ending = 'handoff' if b == end else 'settle'
-      theme='introduced' if i==0 else 'conditional_history',identity=home if i==0 else ()))
+    phrases.append(generator.solo.types.Phrase(f'learned_{i + 1}', a, b, ns, 'learned_sequence', 'learned', ending,
-  result=types.Solo(part,spans,tuple(phrases),analysis.tempo,end,
+      expressions=cs, groups=generator.solo.groups.describe(tuple((n.start, n.end) for n in ns)),
-    {'construction':'learned_joint_events','version':learned.VERSION,'decode_seconds':decoded-began,
+      theme='introduced' if i == 0 else 'conditional_history', identity=home if i == 0 else ()))
-      'placement_seconds':time.monotonic()-decoded,'selection_total':selected.cost,
+  result = generator.solo.types.Solo(part, spans, tuple(phrases), analysis.tempo, end,
-      'objective':'seeded sampling from validated learned joint-event likelihoods; score is diagnostic, not a complete-solo ranking',
+    {'construction': 'learned_joint_events', 'version': generator.solo.learned.VERSION, 'decode_seconds': decoded - began,
-      'budget':{'attempts':builder.attempts},
+      'placement_seconds': time.monotonic() - decoded, 'selection_total': selected.cost,
-      'counts':dict(builder.counts),'frontiers':builder.trace,'completed_candidates':[
+      'objective': 'seeded sampling from validated learned joint-event likelihoods; score is diagnostic, not a complete-solo ranking',
-        {'rank':i+1,'cost':s.cost,'events':[(str(a),str(b),p) for a,b,p in s.events]} for i,s in enumerate(builder.completed)],
+      'budget': {'attempts': builder.attempts},
-      'expressions_enabled':expressions_enabled},profile,start)
+      'counts': dict(builder.counts), 'frontiers': builder.trace, 'completed_candidates': [
-  validate(result);return result
+        {'rank': i + 1, 'cost': s.cost, 'events': [(str(a), str(b), p) for a, b, p in s.events]} for i, s in enumerate(builder.completed)],
+      'expressions_enabled': expressions_enabled}, profile, start)
+  validate(result, composition=composition)
+  return result
 
-
+def validate(solo, *, composition=None):
-def validate(solo):
+  composition = composition or generator.vocabulary.current()
-  previous=None
+  previous = None
   for n in solo.notes():
-    if n.start<solo.start or n.end>solo.end or n.end<=n.start or previous and previous.end>n.start:raise ValueError('overlapping/out-of-bounds sounding event')
+    if n.start < solo.start or n.end > solo.end or n.end <= n.start or previous and previous.end > n.start:
-    if n.position not in placement.positions(n.pitch.number(),solo.profile):raise ValueError('invalid guitar placement')
+      raise ValueError('overlapping/out-of-bounds sounding event')
-    previous=n
+    if n.position not in generator.solo.placement.positions(n.pitch.number(), solo.profile):
-  checked=harmonic.Harmony(solo.harmony).evaluate(solo.notes(),until=solo.end,complete=True,
+      raise ValueError('invalid guitar placement')
+    previous = n
+  checked = generator.solo.harmonic.Harmony(solo.harmony, composition=composition).evaluate(solo.notes(), until=solo.end, complete=True,
     expressions=tuple(c for p in solo.phrases for c in p.expressions))
-  if not checked.valid:raise ValueError(checked.error)
+  if not checked.valid:
-  cursor=solo.start
+    raise ValueError(checked.error)
+  cursor = solo.start
   for p in solo.phrases:
-    if p.start!=cursor or p.end<=p.start:raise ValueError('phrase coverage gap or overlap')
+    if p.start != cursor or p.end <= p.start:
-    cursor=p.end
+      raise ValueError('phrase coverage gap or overlap')
-  if cursor!=solo.end:raise ValueError('incomplete solo duration')
+    cursor = p.end
+  if cursor != solo.end:
+    raise ValueError('incomplete solo duration')

generator/solo/decoder.py

@@ -8,300 +8,369 @@ legacy fallback. A bounded restart handles an exhausted/entirely silent sample.
 from __future__ import annotations
 import collections
 import dataclasses
+import fractions
 import hashlib
-import random
 import math
-from fractions import Fraction as F
+import random
-from generator import model as music, vocabulary
-from generator.solo import expressions, groups, learned, types
 
+import generator.harmony
+import generator.model
+import generator.solo.expressions
+import generator.solo.groups
+import generator.solo.learned
+import generator.solo.types
+import generator.vocabulary
 
-class SearchExhausted(ValueError):pass
+class SearchExhausted(ValueError):
-
+  pass
 
 @dataclasses.dataclass(frozen=True)
 class State:
-  events:tuple=()
+  events: tuple = ()
-  notes:tuple=()
+  notes: tuple = ()
-  contracts:tuple=()
+  contracts: tuple = ()
-  history:tuple=()
+  history: tuple = ()
-  previous:int|None=None
+  previous: int | None = None
-  cost:float=0.
+  cost: float = 0.
-
-
-def event_key(events):return tuple((a,b,-1 if p is None else p) for a,b,p in events)
 
+def event_key(events): return tuple((a, b, -1 if p is None else p) for a, b, p in events)
 
 class Decoder:
-  def __init__(self,analysis,harmony,model,role,start,end,*,expressions_enabled=True,attempts=4,seed=''):
+  def __init__(self, analysis, harmony, model, role, start, end, *, expressions_enabled=True, attempts=4, seed='', composition=None):
-    if type(attempts) is not int or not 1<=attempts<=16:raise ValueError('attempts must be between 1 and 16')
+    self.composition = composition or generator.vocabulary.current()
-    self.analysis,self.harmony,self.model,self.role=analysis,harmony,model,role
+    if type(attempts) is not int or not 1 <= attempts <= 16:
-    self.start,self.end=start,end
+      raise ValueError('attempts must be between 1 and 16')
-    self.attempts=attempts
+    self.analysis, self.harmony, self.model, self.role = analysis, harmony, model, role
-    source=tuple((str(n.start),str(n.end),n.pitch.number()) for n in analysis.melody)
+    self.start, self.end = start, end
-    self.seed=seed or hashlib.sha256(repr((source,role,learned.VERSION)).encode()).hexdigest()
+    self.attempts = attempts
-    self.expressions_enabled=expressions_enabled
+    source = tuple((str(n.start), str(n.end), n.pitch.number()) for n in analysis.melody)
-    self.low,self.high=harmony.profile.register
+    self.seed = seed or hashlib.sha256(repr((source, role, generator.solo.learned.VERSION)).encode()).hexdigest()
-    limits=vocabulary.data('roles')['difficulty']
+    self.expressions_enabled = expressions_enabled
-    self.durations=tuple(d for d in model.durations if 0<d<=4 and d.denominator in (1,2,4) and
+    self.low, self.high = harmony.profile.register
-      (d.denominator!=4 or analysis.tempo<=limits['sixteenth_max_bpm']))
+    limits = self.composition.data('roles')['difficulty']
-    self.allow_triplets=analysis.tempo<=limits['triplet_max_bpm']
+    self.durations = tuple(d for d in model.durations if 0 < d <= 4 and d.denominator in (1, 2, 4) and
-    self._actions={};self._context={};self._pitches={};self._colors={};self._permissions={};self.trace=[];self.counts=collections.Counter()
+      (d.denominator != 4 or analysis.tempo <= limits['sixteenth_max_bpm']))
-
+    self.allow_triplets = analysis.tempo <= limits['triplet_max_bpm']
-  def context(self,time):
+    self._actions = {}
+    self._context = {}
+    self._pitches = {}
+    self._colors = {}
+    self._permissions = {}
+    self.trace = []
+    self.counts = collections.Counter()
+  def context(self, time):
     if time not in self._context:
-      harmonic,root,future=learned.chord_context(self.harmony,time)
+      harmonic, root, future = generator.solo.learned.chord_context(self.harmony, time)
-      self._context[time]=(harmonic,root,learned.phase_at(self.analysis,time),learned.source_feature(self.analysis,time,root),future)
+      self._context[time] = (harmonic, root, generator.solo.learned.phase_at(self.analysis, time), generator.solo.learned.source_feature(self.analysis, time, root), future)
     return self._context[time]
-
+  def pitch(self, number, time):
-  def pitch(self,number,time):
+    span = self.harmony.at(time)
-    span=self.harmony.at(time)
+    identity = (number, span.start if span else None)
-    identity=(number,span.start if span else None)
+    if identity not in self._pitches:
-    if identity not in self._pitches:self._pitches[identity]=self._spell(number,time)
+      self._pitches[identity] = self._spell(number, time)
     return self._pitches[identity]
-
+  def _spell(self, number, time):
-  def _spell(self,number,time):
+    span = self.harmony.at(time)
-    span=self.harmony.at(time)
+    if span and span.chord != 'N.C.':
-    if span and span.chord!='N.C.':
+      chord = generator.solo.expressions.parsed_chord(span.chord)
-      chord=expressions.parsed_chord(span.chord)
+      for degree, alter in generator.harmony.degrees(chord).items():
-      from generator import harmony as chords
+        p = generator.solo.expressions.degree_pitch(chord, degree, alter, number)
-      for degree,alter in chords.degrees(chord).items():
+        if p.number() == number:
-        p=expressions.degree_pitch(chord,degree,alter,number)
+          return p
-        if p.number()==number:return p
+      for letter, alter in zip(generator.model.LETTERS, span.key['alterations']):
-      for letter,alter in zip(music.LETTERS,span.key['alterations']):
+        accidental = generator.model.fraction(alter)
-        accidental=music.fraction(alter)
+        if (generator.model.NATURAL[letter] + accidental) % 12 == number % 12:
-        if (music.NATURAL[letter]+accidental)%12==number%12:
+          return generator.solo.types.Pitch(letter, accidental, int((number - generator.model.NATURAL[letter] - accidental) // 12) - 1)
-          return types.Pitch(letter,accidental,int((number-music.NATURAL[letter]-accidental)//12)-1)
+    names = (('C', 0), ('D', -1), ('D', 0), ('E', -1), ('E', 0), ('F', 0), ('F', 1), ('G', 0), ('A', -1), ('A', 0), ('B', -1), ('B', 0))
-    names=(('C',0),('D',-1),('D',0),('E',-1),('E',0),('F',0),('F',1),('G',0),('A',-1),('A',0),('B',-1),('B',0))
+    letter, alter = names[number % 12]
-    letter,alter=names[number%12]
+    return generator.solo.types.Pitch(letter.lower(), fractions.Fraction(alter), number // 12 - 1)
-    return types.Pitch(letter.lower(),F(alter),number//12-1)
+  def note(self, a, b, p, role='learned'):
-
+    pitch = p if isinstance(p, generator.solo.types.Pitch) else self.pitch(p, a)
-  def note(self,a,b,p,role='learned'):
+    return generator.solo.types.Note(a, b, pitch, role=role, event_id=f'{a}:{b}:{pitch.number()}')
-    pitch=p if isinstance(p,types.Pitch) else self.pitch(p,a)
+  def color_bases(self, n):
-    return types.Note(a,b,pitch,role=role,event_id=f'{a}:{b}:{pitch.number()}')
+    key = (n.start, n.end, n.pitch.number())
-
+    if key in self._colors:
-  def color_bases(self,n):
+      return self._colors[key]
-    key=(n.start,n.end,n.pitch.number())
+    bases = []
-    if key in self._colors:return self._colors[key]
+    regions = [(None, n)]
-    bases=[]
+    regions.extend((s.start, dataclasses.replace(n, start=s.start)) for s in self.harmony.spans
-    regions=[(None,n)]
+      if n.start < s.start < n.end and s.start - n.start <= fractions.Fraction(self.composition.data('harmony')['anticipation_max']))
-    regions.extend((s.start,dataclasses.replace(n,start=s.start)) for s in self.harmony.spans
+    for boundary, arrived in regions:
-      if n.start<s.start<n.end and s.start-n.start<=F(vocabulary.data('harmony')['anticipation_max']))
+      exposed = [s for s, _, _ in self.harmony.segments(arrived) if n.pitch.number() % 12 not in s.roles.basic | s.roles.specified]
-    for boundary,arrived in regions:
+      if not exposed:
-      exposed=[s for s,_,_ in self.harmony.segments(arrived) if n.pitch.number()%12 not in s.roles.basic|s.roles.specified]
+        continue
-      if not exposed:continue
+      for rule in self.composition.data('harmony')['colors']:
-      for rule in vocabulary.data('harmony')['colors']:
+        if arrived.end - arrived.start > fractions.Fraction(rule['max_beats']):
-        if arrived.end-arrived.start>F(rule['max_beats']):continue
+          continue
-        if any(s.chord=='N.C.' or not expressions.eligible_color(rule,s,self.harmony) or
+        if any(s.chord == 'N.C.' or not generator.solo.expressions.eligible_color(rule, s, self.harmony) or
-          expressions.degree_pitch(expressions.parsed_chord(s.chord),rule['degree'],rule['alter'],n.pitch.number()).number()!=n.pitch.number()
+          generator.solo.expressions.degree_pitch(generator.solo.expressions.parsed_chord(s.chord), rule['degree'], rule['alter'], n.pitch.number()).number() != n.pitch.number()
-          for s in exposed):continue
+          for s in exposed):
-        bases.append((rule,boundary))
+          continue
-    self._colors[key]=tuple(bases)
+        bases.append((rule, boundary))
+    self._colors[key] = tuple(bases)
     return self._colors[key]
-
+  def color_contract(self, n, target, rule, boundary):
-  def color_contract(self,n,target,rule,boundary):
+    span = self.harmony.at(boundary if boundary is not None else n.start)
-    span=self.harmony.at(boundary if boundary is not None else n.start)
+    if not span:
-    if not span:return None
+      return None
-    kind='anticipated_color' if boundary is not None else 'color'
+    kind = 'anticipated_color' if boundary is not None else 'color'
-    return types.ExpressionContract(n.event_id+':'+rule['id']+':'+str(boundary),kind,rule['id'],(n,),target,span.chord,boundary)
+    return generator.solo.types.ExpressionContract(n.event_id + ':' + rule['id'] + ':' + str(boundary), kind, rule['id'], (n,), target, span.chord, boundary)
-
+  def permission(self, n):
-  def permission(self,n):
+    key = (n.start, n.end, n.pitch.number())
-    key=(n.start,n.end,n.pitch.number())
+    if key not in self._permissions:
-    if key not in self._permissions:self._permissions[key]=self._permission(n)
+      self._permissions[key] = self._permission(n)
     return self._permissions[key]
-
+  def _permission(self, n):
-  def _permission(self,n):
     """Find justified color/anticipation support, never authorize by label."""
     # Source spans are contiguous. These are precisely the ordinary-event
     # branches of Harmony.evaluate, without rebuilding an empty contract map.
-    if self.harmony.at(n.start) and n.end<=self.harmony.spans[-1].end:
+    if self.harmony.at(n.start) and n.end <= self.harmony.spans[-1].end:
-      if n.pitch.number()%12 in self.harmony.members(n.start,n.end):return ()
+      if n.pitch.number() % 12 in self.harmony.members(n.start, n.end):
-      if n.end-n.start<2 and n.pitch.number()%12 in self.harmony.permitted(n.start,n.end):return ()
+        return ()
-    if not self.expressions_enabled:return None
+      if n.end - n.start < 2 and n.pitch.number() % 12 in self.harmony.permitted(n.start, n.end):
-    span=self.harmony.at(n.start)
+        return ()
-    if not span or span.chord=='N.C.':return None
+    if not self.expressions_enabled:
-    for rule,boundary in self.color_bases(n):
+      return None
-      if rule.get('resolve'):continue
+    span = self.harmony.at(n.start)
-      c=self.color_contract(n,n,rule,boundary)
+    if not span or span.chord == 'N.C.':
-      if self.harmony.evaluate((n,),complete=True,expressions=(c,),infer=False).valid:return (c,)
+      return None
-    for arriving in self.harmony.spans:
+    for rule, boundary in self.color_bases(n):
-      if n.start<arriving.start<n.end and arriving.start-n.start<=F(vocabulary.data('harmony')['anticipation_max']):
-        c=types.ExpressionContract(n.event_id+':anticipation','anticipation','early_chord_tone',(n,),n,arriving.chord,arriving.start)
-        if self.harmony.evaluate((n,),complete=True,expressions=(c,),infer=False).valid:return (c,)
-    return None
-
-  def tails(self,n,target):
-    """Finite resolution timing: direct, real space, or one valid intervening note."""
-    gap=target.start-n.end
-    if gap<0:return
-    if gap==0:
-      yield ((n.start,n.end,n.pitch.number()),(target.start,target.end,target.pitch.number())),(n,target)
-      return
-    yield ((n.start,n.end,n.pitch.number()),(n.end,target.start,None),(target.start,target.end,target.pitch.number())),(n,target)
-    windows={(n.end,target.start)} if gap<=1 else set()
-    if gap>=1:windows|={(n.end,n.end+F(1,2)),(target.start-F(1,2),target.start)}
-    for a,b in sorted(windows):
-      for number in range(max(self.low,n.pitch.number()-3),min(self.high,n.pitch.number()+3)+1):
-        mid=self.note(a,b,number)
-        if number in (n.pitch.number(),target.pitch.number()) or not self.harmony.ordinary(mid):continue
-        events=[(n.start,n.end,n.pitch.number())]
-        if a>n.end:events.append((n.end,a,None))
-        events.append((a,b,number))
-        if b<target.start:events.append((b,target.start,None))
-        events.append((target.start,target.end,target.pitch.number()))
-        yield tuple(events),(n,mid,target)
-
-  def resolutions(self,n):
-    """Joint color/delayed-approach actions; the destination is never optional."""
-    requests=[]
-    for rule,boundary in self.color_bases(n):
       if rule.get('resolve'):
-        for gap in (F(),F(1,2),F(1)):
+        continue
-          if gap<=F(vocabulary.data('harmony')['resolution_gap']):requests.append(('color',rule,boundary,n.end+gap,(-2,-1,1,2)))
+      c = self.color_contract(n, n, rule, boundary)
-    delay=vocabulary.data('lead_ins')['delayed_approach']
+      if self.harmony.evaluate((n,), complete=True, expressions=(c,), infer=False).valid:
-    if n.end-n.start<=F(delay['max_note']) and n.pitch.number()%12 not in self.harmony.permitted(n.start,n.end):
+        return tuple([c])
-      for arrival in map(F,delay['arrivals']):
+    for arriving in self.harmony.spans:
-        if n.end<=n.start+arrival<=n.start+F(delay['deadline']):requests.append(('delayed',delay,None,n.start+arrival,delay['steps']))
+      if n.start < arriving.start < n.end and arriving.start - n.start <= fractions.Fraction(self.composition.data('harmony')['anticipation_max']):
-    for kind,rule,boundary,arrival,steps in requests:
+        c = generator.solo.types.ExpressionContract(n.event_id + ':anticipation', 'anticipation', 'early_chord_tone', (n,), n, arriving.chord, arriving.start)
+        if self.harmony.evaluate((n,), complete=True, expressions=(c,), infer=False).valid:
+          return tuple([c])
+    return None
+  def tails(self, n, target):
+    """Finite resolution timing: direct, real space, or one valid intervening note."""
+    gap = target.start - n.end
+    if gap < 0:
+      return
+    if gap == 0:
+      yield ((n.start, n.end, n.pitch.number()), (target.start, target.end, target.pitch.number())), (n, target)
+      return
+    yield ((n.start, n.end, n.pitch.number()), (n.end, target.start, None), (target.start, target.end, target.pitch.number())), (n, target)
+    windows = {(n.end, target.start)} if gap <= 1 else set()
+    if gap >= 1:
+      windows |= {(n.end, n.end + fractions.Fraction(1, 2)), (target.start - fractions.Fraction(1, 2), target.start)}
+    for a, b in sorted(windows):
+      for number in range(max(self.low, n.pitch.number() - 3), min(self.high, n.pitch.number() + 3) + 1):
+        mid = self.note(a, b, number)
+        if number in (n.pitch.number(), target.pitch.number()) or not self.harmony.ordinary(mid):
+          continue
+        events = [(n.start, n.end, n.pitch.number())]
+        if a > n.end:
+          events.append((n.end, a, None))
+        events.append((a, b, number))
+        if b < target.start:
+          events.append((b, target.start, None))
+        events.append((target.start, target.end, target.pitch.number()))
+        yield tuple(events), (n, mid, target)
+  def resolutions(self, n):
+    """Joint color/delayed-approach actions; the destination is never optional."""
+    requests = []
+    for rule, boundary in self.color_bases(n):
+      if rule.get('resolve'):
+        for gap in (fractions.Fraction(), fractions.Fraction(1, 2), fractions.Fraction(1)):
+          if gap <= fractions.Fraction(self.composition.data('harmony')['resolution_gap']):
+            requests.append(('color', rule, boundary, n.end + gap, (-2, -1, 1, 2)))
+    delay = self.composition.data('lead_ins')['delayed_approach']
+    if n.end - n.start <= fractions.Fraction(delay['max_note']) and n.pitch.number() % 12 not in self.harmony.permitted(n.start, n.end):
+      for arrival in map(fractions.Fraction, delay['arrivals']):
+        if n.end <= n.start + arrival <= n.start + fractions.Fraction(delay['deadline']):
+          requests.append(('delayed', delay, None, n.start + arrival, delay['steps']))
+    for kind, rule, boundary, arrival, steps in requests:
       for delta in steps:
-        number=n.pitch.number()+delta
+        number = n.pitch.number() + delta
-        if not self.low<=number<=self.high:continue
+        if not self.low <= number <= self.high:
-        for hold in (F(1,2),F(1),F(2)):
+          continue
-          if arrival+hold>self.end or number%12 not in self.harmony.members(arrival,arrival+hold):continue
+        for hold in (fractions.Fraction(1, 2), fractions.Fraction(1), fractions.Fraction(2)):
-          target=self.note(arrival,arrival+hold,number,'destination')
+          if arrival + hold > self.end or number % 12 not in self.harmony.members(arrival, arrival + hold):
-          c=self.color_contract(n,target,rule,boundary) if kind=='color' else types.ExpressionContract(n.event_id+':delayed','delayed_approach',rule['id'],(n,),target,self.harmony.at(arrival).chord)
+            continue
+          target = self.note(arrival, arrival + hold, number, 'destination')
+          c = self.color_contract(n, target, rule, boundary) if kind == 'color' else generator.solo.types.ExpressionContract(n.event_id + ':delayed', 'delayed_approach', rule['id'], (n,), target, self.harmony.at(arrival).chord)
           # Color and delayed-approach contracts permit this intervening
           # window. Validate the invariant endpoints once; tails() checks
           # every inserted note's ordinary full-duration permission itself.
-          if not self.harmony.evaluate((n,target),complete=True,expressions=(c,),infer=False).valid:continue
+          if not self.harmony.evaluate((n, target), complete=True, expressions=(c,), infer=False).valid:
-          for events,ns in self.tails(n,target):
+            continue
-            if not groups.valid(tuple((a,b) for a,b,_ in events)):continue
+          for events, ns in self.tails(n, target):
-            yield events,ns,(c,)
+            if not generator.solo.groups.valid(tuple((a, b) for a, b, _ in events)):
-
+              continue
-  def actions(self,time):
+            yield events, ns, (c,)
-    if time in self._actions:return self._actions[time]
+  def actions(self, time):
-    result=[]
+    if time in self._actions:
-    def add(events,notes,contracts=()):result.append((events,notes,contracts))
+      return self._actions[time]
+    result = []
+    def add(events, notes, contracts=()): result.append((events, notes, contracts))
     for duration in self.durations:
-      end=time+duration
+      end = time + duration
-      if end>self.end:continue
+      if end > self.end:
-      add(((time,end,None),),())
+        continue
-      for number in range(self.low,self.high+1):
+      add(((time, end, None),), ())
-        n=self.note(time,end,number);support=self.permission(n)
+      for number in range(self.low, self.high + 1):
-        if support is not None:add(((time,end,number),),(n,),support)
+        n = self.note(time, end, number)
+        support = self.permission(n)
+        if support is not None:
+          add(((time, end, number),), (n,), support)
         if self.expressions_enabled and not self.harmony.ordinary(n):
-          for events,ns,contracts in self.resolutions(n):add(events,ns,contracts)
+          for events, ns, contracts in self.resolutions(n):
-          rule=vocabulary.data('lead_ins')['passing']
+            add(events, ns, contracts)
-          if duration<=F(rule['max_note']):
+          rule = self.composition.data('lead_ins')['passing']
+          if duration <= fractions.Fraction(rule['max_note']):
             for step in rule['steps']:
-              p=number+step
+              p = number + step
-              if not self.low<=p<=self.high:continue
+              if not self.low <= p <= self.high:
-              for hold in (F(1,2),F(1),F(2)):
+                continue
-                if end+hold>self.end:continue
+              for hold in (fractions.Fraction(1, 2), fractions.Fraction(1), fractions.Fraction(2)):
-                target=self.note(end,end+hold,p);extra=self.permission(target)
+                if end + hold > self.end:
-                if extra is None or any(c.kind not in ('color','anticipated_color') for c in extra):continue
+                  continue
-                c=types.ExpressionContract(n.event_id+':passing','passing',rule['id'],(n,),target,self.harmony.at(end).chord)
+                target = self.note(end, end + hold, p)
-                add(((time,end,number),(end,end+hold,p)),(n,target),(c,*extra))
+                extra = self.permission(target)
+                if extra is None or any(c.kind not in ('color', 'anticipated_color') for c in extra):
+                  continue
+                c = generator.solo.types.ExpressionContract(n.event_id + ':passing', 'passing', rule['id'], (n,), target, self.harmony.at(end).chord)
+                add(((time, end, number), (end, end + hold, p)), (n, target), (c, *extra))
     if self.expressions_enabled:
-      limits=vocabulary.data('roles')['difficulty']
+      limits = self.composition.data('roles')['difficulty']
-      for rule in vocabulary.data('lead_ins')['patterns']:
+      for rule in self.composition.data('lead_ins')['patterns']:
-        difficulty=rule.get('difficulty')
+        difficulty = rule.get('difficulty')
-        if difficulty and self.analysis.tempo>limits[difficulty+'_max_bpm']:continue
+        if difficulty and self.analysis.tempo > limits[difficulty + '_max_bpm']:
+          continue
         # Connections require a concrete preceding note, checked at expansion.
-        durations=tuple(map(F,rule['durations']));gaps=tuple(map(F,rule.get('internal_gaps',['0']*(len(durations)-1))))
+        durations = tuple(map(fractions.Fraction, rule['durations']))
-        target_time=time+sum(durations,F())+sum(gaps,F())+F(rule['gap'])
+        gaps = tuple(map(fractions.Fraction, rule.get('internal_gaps', ['0'] * (len(durations) - 1))))
-        span=self.harmony.at(target_time)
+        target_time = time + sum(durations, fractions.Fraction()) + sum(gaps, fractions.Fraction()) + fractions.Fraction(rule['gap'])
-        if not span or span.chord=='N.C.':continue
+        span = self.harmony.at(target_time)
-        holds=(F(1,3),) if difficulty=='triplet' else (F(1,2),F(1),F(2))
+        if not span or span.chord == 'N.C.':
+          continue
+        holds = (fractions.Fraction(1, 3),) if difficulty == 'triplet' else (fractions.Fraction(1, 2), fractions.Fraction(1), fractions.Fraction(2))
         for duration in holds:
-          end=target_time+duration
+          end = target_time + duration
-          if end>self.end:continue
+          if end > self.end:
-          for number in range(self.low,self.high+1):
+            continue
-            if number%12 not in self.harmony.members(target_time,end):continue
+          for number in range(self.low, self.high + 1):
-            target=self.note(target_time,end,number,'destination');cursor=time;notes=[];events=[]
+            if number % 12 not in self.harmony.members(target_time, end):
-            for i,(delta,d) in enumerate(zip(rule['semitones'],durations)):
+              continue
-              n=self.note(cursor,cursor+d,expressions.chromatic(target.pitch,delta),rule['kind'])
+            target = self.note(target_time, end, number, 'destination')
-              notes.append(n);events.append((n.start,n.end,n.pitch.number()));cursor=n.end
+            cursor = time
-              gap=gaps[i] if i<len(gaps) else F(rule['gap'])
+            notes = []
-              if gap:events.append((cursor,cursor+gap,None));cursor+=gap
+            events = []
-            if any(not self.low<=n.pitch.number()<=self.high for n in notes):continue
+            for i, (delta, d) in enumerate(zip(rule['semitones'], durations)):
-            if not groups.valid(tuple((a,b) for a,b,p in (*events,(target_time,end,number)))):continue
+              n = self.note(cursor, cursor + d, generator.solo.expressions.chromatic(target.pitch, delta), rule['kind'])
-            contract=types.ExpressionContract(f'{time}:{number}:{rule["id"]}',rule['kind'],rule['id'],tuple(notes),target,span.chord)
+              notes.append(n)
-            ns=(*notes,target)
+              events.append((n.start, n.end, n.pitch.number()))
-            if rule['kind']=='connection':
+              cursor = n.end
+              gap = gaps[i] if i < len(gaps) else fractions.Fraction(rule['gap'])
+              if gap:
+                events.append((cursor, cursor + gap, None))
+                cursor += gap
+            if any(not self.low <= n.pitch.number() <= self.high for n in notes):
+              continue
+            if not generator.solo.groups.valid(tuple((a, b) for a, b, p in (*events, (target_time, end, number)))):
+              continue
+            contract = generator.solo.types.ExpressionContract(f'{time}:{number}:{rule["id"]}', rule['kind'], rule['id'], tuple(notes), target, span.chord)
+            ns = (*notes, target)
+            if rule['kind'] == 'connection':
               # Validation waits for the actual incoming event.
-              add((*events,(target_time,end,number)),ns,(contract,))
+              add((*events, (target_time, end, number)), ns, (contract,))
-            elif self.harmony.evaluate(ns,complete=True,expressions=(contract,),infer=False).valid:
+            elif self.harmony.evaluate(ns, complete=True, expressions=(contract,), infer=False).valid:
-              add((*events,(target_time,end,number)),ns,(contract,))
+              add((*events, (target_time, end, number)), ns, (contract,))
     # Complete ordinary triplet groups; do not leave a third-beat residual.
-    if self.allow_triplets and time+1<=self.end:
+    if self.allow_triplets and time + 1 <= self.end:
-      for number in range(self.low,self.high+1):
+      for number in range(self.low, self.high + 1):
-        for direction in (-1,0,1):
+        for direction in (-1, 0, 1):
-          numbers=[number];cursor=time
+          numbers = [number]
+          cursor = time
           for _ in range(2):
-            nxt=next((p for p in range(numbers[-1]+direction,numbers[-1]+direction*4,direction or 1)
+            nxt = next((p for p in range(numbers[-1] + direction, numbers[-1] + direction * 4, direction or 1)
-              if self.low<=p<=self.high and p%12 in self.harmony.permitted(cursor+F(1,3),cursor+F(2,3))),None) if direction else number
+              if self.low <= p <= self.high and p % 12 in self.harmony.permitted(cursor + fractions.Fraction(1, 3), cursor + fractions.Fraction(2, 3))), None) if direction else number
-            if nxt is None:break
+            if nxt is None:
-            numbers.append(nxt);cursor+=F(1,3)
+              break
-          if len(numbers)!=3:continue
+            numbers.append(nxt)
-          ns=tuple(self.note(time+i*F(1,3),time+(i+1)*F(1,3),p) for i,p in enumerate(numbers))
+            cursor += fractions.Fraction(1, 3)
-          if self.harmony.evaluate(ns,complete=True,infer=False).valid:add(tuple((n.start,n.end,n.pitch.number()) for n in ns),ns)
+          if len(numbers) != 3:
-    self._actions[time]=tuple(result);self.counts['valid_structures']+=len(result)
+            continue
+          ns = tuple(self.note(time + i * fractions.Fraction(1, 3), time + (i + 1) * fractions.Fraction(1, 3), p) for i, p in enumerate(numbers))
+          if self.harmony.evaluate(ns, complete=True, infer=False).valid:
+            add(tuple((n.start, n.end, n.pitch.number()) for n in ns), ns)
+    self._actions[time] = tuple(result)
+    self.counts['valid_structures'] += len(result)
     return self._actions[time]
-
+  def extend(self, state, action):
-  def extend(self,state,action):
+    events, notes, contracts = action
-    events,notes,contracts=action
     # Training tokenization merges contiguous silence. Never manufacture a
     # rest-to-rest transition absent from that event representation.
-    if state.events and state.events[-1][2] is None and events[0][2] is None:return None
+    if state.events and state.events[-1][2] is None and events[0][2] is None:
-    if notes and state.previous is not None and abs(notes[0].pitch.number()-state.previous)>12:return None
+      return None
-    if any(c.kind in ('connection','passing') and c.preparation is None for c in contracts):
+    if notes and state.previous is not None and abs(notes[0].pitch.number() - state.previous) > 12:
-      if not state.notes:return None
+      return None
-      contracts=tuple(dataclasses.replace(c,preparation=state.notes[-1]) if c.kind in ('connection','passing') and c.preparation is None else c for c in contracts)
+    if any(c.kind in ('connection', 'passing') and c.preparation is None for c in contracts):
+      if not state.notes:
+        return None
+      contracts = tuple(dataclasses.replace(c, preparation=state.notes[-1]) if c.kind in ('connection', 'passing') and c.preparation is None else c for c in contracts)
       if self.harmony.ordinary(state.notes[-1]):
-        _,error=expressions.validate((state.notes[-1],*notes),contracts,self.harmony)
+        _, error = generator.solo.expressions.validate((state.notes[-1], *notes), contracts, self.harmony, composition=self.composition)
       else:
-        _,error=expressions.validate((*state.notes,*notes),(*state.contracts,*contracts),self.harmony)
+        _, error = generator.solo.expressions.validate((*state.notes, *notes), (*state.contracts, *contracts), self.harmony, composition=self.composition)
-      if error:return None
+      if error:
-    history=state.history;previous=state.previous;cost=state.cost
+        return None
-    for a,b,p in events:
+    history = state.history
-      harmonic,root,phase,source,_=self.context(a)
+    previous = state.previous
-      action=learned.token(b-a,p,previous,root)
+    cost = state.cost
-      context=learned.contexts(harmonic,phase,source,history)
+    for a, b, p in events:
-      cost+=self.model.cost(action,context)
+      harmonic, root, phase, source, _ = self.context(a)
+      action = generator.solo.learned.token(b - a, p, previous, root)
+      context = generator.solo.learned.contexts(harmonic, phase, source, history)
+      cost += self.model.cost(action, context)
       if p is not None:
-        if previous is None:cost+=self.model.register_cost(p)
+        if previous is None:
-        previous=p
+          cost += self.model.register_cost(p)
-      history=(*history,action)[-4:]
+        previous = p
-    return State((*state.events,*events),(*state.notes,*notes),(*state.contracts,*contracts),history,previous,cost)
+      history = (*history, action)[-4:]
-
+    return State((*state.events, *events), (*state.notes, *notes), (*state.contracts, *contracts), history, previous, cost)
   def compose(self):
     for attempt in range(self.attempts):
-      rng=random.Random(self.seed+':'+str(attempt));state=State();time=self.start
+      rng = random.Random(self.seed + ':' + str(attempt))
-      trace=[]
+      state = State()
-      while time<self.end:
+      time = self.start
-        actions=self.actions(time)
+      trace = []
-        options=[s for a in actions if (s:=self.extend(state,a)) is not None]
+      while time < self.end:
+        actions = self.actions(time)
+        options = [s for a in actions if (s := self.extend(state, a)) is not None]
         # Different operator names cannot multiply the probability of identical
         # sounded music. Prefer plain permission when no exception is needed.
-        unique={}
+        unique = {}
         for s in options:
-          signature=event_key(s.events[len(state.events):])
+          signature = event_key(s.events[len(state.events):])
-          if signature not in unique or len(s.contracts)<len(unique[signature].contracts):unique[signature]=s
+          if signature not in unique or len(s.contracts) < len(unique[signature].contracts):
-        options=sorted(unique.values(),key=lambda s:event_key(s.events[len(state.events):]))
+            unique[signature] = s
-        self.counts['scored_continuations']+=len(options)
+        options = sorted(unique.values(), key=lambda s: event_key(s.events[len(state.events):]))
-        if not options:break
+        self.counts['scored_continuations'] += len(options)
-        low=min(s.cost-state.cost for s in options)
+        if not options:
-        weights=[math.exp(-(s.cost-state.cost-low)) for s in options]
+          break
-        draw=rng.random()*sum(weights);selected=options[-1]
+        low = min(s.cost - state.cost for s in options)
-        for s,w in zip(options,weights):
+        weights = [math.exp(-(s.cost - state.cost - low)) for s in options]
-          draw-=w
+        draw = rng.random() * sum(weights)
-          if draw<=0:selected=s;break
+        selected = options[-1]
-        trace.append({'time':str(time),'valid_joint_actions':len(options),
+        for s, w in zip(options, weights):
-          'selected_events':[(str(a),str(b),p) for a,b,p in selected.events[len(state.events):]],
+          draw -= w
-          'negative_log_likelihood':selected.cost-state.cost})
+          if draw <= 0:
-        state=selected;time=state.events[-1][1]
+            selected = s
-      if time==self.end and state.notes:
+            break
-        self.trace=trace;self.completed=(state,);self.counts['attempts']=attempt+1
+        trace.append({'time': str(time), 'valid_joint_actions': len(options),
+          'selected_events': [(str(a), str(b), p) for a, b, p in selected.events[len(state.events):]],
+          'negative_log_likelihood': selected.cost - state.cost})
+        state = selected
+        time = state.events[-1][1]
+      if time == self.end and state.notes:
+        self.trace = trace
+        self.completed = (state,)
+        self.counts['attempts'] = attempt + 1
         return state
     raise SearchExhausted(f'{self.role}: no complete non-silent sample within {self.attempts} deterministic attempts; end={self.end}')

generator/solo/export.py

@@ -1,86 +0,0 @@
-"""Optional MusicXML views of finished production roles; never composes."""
-from __future__ import annotations
-
-import copy
-import json
-from pathlib import Path
-from fractions import Fraction as F
-
-from generator import model, musicxml_writer, score
-
-
-def region(document, part):
-  measures = document['staves'][0]['measures']
-  first = next(i for i, m in enumerate(measures) if m.get('section_label') == part.capitalize() + ' Solo')
-  end = next((i for i in range(first + 1, len(measures)) if measures[i].get('section_label')), len(measures))
-  return first, end - first
-
-
-def excerpt(document, first, count):
-  result = copy.deepcopy(document)
-  staff = result['staves'][0]
-  original = staff['measures']
-  staff['measures'] = original[first:first + count]
-  if not staff['measures']:
-    raise ValueError('Empty solo export region')
-  notes = [n for m in staff['measures'] for n in score.measure_notes(m)]
-  identities = {p['id'] for n in notes for p in n['pitches']}
-  events = {n['id'] for n in notes}
-  for note in notes:
-    for pitch in note['pitches']:
-      for key in ('tie_from', 'tie_to'):
-        if pitch.get(key) not in identities:
-          pitch.pop(key, None)
-  staff['tuplets'] = [t for t in staff.get('tuplets', []) if t['start'] in events and t['end'] in events]
-  tuplets = {t['id'] for t in staff['tuplets']}
-  for note in notes:
-    if 'tuplet_ids' in note:
-      note['tuplet_ids'] = [i for i in note['tuplet_ids'] if i in tuplets]
-  offset = sum((model.fraction(m['duration']) for m in original[:first]), F())
-  end = offset + sum((model.fraction(m['duration']) for m in staff['measures']), F())
-  for key in ('scale_spans', 'octave_shifts'):
-    staff[key] = [{**s, 'start': model.rational(max(offset, model.fraction(s['start'])) - offset),
-      'end': model.rational(min(end, model.fraction(s['end'])) - offset)} for s in staff.get(key, [])
-      if model.fraction(s['start']) < end and model.fraction(s['end']) > offset]
-  model.validate_document(result)
-  return result
-
-
-def write_role(result, path):
-  role = result['role']
-  if role not in ('acoustic_solo', 'electric_solo'):
-    raise ValueError('Solo XML export requires an acoustic_solo or electric_solo role')
-  part = role.split('_')[0]
-  document = result['document']
-  first, count = region(document, part)
-  leadin=int(result.get('details',{}).get('allocation',{}).get('lead_in_measures',0))
-  first-=leadin;count+=leadin
-  selected = excerpt(document, first, count)
-  numbers = [m['number'] for m in selected['staves'][0]['measures']]
-  for i, measure in enumerate(selected['staves'][0]['measures'], 1):
-    measure['number'] = str(i)
-  path = Path(path)
-  path.parent.mkdir(parents=True, exist_ok=True)
-  musicxml_writer.write_musicxml(path, selected, tablature=True)
-  return {'role': role, 'file': path.name, 'form_start_measure': 1+leadin,
-    'measure_count': count, 'arrangement_measures': numbers,
-    'arrangement_start_beat': str(sum((model.fraction(m['duration']) for m in document['staves'][0]['measures'][:first]), F())),
-    'pitch_convention': 'guitar-written; explicit octave transposition -1',
-    'trace': result['trace'], 'phrase_report': result.get('details', {})}
-
-
-def export_tree(assets, output=None):
-  assets = Path(assets)
-  output = Path(output) if output else assets / 'solo_xml'
-  output.mkdir(parents=True, exist_ok=True)
-  entries = {}
-  for path in sorted((assets / 'analysis').glob('*.roles')):
-    roles = json.loads(path.read_text())
-    for i, part in enumerate(('acoustic', 'electric'), 1):
-      name = path.stem.replace('_', '-') + f'-solo{i}.xml'
-      entries[path.stem + '/' + part] = write_role(roles[part + '_solo'], output / name)
-  manifest = {'version': 1, 'composition_inputs': False, 'solos': entries}
-  # Runtime score discovery treats *.json under assets as score documents.
-  # Diagnostic manifests deliberately use a non-score extension, like .roles.
-  (output / 'manifest.solo').write_text(json.dumps(manifest, sort_keys=True, indent=2) + '\n')
-  return manifest

generator/solo/expressions.py

@@ -1,228 +1,297 @@
 """Typed, target-driven harmonic expression against written harmony and explicit musical relationships."""
 from __future__ import annotations
 import dataclasses
+import fractions
 import functools
-from fractions import Fraction as F
+
-from generator import harmony as chord_model, model, pitch as spelling, vocabulary
+import generator.harmony
-from generator.solo import types
+import generator.model
+import generator.pitch
+import generator.solo.types
+import generator.vocabulary
 
 @functools.lru_cache(maxsize=512)
 def parsed_chord(symbol):
   """Private read-only chord analysis shared by pure expression operations."""
-  return chord_model.parse_symbol(symbol)
+  return generator.harmony.parse_symbol(symbol)
-
 
 @functools.lru_cache(maxsize=512)
-def degree_spelling(letter,accidental,degree,alter):
+def degree_spelling(letter, accidental, degree, alter):
-  root={'letter':letter,'alter':model.rational(accidental),'octave':4}
+  root = {'letter': letter, 'alter': generator.model.rational(accidental), 'octave': 4}
-  p=spelling.shift_pitch(root,degree-1,chord_model.degree_semitones(degree,alter))
+  p = generator.pitch.shift_pitch(root, degree - 1, generator.harmony.degree_semitones(degree, alter))
-  return types.Pitch(p['letter'],model.alteration(p['alter']),p['octave'])
+  return generator.solo.types.Pitch(p['letter'], generator.model.alteration(p['alter']), p['octave'])
-
 
 def quality(chord):
-  degrees=chord_model.degrees(chord)
+  degrees = generator.harmony.degrees(chord)
-  if degrees.get(3)==0 and degrees.get(7)==-1:return 'dominant'
+  if degrees.get(3) == 0 and degrees.get(7) == -1:
-  if degrees.get(3)==-1:
+    return 'dominant'
-    if degrees.get(5)==-1:return 'half_diminished' if degrees.get(7)==-1 else 'diminished'
+  if degrees.get(3) == -1:
+    if degrees.get(5) == -1:
+      return 'half_diminished' if degrees.get(7) == -1 else 'diminished'
     return 'minor'
-  if chord['kind'] in ('suspended-second','suspended-fourth'):return chord['kind']
+  if chord['kind'] in ('suspended-second', 'suspended-fourth'):
-  if degrees.get(3)==0 and degrees.get(5)==0:return 'major'
+    return chord['kind']
+  if degrees.get(3) == 0 and degrees.get(5) == 0:
+    return 'major'
   return 'other'
 
-
+def degree_pitch(chord, degree, alter, near):
-def degree_pitch(chord,degree,alter,near):
+  value = degree_spelling(chord['root']['letter'], generator.model.alteration(chord['root']['alter']), degree, alter)
-  value=degree_spelling(chord['root']['letter'],model.alteration(chord['root']['alter']),degree,alter)
+  return dataclasses.replace(value, octave=value.octave + round((near - value.number()) / 12))
-  return dataclasses.replace(value,octave=value.octave+round((near-value.number())/12))
-
 
 @functools.lru_cache(maxsize=1024)
-def chromatic(target,delta):
+def chromatic(target, delta):
   # Chromatic neighbors are spelled relative to their exact destination.
-  if delta in (-3,-2,-1,1,2,3):
+  if delta in (-3, -2, -1, 1, 2, 3):
-    step=(1 if delta>0 else -1)*(2 if abs(delta)==3 else 1)
+    step = (1 if delta > 0 else -1) * (2 if abs(delta) == 3 else 1)
-    root={'letter':target.letter,'alter':model.rational(target.alter),'octave':target.octave}
+    root = {'letter': target.letter, 'alter': generator.model.rational(target.alter), 'octave': target.octave}
-    p=spelling.shift_pitch(root,step,delta)
+    p = generator.pitch.shift_pitch(root, step, delta)
-    return types.Pitch(p['letter'],model.alteration(p['alter']),p['octave'])
+    return generator.solo.types.Pitch(p['letter'], generator.model.alteration(p['alter']), p['octave'])
   raise ValueError('unsupported approach interval')
 
-
+def eligible_color(rule, span, harmony):
-def eligible_color(rule,span,harmony):
   # An explicit no-chord span (or missing harmony) has no optional colors.
   # In particular, looking ahead for an anticipation must not try to parse
   # the display label N.C. as a written chord.
-  if not span or span.chord=='N.C.' or not span.chord:return False
+  if not span or span.chord == 'N.C.' or not span.chord:
-  key=(tuple(rule['qualities']),rule['degree'],rule['alter'],rule.get('function'),span.start,span.end,span.chord)
+    return False
-  cache=harmony.color_eligibility
+  key = (tuple(rule['qualities']), rule['degree'], rule['alter'], rule.get('function'), span.start, span.end, span.chord)
-  if key not in cache:cache[key]=_eligible_color(rule,span,harmony)
+  cache = harmony.color_eligibility
+  if key not in cache:
+    cache[key] = _eligible_color(rule, span, harmony)
   return cache[key]
 
-
+def _eligible_color(rule, span, harmony):
-def _eligible_color(rule,span,harmony):
+  chord = parsed_chord(span.chord)
-  chord=parsed_chord(span.chord)
+  degrees = generator.harmony.degrees(chord)
-  degrees=chord_model.degrees(chord)
+  if quality(chord) not in rule['qualities']:
-  if quality(chord) not in rule['qualities']:return False
+    return False
-  degree=rule['degree']; simple=(degree-1)%7+1
+  degree = rule['degree']
+  simple = (degree - 1) % 7 + 1
   # Explicit alterations and omissions take precedence over optional colors.
-  if any((d['value']-1)%7+1==simple and (d['type']=='subtract' or model.alteration(d['alter'])!=rule['alter']) for d in chord.get('degrees',[])):return False
+  if any((d['value'] - 1) % 7 + 1 == simple and (d['type'] == 'subtract' or generator.model.alteration(d['alter']) != rule['alter']) for d in chord.get('degrees', [])):
-  if degree in degrees and degrees[degree]!=rule['alter']:return False
+    return False
-  following=next((s for s in harmony.spans if s.start>=span.end and s.chord!=span.chord and s.chord!='N.C.'),None)
+  if degree in degrees and degrees[degree] != rule['alter']:
-  root=int(model.NATURAL[chord['root']['letter']]+model.alteration(chord['root']['alter']))%12
+    return False
-  next_chord=parsed_chord(following.chord) if following else None
+  following = next((s for s in harmony.spans if s.start >= span.end and s.chord != span.chord and s.chord != 'N.C.'), None)
-  next_root=int(model.NATURAL[next_chord['root']['letter']]+model.alteration(next_chord['root']['alter']))%12 if next_chord else None
+  root = int(generator.model.NATURAL[chord['root']['letter']] + generator.model.alteration(chord['root']['alter'])) % 12
-  function=rule.get('function')
+  next_chord = parsed_chord(following.chord) if following else None
-  if function=='predominant_to_dominant' and not (next_chord and quality(next_chord)=='dominant' and (next_root-root)%12==5):return False
+  next_root = int(generator.model.NATURAL[next_chord['root']['letter']] + generator.model.alteration(next_chord['root']['alter'])) % 12 if next_chord else None
-  if function=='resolving_dominant' and (next_root is None or (root-next_root)%12!=7):return False
+  function = rule.get('function')
-  if function=='tonic_or_predominant':
+  if function == 'predominant_to_dominant' and not (next_chord and quality(next_chord) == 'dominant' and (next_root - root) % 12 == 5):
-    tonic=span.key.get('tonic')
+    return False
-    if not tonic:return False
+  if function == 'resolving_dominant' and (next_root is None or (root - next_root) % 12 != 7):
-    tonic_pc=int(model.NATURAL[tonic['letter']]+model.alteration(tonic['alter']))%12
+    return False
-    if root!=tonic_pc and not (next_chord and quality(next_chord)=='dominant' and (next_root-root)%12==5):return False
+  if function == 'tonic_or_predominant':
+    tonic = span.key.get('tonic')
+    if not tonic:
+      return False
+    tonic_pc = int(generator.model.NATURAL[tonic['letter']] + generator.model.alteration(tonic['alter'])) % 12
+    if root != tonic_pc and not (next_chord and quality(next_chord) == 'dominant' and (next_root - root) % 12 == 5):
+      return False
   return True
 
-
+def validate(notes, contracts, harmony, *, composition=None):
-def validate(notes,contracts,harmony):
   """Return event permissions only after checking complete actual gestures."""
-  actual={n.event_id:n for n in notes};approved={}
+  composition = composition or generator.vocabulary.current()
+  actual = {n.event_id: n for n in notes}
+  approved = {}
+
   def same(expected):
-    n=actual.get(expected.event_id)
+    n = actual.get(expected.event_id)
-    return n and (n.start,n.end,n.pitch)==(expected.start,expected.end,expected.pitch)
+    return n and (n.start, n.end, n.pitch) == (expected.start, expected.end, expected.pitch)
+
   for c in contracts:
-    if not c.events or not all(same(n) for n in (*c.events,c.target)):
+    if not c.events or not all(same(n) for n in (*c.events, c.target)):
-      return {},c.id+': missing or changed expression event/target'
+      return {}, c.id + ': missing or changed expression event/target'
-    if c.preparation and not same(c.preparation):return {},c.id+': changed preparation'
+    if c.preparation and not same(c.preparation):
-    if any(n.event_id in approved for n in c.events):return {},c.id+': overlapping expression permissions'
+      return {}, c.id + ': changed preparation'
-    target=c.target; span=harmony.at(c.boundary if c.boundary is not None else c.events[0].start if c.kind=='color' else target.start)
+    if any(n.event_id in approved for n in c.events):
-    if not span or span.chord=='N.C.' or span.chord!=c.chord:return {},c.id+': wrong target harmony'
+      return {}, c.id + ': overlapping expression permissions'
-    if c.kind in ('anticipation','anticipated_color'):
+    target = c.target
-      previous=next((s for s in harmony.spans if s.end==c.boundary),None)
+    span = harmony.at(c.boundary if c.boundary is not None else c.events[0].start if c.kind == 'color' else target.start)
-      if not previous or previous.chord.strip('()')==span.chord.strip('()'):return {},c.id+': anticipation requires a harmonic change'
+    if not span or span.chord == 'N.C.' or span.chord != c.chord:
-    if c.kind in ('approach','enclosure','connection'):
+      return {}, c.id + ': wrong target harmony'
-      rule=next((r for r in vocabulary.data('lead_ins')['patterns'] if r['id']==c.rule and r['kind']==c.kind),None)
+    if c.kind in ('anticipation', 'anticipated_color'):
-      if not rule or len(c.events)!=len(rule['semitones']):return {},c.id+': unknown approach formula'
+      previous = next((s for s in harmony.spans if s.end == c.boundary), None)
-      arrival=c.boundary if c.boundary is not None else target.start
+      if not previous or previous.chord.strip('()') == span.chord.strip('()'):
-      if c.boundary is not None and not any(a.kind=='anticipation' and a.target==target and a.boundary==c.boundary for a in contracts):return {},c.id+': future target lacks anticipation contract'
+        return {}, c.id + ': anticipation requires a harmonic change'
-      planned_color=any(a.kind=='color' and a.target==target and a.events==(target,) for a in contracts)
+    if c.kind in ('approach', 'enclosure', 'connection'):
-      if not planned_color and (target.pitch.number()%12 not in harmony.members(arrival,target.end) or target.pitch.number()%12 not in harmony.permitted(arrival,target.end)):return {},c.id+': invalid destination hold'
+      rule = next((r for r in composition.data('lead_ins')['patterns'] if r['id'] == c.rule and r['kind'] == c.kind), None)
-      gaps=tuple(map(F,rule.get('internal_gaps',['0']*(len(c.events)-1))))
+      if not rule or len(c.events) != len(rule['semitones']):
-      if len(gaps)!=len(c.events)-1 or any(g<0 for g in gaps):return {},c.id+': invalid internal approach gaps'
+        return {}, c.id + ': unknown approach formula'
-      gap=F(rule['gap']);cursor=target.start-gap-sum((F(d) for d in rule['durations']),F())-sum(gaps,F())
+      arrival = c.boundary if c.boundary is not None else target.start
-      if target.start-cursor>F(vocabulary.data('lead_ins')['max_approach']):return {},c.id+': approach too long'
+      if c.boundary is not None and not any(a.kind == 'anticipation' and a.target == target and a.boundary == c.boundary for a in contracts):
-      for i,(n,step,duration) in enumerate(zip(c.events,rule['semitones'],rule['durations'])):
+        return {}, c.id + ': future target lacks anticipation contract'
-        if n.start!=cursor or n.end!=cursor+F(duration) or n.pitch.number()!=target.pitch.number()+step:return {},c.id+': wrong register, interval, or timing'
+      planned_color = any(a.kind == 'color' and a.target == target and a.events == (target,) for a in contracts)
-        cursor=n.end+(gaps[i] if i<len(gaps) else 0)
+      if not planned_color and (target.pitch.number() % 12 not in harmony.members(arrival, target.end) or target.pitch.number() % 12 not in harmony.permitted(arrival, target.end)):
-      for index in rule.get('diatonic_indices',[]):
+        return {}, c.id + ': invalid destination hold'
-        n=c.events[index]
+      gaps = tuple(map(fractions.Fraction, rule.get('internal_gaps', ['0'] * (len(c.events) - 1))))
-        if n.pitch.number()%12 not in harmony.permitted(n.start,n.end):return {},c.id+': declared diatonic neighbor is outside source-key/chord motion'
+      if len(gaps) != len(c.events) - 1 or any(g < 0 for g in gaps):
-      if any(n.start<c.target.start and n.end>c.events[0].start and n.event_id not in {e.event_id for e in c.events} for n in notes):return {},c.id+': interrupted approach'
+        return {}, c.id + ': invalid internal approach gaps'
-      if c.kind=='connection' and (not c.preparation or c.preparation.end!=c.events[0].start or abs(c.preparation.pitch.number()-c.events[0].pitch.number())>2):return {},c.id+': chromatic connection lacks its incoming motion'
+      gap = fractions.Fraction(rule['gap'])
-    elif c.kind=='passing':
+      cursor = target.start - gap - sum((fractions.Fraction(d) for d in rule['durations']), fractions.Fraction()) - sum(gaps, fractions.Fraction())
-      rule=vocabulary.data('lead_ins')['passing']
+      if target.start - cursor > fractions.Fraction(composition.data('lead_ins')['max_approach']):
-      if c.rule!=rule['id'] or len(c.events)!=1 or not c.preparation:return {},c.id+': unsupported passing figure'
+        return {}, c.id + ': approach too long'
-      n,=c.events;before=c.preparation
+      for i, (n, step, duration) in enumerate(zip(c.events, rule['semitones'], rule['durations'])):
-      left=n.pitch.number()-before.pitch.number();right=target.pitch.number()-n.pitch.number()
+        if n.start != cursor or n.end != cursor + fractions.Fraction(duration) or n.pitch.number() != target.pitch.number() + step:
-      if before.end!=n.start or n.end!=target.start or n.end-n.start>F(rule['max_note']) or left!=right or left not in rule['steps']:
+          return {}, c.id + ': wrong register, interval, or timing'
-        return {},c.id+': interrupted/nonstepwise passing figure'
+        cursor = n.end + (gaps[i] if i < len(gaps) else 0)
+      for index in rule.get('diatonic_indices', []):
+        n = c.events[index]
+        if n.pitch.number() % 12 not in harmony.permitted(n.start, n.end):
+          return {}, c.id + ': declared diatonic neighbor is outside source-key/chord motion'
+      if any(n.start < c.target.start and n.end > c.events[0].start and n.event_id not in {e.event_id for e in c.events} for n in notes):
+        return {}, c.id + ': interrupted approach'
+      if c.kind == 'connection' and (not c.preparation or c.preparation.end != c.events[0].start or abs(c.preparation.pitch.number() - c.events[0].pitch.number()) > 2):
+        return {}, c.id + ': chromatic connection lacks its incoming motion'
+    elif c.kind == 'passing':
+      rule = composition.data('lead_ins')['passing']
+      if c.rule != rule['id'] or len(c.events) != 1 or not c.preparation:
+        return {}, c.id + ': unsupported passing figure'
+      n, = c.events
+      before = c.preparation
+      left = n.pitch.number() - before.pitch.number()
+      right = target.pitch.number() - n.pitch.number()
+      if before.end != n.start or n.end != target.start or n.end - n.start > fractions.Fraction(rule['max_note']) or left != right or left not in rule['steps']:
+        return {}, c.id + ': interrupted/nonstepwise passing figure'
+
       def supported(endpoint):
-        return harmony.ordinary(endpoint) or any(a.kind in ('color','anticipated_color') and endpoint in a.events for a in contracts)
+        return harmony.ordinary(endpoint) or any(a.kind in ('color', 'anticipated_color') and endpoint in a.events for a in contracts)
-      if not supported(before) or not supported(target):return {},c.id+': unsupported passing endpoints'
+
-    elif c.kind=='delayed_approach':
+      if not supported(before) or not supported(target):
-      rule=vocabulary.data('lead_ins').get('delayed_approach',{})
+        return {}, c.id + ': unsupported passing endpoints'
-      if c.rule!=rule.get('id') or len(c.events)!=1:return {},c.id+': unknown delayed approach'
+    elif c.kind == 'delayed_approach':
-      n,=c.events
+      rule = composition.data('lead_ins').get('delayed_approach', {})
-      if n.end-n.start>F(rule['max_note']) or not n.end<=target.start<=n.start+F(rule['deadline']):return {},c.id+': expired delayed approach'
+      if c.rule != rule.get('id') or len(c.events) != 1:
-      if target.pitch.number()-n.pitch.number() not in rule['steps']:return {},c.id+': wrong delayed target register'
+        return {}, c.id + ': unknown delayed approach'
-      if target.pitch.number()%12 not in harmony.members(target.start,target.end) or target.pitch.number()%12 not in harmony.permitted(target.start,target.end):return {},c.id+': invalid delayed destination'
+      n, = c.events
-    elif c.kind=='color':
+      if n.end - n.start > fractions.Fraction(rule['max_note']) or not n.end <= target.start <= n.start + fractions.Fraction(rule['deadline']):
-      rule=next((r for r in vocabulary.data('harmony')['colors'] if r['id']==c.rule),None)
+        return {}, c.id + ': expired delayed approach'
-      if not rule:return {},c.id+': unknown color policy'
+      if target.pitch.number() - n.pitch.number() not in rule['steps']:
-      colored=False
+        return {}, c.id + ': wrong delayed target register'
+      if target.pitch.number() % 12 not in harmony.members(target.start, target.end) or target.pitch.number() % 12 not in harmony.permitted(target.start, target.end):
+        return {}, c.id + ': invalid delayed destination'
+    elif c.kind == 'color':
+      rule = next((r for r in composition.data('harmony')['colors'] if r['id'] == c.rule), None)
+      if not rule:
+        return {}, c.id + ': unknown color policy'
+      colored = False
       for n in c.events:
-        if n.end-n.start>F(rule['max_beats']):return {},c.id+': excessive color duration'
+        if n.end - n.start > fractions.Fraction(rule['max_beats']):
-        for region,a,b in harmony.segments(n):
+          return {}, c.id + ': excessive color duration'
-          if n.pitch.number()%12 in region.roles.basic|region.roles.specified:continue
+        for region, a, b in harmony.segments(n):
-          if region.chord=='N.C.' or not eligible_color(rule,region,harmony):return {},c.id+': incompatible color harmony/function'
+          if n.pitch.number() % 12 in region.roles.basic | region.roles.specified:
-          p=degree_pitch(parsed_chord(region.chord),rule['degree'],rule['alter'],n.pitch.number())
+            continue
+          if region.chord == 'N.C.' or not eligible_color(rule, region, harmony):
+            return {}, c.id + ': incompatible color harmony/function'
+          p = degree_pitch(parsed_chord(region.chord), rule['degree'], rule['alter'], n.pitch.number())
           # The rule retains the functional degree/spelling. Encoded
           # enharmonic notation must not change sounding harmonic validity.
-          if p.number()!=n.pitch.number():return {},c.id+': color degree/register mismatch'
+          if p.number() != n.pitch.number():
-          colored=True
+            return {}, c.id + ': color degree/register mismatch'
-      if not colored:return {},c.id+': no interval expresses the declared color'
+          colored = True
-      last=c.events[-1]
+      if not colored:
+        return {}, c.id + ': no interval expresses the declared color'
+      last = c.events[-1]
       if rule.get('resolve'):
-        if target.event_id==last.event_id or not 0<=target.start-last.end<=F(vocabulary.data('harmony')['resolution_gap']) or not 0<abs(target.pitch.number()-last.pitch.number())<=2 or target.pitch.number()%12 not in harmony.members(target.start,target.end):return {},c.id+': unresolved color'
+        if target.event_id == last.event_id or not 0 <= target.start - last.end <= fractions.Fraction(composition.data('harmony')['resolution_gap']) or not 0 < abs(target.pitch.number() - last.pitch.number()) <= 2 or target.pitch.number() % 12 not in harmony.members(target.start, target.end):
-    elif c.kind=='suspension':
+          return {}, c.id + ': unresolved color'
-      rule=next((r for r in vocabulary.data('harmony').get('suspensions',[]) if r['id']==c.rule),None)
+    elif c.kind == 'suspension':
-      if not rule or len(c.events)!=1 or c.boundary is None:return {},c.id+': unsupported suspension'
+      rule = next((r for r in composition.data('harmony').get('suspensions', []) if r['id'] == c.rule), None)
-      held,=c.events
+      if not rule or len(c.events) != 1 or c.boundary is None:
-      if not held.start<=c.boundary-F(rule['preparation'])<c.boundary<held.end:
+        return {}, c.id + ': unsupported suspension'
-        return {},c.id+': insufficient sounding preparation'
+      held, = c.events
-      if held.pitch.number()%12 not in harmony.members(held.start,c.boundary):return {},c.id+': nonchordal preparation'
+      if not held.start <= c.boundary - fractions.Fraction(rule['preparation']) < c.boundary < held.end:
-      if held.end-c.boundary>F(rule['max_delay']) or target.start!=held.end or target.pitch.number()-held.pitch.number() not in rule['steps']:
+        return {}, c.id + ': insufficient sounding preparation'
-        return {},c.id+': missed suspension resolution'
+      if held.pitch.number() % 12 not in harmony.members(held.start, c.boundary):
-      if target.pitch.number()%12 not in harmony.members(target.start,target.end):return {},c.id+': invalid suspension target'
+        return {}, c.id + ': nonchordal preparation'
-      if held.pitch.number()%12 not in harmony.permitted(held.start,held.end):return {},c.id+': suspension outside source-key/chord motion'
+      if held.end - c.boundary > fractions.Fraction(rule['max_delay']) or target.start != held.end or target.pitch.number() - held.pitch.number() not in rule['steps']:
-    elif c.kind=='anticipated_color':
+        return {}, c.id + ': missed suspension resolution'
-      rule=next((r for r in vocabulary.data('harmony')['colors'] if r['id']==c.rule),None)
+      if target.pitch.number() % 12 not in harmony.members(target.start, target.end):
-      if not rule or len(c.events)!=1 or c.boundary!=span.start:return {},c.id+': unsupported anticipated color'
+        return {}, c.id + ': invalid suspension target'
-      n,=c.events
+      if held.pitch.number() % 12 not in harmony.permitted(held.start, held.end):
-      if not n.start<c.boundary<n.end or c.boundary-n.start>F(vocabulary.data('harmony')['anticipation_max']):return {},c.id+': invalid anticipated color interval'
+        return {}, c.id + ': suspension outside source-key/chord motion'
-      arrived=dataclasses.replace(n,start=c.boundary)
+    elif c.kind == 'anticipated_color':
-      target=arrived if c.target==n else c.target
+      rule = next((r for r in composition.data('harmony')['colors'] if r['id'] == c.rule), None)
-      inner=types.ExpressionContract(c.id+':arrived','color',rule['id'],(arrived,),target,span.chord)
+      if not rule or len(c.events) != 1 or c.boundary != span.start:
-      _,error=validate((arrived,*((target,) if target!=arrived else ())), (inner,),harmony)
+        return {}, c.id + ': unsupported anticipated color'
-      if error:return {},error
+      n, = c.events
-    elif c.kind=='anticipation':
+      if not n.start < c.boundary < n.end or c.boundary - n.start > fractions.Fraction(composition.data('harmony')['anticipation_max']):
-      if c.rule not in ('early_chord_tone','early_reattack') or len(c.events)!=1 or span.start!=c.boundary:
+        return {}, c.id + ': invalid anticipated color interval'
-        return {},c.id+': unknown anticipation or non-boundary target'
+      arrived = dataclasses.replace(n, start=c.boundary)
-      n,=c.events
+      target = arrived if c.target == n else c.target
-      if c.boundary is None or not n.start<c.boundary or c.boundary-n.start>F(vocabulary.data('harmony')['anticipation_max']):return {},c.id+': invalid anticipation interval'
+      inner = generator.solo.types.ExpressionContract(c.id + ':arrived', 'color', rule['id'], (arrived,), target, span.chord)
-      if c.rule=='early_chord_tone' and (n!=target or not c.boundary<n.end):return {},c.id+': invalid held anticipation'
+      _, error = validate((arrived, *((target,) if target != arrived else ())), (inner,), harmony, composition=composition)
-      if c.rule=='early_reattack' and (n.end!=c.boundary or target.start!=c.boundary or target.event_id==n.event_id or target.pitch.number()!=n.pitch.number()):return {},c.id+': invalid anticipated reattack'
+      if error:
-      if target.pitch.number()%12 not in harmony.members(c.boundary,target.end) or target.pitch.number()%12 not in harmony.permitted(c.boundary,target.end):return {},c.id+': invalid future destination'
+        return {}, error
-    elif c.kind=='blues':
+    elif c.kind == 'anticipation':
-      if len(c.events)!=1:return {},c.id+': blues treatment requires one inflected event'
+      if c.rule not in ('early_chord_tone', 'early_reattack') or len(c.events) != 1 or span.start != c.boundary:
-      rule=next((r for r in vocabulary.data('lead_ins')['blues'] if r['id']==c.rule),None)
+        return {}, c.id + ': unknown anticipation or non-boundary target'
-      n,=c.events
+      n, = c.events
-      chord=parsed_chord(span.chord)
+      if c.boundary is None or not n.start < c.boundary or c.boundary - n.start > fractions.Fraction(composition.data('harmony')['anticipation_max']):
-      if not rule or n.end-n.start>F(1,2) or n.end!=target.start or target.pitch.number()%12 not in span.roles.basic:return {},c.id+': unresolved blues inflection'
+        return {}, c.id + ': invalid anticipation interval'
-      expected=degree_pitch(chord,rule['degree'],rule['alter'],n.pitch.number())
+      if c.rule == 'early_chord_tone' and (n != target or not c.boundary < n.end):
-      resolved=degree_pitch(chord,rule['target_degree'],0,target.pitch.number())
+        return {}, c.id + ': invalid held anticipation'
-      if n.pitch!=expected or target.pitch!=resolved or target.pitch.number()-n.pitch.number()!=1:return {},c.id+': unsupported blues destination'
+      if c.rule == 'early_reattack' and (n.end != c.boundary or target.start != c.boundary or target.event_id == n.event_id or target.pitch.number() != n.pitch.number()):
-      if not c.preparation or c.preparation.end!=n.start or abs(c.preparation.pitch.number()-n.pitch.number())>2:return {},c.id+': unprepared blues gesture'
+        return {}, c.id + ': invalid anticipated reattack'
-    else:return {},c.id+': unknown expression kind'
+      if target.pitch.number() % 12 not in harmony.members(c.boundary, target.end) or target.pitch.number() % 12 not in harmony.permitted(c.boundary, target.end):
+        return {}, c.id + ': invalid future destination'
+    elif c.kind == 'blues':
+      if len(c.events) != 1:
+        return {}, c.id + ': blues treatment requires one inflected event'
+      rule = next((r for r in composition.data('lead_ins')['blues'] if r['id'] == c.rule), None)
+      n, = c.events
+      chord = parsed_chord(span.chord)
+      if not rule or n.end - n.start > fractions.Fraction(1, 2) or n.end != target.start or target.pitch.number() % 12 not in span.roles.basic:
+        return {}, c.id + ': unresolved blues inflection'
+      expected = degree_pitch(chord, rule['degree'], rule['alter'], n.pitch.number())
+      resolved = degree_pitch(chord, rule['target_degree'], 0, target.pitch.number())
+      if n.pitch != expected or target.pitch != resolved or target.pitch.number() - n.pitch.number() != 1:
+        return {}, c.id + ': unsupported blues destination'
+      if not c.preparation or c.preparation.end != n.start or abs(c.preparation.pitch.number() - n.pitch.number()) > 2:
+        return {}, c.id + ': unprepared blues gesture'
+    else:
+      return {}, c.id + ': unknown expression kind'
     for n in c.events:
-      covered=n.start
+      covered = n.start
-      for region,a,b in harmony.segments(n):
+      for region, a, b in harmony.segments(n):
-        if a!=covered or region.chord=='N.C.':return {},c.id+': missing harmony'
+        if a != covered or region.chord == 'N.C.':
-        covered=b
+          return {}, c.id + ': missing harmony'
-      if covered!=n.end:return {},c.id+': incomplete harmonic coverage'
+        covered = b
-      approved[n.event_id]=c
+      if covered != n.end:
-  return approved,''
+        return {}, c.id + ': incomplete harmonic coverage'
+      approved[n.event_id] = c
+  return approved, ''
 
-
+def validate_frontier(notes, contracts, harmony, until, complete, *, composition=None):
-def validate_frontier(notes,contracts,harmony,until,complete):
   """Validate heard prefixes and bounded future obligations separately.
 
   Future contract events are used ONLY to establish rule feasibility. They do
   not count as heard resolution; the contract stays pending until its entire
   target hold is present. Missing past events and expired deadlines fail.
   """
+  composition = composition or generator.vocabulary.current()
   if complete:
-    approved,error=validate(notes,contracts,harmony)
+    approved, error = validate(notes, contracts, harmony, composition=composition)
-    return approved,(),error
+    return approved, (), error
-  actual={n.event_id:n for n in notes};future=dict(actual);pending=[]
+  actual = {n.event_id: n for n in notes}
+  future = dict(actual)
+  pending = []
   for c in contracts:
-    required=tuple(dict.fromkeys((*c.events,c.target,*((c.preparation,) if c.preparation else ()))))
+    required = tuple(dict.fromkeys((*c.events, c.target, *((c.preparation,) if c.preparation else ()))))
-    incomplete=False
+    incomplete = False
     for expected in required:
-      heard=actual.get(expected.event_id)
+      heard = actual.get(expected.event_id)
-      if expected.start>=until:
+      if expected.start >= until:
-        future[expected.event_id]=expected;incomplete=True
+        future[expected.event_id] = expected
-      elif heard and heard.pitch==expected.pitch and heard.start==expected.start and heard.end==min(until,expected.end):
+        incomplete = True
-        future[expected.event_id]=expected
+      elif heard and heard.pitch == expected.pitch and heard.start == expected.start and heard.end == min(until, expected.end):
-        incomplete |= heard.end<expected.end
+        future[expected.event_id] = expected
-      elif not heard or (heard.start,heard.end,heard.pitch)!=(expected.start,expected.end,expected.pitch):
+        incomplete |= heard.end < expected.end
-        return {},(),c.id+': missing/changed event before resolution deadline'
+      elif not heard or (heard.start, heard.end, heard.pitch) != (expected.start, expected.end, expected.pitch):
-    if incomplete:pending.append(c)
+        return {}, (), c.id + ': missing/changed event before resolution deadline'
-  approved,error=validate(tuple(sorted(future.values(),key=lambda n:(n.start,n.event_id))),contracts,harmony)
+    if incomplete:
-  return {key:c for key,c in approved.items() if key in actual},tuple(pending),error
+      pending.append(c)
+  approved, error = validate(tuple(sorted(future.values(), key=lambda n: (n.start, n.event_id))), contracts, harmony, composition=composition)
+  return {key: c for key, c in approved.items() if key in actual}, tuple(pending), error

generator/solo/groups.py

@@ -7,59 +7,66 @@ not a rhythmic boundary and must not be used to clip a group.
 from __future__ import annotations
 import dataclasses
 import functools
-from fractions import Fraction as F
+import fractions
-
 
 @dataclasses.dataclass(frozen=True)
 class RhythmGroup:
-  start: F
+  start: fractions.Fraction
-  end: F
+  end: fractions.Fraction
   family: str
-  events: tuple[tuple[F, F], ...]
+  events: tuple[tuple[fractions.Fraction, fractions.Fraction], ...]
-  displacement: F = F()
+  displacement: fractions.Fraction = fractions.Fraction()
   source: str = ''
 
-
+def describe(pattern, *, origin=fractions.Fraction(), source='', imported=False):
-def describe(pattern, *, origin=F(), source='', imported=False):
+  if not pattern:
-  if not pattern:return ()
+    return ()
   if imported:
-    return (RhythmGroup(origin, origin+pattern[-1][1], 'source', pattern, source=source),)
+    return (RhythmGroup(origin, origin + pattern[-1][1], 'source', pattern, source=source),)
-  result=[]; i=0
+  result = []
-  while i<len(pattern):
+  i = 0
-    a,b=pattern[i]
+  while i < len(pattern):
-    if b<=a or i and a<pattern[i-1][1]:raise ValueError('overlapping rhythm group')
+    a, b = pattern[i]
+    if b <= a or i and a < pattern[i - 1][1]:
+      raise ValueError('overlapping rhythm group')
     # A complete ternary figure can be intentionally displaced on an eighth.
-    if (b-a).denominator%3==0:
+    if (b - a).denominator % 3 == 0:
-      if (a*2).denominator!=1:raise ValueError('unexplained tuplet entry phase')
+      if (a * 2).denominator != 1:
-      start=i; cursor=a
+        raise ValueError('unexplained tuplet entry phase')
-      while i<len(pattern):
+      start = i
-        x,y=pattern[i]
+      cursor = a
-        if x!=cursor or (y-x).denominator not in (3,6):break
+      while i < len(pattern):
-        cursor=y;i+=1
+        x, y = pattern[i]
-        if (cursor-a).denominator==1:break
+        if x != cursor or (y - x).denominator not in (3, 6):
-      if i==start or (cursor-a).denominator!=1:
+          break
+        cursor = y
+        i += 1
+        if (cursor - a).denominator == 1:
+          break
+      if i == start or (cursor - a).denominator != 1:
         raise ValueError('incomplete tuplet group')
-      group=pattern[start:i]
+      group = pattern[start:i]
-      result.append(RhythmGroup(origin+a,origin+cursor,'triplet',group,a%1,source))
+      result.append(RhythmGroup(origin + a, origin + cursor, 'triplet', group, a % 1, source))
     else:
-      if (a*4).denominator!=1 or (b*4).denominator!=1:
+      if (a * 4).denominator != 1 or (b * 4).denominator != 1:
         raise ValueError('unexplained binary subdivision phase')
-      family='sixteenth' if (a*2).denominator!=1 or (b*2).denominator!=1 else 'binary'
+      family = 'sixteenth' if (a * 2).denominator != 1 or (b * 2).denominator != 1 else 'binary'
-      result.append(RhythmGroup(origin+a,origin+b,family,((a,b),),a%1,source));i+=1
+      result.append(RhythmGroup(origin + a, origin + b, family, ((a, b),), a % 1, source))
+      i += 1
   return tuple(result)
 
-
 @functools.lru_cache(maxsize=8192)
 def valid(pattern):
-  try:describe(pattern);return True
+  try:
-  except ValueError:return False
+    describe(pattern)
-
+    return True
+  except ValueError:
+    return False
 
 def signature(notes):
   """Translation-invariant duration, inter-onset and rest vocabulary."""
-  return tuple((n.end-n.start, notes[i+1].start-n.end if i+1<len(notes) else F())
+  return tuple((n.end - n.start, notes[i + 1].start - n.end if i + 1 < len(notes) else fractions.Fraction())
-    for i,n in enumerate(notes))
+    for i, n in enumerate(notes))
-
 
 def descriptor(pattern, duration):
   """Retain ordered attacks, durations and space, including a trailing breath.
@@ -69,11 +76,12 @@ def descriptor(pattern, duration):
   """
   return (duration, tuple(pattern))
 
-
 def broaden(pattern, arrival, release):
   """Reopen an editable binary tail; never slice a tuplet or inherit its phase."""
-  if (arrival*2).denominator!=1 or (release*2).denominator!=1:return ()
+  if (arrival * 2).denominator != 1 or (release * 2).denominator != 1:
+    return ()
   for group in describe(pattern):
-    if group.family=='triplet' and group.start<arrival<group.end:return ()
+    if group.family == 'triplet' and group.start < arrival < group.end:
-  result=tuple((a,min(b,arrival)) for a,b in pattern if a<arrival)+((arrival,release),)
+      return ()
+  result = tuple((a, min(b, arrival)) for a, b in pattern if a < arrival) + ((arrival, release),)
   return result if valid(result) else ()

generator/solo/harmonic.py

@@ -5,128 +5,157 @@ permission follows the actual sounded interval and validated resolution.
 """
 from __future__ import annotations
 import dataclasses
+import fractions
 import functools
-from fractions import Fraction as F
+
-from generator import harmony as chords, model
+import generator.harmony
-from generator.solo import types
+import generator.model
+import generator.solo.expressions
+import generator.solo.profiles
+import generator.solo.recognition
+import generator.solo.types
+import generator.vocabulary
 
 @dataclasses.dataclass(frozen=True)
 class Roles:
-  basic:frozenset
+  basic: frozenset
-  specified:frozenset
+  specified: frozenset
-  degrees:tuple
+  degrees: tuple
 
 @functools.lru_cache(maxsize=512)
-def degree_roles(kind,root,degrees):
+def degree_roles(kind, root, degrees):
-  basic,specified,labels=set(),set(),{}
+  basic, specified, labels = set(), set(), {}
-  for degree,alter in degrees:
+  for degree, alter in degrees:
-    pc=int(root+chords.degree_semitones(degree,alter))%12
+    pc = int(root + generator.harmony.degree_semitones(degree, alter)) % 12
-    structural=degree in (1,3,5,6,7) or degree in (2,4) and kind in ('suspended-second','suspended-fourth')
+    structural = degree in (1, 3, 5, 6, 7) or degree in (2, 4) and kind in ('suspended-second', 'suspended-fourth')
-    (basic if structural else specified).add(pc);labels.setdefault(pc,[]).append(degree)
+    (basic if structural else specified).add(pc)
-  return Roles(frozenset(basic),frozenset(specified-basic),tuple((p,tuple(v)) for p,v in sorted(labels.items())))
+    labels.setdefault(pc, []).append(degree)
+  return Roles(frozenset(basic), frozenset(specified - basic), tuple((p, tuple(v)) for p, v in sorted(labels.items())))
 
 def roles(chord):
-  if not chord or chord['kind']=='none':return Roles(frozenset(),frozenset(),())
+  if not chord or chord['kind'] == 'none':
-  root=chord['root'];pc=model.NATURAL[root['letter']]+model.alteration(root['alter'])
+    return Roles(frozenset(), frozenset(), ())
-  return degree_roles(chord['kind'],pc,tuple(sorted(chords.degrees(chord).items())))
+  root = chord['root']
+  pc = generator.model.NATURAL[root['letter']] + generator.model.alteration(root['alter'])
+  return degree_roles(chord['kind'], pc, tuple(sorted(generator.harmony.degrees(chord).items())))
 
 @dataclasses.dataclass(frozen=True)
 class Span:
-  start:F
+  start: fractions.Fraction
-  end:F
+  end: fractions.Fraction
-  roles:Roles
+  roles: Roles
-  chord:str
+  chord: str
-  key:dict
+  key: dict
-  pitches:frozenset
+  pitches: frozenset
 
 @dataclasses.dataclass(frozen=True)
 class Result:
-  valid:bool
+  valid: bool
-  exposure:float
+  exposure: float
-  pending:tuple=()
+  pending: tuple = ()
-  error:str=''
+  error: str = ''
-  contracts:tuple=()
+  contracts: tuple = ()
 
 class Harmony:
-  def __init__(self,spans,profile=None):
+  def __init__(self, spans, profile=None, *, composition=None):
-    from generator.solo.profiles import UNRESTRICTED
+    self.composition = composition or generator.vocabulary.current()
-    self.profile=profile or UNRESTRICTED
+    self.profile = profile or generator.solo.profiles.UNRESTRICTED
-    self.source_spans=spans;self.spans=[];self.color_eligibility={}
+    self.source_spans = spans
+    self.spans = []
+    self.color_eligibility = {}
     for s in spans:
-      r=roles(s.chord)
+      r = roles(s.chord)
-      key=frozenset((model.NATURAL[l]+int(model.fraction(a)))%12 for l,a in zip(model.LETTERS,s.key['alterations']))
+      key = frozenset((generator.model.NATURAL[l] + int(generator.model.fraction(a))) % 12 for l, a in zip(generator.model.LETTERS, s.key['alterations']))
-      self.spans.append(Span(s.start,s.end,r,chords.symbol(s.chord) if s.chord else 'N.C.',s.key,key|r.basic|r.specified))
+      self.spans.append(Span(s.start, s.end, r, generator.harmony.symbol(s.chord) if s.chord else 'N.C.', s.key, key | r.basic | r.specified))
-    self.spans=tuple(self.spans)
+    self.spans = tuple(self.spans)
   @functools.lru_cache(maxsize=2048)
-  def at(self,time):return next((s for s in self.spans if s.start<=time<s.end),None)
+  def at(self, time): return next((s for s in self.spans if s.start <= time < s.end), None)
-  def segments(self,n):return tuple((s,max(n.start,s.start),min(n.end,s.end)) for s in self.spans if s.start<n.end and s.end>n.start)
+  def segments(self, n): return tuple((s, max(n.start, s.start), min(n.end, s.end)) for s in self.spans if s.start < n.end and s.end > n.start)
   @functools.lru_cache(maxsize=8192)
-  def members(self,start,end):
+  def members(self, start, end):
-    spans=[s for s in self.spans if s.start<end and s.end>start]
+    spans = [s for s in self.spans if s.start < end and s.end > start]
-    return frozenset.intersection(*(s.roles.basic|s.roles.specified for s in spans)) if spans else frozenset()
+    return frozenset.intersection(*(s.roles.basic | s.roles.specified for s in spans)) if spans else frozenset()
   @functools.lru_cache(maxsize=8192)
-  def permitted(self,start,end):
+  def permitted(self, start, end):
-    spans=[s for s in self.spans if s.start<end and s.end>start]
+    spans = [s for s in self.spans if s.start < end and s.end > start]
-    return frozenset.intersection(*(s.pitches if s.chord!='N.C.' else frozenset() for s in spans)) if spans else frozenset()
+    return frozenset.intersection(*(s.pitches if s.chord != 'N.C.' else frozenset() for s in spans)) if spans else frozenset()
-  def function(self,number,span):
+  def function(self, number, span):
-    pc=number%12
+    pc = number % 12
     return 'basic' if pc in span.roles.basic else 'specified_extension' if pc in span.roles.specified else 'diatonic_motion' if pc in span.pitches else 'chromatic'
-  def step(self,a,b,time):return 0<abs(a-b)<=2
+  def step(self, a, b, time): return 0 < abs(a - b) <= 2
-  def connected_motion(self,notes,index):
+  def connected_motion(self, notes, index):
-    if not 0<index<len(notes)-1:return ''
+    if not 0 < index < len(notes) - 1:
-    a,n,b=notes[index-1:index+2]
+      return ''
-    if a.end!=n.start or n.end!=b.start:return ''
+    a, n, b = notes[index - 1:index + 2]
-    left=n.pitch.number()-a.pitch.number();right=b.pitch.number()-n.pitch.number()
+    if a.end != n.start or n.end != b.start:
-    return 'neighbor' if 0<abs(left)<=2 and left==-right else 'passing' if 0<abs(left)<=2 and 0<abs(right)<=2 and left*right>0 else ''
+      return ''
+    left = n.pitch.number() - a.pitch.number()
+    right = b.pitch.number() - n.pitch.number()
+    return 'neighbor' if 0 < abs(left) <= 2 and left == -right else 'passing' if 0 < abs(left) <= 2 and 0 < abs(right) <= 2 and left * right > 0 else ''
   @functools.lru_cache(maxsize=8192)
-  def _ordinary(self,start,end,number):
+  def _ordinary(self, start, end, number):
-    covered=start
+    covered = start
     for s in self.spans:
-      a,b=max(start,s.start),min(end,s.end)
+      a, b = max(start, s.start), min(end, s.end)
-      if a>=b:continue
+      if a >= b:
-      if a!=covered or s.chord=='N.C.':return False
+        continue
-      covered=b;pc=number%12
+      if a != covered or s.chord == 'N.C.':
-      if pc not in s.roles.basic|s.roles.specified and (pc not in s.pitches or end-start>=2):return False
+        return False
-    return covered==end and end>start
+      covered = b
-  def ordinary(self,n):return self._ordinary(n.start,n.end,n.pitch.number())
+      pc = number % 12
+      if pc not in s.roles.basic | s.roles.specified and (pc not in s.pitches or end - start >= 2):
+        return False
+    return covered == end and end > start
+  def ordinary(self, n): return self._ordinary(n.start, n.end, n.pitch.number())
   @staticmethod
   def identified(n):
-    return n if n.event_id else dataclasses.replace(n,event_id=f'event:{n.start}:{n.end}:{n.pitch.label()}')
+    return n if n.event_id else dataclasses.replace(n, event_id=f'event:{n.start}:{n.end}:{n.pitch.label()}')
-  def evaluate(self,notes,*,until=None,complete=False,expressions=(),infer=True):
+  def evaluate(self, notes, *, until=None, complete=False, expressions=(), infer=True):
-    from generator.solo.expressions import validate_frontier
+    notes = tuple(self.identified(n) for n in generator.solo.types.sounding_notes(notes))
-    notes=tuple(self.identified(n) for n in types.sounding_notes(notes))
+    expressions = tuple(dataclasses.replace(c, events=tuple(self.identified(n) for n in c.events),
-    expressions=tuple(dataclasses.replace(c,events=tuple(self.identified(n) for n in c.events),
+      target=self.identified(c.target), preparation=self.identified(c.preparation) if c.preparation else None) for c in expressions)
-      target=self.identified(c.target),preparation=self.identified(c.preparation) if c.preparation else None) for c in expressions)
+    frontier = until if until is not None else max((n.end for n in notes), default=fractions.Fraction())
-    frontier=until if until is not None else max((n.end for n in notes),default=F())
+    permissions, pending, error = generator.solo.expressions.validate_frontier(notes, expressions, self, frontier, complete or until is None, composition=self.composition)
-    permissions,pending,error=validate_frontier(notes,expressions,self,frontier,complete or until is None)
+    if error:
-    if error:return Result(False,0,error=error)
+      return Result(False, 0, error=error)
     if infer and not pending and (complete or until is None) and any(not self.ordinary(n) and n.event_id not in permissions for n in notes):
-      from generator.solo.recognition import recognize
+      expressions, error = generator.solo.recognition.recognize(notes, self, expressions, composition=self.composition)
-      expressions,error=recognize(notes,self,expressions)
+      if error:
-      if error:return Result(False,0,error=error)
+        return Result(False, 0, error=error)
-      permissions,pending,error=validate_frontier(notes,expressions,self,frontier,True)
+      permissions, pending, error = generator.solo.expressions.validate_frontier(notes, expressions, self, frontier, True, composition=self.composition)
-      if error:return Result(False,0,error=error)
+      if error:
-    exposed=sounded=F()
+        return Result(False, 0, error=error)
+    exposed = sounded = fractions.Fraction()
     for n in notes:
-      covered=n.start;duration=n.end-n.start;sounded+=duration
+      covered = n.start
-      for s,a,b in self.segments(n):
+      duration = n.end - n.start
-        if a!=covered or s.chord=='N.C.':return Result(False,0,error=f'{n.event_id}: missing written/carried harmony')
+      sounded += duration
-        covered=b;permission=permissions.get(n.event_id)
+      for s, a, b in self.segments(n):
-        if permission and permission.kind=='anticipation' and a>=permission.boundary:permission=None
+        if a != covered or s.chord == 'N.C.':
-        pc=n.pitch.number()%12
+          return Result(False, 0, error=f'{n.event_id}: missing written/carried harmony')
-        if pc in s.roles.basic|s.roles.specified or permission:continue
+        covered = b
-        if pc not in s.pitches:return Result(False,0,error=f'{n.event_id}: unsupported chromatic event')
+        permission = permissions.get(n.event_id)
-        if duration>=2:return Result(False,0,error=f'{n.event_id}: sustained nonchord tone needs an explicit treatment')
+        if permission and permission.kind == 'anticipation' and a >= permission.boundary:
-        exposed+=(b-a)/4
+          permission = None
-      if covered!=n.end:return Result(False,0,error=f'{n.event_id}: incomplete harmonic coverage')
+        pc = n.pitch.number() % 12
-    return Result(True,float(exposed/max(F(1,1000),sounded)),pending,contracts=tuple(expressions))
+        if pc in s.roles.basic | s.roles.specified or permission:
-  def describe(self,notes,expressions=()):
+          continue
-    notes=tuple(self.identified(n) for n in types.sounding_notes(notes))
+        if pc not in s.pitches:
-    checked=self.evaluate(notes,expressions=expressions,complete=True)
+          return Result(False, 0, error=f'{n.event_id}: unsupported chromatic event')
-    if not checked.valid:raise ValueError(checked.error)
+        if duration >= 2:
-    permissions={n.event_id:c for c in checked.contracts for n in c.events}
+          return Result(False, 0, error=f'{n.event_id}: sustained nonchord tone needs an explicit treatment')
-    return [{'start':str(a),'end':str(b),'attack':str(n.start),'sounding_duration':str(n.end-n.start),
+        exposed += (b - a) / 4
-      'pitch':n.pitch.label(),'chord':s.chord,'function':self.function(n.pitch.number(),s),
+      if covered != n.end:
-      'expression':permissions[n.event_id].kind if n.event_id in permissions else '',
+        return Result(False, 0, error=f'{n.event_id}: incomplete harmonic coverage')
-      'expression_rule':permissions[n.event_id].rule if n.event_id in permissions else '',
+    return Result(True, float(exposed / max(fractions.Fraction(1, 1000), sounded)), pending, contracts=tuple(expressions))
-      'protected':n.role=='quote'} for n in notes for s,a,b in self.segments(n)]
+  def describe(self, notes, expressions=()):
+    notes = tuple(self.identified(n) for n in generator.solo.types.sounding_notes(notes))
+    checked = self.evaluate(notes, expressions=expressions, complete=True)
+    if not checked.valid:
+      raise ValueError(checked.error)
+    permissions = {n.event_id: c for c in checked.contracts for n in c.events}
+    return [{'start': str(a), 'end': str(b), 'attack': str(n.start), 'sounding_duration': str(n.end - n.start),
+      'pitch': n.pitch.label(), 'chord': s.chord, 'function': self.function(n.pitch.number(), s),
+      'expression': permissions[n.event_id].kind if n.event_id in permissions else '',
+      'expression_rule': permissions[n.event_id].rule if n.event_id in permissions else '',
+      'protected': n.role == 'quote'} for n in notes for s, a, b in self.segments(n)]

generator/solo/learned.py

@@ -9,122 +9,127 @@ import collections
 import json
 import math
 import functools
-from fractions import Fraction as F
+import fractions
-from pathlib import Path
+import pathlib
-from generator import model
+import generator.model
-from generator.solo import expressions
+import generator.solo.expressions
 
 VERSION = 'pooled-events-1'
 PRIOR = 8
-DURATIONS = ('1/4','1/3','1/2','2/3','3/4','1','3/2','2','5/2','3','4')
+DURATIONS = ('1/4', '1/3', '1/2', '2/3', '3/4', '1', '3/2', '2', '5/2', '3', '4')
-
 
 def key(value):
-  return json.dumps(value,separators=(',',':'))
+  return json.dumps(value, separators=(',', ':'))
 
+def chord_context(harmony, time):
+  span = harmony.at(time)
+  current = generator.solo.expressions.parsed_chord(span.chord) if span and span.chord != 'N.C.' else None
+  future = next((s for s in harmony.spans if s.start > time and s.chord != 'N.C.' and (not span or s.chord != span.chord)), None)
+  following = generator.solo.expressions.parsed_chord(future.chord) if future else None
 
-def chord_context(harmony,time):
+  def root(c): return int(generator.model.NATURAL[c['root']['letter']] + generator.model.alteration(c['root']['alter'])) % 12 if c else 0
-  span=harmony.at(time)
-  current=expressions.parsed_chord(span.chord) if span and span.chord!='N.C.' else None
-  future=next((s for s in harmony.spans if s.start>time and s.chord!='N.C.' and (not span or s.chord!=span.chord)),None)
-  following=expressions.parsed_chord(future.chord) if future else None
-  def root(c):return int(model.NATURAL[c['root']['letter']]+model.alteration(c['root']['alter']))%12 if c else 0
-  return (expressions.quality(current) if current else 'unknown',
-    expressions.quality(following) if following else 'end',
-    (root(following)-root(current))%12 if current and following else -1),root(current),future
 
+  return (generator.solo.expressions.quality(current) if current else 'unknown',
+    generator.solo.expressions.quality(following) if following else 'end',
+    (root(following) - root(current)) % 12 if current and following else -1), root(current), future
 
-def token(duration,pitch,previous,root):
+def token(duration, pitch, previous, root):
-  return ('rest',str(duration),0,0) if pitch is None else ('note',str(duration),
+  return ('rest', str(duration), 0, 0) if pitch is None else ('note', str(duration), pitch - previous if previous is not None else 'start', (pitch - root) % 12)
-    pitch-previous if previous is not None else 'start',(pitch-root)%12)
 
+def source_feature(analysis, time, root=0):
+  time += analysis.body_offset
+  before = [n for n in analysis.melody if n.start <= time][-2:]
+  following = next((n for n in analysis.melody if n.start > time), None)
+  direction = 0 if len(before) < 2 else (before[-1].pitch.number() > before[-2].pitch.number()) - (before[-1].pitch.number() < before[-2].pitch.number())
+  held = bool(before and before[-1].end > time)
+  upcoming = 0 if not before or not following else (following.pitch.number() > before[-1].pitch.number()) - (following.pitch.number() < before[-1].pitch.number())
+  return (direction, held, upcoming, str(before[-1].end - before[-1].start) if before else 'none',
+    (before[-1].pitch.number() - root) % 12 if before else -1,
+    before[-1].pitch.number() - before[-2].pitch.number() if len(before) > 1 else 0)
 
-def source_feature(analysis,time,root=0):
+def phase_at(analysis, time):
-  time+=analysis.body_offset
+  absolute = time + analysis.body_offset
-  before=[n for n in analysis.melody if n.start<=time][-2:]
+  start = next((a for a, b in zip(analysis.boundaries, analysis.boundaries[1:]) if a <= absolute < b), analysis.boundaries[-2])
-  following=next((n for n in analysis.melody if n.start>time),None)
+  return absolute - start
-  direction=0 if len(before)<2 else (before[-1].pitch.number()>before[-2].pitch.number())-(before[-1].pitch.number()<before[-2].pitch.number())
-  held=bool(before and before[-1].end>time)
-  upcoming=0 if not before or not following else (following.pitch.number()>before[-1].pitch.number())-(following.pitch.number()<before[-1].pitch.number())
-  return (direction,held,upcoming,str(before[-1].end-before[-1].start) if before else 'none',
-    (before[-1].pitch.number()-root)%12 if before else -1,
-    before[-1].pitch.number()-before[-2].pitch.number() if len(before)>1 else 0)
-
-
-def phase_at(analysis,time):
-  absolute=time+analysis.body_offset
-  start=next((a for a,b in zip(analysis.boundaries,analysis.boundaries[1:]) if a<=absolute<b),analysis.boundaries[-2])
-  return absolute-start
-
 
 @functools.lru_cache(maxsize=65536)
-def contexts(harmonic,phase,source,history):
+def contexts(harmonic, phase, source, history):
   # No title, path, absolute bar number, progression ID, or target-derived ID.
-  base=(*harmonic,str(phase))
+  base = (*harmonic, str(phase))
-  result=[(),base,(*base,source)]
+  result = [(), base, (*base, source)]
   # A complete four-event rhythmic/melodic relationship can influence the next
   # event, while unseen histories smoothly return to broader musical evidence.
-  for count in range(1,min(4,len(history))+1):
+  for count in range(1, min(4, len(history)) + 1):
-    result.append((*base,source,tuple(history[-count:])))
+    result.append((*base, source, tuple(history[-count:])))
   return tuple(key(c) for c in result)
 
-
 class Model:
-  def __init__(self,value):
+  def __init__(self, value):
-    if value.get('version')!=VERSION:raise ValueError('Missing/incompatible solo model; run python -m generator.train')
+    if value.get('version') != VERSION:
-    self.value=value
+      raise ValueError('Missing/incompatible solo model; run python -m generator.train')
-    self.tables={k:(v['total'],v['counts']) for k,v in value['contexts'].items()}
+    self.value = value
-    self.durations=tuple(sorted(map(F,value['durations'])))
+    self.tables = {k: (v['total'], v['counts']) for k, v in value['contexts'].items()}
-    self.rhythms=value['rhythms'];self.steps=value['steps'];self.degrees=value['degrees']
+    self.durations = tuple(sorted(map(fractions.Fraction, value['durations'])))
-    self._prob={}
+    self.rhythms = value['rhythms']
-    self._totals={id(t):sum(t.values()) for t in (self.rhythms,self.steps,self.degrees)}
+    self.steps = value['steps']
-
+    self.degrees = value['degrees']
+    self._prob = {}
+    self._totals = {id(t): sum(t.values()) for t in (self.rhythms, self.steps, self.degrees)}
   @classmethod
-  def load(cls,path):
+  def load(cls, path):
-    try:return cls(json.loads(Path(path).read_text()))
+    try:
-    except FileNotFoundError as e:raise ValueError('Solo model not found; run python -m generator.train') from e
+      return cls(json.loads(pathlib.Path(path).read_text()))
-
+    except FileNotFoundError as e:
-  def probability(self,action,context):
+      raise ValueError('Solo model not found; run python -m generator.train') from e
-    cache=(action,context)
+  def probability(self, action, context):
-    if cache in self._prob:return self._prob[cache]
+    cache = (action, context)
-    kind,duration,step,degree=action
+    if cache in self._prob:
-    def p(table,item,size):return (table.get(str(item),0)+1)/(self._totals[id(table)]+size)
+      return self._prob[cache]
-    probability=p(self.rhythms,kind+':'+duration,2*len(self.durations))
+    kind, duration, step, degree = action
-    if kind=='note':probability*=p(self.steps,step,50)*p(self.degrees,degree,12)
+    def p(table, item, size): return (table.get(str(item), 0) + 1) / (self._totals[id(table)] + size)
-    encoded=key(action)
+    probability = p(self.rhythms, kind + ':' + duration, 2 * len(self.durations))
+    if kind == 'note':
+      probability *= p(self.steps, step, 50) * p(self.degrees, degree, 12)
+    encoded = key(action)
     for c in context:
       if c in self.tables:
-        total,counts=self.tables[c]
+        total, counts = self.tables[c]
-        probability=(counts.get(encoded,0)+PRIOR*probability)/(total+PRIOR)
+        probability = (counts.get(encoded, 0) + PRIOR * probability) / (total + PRIOR)
-    if len(self._prob)>100000:self._prob.clear()
+    if len(self._prob) > 100000:
-    self._prob[cache]=probability
+      self._prob.clear()
+    self._prob[cache] = probability
     return probability
+  def cost(self, action, context): return -math.log(self.probability(action, context))
+  def register_cost(self, number):
+    counts = self.value['registers']
+    return -math.log((counts.get(str(number // 12), 0) + PRIOR) / (sum(counts.values()) + PRIOR * 4))
 
-  def cost(self,action,context):return -math.log(self.probability(action,context))
+def fit(examples, provenance):
-
+  tables = collections.defaultdict(collections.Counter)
-  def register_cost(self,number):
+  rhythms = collections.Counter()
-    counts=self.value['registers']
+  steps = collections.Counter()
-    return -math.log((counts.get(str(number//12),0)+PRIOR)/(sum(counts.values())+PRIOR*4))
+  degrees = collections.Counter()
-
+  registers = collections.Counter()
-
+  durations = set(DURATIONS)
-def fit(examples,provenance):
-  tables=collections.defaultdict(collections.Counter)
-  rhythms=collections.Counter();steps=collections.Counter();degrees=collections.Counter()
-  registers=collections.Counter();durations=set(DURATIONS)
   for example in examples:
-    history=[];previous=None
+    history = []
-    for start,end,pitch in example['events']:
+    previous = None
-      harmonic,root,_=chord_context(example['harmony'],start)
+    for start, end, pitch in example['events']:
-      action=token(end-start,pitch,previous,root)
+      harmonic, root, _ = chord_context(example['harmony'], start)
-      context=contexts(harmonic,phase_at(example['analysis'],start-example['analysis'].body_offset),
+      action = token(end - start, pitch, previous, root)
-        source_feature(example['analysis'],start-example['analysis'].body_offset,root),tuple(history[-4:]))
+      context = contexts(harmonic, phase_at(example['analysis'], start - example['analysis'].body_offset),
-      for c in context:tables[c][key(action)]+=1
+        source_feature(example['analysis'], start - example['analysis'].body_offset, root), tuple(history[-4:]))
-      durations.add(str(end-start));rhythms[action[0]+':'+action[1]]+=1
+      for c in context:
+        tables[c][key(action)] += 1
+      durations.add(str(end - start))
+      rhythms[action[0] + ':' + action[1]] += 1
       if pitch is not None:
-        steps[str(action[2])]+=1;degrees[str(action[3])]+=1
+        steps[str(action[2])] += 1
-        registers[str(pitch//12)]+=1;previous=pitch
+        degrees[str(action[3])] += 1
+        registers[str(pitch // 12)] += 1
+        previous = pitch
       history.append(action)
-  return {'version':VERSION,'provenance':provenance,'prior':PRIOR,
+  return {'version': VERSION, 'provenance': provenance, 'prior': PRIOR,
-    'durations':sorted(durations,key=F),'rhythms':dict(rhythms),'steps':dict(steps),'degrees':dict(degrees),
+    'durations': sorted(durations, key=fractions.Fraction), 'rhythms': dict(rhythms), 'steps': dict(steps), 'degrees': dict(degrees),
-    'registers':dict(registers),
+    'registers': dict(registers),
-    'contexts':{k:{'total':sum(v.values()),'counts':dict(sorted(v.items()))} for k,v in sorted(tables.items())}}
+    'contexts': {k: {'total': sum(v.values()), 'counts': dict(sorted(v.items()))} for k, v in sorted(tables.items())}}

generator/solo/notation.py

@@ -61,27 +61,33 @@ def measures(sources: list[dict], notes: tuple[generator.solo.types.Note, ...],
 
 def tuplets(measures):
   # Group by sounding duration, including notation fragments across barlines.
-  spans=[]
+  spans = []
-  events=[n for m in measures for n in generator.score.measure_notes(m)]
+  events = [n for m in measures for n in generator.score.measure_notes(m)]
-  for ratio,values in itertools.groupby(events,key=lambda n:generator.model.fraction(n['written'].get('ratio',generator.model.rational(1)))):
+  for ratio, values in itertools.groupby(events, key=lambda n: generator.model.fraction(n['written'].get('ratio', generator.model.rational(1)))):
-    if ratio==1:continue
+    if ratio == 1:
-    members=[];elapsed=fractions.Fraction();length=None
+      continue
+    members = []
+    elapsed = fractions.Fraction()
+    length = None
     for note in values:
       if not members:
-        length=generator.model.TYPE_LENGTHS[note['written']['type']]*ratio.numerator
+        length = generator.model.TYPE_LENGTHS[note['written']['type']] * ratio.numerator
-      members.append(note);elapsed+=generator.model.fraction(note['duration'])
+      members.append(note)
-      if elapsed>=length:
+      elapsed += generator.model.fraction(note['duration'])
-        identifier=members[0]['id']+'_tuplet'
+      if elapsed >= length:
-        spans.append({'id':identifier,'voice':'voice_arranged','start':members[0]['id'],'end':members[-1]['id'],'note_count':ratio.denominator,'space_count':ratio.numerator,'depth':0,'bracket':'auto','show_number':'actual'})
+        identifier = members[0]['id'] + '_tuplet'
-        for n in members:n['tuplet_ids']=[identifier]
+        spans.append({'id': identifier, 'voice': 'voice_arranged', 'start': members[0]['id'], 'end': members[-1]['id'], 'note_count': ratio.denominator, 'space_count': ratio.numerator, 'depth': 0, 'bracket': 'auto', 'show_number': 'actual'})
-        members=[];elapsed=fractions.Fraction()
+        for n in members:
+          n['tuplet_ids'] = [identifier]
+        members = []
+        elapsed = fractions.Fraction()
     if members:
-      identifier=members[0]['id']+'_tuplet'
+      identifier = members[0]['id'] + '_tuplet'
-      spans.append({'id':identifier,'voice':'voice_arranged','start':members[0]['id'],'end':members[-1]['id'],'note_count':ratio.denominator,'space_count':ratio.numerator,'depth':0,'bracket':'auto','show_number':'actual'})
+      spans.append({'id': identifier, 'voice': 'voice_arranged', 'start': members[0]['id'], 'end': members[-1]['id'], 'note_count': ratio.denominator, 'space_count': ratio.numerator, 'depth': 0, 'bracket': 'auto', 'show_number': 'actual'})
-      for n in members:n['tuplet_ids']=[identifier]
+      for n in members:
+        n['tuplet_ids'] = [identifier]
   return spans
 
-
 def replace_region(document: dict, first: int, replacement: list[dict]) -> None:
   staff = document['staves'][0]
   replacement = copy.deepcopy(replacement)

generator/solo/placement.py

@@ -5,80 +5,107 @@ import fractions
 import functools
 
 import generator.fretboard
+import generator.hand_position
 import generator.solo.types
-from generator.solo.profiles import UNRESTRICTED
+import generator.solo.profiles
-
+import generator.solo.policy
-def travel(previous: generator.solo.types.Note | None, current: generator.solo.types.Note, tempo: fractions.Fraction) -> int:
-  if previous is None:
-    return 0
-  return transition(previous.position, current.position, current.start - previous.start, tempo)
-
-@functools.lru_cache(maxsize=16384)
-def transition(previous, current, duration, tempo):
-  distance = abs(previous.fret - current.fret) + abs(previous.string_number - current.string_number)
-  seconds = float(duration * 60 / tempo)
-  # Movement is a preference, not an unsupported claim about a player's limits.
-  return round(100 * distance / max(seconds, 0.05))
 
 def hand_positions(position):
-  return tuple(h for h in range(1, 10) if h <= position.fret <= h + 3)
+  return generator.hand_position.regions((position,))
-
 
 @functools.lru_cache(maxsize=512)
 def position_rank(position):
-  from generator.solo.policy import stable_rank
+  return generator.solo.policy.stable_rank('guitar-position', position)
-  return stable_rank('guitar-position', position)
 
-def stable_path(path):
+def seconds(duration, tempo):
-  return tuple(position_rank(n.position) for n in path)
+  return max(fractions.Fraction(1, 20), duration * 60 / tempo)
 
+@dataclasses.dataclass(frozen=True)
+class Path:
+  travel: int = 0
+  peak: int = 0
+  effort: fractions.Fraction = fractions.Fraction()
+  notes: tuple = ()
+  hands: tuple = ()
+  ties: tuple = ()
+  @property
+  def rank(self):
+    return self.travel, self.peak, self.effort, self.ties
 
-@functools.lru_cache(maxsize=4096)
+@functools.lru_cache(maxsize=512)
-def alternatives(notes, tempo, previous=None, limit=2, profile=UNRESTRICTED):
+def paths(notes, tempo, previous=None, profile=generator.solo.profiles.UNRESTRICTED, fixed=False):
-  """Phrase paths with a four-fret hand region and distinct endpoint choices."""
+  """Complete TAB paths: hand travel first, then maximum hand position.
-  states = [(0, (), h) for h in hand_positions(previous.position)] if previous and previous.position else [(0, (), None)]
+
+  Preserve different peaks at a common endpoint. No barline or tied fragment
+  resets the hand or shortens the elapsed time from the previous attack.
+  """
+  states = [(Path(peak=h), h) for h in hand_positions(previous.position)] if previous and previous.position else [(Path(), 0)]
+  last_attack = previous.start if previous else None
   for note in notes:
     following = {}
-    for position in positions(note.pitch.number(), profile):
+    options = (note.position,) if fixed and note.position else positions(note.pitch.number(), profile)
+    for position in options:
       current = dataclasses.replace(note, position=position)
       for hand in hand_positions(position):
-        best = None
+        for path, old_hand in states:
-        for cost, path, old_hand in states:
+          prior = path.notes[-1] if path.notes else previous
-          prior = path[-1] if path else previous
+          if note.continuation and (prior is None or prior.pitch != note.pitch or prior.position != position or prior.end != note.start or hand != old_hand):
-          if note.continuation and (prior is None or prior.pitch != note.pitch or prior.position != position or prior.end != note.start):
             continue
-          movement = 0
+          motion = generator.hand_position.Motion()
+          effort = fractions.Fraction()
           if prior and not note.continuation:
-            seconds = max(.05, float((note.start-prior.start)*60/tempo))
+            motion = generator.hand_position.motion((prior.position,), (position,), old_hand, hand)
-            distance = abs(hand-old_hand) if old_hand is not None else 0
+            effort = 100 * (motion.shift + motion.ergonomics) / seconds(note.start - last_attack, tempo)
-            distance += .25*abs(position.fret-prior.position.fret) + .5*abs(position.string_number-prior.position.string_number)
+          candidate = Path(path.travel + motion.shift, max(path.peak, hand), path.effort + effort, (*path.notes, current), (*path.hands, hand), (*path.ties, (position_rank(position), hand)))
-            movement = round(100*distance/seconds)
+          key = (position, hand, candidate.peak)
-          candidate = (cost+movement, (*path,current), hand)
+          if key not in following or candidate.rank < following[key].rank:
-          if best is None or candidate[0] < best[0] or candidate[0] == best[0] and stable_path(candidate[1]) < stable_path(best[1]):
+            following[key] = candidate
-            best = candidate
-        if best is not None:
-          following[position,hand] = best
     if not following:
       return ()
-    states = list(following.values())
+    # A strictly worse hand travel at the identical endpoint/hand cannot recover.
-  # Endpoints survive so the following phrase can choose a better transition.
+    best = {}
+    for (position, hand, _), path in following.items():
+      best[position, hand] = min(best.get((position, hand), path.travel), path.travel)
+    states = [(path, hand) for (position, hand, _), path in following.items() if path.travel == best[position, hand]]
+    if not note.continuation:
+      last_attack = note.start
+  return tuple(sorted((path for path, _ in states), key=lambda path: path.rank))
+
+@functools.lru_cache(maxsize=512)
+def alternatives(notes, tempo, previous=None, limit=2, profile=generator.solo.profiles.UNRESTRICTED):
   selected, seen = [], set()
-  for cost,path,hand in sorted(states,key=lambda v:(v[0],stable_path(v[1]))):
+  for path in paths(notes, tempo, previous, profile):
-    endpoint = path[-1].position if path else None
+    endpoint = (path.notes[-1].position, path.hands[-1]) if path.notes else None
     if endpoint in seen:
       continue
-    seen.add(endpoint); selected.append((cost,path))
+    seen.add(endpoint)
+    selected.append((path.travel, path.notes))
     if len(selected) == limit:
       break
   return tuple(selected)
 
-
+@functools.lru_cache(maxsize=512)
-@functools.lru_cache(maxsize=4096)
+def place(notes, tempo, previous=None, profile=generator.solo.profiles.UNRESTRICTED):
-def place(notes, tempo, previous=None, profile=UNRESTRICTED):
+  result = alternatives(notes, tempo, previous, 1, profile)
-  result = alternatives(notes,tempo,previous,1,profile)
   return result[0] if result else None
 
+def describe(notes, tempo):
+  candidates = paths(notes, tempo, fixed=True)
+  if not candidates:
+    raise ValueError('cannot describe an invalid guitar path')
+  path = candidates[0]
+  result, prior, old_hand, last_attack = [], None, 0, None
+  for note, hand in zip(path.notes, path.hands):
+    motion = generator.hand_position.motion((prior.position,), (note.position,), old_hand, hand) if prior and not note.continuation else generator.hand_position.Motion()
+    effort = 100 * (motion.shift + motion.ergonomics) / seconds(note.start - last_attack, tempo) if prior and not note.continuation else 0
+    result.append({'start': str(note.start), 'end': str(note.end), 'hand_position': hand, 'string': note.position.string_number,
+      'fret': note.position.fret, 'hand_shift': motion.shift, 'finger_adjustment': motion.fingers, 'string_changes': motion.strings, 'secondary_effort': str(effort)})
+    if not note.continuation:
+      last_attack = note.start
+    prior, old_hand = note, hand
+  return {'version': generator.hand_position.VERSION, **generator.hand_position.summary(r['hand_shift'] for r in result), 'secondary_effort': str(path.effort), 'peak_position': path.peak, 'positions': result}
+
 @functools.lru_cache(maxsize=2048)
-def positions(number: int, profile=UNRESTRICTED):
+def positions(number: int, profile=generator.solo.profiles.UNRESTRICTED):
-  profile = profile or UNRESTRICTED
+  profile = profile or generator.solo.profiles.UNRESTRICTED
   return tuple(p for p in generator.fretboard.single_positions_for_pitch(fractions.Fraction(number)) if p.string_number in profile.strings)

generator/solo/policy.py

@@ -1,5 +1,5 @@
 """Stable deterministic tie breaking shared with guitar and accompaniment."""
 import hashlib
 
-def stable_rank(seed,value):
+def stable_rank(seed, value):
-  return hashlib.sha256((seed+repr(value)).encode()).hexdigest()
+  return hashlib.sha256((seed + repr(value)).encode()).hexdigest()

generator/solo/profiles.py

@@ -2,12 +2,10 @@
 from __future__ import annotations
 
 import dataclasses
+import fractions
 import functools
-from fractions import Fraction as F
 
 import generator.fretboard
-import generator.vocabulary
-
 
 @dataclasses.dataclass(frozen=True)
 class Instrument:
@@ -15,23 +13,20 @@ class Instrument:
   strings: tuple[int, ...] = (1, 2, 3, 4, 5, 6)
   reference: int | None = None
   melody_range: tuple[int, int] | None = None
-
   @property
   def register(self):
     tuning = [p for s, p in generator.fretboard.STANDARD_TUNING if s in self.strings]
     return min(tuning) + 1, max(tuning) + 12
-
   @functools.lru_cache(maxsize=4096)
   def permits(self, number):
-    return any(p.string_number in self.strings for p in generator.fretboard.single_positions_for_pitch(F(number)))
+    return any(p.string_number in self.strings for p in generator.fretboard.single_positions_for_pitch(fractions.Fraction(number)))
 
 UNRESTRICTED = Instrument()
 
-
 def for_part(part, analysis=None):
   if part not in ('acoustic', 'electric'):
     raise ValueError('unknown solo instrument: ' + part)
   strings = (1, 2, 3, 4, 5, 6)
   if analysis is None or not analysis.melody:
     return Instrument(part, strings)
-  return Instrument(part,strings)
+  return Instrument(part, strings)

generator/solo/recognition.py

@@ -4,138 +4,166 @@ No filenames, annotations, learned examples or role exemptions enter here.
 Existing expression validators prove each candidate; deterministic set-cover
 search chooses a mutually consistent explanation, never edits the notes.
 """
-from fractions import Fraction as F
+import fractions
-from generator import vocabulary
-from generator.solo import expressions, types
 
+import generator.solo.expressions
+import generator.solo.types
+import generator.vocabulary
 
 def signature(c):
-  return (c.kind,c.rule,tuple(n.event_id for n in c.events),c.target.event_id,
+  return (c.kind, c.rule, tuple(n.event_id for n in c.events), c.target.event_id,
-    str(c.boundary),c.preparation.event_id if c.preparation else '')
+    str(c.boundary), c.preparation.event_id if c.preparation else '')
-
 
 def merge(*groups):
-  result={}
+  result = {}
   for group in groups:
-    for c in group:result.setdefault(signature(c),c)
+    for c in group:
+      result.setdefault(signature(c), c)
   return tuple(result.values())
 
+def recognize(notes, harmony, provided=(), *, budget=4096, composition=None):
+  composition = composition or generator.vocabulary.current()
+  required = {n.event_id for n in notes if not harmony.ordinary(n)}
+  fixed = {n.event_id for c in provided for n in c.events}
+  if required <= fixed:
+    return tuple(provided), ''
+  candidates = {}
+  single = {}
+  by_id = {n.event_id: n for n in notes}
+  if len(by_id) != len(notes):
+    return (), 'Harmonic analysis requires unique sounding-event IDs'
 
-def recognize(notes,harmony,provided=(),*,budget=4096):
+  def contract(kind, rule, events, target, boundary=None, preparation=None):
-  required={n.event_id for n in notes if not harmony.ordinary(n)}
+    span = harmony.at(boundary if boundary is not None else target.start)
-  fixed={n.event_id for c in provided for n in c.events}
+    if not span or span.chord == 'N.C.':
-  if required<=fixed:return tuple(provided),''
+      return None
-  candidates={};single={};by_id={n.event_id:n for n in notes}
+    identifier = f'inferred:{kind}:{rule}:{events[0].event_id}:{target.event_id}:{boundary}'
-  if len(by_id)!=len(notes):return (), 'Harmonic analysis requires unique sounding-event IDs'
+    return generator.solo.types.ExpressionContract(identifier, kind, rule, tuple(events), target, span.chord, boundary, preparation)
 
-  def contract(kind,rule,events,target,boundary=None,preparation=None):
+  def add(c, extras=()):
-    span=harmony.at(boundary if boundary is not None else target.start)
+    if c is None:
-    if not span or span.chord=='N.C.':return None
+      return False
-    identifier=f'inferred:{kind}:{rule}:{events[0].event_id}:{target.event_id}:{boundary}'
+    bundle = merge(extras, (c,))
-    return types.ExpressionContract(identifier,kind,rule,tuple(events),target,span.chord,boundary,preparation)
+    covered = frozenset(n.event_id for a in bundle for n in a.events)
-
+    if not (covered & required):
-  def add(c,extras=()):
+      return False
-    if c is None:return False
+    combined = merge(provided, bundle)
-    bundle=merge(extras,(c,));covered=frozenset(n.event_id for a in bundle for n in a.events)
+    _, error = generator.solo.expressions.validate(notes, combined, harmony, composition=composition)
-    if not (covered & required):return False
+    if error:
-    combined=merge(provided,bundle)
+      return False
-    _,error=expressions.validate(notes,combined,harmony)
+    candidates[tuple(signature(a) for a in bundle)] = (bundle, covered)
-    if error:return False
-    candidates[tuple(signature(a) for a in bundle)]=(bundle,covered)
     return True
 
   # First discover independently supported colors and harmonic anticipations.
   # These can also support a later approach's destination or passing endpoint.
-  for i,n in enumerate(notes):
+  for i, n in enumerate(notes):
-    if n.event_id not in required:continue
+    if n.event_id not in required:
-    future=[t for t in notes[i+1:] if n.end<=t.start<=n.end+F(vocabulary.data('harmony')['resolution_gap'])
+      continue
-      and 0<abs(t.pitch.number()-n.pitch.number())<=2]
+    future = [t for t in notes[i + 1:] if n.end <= t.start <= n.end + fractions.Fraction(composition.data('harmony')['resolution_gap'])
-    regions=[(None,harmony.at(n.start))]
+      and 0 < abs(t.pitch.number() - n.pitch.number()) <= 2]
-    regions.extend((s.start,s) for s in harmony.spans if n.start<s.start<n.end
+    regions = [(None, harmony.at(n.start))]
-      and s.start-n.start<=F(vocabulary.data('harmony')['anticipation_max']))
+    regions.extend((s.start, s) for s in harmony.spans if n.start < s.start < n.end
-    for boundary,span in regions:
+      and s.start - n.start <= fractions.Fraction(composition.data('harmony')['anticipation_max']))
-      if not span or span.chord=='N.C.':continue
+    for boundary, span in regions:
+      if not span or span.chord == 'N.C.':
+        continue
       if boundary is not None:
-        c=contract('anticipation','early_chord_tone',(n,),n,boundary)
+        c = contract('anticipation', 'early_chord_tone', (n,), n, boundary)
-        if add(c):single.setdefault(n.event_id,[]).append((c,))
+        if add(c):
-      for rule in vocabulary.data('harmony')['colors']:
+          single.setdefault(n.event_id, []).append((c,))
-        if not expressions.eligible_color(rule,span,harmony) and boundary is not None:continue
+      for rule in composition.data('harmony')['colors']:
-        kind='anticipated_color' if boundary is not None else 'color'
+        if not generator.solo.expressions.eligible_color(rule, span, harmony) and boundary is not None:
-        for target in future if rule.get('resolve') else (n,):
+          continue
-          c=contract(kind,rule['id'],(n,),target,boundary)
+        kind = 'anticipated_color' if boundary is not None else 'color'
+        for target in future if rule.get('resolve') else [n]:
+          c = contract(kind, rule['id'], (n,), target, boundary)
           # Non-anticipated color permission is scoped to the note's initial
           # harmony, even when its resolution occurs after a chord boundary.
-          if boundary is None and c:c=types.ExpressionContract(c.id,c.kind,c.rule,c.events,c.target,span.chord)
+          if boundary is None and c:
-          if add(c):single.setdefault(n.event_id,[]).append((c,))
+            c = generator.solo.types.ExpressionContract(c.id, c.kind, c.rule, c.events, c.target, span.chord)
-    if i+1<len(notes):
+          if add(c):
-      target=notes[i+1]
+            single.setdefault(n.event_id, []).append((c,))
+    if i + 1 < len(notes):
+      target = notes[i + 1]
       for s in harmony.spans:
-        if n.start<s.start<n.end:
+        if n.start < s.start < n.end:
-          for rule in vocabulary.data('harmony').get('suspensions',[]):
+          for rule in composition.data('harmony').get('suspensions', []):
-            add(contract('suspension',rule['id'],(n,),target,s.start))
+            add(contract('suspension', rule['id'], (n,), target, s.start))
-        elif s.start==n.end==target.start and n.pitch.number()==target.pitch.number():
+        elif s.start == n.end == target.start and n.pitch.number() == target.pitch.number():
-          add(contract('anticipation','early_reattack',(n,),target,s.start))
+          add(contract('anticipation', 'early_reattack', (n,), target, s.start))
-
+  for i, target in enumerate(notes):
-  for i,target in enumerate(notes):
+    targets = [(None, ())]
-    targets=[(None,())]
+    for support in single.get(target.event_id, ()):
-    for support in single.get(target.event_id,()):
+      c = support[0]
-      c=support[0]
       # An approach into an anticipated color needs both permissions; that
       # combination is not silently interpreted as a chord-tone anticipation.
-      if c.kind=='color':targets.append((None,support))
+      if c.kind == 'color':
-      elif c.kind=='anticipation':targets.append((c.boundary,support))
+        targets.append((None, support))
-    for rule in vocabulary.data('lead_ins')['patterns']:
+      elif c.kind == 'anticipation':
-      size=len(rule['semitones'])
+        targets.append((c.boundary, support))
-      if i<size:continue
+    for rule in composition.data('lead_ins')['patterns']:
-      ns=notes[i-size:i]
+      size = len(rule['semitones'])
-      if not any(n.event_id in required for n in ns):continue
+      if i < size:
-      if tuple(n.pitch.number()-target.pitch.number() for n in ns)!=tuple(rule['semitones']):continue
+        continue
-      preparation=notes[i-size-1] if i>size and rule['kind']=='connection' else None
+      ns = notes[i - size:i]
-      for boundary,extras in targets:
+      if not any(n.event_id in required for n in ns):
-        add(contract(rule['kind'],rule['id'],ns,target,boundary,preparation),extras)
+        continue
+      if tuple(n.pitch.number() - target.pitch.number() for n in ns) != tuple(rule['semitones']):
+        continue
+      preparation = notes[i - size - 1] if i > size and rule['kind'] == 'connection' else None
+      for boundary, extras in targets:
+        add(contract(rule['kind'], rule['id'], ns, target, boundary, preparation), extras)
     # Delayed approaches may include real space or intervening valid music.
     # They still have an exact semitone target and a finite deadline.
-    rule=vocabulary.data('lead_ins')['delayed_approach']
+    rule = composition.data('lead_ins')['delayed_approach']
     for n in reversed(notes[:i]):
-      if target.start-n.start>F(rule['deadline']):break
+      if target.start - n.start > fractions.Fraction(rule['deadline']):
-      if n.event_id in required and target.pitch.number()-n.pitch.number() in rule['steps']:
+        break
-        add(contract('delayed_approach',rule['id'],(n,),target))
+      if n.event_id in required and target.pitch.number() - n.pitch.number() in rule['steps']:
-    if i>=2:
+        add(contract('delayed_approach', rule['id'], (n,), target))
-      before,n=notes[i-2:i]
+    if i >= 2:
+      before, n = notes[i - 2:i]
       if n.event_id in required:
-        for left in [(),*single.get(before.event_id,())]:
+        for left in [(), *single.get(before.event_id, ())]:
-          for right in [(),*single.get(target.event_id,())]:
+          for right in [(), *single.get(target.event_id, ())]:
-            add(contract('passing',vocabulary.data('lead_ins')['passing']['id'],(n,),target,preparation=before),merge(left,right))
+            add(contract('passing', composition.data('lead_ins')['passing']['id'], (n,), target, preparation=before), merge(left, right))
-        for rule in vocabulary.data('lead_ins')['blues']:
+        for rule in composition.data('lead_ins')['blues']:
-          add(contract('blues',rule['id'],(n,),target,preparation=before))
+          add(contract('blues', rule['id'], (n,), target, preparation=before))
-
+  options = {identifier: [] for identifier in sorted(required - fixed, key=lambda identifier: (by_id[identifier].start, identifier))}
-  options={identifier:[] for identifier in sorted(required-fixed,key=lambda identifier:(by_id[identifier].start,identifier))}
+  for bundle, covered in candidates.values():
-  for bundle,covered in candidates.values():
+    for identifier in covered & options.keys():
-    for identifier in covered & options.keys():options[identifier].append((bundle,covered))
+      options[identifier].append((bundle, covered))
-  for identifier,choices in options.items():
+  for identifier, choices in options.items():
     if not choices:
-      n=by_id[identifier]
+      n = by_id[identifier]
-      return (),f'{identifier} at {n.start}: unsupported harmonic relationship for {n.pitch.label()}'
+      return (), f'{identifier} at {n.start}: unsupported harmonic relationship for {n.pitch.label()}'
-    choices.sort(key=lambda x:(sum((n.end-n.start for c in x[0] for n in c.events),F()),
+    choices.sort(key=lambda x: (sum((n.end - n.start for c in x[0] for n in c.events), fractions.Fraction()),
-      sum((max(F(),c.target.start-c.events[-1].end) for c in x[0]),F()),tuple(signature(c) for c in x[0])))
+      sum((max(fractions.Fraction(), c.target.start - c.events[-1].end) for c in x[0]), fractions.Fraction()), tuple(signature(c) for c in x[0])))
+  visits = 0
+  failed = set()
 
-  visits=0;failed=set()
+  def solve(chosen, covered):
-  def solve(chosen,covered):
     nonlocal visits
-    remaining=required-covered
+    remaining = required - covered
-    if not remaining:return chosen
+    if not remaining:
-    key=tuple(sorted(signature(c) for c in chosen))
+      return chosen
-    if key in failed or visits>=budget:return None
+    key = tuple(sorted(signature(c) for c in chosen))
-    visits+=1
+    if key in failed or visits >= budget:
-    identifier=min(remaining,key=lambda x:(len(options[x]),by_id[x].start,x))
+      return None
-    for bundle,added in options[identifier]:
+    visits += 1
-      combined=merge(chosen,bundle)
+    identifier = min(remaining, key=lambda x: (len(options[x]), by_id[x].start, x))
-      _,error=expressions.validate(notes,combined,harmony)
+    for bundle, added in options[identifier]:
-      if error:continue
+      combined = merge(chosen, bundle)
-      result=solve(combined,covered|added)
+      _, error = generator.solo.expressions.validate(notes, combined, harmony, composition=composition)
-      if result is not None:return result
+      if error:
+        continue
+      result = solve(combined, covered | added)
+      if result is not None:
+        return result
     failed.add(key)
     return None
-  result=solve(tuple(provided),fixed)
+
-  if result is None:return (), 'Harmonic treatment search budget exhausted' if visits>=budget else 'No mutually consistent harmonic treatment coverage'
+  result = solve(tuple(provided), fixed)
-  return result,''
+  if result is None:
+    return (), 'Harmonic treatment search budget exhausted' if visits >= budget else 'No mutually consistent harmonic treatment coverage'
+  return result, ''

generator/solo/report.py

@@ -1,60 +1,74 @@
 """Compact reports of actual selected phrases, not duplicated draft ancestry."""
+import collections
 import dataclasses, fractions, json
+
+import generator.solo.harmonic
 import generator.solo.placement
-from generator.solo import harmonic
+import generator.vocabulary
 
 def plain(value):
-  if dataclasses.is_dataclass(value):return plain(dataclasses.asdict(value))
+  if dataclasses.is_dataclass(value):
-  if isinstance(value,dict):return {k:plain(v) for k,v in value.items()}
+    return plain(dataclasses.asdict(value))
-  if isinstance(value,(tuple,list,set,frozenset)):return [plain(v) for v in value]
+  if isinstance(value, dict):
-  if isinstance(value,fractions.Fraction):return str(value)
+    return {k: plain(v) for k, v in value.items()}
+  if isinstance(value, (tuple, list, set, frozenset)):
+    return [plain(v) for v in value]
+  if isinstance(value, fractions.Fraction):
+    return str(value)
   return value
 
-def structure(solo):
+def structure(solo, *, composition=None):
-  harmony=harmonic.Harmony(solo.harmony)
+  composition = composition or generator.vocabulary.current()
-  contracts=tuple(c for p in solo.phrases for c in p.expressions)
+  harmony = generator.solo.harmonic.Harmony(solo.harmony, composition=composition)
-  checked=harmony.evaluate(solo.notes(),until=solo.end,complete=True,expressions=contracts)
+  contracts = tuple(c for p in solo.phrases for c in p.expressions)
-  phrases=[]
+  checked = harmony.evaluate(solo.notes(), until=solo.end, complete=True, expressions=contracts)
+  phrases = []
   for p in solo.phrases:
-    entry={'id':p.id,'start':str(p.start),'end':str(p.end),'purpose':p.purpose,'direction':p.direction,'ending':p.ending,
+    entry = {'id': p.id, 'start': str(p.start), 'end': str(p.end), 'purpose': p.purpose, 'direction': p.direction, 'ending': p.ending,
-      'theme':p.theme,'notes':plain(p.notes),'rhythm_groups':plain(p.groups),'sources':p.sources,
+      'theme': p.theme, 'notes': plain(p.notes), 'rhythm_groups': plain(p.groups), 'sources': p.sources,
-      'identifying_events':plain(p.identity),
+      'identifying_events': plain(p.identity),
-      'evaluation':plain(p.evaluation),'score':p.score,'expressions':plain(p.expressions)}
+      'evaluation': plain(p.evaluation), 'score': p.score, 'expressions': plain(p.expressions)}
     phrases.append(entry)
-  ns=solo.notes()
+  ns = solo.notes()
-  duration=solo.end-solo.start
+  duration = solo.end - solo.start
-  return {'schema':'composer-report-3','selection_trace':plain({k:v for k,v in solo.diagnostics.items() if not k.endswith('_seconds')}),
+  return {'schema': 'composer-report-3', 'selection_trace': plain({k: v for k, v in solo.diagnostics.items() if not k.endswith('_seconds')}),
-    'instrument_profile':plain(solo.profile),'phrases':phrases,'expressions':plain(contracts),'fretboard':fretboard_report(solo),
+    'instrument_profile': plain(solo.profile), 'phrases': phrases, 'expressions': plain(contracts), 'fretboard': fretboard_report(solo), 'guitar_placement': generator.solo.placement.describe(solo.notes(), solo.tempo),
-    'phrasing':{'attacks':len(ns),'allocation_beats':str(duration),
+    'phrasing': {'attacks': len(ns), 'allocation_beats': str(duration),
-      'attacks_per_four_beats':4*len(ns)/float(duration) if duration else 0,
+      'attacks_per_four_beats': 4 * len(ns) / float(duration) if duration else 0,
-      'occupancy':sum(float(n.end-n.start) for n in ns)/float(duration) if duration else 0},
+      'occupancy': sum(float(n.end - n.start) for n in ns) / float(duration) if duration else 0},
-    'harmonic_roles':harmony.describe(ns,expressions=contracts),'harmonic_policy':'contextual (shared by authored and predicted music)', 'harmonic_valid':checked.valid,'harmonic_error':checked.error,
+    'harmonic_roles': harmony.describe(ns, expressions=contracts), 'harmonic_policy': 'contextual (shared by authored and predicted music)', 'harmonic_valid': checked.valid, 'harmonic_error': checked.error,
-    'harmonic_exposure':checked.exposure,'unresolved_treatments':plain(checked.pending)}
+    'harmonic_exposure': checked.exposure, 'unresolved_treatments': plain(checked.pending)}
 
-def describe(solo,measure_offset=0,time_offset=0,*,detail=None):
+def describe(solo, measure_offset=0, time_offset=0, *, detail=None, composition=None):
-  detail=detail or structure(solo)
+  composition = composition or generator.vocabulary.current()
-  lines=[f'{solo.diagnostics.get("construction","solo")}: {solo.part}', 'Times are exact quarter beats from the chorus start; negative times are allocated lead-ins. Pitches sound one octave below guitar notation.',
+  detail = detail or structure(solo, composition=composition)
-    'Approximate fingering costs are not a biomechanical guarantee.',json.dumps(detail['phrasing'])]
+  lines = [f'{solo.diagnostics.get("construction", "solo")}: {solo.part}', 'Times are exact quarter beats from the chorus start; negative times are allocated lead-ins. Pitches sound one octave below guitar notation.',
+    'Approximate fingering costs are not a biomechanical guarantee.', json.dumps(detail['phrasing'])]
   for p in solo.phrases:
     lines.append(f'{p.id}: {p.start}–{p.end}; {p.purpose}, {p.direction}, {p.ending}; theme {p.theme}; score {p.score:.4f}')
-    for n in p.notes:lines.append(f'  {n.start}–{n.end}: {n.pitch.label()}, {n.role}; string {n.position.string_number}, fret {n.position.fret}')
+    for n in p.notes:
-    for c in p.expressions:lines.append(f'  {c.kind}/{c.rule} → {c.target.pitch.label()} at {c.target.start}, held to {c.target.end}')
+      lines.append(f'  {n.start}–{n.end}: {n.pitch.label()}, {n.role}; string {n.position.string_number}, fret {n.position.fret}')
-  return '\n'.join(lines)+'\n'
+    for c in p.expressions:
+      lines.append(f'  {c.kind}/{c.rule} → {c.target.pitch.label()} at {c.target.start}, held to {c.target.end}')
+  return '\n'.join(lines) + '\n'
 
 def fretboard_report(solo):
-  import collections
+  notes = solo.notes()
-  import generator.fretboard
+  attacks = collections.Counter()
-  notes=solo.notes()
+  durations = collections.Counter()
-  attacks=collections.Counter(); durations=collections.Counter(); forced=optional=0
+  forced = optional = 0
   for note in notes:
-    string=note.position.string_number
+    string = note.position.string_number
-    attacks[string]+=1;durations[string]+=float(note.end-note.start)
+    attacks[string] += 1
-    if string==1:
+    durations[string] += float(note.end - note.start)
-      if len(generator.solo.placement.positions(note.pitch.number(),solo.profile))==1:forced+=1
+    if string == 1:
-      else:optional+=1
+      if len(generator.solo.placement.positions(note.pitch.number(), solo.profile)) == 1:
-  total=sum(durations.values())
+        forced += 1
-  return {'attacks_by_string':dict(sorted(attacks.items())),
+      else:
-    'duration_fraction_by_string':{s:d/max(.001,total) for s,d in sorted(durations.items())},
+        optional += 1
-    'first_string_forced':forced,'first_string_optional':optional,
+  total = sum(durations.values())
-    'range_midi': [min(n.pitch.number() for n in notes),max(n.pitch.number() for n in notes)] if notes else [],
+  return {'attacks_by_string': dict(sorted(attacks.items())),
-    'cross_bar_sounding_events':sum(any(n.start < t < n.end for t in {c.start for c in solo.harmony if c.beat == 1}) for n in notes)}
+    'duration_fraction_by_string': {s: d / max(.001, total) for s, d in sorted(durations.items())},
+    'first_string_forced': forced, 'first_string_optional': optional,
+    'range_midi': [min(n.pitch.number() for n in notes), max(n.pitch.number() for n in notes)] if notes else [],
+    'cross_bar_sounding_events': sum(any(n.start < t < n.end for t in {c.start for c in solo.harmony if c.beat == 1}) for n in notes)}

generator/solo/types.py

@@ -5,7 +5,6 @@ import generator.model
 import generator.fretboard
 import generator.solo.profiles
 
-
 @dataclasses.dataclass(frozen=True, order=True)
 class Pitch:
   letter: str

generator/source.py

@@ -0,0 +1,81 @@
+"""Immutable source events, meter boundaries and tempo. No solo construction."""
+from __future__ import annotations
+import copy
+import dataclasses
+import fractions
+import hashlib
+import json
+
+import generator.model
+import generator.score
+import generator.solo.types
+
+@dataclasses.dataclass(frozen=True)
+class Analysis:
+  boundaries: tuple
+  tempo: fractions.Fraction
+  melody: tuple
+  body_offset: fractions.Fraction = fractions.Fraction()
+
+def attacks(document: dict) -> tuple[generator.solo.types.Note, ...]:
+  """Collapse tied tones before extracting monophonic melody fragments."""
+  result, pending = [], {}
+  absolute = fractions.Fraction()
+  for measure in generator.score.source_staff(document)['measures']:
+    for event in generator.score.measure_notes(measure):
+      if event.get('grace') or event.get('cue'):
+        continue
+      for tone in event['pitches']:
+        number = generator.model.pitch_number(tone)
+        if number.denominator != 1:
+          continue
+        start = absolute + generator.model.fraction(event['onset'])
+        end = start + generator.model.fraction(tone['duration'])
+        pitch = generator.solo.types.Pitch(tone['letter'], generator.model.alteration(tone['alter']), tone['octave'])
+        source = pending.pop(tone.get('tie_from'), None)
+        if source is not None:
+          result[source] = dataclasses.replace(result[source], end=end, sources=(*result[source].sources, event['id']))
+        else:
+          source = len(result)
+          result.append(generator.solo.types.Note(start, end, pitch, (event['id'],)))
+        if 'tie_to' in tone:
+          pending[tone['id']] = source
+    absolute += generator.model.fraction(measure['duration'])
+  return tuple(sorted(result, key=lambda n: (n.start, -n.pitch.number(), n.sources)))
+
+def analyze(document):
+  measures = generator.score.source_staff(document)['measures']
+  boundaries = [fractions.Fraction()]
+  for m in measures:
+    boundaries.append(boundaries[-1] + generator.model.fraction(m['duration']))
+  tempo = next((generator.model.fraction(t['bpm']) * generator.model.written_length(t['written']) for m in measures for t in m.get('tempos', []) if 'bpm' in t), fractions.Fraction(120))
+  return Analysis(tuple(boundaries), tempo, attacks(document), boundaries[generator.score.body_start_measure_index(measures)])
+
+@dataclasses.dataclass(frozen=True)
+class SourceAnalysis:
+  document_json: str
+  musical_key: str
+  @classmethod
+  def read(cls, document):
+    prepared = copy.deepcopy(document)
+    staff = generator.score.source_staff(prepared)
+    for i, m in enumerate(staff['measures']):
+      m['id'], m['number'] = f'source_{i}', str(i + 1)
+    staff['tuplets'] = generator.score.reidentify(staff['measures'], staff.get('tuplets', []))
+    prepared['metadata'] = {k: v for k, v in prepared['metadata'].items() if k not in ('title', 'composer', 'copyright', 'software')}
+    prepared['metadata'].update(title='Source', composer='')
+    staff.update(id='source', name='Source')
+    melodic = analyze(prepared)
+    content = {'melody': [(str(n.start), str(n.end), dataclasses.asdict(n.pitch)) for n in melodic.melody],
+      'measures': [{k: m.get(k) for k in ('duration', 'time_signature', 'key', 'harmonies', 'tempos', 'section_label', 'implicit')} for m in staff['measures']]}
+    # Fraction/string spelling is canonical and never includes filesystem IDs.
+    key = hashlib.sha256(json.dumps(content, sort_keys=True, default=str).encode()).hexdigest()
+    return cls(json.dumps(prepared, sort_keys=True), key)
+  def document(self):
+    return json.loads(self.document_json)
+  def accompaniment_key(self):
+    document = self.document()
+    document['metadata'].pop('tempo', None)
+    for measure in generator.score.source_staff(document)['measures']:
+      measure['tempos'] = []
+    return SourceAnalysis.read(document).musical_key

generator/summary.py

@@ -6,6 +6,7 @@ import typing
 
 import generator.model
 import generator.score
+import generator.score_io
 
 def _arranged_staff(document: dict[str, typing.Any]) -> dict[str, typing.Any]:
   staves = document.get("staves")
@@ -61,7 +62,7 @@ def performance_document_summary(document: dict[str, typing.Any]) -> dict[str, t
 
 def performance_tree_summary(root: pathlib.Path) -> dict[str, typing.Any]:
   return {
-    path.name: performance_document_summary(generator.score.load_json(path))
+    path.name: performance_document_summary(generator.score_io.load_json(path))
     for path in sorted(root.glob("*.json"))
   }
 

generator/train.py

@@ -3,154 +3,200 @@ from __future__ import annotations
 import argparse
 import collections
 import dataclasses
+import fractions
 import hashlib
 import json
 import os
-from pathlib import Path
+import pathlib
+import shutil
 import tempfile
 import time
-from fractions import Fraction as F
-from generator import harmony, model, musicxml, parts, training_io, vocabulary
-from generator import harmonic_time as timeline
-from generator.solo import analysis, learned
 
+import generator.fretboard
+import generator.harmonic_time
+import generator.harmony
+import generator.musicxml
+import generator.parts
+import generator.solo.authored
+import generator.solo.learned
+import generator.source
+import generator.training_io
+import generator.vocabulary
 
 @dataclasses.dataclass(frozen=True)
 class Span:
-  start:F
+  start: fractions.Fraction
-  end:F
+  end: fractions.Fraction
-  chord:str
+  chord: str
-
 
 class Context:
-  def __init__(self,document):
+  def __init__(self, document):
-    self.spans=tuple(Span(s.start,s.end,harmony.symbol(s.chord) if s.chord else 'N.C.')
+    self.spans = tuple(Span(s.start, s.end, generator.harmony.symbol(s.chord) if s.chord else 'N.C.')
-      for s in timeline.read(document['staves'][0]))
+      for s in generator.harmonic_time.read(document['staves'][0]))
-  def at(self,time):return next((s for s in self.spans if s.start<=time<s.end),None)
+  def at(self, time): return next((s for s in self.spans if s.start <= time < s.end), None)
-
 
 def load_examples(manifest):
-  manifest=Path(manifest);value=json.loads(manifest.read_text())
+  manifest = pathlib.Path(manifest)
-  if value.get('version')!=1:raise ValueError('Unsupported training manifest')
+  value = json.loads(manifest.read_text())
-  examples=[];provenance=[];seen=set();pairs=set()
+  if value.get('version') != 1:
-  for entry in sorted(value['examples'],key=lambda e:e['id']):
+    raise ValueError('Unsupported training manifest')
-    if entry['id'] in seen or entry['slot'] not in (1,2):raise ValueError('Duplicate example or invalid role')
+  examples = []
-    seen.add(entry['id']);role='acoustic' if entry['slot']==1 else 'electric'
+  provenance = []
-    source=(manifest.parent/entry['source']).resolve();solo=(manifest.parent/entry['solo']).resolve()
+  seen = set()
-    if not solo.is_relative_to(manifest.parent.resolve()):raise ValueError('Training solo must be inside the training directory')
+  pairs = set()
-    if not source.is_relative_to((manifest.parent.parent/'songs').resolve()):raise ValueError('Training sources must be inside data/songs')
+  for entry in sorted(value['examples'], key=lambda e: e['id']):
-    document=parts.prepare_document(musicxml.load_musicxml(source));a=analysis.analyze(document)
+    if entry['id'] in seen or entry['slot'] not in (1, 2):
-    from generator.production import SourceAnalysis
+      raise ValueError('Duplicate example or invalid role')
-    pair=(SourceAnalysis.read(document).musical_key,entry['slot'])
+    seen.add(entry['id'])
-    if pair in pairs:raise ValueError('Duplicate source music and solo slot')
+    role = 'acoustic' if entry['slot'] == 1 else 'electric'
+    source = (manifest.parent / entry['source']).resolve()
+    solo = (manifest.parent / entry['solo']).resolve()
+    if not solo.is_relative_to(manifest.parent.resolve()):
+      raise ValueError('Training solo must be inside the training directory')
+    if not source.is_relative_to((manifest.parent.parent / 'songs').resolve()):
+      raise ValueError('Training sources must be inside data/songs')
+    document = generator.parts.prepare_document(generator.musicxml.load_musicxml(source))
+    a = generator.source.analyze(document)
+    pair = (generator.source.SourceAnalysis.read(document).musical_key, entry['slot'])
+    if pair in pairs:
+      raise ValueError('Duplicate source music and solo slot')
     pairs.add(pair)
-    evidence=training_io.read(solo,role,pitch_convention=entry['pitch_convention'])
+    evidence = generator.training_io.read(solo, role, pitch_convention=entry['pitch_convention'])
-    spans=[(F(m['start']),F(m['duration'])) for m in evidence['measures']]
+    spans = [(fractions.Fraction(m['start']), fractions.Fraction(m['duration'])) for m in evidence['measures']]
-    if spans!=[(x,y-x) for x,y in zip(a.boundaries,a.boundaries[1:])]:raise ValueError(entry['id']+': source/solo measure timing differs; no padding or pickup removal allowed')
+    if spans != [(x, y - x) for x, y in zip(a.boundaries, a.boundaries[1:])]:
-    if evidence['discrepancies']:raise ValueError(entry['id']+': '+str(evidence['discrepancies']))
+      raise ValueError(entry['id'] + ': source/solo measure timing differs; no padding or pickup removal allowed')
-    ctx=Context(document)
+    if evidence['discrepancies']:
-    training_io.validate_harmony(document,evidence,entry['id'])
+      raise ValueError(entry['id'] + ': ' + str(evidence['discrepancies']))
-    from generator.solo.authored import validate_evidence
+    ctx = Context(document)
-    checked=validate_evidence(document,evidence)
+    generator.training_io.validate_harmony(document, evidence, entry['id'])
-    events=[];cursor=F()
+    checked = generator.solo.authored.validate_evidence(document, evidence)
-    from generator import fretboard
+    events = []
+    cursor = fractions.Fraction()
     for n in evidence['sounding_events']:
-      start,end=F(n['start']),F(n['end']);number=F(n['sounding_midi'])
+      start, end = fractions.Fraction(n['start']), fractions.Fraction(n['end'])
-      if number.denominator!=1:raise ValueError(entry['id']+': microtonal guitar pitch is unsupported')
+      number = fractions.Fraction(n['sounding_midi'])
-      pitch=int(number)
+      if number.denominator != 1:
-      if not fretboard.single_positions_for_pitch(F(pitch)):raise ValueError(entry['id']+': unplayable sounding pitch '+str(pitch)+' at '+str(start))
+        raise ValueError(entry['id'] + ': microtonal guitar pitch is unsupported')
-      if start<cursor or end<=start:raise ValueError(entry['id']+': non-monophonic/invalid event')
+      pitch = int(number)
-      if start>cursor:events.append((cursor,start,None))
+      if not generator.fretboard.single_positions_for_pitch(fractions.Fraction(pitch)):
-      events.append((start,end,pitch));cursor=end
+        raise ValueError(entry['id'] + ': unplayable sounding pitch ' + str(pitch) + ' at ' + str(start))
-    if cursor<a.boundaries[-1]:events.append((cursor,a.boundaries[-1],None))
+      if start < cursor or end <= start:
-    source_hash=hashlib.sha256(source.read_bytes()).hexdigest()
+        raise ValueError(entry['id'] + ': non-monophonic/invalid event')
-    examples.append({'id':entry['id'],'song':source_hash,'role':role,'analysis':a,'harmony':ctx,'events':tuple(events),
+      if start > cursor:
-      'harmonic_validation':{'valid':checked.valid,'treatments':dict(sorted(collections.Counter(c.rule for c in checked.contracts).items()))}})
+        events.append((cursor, start, None))
-    provenance.append({'id':entry['id'],'source_sha256':source_hash,'solo_sha256':evidence['sha256'],
+      events.append((start, end, pitch))
-      'pitch_convention':entry['pitch_convention'],'measures':len(spans),'pickup_beats':str(a.body_offset),
+      cursor = end
-      'slot':entry['slot'],'attacks':len(evidence['sounding_events']),'annotations':len(evidence['annotations']),
+    if cursor < a.boundaries[-1]:
-      'adapter_notes':['Trailing chord-display parentheses interpreted without modifying XML.'] if any(h['display'] and h['display'].endswith(')') and '(' not in h['display'] for h in evidence['harmonies']) else []})
+      events.append((cursor, a.boundaries[-1], None))
-  if not examples:raise ValueError('No training examples')
+    source_hash = hashlib.sha256(source.read_bytes()).hexdigest()
-  return examples,{'description':value['provenance'],'examples':provenance,'training_version':learned.VERSION}
+    examples.append({'id': entry['id'], 'song': source_hash, 'role': role, 'analysis': a, 'harmony': ctx, 'events': tuple(events),
+      'harmonic_validation': {'valid': checked.valid, 'treatments': dict(sorted(collections.Counter(c.rule for c in checked.contracts).items()))}})
+    provenance.append({'id': entry['id'], 'source_sha256': source_hash, 'solo_sha256': evidence['sha256'],
+      'pitch_convention': entry['pitch_convention'], 'measures': len(spans), 'pickup_beats': str(a.body_offset),
+      'slot': entry['slot'], 'attacks': len(evidence['sounding_events']), 'annotations': len(evidence['annotations']),
+      'adapter_notes': ['Trailing chord-display parentheses interpreted without modifying XML.'] if any(h['display'] and h['display'].endswith(')') and '(' not in h['display'] for h in evidence['harmonies']) else []})
+  if not examples:
+    raise ValueError('No training examples')
+  return examples, {'description': value['provenance'], 'examples': provenance, 'training_version': generator.solo.learned.VERSION}
 
+def likelihood(predictor, example):
+  history = []
+  previous = None
+  cost = 0
+  for start, end, pitch in example['events']:
+    harmonic, root, _ = generator.solo.learned.chord_context(example['harmony'], start)
+    action = generator.solo.learned.token(end - start, pitch, previous, root)
+    relative = start - example['analysis'].body_offset
+    c = generator.solo.learned.contexts(harmonic, generator.solo.learned.phase_at(example['analysis'], relative), generator.solo.learned.source_feature(example['analysis'], relative, root), tuple(history[-4:]))
+    cost += predictor.cost(action, c)
+    history.append(action)
+    if pitch is not None:
+      previous = pitch
+  return cost / len(history)
 
-def likelihood(predictor,example):
+def atomic_json(path, value):
-  history=[];previous=None;cost=0
+  path = pathlib.Path(path)
-  for start,end,pitch in example['events']:
+  path.parent.mkdir(parents=True, exist_ok=True)
-    harmonic,root,_=learned.chord_context(example['harmony'],start)
+  with tempfile.NamedTemporaryFile('w', dir=path.parent, delete=False) as stream:
-    action=learned.token(end-start,pitch,previous,root)
+    json.dump(value, stream, sort_keys=True, separators=(',', ':'))
-    relative=start-example['analysis'].body_offset
+    stream.write('\n')
-    c=learned.contexts(harmonic,learned.phase_at(example['analysis'],relative),learned.source_feature(example['analysis'],relative,root),tuple(history[-4:]))
+    name = stream.name
-    cost+=predictor.cost(action,c);history.append(action)
+  os.chmod(name, 0o644)
-    if pitch is not None:previous=pitch
+  os.replace(name, path)
-  return cost/len(history)
 
-
+def register_example(manifest, source, solo, slot, pitch_convention):
-def atomic_json(path,value):
-  path=Path(path);path.parent.mkdir(parents=True,exist_ok=True)
-  with tempfile.NamedTemporaryFile('w',dir=path.parent,delete=False) as stream:
-    json.dump(value,stream,sort_keys=True,separators=(',',':'));stream.write('\n');name=stream.name
-  os.chmod(name,0o644)
-  os.replace(name,path)
-
-
-def register_example(manifest,source,solo,slot,pitch_convention):
   """Copy and validate a new pair before atomically publishing its manifest."""
-  import shutil
+  manifest = pathlib.Path(manifest).resolve()
-  manifest=Path(manifest).resolve();root=manifest.parent
+  root = manifest.parent
-  sources=root.parent/'songs';root.mkdir(parents=True,exist_ok=True);sources.mkdir(parents=True,exist_ok=True)
+  sources = root.parent / 'songs'
-  value=json.loads(manifest.read_text()) if manifest.exists() else {'version':1,'provenance':'User-authored paired song/solo training corpus.','examples':[]}
+  root.mkdir(parents=True, exist_ok=True)
-  source,solo=Path(source).resolve(),Path(solo).resolve()
+  sources.mkdir(parents=True, exist_ok=True)
-  destinations=((source,sources/source.name),(solo,root/solo.name));created=[];temporary=None
+  value = json.loads(manifest.read_text()) if manifest.exists() else {'version': 1, 'provenance': 'User-authored paired song/solo training corpus.', 'examples': []}
+  source, solo = pathlib.Path(source).resolve(), pathlib.Path(solo).resolve()
+  destinations = ((source, sources / source.name), (solo, root / solo.name))
+  created = []
+  temporary = None
   try:
-    for original,destination in destinations:
+    for original, destination in destinations:
       if destination.exists():
-        if original.read_bytes()!=destination.read_bytes():raise ValueError(str(destination)+': already exists with different contents; no overwrite performed')
+        if original.read_bytes() != destination.read_bytes():
-      else:shutil.copyfile(original,destination);created.append(destination)
+          raise ValueError(str(destination) + ': already exists with different contents; no overwrite performed')
-    entry={'id':source.stem+'/solo'+str(slot),'source':'../songs/'+source.name,'solo':solo.name,
+      else:
-      'slot':slot,'pitch_convention':pitch_convention}
+        shutil.copyfile(original, destination)
-    if any(e['id']==entry['id'] for e in value['examples']):raise ValueError('Example already registered: '+entry['id'])
+        created.append(destination)
+    entry = {'id': source.stem + '/solo' + str(slot), 'source': '../songs/' + source.name, 'solo': solo.name,
+      'slot': slot, 'pitch_convention': pitch_convention}
+    if any(e['id'] == entry['id'] for e in value['examples']):
+      raise ValueError('Example already registered: ' + entry['id'])
     value['examples'].append(entry)
-    with tempfile.NamedTemporaryFile('w',dir=root,suffix='.json',delete=False) as stream:
+    with tempfile.NamedTemporaryFile('w', dir=root, suffix='.json', delete=False) as stream:
-      json.dump(value,stream);temporary=Path(stream.name)
+      json.dump(value, stream)
+      temporary = pathlib.Path(stream.name)
     load_examples(temporary)
-    atomic_json(manifest,value)
+    atomic_json(manifest, value)
   except BaseException:
-    for p in created:p.unlink()
+    for p in created:
+      p.unlink()
     raise
   finally:
-    if temporary:temporary.unlink(missing_ok=True)
+    if temporary:
-
+      temporary.unlink(missing_ok=True)
 
 def main():
-  parser=argparse.ArgumentParser(description=__doc__)
+  parser = argparse.ArgumentParser(description=__doc__)
-  parser.add_argument('--manifest',type=Path,default=Path('data/training/manifest.json'))
+  parser.add_argument('--manifest', type=pathlib.Path, default=pathlib.Path('data/training/manifest.json'))
-  parser.add_argument('--output',type=Path,default=Path('data/models/solo.json'))
+  parser.add_argument('--output', type=pathlib.Path, default=pathlib.Path('data/models/solo.json'))
-  parser.add_argument('--report',type=Path,default=Path('build/training/report.json'))
+  parser.add_argument('--report', type=pathlib.Path, default=pathlib.Path('build/training/report.json'))
-  parser.add_argument('--leave-one-song-out',action='store_true')
+  parser.add_argument('--leave-one-song-out', action='store_true')
-  parser.add_argument('--add-source',type=Path,help='Register a source song (copied to data/songs if needed).')
+  parser.add_argument('--add-source', type=pathlib.Path, help='Register a source song (copied to data/songs if needed).')
-  parser.add_argument('--add-solo',type=Path,help='Register its authored solo (copied unchanged to data/training).')
+  parser.add_argument('--add-solo', type=pathlib.Path, help='Register its authored solo (copied unchanged to data/training).')
-  parser.add_argument('--slot',type=int,choices=(1,2),help='Arrangement mapping only: 1 acoustic, 2 electric; learning is pooled.')
+  parser.add_argument('--slot', type=int, choices=(1, 2), help='Arrangement mapping only: 1 acoustic, 2 electric; learning is pooled.')
-  parser.add_argument('--pitch-convention',choices=('guitar-written','encoded'),default='guitar-written')
+  parser.add_argument('--pitch-convention', choices=('guitar-written', 'encoded'), default='guitar-written')
-  args=parser.parse_args();began=time.monotonic()
+  args = parser.parse_args()
-  if any((args.add_source,args.add_solo,args.slot)):
+  began = time.monotonic()
-    if not all((args.add_source,args.add_solo,args.slot)):parser.error('--add-source, --add-solo and --slot must be supplied together')
+  if any((args.add_source, args.add_solo, args.slot)):
-    register_example(args.manifest,args.add_source,args.add_solo,args.slot,args.pitch_convention)
+    if not all((args.add_source, args.add_solo, args.slot)):
-  examples,provenance=load_examples(args.manifest)
+      parser.error('--add-source, --add-solo and --slot must be supplied together')
-  value=learned.fit(examples,provenance);predictor=learned.Model(value)
+    register_example(args.manifest, args.add_source, args.add_solo, args.slot, args.pitch_convention)
-  report={'version':learned.VERSION,'training':provenance,'model_sha256':vocabulary.fingerprint(value),
+  examples, provenance = load_examples(args.manifest)
-    'contexts':len(value['contexts']),'in_sample_event_nll':{e['id']:likelihood(predictor,e) for e in examples},
+  value = generator.solo.learned.fit(examples, provenance)
-    'harmonic_validation':{e['id']:e['harmonic_validation'] for e in examples},
+  predictor = generator.solo.learned.Model(value)
-    'limitations':'A small corpus does not establish generalization. Likelihood is teacher-forced, not free generation or musical preference.'}
+  report = {'version': generator.solo.learned.VERSION, 'training': provenance, 'model_sha256': generator.vocabulary.fingerprint(value),
+    'contexts': len(value['contexts']), 'in_sample_event_nll': {e['id']: likelihood(predictor, e) for e in examples},
+    'harmonic_validation': {e['id']: e['harmonic_validation'] for e in examples},
+    'limitations': 'A small corpus does not establish generalization. Likelihood is teacher-forced, not free generation or musical preference.'}
   if args.leave_one_song_out:
-    report['held_out_event_nll']={}
+    report['held_out_event_nll'] = {}
     for song in sorted({e['song'] for e in examples}):
-      remaining=[e for e in examples if e['song']!=song]
+      remaining = [e for e in examples if e['song'] != song]
-      if not remaining:continue
+      if not remaining:
-      fitted=learned.Model(learned.fit(remaining,{'evaluation_only':True}))
+        continue
-      report['held_out_event_nll'].update({e['id']:likelihood(fitted,e) for e in examples if e['song']==song})
+      fitted = generator.solo.learned.Model(generator.solo.learned.fit(remaining, {'evaluation_only': True}))
-  atomic_json(args.output,value);report['training_seconds']=time.monotonic()-began;atomic_json(args.report,report)
+      report['held_out_event_nll'].update({e['id']: likelihood(fitted, e) for e in examples if e['song'] == song})
+  atomic_json(args.output, value)
+  report['training_seconds'] = time.monotonic() - began
+  atomic_json(args.report, report)
   print(f'Trained {len(examples)} solos: {len(value["contexts"])} contexts in {report["training_seconds"]:.3f}s; {args.output}')
 
-
+if __name__ == '__main__':
-if __name__=='__main__':main()
+  main()

generator/training_io.py

@@ -1,152 +1,193 @@
 """Literal authored-score reader for training, registered repertoire and comparison."""
 from __future__ import annotations
-import hashlib,json
+import hashlib, json
-from pathlib import Path
+import pathlib
-from fractions import Fraction as F
+import fractions
-import xml.etree.ElementTree as ET
+import xml.etree.ElementTree
+import generator.fretboard
 
 def read(path, role, form_start=None, pitch_convention='encoded'):
-  if pitch_convention not in ('encoded','guitar-written'):
+  if pitch_convention not in ('encoded', 'guitar-written'):
-    raise ValueError('Unknown reference pitch convention: '+pitch_convention)
+    raise ValueError('Unknown reference pitch convention: ' + pitch_convention)
-  raw=Path(path).read_bytes();tree=ET.fromstring(raw)
+  raw = pathlib.Path(path).read_bytes()
-  result={'path':str(Path(path).resolve()),'sha256':hashlib.sha256(raw).hexdigest(),'validation_role':role,
+  tree = xml.etree.ElementTree.fromstring(raw)
-    'pitch_convention':pitch_convention,'part_names':[n.text for n in tree.findall('./part-list/score-part/part-name')],
+  result = {'path': str(pathlib.Path(path).resolve()), 'sha256': hashlib.sha256(raw).hexdigest(), 'validation_role': role,
-    'measures':[],'notes':[],'harmonies':[],'annotations':[],'discrepancies':[]}
+    'pitch_convention': pitch_convention, 'part_names': [n.text for n in tree.findall('./part-list/score-part/part-name')],
-  divisions=F(1);absolute=F();nominal=F(4);pending={};active=None;transpose=F()
+    'measures': [], 'notes': [], 'harmonies': [], 'annotations': [], 'discrepancies': []}
-  for index,m in enumerate(tree.findall('./part/measure')):
+  divisions = fractions.Fraction(1)
-    cursor=F();extent=F();last_start=F();last_duration=F()
+  absolute = fractions.Fraction()
-    if (value:=m.findtext('attributes/divisions')) is not None:divisions=F(value)
+  nominal = fractions.Fraction(4)
-    meter=m.find('attributes/time')
+  pending = {}
+  active = None
+  transpose = fractions.Fraction()
+  for index, m in enumerate(tree.findall('./part/measure')):
+    cursor = fractions.Fraction()
+    extent = fractions.Fraction()
+    last_start = fractions.Fraction()
+    last_duration = fractions.Fraction()
+    if (value := m.findtext('attributes/divisions')) is not None:
+      divisions = fractions.Fraction(value)
+    meter = m.find('attributes/time')
     if meter is not None and meter.findtext('beats'):
-      nominal=sum((F(v) for v in meter.findtext('beats').split('+')),F())*4/F(meter.findtext('beat-type'))
+      nominal = sum((fractions.Fraction(v) for v in meter.findtext('beats').split('+')), fractions.Fraction()) * 4 / fractions.Fraction(meter.findtext('beat-type'))
     for item in m:
-      offset=F(item.findtext('offset','0'))/divisions
+      offset = fractions.Fraction(item.findtext('offset', '0')) / divisions
-      if item.tag=='attributes' and item.find('transpose') is not None:
+      if item.tag == 'attributes' and item.find('transpose') is not None:
-        transpose=F(item.findtext('transpose/chromatic','0'))+12*F(item.findtext('transpose/octave-change','0'))
+        transpose = fractions.Fraction(item.findtext('transpose/chromatic', '0')) + 12 * fractions.Fraction(item.findtext('transpose/octave-change', '0'))
-      elif item.tag=='backup':cursor-=F(item.findtext('duration'))/divisions
+      elif item.tag == 'backup':
-      elif item.tag=='forward':cursor+=F(item.findtext('duration'))/divisions;extent=max(extent,cursor)
+        cursor -= fractions.Fraction(item.findtext('duration')) / divisions
-      elif item.tag=='harmony':
+      elif item.tag == 'forward':
-        active={'time':str(absolute+cursor+offset),'measure':index+1,'offset':str(cursor+offset),
+        cursor += fractions.Fraction(item.findtext('duration')) / divisions
-          'root':item.findtext('root/root-step'),'alter':item.findtext('root/root-alter','0'),
+        extent = max(extent, cursor)
-          'kind':item.findtext('kind'),'display':item.find('kind').get('text') if item.find('kind') is not None else None,
+      elif item.tag == 'harmony':
-          'bass':ET.tostring(item.find('bass'),encoding='unicode') if item.find('bass') is not None else None,
+        active = {'time': str(absolute + cursor + offset), 'measure': index + 1, 'offset': str(cursor + offset),
-          'degrees':[{'value':d.findtext('degree-value'),'alter':d.findtext('degree-alter'),'type':d.findtext('degree-type')} for d in item.findall('degree')]}
+          'root': item.findtext('root/root-step'), 'alter': item.findtext('root/root-alter', '0'),
+          'kind': item.findtext('kind'), 'display': item.find('kind').get('text') if item.find('kind') is not None else None,
+          'bass': xml.etree.ElementTree.tostring(item.find('bass'), encoding='unicode') if item.find('bass') is not None else None,
+          'degrees': [{'value': d.findtext('degree-value'), 'alter': d.findtext('degree-alter'), 'type': d.findtext('degree-type')} for d in item.findall('degree')]}
         result['harmonies'].append(active)
-      elif item.tag=='direction':
+      elif item.tag == 'direction':
         for words in item.findall('direction-type/words'):
-          result['annotations'].append({'time':str(absolute+cursor+offset),'measure':index+1,'text':''.join(words.itertext()),'interpretation':'descriptive; no production permission'})
+          result['annotations'].append({'time': str(absolute + cursor + offset), 'measure': index + 1, 'text': ''.join(words.itertext()), 'interpretation': 'descriptive; no production permission'})
-      elif item.tag=='note':
+      elif item.tag == 'note':
-        duration=F(item.findtext('duration','0'))/divisions
+        duration = fractions.Fraction(item.findtext('duration', '0')) / divisions
-        start=last_start if item.find('chord') is not None else cursor
+        start = last_start if item.find('chord') is not None else cursor
-        end=start+duration;voice=item.findtext('voice','1');p=item.find('pitch')
+        end = start + duration
-        encoded={'step':p.findtext('step'),'alter':p.findtext('alter','0'),'octave':int(p.findtext('octave'))} if p is not None else None
+        voice = item.findtext('voice', '1')
-        kinds={t.get('type') for t in item.findall('tie')} | {t.get('type') for t in item.findall('notations/tied')}
+        p = item.find('pitch')
-        event={'start':str(absolute+start),'end':str(absolute+end),'measure':index+1,'offset':str(start),'pitch':encoded,'rest':item.find('rest') is not None,
+        encoded = {'step': p.findtext('step'), 'alter': p.findtext('alter', '0'), 'octave': int(p.findtext('octave'))} if p is not None else None
-          'written_type':item.findtext('type'),'dots':len(item.findall('dot')),'voice':voice,'ties':sorted(kinds),
+        kinds = {t.get('type') for t in item.findall('tie')} | {t.get('type') for t in item.findall('notations/tied')}
-          'sounding_transposition':str(-12 if pitch_convention=='guitar-written' else transpose),
+        event = {'start': str(absolute + start), 'end': str(absolute + end), 'measure': index + 1, 'offset': str(start), 'pitch': encoded, 'rest': item.find('rest') is not None,
-          'tablature':{'string':int(item.findtext('notations/technical/string')),'fret':int(item.findtext('notations/technical/fret'))} if item.find('notations/technical/string') is not None and item.find('notations/technical/fret') is not None else None,
+          'written_type': item.findtext('type'), 'dots': len(item.findall('dot')), 'voice': voice, 'ties': sorted(kinds),
-          'time_modification':ET.tostring(item.find('time-modification'),encoding='unicode') if item.find('time-modification') is not None else None,
+          'sounding_transposition': str(-12 if pitch_convention == 'guitar-written' else transpose),
-          'tuplets':[dict(t.attrib) for t in item.findall('notations/tuplet')],'grace':item.find('grace') is not None,'new_attack':'stop' not in kinds and p is not None}
+          'tablature': {'string': int(item.findtext('notations/technical/string')), 'fret': int(item.findtext('notations/technical/fret'))} if item.find('notations/technical/string') is not None and item.find('notations/technical/fret') is not None else None,
-        key=(voice,json.dumps(encoded,sort_keys=True))
+          'time_modification': xml.etree.ElementTree.tostring(item.find('time-modification'), encoding='unicode') if item.find('time-modification') is not None else None,
+          'tuplets': [dict(t.attrib) for t in item.findall('notations/tuplet')], 'grace': item.find('grace') is not None, 'new_attack': 'stop' not in kinds and p is not None}
+        key = (voice, json.dumps(encoded, sort_keys=True))
         if 'stop' in kinds:
-          old=pending.get(key)
+          old = pending.get(key)
-          if old is None or result['notes'][old]['end']!=event['start']:
+          if old is None or result['notes'][old]['end'] != event['start']:
-            result['discrepancies'].append({'measure':index+1,'issue':'unmatched/noncontiguous tie; retained literally'})
+            result['discrepancies'].append({'measure': index + 1, 'issue': 'unmatched/noncontiguous tie; retained literally'})
-          else:event['continuation_of']=old
+          else:
-          if 'start' not in kinds:pending.pop(key,None)
+            event['continuation_of'] = old
-        if 'start' in kinds:pending[key]=len(result['notes'])
+          if 'start' not in kinds:
+            pending.pop(key, None)
+        if 'start' in kinds:
+          pending[key] = len(result['notes'])
         result['notes'].append(event)
-        for container in ('notations/technical/fingering','notations/technical/other-technical','notations/other-notation'):
+        for container in ('notations/technical/fingering', 'notations/technical/other-technical', 'notations/other-notation'):
           for label in item.findall(container):
-            text=''.join(label.itertext()).strip()
+            text = ''.join(label.itertext()).strip()
-            if text:result['annotations'].append({'time':str(absolute+start),'measure':index+1,'text':text,
+            if text:
-              'container':container,'interpretation':'Original note annotation; container does not establish musical permission or guitar fingering.'})
+              result['annotations'].append({'time': str(absolute + start), 'measure': index + 1, 'text': text,
-        if item.find('chord') is None and item.find('grace') is None:cursor=end
+                'container': container, 'interpretation': 'Original note annotation; container does not establish musical permission or guitar fingering.'})
-        last_start,last_duration=start,duration;extent=max(extent,end,cursor)
+        if item.find('chord') is None and item.find('grace') is None:
-    if extent!=nominal and m.get('implicit')!='yes':result['discrepancies'].append({'measure':index+1,'issue':f'encoded span {extent} differs from nominal {nominal}; no padding applied'})
+          cursor = end
-    result['measures'].append({'number':m.get('number'),'start':str(absolute),'duration':str(extent),'implicit':m.get('implicit'),'initial_harmony_known':any(F(h['time'])<=absolute for h in result['harmonies'])})
+        last_start, last_duration = start, duration
-    absolute+=extent
+        extent = max(extent, end, cursor)
-  if pending:result['discrepancies'].append({'issue':'unterminated ties retained'})
+    if extent != nominal and m.get('implicit') != 'yes':
-  result['harmonies'].sort(key=lambda h:F(h['time']))
+      result['discrepancies'].append({'measure': index + 1, 'issue': f'encoded span {extent} differs from nominal {nominal}; no padding applied'})
+    result['measures'].append({'number': m.get('number'), 'start': str(absolute), 'duration': str(extent), 'implicit': m.get('implicit'), 'initial_harmony_known': any(fractions.Fraction(h['time']) <= absolute for h in result['harmonies'])})
+    absolute += extent
+  if pending:
+    result['discrepancies'].append({'issue': 'unterminated ties retained'})
+  result['harmonies'].sort(key=lambda h: fractions.Fraction(h['time']))
   # Do not run the strict score importer over these references: annotations,
   # unusual timing and high encoded pitches are evidence, not repair requests.
-  from generator import harmony, model, fretboard
   for event in result['notes']:
-    if not event['pitch']:continue
+    if not event['pitch']:
-    p=event['pitch'];number=12*(p['octave']+1)+model.NATURAL[p['step'].lower()]+F(p['alter'])
+      continue
-    event['encoded_midi']=str(number)
+    p = event['pitch']
-    event['sounding_midi']=str(number+F(event['sounding_transposition']))
+    number = 12 * (p['octave'] + 1) + generator.model.NATURAL[p['step'].lower()] + fractions.Fraction(p['alter'])
-    event['legal_positions_at_encoded_pitch']=len(fretboard.single_positions_for_pitch(number))
+    event['encoded_midi'] = str(number)
-    event['legal_positions_at_sounding_pitch']=len(fretboard.single_positions_for_pitch(F(event['sounding_midi'])))
+    event['sounding_midi'] = str(number + fractions.Fraction(event['sounding_transposition']))
-    event['harmonic_spans']=[]
+    event['legal_positions_at_encoded_pitch'] = len(generator.fretboard.single_positions_for_pitch(number))
-    start,end=F(event['start']),F(event['end'])
+    event['legal_positions_at_sounding_pitch'] = len(generator.fretboard.single_positions_for_pitch(fractions.Fraction(event['sounding_midi'])))
-    changes=[F(h['time']) for h in result['harmonies'] if start<F(h['time'])<end]
+    event['harmonic_spans'] = []
-    for a,b in zip([start,*changes],[*changes,end]):
+    start, end = fractions.Fraction(event['start']), fractions.Fraction(event['end'])
-      current=next((h for h in reversed(result['harmonies']) if F(h['time'])<=a),None)
+    changes = [fractions.Fraction(h['time']) for h in result['harmonies'] if start < fractions.Fraction(h['time']) < end]
-      info={'start':str(a),'end':str(b),'chord':current,'chord_degrees':[]}
+    for a, b in zip([start, *changes], [*changes, end]):
+      current = next((h for h in reversed(result['harmonies']) if fractions.Fraction(h['time']) <= a), None)
+      info = {'start': str(a), 'end': str(b), 'chord': current, 'chord_degrees': []}
       if current and current['root']:
         try:
-          chord=harmony.parse_symbol(training_chord(current));root=model.NATURAL[chord['root']['letter']]+model.alteration(chord['root']['alter'])
+          chord = generator.harmony.parse_symbol(training_chord(current))
-          info['chord_degrees']=[d for d,alter in harmony.degrees(chord).items() if (root+harmony.degree_semitones(d,alter)-number)%12==0]
+          root = generator.model.NATURAL[chord['root']['letter']] + generator.model.alteration(chord['root']['alter'])
-          info['root_relative_semitones']=str((number-root)%12)
+          info['chord_degrees'] = [d for d, alter in generator.harmony.degrees(chord).items() if (root + generator.harmony.degree_semitones(d, alter) - number) % 12 == 0]
-          info['spelled_simple_degree']=(model.LETTERS.index(p['step'].lower())-model.LETTERS.index(chord['root']['letter']))%7+1
+          info['root_relative_semitones'] = str((number - root) % 12)
-        except (ValueError,KeyError,AttributeError) as error:
+          info['spelled_simple_degree'] = (generator.model.LETTERS.index(p['step'].lower()) - generator.model.LETTERS.index(chord['root']['letter'])) % 7 + 1
-          info['uncertainty']=str(error)
+        except (ValueError, KeyError, AttributeError) as error:
-      else:info['uncertainty']='No written/carried harmony; none invented.'
+          info['uncertainty'] = str(error)
+      else:
+        info['uncertainty'] = 'No written/carried harmony; none invented.'
       event['harmonic_spans'].append(info)
-  sounded=[];tie_roots={}
+  sounded = []
-  for i,n in enumerate(result['notes']):
+  tie_roots = {}
-    if not n['pitch']:continue
+  for i, n in enumerate(result['notes']):
-    previous=n.get('continuation_of')
+    if not n['pitch']:
+      continue
+    previous = n.get('continuation_of')
     if previous is not None and previous in tie_roots:
-      root=tie_roots[previous];sounded[root]['end']=n['end'];sounded[root]['notation_events'].append(i);tie_roots[i]=root
+      root = tie_roots[previous]
+      sounded[root]['end'] = n['end']
+      sounded[root]['notation_events'].append(i)
+      tie_roots[i] = root
     else:
-      tie_roots[i]=len(sounded);sounded.append({'start':n['start'],'end':n['end'],'pitch':n['pitch'],
+      tie_roots[i] = len(sounded)
-        'encoded_midi':n['encoded_midi'],'sounding_midi':n['sounding_midi'],'measure':n['measure'],'notation_events':[i]})
+      sounded.append({'start': n['start'], 'end': n['end'], 'pitch': n['pitch'],
-  result['sounding_events']=sounded
+        'encoded_midi': n['encoded_midi'], 'sounding_midi': n['sounding_midi'], 'measure': n['measure'], 'notation_events': [i]})
-  first=form_start or next((h['measure'] for h in result['harmonies']),None)
+  result['sounding_events'] = sounded
-  offset=F(result['measures'][first-1]['start']) if first else None
+  first = form_start or next((h['measure'] for h in result['harmonies']), None)
-  result['alignment']={'form_start_measure':first,'encoded_offset':str(offset) if offset is not None else None,'basis':'explicit override' if form_start else 'first written harmony; review required','preceding_material':'retained, without invented harmony'}
+  offset = fractions.Fraction(result['measures'][first - 1]['start']) if first else None
+  result['alignment'] = {'form_start_measure': first, 'encoded_offset': str(offset) if offset is not None else None, 'basis': 'explicit override' if form_start else 'first written harmony; review required', 'preceding_material': 'retained, without invented harmony'}
   for label in result['annotations']:
-    t=F(label['time']); current=[h for h in result['harmonies'] if F(h['time'])<=t]
+    t = fractions.Fraction(label['time'])
-    future=next((h for h in result['harmonies'] if F(h['time'])>t),None)
+    current = [h for h in result['harmonies'] if fractions.Fraction(h['time']) <= t]
-    label['current_harmony']=current[-1] if current else None
+    future = next((h for h in result['harmonies'] if fractions.Fraction(h['time']) > t), None)
-    label['following_harmony']=future
+    label['current_harmony'] = current[-1] if current else None
-    label['events']=[n for n in result['notes'] if F(n['start'])<=t<F(n['end']) or t<=F(n['start'])<t+1]
+    label['following_harmony'] = future
-    label['uncertainty']='Label meaning must be assessed from these events and harmonic time; label preserved verbatim.'
+    label['events'] = [n for n in result['notes'] if fractions.Fraction(n['start']) <= t < fractions.Fraction(n['end']) or t <= fractions.Fraction(n['start']) < t + 1]
-    expected={'b9':F(1),'#9':F(3),'b13':F(8),'#11':F(6),'6':F(9)}.get(label['text'].strip())
+    label['uncertainty'] = 'Label meaning must be assessed from these events and harmonic time; label preserved verbatim.'
+    expected = {'b9': fractions.Fraction(1), '#9': fractions.Fraction(3), 'b13': fractions.Fraction(8), '#11': fractions.Fraction(6), '6': fractions.Fraction(9)}.get(label['text'].strip())
     if expected is not None:
-      relationships=[s.get('root_relative_semitones') for n in label['events'] for s in n.get('harmonic_spans',[])]
+      relationships = [s.get('root_relative_semitones') for n in label['events'] for s in n.get('harmonic_spans', [])]
-      label['literal_interval_present']=str(expected) in relationships
+      label['literal_interval_present'] = str(expected) in relationships
-      if not label['literal_interval_present']:label['uncertainty']='No matching interval in the immediate annotated context; label retained, intended scope uncertain.'
+      if not label['literal_interval_present']:
+        label['uncertainty'] = 'No matching interval in the immediate annotated context; label retained, intended scope uncertain.'
     # Lead in and Bluesy stay contextual descriptions, never enclosure permits.
   return result
 
-from generator import harmony,model
+import generator.harmony
+import generator.model
 
 def training_chord(h):
-  result={'root':{'letter':h['root'].lower(),'alter':model.rational(F(h['alter']))},'kind':h['kind'],'degrees':[]}
+  result = {'root': {'letter': h['root'].lower(), 'alter': generator.model.rational(fractions.Fraction(h['alter']))}, 'kind': h['kind'], 'degrees': []}
   for d in h['degrees']:
-    degree=int(d['value']);alter=F(d['alter'])
+    degree = int(d['value'])
-    if d['type']=='alter':alter+=harmony.KINDS.get(result['kind'],{}).get(degree,0)
+    alter = fractions.Fraction(d['alter'])
-    elif d['type']=='add' and (degree-1)%7==6:alter-=1
+    if d['type'] == 'alter':
-    result['degrees'].append({'value':degree,'alter':model.rational(alter),'type':d['type']})
+      alter += generator.harmony.KINDS.get(result['kind'], {}).get(degree, 0)
+    elif d['type'] == 'add' and (degree - 1) % 7 == 6:
+      alter -= 1
+    result['degrees'].append({'value': degree, 'alter': generator.model.rational(alter), 'type': d['type']})
   if h['bass']:
-    import xml.etree.ElementTree as ET
+    bass = xml.etree.ElementTree.fromstring(h['bass'])
-    bass=ET.fromstring(h['bass']);result['bass']={'letter':bass.findtext('bass-step').lower(),'alter':model.rational(F(bass.findtext('bass-alter','0')))}
+    result['bass'] = {'letter': bass.findtext('bass-step').lower(), 'alter': generator.model.rational(fractions.Fraction(bass.findtext('bass-alter', '0')))}
-  display=h['display']
+  display = h['display']
   if display:
     # Soundslice exports an optional chord's suffix with a trailing closing
     # parenthesis, but its actual root separately. Evidence stays untouched.
-    if display.endswith(')') and '(' not in display:display=display[:-1]
+    if display.endswith(')') and '(' not in display:
-    harmony.normalize_text(result,display)
+      display = display[:-1]
-  return harmony.symbol(result).strip('()')
+    generator.harmony.normalize_text(result, display)
+  return generator.harmony.symbol(result).strip('()')
 
-
+def validate_harmony(document, evidence, identity):
-def validate_harmony(document,evidence,identity):
+  source_chords = []
-  source_chords=[];cursor=F()
+  cursor = fractions.Fraction()
   for measure in document['staves'][0]['measures']:
-    source_chords.extend((cursor+model.fraction(h['onset']),harmony.symbol(h).strip('()')) for h in measure['harmonies'])
+    source_chords.extend((cursor + generator.model.fraction(h['onset']), generator.harmony.symbol(h).strip('()')) for h in measure['harmonies'])
-    cursor+=model.fraction(measure['duration'])
+    cursor += generator.model.fraction(measure['duration'])
-  authored=[(F(h['time']),training_chord(h)) for h in evidence['harmonies']]
+  authored = [(fractions.Fraction(h['time']), training_chord(h)) for h in evidence['harmonies']]
-  if authored!=source_chords:raise ValueError(identity+': authored harmony differs from authoritative source')
+  if authored != source_chords:
+    raise ValueError(identity + ': authored harmony differs from authoritative source')

generator/validation.py

@@ -0,0 +1,161 @@
+"""Score document validation, above pure musical values and instrument rules."""
+from __future__ import annotations
+import fractions
+
+import generator.fretboard
+import generator.model
+import generator.pitch
+
+def validate_document(document: dict) -> None:
+  if not isinstance(document, dict) or not isinstance(document.get('metadata'), dict):
+    raise ValueError('score requires a metadata object')
+  if any(not isinstance(document['metadata'].get(name), str) for name in ('title', 'composer', 'key_signature', 'time_signature')):
+    raise ValueError('score metadata requires title, composer, key_signature and time_signature strings')
+  staves = document.get('staves', [])
+  if not isinstance(staves, list) or len(staves) != 1 or not isinstance(staves[0], dict):
+    raise ValueError('exactly one staff is currently supported')
+  if not isinstance(staves[0].get('measures'), list) or not staves[0]['measures']:
+    raise ValueError('score requires measures')
+  instruments = staves[0].get('instruments', [])
+  instrument_ids = {i['id'] for i in instruments}
+  if len(instruments) != len(instrument_ids) or any(not isinstance(i, str) or not i for i in instrument_ids):
+    raise ValueError('score instruments require unique nonempty IDs')
+  ids: set[str] = set()
+  events: dict[str, dict] = {}
+  tones: dict[str, tuple[dict, fractions.Fraction, fractions.Fraction, str]] = {}
+  ownership: dict[str, str] = {}
+  absolute = fractions.Fraction()
+  for measure in staves[0]['measures']:
+    if {'repeats', 'endings', 'initial_attributes'} & measure.keys():
+      raise ValueError('repeat notation is unsupported; regenerate scores from music written out in reading order')
+    length = generator.model.fraction(measure['duration'])
+    if length <= 0:
+      raise ValueError('measure duration must be positive')
+    voices = measure.get('voices', [])
+    if not voices or len({v['id'] for v in voices}) != len(voices) or any(not v['id'] for v in voices):
+      raise ValueError('measure requires uniquely identified voice lanes')
+    previous_ends = {v['id']: fractions.Fraction() for v in voices}
+    for voice, note in ((v, n) for v in voices for n in v['notes']):
+      if any({'sound_ties', 'printed_ties'} & pitch.keys() for pitch in note['pitches']):
+        raise ValueError('pass-specific tie metadata is unsupported; regenerate scores')
+      if not note['id'] or note['id'] in ids:
+        raise ValueError(f'duplicate event ID: {note["id"]}')
+      ids.add(note['id'])
+      events[note['id']] = note
+      ownership[note['id']] = voice['id']
+      if note.get('voice', voice['id']) != voice['id']:
+        raise ValueError('event voice ownership disagrees with its lane')
+      onset, duration = generator.model.fraction(note['onset']), generator.model.fraction(note['duration'])
+      if onset < 0 or duration < 0 or onset + duration > length:
+        raise ValueError(f'event outside measure: {note["id"]}')
+      if note['kind'] not in ('note', 'rest', 'spacer'):
+        raise ValueError('invalid event kind')
+      if not note.get('grace'):
+        if duration == 0 or onset < previous_ends[voice['id']]:
+          raise ValueError('overlapping events in one rhythmic voice')
+        previous_ends[voice['id']] = onset + duration
+      elif duration != 0:
+        raise ValueError('grace notes must not advance metric time')
+      expected = generator.model.metric_length(note['written'])
+      if not note.get('grace') and not note.get('measure_rest') and duration != expected:
+        raise ValueError('metric and written event durations disagree')
+      if note['kind'] != 'note' and note.get('pitches'):
+        raise ValueError('rests and spacers cannot contain pitches')
+      if note['kind'] == 'note' and not note.get('pitches') and note.get('notehead') != 'slash':
+        raise ValueError('pitched event requires at least one pitch')
+      for p in note.get('pitches', []):
+        if p.get('instrument') and p['instrument'] not in instrument_ids:
+          raise ValueError('note refers to an unknown score instrument')
+        if 'unpitched' in p:
+          position = p['unpitched']
+          if position['letter'] not in generator.model.LETTERS or type(position['octave']) is not int:
+            raise ValueError('invalid unpitched display position')
+        else:
+          generator.model.pitch_number(p)
+        if staves[0].get('tablature'):
+          tab = p.get('tablature', {})
+          string, fret = tab.get('string'), tab.get('fret')
+          if type(string) is not int or type(fret) is not int or string not in dict(generator.fretboard.STANDARD_TUNING) or not generator.fretboard.MIN_FRET <= fret <= generator.fretboard.MAX_FRET:
+            raise ValueError(f'{p["id"]}: invalid generated tablature position')
+          state = dict(measure)
+          for change in measure.get('attributes', []):
+            if generator.model.fraction(change['onset']) <= onset:
+              state.update(change)
+          if dict(generator.fretboard.STANDARD_TUNING)[string] + fret != generator.pitch.sounding_pitch(p, state):
+            raise ValueError(f'{p["id"]}: tablature disagrees with sounding pitch')
+        pd = generator.model.fraction(p['duration'])
+        if pd < 0 or (pd == 0 and not note.get('grace')) or pd > duration or not p['id'] or p['id'] in tones:
+          raise ValueError('invalid chord-tone duration or duplicate ID')
+        if not note.get('grace') and pd != generator.model.metric_length(p['written']):
+          raise ValueError('metric and written chord-tone durations disagree')
+        tones[p['id']] = (p, absolute + onset, pd, voice['id'])
+    for harmony in measure.get('harmonies', []):
+      if not 0 <= generator.model.fraction(harmony['onset']) < length:
+        raise ValueError('harmony outside measure')
+    for change in measure.get('attributes', []):
+      if not 0 <= generator.model.fraction(change['onset']) < length:
+        raise ValueError('attribute change outside measure')
+    for state in [measure, *measure.get('attributes', [])]:
+      if 'meter' in state:
+        generator.model.validate_meter(state['meter'])
+      if 'key' in state:
+        if len(state['key']['alterations']) != 7:
+          raise ValueError('key alterations must contain seven exact alterations')
+        for value in state['key']['alterations']:
+          generator.model.alteration(value)
+      if 'clef' in state and (state['clef']['sign'] not in ('G', 'F', 'C', 'percussion', 'none') or not 1 <= state['clef']['line'] <= 6 or not -3 <= state['clef']['octave_change'] <= 3):
+        raise ValueError('invalid clef')
+    absolute += length
+  for identifier, (p, start, duration, voice) in tones.items():
+    if target := p.get('tie_to'):
+      if target not in tones:
+        raise ValueError(f'dangling tie: {identifier} -> {target}')
+      other, next_start, _, next_voice = tones[target]
+      same_pitch = all(p.get(k) == other.get(k) for k in ('unpitched', 'instrument')) if 'unpitched' in p else all(p.get(k) == other.get(k) for k in ('letter', 'alter', 'octave'))
+      if other.get('tie_from') != identifier or not same_pitch or next_voice != voice or next_start != start + duration:
+        raise ValueError(f'invalid tie: {identifier} -> {target}')
+      if staves[0].get('tablature') and p['tablature'] != other['tablature']:
+        raise ValueError(f'{identifier}: tied guitar tones must retain their string and fret')
+    if source := p.get('tie_from'):
+      if source not in tones or tones[source][0].get('tie_to') != identifier:
+        raise ValueError(f'dangling incoming tie: {identifier}')
+  span_ids = set()
+  event_order = {identifier: index for index, identifier in enumerate(events)}
+  for span in staves[0].get('tuplets', []):
+    if not span['id'] or span['id'] in span_ids or span['start'] not in ids or span['end'] not in ids or not 0 < span['note_count'] <= 1000000 or not 0 < span['space_count'] <= 1000000 or not 0 <= span['depth'] <= 16:
+      raise ValueError('invalid tuplet span')
+    members = [n['id'] for n in events.values() if span['id'] in n.get('tuplet_ids', [])]
+    if not members or members[0] != span['start'] or members[-1] != span['end'] or event_order[span['start']] > event_order[span['end']]:
+      raise ValueError('tuplet endpoints disagree with member events')
+    if len({ownership[n] for n in members}) != 1 or span.get('voice', ownership[members[0]]) != ownership[members[0]]:
+      raise ValueError('tuplet crosses independent voice lanes')
+    if span.get('bracket', 'auto') not in ('yes', 'no', 'auto') or span.get('show_number', 'actual') not in ('none', 'actual', 'both'):
+      raise ValueError('invalid tuplet appearance')
+    span_ids.add(span['id'])
+  for note in events.values():
+    for verse in note.get('lyrics', []):
+      start = tones[note['pitches'][0]['id']][1] if note['pitches'] else fractions.Fraction()
+      if 'extend_to' in verse and not start < generator.model.fraction(verse['extend_to']) <= absolute:
+        raise ValueError('lyric extension endpoint is outside the score')
+    if any(t not in span_ids for t in note.get('tuplet_ids', [])):
+      raise ValueError('event refers to an unknown tuplet')
+    if tremolo := note.get('tremolo'):
+      if tremolo['type'] not in ('single', 'start', 'stop') or type(tremolo['marks']) is not int or not 0 <= tremolo['marks'] <= 8:
+        raise ValueError('invalid measured tremolo')
+      if tremolo['type'] in ('start', 'stop'):
+        other = events.get(tremolo.get('partner'))
+        expected_type = 'stop' if tremolo['type'] == 'start' else 'start'
+        if not other or other.get('tremolo', {}).get('partner') != note['id'] or other['tremolo']['type'] != expected_type or other['tremolo']['marks'] != tremolo['marks']:
+          raise ValueError('invalid paired tremolo')
+  for span in staves[0].get('octave_shifts', []):
+    if not 0 <= generator.model.fraction(span['start']) < generator.model.fraction(span['end']) <= absolute or not -3 <= span['octaves'] <= 3:
+      raise ValueError('invalid octave-shift span')
+  previous_end = fractions.Fraction()
+  for span in staves[0].get('scale_spans', []):
+    start, end = generator.model.fraction(span['start']), generator.model.fraction(span['end'])
+    if not previous_end <= start < end <= absolute or not isinstance(span.get('label'), str) or not span['label'].strip() or type(span.get('matched')) is not bool:
+      raise ValueError('invalid solo scale span')
+    if not isinstance(span.get('id'), str) or not span['id'] or span['id'] in ids:
+      raise ValueError('solo scale spans require unique IDs')
+    ids.add(span['id'])
+    previous_end = end

generator/vocabulary.py

@@ -2,29 +2,103 @@
 from __future__ import annotations
 import contextlib
 import contextvars
+import copy
 import dataclasses
+import fractions
 import functools
 import hashlib
 import json
-from pathlib import Path
+import pathlib
-from fractions import Fraction as F
+import types
 
-ROOT = Path(__file__).resolve().parents[1] / 'data' / 'composition'
+ROOT = pathlib.Path(__file__).resolve().parents[1] / 'data' / 'composition'
 ACTIVE = contextvars.ContextVar('composition_vocabulary', default=None)
 
 def load(root=ROOT):
   # Content, not just the directory, keys memoization. Long-lived individual
   # role builds must observe edited production data without restarting Python.
   names = ('roles', 'harmony', 'lead_ins', 'accompaniment')
-  return _load(tuple((name, (Path(root)/(name+'.json')).read_text()) for name in names))
+  return copy.deepcopy(_load(tuple((name, (pathlib.Path(root) / (name + '.json')).read_text()) for name in names)))
 
 @functools.lru_cache(maxsize=16)
 def _load(contents):
   result = {}
   for name, text in contents:
     value = json.loads(text)
-    if value.get('version') != 1 or not value.get('provenance') or not value.get('license'):
+    if value.get('version') != (6 if name == 'accompaniment' else 1) or not value.get('provenance') or not value.get('license'):
       raise ValueError(f'{name}: unsupported or unprovenanced production vocabulary')
+    if name == 'accompaniment':
+      signatures = {}
+      for part in ('electric', 'acoustic'):
+        policy = value['instruments'][part]
+        if part == 'acoustic' and policy.get('first_attack') != 'bass_per_harmony':
+          raise ValueError('acoustic patterns require bass at the first attack of each harmony')
+        if part == 'acoustic' and policy.get('attack_assignment') != {'version': 1, 'mode': 'combined_except_eighth_pairs'}:
+          raise ValueError('acoustic patterns require automatic eighth-pair assignment')
+        groove = policy['groove']
+        if any(type(groove[k]) is not int for k in ('cycle_patterns', 'slot_candidates', 'fill_spacing_measures')) or not 1 <= groove['cycle_patterns'] <= 4 or not 4 <= groove['slot_candidates'] <= 8 or groove['fill_spacing_measures'] < 8:
+          raise ValueError('invalid bounded accompaniment groove policy')
+        identifiers = set()
+        signatures[part] = set()
+        for pattern in policy['patterns']:
+          if pattern['id'] in identifiers or not pattern['family']:
+            raise ValueError('duplicate or unnamed accompaniment pattern')
+          identifiers.add(pattern['id'])
+          lengths = tuple(map(fractions.Fraction, pattern['lengths']))
+          if not 1 <= len(lengths) <= 2 or any(n <= 0 for n in lengths):
+            raise ValueError('patterns require one or two positive measure lengths')
+          meter = pattern.get('meter')
+          grouping = tuple(pattern.get('grouping', ()))
+          if meter == 'partial':
+            if len(lengths) != 1 or grouping:
+              raise ValueError('partial patterns require one measure and no grouping')
+          elif meter not in ('3/4', '4/4', '5/4') or lengths != (fractions.Fraction(meter[0]),) * 2:
+            raise ValueError('full patterns require an explicit supported meter and two complete bars')
+          if (meter == '5/4' and grouping not in ((3, 2), (2, 3))) or (meter != '5/4' and grouping):
+            raise ValueError('5/4 patterns require a 3+2 or 2+3 grouping')
+          # Validate musical events before any measure projection or ties.
+          previous, releases = fractions.Fraction(), {}
+          onsets = []
+          for row in pattern['events']:
+            onset, duration = fractions.Fraction(row['onset']), fractions.Fraction(row['duration'])
+            if onset < previous or duration <= 0 or onset + duration > sum(lengths):
+              raise ValueError('overlapping or out-of-bounds accompaniment pattern')
+            if onset.denominator > 2 or duration.denominator > 2:
+              raise ValueError('comping vocabulary uses a quarter/eighth grid')
+            if part == 'acoustic':
+              if row['layer'] != 'acoustic':
+                raise ValueError('acoustic pattern layers are assigned from the complete rhythm')
+            elif row['layer'] != 'electric':
+              raise ValueError('electric pattern must contain electric chord attacks')
+            if row.get('anticipate') and row['layer'] not in ('acoustic', 'electric'):
+              raise ValueError('only acoustic/electric chords can anticipate')
+            # Acoustic overlaps are checked after deterministic string-lane
+            # assignment, including neighboring figures in the source form.
+            lanes = () if part == 'acoustic' else (row['layer'],)
+            if any(releases.get(lane, fractions.Fraction()) > onset for lane in lanes):
+              raise ValueError('overlapping accompaniment attacks on the same strings')
+            releases.update({lane: onset + duration for lane in lanes})
+            previous = onset
+            onsets.append(onset)
+          if len(lengths) == 2:
+            signatures[part].add((meter, grouping, tuple(onsets)))
+        families = policy['voicing_families']
+        if not families.get('provenance'):
+          raise ValueError('guitar voicing families require provenance')
+        if part == 'electric':
+          if set(families['rank']) != {'drop-two', 'close-position', 'compact-open', 'compact-doubled', 'three-note-shell'} or any(type(v) is not int or not 0 <= v <= 2 for v in families['rank'].values()):
+            raise ValueError('invalid electric family ranking')
+        else:
+          for kind in ('triad', 'tetrad', 'extended'):
+            if set(families[kind]) != {'5/1', '5/5', '6/1', '6/5'}:
+              raise ValueError('acoustic families require root/fifth bass on both bass strings')
+            for orders in families[kind].values():
+              if not orders or any(len(o) != 3 or set(o) - {'root', 'fifth', 'guide', 'terminal', 'color'} for o in orders):
+                raise ValueError('invalid acoustic upper chord order')
+      if signatures['acoustic'] & signatures['electric']:
+        raise ValueError('acoustic/electric two-bar onset patterns must be distinct')
+      if not 1 <= value['pattern_candidates'] <= 64:
+        raise ValueError('accompaniment pattern budget must be 1–64')
     entries = value.get('cells', value.get('patterns', value.get('colors', value.get('transformations', []))))
     if len({v['id'] for v in entries}) != len(entries):
       raise ValueError(f'{name}: duplicate vocabulary ID')
@@ -33,22 +107,54 @@ def _load(contents):
 
 @functools.lru_cache(maxsize=1)
 def _default():
-  return {'data': load(), 'role': '', 'settings': {}}
+  return CompositionContext.create(load())
+
+def _freeze(value):
+  if isinstance(value, dict):
+    return types.MappingProxyType({key: _freeze(item) for key, item in value.items()})
+  if isinstance(value, list):
+    return tuple(_freeze(item) for item in value)
+  return value
+
+@functools.lru_cache(maxsize=32)
+def _frozen_json(contents):
+  return _freeze(json.loads(contents))
+
+@dataclasses.dataclass(frozen=True)
+class CompositionContext:
+  """Owned, immutable, pickleable policy inputs with stable iteration order."""
+  resources_json: str
+  name: str = ''
+  settings_json: str = '{}'
+  @classmethod
+  def create(cls, resources, name='', settings=None):
+    return cls(json.dumps(resources), name, json.dumps(settings or {}))
+  def data(self, name):
+    return _frozen_json(self.resources_json)[name]
+  def role(self, name=None):
+    return {**self.data('roles')['roles'].get(name or self.name, {}), **_frozen_json(self.settings_json)}
+  def accompaniment(self, part):
+    value = self.data('accompaniment')
+    return {**{key: item for key, item in value.items() if key != 'instruments'}, **value['instruments'][part]}
 
 def current():
   return ACTIVE.get() or _default()
 
 def data(name):
-  return current()['data'][name]
+  return current().data(name)
+
+def accompaniment(part, resources=None):
+  if resources is None:
+    return current().accompaniment(part)
+  value = resources['accompaniment']
+  return {**{k: v for k, v in value.items() if k != 'instruments'}, **value['instruments'][part]}
 
 def role(name=None):
-  active = current()
+  return current().role(name)
-  name = name or active['role']
-  return {**active['data']['roles']['roles'].get(name, {}), **active['settings']}
 
 @contextlib.contextmanager
 def using(resources, name, settings=None):
-  token = ACTIVE.set({'data': resources, 'role': name, 'settings': settings or {}})
+  token = ACTIVE.set(CompositionContext.create(resources, name, settings))
   try:
     yield
   finally:

include/vamp_pico/artwork.hpp

@@ -42,13 +42,20 @@ enum class sprite : std::size_t {
   MAIN_MENU,
   PERFORMANCE,
   SETTINGS,
-  PLAYBACK,
+  BACKING,
   METRONOME,
+  SOLOS,
   TEMPO,
   BACK,
   ON,
   OFF,
   BPM,
+  MANUAL,
+  PREPARING,
+  COUNT_IN,
+  PLAYING,
+  SUSTAIN,
+  HOLD_TO_EXIT,
   DIGIT_0,
   DIGIT_1,
   DIGIT_2,

include/vamp_pico/http_client.hpp

@@ -20,15 +20,19 @@ struct remote_display_state {
   std::char_t part[48]{};
   std::char_t mode[24]{};
   std::char_t focus[32]{};
-  std::bool_t playback{};
+  std::bool_t backing{};
+  std::bool_t backing_available{true};
+  std::bool_t error{};
+  std::array<std::char_t, 16> phase{};
   std::bool_t metronome{};
+  std::bool_t solos{true};
   std::bool_t editing{};
   std::uint32_t bpm{120U};
   std::array<std::char_t, vamp_protocol::title_id_size + 1U> title_image{};
   std::uint32_t page{};
   std::uint32_t pages{};
   auto operator==(remote_display_state const &other) const -> std::bool_t {
-    return active == other.active && std::string_view{title} == other.title && std::string_view{part} == other.part && std::string_view{mode} == other.mode && std::string_view{focus} == other.focus && playback == other.playback && metronome == other.metronome && editing == other.editing && bpm == other.bpm && title_image == other.title_image && page == other.page && pages == other.pages;
+    return backing_available == other.backing_available && error == other.error && phase == other.phase && active == other.active && std::string_view{title} == other.title && std::string_view{part} == other.part && std::string_view{mode} == other.mode && std::string_view{focus} == other.focus && backing == other.backing && metronome == other.metronome && solos == other.solos && editing == other.editing && bpm == other.bpm && title_image == other.title_image && page == other.page && pages == other.pages;
   }
 };
 

include/vamp_zero/audio.hpp

@@ -16,6 +16,7 @@
 #include <vamp_zero/config.hpp>
 #include <vamp_zero/types.hpp>
 #include <vamp_zero/score_loader.hpp>
+#include <vamp_zero/performance.hpp>
 
 
 namespace vamp_zero {
@@ -28,19 +29,18 @@ public:
   ~audio_controller();
   auto sync(vamp_zero::app_state const &state) -> void;
   auto tick(vamp_zero::app_state const &state, vamp_zero::score_load_state const &loaded, std::filesystem::path const &assets) -> std::optional<std::expected<void, modulus::error>>;
+  auto position(vamp_zero::score_document const &score, std::uint32_t bpm) const -> std::optional<vamp_zero::performance_position>;
 private:
-  auto make_metronome_config(vamp_zero::app_state const &state) const -> sonus::metronome_config;
   vamp_zero::audio_config config_{};
   sonus::engine engine_;
   mutable std::mutex mutex_{};
-  std::bool_t metronome_running_{false};
-  std::uint32_t last_metronome_bpm_{};
   boost::asio::thread_pool worker_{1};
-  std::future<std::expected<sonus::prepared_sequence, modulus::error>> preparation_{};
+  std::future<std::expected<sonus::prepared_transport, modulus::error>> preparation_{};
   std::uint64_t preparing_session_{};
+  std::shared_ptr<vamp_zero::catalog const> preparing_catalog_{};
   std::optional<std::uint64_t> attempted_session_{};
   std::shared_ptr<vamp_zero::catalog const> catalog_{};
-  sonus::playback_session playback_{};
+  sonus::transport_session playback_{};
   std::bool_t reported_{true};
 };
 

include/vamp_zero/config.hpp

@@ -46,13 +46,9 @@ struct control_config {
 
 struct audio_config {
   std::bool_t enabled{true};
-  std::bool_t playback_enabled{false};
+  std::bool_t backing_enabled{false};
   std::bool_t metronome_enabled{false};
-  std::uint32_t metronome_bpm{120};
+  std::uint32_t tempo_bpm{120};
-  std::uint32_t metronome_min_bpm{40};
-  std::uint32_t metronome_max_bpm{240};
-  std::uint32_t metronome_step{5};
-  std::uint32_t beats_per_bar{4};
   std::float32_t master_volume{0.70F};
   std::float32_t metronome_volume{0.45F};
 };

include/vamp_zero/control.hpp

@@ -25,6 +25,7 @@ public:
   auto refresh_connection() -> std::optional<vamp_zero::app_state>;
   auto set_observer(std::function<void(vamp_zero::app_state)> observer) -> void;
   auto finish_playback(std::uint64_t session, std::string error = {}, std::shared_ptr<vamp_zero::catalog const> catalog = {}) -> void;
+  auto update_performance(std::uint64_t session, std::size_t page, std::string phase, std::shared_ptr<vamp_zero::catalog const> catalog, std::optional<std::size_t> measure = {}) -> void;
 private:
   auto pico_connected_locked() const -> std::bool_t;
   auto snapshot_locked() const -> vamp_zero::app_state;

include/vamp_zero/menu.hpp

@@ -7,6 +7,14 @@
 
 namespace vamp_zero {
 
+namespace settings_row {
+inline constexpr std::size_t backing = 1U;
+inline constexpr std::size_t metronome = 2U;
+inline constexpr std::size_t solos = 3U;
+inline constexpr std::size_t tempo = 4U;
+inline constexpr std::size_t count = 5U;
+}
+
 enum class menu_action_kind { ACTIVATE,
                               PREVIOUS,
                               NEXT,

include/vamp_zero/navigation.hpp

@@ -12,6 +12,7 @@
 namespace vamp_zero {
 
 auto selected_group(vamp_zero::app_state const &state) -> vamp_zero::song_group const *;
+auto visible_parts(vamp_zero::app_state const &state) -> std::vector<std::size_t>;
 auto selected_part(vamp_zero::app_state const &state) -> vamp_zero::library_part const *;
 auto active_part(vamp_zero::app_state const &state) -> vamp_zero::library_part const *;
 auto active_pages(vamp_zero::app_state const &state) -> std::size_t;

include/vamp_zero/performance.hpp

@@ -0,0 +1,28 @@
+#pragma once
+#include <expected>
+#include <filesystem>
+#include <string_view>
+#include <boost/json.hpp>
+#include <sonus.hpp>
+#include <vamp_zero/types.hpp>
+
+
+namespace vamp_zero {
+
+auto parse_beat(boost::json::value const &value) -> std::expected<vamp_zero::beat_position, modulus::error>;
+auto frame_at(vamp_zero::beat_position beat, std::uint32_t bpm) -> std::uint64_t;
+auto available_tempos(vamp_zero::app_state const &state) -> std::vector<std::uint32_t>;
+auto parse_audio_assets(std::string_view json) -> std::expected<std::vector<vamp_zero::audio_asset>, modulus::error>;
+auto validate_audio_assets(vamp_zero::catalog const &catalog, std::filesystem::path const &assets) -> std::expected<void, modulus::error>;
+auto prepare_performance(vamp_zero::score_document const &score, vamp_zero::app_state const &state, std::filesystem::path const &assets, std::float32_t click_volume) -> std::expected<sonus::prepared_transport, modulus::error>;
+
+struct performance_position {
+  std::size_t page{};
+  std::string phase{};
+  std::optional<std::size_t> measure{};
+};
+
+auto performance_at(vamp_zero::score_document const &score, std::uint64_t frames, std::uint32_t bpm) -> vamp_zero::performance_position;
+auto automatic(vamp_zero::audio_state const &audio) -> bool;
+
+}

include/vamp_zero/playback.hpp

@@ -1,18 +0,0 @@
-#pragma once
-
-#include <cstdint>
-#include <expected>
-#include <filesystem>
-
-#include <modulus.hpp>
-#include <sonus.hpp>
-
-#include <vamp_zero/types.hpp>
-
-
-namespace vamp_zero {
-
-auto prepare_playback(vamp_zero::score_document const &score, std::filesystem::path const &assets, std::uint32_t bpm) -> std::expected<sonus::sample_sequence, modulus::error>;
-auto validate_playback_assets(vamp_zero::score_document const &score, std::filesystem::path const &assets) -> std::expected<void, modulus::error>;
-
-}

include/vamp_zero/score.hpp

@@ -28,8 +28,10 @@ private:
   std::vector<entry> entries_{};
 };
 
-auto score_page_count(vamp_zero::score_document const &score) -> std::size_t;
+auto score_page_count(vamp_zero::score_document const &score, bool playback = false) -> std::size_t;
-auto validated_page_path(vamp_zero::score_document const &score, std::size_t index) -> std::expected<std::string, modulus::error>;
+auto validated_page_path(vamp_zero::score_document const &score, std::size_t index, bool playback = false) -> std::expected<std::string, modulus::error>;
-auto validate_score_page(vamp_zero::score_document const &score, std::size_t index) -> std::expected<void, modulus::error>;
+auto validate_score_page(vamp_zero::score_document const &score, std::size_t index, bool playback = false) -> std::expected<void, modulus::error>;
+auto page_rectangle(luma::rect bounds) -> luma::rect;
+auto measure_highlight(vamp_zero::score_page const &page, std::optional<std::size_t> measure, luma::rect image) -> std::optional<luma::rect>;
 
 }

include/vamp_zero/score_loader.hpp

@@ -17,6 +17,7 @@ struct score_load_state {
   std::shared_ptr<vamp_zero::score_document const> document{};
   std::optional<modulus::error> error{};
   std::bool_t loading{};
+  std::bool_t pages_ready{};
 };
 
 class score_loader {
@@ -24,13 +25,15 @@ public:
   // An already-running completion may overlap stop; capture its consumer weakly.
   score_loader(boost::asio::any_io_executor executor,
                std::function<boost::asio::awaitable<std::expected<vamp_zero::score_document, modulus::error>>(vamp_zero::score_reference)> load,
-               std::function<void()> completed = {});
+               std::function<void()> completed = {},
+               std::optional<boost::asio::any_io_executor> worker = {});
   ~score_loader();
   score_loader(score_loader const &) = delete;
   auto operator=(score_loader const &) -> score_loader & = delete;
   auto select(std::shared_ptr<vamp_zero::catalog const> catalog, std::optional<vamp_zero::score_reference> reference) -> void;
   auto snapshot() const -> vamp_zero::score_load_state;
   auto stop() -> void;
+  auto pages_prepared(std::shared_ptr<vamp_zero::score_document const> const &document, std::expected<void, modulus::error> result) -> void;
 private:
   struct state;
   std::shared_ptr<state> state_;

include/vamp_zero/types.hpp

@@ -3,6 +3,8 @@
 #include <cstddef>
 #include <cstdint>
 #include <memory>
+#include <optional>
+#include <array>
 #include <string>
 #include <vector>
 
@@ -11,9 +13,30 @@
 
 namespace vamp_zero {
 
+struct beat_position {
+  std::uint64_t numerator{};
+  std::uint64_t denominator{1};
+};
+
 struct score_page {
   std::string path{};
   std::string sha256{};
+  std::size_t first_measure{};
+  std::size_t measure_count{};
+  vamp_zero::beat_position start_beat{};
+  vamp_zero::beat_position end_beat{};
+  struct measure_label {
+    std::size_t measure{};
+    std::array<double, 4> bounds{};
+  };
+  struct system {
+    std::size_t first_measure{};
+    std::size_t measure_count{};
+    vamp_zero::beat_position start_beat{};
+    vamp_zero::beat_position end_beat{};
+    std::vector<measure_label> labels{};
+  };
+  std::vector<system> systems{};
 };
 
 struct score_document {
@@ -28,6 +51,12 @@ struct score_document {
   std::string source_json{};
   std::size_t measure_count{};
   std::vector<vamp_zero::score_page> pages{};
+  std::string song_id{};
+  vamp_zero::beat_position duration{};
+  vamp_zero::beat_position sustain_start{};
+  std::uint32_t count_in_beats{4};
+  std::vector<vamp_zero::score_page> playback_pages{};
+  std::vector<vamp_zero::beat_position> measure_boundaries{};
 };
 
 struct library_part {
@@ -40,6 +69,9 @@ struct library_part {
   std::size_t measure_count{};
   std::size_t total_pages{};
   std::string title_image{};
+  std::string song_id{};
+  std::size_t playback_pages{};
+  std::bool_t solos_enabled{true};
 };
 
 struct song_group {
@@ -48,9 +80,20 @@ struct song_group {
   std::vector<std::size_t> parts{};
 };
 
+struct audio_asset {
+  std::string path{};
+  std::string sha256{};
+  std::string song_id{};
+  std::uint32_t bpm{};
+  std::uint32_t sample_rate{};
+  std::uint32_t channels{};
+  std::uint64_t frames{};
+};
+
 struct catalog {
   std::vector<vamp_zero::library_part> parts{};
   std::vector<vamp_zero::song_group> groups{};
+  std::vector<vamp_zero::audio_asset> audio_assets{};
 };
 
 auto empty_catalog() -> std::shared_ptr<vamp_zero::catalog const> const &;
@@ -70,12 +113,10 @@ enum class app_mode {
 
 struct audio_state {
   std::string playback_error{};
-  std::bool_t playback_enabled{false};
+  std::string phase{"Manual"};
+  std::bool_t backing_enabled{false};
   std::bool_t metronome_enabled{false};
-  std::uint32_t metronome_bpm{120};
+  std::uint32_t tempo_bpm{120};
-  std::uint32_t metronome_min_bpm{40};
-  std::uint32_t metronome_max_bpm{240};
-  std::uint32_t metronome_step{5};
   std::bool_t tempo_editing{false};
 };
 
@@ -83,6 +124,7 @@ struct app_state {
   std::shared_ptr<vamp_zero::catalog const> catalog{vamp_zero::empty_catalog()};
   vamp_zero::app_mode mode{vamp_zero::app_mode::MAIN_MENU};
   vamp_zero::audio_state audio{};
+  std::bool_t solos_enabled{true};
   std::size_t selected_main_menu_index{};
   std::size_t selected_settings_index{1U};
   std::size_t remembered_settings_index{1U};
@@ -93,6 +135,7 @@ struct app_state {
   std::uint64_t score_session{};
   std::uint64_t input_revision{};
   std::bool_t pico_connected{false};
+  std::optional<std::size_t> active_measure{};
 };
 
 auto to_string(vamp_zero::app_mode value) -> std::string;

include/vamp_zero/ui.hpp

@@ -4,6 +4,7 @@
 #include <cstdint>
 #include <functional>
 #include <memory>
+#include <optional>
 #include <string>
 
 #include <boost/asio/any_io_executor.hpp>
@@ -24,7 +25,7 @@ class score_loader;
 
 class zero_ui {
 public:
-  zero_ui(boost::asio::any_io_executor executor, luma::application_config config, vamp_zero::database_config database, std::string assets_path, std::function<void(vamp_protocol::page_command)> command_handler = {}, std::function<void(vamp_zero::menu_action)> menu_handler = {});
+  zero_ui(boost::asio::any_io_executor executor, luma::application_config config, vamp_zero::database_config database, std::string assets_path, std::function<void(vamp_protocol::page_command)> command_handler = {}, std::function<void(vamp_zero::menu_action)> menu_handler = {}, std::optional<boost::asio::any_io_executor> score_worker = {});
   ~zero_ui();
   zero_ui(zero_ui const &) = delete;
   auto operator=(zero_ui const &) -> zero_ui & = delete;

package/config/vamp.example.json

@@ -37,13 +37,9 @@
   },
   "audio": {
     "enabled": true,
-    "playback_enabled": false,
+    "backing_enabled": false,
     "metronome_enabled": false,
-    "metronome_bpm": 120,
+    "tempo_bpm": 120,
-    "metronome_min_bpm": 40,
-    "metronome_max_bpm": 240,
-    "metronome_step": 5,
-    "beats_per_bar": 4,
     "master_volume": 0.7,
     "metronome_volume": 0.45
   },

src/vamp_pico/remote_display.cpp

@@ -71,12 +71,20 @@ auto vamp_pico::parse_remote_display_body(std::string_view body, vamp_pico::remo
           return false;
         }
         copy_value(value, parsed.focus, sizeof(parsed.focus));
-      } else if (key == "playback" || key == "metronome" || key == "editing") {
+      } else if (key == "phase") {
+        if (value.size() >= parsed.phase.size()) {
+          return false;
+        }
+        std::copy(value.begin(), value.end(), parsed.phase.begin());
+      } else if (key == "backing" || key == "metronome" || key == "solos" || key == "editing" || key == "error" || key == "backing_available") {
         if (value != "0" && value != "1") {
           return false;
         }
-        auto &flag = key == "playback" ? parsed.playback : key == "metronome" ? parsed.metronome
+        auto &flag = key == "backing_available" ? parsed.backing_available : key == "error"     ? parsed.error
-                                                                              : parsed.editing;
+                                                                           : key == "backing"   ? parsed.backing
+                                                                           : key == "metronome" ? parsed.metronome
+                                                                           : key == "solos"     ? parsed.solos
+                                                                                                : parsed.editing;
         flag = value == "1";
       } else if (key == "bpm") {
         if (!parse_uint(value, parsed.bpm) || parsed.bpm == 0U || parsed.bpm > 400U) {

src/vamp_pico/screen.cpp

@@ -11,8 +11,8 @@ auto blend(std::uint16_t background, vamp_style::color foreground, std::uint8_t
 }
 
 auto label_sprite(std::string_view text) -> vamp_pico::sprite {
-  constexpr auto names = std::array<std::string_view, 10>{"Acoustic", "Electric", "Vocal", "Performance", "Settings", "Playback", "Metronome", "Tempo", "Back", "Main Menu"};
+  constexpr auto names = std::array<std::string_view, 11>{"Acoustic", "Electric", "Vocal", "Performance", "Settings", "Backing", "Metronome", "Solos", "Tempo", "Back", "Main Menu"};
-  constexpr auto sprites = std::array{vamp_pico::sprite::ACOUSTIC, vamp_pico::sprite::ELECTRIC, vamp_pico::sprite::VOCAL, vamp_pico::sprite::PERFORMANCE, vamp_pico::sprite::SETTINGS, vamp_pico::sprite::PLAYBACK, vamp_pico::sprite::METRONOME, vamp_pico::sprite::TEMPO, vamp_pico::sprite::BACK, vamp_pico::sprite::MAIN_MENU};
+  constexpr auto sprites = std::array{vamp_pico::sprite::ACOUSTIC, vamp_pico::sprite::ELECTRIC, vamp_pico::sprite::VOCAL, vamp_pico::sprite::PERFORMANCE, vamp_pico::sprite::SETTINGS, vamp_pico::sprite::BACKING, vamp_pico::sprite::METRONOME, vamp_pico::sprite::SOLOS, vamp_pico::sprite::TEMPO, vamp_pico::sprite::BACK, vamp_pico::sprite::MAIN_MENU};
   for (auto i = std::size_t{}; i < names.size(); ++i) {
     if (names[i] == text) {
       return sprites[i];
@@ -24,7 +24,7 @@ auto label_sprite(std::string_view text) -> vamp_pico::sprite {
 }
 
 auto vamp_pico::render_scanline(std::array<std::uint16_t, vamp_pico::display_width> &row, std::size_t y, vamp_pico::remote_display_state const &state, vamp_pico::link_status link, std::bool_t busy, vamp_pico::title_cache const &title, std::string_view atlas) -> void {
-  row.fill(vamp_style::rgb565(y < 44U ? vamp_style::panel : vamp_style::background));
+  row.fill(vamp_style::rgb565(vamp_style::background));
   auto fill = [&](std::size_t left, std::size_t right, vamp_style::color color) {
     std::fill(row.begin() + left, row.begin() + right, vamp_style::rgb565(color));
   };
@@ -40,7 +40,7 @@ auto vamp_pico::render_scanline(std::array<std::uint16_t, vamp_pico::display_wid
     auto image = static_sprite(atlas, value);
     overlay(image, (row.size() - image.width) / 2U, top, ink);
   };
-  auto number = [&](std::string_view text, sprite suffix) {
+  auto number = [&](std::string_view text, sprite suffix, std::size_t top = 192U) {
     auto label = suffix == sprite::COUNT ? coverage_image{} : static_sprite(atlas, suffix);
     auto width = text.size() * digit_width + (label.width == 0U ? 0U : label.width + 8U);
     if (text.empty() || width > row.size() - 16U) {
@@ -49,10 +49,10 @@ auto vamp_pico::render_scanline(std::array<std::uint16_t, vamp_pico::display_wid
     auto x = (row.size() - width) / 2U;
     for (auto character : text) {
       auto value = character == '/' ? sprite::SLASH : static_cast<sprite>(static_cast<std::size_t>(sprite::DIGIT_0) + character - '0');
-      overlay(static_sprite(atlas, value), x, 192U, vamp_style::ink);
+      overlay(static_sprite(atlas, value), x, top, vamp_style::ink);
       x += digit_width;
     }
-    overlay(label, x + 8U, 192U, vamp_style::muted);
+    overlay(label, x + 8U, top, vamp_style::muted);
   };
   centered(sprite::VAMP, 8U, vamp_style::ink);
   if (y >= 18U && y < 26U) {
@@ -66,7 +66,7 @@ auto vamp_pico::render_scanline(std::array<std::uint16_t, vamp_pico::display_wid
   auto mode = std::string_view{state.mode};
   if (has_title || state.active) {
     if (title.ready()) {
-      overlay(title.image(), 12U, 60U, vamp_style::ink);
+      overlay(title.image(), 12U, state.active ? 48U : 60U, vamp_style::ink);
     } else {
       centered(title.failed() || !has_title ? sprite::UNAVAILABLE : sprite::LOADING, 78U, vamp_style::muted);
     }
@@ -79,7 +79,27 @@ auto vamp_pico::render_scanline(std::array<std::uint16_t, vamp_pico::display_wid
     fill(16U, 304U, vamp_style::selection);
     fill(16U, 19U, vamp_style::accent);
   }
-  centered(label_sprite(state.active ? std::string_view{state.part} : std::string_view{state.focus}), 138U, vamp_style::ink);
+  if (state.active) {
+    auto phase = std::string_view{state.phase.data()};
+    auto label = state.error ? sprite::ERROR : phase == "Count-in"  ? sprite::COUNT_IN
+                                             : phase == "Playing"   ? sprite::PLAYING
+                                             : phase == "Sustain"   ? sprite::SUSTAIN
+                                             : phase == "Preparing" ? sprite::PREPARING
+                                                                    : sprite::MANUAL;
+    overlay(static_sprite(atlas, label), 16U, 112U, state.error ? vamp_style::danger : vamp_style::accent);
+    auto digits = std::array<char, 4>{};
+    std::to_chars(digits.data(), digits.data() + 3, state.bpm);
+    auto x = std::size_t{174};
+    for (auto ch : std::string_view{digits.data()}) {
+      overlay(static_sprite(atlas, static_cast<sprite>(static_cast<std::size_t>(sprite::DIGIT_0) + ch - '0')), x, 112U, vamp_style::muted);
+      x += digit_width;
+    }
+    overlay(static_sprite(atlas, sprite::BPM), x + 6U, 112U, vamp_style::muted);
+    centered(label_sprite(state.part), 142U, vamp_style::ink);
+    centered(sprite::HOLD_TO_EXIT, 210U, vamp_style::muted);
+  } else {
+    centered(label_sprite(state.focus), 138U, vamp_style::ink);
+  }
   if (busy || link != link_status::CONNECTED) {
     centered(busy ? sprite::BUSY : link == link_status::CONNECTING ? sprite::CONNECTING
                                  : link == link_status::ERROR      ? sprite::ERROR
@@ -87,7 +107,7 @@ auto vamp_pico::render_scanline(std::array<std::uint16_t, vamp_pico::display_wid
              192U, vamp_style::muted);
   } else if (state.active) {
     auto text = counter_text(state.page, state.pages);
-    number(text.data(), sprite::COUNT);
+    number(text.data(), sprite::COUNT, 176U);
   } else if (mode == "settings_menu") {
     auto focus = std::string_view{state.focus};
     if (focus == "Tempo") {
@@ -98,8 +118,12 @@ auto vamp_pico::render_scanline(std::array<std::uint16_t, vamp_pico::display_wid
       if (state.editing && (y == 184U || y == 226U)) {
         fill(64U, 256U, vamp_style::accent);
       }
-    } else if (focus == "Playback" || focus == "Metronome") {
+    } else if (focus == "Backing" || focus == "Metronome" || focus == "Solos") {
-      centered((focus == "Playback" ? state.playback : state.metronome) ? sprite::ON : sprite::OFF, 192U, vamp_style::accent);
+      if (focus == "Backing" && !state.backing_available) {
+        centered(sprite::UNAVAILABLE, 192U, vamp_style::muted);
+      } else {
+        centered((focus == "Backing" ? state.backing : focus == "Metronome" ? state.metronome : state.solos) ? sprite::ON : sprite::OFF, 192U, vamp_style::accent);
+      }
     }
   }
 }

src/vamp_zero/app.cpp

@@ -22,6 +22,7 @@
 #include <boost/asio/post.hpp>
 #include <boost/asio/signal_set.hpp>
 #include <boost/asio/steady_timer.hpp>
+#include <boost/asio/thread_pool.hpp>
 #include <boost/asio/use_future.hpp>
 #include <boost/json/object.hpp>
 #include <boost/json/serialize.hpp>
@@ -32,6 +33,7 @@
 #include <vamp_zero/app.hpp>
 #include <vamp_zero/config.hpp>
 #include <vamp_zero/audio.hpp>
+#include <vamp_zero/performance.hpp>
 #include <vamp_zero/database.hpp>
 #include <vamp_zero/errors.hpp>
 #include <vamp_zero/health.hpp>
@@ -39,7 +41,7 @@
 #include <vamp_zero/page_control.hpp>
 #include <vamp_zero/control.hpp>
 #include <vamp_zero/score.hpp>
-#include <vamp_zero/playback.hpp>
+
 #include <vamp_zero/ui.hpp>
 
 
@@ -163,12 +165,9 @@ auto load_initial_state(vamp_zero::config const &config) -> std::expected<vamp_z
     .catalog = std::make_shared<vamp_zero::catalog const>(std::move(*performance)),
     .audio =
       {
-        .playback_enabled = config.audio.playback_enabled,
+        .backing_enabled = config.audio.backing_enabled,
         .metronome_enabled = config.audio.metronome_enabled,
-        .metronome_bpm = config.audio.metronome_bpm,
+        .tempo_bpm = config.audio.tempo_bpm,
-        .metronome_min_bpm = config.audio.metronome_min_bpm,
-        .metronome_max_bpm = config.audio.metronome_max_bpm,
-        .metronome_step = config.audio.metronome_step,
       },
   };
   if (auto valid = vamp_zero::validate_app_state(state); !valid) {
@@ -182,20 +181,21 @@ auto validate_score_pages(boost::asio::any_io_executor executor, vamp_zero::conf
   if (!score) {
     co_return std::unexpected(score.error());
   }
-  for (auto page = std::size_t{}; page < vamp_zero::score_page_count(*score); ++page) {
+  for (auto playback : {false, true}) {
-    auto valid = vamp_zero::validate_score_page(*score, page);
+    for (auto page = std::size_t{}; page < vamp_zero::score_page_count(*score, playback); ++page) {
-    if (!valid) {
+      auto valid = vamp_zero::validate_score_page(*score, page, playback);
-      co_return std::unexpected(valid.error());
+      if (!valid) {
+        co_return std::unexpected(valid.error());
+      }
     }
   }
-  auto playback = vamp_zero::validate_playback_assets(*score, config.runtime.assets_path);
-  if (!playback) {
-    co_return std::unexpected(playback.error());
-  }
   co_return std::expected<void, modulus::error>{};
 }
 
 auto validate_catalog(boost::asio::any_io_executor executor, vamp_zero::config const &config, vamp_zero::catalog const &catalog) -> boost::asio::awaitable<std::expected<void, modulus::error>> {
+  if (auto audio = vamp_zero::validate_audio_assets(catalog, config.runtime.assets_path); !audio) {
+    co_return audio;
+  }
   for (auto const &part : catalog.parts) {
     auto title = vamp_zero::validated_title_path(config.runtime.assets_path, part.title_image);
     if (!title) {
@@ -294,11 +294,20 @@ private:
       if (selects_) {
         loader_->select(app.catalog, vamp_zero::selected_score_reference(app));
       }
+      auto loaded = loader_->snapshot();
+      if (selects_) {
+        loaded.pages_ready = true;
+      }
       if (audio_) {
-        if (auto result = audio_->tick(app, loader_->snapshot(), assets_)) {
+        if (auto result = audio_->tick(app, loaded, assets_)) {
           state_.finish_playback(app.score_session, *result ? std::string{} : result->error().describe(), app.catalog);
         }
-      } else if (app.audio.playback_enabled && vamp_zero::selected_score_reference(app) && app.audio.playback_error.empty()) {
+        if (loaded.document) {
+          if (auto position = audio_->position(*loaded.document, app.audio.tempo_bpm)) {
+            state_.update_performance(app.score_session, position->page, position->phase, app.catalog, position->measure);
+          }
+        }
+      } else if (vamp_zero::automatic(app.audio) && vamp_zero::selected_score_reference(app) && app.audio.playback_error.empty()) {
         state_.finish_playback(app.score_session, "Audio output is unavailable");
       }
       schedule();
@@ -314,6 +323,7 @@ private:
 
 auto run_headless(vamp_zero::config const &config, vamp_zero::control_state &state, vamp_zero::audio_controller *audio) -> std::expected<void, modulus::error> {
   auto io = boost::asio::io_context{};
+  auto score_worker = boost::asio::thread_pool{1};
   auto endpoint = make_server_endpoint(config);
   if (!endpoint) {
     return std::unexpected(endpoint.error());
@@ -327,7 +337,7 @@ auto run_headless(vamp_zero::config const &config, vamp_zero::control_state &sta
   if (audio != nullptr) {
     audio->sync(state.snapshot());
   }
-  auto loader = std::make_shared<vamp_zero::score_loader>(io.get_executor(), [executor = io.get_executor(), database = config.database, assets = config.runtime.assets_path](vamp_zero::score_reference reference) { return vamp_zero::async_load_score(executor, database, std::move(reference), assets); });
+  auto loader = std::make_shared<vamp_zero::score_loader>(io.get_executor(), [executor = score_worker.get_executor(), database = config.database, assets = config.runtime.assets_path](vamp_zero::score_reference reference) { return vamp_zero::async_load_score(executor, database, std::move(reference), assets); }, std::function<void()>{}, score_worker.get_executor());
   auto playback = playback_poll{io, state, audio, loader, config.runtime.assets_path, true};
   auto heartbeat = write_heartbeat(config);
   if (!heartbeat) {
@@ -346,11 +356,12 @@ auto run_headless(vamp_zero::config const &config, vamp_zero::control_state &sta
 
 auto run_display(vamp_zero::config const &config, vamp_zero::control_state &state, vamp_zero::audio_controller *audio) -> std::expected<void, modulus::error> {
   auto io = boost::asio::io_context{};
+  auto score_worker = boost::asio::thread_pool{1};
   auto endpoint = make_server_endpoint(config);
   if (!endpoint) {
     return std::unexpected(endpoint.error());
   }
-  auto ui = vamp_zero::zero_ui{io.get_executor(), config.display, config.database, config.runtime.assets_path, [&state](vamp_protocol::page_command command) { static_cast<void>(state.apply(command)); }, [&state](vamp_zero::menu_action action) { static_cast<void>(state.apply(action)); }};
+  auto ui = vamp_zero::zero_ui{io.get_executor(), config.display, config.database, config.runtime.assets_path, [&state](vamp_protocol::page_command command) { static_cast<void>(state.apply(command)); }, [&state](vamp_zero::menu_action action) { static_cast<void>(state.apply(action)); }, score_worker.get_executor()};
   state.set_observer([&ui](vamp_zero::app_state app) {
     auto status = ui.publish(std::move(app));
     if (!status) {

src/vamp_zero/audio.cpp

@@ -10,7 +10,7 @@
 
 #include <vamp_zero/audio.hpp>
 #include <vamp_zero/navigation.hpp>
-#include <vamp_zero/playback.hpp>
+#include <vamp_zero/performance.hpp>
 #include <vamp_zero/errors.hpp>
 
 
@@ -23,74 +23,40 @@ auto score_active(vamp_zero::app_state const &state) -> std::bool_t {
   return false;
 }
 
-auto log_audio_error(std::string const &action, modulus::error const &error) -> void {
-  modulus::log_error(action + ": " + error.describe());
-}
-
 }
 
 vamp_zero::audio_controller::audio_controller(vamp_zero::audio_config config, sonus::engine engine)
     : config_{config}, engine_{std::move(engine)} {}
 
 vamp_zero::audio_controller::~audio_controller() {
+  playback_.stop();
   worker_.join();
   auto lock = std::scoped_lock{mutex_};
-  engine_.stop_metronome();
   engine_.stop_all();
   engine_.stop();
 }
 
-auto vamp_zero::audio_controller::make_metronome_config(vamp_zero::app_state const &state) const -> sonus::metronome_config {
-  auto result = sonus::metronome_config{};
-  result.bpm = state.audio.metronome_bpm;
-  result.beats_per_bar = config_.beats_per_bar;
-  result.volume = config_.metronome_volume;
-  return result;
-}
-
 auto vamp_zero::audio_controller::sync(vamp_zero::app_state const &state) -> void {
   auto lock = std::scoped_lock{mutex_};
-  auto const should_run_metronome = state.audio.metronome_enabled && !state.audio.playback_enabled && score_active(state);
+  if (!vamp_zero::automatic(state.audio) || !score_active(state) || attempted_session_ != state.score_session || catalog_ != state.catalog || !state.audio.playback_error.empty()) {
-  if (!state.audio.playback_enabled || !score_active(state) || attempted_session_ != state.score_session || catalog_ != state.catalog) {
     playback_.stop();
+    reported_ = true;
   }
-  if (!should_run_metronome) {
+}
-    if (metronome_running_) {
+
-      engine_.stop_metronome();
+auto vamp_zero::audio_controller::position(vamp_zero::score_document const &score, std::uint32_t bpm) const -> std::optional<vamp_zero::performance_position> {
-      metronome_running_ = false;
+  auto lock = std::scoped_lock{mutex_};
-    }
+  if (reported_) {
-    return;
+    return std::nullopt;
-  }
-  auto metronome = make_metronome_config(state);
-  if (!metronome_running_) {
-    auto started = engine_.start_metronome(metronome);
-    if (!started) {
-      log_audio_error("failed to start metronome", started.error());
-      engine_.stop_metronome();
-      metronome_running_ = false;
-      return;
-    }
-    metronome_running_ = true;
-    last_metronome_bpm_ = metronome.bpm;
-    return;
-  }
-  if (last_metronome_bpm_ != metronome.bpm) {
-    auto updated = engine_.update_metronome(metronome);
-    if (!updated) {
-      log_audio_error("failed to update metronome", updated.error());
-      engine_.stop_metronome();
-      metronome_running_ = false;
-      return;
-    }
-    last_metronome_bpm_ = metronome.bpm;
   }
+  return vamp_zero::performance_at(score, playback_.frames(), bpm);
 }
 
 auto vamp_zero::audio_controller::tick(vamp_zero::app_state const &state, vamp_zero::score_load_state const &loaded, std::filesystem::path const &assets) -> std::optional<std::expected<void, modulus::error>> {
   sync(state);
   auto lock = std::scoped_lock{mutex_};
   engine_.collect_finished();
-  auto wanted = state.audio.playback_enabled && score_active(state);
+  auto wanted = vamp_zero::automatic(state.audio) && score_active(state) && state.audio.playback_error.empty();
   if (!wanted || catalog_ != state.catalog) {
     attempted_session_.reset();
     reported_ = true;
@@ -101,12 +67,12 @@ auto vamp_zero::audio_controller::tick(vamp_zero::app_state const &state, vamp_z
       return std::nullopt;
     }
     auto prepared = preparation_.get();
-    if (wanted && attempted_session_ == state.score_session && preparing_session_ == state.score_session) {
+    if (wanted && !loaded.error && loaded.pages_ready && attempted_session_ == state.score_session && preparing_session_ == state.score_session && preparing_catalog_ == state.catalog) {
       if (!prepared) {
         reported_ = true;
         return std::unexpected(prepared.error());
       }
-      auto started = engine_.play_sequence(std::move(*prepared));
+      auto started = engine_.play_transport(std::move(*prepared));
       if (!started) {
         reported_ = true;
         return std::unexpected(started.error());
@@ -118,27 +84,26 @@ auto vamp_zero::audio_controller::tick(vamp_zero::app_state const &state, vamp_z
   if (!wanted) {
     return std::nullopt;
   }
+  if (loaded.error) {
+    playback_.stop();
+    reported_ = true;
+    return std::unexpected(*loaded.error);
+  }
   if (attempted_session_ != state.score_session) {
     auto reference = vamp_zero::selected_score_reference(state);
     if (loaded.document && (!reference || loaded.document->id != reference->id)) {
       return std::nullopt;
     }
-    if (loaded.loading || (!loaded.document && !loaded.error)) {
+    if (loaded.loading || !loaded.pages_ready || !loaded.document) {
       return std::nullopt;
     }
     attempted_session_ = state.score_session;
     reported_ = true;
-    if (loaded.error) {
-      return std::unexpected(*loaded.error);
-    }
     preparing_session_ = state.score_session;
-    auto task = std::make_shared<std::packaged_task<std::expected<sonus::prepared_sequence, modulus::error>()>>([score = loaded.document, assets, bpm = state.audio.metronome_bpm]() -> std::expected<sonus::prepared_sequence, modulus::error> {
+    preparing_catalog_ = state.catalog;
+    auto task = std::make_shared<std::packaged_task<std::expected<sonus::prepared_transport, modulus::error>()>>([score = loaded.document, assets, state, volume = config_.metronome_volume]() -> std::expected<sonus::prepared_transport, modulus::error> {
       try {
-        auto sequence = vamp_zero::prepare_playback(*score, assets, bpm);
+        return vamp_zero::prepare_performance(*score, state, assets, volume);
-        if (!sequence) {
-          return std::unexpected(sequence.error());
-        }
-        return sonus::prepared_sequence::create(std::move(*sequence), 48000U);
       } catch (...) {
         return std::unexpected(vamp_zero::make_error(vamp_zero::error_code::INTERNAL, "audio preparation failed"));
       }
@@ -153,7 +118,7 @@ auto vamp_zero::audio_controller::tick(vamp_zero::app_state const &state, vamp_z
       return std::expected<void, modulus::error>{};
     }
     if (playback_.status() == sonus::playback_status::FAILED) {
-      return std::unexpected(vamp_zero::make_error(vamp_zero::error_code::UNAVAILABLE, "sample playback failed"));
+      return std::unexpected(vamp_zero::make_error(vamp_zero::error_code::UNAVAILABLE, "Recording playback failed"));
     }
   }
   return std::nullopt;

src/vamp_zero/config.cpp

@@ -171,42 +171,28 @@ auto parse_audio(boost::json::object const &root, vamp_zero::audio_config fallba
   if (*object == nullptr) {
     return fallback;
   }
-  if ((*object)->if_contains("track_enabled")) {
+  for (auto name : {"track_enabled", "playback_enabled", "metronome_bpm", "metronome_min_bpm", "metronome_max_bpm", "metronome_step", "beats_per_bar"}) {
-    return std::unexpected(invalid("audio.track_enabled is no longer supported; use audio.playback_enabled"));
+    if ((*object)->if_contains(name)) {
+      return std::unexpected(invalid("obsolete audio option: " + std::string{name} + "; use backing_enabled, metronome_enabled and tempo_bpm"));
+    }
   }
   auto enabled = modulus::json_config::optional_bool_field(**object, "enabled", fallback.enabled, field_options);
-  auto playback_enabled = modulus::json_config::optional_bool_field(**object, "playback_enabled", fallback.playback_enabled, field_options);
+  auto backing_enabled = modulus::json_config::optional_bool_field(**object, "backing_enabled", fallback.backing_enabled, field_options);
   auto metronome_enabled = modulus::json_config::optional_bool_field(**object, "metronome_enabled", fallback.metronome_enabled, field_options);
-  auto metronome_bpm = optional_uint32(**object, "metronome_bpm", fallback.metronome_bpm, 1U, 400U);
+  auto tempo_bpm = optional_uint32(**object, "tempo_bpm", fallback.tempo_bpm, 1U, 400U);
-  auto metronome_min_bpm = optional_uint32(**object, "metronome_min_bpm", fallback.metronome_min_bpm, 1U, 400U);
-  auto metronome_max_bpm = optional_uint32(**object, "metronome_max_bpm", fallback.metronome_max_bpm, 1U, 400U);
-  auto metronome_step = optional_uint32(**object, "metronome_step", fallback.metronome_step, 1U, 100U);
-  auto beats_per_bar = optional_uint32(**object, "beats_per_bar", fallback.beats_per_bar, 1U, 16U);
   auto master_volume = optional_float(**object, "master_volume", fallback.master_volume, 0.0F, 1.0F);
   auto metronome_volume = optional_float(**object, "metronome_volume", fallback.metronome_volume, 0.0F, 1.0F);
   if (!enabled) {
     return std::unexpected(enabled.error());
   }
-  if (!playback_enabled) {
+  if (!backing_enabled) {
-    return std::unexpected(playback_enabled.error());
+    return std::unexpected(backing_enabled.error());
   }
   if (!metronome_enabled) {
     return std::unexpected(metronome_enabled.error());
   }
-  if (!metronome_bpm) {
+  if (!tempo_bpm) {
-    return std::unexpected(metronome_bpm.error());
+    return std::unexpected(tempo_bpm.error());
-  }
-  if (!metronome_min_bpm) {
-    return std::unexpected(metronome_min_bpm.error());
-  }
-  if (!metronome_max_bpm) {
-    return std::unexpected(metronome_max_bpm.error());
-  }
-  if (!metronome_step) {
-    return std::unexpected(metronome_step.error());
-  }
-  if (!beats_per_bar) {
-    return std::unexpected(beats_per_bar.error());
   }
   if (!master_volume) {
     return std::unexpected(master_volume.error());
@@ -214,21 +200,14 @@ auto parse_audio(boost::json::object const &root, vamp_zero::audio_config fallba
   if (!metronome_volume) {
     return std::unexpected(metronome_volume.error());
   }
-  if (*metronome_min_bpm > *metronome_max_bpm) {
+  if (*tempo_bpm < 60U || *tempo_bpm > 120U || *tempo_bpm % 10U != 0U) {
-    return std::unexpected(invalid("audio.metronome_min_bpm must be less than or equal to audio.metronome_max_bpm"));
+    return std::unexpected(invalid("audio.tempo_bpm must be 60, 70, 80, 90, 100, 110 or 120"));
-  }
-  if (*metronome_bpm < *metronome_min_bpm || *metronome_bpm > *metronome_max_bpm) {
-    return std::unexpected(invalid("audio.metronome_bpm must be within the configured metronome range"));
   }
   return vamp_zero::audio_config{
     .enabled = *enabled,
-    .playback_enabled = *playback_enabled,
+    .backing_enabled = *backing_enabled,
     .metronome_enabled = *metronome_enabled,
-    .metronome_bpm = *metronome_bpm,
+    .tempo_bpm = *tempo_bpm,
-    .metronome_min_bpm = *metronome_min_bpm,
-    .metronome_max_bpm = *metronome_max_bpm,
-    .metronome_step = *metronome_step,
-    .beats_per_bar = *beats_per_bar,
     .master_volume = *master_volume,
     .metronome_volume = *metronome_volume,
   };

src/vamp_zero/control.cpp

@@ -1,4 +1,5 @@
 #include <vamp_zero/control.hpp>
+#include <vamp_zero/performance.hpp>
 
 #include <chrono>
 #include <array>
@@ -218,13 +219,15 @@ auto vamp_zero::control_state::finish_playback(std::uint64_t session, std::strin
   auto updated = vamp_zero::app_state{};
   {
     auto lock = std::scoped_lock{mutex_};
-    if (!state_.audio.playback_enabled || state_.score_session != session || (catalog && catalog != state_.catalog) || state_.mode != vamp_zero::app_mode::PERFORMANCE_SCORE) {
+    if (!vamp_zero::automatic(state_.audio) || state_.score_session != session || (catalog && catalog != state_.catalog) || state_.mode != vamp_zero::app_mode::PERFORMANCE_SCORE) {
       return;
     }
     if (error.empty()) {
       state_ = vamp_zero::apply_app_command(std::move(state_), vamp_protocol::page_command::SELECT);
     } else {
       state_.audio.playback_error = std::move(error);
+      state_.audio.phase = "Error";
+      state_.active_measure.reset();
     }
     updated = snapshot_locked();
     observer = observer_;
@@ -277,3 +280,26 @@ auto vamp_zero::make_router(vamp_zero::control_state &state) -> ligo::service_ro
   });
   return router;
 }
+
+auto vamp_zero::control_state::update_performance(std::uint64_t session, std::size_t page, std::string phase, std::shared_ptr<vamp_zero::catalog const> catalog, std::optional<std::size_t> measure) -> void {
+  std::function<void(vamp_zero::app_state)> observer;
+  vamp_zero::app_state updated;
+  {
+    auto lock = std::scoped_lock{mutex_};
+    if (!vamp_zero::automatic(state_.audio) || state_.mode != vamp_zero::app_mode::PERFORMANCE_SCORE || state_.score_session != session || state_.catalog != catalog || !state_.audio.playback_error.empty()) {
+      return;
+    }
+    if (state_.page_index == page && state_.audio.phase == phase && state_.active_measure == measure) {
+      return;
+    }
+    state_.page_index = page;
+    state_.audio.phase = std::move(phase);
+    state_.active_measure = measure;
+    ++state_.input_revision;
+    updated = snapshot_locked();
+    observer = observer_;
+  }
+  if (observer) {
+    observer(std::move(updated));
+  }
+}

src/vamp_zero/database.cpp

@@ -20,6 +20,7 @@
 #include <vamp_zero/errors.hpp>
 #include <vamp_zero/library.hpp>
 #include <vamp_zero/score.hpp>
+#include <vamp_zero/performance.hpp>
 
 
 namespace {
@@ -94,7 +95,7 @@ auto valid_title_image(noto::sqlite_value const &value) -> std::bool_t {
 }
 
 auto compatible_catalog(noto::connection &connection) -> boost::asio::awaitable<std::bool_t> {
-  auto columns = std::vector<std::string_view>{"id", "title", "composer", "key_signature", "source_path", "source_json", "measure_count", "total_pages", "title_image", "part"};
+  auto columns = std::vector<std::string_view>{"id", "title", "composer", "key_signature", "source_path", "source_json", "measure_count", "total_pages", "title_image", "part", "solos_enabled"};
   auto parameters = std::vector<noto::sqlite_value>{noto::sqlite_value{"songs"}};
   auto selected = co_await connection.async_execute("SELECT name, type, \"notnull\", pk FROM pragma_table_info(?1) "
                                                     "WHERE EXISTS (SELECT 1 FROM pragma_table_list WHERE schema = 'main' AND name = ?1 AND type = 'table' AND strict = 1) ORDER BY cid",
@@ -109,7 +110,7 @@ auto compatible_catalog(noto::connection &connection) -> boost::asio::awaitable<
     }
     auto required = std::get_if<std::int64_t>(&row[2].value);
     auto primary = std::get_if<std::int64_t>(&row[3].value);
-    auto type = index == 6U || index == 7U ? "INTEGER" : "TEXT";
+    auto type = index == 6U || index == 7U || index == 10U ? "INTEGER" : "TEXT";
     if (text_value(row[0]) != columns[index] || text_value(row[1]) != type || required == nullptr || *required != 1 || primary == nullptr || *primary != (index == 0U ? 1 : 0)) {
       co_return false;
     }
@@ -137,6 +138,17 @@ auto vamp_zero::async_validate_database(boost::asio::any_io_executor executor, v
   if (status && !(co_await compatible_catalog(connection))) {
     status = std::unexpected(vamp_zero::make_error(vamp_zero::error_code::PERSISTENCE, "incompatible songs catalog: " + config.path + "; rebuild or reinstall Product Vamp"));
   }
+  if (status) {
+    auto variants = co_await connection.async_execute(
+      "SELECT json_extract(source_json, '$.music.metadata.song_id'), part FROM songs "
+      "WHERE json_extract(source_json, '$.music.staves[0].arranged') = 1 "
+      "GROUP BY json_extract(source_json, '$.music.metadata.song_id'), part "
+      "HAVING (part IN ('Acoustic', 'Electric') AND (count(*) != 2 OR sum(solos_enabled) != 1)) "
+      "OR (part = 'Vocal' AND (count(*) != 1 OR sum(solos_enabled) != 1)) LIMIT 1");
+    if (!variants || !variants->rows.empty()) {
+      status = std::unexpected(vamp_zero::make_error(vamp_zero::error_code::PERSISTENCE, "missing or duplicate solo variants; rebuild or reinstall Product Vamp"));
+    }
+  }
   if (status && integrity) {
     auto checked = co_await connection.async_quick_check();
     if (!checked || *checked != std::vector<std::string>{"ok"}) {
@@ -158,7 +170,7 @@ auto vamp_zero::async_load_catalog(boost::asio::any_io_executor executor, vamp_z
     co_return std::unexpected(connected.error());
   }
   auto selected =
-    co_await connection.async_prepare("SELECT id, title, composer, part, source_path, measure_count, key_signature, total_pages, title_image FROM "
+    co_await connection.async_prepare("SELECT id, title, composer, part, source_path, measure_count, key_signature, total_pages, title_image, json_extract(source_json, '$.music.metadata.song_id'), json_array_length(source_json, '$.presentation.playback_pages'), solos_enabled FROM "
                                       "songs ORDER BY title, composer, part, id");
   if (!selected) {
     auto close = co_await connection.async_close();
@@ -179,8 +191,9 @@ auto vamp_zero::async_load_catalog(boost::asio::any_io_executor executor, vamp_z
       break;
     }
     auto const &row = **next;
-    if (row.size() != 9U || count_value(row[5]) == 0U || count_value(row[7]) == 0U || !valid_title_image(row[8])) {
+    auto solos = row.size() == 12U ? std::get_if<std::int64_t>(&row[11].value) : nullptr;
-      status = std::unexpected(vamp_zero::make_error(vamp_zero::error_code::PERSISTENCE, "invalid performance metadata"));
+    if (!solos || (*solos != 0 && *solos != 1) || count_value(row[5]) == 0U || count_value(row[7]) == 0U || count_value(row[10]) == 0U || !valid_title_image(row[8])) {
+      status = std::unexpected(vamp_zero::make_error(vamp_zero::error_code::PERSISTENCE, "invalid performance metadata; regenerate score assets"));
       break;
     }
     auto source = resolve_source_path(assets_path, text_value(row[4]));
@@ -198,13 +211,30 @@ auto vamp_zero::async_load_catalog(boost::asio::any_io_executor executor, vamp_z
       .measure_count = count_value(row[5]),
       .total_pages = count_value(row[7]),
       .title_image = text_value(row[8]),
+      .song_id = text_value(row[9]),
+      .playback_pages = count_value(row[10]),
+      .solos_enabled = *solos == 1,
     });
   }
+  auto audio_assets = std::expected<std::vector<vamp_zero::audio_asset>, modulus::error>{};
+  if (status) {
+    auto resources = co_await connection.async_execute("SELECT value FROM resources WHERE id = 'audio'");
+    if (!resources || resources->rows.size() != 1 || resources->rows.front().size() != 1) {
+      status = std::unexpected(vamp_zero::make_error(vamp_zero::error_code::PERSISTENCE, "missing audio catalog; regenerate assets"));
+    } else {
+      audio_assets = vamp_zero::parse_audio_assets(text_value(resources->rows.front().front()));
+      if (!audio_assets) {
+        status = std::unexpected(audio_assets.error());
+      }
+    }
+  }
   auto closed = co_await close_catalog_query(connection, *selected, std::move(status));
   if (!closed) {
     co_return std::unexpected(closed.error());
   }
-  co_return vamp_zero::group_ordered_parts(std::move(parts));
+  auto catalog = vamp_zero::group_ordered_parts(std::move(parts));
+  catalog.audio_assets = std::move(*audio_assets);
+  co_return catalog;
 }
 
 auto vamp_zero::async_load_score(boost::asio::any_io_executor executor, vamp_zero::database_config config, vamp_zero::score_reference reference, std::string assets_path) -> boost::asio::awaitable<std::expected<vamp_zero::score_document, modulus::error>> {

src/vamp_zero/menu.cpp

@@ -4,6 +4,7 @@
 
 #include <vamp_zero/navigation.hpp>
 #include <vamp_zero/page_control.hpp>
+#include <vamp_zero/performance.hpp>
 
 
 auto vamp_zero::menu_count(vamp_zero::app_state const &state) -> std::size_t {
@@ -11,14 +12,11 @@ auto vamp_zero::menu_count(vamp_zero::app_state const &state) -> std::size_t {
   case vamp_zero::app_mode::MAIN_MENU:
     return 2U;
   case vamp_zero::app_mode::SETTINGS_MENU:
-    return 4U;
+    return settings_row::count;
   case vamp_zero::app_mode::PERFORMANCE_SONG_MENU:
     return state.catalog->groups.size() + 1U;
   case vamp_zero::app_mode::PERFORMANCE_PART_MENU:
-    if (auto group = selected_group(state)) {
+    return visible_parts(state).size() + 1U;
-      return group->parts.size() + 1U;
-    }
-    return 1U;
   case vamp_zero::app_mode::PERFORMANCE_SCORE:
     return 0U;
   }
@@ -71,20 +69,26 @@ auto vamp_zero::menu_row(vamp_zero::app_state const &state, std::size_t row) ->
     return {"Back", ""};
   }
   if (state.mode == vamp_zero::app_mode::SETTINGS_MENU) {
-    if (row == 1U) {
+    if (row == settings_row::backing) {
-      return {"Playback", state.audio.playback_enabled ? "On" : "Off"};
+      auto enabled = state;
+      enabled.audio.backing_enabled = true;
+      return {"Backing", vamp_zero::available_tempos(enabled).empty() ? "Unavailable" : state.audio.backing_enabled ? "On"
+                                                                                                                    : "Off"};
     }
-    if (row == 2U) {
+    if (row == settings_row::metronome) {
       return {"Metronome", state.audio.metronome_enabled ? "On" : "Off"};
     }
-    return {"Tempo", std::to_string(state.audio.metronome_bpm) + " BPM"};
+    if (row == settings_row::solos) {
+      return {"Solos", state.solos_enabled ? "On" : "Off"};
+    }
+    return {"Tempo", std::to_string(state.audio.tempo_bpm) + " BPM"};
   }
   if (state.mode == vamp_zero::app_mode::PERFORMANCE_SONG_MENU) {
     auto const &group = state.catalog->groups[row - 1U];
     return {group.title, group.composer};
   }
-  if (auto group = selected_group(state)) {
+  if (auto parts = visible_parts(state); row <= parts.size()) {
-    auto const &part = state.catalog->parts[group->parts[row - 1U]];
+    auto const &part = state.catalog->parts[parts[row - 1U]];
     return {part.part, std::to_string(part.total_pages) + " pages"};
   }
   return {};
@@ -101,10 +105,10 @@ auto vamp_zero::apply_menu_action(vamp_zero::app_state state, vamp_zero::menu_ac
     return state;
   }
   if (action.kind == vamp_zero::menu_action_kind::TEMPO_DOWN || action.kind == vamp_zero::menu_action_kind::TEMPO_UP) {
-    if (state.mode != vamp_zero::app_mode::SETTINGS_MENU || action.row != 3U) {
+    if (state.mode != vamp_zero::app_mode::SETTINGS_MENU || action.row != settings_row::tempo) {
       return state;
     }
-    state.selected_settings_index = 3U;
+    state.selected_settings_index = settings_row::tempo;
     state.audio.tempo_editing = true;
     state = apply_app_command(std::move(state), action.kind == vamp_zero::menu_action_kind::TEMPO_UP ? vamp_protocol::page_command::NEXT : vamp_protocol::page_command::PREVIOUS);
     state.audio.tempo_editing = false;

src/vamp_zero/menu_view.cpp

@@ -5,6 +5,7 @@
 
 #include <vamp_zero/color.hpp>
 #include <vamp_zero/navigation.hpp>
+#include <vamp_zero/performance.hpp>
 
 
 namespace {
@@ -49,7 +50,7 @@ auto vamp_zero::menu_layout(luma::rect bounds, vamp_zero::app_state const &state
   for (auto i = std::size_t{}; i < visible; ++i) {
     auto row = first + i;
     auto rectangle = luma::rect{bounds.x + 36.0F, bounds.y + 112.0F + static_cast<std::float32_t>(i) * 72.0F, bounds.width - 72.0F, 72.0F};
-    if (state.mode == vamp_zero::app_mode::SETTINGS_MENU && row == 3U) {
+    if (state.mode == vamp_zero::app_mode::SETTINGS_MENU && row == settings_row::tempo) {
       add({rectangle.x + rectangle.width - 240.0F, rectangle.y + 12.0F, 48.0F, 48.0F}, vamp_zero::menu_action_kind::TEMPO_DOWN, row);
       add({rectangle.x + rectangle.width - 64.0F, rectangle.y + 12.0F, 48.0F, 48.0F}, vamp_zero::menu_action_kind::TEMPO_UP, row);
     }
@@ -87,12 +88,13 @@ auto vamp_zero::draw_label(luma::draw_context &context, luma::rect bounds, std::
 }
 
 auto vamp_zero::draw_header(luma::draw_context &context, luma::rect bounds, vamp_zero::app_state const &state) -> void {
-  context.fill_rect({bounds.x, bounds.y, bounds.width, 80.0F}, vamp_zero::to_luma(vamp_style::panel));
+  context.fill_rect({bounds.x, bounds.y, bounds.width, 80.0F}, vamp_zero::to_luma(vamp_style::background));
   context.stroke_line({bounds.x + 36.0F, bounds.y + 79.0F}, {bounds.x + bounds.width - 36.0F, bounds.y + 79.0F}, 1.0F, vamp_zero::to_luma(vamp_style::rule));
   draw_label(context, {bounds.x + 36.0F, bounds.y + 8.0F, 260.0F, 60.0F}, "Product Vamp", 30.0F, vamp_style::ink);
   if (auto part = active_part(state); state.mode == vamp_zero::app_mode::PERFORMANCE_SCORE && part != nullptr) {
-    auto summary = part->part + "  " + std::to_string(state.page_index + 1U) + " / " + std::to_string(part->total_pages);
+    auto summary = state.audio.phase + "   " + std::to_string(state.audio.tempo_bpm) + " BPM   " + (vamp_zero::automatic(state.audio) ? "Line " : "Page ") + std::to_string(state.page_index + 1U) + "/" + std::to_string(vamp_zero::active_pages(state));
-    draw_label(context, {bounds.x + 310.0F, bounds.y + 14.0F, bounds.width - 550.0F, 48.0F}, state.audio.playback_error.empty() ? summary : state.audio.playback_error, 20.0F, state.audio.playback_error.empty() ? vamp_style::muted : vamp_style::danger);
+    draw_label(context, {bounds.x + 310.0F, bounds.y + 46.0F, bounds.width - 550.0F, 26.0F}, "Hold to exit", 15.0F, vamp_style::muted);
+    draw_label(context, {bounds.x + 310.0F, bounds.y + 7.0F, bounds.width - 550.0F, 38.0F}, state.audio.playback_error.empty() ? summary : state.audio.playback_error, 20.0F, state.audio.playback_error.empty() ? vamp_style::muted : vamp_style::danger);
   }
   context.fill_ellipse({bounds.x + bounds.width - 184.0F, bounds.y + 33.0F, 10.0F, 10.0F}, vamp_zero::to_luma(state.pico_connected ? vamp_style::connected : vamp_style::muted));
   draw_label(context, {bounds.x + bounds.width - 162.0F, bounds.y + 12.0F, 132.0F, 52.0F}, state.pico_connected ? "Pico connected" : "Pico offline", 17.0F, vamp_style::muted);
@@ -128,8 +130,9 @@ auto vamp_zero::draw_menu(luma::draw_context &context, luma::rect bounds, vamp_z
     draw_label(context, {row.x + 22.0F, row.y + (songs ? 4.0F : 12.0F), row.width - (songs ? 90.0F : 290.0F), songs ? 36.0F : 48.0F}, item.label, 26.0F, vamp_style::ink);
     if (songs) {
       draw_label(context, {row.x + 22.0F, row.y + 39.0F, row.width - 90.0F, 28.0F}, item.detail, 17.0F, vamp_style::muted);
-    } else if (settings && action.row < 3U) {
+    } else if (settings && action.row != settings_row::tempo) {
-      draw_switch(context, row, action.row == 1U ? state.audio.playback_enabled : state.audio.metronome_enabled);
+      draw_label(context, {row.x + row.width - 245.0F, row.y + 12.0F, 145.0F, 48.0F}, item.detail, 20.0F, vamp_style::muted);
+      draw_switch(context, row, action.row == settings_row::backing ? state.audio.backing_enabled : action.row == settings_row::metronome ? state.audio.metronome_enabled : state.solos_enabled);
     } else {
       draw_label(context, {row.x + row.width - (settings ? 181.0F : 250.0F), row.y + 12.0F, settings ? 112.0F : 180.0F, 48.0F}, item.detail, 20.0F, settings && state.audio.tempo_editing ? vamp_style::accent : vamp_style::muted);
     }
@@ -149,7 +152,8 @@ auto vamp_zero::draw_menu(luma::draw_context &context, luma::rect bounds, vamp_z
       continue;
     }
     auto up = kind == vamp_zero::menu_action_kind::TEMPO_UP;
-    auto enabled = up ? state.audio.metronome_bpm < state.audio.metronome_max_bpm : state.audio.metronome_bpm > state.audio.metronome_min_bpm;
+    auto tempos = vamp_zero::available_tempos(state);
+    auto enabled = !tempos.empty() && (up ? state.audio.tempo_bpm < tempos.back() : state.audio.tempo_bpm > tempos.front());
     auto color = vamp_zero::to_luma(enabled ? vamp_style::accent : vamp_style::rule);
     context.stroke_rect(rect, 1.0F, color);
     context.stroke_line({rect.x + 16.0F, rect.y + 24.0F}, {rect.x + 32.0F, rect.y + 24.0F}, 2.0F, color);

src/vamp_zero/navigation.cpp

@@ -1,6 +1,7 @@
 #include <algorithm>
 
 #include <vamp_zero/navigation.hpp>
+#include <vamp_zero/performance.hpp>
 
 
 auto vamp_zero::selected_group(vamp_zero::app_state const &state) -> vamp_zero::song_group const * {
@@ -14,16 +15,31 @@ auto vamp_zero::selected_group(vamp_zero::app_state const &state) -> vamp_zero::
   return &state.catalog->groups[index];
 }
 
+auto vamp_zero::visible_parts(vamp_zero::app_state const &state) -> std::vector<std::size_t> {
+  auto parts = std::vector<std::size_t>{};
+  if (auto group = selected_group(state); group != nullptr) {
+    for (auto index : group->parts) {
+      if (index < state.catalog->parts.size()) {
+        auto const &part = state.catalog->parts[index];
+        if (part.part == "Vocal" || part.solos_enabled == state.solos_enabled) {
+          parts.push_back(index);
+        }
+      }
+    }
+  }
+  return parts;
+}
+
 auto vamp_zero::selected_part(vamp_zero::app_state const &state) -> vamp_zero::library_part const * {
-  auto const *group = vamp_zero::selected_group(state);
+  auto parts = visible_parts(state);
-  if (state.selected_part_index == 0U || group == nullptr || group->parts.empty()) {
+  if (state.selected_part_index == 0U) {
     return nullptr;
   }
   auto index = state.selected_part_index - 1U;
-  if (index >= group->parts.size()) {
+  if (index >= parts.size()) {
     return nullptr;
   }
-  auto part_index = group->parts[index];
+  auto part_index = parts[index];
   return part_index < state.catalog->parts.size() ? &state.catalog->parts[part_index] : nullptr;
 }
 
@@ -40,7 +56,7 @@ auto vamp_zero::active_part(vamp_zero::app_state const &state) -> vamp_zero::lib
 auto vamp_zero::active_pages(vamp_zero::app_state const &state) -> std::size_t {
   if (state.mode == vamp_zero::app_mode::PERFORMANCE_SCORE) {
     auto const *part = vamp_zero::active_part(state);
-    return part == nullptr ? 0U : part->total_pages;
+    return part == nullptr ? 0U : (automatic(state.audio) ? part->playback_pages : part->total_pages);
   }
   return 0U;
 }