Cleanupf

.gitea/workflows/ci.yml

@@ -107,7 +107,7 @@ jobs:
         uses: actions/cache/restore@v4
         with:
           path: build/build_common/engraving-cache
-          key: product-vamp-pages-v1-${{ runner.os }}-${{ hashFiles('generator/engraving/*.py', 'generator/model.py', 'generator/harmony.py', 'generator/requirements.txt', 'package/assets/fonts/petaluma/*') }}-${{ hashFiles('data/**/*.xml', 'data/*.xml') }}
+          key: product-vamp-pages-v1-${{ runner.os }}-${{ hashFiles('generator/engraving/*.py', 'generator/model.py', 'generator/harmony.py', 'generator/requirements.txt', 'package/assets/fonts/petaluma/*') }}-${{ hashFiles('generator/**/*.py', 'generator/*.py', 'data/**/*.xml', 'data/*.xml') }}
           restore-keys: product-vamp-pages-v1-${{ runner.os }}-${{ hashFiles('generator/engraving/*.py', 'generator/model.py', 'generator/harmony.py', 'generator/requirements.txt', 'package/assets/fonts/petaluma/*') }}-
       - name: Build native release
         run: python3 modulus.py build --target x86_64_gcc12_release

TECHNICAL_GUIDE.md

@@ -15,7 +15,11 @@ Product Vamp follows the Modulus product layout used by the khodo repos:
 
 ## Zero Runtime
 
-The Zero process can run headless for local/API quick checks or with Luma for the Waveshare Zero-DISP-7A display. The API is plain HTTP on the private AP:
+The Zero process can run headless for local/API quick checks or with Luma for the Waveshare Zero-DISP-7A display.
+
+`runtime.headless` controls display startup and `audio.enabled` controls audio startup. The unused `runtime.hardware` setting has been removed; older copies of that field are ignored like other unknown configuration fields.
+
+The API is plain HTTP on the private AP:
 
 - `GET /api/status`
 - `GET /api/health`
@@ -116,6 +120,10 @@ Measure durations, event onsets/durations, harmony offsets, span times, and pitc
 
 Practice acoustic accompaniment is derived from structured harmony roots, kinds, degrees, inversions, and slash basses using the shared accompaniment rhythm. A missing harmony or N.C. produces rests, not guessed chords. Electric retains the exact melody, spelling, rhythm, ties, and tuplets; vocal retains those same authored notes and receives scat lyrics. Song vocal lyrics are unchanged. Practice optional-turnaround harmonies are played once, without creating extra sections.
 
+Accompaniment attacks at each bar start and authored harmony onset, sustaining until the next harmony onset or bar end. Harmony carries into subsequent bars until changed or explicitly cancelled by N.C.; melody attacks never suppress accompaniment. This same rule applies to electric accompaniment sections in songs. Song acoustic rhythm and solo-slash patterns are unchanged.
+
+Each source is parsed, validated, and normalized once for all three parts. Part builders treat the prepared source as read-only and own their generated notes and spans. Source-path checks reject colliding song slugs before writing parts: for example, `one-two.xml` and `one_two.xml` would both produce `one-two-acoustic.json` and cannot coexist.
+
 All three parts, for both songs and practice, use octave-transposing notation: written notes sound one octave lower. This includes the vocal part for the singer's range. Unmarked source notation uses that convention by default; copied melodies retain their written pitches and octave numbers. For example, written C4 sounds C3 (A-string fret 3), and written C#6 sounds C#5, within the 0-20 fret range. Generated accompaniment is notated using the same convention.
 
 `pitch.py` resolves explicit MusicXML instrument transposition once before part generation. Explicit zero transposition is also authoritative. Octave-clef exports already encode sounding pitch in their note values, so they do not receive another implicit octave subtraction. Generated parts carry an explicit `transpose.octave_change: -1`; guitar placement and audio read that shared state. Internal concert-pitch normalization and written-pitch conversion cancel for ordinary source notation. There is no octave fitting, vocal-specific range shift, or key transposition.
@@ -136,6 +144,8 @@ The `soft-struck-1` profile generates shared mono 48 kHz PCM16 WAVs with a short
 
 Grace groups honor explicit steal-time-previous/following or make-time; unspecified groups share 25 percent of the following event. Conflicting grace modes, trailing grace groups, unsupported unpitched sound, tied measured tremolo, and incompatible paired tremolo make playback explicitly unavailable while retaining the notation. Measured tremolo uses written subdivision and tuplet ratios. The tone range is 20-18,000 Hz. Asset validation checks containment, SHA-256, PCM format/length, loop endpoints, envelope shape, event ordering, references, and rational timing. Generation remains transactional if validation fails.
 
+When grace groups steal from both ends of one donor, their adjustments accumulate against its original duration. Consuming the entire donor or any chord tone makes playback unavailable rather than overlapping the grace group and principal note.
+
 At runtime the shared on-demand score loader serves both display and audio. A background worker validates and loads only the selected score's samples, converts absolute beat positions to rounded sample frames at the menu BPM, and prepares a Sonus sequence. Pitch does not change with BPM, and rounding absolute positions avoids cumulative timing drift. The I/O loop publishes the prepared sequence and polls completion without doing disk I/O or musical interpretation. Sonus mixes preallocated voices from immutable PCM and retires them on control threads; the callback never synthesizes tones or loads files. Limits are one million events, 512 distinct samples, 64 MiB decoded samples per selection, 64 simultaneous voices including releases, and 24 hours at the selected BPM. Exceeding them reports an error rather than dropping notes.
 
 `audio.playback_enabled` defaults to false and replaces the old inert `track_enabled` setting. The old config name is accepted for migration, but conflicting values fail. Playback suppresses the metronome without changing its stored toggle. Authored tempo marks remain notation; menu BPM is constant throughout a session. Selecting a practice or performance part starts a new session. Manual page changes never restart or seek. Natural completion, including trailing silence/release, returns to the corresponding part menu with its selection preserved. Leaving the score cancels playback; stale results cannot close a newer score. Errors remain visible rather than being mistaken for completion.
@@ -164,6 +174,8 @@ Pages are 976x484 pixels with 12-pixel staff spacing, 28-pixel horizontal margin
 
 Generation validates sources and complete staged output before replacing the previous asset directory. Replacement is always complete; deleted sources cannot leave stale JSON, pages, or audio. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalogue. Runtime packages contain `catalogue.db` with embedded semantic JSON, PNG pages, generated PCM WAVs, and the OTF UI text font, not MusicXML, MuseScore, or Python dependencies. Python test discovery runs model/import/export tests; the normal asset build additionally qualifies and audits every catalogue score.
 
+CI page-cache snapshot keys include every generator Python module and XML source, while retaining a renderer/font-compatible restore prefix. Individual cached pages remain validated by their semantic-content and renderer fingerprints.
+
 Diagnostics identify the file, part, measure, and unsupported element or invalid relationship. Examples include `part P1, measure 4: unsupported musical element <slur>`, `tie stop has no contiguous matching start`, and `optional-chord marker must identify exactly one harmony`.
 
 The six compact source files were replaced, not retained as a second authoring format. Installation replaces the entire catalogue, rather than retaining or migrating rows from older releases. Legacy data must be authored as supported MusicXML and regenerated; the runtime never upgrades old databases.

conanfile.py

@@ -21,6 +21,7 @@ class product_vamp_conan(conan.ConanFile):
   python_requires = "modulus/latest"
   exports = "modulus.json"
   exports_sources = (
+    ".gitea/workflows/ci.yml",
     "CMakeLists.txt",
     "modulus.py",
     "modulus.json",

generator/__main__.py

@@ -10,7 +10,7 @@ from generator.arrange.generate import generate_performance_scores
 from generator.practice.generate import generate_practice_scores
 from generator.score import count_json_files, validate_generated_tree
 from generator.summary import validate_expected_performance_summary, validate_practice_parts
-from generator.musicxml import load_musicxml
+from generator.parts import source_paths
 from generator.engraving import engrave_tree
 from generator.playback import compile_tree
 from generator.catalogue import build_catalogue
@@ -50,11 +50,12 @@ def generate_assets(args: argparse.Namespace) -> None:
     if progress:
       print(f'Assets [{monotonic() - started:.0f}s]: {message}', flush=True)
 
-  report('validating MusicXML sources')
+  report('checking source paths')
   for source in sorted(data_dir.rglob('*.xml')):
     if source.parent not in (data_dir / 'practice', data_dir / 'songs'):
       raise ValueError(f'{source}: MusicXML sources belong directly under data/practice or data/songs')
-    load_musicxml(source)
+  source_paths(data_dir / 'songs')
+  source_paths(data_dir / 'practice', practice=True)
   output_dir.parent.mkdir(parents=True, exist_ok=True)
   with tempfile.TemporaryDirectory(prefix='.vamp-assets-', dir=output_dir.parent) as temporary:
     staged = Path(temporary) / 'assets'

generator/arrange/generate.py

@@ -6,19 +6,19 @@ from pathlib import Path
 
 from generator.arrange.form import full_arrangement_sections
 from generator.guitar import place_guitar
-from generator.parts import PART_TYPES, build_measure, propagated_chords, validate_source
+from generator.parts import PART_TYPES, build_measure, prepare_document, propagated_chords, source_paths
 from generator.model import fraction, key_label, make_event, meter_label, rational
 from generator.musicxml import load_musicxml
-from generator.pitch import concert_document
 from generator.score import copy_source_voices, make_arranged_document, reidentify, slugify, source_staff, write_json
 
 FINAL_SECTION_INDEX = 6
 
 def arrange_document(document: dict, part_type: str) -> dict:
+  return _arrange_document(prepare_document(document), part_type)
+
+def _arrange_document(document: dict, part_type: str) -> dict:
   if part_type not in PART_TYPES:
     raise ValueError(f'unknown arrangement part: {part_type}')
-  validate_source(document)
-  document = concert_document(document, octave_transposing=True)
   staff = source_staff(document)
   source = staff['measures']
   sections = full_arrangement_sections(source, part_type)
@@ -90,14 +90,15 @@ def arrange_document(document: dict, part_type: str) -> dict:
   return result
 
 def generate_performance_scores(source_dir: Path, output_dir: Path) -> int:
-  if not source_dir.exists():
-    raise ValueError(f'performance source directory not found: {source_dir}')
   count = 0
-  for path in sorted(source_dir.glob('*.xml')):
+  for path in source_paths(source_dir):
-    document = load_musicxml(path)
+    try:
+      document = prepare_document(load_musicxml(path))
+    except ValueError as error:
+      raise ValueError(f'{path}: {error}') from error
     for part in PART_TYPES:
       try:
-        write_json(output_dir / f'{slugify(path.stem)}-{part}.json', arrange_document(document, part))
+        write_json(output_dir / f'{slugify(path.stem)}-{part}.json', _arrange_document(document, part))
       except ValueError as error:
         raise ValueError(f'{path}, part {part}: {error}') from error
       count += 1

generator/engraving/layout.py

@@ -1,12 +1,13 @@
 from __future__ import annotations
 
 from dataclasses import replace
+from bisect import bisect_left, bisect_right
 from fractions import Fraction
 import math
 
 from generator import harmony
 from generator.engraving import tablature
-from generator.model import fraction, validate_document
+from generator.model import validate_document
 from generator.engraving.attributes import barline, staff_attributes, tempo
 from generator.engraving.fonts import Fonts
 from generator.engraving.notation import ScoreIndex
@@ -19,7 +20,7 @@ def lyric_text(verse: dict) -> str:
 
 def column(index: ScoreIndex, time: Fraction, start: Fraction, end: Fraction, fonts: Fonts, style: Style) -> tuple[list[Primitive], list[Anchor], list[Fragment]]:
   items, anchors, fragments = [], [], []
-  events = [e for e in index.events if e.start == time or (time == start and e.start < start < e.end)]
+  events = index.events_at(time, include_sustained=time == start)
   grace_width = 0
   occupied = []
   for event in events:
@@ -47,48 +48,45 @@ def column(index: ScoreIndex, time: Fraction, start: Fraction, end: Fraction, fo
         text = lyric_text(verse)
         if text:
           items.append(fonts.text(text, x + style.space / 2, lane * 23, 18, 'lyric', (event.value['id'] + ':lyric:' + verse['number'], event.value['id']), True))
-  for measure_start, measure_end, measure in index.measures:
+  measure_start, _, measure = index.measure_at(time)
-    if not measure_start <= time < measure_end:
+  for owner, _, item in index.harmonies_by_onset.get(time, []):
-      continue
+    if owner is measure:
-    for item in measure['harmonies']:
+      items.append(fonts.text(harmony.symbol(item), 0, 0, 21, 'harmony', (item['id'],)))
-      if measure_start + fraction(item['onset']) == time:
+  for owner, i, value in index.tempos_by_onset.get(time, []):
-        label = harmony.symbol(item)
+    if owner is measure:
-        items.append(fonts.text(label, 0, 0, 21, 'harmony', (item['id'],)))
+      items.extend(tempo(value, fonts, 0, 0, style.space, measure['id'] + f':tempo:{i}'))
-    for i, value in enumerate(measure.get('tempos', [])):
+  if time == measure_start:
-      if measure_start + fraction(value['onset']) == time:
+    items.append(fonts.text(str(measure['number']), 0, -style.space * (index.state(time)['staff_details']['lines'] + 0.5), 12, 'measure-number', (measure['id'],)))
-        items.extend(tempo(value, fonts, 0, 0, style.space, measure['id'] + f':tempo:{i}'))
+    if label := measure.get('section_label'):
-    if time == measure_start:
+      items.append(fonts.text(label, 0, 0, 19, 'section', (measure['id'] + ':section',)))
-      items.append(fonts.text(str(measure['number']), 0, -style.space * (index.state(time)['staff_details']['lines'] + 0.5), 12, 'measure-number', (measure['id'],)))
-      if label := measure.get('section_label'):
-        items.append(fonts.text(label, 0, 0, 19, 'section', (measure['id'] + ':section',)))
   return items, anchors, fragments
 
 def make_system(index: ScoreIndex, start: Fraction, end: Fraction, fonts: Fonts, style: Style, justify: bool = False) -> System:
   items, anchors, fragments = [], [], []
   state = index.state(start)
-  measure = next(m for a, b, m in index.measures if a <= start < b)
+  measure = index.measure_at(start)[2]
   header = staff_attributes(index, state, start, fonts, style.space, measure['id'] + ':initial', ('clef', 'key', 'meter', 'staff_details'))
   items.extend(header)
   cursor = max((p.box.right for p in header), default=0) + 22
   content_left = cursor - 9
-  times = sorted({start} | {t for t in index.columns if start < t < end})
+  times = [start] + index.column_times[bisect_right(index.column_times, start):bisect_left(index.column_times, end)]
   placements = []
   groups = []
   for ti, time in enumerate(times):
     item_start, anchor_start = len(items), len(anchors)
     drawn, placed, pieces = column(index, time, start, end, fonts, style)
     leading = []
-    for measure_start, measure_end, m in index.measures:
+    measure_start, _, m = index.measure_at(time)
-      if time == measure_start and time > start:
+    if time == measure_start and time > start:
-        previous = next(previous for a, b, previous in index.measures if b == time)
+      previous = index.measure_at(time, before=True)[2]
-        leading.extend(barline(previous.get('barlines', {}).get('right', 'regular'), 0, -style.space * (index.state(time)['staff_details']['lines'] - 1), previous['id']))
+      leading.extend(barline(previous.get('barlines', {}).get('right', 'regular'), 0, -style.space * (index.state(time)['staff_details']['lines'] - 1), previous['id']))
-        old_state = index.state(time, before=True)
+      old_state = index.state(time, before=True)
-        fields = tuple(k for k in ('clef', 'key', 'meter', 'staff_details') if old_state[k] != index.state(time)[k])
+      fields = tuple(k for k in ('clef', 'key', 'meter', 'staff_details') if old_state[k] != index.state(time)[k])
-        leading.extend(p.moved(12, 0) for p in staff_attributes(index, index.state(time), time, fonts, style.space, m['id'] + ':initial', fields))
+      leading.extend(p.moved(12, 0) for p in staff_attributes(index, index.state(time), time, fonts, style.space, m['id'] + ':initial', fields))
-      for ci, change in enumerate(m.get('attributes', [])):
+    if time > start:
-        if measure_start + fraction(change['onset']) == time and time > start:
+      for owner, ci, change in index.attributes_by_onset.get(time, []):
-          extra = staff_attributes(index, index.state(time), time, fonts, style.space, m['id'] + f':attributes:{ci}', tuple(change))
+        extra = staff_attributes(index, index.state(time), time, fonts, style.space, owner['id'] + f':attributes:{ci}', tuple(change))
-          leading.extend(p.moved(max((p.box.right for p in leading), default=0) + 10, 0) for p in extra)
+        leading.extend(p.moved(max((p.box.right for p in leading), default=0) + 10, 0) for p in extra)
     if leading:
       width = bounds(leading).right - bounds(leading).left + 14
       items.extend(p.moved(cursor - bounds(leading).left, 0) for p in leading)
@@ -113,7 +111,7 @@ def make_system(index: ScoreIndex, start: Fraction, end: Fraction, fonts: Fonts,
     placements = [(time, x + i * stretch) for i, (time, x) in enumerate(placements)]
     width = available
   right = width - 5
-  last_measure = next(m for a, b, m in index.measures if a < end <= b)
+  last_measure = index.measure_at(end, before=True)[2]
   tab_items = [p for p in items if p.role.startswith('tab')]
   items = [p for p in items if not p.role.startswith('tab')]
   items = draw_beams(index, anchors, items, start, end, content_left, right, style)
@@ -200,7 +198,7 @@ def layout_score(document: dict, style: Style, fonts: Fonts) -> LayoutDocument:
   start = Fraction()
   available = style.width - style.margin_x * 2
   while start < index.end:
-    candidates = sorted(b for b in index.boundaries if b > start)
+    candidates = index.boundary_times[bisect_right(index.boundary_times, start):]
     selected = None
     # Reserve the same title/credit and footer bands when considering a split,
     # so a wide pitch range can break vertically as well as horizontally.
@@ -212,7 +210,7 @@ def layout_score(document: dict, style: Style, fonts: Fonts) -> LayoutDocument:
       selected = trial
     if selected is None:
       measure_end = candidates[0]
-      for end in sorted(t for t in index.columns if start < t < measure_end):
+      for end in index.column_times[bisect_right(index.column_times, start):bisect_left(index.column_times, measure_end)]:
         trial = make_system(index, start, end, fonts, style)
         if trial.box.right - trial.box.left <= available and trial.box.bottom - trial.box.top <= height_limit:
           selected = trial

generator/engraving/notation.py

@@ -26,6 +26,10 @@ class ScoreIndex:
     self.staff = document['staves'][0]
     self.measures = []
     self.events = []
+    self.events_by_onset = {}
+    self.attributes_by_onset = {}
+    self.harmonies_by_onset = {}
+    self.tempos_by_onset = {}
     self.by_id = {}
     self.tones = {}
     self.boundaries = {Fraction()}
@@ -48,18 +52,46 @@ class ScoreIndex:
           for tone in note['pitches']:
             self.tones[tone['id']] = (event, tone)
       for field in ('attributes', 'harmonies', 'tempos'):
-        self.columns.update(time + fraction(item['onset']) for item in measure.get(field, []))
+        bucket = {'attributes': self.attributes_by_onset, 'harmonies': self.harmonies_by_onset, 'tempos': self.tempos_by_onset}[field]
+        for index, item in enumerate(measure.get(field, [])):
+          onset = time + fraction(item['onset'])
+          self.columns.add(onset)
+          bucket.setdefault(onset, []).append((measure, index, item))
       time = end
     self.end = time
     self.measure_starts = [item[0] for item in self.measures]
     self.events.sort(key=lambda e: (e.start, not bool(e.value.get('grace')), e.lane))
+    self.event_starts = []
+    self.event_max_ends = []
+    for event in self.events:
+      self.events_by_onset.setdefault(event.start, []).append(event)
+      self.event_starts.append(event.start)
+      self.event_max_ends.append(max(event.end, self.event_max_ends[-1] if self.event_max_ends else event.end))
+    self.onsets = list(self.events_by_onset)
+    self.column_times = sorted(self.columns)
+    self.boundary_times = sorted(self.boundaries)
     self.lyric_lanes = list(dict.fromkeys((e.voice, verse['number']) for e in self.events for verse in e.value.get('lyrics', [])))
     self.accidentals = self.accidental_state()
+  def measure_at(self, time: Fraction, before: bool = False) -> tuple:
+    position = bisect_left(self.measure_starts, time) if before else bisect_right(self.measure_starts, time)
+    return self.measures[max(0, position - 1)]
+  def events_at(self, time: Fraction, include_sustained: bool = False) -> list[Event]:
+    attacks = self.events_by_onset.get(time, [])
+    if not include_sustained:
+      return attacks
+    sustained = []
+    position = bisect_left(self.event_starts, time) - 1
+    # Prefix maxima let system continuations stop searching once every earlier note has ended.
+    while position >= 0 and self.event_max_ends[position] > time:
+      event = self.events[position]
+      if event.end > time:
+        sustained.append(event)
+      position -= 1
+    return list(reversed(sustained)) + attacks
   def state(self, time: Fraction, before: bool = False) -> dict:
     if (time, before) in self._states:
       return self._states[time, before]
-    position = bisect_left(self.measure_starts, time) if before else bisect_right(self.measure_starts, time)
+    start, _, measure = self.measure_at(time, before)
-    start, _, measure = self.measures[max(0, position - 1)]
     state = {key: measure[key] for key in ('key', 'meter', 'clef', 'transpose', 'staff_details')}
     for change in measure.get('attributes', []):
       onset = start + fraction(change['onset'])
@@ -82,13 +114,13 @@ class ScoreIndex:
     for start, end, measure in self.measures:
       memory = {}
       key = None
-      onsets = sorted({e.start for e in self.events if start <= e.start < end})
+      onsets = self.onsets[bisect_left(self.onsets, start):bisect_left(self.onsets, end)]
       for onset in onsets:
         state = self.state(onset)
         if key != state['key']:
           key, memory = state['key'], {}
         simultaneous = {}
-        for event in (e for e in self.events if e.start == onset):
+        for event in self.events_by_onset[onset]:
           for tone in event.value['pitches']:
             hidden_head = isinstance(tone.get('notehead'), dict) and tone['notehead'].get('name') == 'none'
             if event.value.get('hidden') or tone.get('print', {}).get('print-object') == 'no' or hidden_head:
@@ -108,7 +140,7 @@ class ScoreIndex:
         for identity, values in simultaneous.items():
           memory[identity] = next(iter(values)) if len(values) == 1 else None
         if any(len(values) > 1 for values in simultaneous.values()):
-          for event in (e for e in self.events if e.start == onset):
+          for event in self.events_by_onset[onset]:
             for tone in event.value['pitches']:
               if 'letter' in tone and len(simultaneous.get((tone['letter'], tone['octave']), ())) > 1:
                 result[tone['id']] = True

generator/parts.py

@@ -1,14 +1,36 @@
 from __future__ import annotations
 
 from fractions import Fraction
+from pathlib import Path
 
 from generator.guitar import validate_tuning
 from generator.model import beat_lengths, fraction, make_event
-from generator.pitch import shift_pitch
+from generator.pitch import concert_document, shift_pitch
-from generator.score import copy_source_voices, make_measure, measure_notes, source_staff
+from generator.score import copy_source_voices, make_measure, slugify, source_staff
 
 PART_TYPES = ('acoustic', 'electric', 'vocal')
 
+def source_paths(directory: Path, *, practice: bool = False) -> list[Path]:
+  if not directory.is_dir():
+    raise ValueError(f'{"practice" if practice else "performance"} source directory not found: {directory}')
+  paths = sorted(directory.rglob('*.xml'))
+  identities = {}
+  for path in paths:
+    if practice:
+      if path.parent != directory or len(path.stem) != 3 or not path.stem.isascii() or not path.stem.isdecimal():
+        raise ValueError(f'{path}: practice sources require a three-digit filename directly under data/practice')
+    elif path.parent != directory:
+      raise ValueError(f'{path}: performance sources belong directly under data/songs')
+    identity = path.stem if practice else slugify(path.stem)
+    if identity in identities:
+      raise ValueError(f'{identities[identity]} and {path}: conflicting output path {identity}-acoustic.json')
+    identities[identity] = path
+  return paths
+
+def prepare_document(document: dict, *, practice: bool = False) -> dict:
+  validate_source(document, practice=practice)
+  return concert_document(document, octave_transposing=True)
+
 def propagated_chords(measures: list[dict], *, include_optional: bool = True) -> list[dict | None]:
   result = []
   current = None
@@ -34,8 +56,9 @@ def generated_voice(measure: dict, role: str, previous: dict | None) -> list[dic
   common_time = measure['time_signature'] == '4/4' and not any('meter' in a for a in measure.get('attributes', []))
   if role == 'rhythm' and common_time:
     attacks = [(Fraction(n, 2), Fraction(1, 2)) for n in (0, 3, 4, 5, 7)]
-  elif role == 'accompaniment' and common_time:
+  elif role == 'accompaniment':
-    attacks = [(Fraction(1, 2), Fraction(1, 2))]
+    boundaries = sorted({Fraction(), length} | {fraction(h['onset']) for h in measure['harmonies'] if fraction(h['onset']) < length})
+    attacks = [(start, end - start) for start, end in zip(boundaries, boundaries[1:])]
   else:
     attacks = []
     states = [(Fraction(), measure['meter'])]
@@ -59,11 +82,10 @@ def generated_voice(measure: dict, role: str, previous: dict | None) -> list[dic
       events.append(make_event(f'rest-{len(events)}', cursor, onset - cursor, 'rest'))
     duration = min(duration, length - onset)
     chord = active_chord(measure, onset, previous)
-    source_attack = any(fraction(n['onset']) == onset and n['kind'] == 'note' and not n.get('grace') and any(not p.get('tie_from') for p in n['pitches']) for n in measure_notes(measure))
     if role == 'slash':
       event = make_event(f'slash-{len(events)}', onset, duration, 'note')
       event['notehead'] = 'slash'
-    elif not chord or role == 'accompaniment' and source_attack:
+    elif not chord:
       event = make_event(f'rest-{len(events)}', onset, duration, 'rest')
     else:
       event = make_event(f'chord-{len(events)}', onset, duration, 'note')

generator/playback.py

@@ -75,11 +75,15 @@ def compile_timeline(document: dict) -> dict:
             raise PlaybackUnavailable('grace group must steal between zero and 100 percent')
           lengths = [fraction(donor['duration']) * amount / 100 for amount in percentages]
           total = sum(lengths, Fraction())
+          delay, extension = adjustments.get(donor['id'], (Fraction(), Fraction()))
           if mode == 'steal-time-previous':
             cursor -= total
-            adjustments[donor['id']] = (Fraction(), -total)
           else:
-            adjustments[donor['id']] = (total, -total)
+            delay += total
+          extension -= total
+          if fraction(donor['duration']) + extension <= 0 or any(fraction(p['duration']) + extension <= 0 for p in donor['pitches']):
+            raise PlaybackUnavailable(f'{donor["id"]}: grace timing consumes a donor or chord tone')
+          adjustments[donor['id']] = (delay, extension)
         for item, length in zip(pending, lengths):
           value, _, pitch_state, lane_id = item
           adjustments[value['id']] = (cursor - item[1], length)

generator/practice/generate.py

@@ -4,16 +4,16 @@ from pathlib import Path
 
 from generator.guitar import place_guitar
 from generator.musicxml import load_musicxml
-from generator.parts import PART_TYPES, build_measure, propagated_chords, validate_source
+from generator.parts import PART_TYPES, build_measure, prepare_document, propagated_chords, source_paths
-from generator.pitch import concert_document
 from generator.scat import add_scat
 from generator.score import make_arranged_document, reidentify, source_staff, write_json
 
 def practice_document(document: dict, part_type: str) -> dict:
+  return _practice_document(prepare_document(document, practice=True), part_type)
+
+def _practice_document(document: dict, part_type: str) -> dict:
   if part_type not in PART_TYPES:
     raise ValueError(f'unknown practice part: {part_type}')
-  validate_source(document, practice=True)
-  document = concert_document(document, octave_transposing=True)
   staff = source_staff(document)
   previous = propagated_chords(staff['measures'])
   role = {'acoustic': 'accompaniment', 'electric': 'melody', 'vocal': 'vocal'}[part_type]
@@ -30,17 +30,15 @@ def practice_document(document: dict, part_type: str) -> dict:
   return result
 
 def generate_practice_scores(source_dir: Path, output_dir: Path) -> int:
-  if not source_dir.is_dir():
+  sources = source_paths(source_dir, practice=True)
-    raise ValueError(f'practice source directory not found: {source_dir}')
-  sources = sorted(source_dir.rglob('*.xml'))
   for path in sources:
-    if path.parent != source_dir or len(path.stem) != 3 or not path.stem.isascii() or not path.stem.isdecimal():
+    try:
-      raise ValueError(f'{path}: practice sources require a three-digit filename directly under data/practice')
+      document = prepare_document(load_musicxml(path), practice=True)
-  for path in sources:
+    except ValueError as error:
-    document = load_musicxml(path)
+      raise ValueError(f'{path}: {error}') from error
     for part in PART_TYPES:
       try:
-        write_json(output_dir / path.stem / (part + '.json'), practice_document(document, part))
+        write_json(output_dir / path.stem / (part + '.json'), _practice_document(document, part))
       except ValueError as error:
         raise ValueError(f'{path}, part {part}: {error}') from error
   return len(sources) * len(PART_TYPES)

generator/score.py

@@ -114,7 +114,7 @@ def reidentify(measures: list[dict], source_tuplets: list[dict]) -> list[dict]:
   return tuplets
 
 def make_arranged_document(source_document: dict, staff: dict, *, staff_id: str, staff_name: str, role: str, arrangement_part_type: str, measures: list[dict], tuplets=None, octave_shifts=None) -> dict:
-  return {'schema_version': SCHEMA_VERSION, 'metadata': copy.deepcopy(source_document['metadata']), 'staves': [{'id': staff_id, 'name': staff_name, 'clef': copy.deepcopy(staff['clef']), 'role': role, 'arranged': True, 'arrangement_part_type': arrangement_part_type, 'measures': measures, 'tuplets': tuplets or [], 'octave_shifts': octave_shifts or [], 'instruments': copy.deepcopy(staff.get('instruments', []))}]}
+  return {'schema_version': SCHEMA_VERSION, 'metadata': copy.deepcopy(source_document['metadata']), 'staves': [{'id': staff_id, 'name': staff_name, 'clef': copy.deepcopy(staff['clef']), 'role': role, 'arranged': True, 'arrangement_part_type': arrangement_part_type, 'measures': measures, 'tuplets': tuplets or [], 'octave_shifts': copy.deepcopy(octave_shifts or []), 'instruments': copy.deepcopy(staff.get('instruments', []))}]}
 
 def count_json_files(root: Path) -> int:
   return sum(1 for p in root.rglob('*.json') if p.is_file())

include/vamp_zero/config.hpp

@@ -12,13 +12,11 @@
 #include <modulus.hpp>
 
 #include <modulus/types.hpp>
-#include <vamp_zero/types.hpp>
 
 
 namespace vamp_zero {
 
 struct runtime_config {
-  vamp_zero::hardware_mode hardware{vamp_zero::hardware_mode::REAL};
   std::bool_t headless{false};
   std::bool_t exit_after_startup{false};
   std::string assets_path{"/usr/share/product-vamp"};

include/vamp_zero/types.hpp

@@ -12,11 +12,6 @@
 
 namespace vamp_zero {
 
-enum class hardware_mode {
-  FAKE,
-  REAL,
-};
-
 struct score_page {
   std::string path{};
   std::string sha256{};
@@ -129,7 +124,6 @@ struct app_state {
   std::bool_t pico_connected{false};
 };
 
-auto to_string(vamp_zero::hardware_mode value) -> std::string;
 auto to_string(vamp_zero::app_mode value) -> std::string;
 auto practice_part_names() -> std::vector<std::string_view> const &;
 

package/config/vamp.example.json

@@ -11,7 +11,6 @@
     }
   },
   "runtime": {
-    "hardware": "real",
     "headless": false,
     "exit_after_startup": false,
     "assets_path": "/usr/share/product-vamp",

src/vamp_zero/config.cpp

@@ -111,16 +111,6 @@ auto optional_float(boost::json::object const &object, std::string_view field, s
   return static_cast<std::float32_t>(value);
 }
 
-auto parse_hardware(std::string const &value) -> std::expected<vamp_zero::hardware_mode, modulus::error> {
-  if (value == "fake") {
-    return vamp_zero::hardware_mode::FAKE;
-  }
-  if (value == "real") {
-    return vamp_zero::hardware_mode::REAL;
-  }
-  return std::unexpected(invalid("runtime.hardware must be fake or real"));
-}
-
 auto parse_runtime(boost::json::object const &root, vamp_zero::runtime_config fallback) -> std::expected<vamp_zero::runtime_config, modulus::error> {
   auto object = optional_object(root, "runtime");
   if (!object) {
@@ -129,14 +119,10 @@ auto parse_runtime(boost::json::object const &root, vamp_zero::runtime_config fa
   if (*object == nullptr) {
     return fallback;
   }
-  auto hardware_text = optional_string(**object, "hardware", vamp_zero::to_string(fallback.hardware));
   auto headless = optional_bool(**object, "headless", fallback.headless);
   auto exit_after_startup = optional_bool(**object, "exit_after_startup", fallback.exit_after_startup);
   auto assets_path = optional_string(**object, "assets_path", fallback.assets_path);
   auto heartbeat_path = optional_string(**object, "heartbeat_path", fallback.heartbeat_path);
-  if (!hardware_text) {
-    return std::unexpected(hardware_text.error());
-  }
   if (!headless) {
     return std::unexpected(headless.error());
   }
@@ -149,12 +135,7 @@ auto parse_runtime(boost::json::object const &root, vamp_zero::runtime_config fa
   if (!heartbeat_path) {
     return std::unexpected(heartbeat_path.error());
   }
-  auto hardware = parse_hardware(*hardware_text);
-  if (!hardware) {
-    return std::unexpected(hardware.error());
-  }
   return vamp_zero::runtime_config{
-    .hardware = *hardware,
     .headless = *headless,
     .exit_after_startup = *exit_after_startup,
     .assets_path = std::move(*assets_path),

src/vamp_zero/types.cpp

@@ -9,16 +9,6 @@ auto vamp_zero::empty_catalogue() -> std::shared_ptr<vamp_zero::catalogue const>
   return empty;
 }
 
-auto vamp_zero::to_string(vamp_zero::hardware_mode value) -> std::string {
-  switch (value) {
-  case vamp_zero::hardware_mode::FAKE:
-    return "fake";
-  case vamp_zero::hardware_mode::REAL:
-    return "real";
-  }
-  return "fake";
-}
-
 auto vamp_zero::to_string(vamp_zero::app_mode value) -> std::string {
   switch (value) {
   case vamp_zero::app_mode::MAIN_MENU:

src/vamp_zero/ui.cpp

@@ -152,6 +152,24 @@ auto first_visible_row(std::size_t selected, std::size_t total, std::size_t max_
   return selected - max_rows + 1U;
 }
 
+auto draw_menu_rows(luma::draw_context &context, luma::rect bounds, std::size_t count, std::size_t selected, auto const &draw_item) -> void {
+  auto total = count + 1U;
+  auto visible = std::min<std::size_t>(total, 7U);
+  auto first = first_visible_row(selected, total, visible);
+  for (auto row = std::size_t{}; row < visible; ++row) {
+    auto index = first + row;
+    auto rectangle = luma::rect{.x = bounds.x + 36.0F,
+                                .y = bounds.y + 112.0F + static_cast<std::float32_t>(row) * 54.0F,
+                                .width = bounds.width - 72.0F,
+                                .height = 54.0F};
+    if (index == 0U) {
+      draw_row(context, rectangle, "BACK", "", selected == 0U);
+    } else {
+      draw_item(index - 1U, rectangle, index == selected);
+    }
+  }
+}
+
 auto draw_main_menu(luma::draw_context &context, luma::rect bounds, vamp_zero::app_state const &state) -> void {
   draw_text(context, bounds.x + 36.0F, bounds.y + 28.0F, "MODE", 4, vamp_style::ink, 260.0F);
   draw_text(context, bounds.x + 38.0F, bounds.y + 66.0F, "SELECT WORKFLOW", 1, vamp_style::muted, 320.0F);
@@ -192,37 +210,12 @@ auto draw_main_menu(luma::draw_context &context, luma::rect bounds, vamp_zero::a
 auto draw_performance_song_menu(luma::draw_context &context, luma::rect bounds, vamp_zero::app_state const &state) -> void {
   draw_text(context, bounds.x + 36.0F, bounds.y + 28.0F, "PERFORMANCE", 4, vamp_style::ink, 360.0F);
   draw_text(context, bounds.x + 38.0F, bounds.y + 66.0F, std::to_string(state.catalog->performance.groups.size()) + " SONGS  " + std::to_string(state.catalog->performance.parts.size()) + " PARTS", 1, vamp_style::muted, 320.0F);
-  auto list_y = bounds.y + 112.0F;
+  draw_menu_rows(context, bounds, state.catalog->performance.groups.size(), state.selected_song_index, [&](std::size_t index, luma::rect row, std::bool_t selected) {
-  auto row_height = std::float32_t{54.0F};
+    auto const &group = state.catalog->performance.groups[index];
-  auto row_width = bounds.width - 72.0F;
+    draw_row(context, row, group.title, std::to_string(group.parts.size()) + " PARTS", selected);
-  auto total_rows = state.catalog->performance.groups.size() + 1U;
+  });
-  auto max_rows = std::min<std::size_t>(total_rows, 7U);
-  auto first = first_visible_row(state.selected_song_index, total_rows, max_rows);
-  for (auto row = std::size_t{}; row < max_rows; ++row) {
-    auto index = first + row;
-    if (index == 0U) {
-      draw_row(context,
-               {.x = bounds.x + 36.0F,
-                .y = list_y + static_cast<std::float32_t>(row) * row_height,
-                .width = row_width,
-                .height = row_height},
-               "BACK",
-               "",
-               state.selected_song_index == 0U);
-      continue;
-    }
-    auto const &group = state.catalog->performance.groups[index - 1U];
-    draw_row(context,
-             {.x = bounds.x + 36.0F,
-              .y = list_y + static_cast<std::float32_t>(row) * row_height,
-              .width = row_width,
-              .height = row_height},
-             group.title,
-             std::to_string(group.parts.size()) + " PARTS",
-             index == state.selected_song_index);
-  }
   if (state.catalog->performance.groups.empty()) {
-    draw_text(context, bounds.x + 38.0F, list_y + row_height + 28.0F, "NO SONGS FOUND", 3, vamp_style::retrying, 420.0F);
+    draw_text(context, bounds.x + 38.0F, bounds.y + 194.0F, "NO SONGS FOUND", 3, vamp_style::retrying, 420.0F);
   }
 }
 
@@ -240,77 +233,27 @@ auto draw_performance_part_menu(luma::draw_context &context, luma::rect bounds,
     return;
   }
   draw_text(context, bounds.x + 38.0F, bounds.y + 66.0F, group->composer, 1, vamp_style::muted, bounds.width - 72.0F);
-  auto list_y = bounds.y + 112.0F;
+  draw_menu_rows(context, bounds, group->parts.size(), state.selected_part_index, [&](std::size_t index, luma::rect row, std::bool_t selected) {
-  auto row_height = std::float32_t{54.0F};
+    auto part_index = group->parts[index];
-  auto row_width = bounds.width - 72.0F;
-  auto total_rows = group->parts.size() + 1U;
-  auto max_rows = std::min<std::size_t>(total_rows, 7U);
-  auto first = first_visible_row(state.selected_part_index, total_rows, max_rows);
-  for (auto row = std::size_t{}; row < max_rows; ++row) {
-    auto index = first + row;
-    if (index == 0U) {
-      draw_row(context,
-               {.x = bounds.x + 36.0F,
-                .y = list_y + static_cast<std::float32_t>(row) * row_height,
-                .width = row_width,
-                .height = row_height},
-               "BACK",
-               "",
-               state.selected_part_index == 0U);
-      continue;
-    }
-    auto part_index = group->parts[index - 1U];
     if (part_index >= state.catalog->performance.parts.size()) {
-      continue;
+      return;
     }
     auto const &part = state.catalog->performance.parts[part_index];
-    draw_row(context,
+    draw_row(context, row, part.part, std::to_string(part.total_pages) + " PAGES", selected);
-             {.x = bounds.x + 36.0F,
+  });
-              .y = list_y + static_cast<std::float32_t>(row) * row_height,
-              .width = row_width,
-              .height = row_height},
-             part.part,
-             std::to_string(part.total_pages) + " PAGES",
-             index == state.selected_part_index);
-  }
 }
 
 auto draw_practice_exercise_menu(luma::draw_context &context, luma::rect bounds, vamp_zero::app_state const &state) -> void {
   auto const &exercises = state.catalog->practice.exercises;
   draw_text(context, bounds.x + 36.0F, bounds.y + 28.0F, "PRACTICE", 4, vamp_style::ink, bounds.width - 72.0F);
   draw_text(context, bounds.x + 38.0F, bounds.y + 66.0F, std::to_string(exercises.size()) + " EXERCISES", 1, vamp_style::muted, 320.0F);
-  auto list_y = bounds.y + 112.0F;
+  draw_menu_rows(context, bounds, exercises.size(), state.selected_practice_exercise_index, [&](std::size_t index, luma::rect row, std::bool_t selected) {
-  auto row_height = std::float32_t{54.0F};
+    auto const &exercise = exercises[index];
-  auto row_width = bounds.width - 72.0F;
-  auto total_rows = exercises.size() + 1U;
-  auto max_rows = std::min<std::size_t>(total_rows, 7U);
-  auto first = first_visible_row(state.selected_practice_exercise_index, total_rows, max_rows);
-  for (auto row = std::size_t{}; row < max_rows; ++row) {
-    auto index = first + row;
-    if (index == 0U) {
-      draw_row(context,
-               {.x = bounds.x + 36.0F,
-                .y = list_y + static_cast<std::float32_t>(row) * row_height,
-                .width = row_width,
-                .height = row_height},
-               "BACK",
-               "",
-               state.selected_practice_exercise_index == 0U);
-      continue;
-    }
-    auto const &exercise = exercises[index - 1U];
     auto title = exercise.title == exercise.id ? exercise.id : exercise.id + " - " + exercise.title;
-    draw_row(context,
+    draw_row(context, row, title, "", selected);
-             {.x = bounds.x + 36.0F,
+  });
-              .y = list_y + static_cast<std::float32_t>(row) * row_height,
-              .width = row_width,
-              .height = row_height},
-             title,
-             "",
-             index == state.selected_practice_exercise_index);
-  }
   if (exercises.empty()) {
-    draw_text(context, bounds.x + 38.0F, list_y + row_height + 28.0F, "NO EXERCISES FOUND", 3, vamp_style::retrying, 440.0F);
+    draw_text(context, bounds.x + 38.0F, bounds.y + 194.0F, "NO EXERCISES FOUND", 3, vamp_style::retrying, 440.0F);
   }
 }
 
@@ -324,36 +267,10 @@ auto draw_practice_part_menu(luma::draw_context &context, luma::rect bounds, vam
             vamp_style::ink,
             bounds.width - 72.0F);
   draw_text(context, bounds.x + 38.0F, bounds.y + 66.0F, "SELECT PRACTICE PART", 1, vamp_style::muted, 360.0F);
-  auto list_y = bounds.y + 112.0F;
+  draw_menu_rows(context, bounds, exercise == nullptr ? 0U : exercise->parts.size(), state.selected_practice_part_index, [&](std::size_t index, luma::rect row, std::bool_t selected) {
-  auto row_height = std::float32_t{54.0F};
+    auto const &part = exercise->parts[index];
-  auto row_width = bounds.width - 72.0F;
+    draw_row(context, row, part.part, part.key_signature, selected);
-  auto total_rows = (exercise == nullptr ? 0U : exercise->parts.size()) + 1U;
+  });
-  auto max_rows = std::min<std::size_t>(total_rows, 7U);
-  auto first = first_visible_row(state.selected_practice_part_index, total_rows, max_rows);
-  for (auto row = std::size_t{}; row < max_rows; ++row) {
-    auto index = first + row;
-    if (index == 0U) {
-      draw_row(context,
-               {.x = bounds.x + 36.0F,
-                .y = list_y + static_cast<std::float32_t>(row) * row_height,
-                .width = row_width,
-                .height = row_height},
-               "BACK",
-               "",
-               state.selected_practice_part_index == 0U);
-      continue;
-    }
-    auto part = exercise == nullptr ? std::string{} : exercise->parts[index - 1U].part;
-    auto key = exercise == nullptr ? std::string{} : exercise->parts[index - 1U].key_signature;
-    draw_row(context,
-             {.x = bounds.x + 36.0F,
-              .y = list_y + static_cast<std::float32_t>(row) * row_height,
-              .width = row_width,
-              .height = row_height},
-             part,
-             key,
-             index == state.selected_practice_part_index);
-  }
 }
 
 auto draw_score(luma::draw_context &context, luma::rect bounds, vamp_zero::app_state const &state, vamp_zero::score_load_state const &loaded, score_draw_cache &score_cache) -> void {

tests/test.cpp

@@ -152,6 +152,15 @@ void write_text(std::filesystem::path const &path, std::string const &text) {
 
 }
 
+BOOST_AUTO_TEST_CASE(runtime_config_uses_headless_and_ignores_unknown_fields) {
+  auto parsed = vamp_zero::parse_config(boost::json::parse(R"json({"runtime":{"headless":true,"obsolete_setting":"ignored"},"audio":{"enabled":false}})json"));
+  BOOST_REQUIRE(parsed);
+  BOOST_TEST(parsed->runtime.headless);
+  BOOST_TEST(!parsed->audio.enabled);
+  auto invalid = vamp_zero::parse_config(boost::json::parse(R"json({"runtime":{"headless":"yes"}})json"));
+  BOOST_TEST(!invalid);
+}
+
 BOOST_AUTO_TEST_CASE(product_vamp_audio_config_parses_defaults_and_values) {
   auto defaults = vamp_zero::parse_config(boost::json::parse(R"json({})json"));
   BOOST_REQUIRE(defaults);

tests/test_catalogue.py

@@ -18,7 +18,7 @@ class CatalogueTests(unittest.TestCase):
     self.assertEqual([p.name for p in sources], [f'{i:03}.xml' for i in range(1, 367)])
     expected = {Path(path.stem) / (part + '.json') for path in sources for part in PART_TYPES}
     self.assertEqual(len(expected), 1098)
-    with tempfile.TemporaryDirectory() as temporary, patch('generator.practice.generate.practice_document', side_effect=lambda document, part: document):
+    with tempfile.TemporaryDirectory() as temporary, patch('generator.practice.generate._practice_document', side_effect=lambda document, part: document):
       output = Path(temporary)
       self.assertEqual(generate_practice_scores(source, output), len(expected))
       self.assertEqual({p.relative_to(output) for p in output.rglob('*.json')}, expected)

tests/test_engraving.py

@@ -30,6 +30,45 @@ class EngravingTests(unittest.TestCase):
     actual = rasterize_page(layout.pages[0], self.fonts)
     with Image.open('tests/fixtures/notation-python.png') as expected:
       self.assertIsNone(ImageChops.difference(actual, expected.convert('RGB')).getbbox())
+  def test_accidental_work_is_bounded_by_local_event_count(self):
+    class Unscannable(list):
+      def __iter__(self):
+        raise AssertionError('whole-score event scan')
+    class Counted(list):
+      visited = 0
+      def __iter__(self):
+        for value in super().__iter__():
+          Counted.visited += 1
+          yield value
+    for count in (100, 200, 400):
+      content = ''.join(xml_note(duration=1, kind='quarter', alter=i % 2) for i in range(4))
+      extra = ''.join(f'<measure number="{i + 1}">{content}</measure>' for i in range(1, count))
+      document = parse(xml_score(content).replace('</part></score-partwise>', extra + '</part></score-partwise>'))
+      index = ScoreIndex(document)
+      expected = index.accidentals
+      index.events = Unscannable(index.events)
+      index.events_by_onset = {time: Counted(events) for time, events in index.events_by_onset.items()}
+      Counted.visited = 0
+      self.assertEqual(index.accidental_state(), expected)
+      self.assertEqual(Counted.visited, count * 4)
+  def test_indexed_columns_preserve_sustains_and_timed_annotations(self):
+    from generator.engraving.layout import column
+    class Unscannable(list):
+      def __iter__(self):
+        raise AssertionError('whole-score scan from column layout')
+    content = xml_note(extra='<voice>held</voice>') + '<backup><duration>4</duration></backup>'
+    content += ''.join(xml_note('G', 1, 'quarter', extra='<voice>moving</voice>') for _ in range(4))
+    content += '<harmony><root><root-step>F</root-step></root><kind>major</kind><offset>-2</offset></harmony>'
+    content += '<direction><direction-type><metronome><beat-unit>quarter</beat-unit><per-minute>90</per-minute></metronome></direction-type><offset>-2</offset></direction>'
+    index = ScoreIndex(parse(xml_score(content)))
+    expected = column(index, Fraction(2), Fraction(2), Fraction(4), self.fonts, Style())
+    index.events = Unscannable(index.events)
+    index.measures = Unscannable(index.measures)
+    actual = column(index, Fraction(2), Fraction(2), Fraction(4), self.fonts, Style())
+    self.assertEqual(actual, expected)
+    self.assertEqual(len(index.events_at(Fraction(2), include_sustained=True)), 2)
+    self.assertTrue(any(p.role == 'harmony' for p in actual[0]))
+    self.assertEqual(len(index.events_at(Fraction(4), include_sustained=True)), 0)
   def test_every_written_note_and_rest_value(self):
     for kind in NOTE_TYPES:
       for dots in (0, 1, 3):

tests/test_engraving_cache.py

@@ -13,6 +13,12 @@ from generator.engraving.cache import renderer_fingerprint, restore_pages, score
 from tests.test_musicxml import parse, xml_note, xml_score
 
 class EngravingCacheTests(unittest.TestCase):
+  def test_ci_snapshot_tracks_all_generator_inputs_and_keeps_fallback(self):
+    workflow = Path('.gitea/workflows/ci.yml').read_text()
+    self.assertIn("hashFiles('generator/**/*.py', 'generator/*.py', 'data/**/*.xml', 'data/*.xml')", workflow)
+    self.assertIn('restore-keys: product-vamp-pages-v1-${{ runner.os }}-', workflow)
+    self.assertIn("if: steps.engraving-cache.outputs.cache-hit != 'true'", workflow)
+    self.assertIn('key: ${{ steps.engraving-cache.outputs.cache-primary-key }}', workflow)
   def setUp(self):
     self.temporary = tempfile.TemporaryDirectory()
     self.addCleanup(self.temporary.cleanup)

tests/test_health.cpp

@@ -132,7 +132,7 @@ auto fixture_config(std::filesystem::path const &root) -> vamp_zero::config {
 auto config_file(std::filesystem::path const &root, vamp_zero::config const &config) -> std::string {
   auto json = boost::json::object{
     {"modulus", modulus::to_json(config.modulus)},
-    {"runtime", {{"assets_path", config.runtime.assets_path}, {"heartbeat_path", config.runtime.heartbeat_path}, {"hardware", "real"}, {"headless", false}, {"exit_after_startup", config.runtime.exit_after_startup}}},
+    {"runtime", {{"assets_path", config.runtime.assets_path}, {"heartbeat_path", config.runtime.heartbeat_path}, {"headless", false}, {"exit_after_startup", config.runtime.exit_after_startup}}},
     {"database", {{"path", config.database.path}}},
     {"control", {{"endpoint", ligo::to_json(config.control.endpoint)}, {"zero_ip", config.control.zero_ip}}},
     {"audio", {{"enabled", true}, {"playback_enabled", true}, {"metronome_enabled", true}}},

tests/test_parts.py

@@ -0,0 +1,114 @@
+import argparse
+import copy
+from fractions import Fraction
+import json
+from pathlib import Path
+import tempfile
+import unittest
+from unittest.mock import patch
+
+from generator.__main__ import generate_assets
+from generator.arrange.generate import _arrange_document, arrange_document, generate_performance_scores
+from generator.model import fraction, rational, validate_document
+from generator.musicxml import load_musicxml
+from generator.parts import PART_TYPES, generated_voice, prepare_document
+from generator.pitch import concert_document
+from generator.playback import compile_timeline
+from generator.practice.generate import _practice_document, practice_document
+from tests.test_musicxml import notes, parse, xml_note, xml_score
+
+def xml_harmony(root='C', kind='major', offset=0):
+  return f'<harmony><root><root-step>{root}</root-step></root><kind>{kind}</kind><offset>{offset}</offset></harmony>'
+
+class PartGenerationTests(unittest.TestCase):
+  def test_accompaniment_ignores_melody_attacks_and_sustains_to_changes(self):
+    source = parse(xml_score(xml_harmony() + xml_harmony('F', offset=3) + ''.join(xml_note(duration=1, kind='eighth') for _ in range(8)), divisions=2))
+    for result in (practice_document(source, 'acoustic'), arrange_document(source, 'electric')):
+      validate_document(result)
+      for measure in result['staves'][0]['measures']:
+        events = measure['voices'][0]['notes']
+        if measure.get('section_label') in ('Rhythm Lead-in', 'Guitar Melody', 'Guitar Solo'):
+          continue
+        self.assertEqual([fraction(n['onset']) for n in events], [0, Fraction(3, 2)])
+        self.assertEqual([fraction(n['duration']) for n in events], [Fraction(3, 2), Fraction(5, 2)])
+        self.assertTrue(all(n['pitches'] for n in events))
+      self.assertTrue(compile_timeline(result)['events'])
+  def test_harmony_carry_and_explicit_silence(self):
+    content = xml_harmony(offset=1) + xml_harmony(kind='none', offset=2) + xml_harmony('F', offset=3) + xml_note()
+    source = xml_score(content).replace('</measure></part>', '</measure><measure number="2">' + xml_note() + '</measure><measure number="3">' + xml_harmony(kind='none') + xml_note() + '</measure><measure number="4">' + xml_note() + '</measure></part>')
+    result = practice_document(parse(source), 'acoustic')
+    self.assertEqual([n['kind'] for n in notes(result)], ['rest', 'note', 'rest', 'note'])
+    self.assertEqual([fraction(n['onset']) for n in notes(result)], [0, 1, 2, 3])
+    self.assertEqual([n['kind'] for n in notes(result, 1)], ['note'])
+    self.assertEqual(fraction(notes(result, 1)[0]['duration']), 4)
+    self.assertEqual([n['kind'] for n in notes(result, 2)], ['rest'])
+    self.assertEqual([n['kind'] for n in notes(result, 3)], ['rest'])
+  def test_accompaniment_handles_pickups_meters_and_coincident_markers(self):
+    for duration, kind, attributes, implicit in ((1, 'quarter', '', True), (3, 'half', '<time><beats>3</beats><beat-type>4</beat-type></time>', False)):
+      source = parse(xml_score(xml_harmony() + xml_harmony('F') + xml_note(duration=duration, kind=kind, extra='<dot/>' if duration == 3 else ''), attributes, implicit=implicit))
+      measure = source['staves'][0]['measures'][0]
+      events = generated_voice(measure, 'accompaniment', None)
+      self.assertEqual(len(events), 1)
+      self.assertEqual(fraction(events[0]['duration']), duration)
+      self.assertEqual(events[0]['_harmony']['root']['letter'], 'f')
+    source = parse(xml_score(xml_harmony() + xml_note(duration=1, kind='quarter') + '<attributes><time><beats>6</beats><beat-type>8</beat-type></time></attributes>' + xml_note(duration=3, kind='half', extra='<dot/>')))
+    self.assertEqual(len(generated_voice(source['staves'][0]['measures'][0], 'accompaniment', None)), 1)
+  def test_exercise_006_has_accompaniment_in_every_measure(self):
+    result = practice_document(load_musicxml(Path('data/practice/006.xml')), 'acoustic')
+    self.assertEqual(len(result['staves'][0]['measures']), 12)
+    for measure in result['staves'][0]['measures']:
+      self.assertTrue(any(n['pitches'] for v in measure['voices'] for n in v['notes']), measure['id'])
+    validate_document(result)
+    self.assertTrue(compile_timeline(result)['events'])
+  def test_prepared_document_and_parts_have_independent_ownership(self):
+    source = parse(xml_score(xml_harmony() + xml_note()))
+    source['staves'][0]['octave_shifts'] = [{'id': 'octave', 'start': rational(0), 'end': rational(4), 'octaves': -1}]
+    original = copy.deepcopy(source)
+    for builder, wrapper, practice in ((_practice_document, practice_document, True), (_arrange_document, arrange_document, False)):
+      prepared = prepare_document(source, practice=practice)
+      before = copy.deepcopy(prepared)
+      parts = [builder(prepared, part) for part in PART_TYPES]
+      self.assertEqual(prepared, before)
+      self.assertEqual(parts, [wrapper(source, part) for part in PART_TYPES])
+      vocal = copy.deepcopy(parts[2])
+      parts[1]['metadata']['title'] = 'changed'
+      notes(parts[1])[0]['pitches'].clear()
+      if parts[1]['staves'][0]['octave_shifts']:
+        parts[1]['staves'][0]['octave_shifts'][0]['octaves'] = 9
+      self.assertEqual(parts[2], vocal)
+      self.assertEqual(prepared, before)
+    self.assertEqual(source, original)
+  def test_song_slug_collisions_fail_before_writes(self):
+    for names in (('one-two', 'one_two'), ('Song', 'song'), ('!!!', '???')):
+      with self.subTest(names=names), tempfile.TemporaryDirectory() as directory:
+        root = Path(directory)
+        source, output = root / 'songs', root / 'output'
+        source.mkdir()
+        output.mkdir()
+        (output / 'sentinel').write_text('preserved')
+        for name in names:
+          (source / (name + '.xml')).write_text(xml_score(xml_note()))
+        with self.assertRaisesRegex(ValueError, 'conflicting output path') as raised:
+          generate_performance_scores(source, output)
+        for name in names:
+          self.assertIn(str(source / (name + '.xml')), str(raised.exception))
+        self.assertEqual([p.name for p in output.iterdir()], ['sentinel'])
+  def test_pipeline_parses_and_normalizes_each_source_once(self):
+    with tempfile.TemporaryDirectory() as directory:
+      root = Path(directory)
+      for category, name in (('songs', 'song'), ('practice', '001')):
+        (root / 'data' / category).mkdir(parents=True)
+        (root / 'data' / category / (name + '.xml')).write_text(xml_score(xml_harmony() + xml_note()))
+      args = argparse.Namespace(data_dir=root / 'data', output_dir=root / 'output', static_assets_dir=Path('package/assets'), check=False)
+      with patch('generator.arrange.generate.load_musicxml', wraps=load_musicxml) as song_load, patch('generator.practice.generate.load_musicxml', wraps=load_musicxml) as practice_load, patch('generator.parts.concert_document', wraps=concert_document) as normalize, patch('generator.__main__.engrave_tree'), patch('generator.__main__.build_catalogue'), patch('generator.__main__.compile_tree'):
+        generate_assets(args)
+      song_load.assert_called_once()
+      practice_load.assert_called_once()
+      self.assertEqual(normalize.call_count, 2)
+      original = {p.relative_to(args.output_dir): p.read_bytes() for p in args.output_dir.rglob('*') if p.is_file()}
+      (root / 'data/practice/002.xml').write_text('<broken')
+      with self.assertRaises(ValueError):
+        generate_assets(args)
+      self.assertEqual(original, {p.relative_to(args.output_dir): p.read_bytes() for p in args.output_dir.rglob('*') if p.is_file()})
+      self.assertEqual(len(list(args.output_dir.rglob('*.json'))), 6)
+      self.assertEqual(json.loads((args.output_dir / 'songs/song-vocal.json').read_text())['metadata']['title'], 'Fixture')

tests/test_playback.py

@@ -10,6 +10,23 @@ from generator.playback import PlaybackUnavailable, compile_timeline, generate_t
 from tests.test_musicxml import notes, parse, xml_note, xml_score
 
 class PlaybackTests(unittest.TestCase):
+  def test_grace_steals_accumulate_at_both_ends_of_a_donor(self):
+    grace = '<note><grace {}/><pitch><step>{}</step><octave>4</octave></pitch><type>eighth</type></note>'
+    content = grace.format('steal-time-following="25"', 'D') + xml_note(duration=2, kind='half') + grace.format('steal-time-previous="25"', 'E') + xml_note('F', 2, 'half')
+    result = compile_timeline(parse(xml_score(content)))
+    self.assertEqual([(fraction(e['pitch']), fraction(e['onset']), fraction(e['duration'])) for e in result['events']], [(62, 0, Fraction(1, 2)), (60, Fraction(1, 2), 1), (64, Fraction(3, 2), Fraction(1, 2)), (65, 2, 2)])
+    for percent in (50, 60):
+      with self.subTest(percent=percent), self.assertRaisesRegex(PlaybackUnavailable, 'grace timing consumes'):
+        compile_timeline(parse(xml_score(content.replace('"25"', f'"{percent}"'))))
+    chord = content.replace(xml_note(duration=2, kind='half'), xml_note(duration=2, kind='half') + xml_note('G', 1, 'quarter', prefix='<chord/>'))
+    with self.assertRaisesRegex(PlaybackUnavailable, 'chord tone'):
+      compile_timeline(parse(xml_score(chord)))
+  def test_grace_previous_and_make_time_preserve_their_timing(self):
+    for mode, expected, total in [('steal-time-previous="25"', [(0, Fraction(3, 2)), (Fraction(3, 2), Fraction(1, 2)), (2, 2)], 4), ('make-time="1"', [(0, 2), (2, 1), (3, 2)], 5)]:
+      grace = f'<note><grace {mode}/><pitch><step>D</step><octave>4</octave></pitch><type>eighth</type></note>'
+      result = compile_timeline(parse(xml_score(xml_note(duration=2, kind='half') + grace + xml_note('E', 2, 'half'))))
+      self.assertEqual([(fraction(e['onset']), fraction(e['duration'])) for e in result['events']], expected)
+      self.assertEqual(fraction(result['duration']), total)
   def test_chords_and_trailing_rest(self):
     document = parse(xml_score(xml_note(duration=2, kind='half') + xml_note('E', 1, 'quarter', prefix='<chord/>') + '<note><rest/><duration>2</duration><type>half</type></note>'))
     result = compile_timeline(document)