Scale Patterns

.gitea/workflows/ci.yml

@@ -112,7 +112,7 @@ jobs:
         uses: actions/cache/restore@v4
         with:
           path: build/build_common/engraving-cache/${{ steps.renderer.outputs.fingerprint }}
-          key: product-vamp-pages-${{ steps.renderer.outputs.fingerprint }}-${{ hashFiles('generator/**/*.py', 'generator/*.py', 'data/songs/*.xml', 'data/rhythms/**/*.xml') }}
+          key: product-vamp-pages-${{ steps.renderer.outputs.fingerprint }}-${{ hashFiles('generator/**/*.py', 'generator/*.py', 'data/songs/*.xml', 'data/rhythms/**/*.xml', 'data/scales/**/*.xml') }}
           restore-keys: product-vamp-pages-${{ steps.renderer.outputs.fingerprint }}-
       - name: Build native release
         run: python3 modulus.py build --target x86_64_gcc12_release

README.md

@@ -90,7 +90,7 @@ Uncompressed MusicXML 3.1/4.0 scores exported from Soundslice or MuseScore live
 
 `data/songs` supplies the authored music. `data/rhythms/<meter>/<series>/` supplies the accompaniment rhythm library for 2/4, 3/4, 4/4, and 5/4. All three series are active: the original quarter/eighth patterns plus series `3` with half, dotted-half, and whole notes/rests. Rhythm snippets are build inputs, not separate catalog entries or runtime files. Generation loads the library once and shares it across parts.
 
-`data/scales` is reserved for unfinished scale sources. These files are not parsed, validated, generated, or packaged, and editing them does not trigger asset regeneration.
+`data/scales` supplies the 21 C-centered modes of major, melodic minor, and harmonic minor. The Python build transposes these patterns internally to label the acoustic and electric solo passages; it never transposes the songs or stores scale assignments in song XML. Scale changes appear above the existing solo chord symbols, with detailed build-only reports in `generated-assets/analysis/`. Editing a scale regenerates assets, and invalid patterns fail before replacing the previous output.
 
 To inspect generated assets without building the full package, run:
 

TECHNICAL_GUIDE.md

@@ -171,6 +171,18 @@ At runtime the shared on-demand score loader serves both display and audio. A ba
 
 `--once` additionally validates referenced audio without opening display/audio hardware. Runtime packages contain WAVs alongside pages and the SQLite catalog; the source XML and Python dependencies remain build-only.
 
+### Solo Scale Maps
+
+`generator/scales/` loads the 21 MusicXML patterns in `data/scales` once per build. Each file spells an ascending C-centered octave; seven distinct degrees define its intervals, including spellings such as B double-flat. The seven files in each family must be rotations of that family's first mode. The library transposes patterns internally, not song notes, and searches the 36 distinct pitch collections formed by the three families in twelve roots. No assignments or overrides are stored in song XML.
+
+The analyzer requires a declared tonic and mode in the song's MusicXML key. A one-flat signature with `<mode>minor</mode>` means D minor, not F major. Exact rational harmony offsets are read after arrangement and pickup overlap handling. Full chord semantics, explicit extensions and alterations, degree omissions, and slash basses constrain compatibility; reduced guitar grips never remove these requirements. Existing chord syntax is unchanged: `7alt` shorthand is still rejected, while explicit alterations are supported.
+
+Selection first preserves every required pitch. Recognized ii-V pairs, resolving dominants, and leading-tone diminished resolutions establish local destinations; the declared key provides the default context elsewhere. Minor cadences prefer harmonic minor and major cadences prefer the destination's major collection when compatible. The arrival chord is evaluated afresh, so a plain D-minor tonic need not retain a preceding cadence's raised seventh. Among equally appropriate choices, a deterministic dynamic program minimizes collection changes, then contextual pitch differences. Equivalent modes share a pitch collection rather than forcing new boundaries. This is a conservative harmonic heuristic, not a claim that ambiguous harmony has only one interpretation.
+
+Only the soloing part receives compact `scale_spans`, with absolute rational start/end offsets, labels, and matched status. `N.C.` and solo boundaries stop annotations. Accompaniment, melody, pitches, TAB, and playback are unchanged; no solo notes are synthesized. Original melody notes are not constraints on these slash-only solo passages. Unmatched chords print `No exact scale match` and emit a source/measure/beat warning without changing the harmony. Invalid scale data or missing tonal metadata instead fail generation before publication.
+
+Scale labels appear above chord symbols at collection changes and system starts. Detailed parent collections, source and arranged positions, chord-root modes, degrees, chord-tone targets, and selection reasons are written to `generated-assets/analysis/<song>.txt`. These reports are published transactionally with the build output but excluded from both installation and Docker runtime assets. The C++ runtime only displays the generated pages; it does not load the scale library or run harmonic analysis.
+
 ### Engraving Boundary
 
 `generator/engraving/` is a custom Python engraver, not a wrapper around MuseScore or another notation engine. `generator.engraving.layout.layout_score(document, style, fonts)` produces immutable layout pages with exact source intervals, notation anchors and drawing primitives. `generator.engraving.raster.rasterize_page(page, fonts)` draws one page at four-times resolution and downsamples it. `generator.engraving.pipeline.engrave_tree(root, fonts)` is the asset-pipeline entry point. `fonts.Fonts`, `types.Style`, and `types.EngravingError` live in their respective engraving modules; page constants and `check_page` live in `validation`. Package initializers do not re-export these APIs. `generator/musicxml_writer.py` remains a round-trip/export utility and is not involved in engraving.

cmake/runtime_assets.cmake

@@ -1,4 +1,4 @@
 if(NOT EXISTS "${SOURCE}/catalog.db")
   message(FATAL_ERROR "Missing generated catalog: ${SOURCE}/catalog.db")
 endif()
-file(COPY "${SOURCE}/" DESTINATION "${DESTINATION}" PATTERN "*.json" EXCLUDE)
+file(COPY "${SOURCE}/" DESTINATION "${DESTINATION}" PATTERN "*.json" EXCLUDE PATTERN "analysis" EXCLUDE)

cmake/zero.cmake

@@ -139,7 +139,6 @@ file(GLOB_RECURSE VAMP_GENERATOR_INPUTS CONFIGURE_DEPENDS
   ${CMAKE_CURRENT_SOURCE_DIR}/package/assets/fonts/*
   ${CMAKE_CURRENT_SOURCE_DIR}/package/assets/fonts/**/*
 )
-list(FILTER VAMP_GENERATOR_INPUTS EXCLUDE REGEX "/data/scales/")
 set(VAMP_GENERATED_ASSETS_DIR ${CMAKE_CURRENT_BINARY_DIR}/generated-assets)
 set(VAMP_GENERATED_ASSETS_STAMP ${CMAKE_CURRENT_BINARY_DIR}/generated-assets.stamp)
 set(VAMP_ENGRAVING_JOBS 2 CACHE STRING "Maximum parallel Python engraving workers (1-16)")
@@ -180,7 +179,7 @@ install(TARGETS vamp-zero
 )
 install(FILES package/config/vamp.example.json DESTINATION zero/config)
 install(DIRECTORY ${VAMP_GENERATED_ASSETS_DIR}/ DESTINATION zero/share/product-vamp
-  PATTERN "*.json" EXCLUDE)
+  PATTERN "*.json" EXCLUDE PATTERN "analysis" EXCLUDE)
 install(PROGRAMS
   package/zero/install-zero.sh
   package/zero/start-ap.sh

conanfile.py

@@ -32,6 +32,7 @@ class product_vamp_conan(conan.ConanFile):
     "data/expected/*.json",
     "data/songs/*.xml",
     "data/rhythms/**/*.xml",
+    "data/scales/**/*.xml",
     "tests/fixtures/*",
     "tests/fixtures/**/*",
     "generator/*.py",

data/expected/performance_summary.json

@@ -2,7 +2,7 @@
   "autumn-leaves-acoustic.json": {
     "title": "autumn-leaves",
     "composer": "",
-    "key_signature": "F",
+    "key_signature": "D minor",
     "part_type": "acoustic",
     "measure_count": 176,
     "section_labels": [
@@ -2074,7 +2074,7 @@
   "autumn-leaves-electric.json": {
     "title": "autumn-leaves",
     "composer": "",
-    "key_signature": "F",
+    "key_signature": "D minor",
     "part_type": "electric",
     "measure_count": 176,
     "section_labels": [
@@ -4146,7 +4146,7 @@
   "autumn-leaves-vocal.json": {
     "title": "autumn-leaves",
     "composer": "",
-    "key_signature": "F",
+    "key_signature": "D minor",
     "part_type": "vocal",
     "measure_count": 176,
     "section_labels": [

data/scales/harmonic-minor-scale-modes/0003.xml

@@ -66,14 +66,6 @@
       </note>
     </measure>
     <measure number="2">
-      <note>
-        <pitch>
-          <step>G</step>
-          <octave>4</octave>
-        </pitch>
-        <duration>4</duration>
-        <type>quarter</type>
-      </note>
       <note>
         <pitch>
           <step>G</step>
@@ -84,6 +76,14 @@
         <type>quarter</type>
         <accidental>sharp</accidental>
       </note>
+      <note>
+        <pitch>
+          <step>A</step>
+          <octave>4</octave>
+        </pitch>
+        <duration>4</duration>
+        <type>quarter</type>
+      </note>
       <note>
         <pitch>
           <step>B</step>

data/songs/autumn-leaves.xml

@@ -20,7 +20,7 @@
         <divisions>4</divisions>
         <key>
           <fifths>-1</fifths>
-          <mode>major</mode>
+          <mode>minor</mode>
         </key>
         <time>
           <beats>4</beats>

generator/__main__.py

@@ -16,6 +16,7 @@ import generator.parts
 import generator.pico_artwork
 import generator.playback
 import generator.rhythm
+import generator.scales.library
 import generator.score
 import generator.summary
 
@@ -67,13 +68,15 @@ def generate_assets(args: argparse.Namespace) -> None:
   generator.parts.source_paths(data_dir / 'songs')
   report('loading rhythm snippets')
   rhythms = generator.rhythm.load_library(data_dir / 'rhythms')
+  report('loading solo scale patterns')
+  scales = generator.scales.library.load_library(data_dir / 'scales')
   output_dir.parent.mkdir(parents=True, exist_ok=True)
   with tempfile.TemporaryDirectory(prefix='.vamp-assets-', dir=output_dir.parent) as temporary:
     staged = pathlib.Path(temporary) / 'assets'
     staged.mkdir(parents=True, exist_ok=True)
     copy_runtime_fonts(static_assets_dir, staged)
     report('generating performance parts')
-    performance_count = generator.arrange.generate.generate_performance_scores(data_dir / 'songs', staged / 'songs', rhythms=rhythms)
+    performance_count = generator.arrange.generate.generate_performance_scores(data_dir / 'songs', staged / 'songs', rhythms=rhythms, scales=scales)
     generator.score.validate_generated_tree(staged)
     if args.check:
       generator.summary.validate_expected_performance_summary(data_dir / 'expected' / 'performance_summary.json', staged / 'songs')

generator/arrange/generate.py

@@ -15,14 +15,17 @@ import generator.model
 import generator.musicxml
 import generator.parts
 import generator.rhythm
+import generator.scales.library
+import generator.scales.mapping
+import generator.scales.timeline
 import generator.score
 
 FINAL_SECTION_INDEX = 6
 
-def arrange_document(document: dict, part_type: str, *, rhythms: generator.rhythm.Library) -> dict:
+def arrange_document(document: dict, part_type: str, *, rhythms: generator.rhythm.Library, scales: generator.scales.library.Library | None = None) -> dict:
   document = generator.parts.prepare_document(document)
   plans = plan_arrangement(document, rhythms) if part_type != 'vocal' else {}
-  return _arrange_document(document, part_type, plans)
+  return _arrange_document(document, part_type, plans, scales=scales)
 
 def section_source(source: list[dict], section, next_section=None) -> list[dict]:
   result = copy.deepcopy(source)
@@ -48,13 +51,14 @@ def plan_arrangement(document: dict, rhythms: generator.rhythm.Library) -> dict[
     result[section.section_index] = generator.accompaniment.plan_section(measures[start:start + section.measure_count], rhythms, initial=previous[start], electric=section.section_index in (1, 5, 6))
   return result
 
-def _arrange_document(document: dict, part_type: str, plans: dict[int, generator.accompaniment.Plan]) -> dict:
+def _arrange_document(document: dict, part_type: str, plans: dict[int, generator.accompaniment.Plan], *, scales: generator.scales.library.Library | None = None, reports: list[str] | None = None, source_name: str = '') -> dict:
   if part_type not in generator.parts.PART_TYPES:
     raise ValueError(f'unknown arrangement part: {part_type}')
   staff = generator.score.source_staff(document)
   source = staff['measures']
   sections = generator.arrange.form.full_arrangement_sections(source, part_type)
   output, tuplets, octave_shifts, voicing_origins = [], [], [], []
+  solos = []
   absolute = fractions.Fraction()
   source_starts = []
   source_time = fractions.Fraction()
@@ -76,6 +80,8 @@ def _arrange_document(document: dict, part_type: str, plans: dict[int, generator
       if part_type == 'electric' and section.role in ('melody', 'accompaniment'):
         voicing_origins.append(generator.electric.MeasureOrigin(source_starts[si], section.role, result))
     segment_length = sum((generator.model.fraction(m['duration']) for m in segment), fractions.Fraction())
+    if section.role == 'slash':
+      solos.append(generator.scales.timeline.Solo(absolute, absolute + segment_length, section.label))
     if section.role in ('vocal', 'melody'):
       start = source_starts[section.source_measure_index]
       for shift in staff.get('octave_shifts', []):
@@ -112,9 +118,13 @@ def _arrange_document(document: dict, part_type: str, plans: dict[int, generator
       generator.guitar.place_guitar(result)
     generator.electric.reuse_voicings(voicing_links)
     generator.acoustic.combine(result)
+  if scales and solos:
+    report = generator.scales.mapping.annotate(result, solos, scales, source_name or document['metadata']['title'])
+    if reports is not None:
+      reports.append(report)
   return result
 
-def generate_performance_scores(source_dir: pathlib.Path, output_dir: pathlib.Path, *, rhythms: generator.rhythm.Library) -> int:
+def generate_performance_scores(source_dir: pathlib.Path, output_dir: pathlib.Path, *, rhythms: generator.rhythm.Library, scales: generator.scales.library.Library | None = None) -> int:
   count = 0
   for path in generator.parts.source_paths(source_dir):
     try:
@@ -122,10 +132,15 @@ def generate_performance_scores(source_dir: pathlib.Path, output_dir: pathlib.Pa
       plans = plan_arrangement(document, rhythms)
     except ValueError as error:
       raise ValueError(f'{path}: {error}') from error
+    reports = []
     for part in generator.parts.PART_TYPES:
       try:
-        generator.score.write_json(output_dir / f'{generator.score.slugify(path.stem)}-{part}.json', _arrange_document(document, part, plans))
+        generator.score.write_json(output_dir / f'{generator.score.slugify(path.stem)}-{part}.json', _arrange_document(document, part, plans, scales=scales, reports=reports, source_name=path.name))
       except ValueError as error:
         raise ValueError(f'{path}, part {part}: {error}') from error
       count += 1
+    if reports:
+      report_path = output_dir.parent / 'analysis' / f'{generator.score.slugify(path.stem)}.txt'
+      report_path.parent.mkdir(parents=True, exist_ok=True)
+      report_path.write_text(document['metadata']['title'] + '\nSource: ' + path.name + '\n\n' + '\n'.join(reports), encoding='utf-8')
   return count

generator/engraving/audit.py

@@ -78,15 +78,19 @@ def audit_layout(document: dict, layout: generator.engraving.types.LayoutDocumen
         fragments.setdefault(fragment.source, []).append(fragment)
       # Staff, ledger, stem, beam and tie contacts are intentional. Distinct
       # noteheads, accidentals and text in the same lane must remain separate.
-      checked = [p for p in system.primitives if p.role in ('head', 'tab', 'accidental', 'harmony', 'lyric')]
+      checked = [p for p in system.primitives if p.role in ('head', 'tab', 'accidental', 'harmony', 'lyric', 'scale', 'section')]
       for i, item in enumerate(checked):
         for other in checked[i + 1:]:
           if set(item.sources) & set(other.sources) and (item.role != 'tab' or other.role != 'tab'):
             continue
-          if item.role != other.role and {item.role, other.role} - {'head', 'tab', 'accidental'}:
+          if item.role != other.role and 'scale' not in (item.role, other.role) and {item.role, other.role} - {'head', 'tab', 'accidental'}:
             continue
           if item.box.overlaps(other.box, 0.1 * system.scale):
             raise generator.engraving.types.EngravingError(f'{item.sources} and {other.sources}: colliding {item.role}/{other.role}')
+      expected_scales = [span for span in index.scale_spans if generator.model.fraction(span['start']) < system.end and generator.model.fraction(span['end']) > system.start]
+      drawn_scales = [(p.sources[0], p.value) for p in system.primitives if p.role == 'scale']
+      if drawn_scales != [(span['id'], span['label']) for span in expected_scales]:
+        raise generator.engraving.types.EngravingError('solo scale labels are missing, duplicated, or changed')
   if previous_end != index.end:
     raise generator.engraving.types.EngravingError('layout omits the end of the score')
   for event in index.events:

generator/engraving/layout.py

@@ -11,6 +11,7 @@ import generator.engraving.audit
 import generator.engraving.fonts
 import generator.engraving.notation
 import generator.engraving.notes
+import generator.engraving.scales
 import generator.engraving.spans
 import generator.engraving.tablature
 import generator.engraving.types
@@ -54,6 +55,7 @@ def column(index: generator.engraving.notation.ScoreIndex, time: fractions.Fract
   for owner, _, item in index.harmonies_by_onset.get(time, []):
     if owner is measure:
       items.append(fonts.text(generator.harmony.symbol(item), 0, 0, 18 if index.staff.get('tablature') else 21, 'harmony', (item['id'],)))
+  items.extend(generator.engraving.scales.column(index, time, start, fonts))
   if time == measure_start:
     items.append(fonts.text(str(measure['number']), 0, -style.space * (index.state(time)['staff_details']['lines'] + 0.5), 12, 'measure-number', (measure['id'],)))
     if label := measure.get('section_label'):

generator/engraving/notation.py

@@ -57,6 +57,9 @@ class ScoreIndex:
           bucket.setdefault(onset, []).append((measure, index, item))
       time = end
     self.end = time
+    self.scale_spans = self.staff.get('scale_spans', [])
+    self.scale_starts = [generator.model.fraction(span['start']) for span in self.scale_spans]
+    self.columns.update(self.scale_starts)
     self.measure_starts = [item[0] for item in self.measures]
     self.measure_starts_by_id = {measure['id']: start for start, _, measure in self.measures}
     self.events.sort(key=lambda e: (e.start, not bool(e.value.get('grace')), e.lane))

generator/engraving/scales.py

@@ -0,0 +1,14 @@
+from __future__ import annotations
+
+import bisect
+
+import generator.model
+
+def column(index, time, system_start, fonts):
+  position = bisect.bisect_right(index.scale_starts, time) - 1
+  if position < 0:
+    return []
+  span = index.scale_spans[position]
+  if time >= generator.model.fraction(span['end']) or time not in (system_start, index.scale_starts[position]):
+    return []
+  return [fonts.text(span['label'], 0, 0, 16, 'scale', (span['id'],))]

generator/engraving/spans.py

@@ -123,7 +123,7 @@ def draw_spans(index: generator.engraving.notation.ScoreIndex, anchors: list[gen
       y = anchor.y - anchor.direction * 8
       if anchor.x - 2 > left:
         result.append(generator.engraving.types.curve('tie', (identifier,), ((left, y), (left + 5, y - anchor.direction * 9), (anchor.x - 7, y - anchor.direction * 9), (anchor.x - 2, y)), 1.6))
-  top = min(-style.space * 5, min((p.box.top for p in items if p.role not in ('harmony', 'lyric', 'section', 'tempo')), default=0))
+  top = min(-style.space * 5, min((p.box.top for p in items if p.role not in ('harmony', 'lyric', 'section', 'scale', 'tempo')), default=0))
   tuplet_lanes = []
   for span in index.staff.get('tuplets', []):
     a, b = index.by_id[span['start']], index.by_id[span['end']]
@@ -203,11 +203,11 @@ def draw_spans(index: generator.engraving.notation.ScoreIndex, anchors: list[gen
   return items + result
 
 def place_text_lanes(items: list[generator.engraving.types.Primitive], fonts: generator.engraving.fonts.Fonts, style: generator.engraving.types.Style) -> list[generator.engraving.types.Primitive]:
-  music = [p for p in items if p.role not in ('harmony', 'lyric', 'section')]
+  music = [p for p in items if p.role not in ('harmony', 'lyric', 'section', 'scale')]
   box = generator.engraving.types.bounds(music)
   result = list(music)
   top = box.top - 12
-  for role in ('harmony', 'section'):
+  for role in ('harmony', 'scale', 'section'):
     lane = [p for p in items if p.role == role]
     if lane:
       extent = generator.engraving.types.bounds(lane)

generator/model.py

@@ -285,3 +285,12 @@ def validate_document(document: dict) -> None:
   for span in staves[0].get('octave_shifts', []):
     if not 0 <= fraction(span['start']) < fraction(span['end']) <= absolute or not -3 <= span['octaves'] <= 3:
       raise ValueError('invalid octave-shift span')
+  previous_end = fractions.Fraction()
+  for span in staves[0].get('scale_spans', []):
+    start, end = fraction(span['start']), fraction(span['end'])
+    if not previous_end <= start < end <= absolute or not isinstance(span.get('label'), str) or not span['label'].strip() or type(span.get('matched')) is not bool:
+      raise ValueError('invalid solo scale span')
+    if not isinstance(span.get('id'), str) or not span['id'] or span['id'] in ids:
+      raise ValueError('solo scale spans require unique IDs')
+    ids.add(span['id'])
+    previous_end = end

generator/scales/library.py

@@ -0,0 +1,92 @@
+from __future__ import annotations
+
+import dataclasses
+import pathlib
+
+import generator.model
+import generator.musicxml
+import generator.pitch
+
+NAMES = {
+  'major': ('Ionian', 'Dorian', 'Phrygian', 'Lydian', 'Mixolydian', 'Aeolian', 'Locrian'),
+  'melodic-minor': ('Melodic Minor', 'Dorian b2', 'Lydian Augmented', 'Lydian Dominant', 'Mixolydian b6', 'Locrian natural 2', 'Altered'),
+  'harmonic-minor': ('Harmonic Minor', 'Locrian natural 6', 'Ionian #5', 'Dorian #4', 'Phrygian Dominant', 'Lydian #2', 'Ultra Locrian'),
+}
+ROOTS = ('C', 'Db', 'D', 'Eb', 'E', 'F', 'F#', 'G', 'Ab', 'A', 'Bb', 'B')
+
+@dataclasses.dataclass(frozen=True)
+class Pattern:
+  family: str
+  mode: int
+  intervals: tuple[int, ...]
+  def name(self) -> str:
+    return NAMES[self.family][self.mode - 1]
+
+@dataclasses.dataclass(frozen=True)
+class Scale:
+  pattern: Pattern
+  root: str
+  notes: tuple[str, ...]
+  pitches: frozenset[int]
+  def label(self) -> str:
+    return self.root + ' ' + self.pattern.name()
+
+@dataclasses.dataclass(frozen=True)
+class Library:
+  patterns: tuple[Pattern, ...]
+  def scale(self, family: str, mode: int, root: str) -> Scale:
+    pattern = next(p for p in self.patterns if (p.family, p.mode) == (family, mode))
+    tonic = generator.model.parse_pitch_name(root)
+    number = generator.model.NATURAL[tonic['letter']] + generator.model.alteration(tonic['alter'])
+    notes = tuple(generator.model.pitch_name(generator.pitch.shift_pitch({**tonic, 'octave': 4}, degree, interval)) for degree, interval in enumerate(pattern.intervals))
+    return Scale(pattern, root, notes, frozenset(int(number + n) % 12 for n in pattern.intervals))
+  def collections(self) -> tuple[Scale, ...]:
+    # Each mode is a rotation of its family, so only the three parents need searching.
+    return tuple(self.scale(family, 1, root) for family in NAMES for root in ROOTS)
+  def for_key(self, key: dict) -> Scale:
+    if not key.get('tonic') or key.get('mode') not in generator.model.MODE_STEPS:
+      raise ValueError('solo scale analysis requires a declared tonic and mode in the MusicXML key')
+    return self.scale('major', generator.model.MODE_STEPS[key['mode']] + 1, generator.model.pitch_name(key['tonic']))
+  def chord_mode(self, scale: Scale, root: str) -> tuple[int, str]:
+    pitch = generator.model.parse_pitch_name(root)
+    number = (generator.model.NATURAL[pitch['letter']] + generator.model.alteration(pitch['alter'])) % 12
+    values = [generator.model.parse_pitch_name(note) for note in scale.notes]
+    degree = next(i for i, note in enumerate(values) if (generator.model.NATURAL[note['letter']] + generator.model.alteration(note['alter'])) % 12 == number)
+    mode = (scale.pattern.mode - 1 + degree) % 7 + 1
+    return degree + 1, root + ' ' + NAMES[scale.pattern.family][mode - 1]
+
+def read_pattern(path: pathlib.Path, family: str, mode: int) -> Pattern:
+  document = generator.musicxml.load_musicxml(path)
+  if len(document['staves']) != 1:
+    raise ValueError(f'{path}: scale patterns require one staff')
+  notes = []
+  for measure in document['staves'][0]['measures']:
+    if len(measure['voices']) != 1:
+      raise ValueError(f'{path}: scale patterns require one melodic voice')
+    for event in measure['voices'][0]['notes']:
+      if event['kind'] != 'note' or len(event['pitches']) != 1 or event.get('grace'):
+        raise ValueError(f'{path}: scale patterns require eight ascending notes, including the octave')
+      notes.append(event['pitches'][0])
+  if len(notes) != 8 or ''.join(note.get('letter', '?') for note in notes) != 'cdefgabc':
+    raise ValueError(f'{path}: scale patterns must spell C through B followed by C')
+  numbers = [generator.model.pitch_number(note) for note in notes]
+  intervals = tuple(number - numbers[0] for number in numbers)
+  if generator.model.alteration(notes[0]['alter']) or intervals[-1] != 12 or any(n.denominator != 1 for n in intervals) or any(a >= b for a, b in zip(intervals, intervals[1:])):
+    raise ValueError(f'{path}: expected seven distinct ascending semitones rooted on C and octave closure')
+  return Pattern(family, mode, tuple(int(n) for n in intervals[:-1]))
+
+def load_library(root: pathlib.Path) -> Library:
+  expected = {root / f'{family}-scale-modes' / f'{mode:04}.xml' for family in NAMES for mode in range(1, 8)}
+  actual = set(root.rglob('*.xml'))
+  if actual != expected:
+    missing, unexpected = sorted(expected - actual), sorted(actual - expected)
+    raise ValueError(f'{root}: expected all 21 scale patterns; missing: {missing}; unexpected: {unexpected}')
+  patterns = tuple(read_pattern(root / f'{family}-scale-modes' / f'{mode:04}.xml', family, mode) for family in NAMES for mode in range(1, 8))
+  for family in NAMES:
+    modes = [pattern for pattern in patterns if pattern.family == family]
+    parent = modes[0].intervals
+    for index, pattern in enumerate(modes):
+      rotation = tuple((parent[(index + n) % 7] - parent[index]) % 12 for n in range(7))
+      if pattern.intervals != rotation:
+        raise ValueError(f'{root / (family + "-scale-modes") / f"{pattern.mode:04}.xml"}: pattern is not mode {pattern.mode} of its family parent')
+  return Library(patterns)

generator/scales/mapping.py

@@ -0,0 +1,32 @@
+from __future__ import annotations
+
+import generator.harmony
+import generator.model
+import generator.scales.library
+import generator.scales.report
+import generator.scales.selection
+import generator.scales.timeline
+
+def annotate(document: dict, solos: list[generator.scales.timeline.Solo], library: generator.scales.library.Library, source: str) -> str:
+  staff = document['staves'][0]
+  timeline = generator.scales.timeline.read(staff)
+  spans, reports = [], []
+  for solo in solos:
+    choices = generator.scales.selection.select(timeline, solo, library)
+    previous = None
+    for choice in choices:
+      if choice.span.chord is None:
+        previous = None
+        continue
+      label = choice.scale.label() if choice.scale else 'No exact scale match'
+      if previous is not None and previous['label'] == label and generator.model.fraction(previous['end']) == choice.span.start:
+        previous['end'] = generator.model.rational(choice.span.end)
+      else:
+        previous = {'id': f'scale-{len(spans) + 1}', 'start': generator.model.rational(choice.span.start), 'end': generator.model.rational(choice.span.end), 'label': label, 'matched': choice.scale is not None}
+        spans.append(previous)
+      if choice.scale is None:
+        print(f'{source}, {staff["name"]}, source measure {choice.span.source_measure}, beat {choice.span.beat}: {choice.reason} Chord: {generator.harmony.symbol(choice.span.chord)}', flush=True)
+    reports.append(generator.scales.report.passage(staff['arrangement_part_type'], solo, choices, library))
+  if spans:
+    staff['scale_spans'] = spans
+  return '\n'.join(reports)

generator/scales/report.py

@@ -0,0 +1,44 @@
+from __future__ import annotations
+
+import generator.harmony
+import generator.model
+import generator.pitch
+
+def chord_notes(chord: dict) -> tuple[tuple[int, str], ...]:
+  return tuple((degree, generator.model.pitch_name(generator.pitch.shift_pitch({**chord['root'], 'octave': 4}, degree - 1, generator.harmony.degree_semitones(degree, alter)))) for degree, alter in generator.harmony.degrees(chord).items())
+
+def passage(part: str, solo, choices, library) -> str:
+  lines = [f'{part.capitalize()}: {solo.label}', 'Positions use source measure numbers and quarter-note beats; arranged measure numbers are also shown.', '']
+  lines.append('Scale map:')
+  previous = None
+  for choice in choices:
+    label = choice.scale.label() if choice.scale else 'No exact match within the permitted catalog.' if choice.span.chord else 'N.C.'
+    if label != previous:
+      lines.append(f'  Source measure {choice.span.source_measure}, beat {choice.span.beat}: {label}')
+      lines.append('    ' + choice.reason)
+    previous = label
+  lines.extend(('', 'Chord detail:'))
+  for choice in choices:
+    span, scale = choice.span, choice.scale
+    if span.chord is None:
+      continue
+    chord = span.chord
+    notes = chord_notes(chord)
+    required = [name for _, name in notes]
+    if bass := chord.get('bass'):
+      required.append(generator.model.pitch_name(bass))
+    targets = [name for degree, name in notes if degree in (3, 7)] or [name for _, name in notes]
+    lines.append(f'Source measure {span.source_measure}, beat {span.beat}; arranged measure {span.measure}; offset {span.start} through {span.end}')
+    lines.append('  Chord: ' + generator.harmony.symbol(chord) + '; required pitches: ' + ', '.join(dict.fromkeys(required)))
+    if scale:
+      root = generator.model.pitch_number({**chord['root'], 'octave': 4}) % 12
+      degree, mode = library.chord_mode(scale, generator.model.pitch_name(chord['root'])) if root in scale.pitches else (None, None)
+      lines.append('  Parent: ' + scale.label() + ' [' + ', '.join(scale.notes) + ']')
+      lines.append(f'  Parent degree: {degree or "omitted root"}; chord-root mode: {mode or "not assigned"}')
+      lines.append('  Chord-tone targets: ' + ', '.join(targets))
+      if generator.model.pitch_name(chord['root']) == scale.root:
+        lines.append('  Tonic resolution target: ' + scale.root)
+    else:
+      lines.append('  No exact match within the permitted catalog.')
+    lines.extend(('  Reason: ' + choice.reason, ''))
+  return '\n'.join(lines)

generator/scales/selection.py

@@ -0,0 +1,109 @@
+from __future__ import annotations
+
+import dataclasses
+import fractions
+
+import generator.harmony
+import generator.model
+import generator.pitch
+import generator.scales.library
+import generator.scales.timeline
+
+@dataclasses.dataclass(frozen=True)
+class Choice:
+  span: generator.scales.timeline.ChordSpan
+  scale: generator.scales.library.Scale | None
+  reason: str
+
+def root_number(chord: dict) -> fractions.Fraction:
+  root = chord['root']
+  return (generator.model.NATURAL[root['letter']] + generator.model.alteration(root['alter'])) % 12
+
+def dominant(chord: dict) -> bool:
+  degrees = generator.harmony.degrees(chord)
+  return degrees.get(3) == 0 and degrees.get(7) == -1
+
+def destination(chord: dict, following: dict | None, library: generator.scales.library.Library):
+  if following and dominant(chord) and (root_number(chord) - root_number(following)) % 12 == 7:
+    third = generator.harmony.degrees(following).get(3)
+    if third in (-1, 0):
+      return library.scale('harmonic-minor' if third == -1 else 'major', 1, generator.model.pitch_name(following['root']))
+  return None
+
+def preference(chord: dict, following: dict | None, after: dict | None, home: generator.scales.library.Scale, library: generator.scales.library.Library) -> tuple[generator.scales.library.Scale, str]:
+  if any(root_number(item).denominator != 1 for item in (chord, following, after) if item):
+    return home, 'non-semitone harmony cannot establish a destination in the permitted catalog'
+  if target := destination(chord, following, library):
+    return target, 'dominant approaching ' + generator.model.pitch_name(following['root']) + (' minor' if target.pattern.family == 'harmonic-minor' else ' major')
+  degrees = generator.harmony.degrees(chord)
+  if following and degrees.get(3) == -1 and degrees.get(7) == -1 and dominant(following) and (root_number(following) - root_number(chord)) % 12 == 5:
+    tonic = generator.pitch.shift_pitch({**chord['root'], 'octave': 4}, -1, -2)
+    target = destination(following, after, library) or library.scale('harmonic-minor' if degrees.get(5) == -1 else 'major', 1, generator.model.pitch_name(tonic))
+    if set(generator.harmony.pitch_classes(chord)) | set(generator.harmony.pitch_classes(following)) <= target.pitches:
+      return target, 'shared ' + ('minor ii-V' if target.pattern.family == 'harmonic-minor' else 'major ii-V') + ' destination ' + target.root
+  if following and degrees.get(3) == -1 and degrees.get(5) == -1 and degrees.get(7) == -2 and (root_number(following) - root_number(chord)) % 12 == 1:
+    return library.scale('harmonic-minor', 1, generator.model.pitch_name(following['root'])), 'leading-tone diminished resolution to ' + generator.model.pitch_name(following['root'])
+  return home, 'declared tonal center ' + home.label()
+
+def following_indices(timeline: tuple[generator.scales.timeline.ChordSpan, ...]) -> list[int | None]:
+  result = [None] * len(timeline)
+  for index in range(len(timeline) - 2, -1, -1):
+    here, there = timeline[index], timeline[index + 1]
+    if not there.chord or here.key != there.key:
+      continue
+    fields = ('root', 'kind', 'degrees', 'bass')
+    same = here.chord and all(here.chord.get(k) == there.chord.get(k) for k in fields)
+    result[index] = result[index + 1] if same else index + 1
+  return result
+
+def select(timeline: tuple[generator.scales.timeline.ChordSpan, ...], solo: generator.scales.timeline.Solo, library: generator.scales.library.Library) -> tuple[Choice, ...]:
+  collections = library.collections()
+  following = following_indices(timeline)
+  rows, layers = [], []
+  for index, original in enumerate(timeline):
+    if original.end <= solo.start or original.start >= solo.end:
+      continue
+    span = dataclasses.replace(original, start=max(original.start, solo.start), end=min(original.end, solo.end), beat=original.beat + max(0, solo.start - original.start))
+    try:
+      home = library.for_key(span.key)
+    except ValueError as error:
+      raise ValueError(f'measure {span.source_measure}, beat {span.beat}: {error}') from error
+    next_index = following[index]
+    after_index = following[next_index] if next_index is not None else None
+    next_chord = timeline[next_index].chord if next_index is not None else None
+    after = timeline[after_index].chord if after_index is not None else None
+    target, reason = preference(span.chord, next_chord, after, home, library) if span.chord else (home, 'no harmony')
+    # Name the home collection from the declared tonic, not its relative major.
+    if target.pitches == home.pitches:
+      target = home
+    required = set(generator.harmony.pitch_classes(span.chord)) if span.chord else set()
+    options = [target if scale.pitches == target.pitches else home if scale.pitches == home.pitches else scale for scale in collections if required <= scale.pitches] if span.chord else []
+    options = options or [None]
+    layer = []
+    for scale in options:
+      local = (int(scale is not None and scale.pitches != target.pitches), 0, len(scale.pitches ^ target.pitches) if scale else 0)
+      predecessors = []
+      for previous_index, (previous_scale, cost, _) in enumerate(layers[-1] if layers else [(None, (0, 0, 0), None)]):
+        changed = int(scale is not None and previous_scale is not None and scale.pitches != previous_scale.pitches)
+        predecessors.append((tuple(a + b for a, b in zip(cost, (local[0], changed, local[2]))), previous_index))
+      cost, predecessor = min(predecessors)
+      layer.append((scale, cost, predecessor))
+    rows.append((span, target, reason))
+    layers.append(layer)
+  if not layers:
+    return ()
+  position = min(range(len(layers[-1])), key=lambda i: layers[-1][i][1])
+  choices = []
+  for (span, target, reason), layer in reversed(list(zip(rows, layers))):
+    scale, _, position = layer[position]
+    if span.chord and scale is None:
+      reason = 'No exact match within the permitted catalog.'
+    elif scale and scale.pitches != target.pitches:
+      reason = 'compatible alternative to ' + target.label() + '; selected for collection continuity and closest contextual pitches (' + reason + ')'
+    choices.append(Choice(span, scale, reason))
+  result = []
+  for choice in reversed(choices):
+    if result and choice.scale and result[-1].scale and choice.scale.pitches == result[-1].scale.pitches:
+      choice = dataclasses.replace(choice, scale=result[-1].scale)
+    result.append(choice)
+  return tuple(result)

generator/scales/timeline.py

@@ -0,0 +1,39 @@
+from __future__ import annotations
+
+import dataclasses
+import fractions
+
+import generator.model
+
+@dataclasses.dataclass(frozen=True)
+class ChordSpan:
+  start: fractions.Fraction
+  end: fractions.Fraction
+  measure: int
+  source_measure: str
+  beat: fractions.Fraction
+  key: dict
+  chord: dict | None
+
+@dataclasses.dataclass(frozen=True)
+class Solo:
+  start: fractions.Fraction
+  end: fractions.Fraction
+  label: str
+
+def read(staff: dict) -> tuple[ChordSpan, ...]:
+  result = []
+  absolute = fractions.Fraction()
+  active = None
+  for number, measure in enumerate(staff['measures'], 1):
+    length = generator.model.fraction(measure['duration'])
+    harmonies = {generator.model.fraction(h['onset']): h for h in measure['harmonies']}
+    changes = {generator.model.fraction(a['onset']): a['key'] for a in measure.get('attributes', []) if 'key' in a}
+    boundaries = sorted({fractions.Fraction(), length} | set(harmonies) | set(changes))
+    key = measure['key']
+    for start, end in zip(boundaries, boundaries[1:]):
+      key = changes.get(start, key)
+      active = harmonies.get(start, active)
+      result.append(ChordSpan(absolute + start, absolute + end, number, str(measure['number']), start + 1, key, active if active and active['kind'] != 'none' else None))
+    absolute += length
+  return tuple(result)

tests/scale_fixtures.py

@@ -0,0 +1,8 @@
+import functools
+import pathlib
+
+import generator.scales.library
+
+@functools.cache
+def library() -> generator.scales.library.Library:
+  return generator.scales.library.load_library(pathlib.Path(__file__).resolve().parents[1] / 'data/scales')

tests/test_engraving_cache.py

@@ -24,12 +24,13 @@ import generator.engraving.validation
 import generator.pico_artwork
 import generator.score
 import tests.rhythm_fixtures
+import tests.scale_fixtures
 import tests.test_musicxml
 
 class EngravingCacheTests(unittest.TestCase):
   def test_ci_snapshot_tracks_all_generator_inputs_and_keeps_fallback(self):
     workflow = pathlib.Path('.gitea/workflows/ci.yml').read_text()
-    self.assertIn("hashFiles('generator/**/*.py', 'generator/*.py', 'data/songs/*.xml', 'data/rhythms/**/*.xml')", workflow)
+    self.assertIn("hashFiles('generator/**/*.py', 'generator/*.py', 'data/songs/*.xml', 'data/rhythms/**/*.xml', 'data/scales/**/*.xml')", workflow)
     self.assertIn('python3 -m generator --renderer-fingerprint', workflow)
     self.assertIn('restore-keys: product-vamp-pages-${{ steps.renderer.outputs.fingerprint }}-', workflow)
     self.assertIn('path: build/build_common/engraving-cache/${{ steps.renderer.outputs.fingerprint }}', workflow)
@@ -344,6 +345,7 @@ class EngravingCacheTests(unittest.TestCase):
       with self.assertRaisesRegex(ValueError, 'must not overlap'):
         generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=cache)
   def test_parallel_generation_preserves_assets_on_worker_failure(self):
+    self.enterContext(unittest.mock.patch('generator.scales.library.load_library', return_value=tests.scale_fixtures.library()))
     self.enterContext(unittest.mock.patch('generator.rhythm.load_library', return_value=tests.rhythm_fixtures.library()))
     data, output = self.root / 'data', self.root / 'output'
     (data / 'songs').mkdir(parents=True)

tests/test_guitar.py

@@ -17,6 +17,7 @@ import generator.pitch
 import generator.playback
 import tests.arrangement_fixtures
 import tests.rhythm_fixtures
+import tests.scale_fixtures
 import tests.test_musicxml
 
 class GuitarTests(unittest.TestCase):
@@ -233,6 +234,7 @@ class GuitarTests(unittest.TestCase):
       generator.arrange.generate.generate_performance_scores(source, output, rhythms=tests.rhythm_fixtures.library())
       self.assertEqual(first, {p.name: p.read_bytes() for p in output.rglob('*.json')})
   def test_unplayable_generation_preserves_previous_assets(self):
+    self.enterContext(unittest.mock.patch('generator.scales.library.load_library', return_value=tests.scale_fixtures.library()))
     with tempfile.TemporaryDirectory() as temporary:
       root = pathlib.Path(temporary)
       (root / 'data/songs').mkdir(parents=True)

tests/test_harmony.py

@@ -91,7 +91,7 @@ class HarmonyTests(unittest.TestCase):
     source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(chord_xml('7', 'dominant', extra='<degree><degree-value>9</degree-value><degree-alter>-1</degree-alter><degree-type>add</degree-type></degree>') + tests.test_musicxml.xml_note()))
     self.assertEqual(source['staves'][0]['measures'][0]['harmonies'][0]['symbol'], 'C7b9')
   def test_current_song_exports_import_without_edits(self):
-    for name, count, key in (('autumn-leaves', 33, 'F'), ('it-could-happen-to-you', 32, 'Eb')):
+    for name, count, key in (('autumn-leaves', 33, 'D minor'), ('it-could-happen-to-you', 32, 'Eb')):
       source = generator.musicxml.load_musicxml(pathlib.Path('data/songs') / f'{name}.xml')
       self.assertEqual(len(source['staves'][0]['measures']), count)
       self.assertEqual(source['metadata']['key_signature'], key)

tests/test_music_model.py

@@ -15,10 +15,12 @@ import generator.musicxml
 import generator.parts
 import generator.pitch
 import tests.rhythm_fixtures
+import tests.scale_fixtures
 import tests.test_musicxml
 
 class MusicModelTests(unittest.TestCase):
   def setUp(self):
+    self.enterContext(unittest.mock.patch('generator.scales.library.load_library', return_value=tests.scale_fixtures.library()))
     engraving = unittest.mock.patch('generator.engraving.pipeline.engrave_tree')
     engraving.start()
     self.addCleanup(engraving.stop)

tests/test_parts.py

@@ -18,6 +18,7 @@ import generator.pitch
 import generator.playback
 import tests.arrangement_fixtures
 import tests.rhythm_fixtures
+import tests.scale_fixtures
 import tests.test_musicxml
 
 def xml_harmony(root='C', kind='major', offset=0):
@@ -170,6 +171,7 @@ class PartGenerationTests(unittest.TestCase):
           self.assertIn(str(source / (name + '.xml')), str(raised.exception))
         self.assertEqual([p.name for p in output.iterdir()], ['sentinel'])
   def test_pipeline_parses_and_normalizes_each_source_once(self):
+    self.enterContext(unittest.mock.patch('generator.scales.library.load_library', return_value=tests.scale_fixtures.library()))
     with tempfile.TemporaryDirectory() as directory:
       root = pathlib.Path(directory)
       for category, name in (('songs', 'song'), ('songs', 'other')):

tests/test_scales.py

@@ -0,0 +1,192 @@
+import contextlib
+import copy
+import dataclasses
+import fractions
+import io
+import pathlib
+import shutil
+import tempfile
+import unittest
+
+import generator.arrange.generate
+import generator.engraving.cache
+import generator.engraving.fonts
+import generator.engraving.layout
+import generator.engraving.types
+import generator.harmony
+import generator.model
+import generator.musicxml
+import generator.parts
+import generator.playback
+import generator.scales.mapping
+import generator.scales.library
+import generator.scales.selection
+import generator.scales.timeline
+import tests.rhythm_fixtures
+import tests.scale_fixtures
+import tests.test_accompaniment
+import tests.test_musicxml
+
+def timeline(symbols, key=None, duration=4):
+  key = key or generator.model.key_from_fifths(-1, 'minor')
+  result = []
+  for index, symbol in enumerate(symbols):
+    result.append(generator.scales.timeline.ChordSpan(fractions.Fraction(index * duration), fractions.Fraction((index + 1) * duration), index + 1, str(index + 1), fractions.Fraction(1), key, generator.harmony.parse_symbol(symbol) if symbol else None))
+  return tuple(result)
+
+def choose(symbols, key=None):
+  spans = timeline(symbols, key)
+  return generator.scales.selection.select(spans, generator.scales.timeline.Solo(spans[0].start, spans[-1].end, 'Solo'), tests.scale_fixtures.library())
+
+class ScaleLibraryTests(unittest.TestCase):
+  def test_all_patterns_and_transpositions_preserve_spelling_and_rotations(self):
+    library = tests.scale_fixtures.library()
+    self.assertEqual(len(library.patterns), 21)
+    self.assertEqual(len({scale.pitches for scale in library.collections()}), 36)
+    for pattern in library.patterns:
+      for root in generator.scales.library.ROOTS:
+        scale = library.scale(pattern.family, pattern.mode, root)
+        self.assertEqual(len(scale.pitches), 7)
+        self.assertEqual(scale.notes[0], root)
+        self.assertEqual(library.chord_mode(scale, root), (1, scale.label()))
+    self.assertEqual(library.scale('harmonic-minor', 3, 'C').notes, ('C', 'D', 'E', 'F', 'G#', 'A', 'B'))
+    self.assertEqual(library.scale('harmonic-minor', 7, 'C').notes, ('C', 'Db', 'Eb', 'Fb', 'Gb', 'Ab', 'Bbb'))
+    self.assertEqual(library.scale('major', 1, 'C#').pitches, library.scale('major', 1, 'Db').pitches)
+    self.assertNotEqual(library.scale('major', 1, 'C#').notes, library.scale('major', 1, 'Db').notes)
+  def test_incomplete_unexpected_and_inconsistent_xml_patterns_are_rejected(self):
+    with tempfile.TemporaryDirectory() as directory:
+      root = pathlib.Path(directory) / 'scales'
+      shutil.copytree('data/scales', root)
+      path = root / 'harmonic-minor-scale-modes/0003.xml'
+      original = path.read_text()
+      path.unlink()
+      with self.assertRaisesRegex(ValueError, '21 scale patterns'):
+        generator.scales.library.load_library(root)
+      path.write_text(original.replace('<step>A</step>', '<step>G</step>'))
+      with self.assertRaisesRegex(ValueError, '0003.xml.*C through B'):
+        generator.scales.library.load_library(root)
+      path.write_text(original.replace('<alter>1</alter>', '<alter>0</alter>').replace('<accidental>sharp</accidental>', '<accidental>natural</accidental>'))
+      with self.assertRaisesRegex(ValueError, '0003.xml.*not mode 3'):
+        generator.scales.library.load_library(root)
+      path.write_text(original)
+      extra = root / 'unknown.xml'
+      extra.write_text(original)
+      with self.assertRaisesRegex(ValueError, 'unexpected'):
+        generator.scales.library.load_library(root)
+
+class ScaleSelectionTests(unittest.TestCase):
+  def test_autumn_leaves_guide_collections_and_modes(self):
+    first = ['Gm7', 'C7', 'FM7', 'BbM7', 'Em7b5', 'A7', 'Dm7', 'Dm7']
+    bridge = ['Em7b5', 'A7', 'Dm7', 'Dm7', 'Gm7', 'C7', 'FM7', 'FM7']
+    labels = [c.scale.label() for c in choose(first + first + bridge)]
+    phrase = ['D Aeolian'] * 4 + ['D Harmonic Minor'] * 2 + ['D Aeolian'] * 2
+    self.assertEqual(labels, phrase * 2 + ['D Harmonic Minor'] * 2 + ['D Aeolian'] * 6)
+    ending = choose(['Em7b5', 'A7', 'Dm7', 'G7', 'Cm7', 'F7', 'Em7b5', 'A7', 'Dm7'])
+    self.assertEqual([c.scale.label() for c in ending], ['D Harmonic Minor'] * 2 + ['D Aeolian', 'C Harmonic Minor', 'Bb Ionian', 'Bb Ionian'] + ['D Harmonic Minor'] * 2 + ['D Aeolian'])
+    library = tests.scale_fixtures.library()
+    self.assertEqual(library.chord_mode(ending[0].scale, 'E'), (2, 'E Locrian natural 6'))
+    self.assertEqual(library.chord_mode(ending[1].scale, 'A'), (5, 'A Phrygian Dominant'))
+    self.assertEqual(library.chord_mode(ending[4].scale, 'C'), (2, 'C Dorian'))
+  def test_actual_song_uses_minor_tonic_triads_without_forcing_sevenths(self):
+    source = generator.musicxml.load_musicxml(pathlib.Path('data/songs/autumn-leaves.xml'))
+    spans = generator.scales.timeline.read(source['staves'][0])
+    choices = generator.scales.selection.select(spans, generator.scales.timeline.Solo(4, spans[-1].end, 'Solo'), tests.scale_fixtures.library())
+    self.assertTrue(all(c.scale for c in choices))
+    for choice in choices:
+      chord = choice.span.chord
+      if chord['symbol'] == 'Dm':
+        self.assertNotIn(7, generator.harmony.degrees(chord))
+        self.assertEqual(choice.scale.label(), 'D Aeolian')
+      self.assertLessEqual(set(generator.harmony.pitch_classes(chord)), choice.scale.pitches)
+    half_bar = [(c.span.source_measure, c.span.beat, c.scale.label()) for c in choices if c.span.source_measure in ('28', '29')]
+    self.assertEqual(half_bar, [('28', 1, 'D Aeolian'), ('28', 3, 'C Harmonic Minor'), ('29', 1, 'Bb Ionian'), ('29', 3, 'Bb Ionian')])
+  def test_extensions_slash_basses_omissions_and_diminished_chords(self):
+    for symbol in ('Em9b5', 'DmM7', 'Dm6', 'FM7#5', 'C#dim7', 'Dm7/F', 'C7no5', 'C7b5b9'):
+      with self.subTest(symbol=symbol):
+        choice = choose([symbol])[0]
+        self.assertIsNotNone(choice.scale)
+        self.assertLessEqual(set(generator.harmony.pitch_classes(choice.span.chord)), choice.scale.pitches)
+    choices = choose(['C#dim7', 'Dm'])
+    self.assertEqual([c.scale.label() for c in choices], ['D Harmonic Minor', 'D Aeolian'])
+    self.assertIsNone(choose(['A13b9'])[0].scale)
+    with self.assertRaises(ValueError):
+      generator.harmony.parse_symbol('A7alt')
+  def test_shared_collection_wins_over_independent_compatible_colors(self):
+    choices = choose(['E7', 'F#7'], generator.model.key_from_fifths(0))
+    self.assertEqual([choice.scale.label() for choice in choices], ['B Melodic Minor'] * 2)
+    self.assertEqual(choices, choose(['E7', 'F#7'], generator.model.key_from_fifths(0)))
+  def test_carried_harmony_and_fractional_key_boundaries_are_exact(self):
+    document = tests.test_musicxml.parse(tests.test_accompaniment.xml_bars([tests.test_musicxml.xml_note()] * 3))
+    measures = document['staves'][0]['measures']
+    measures[0]['harmonies'] = [{'id': 'h1', 'onset': generator.model.rational(0), **generator.harmony.parse_symbol('CM7')}]
+    measures[1]['attributes'] = [{'onset': generator.model.rational(fractions.Fraction(3, 2)), 'key': generator.model.key_from_fifths(1)}]
+    measures[2]['harmonies'] = [{'id': 'h2', 'onset': generator.model.rational(0), 'kind': 'none', 'degrees': []}]
+    spans = generator.scales.timeline.read(document['staves'][0])
+    self.assertEqual([(s.start, s.end) for s in spans], [(0, 4), (4, fractions.Fraction(11, 2)), (fractions.Fraction(11, 2), 8), (8, 12)])
+    self.assertEqual(spans[1].chord, spans[0].chord)
+    self.assertEqual(spans[2].key['tonic']['letter'], 'g')
+    self.assertIsNone(spans[3].chord)
+  def test_microtonal_harmonies_are_explicitly_unmatched(self):
+    spans = timeline(['A7', 'Dm'])
+    for span in spans:
+      span.chord['root']['alter'] = generator.model.rational(fractions.Fraction(1, 2))
+    result = generator.scales.selection.select(spans, generator.scales.timeline.Solo(0, 8, 'Solo'), tests.scale_fixtures.library())
+    self.assertTrue(all(c.scale is None and 'No exact match' in c.reason for c in result))
+  def test_local_key_and_no_chord_break_continuity_and_missing_tonic_fails(self):
+    spans = list(timeline(['Dm', None, 'Cm']))
+    spans[-1] = dataclasses.replace(spans[-1], key=generator.model.key_from_fifths(-3, 'minor'))
+    solo = generator.scales.timeline.Solo(0, 12, 'Solo')
+    result = generator.scales.selection.select(tuple(spans), solo, tests.scale_fixtures.library())
+    self.assertEqual([c.scale.label() if c.scale else None for c in result], ['D Aeolian', None, 'C Aeolian'])
+    spans[0] = dataclasses.replace(spans[0], key=generator.model.key_from_fifths(0, 'none'))
+    with self.assertRaisesRegex(ValueError, 'measure 1.*declared tonic'):
+      generator.scales.selection.select(tuple(spans), solo, tests.scale_fixtures.library())
+
+class ScaleOutputTests(unittest.TestCase):
+  def test_annotations_reports_and_warnings_preserve_music_and_playback(self):
+    source = tests.test_musicxml.parse(tests.test_accompaniment.xml_bars([tests.test_musicxml.xml_note()] * 4))
+    staff = source['staves'][0]
+    staff.update(name='Acoustic', arrangement_part_type='acoustic')
+    for measure, symbol in zip(staff['measures'], ('CM7', 'CM7', 'A13b9', 'CM7')):
+      measure['harmonies'] = [{'id': measure['id'] + '-h1', 'onset': generator.model.rational(0), **generator.harmony.parse_symbol(symbol)}]
+    before = copy.deepcopy(source)
+    warnings = io.StringIO()
+    with contextlib.redirect_stdout(warnings):
+      report = generator.scales.mapping.annotate(source, [generator.scales.timeline.Solo(0, 12, 'Guitar Solo')], tests.scale_fixtures.library(), 'fixture.xml')
+    self.assertIn('fixture.xml, Acoustic, source measure 3', warnings.getvalue())
+    self.assertIn('No exact match within the permitted catalog', report)
+    self.assertIn('Parent degree: 1; chord-root mode: C Ionian', report)
+    spans = staff.pop('scale_spans')
+    self.assertEqual(source, before)
+    self.assertEqual([(generator.model.fraction(s['start']), generator.model.fraction(s['end']), s['label']) for s in spans], [(0, 8, 'C Ionian'), (8, 12, 'No exact scale match')])
+    staff['scale_spans'] = spans
+    generator.model.validate_document(source)
+    self.assertNotEqual(generator.engraving.cache.score_fingerprint(source, 'renderer'), generator.engraving.cache.score_fingerprint(before, 'renderer'))
+    self.assertEqual(generator.playback.compile_timeline(source), generator.playback.compile_timeline(before))
+  def test_layout_keeps_exact_offsets_continuations_and_uniform_size(self):
+    source = tests.test_musicxml.parse(tests.test_accompaniment.xml_bars([tests.test_musicxml.xml_note()] * 40))
+    staff = source['staves'][0]
+    staff['measures'][0]['section_label'] = 'Guitar Solo'
+    staff['scale_spans'] = [{'id': f'scale-{i}', 'start': generator.model.rational(a), 'end': generator.model.rational(b), 'label': label, 'matched': True} for i, (a, b, label) in enumerate(((0, fractions.Fraction(3, 2), 'C Ionian'), (fractions.Fraction(3, 2), 60, 'D Harmonic Minor')), 1)]
+    fonts = generator.engraving.fonts.Fonts(pathlib.Path('package/assets/fonts/petaluma'))
+    layout = generator.engraving.layout.layout_score(source, generator.engraving.types.Style(), fonts)
+    systems = [s for p in layout.pages for s in p.systems]
+    self.assertEqual(len({s.scale for s in systems}), 1)
+    self.assertTrue(all(len(p.systems) == 2 for p in layout.pages[:-1]))
+    self.assertGreater(len(systems), 2)
+    for system in systems:
+      expected = [s['label'] for s in staff['scale_spans'] if generator.model.fraction(s['start']) < system.end and generator.model.fraction(s['end']) > system.start]
+      self.assertEqual([p.value for p in system.primitives if p.role == 'scale'], expected)
+    self.assertIn(fractions.Fraction(3, 2), generator.engraving.notation.ScoreIndex(source).column_times)
+  def test_only_soloing_part_is_annotated_without_revoicing(self):
+    content = '<harmony><root><root-step>C</root-step></root><kind>major</kind></harmony>' + tests.test_musicxml.xml_note()
+    source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))
+    for part in ('acoustic', 'electric', 'vocal'):
+      before = generator.arrange.generate.arrange_document(source, part, rhythms=tests.rhythm_fixtures.library())
+      result = generator.arrange.generate.arrange_document(source, part, rhythms=tests.rhythm_fixtures.library(), scales=tests.scale_fixtures.library())
+      spans = result['staves'][0].pop('scale_spans', [])
+      self.assertEqual(result, before)
+      self.assertEqual(len(spans), int(part != 'vocal'))
+      if spans:
+        self.assertEqual(generator.model.fraction(spans[0]['start']), 12 if part == 'acoustic' else 16)
+        self.assertEqual(generator.model.fraction(spans[0]['end']), 16 if part == 'acoustic' else 20)

tests/test_score_bundle.py

@@ -4,6 +4,7 @@ import fractions
 import hashlib
 import json
 import pathlib
+import shutil
 import tempfile
 import unittest
 import unittest.mock
@@ -39,6 +40,7 @@ class ScoreBundleTests(unittest.TestCase):
       root = pathlib.Path(temporary)
       data, output = root / 'data', root / 'output'
       (data / 'songs').mkdir(parents=True)
+      shutil.copytree(pathlib.Path('data/scales'), data / 'scales')
       source = data / 'songs/fixture.xml'
       source.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
       args = argparse.Namespace(data_dir=data, output_dir=output, static_assets_dir=pathlib.Path('package/assets'), check=False)
@@ -46,11 +48,17 @@ class ScoreBundleTests(unittest.TestCase):
       original = {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()}
       self.assertTrue(list(output.rglob('*.png')))
       (data / 'rhythms' / 'drafts').mkdir(parents=True)
-      (data / 'scales' / 'major-scale-modes').mkdir(parents=True)
+      for path in ('rhythms/0001.xml', 'rhythms/drafts/0002.xml'):
-      for path in ('rhythms/0001.xml', 'rhythms/drafts/0002.xml', 'scales/0001.xml', 'scales/major-scale-modes/0002.xml'):
         (data / path).write_text('<unfinished')
       generator.__main__.generate_assets(args)
       self.assertEqual(original, {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()})
+      scale = data / 'scales/major-scale-modes/0002.xml'
+      previous = scale.read_bytes()
+      scale.write_text('<unfinished')
+      with self.assertRaisesRegex(ValueError, '0002.xml'):
+        generator.__main__.generate_assets(args)
+      self.assertEqual(original, {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()})
+      scale.write_bytes(previous)
       misplaced = data / '0001.xml'
       misplaced.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
       with self.assertRaisesRegex(ValueError, 'belong directly under'):