Compare commits

...

2 Commits

Author SHA1 Message Date
khodowany a0e13434e4 Rhythm Patterns
ci / checks (push) Successful in 51m14s
2026-10-07 00:22:19 -05:00
khodowany 23b606e47f Rhythm Patterns 2026-10-06 23:55:58 -05:00
12 changed files with 6112 additions and 388 deletions
+6 -2
View File
@@ -102,11 +102,15 @@ Six roles generate independently: acoustic accompaniment/solo; electric accompan
Electric accompaniment and electric melody share compact chord families on strings 1–4. Melody harmonization adapts these to keep the original melody on the lowest-numbered feasible string, preferring it above the added chord tones. The separate Electric Solo stays monophonic. Acoustic accompaniment uses one bass note on string 5 or 6 and three upper notes on strings 2, 3 and 4. Transposable degree families cover triads, sevenths/sixths, suspensions, diminished/augmented and extended/altered chords, with root/fifth bass alternatives on both bass strings. Plain major/minor chords also allow third basses; explicit slash basses and inversions take priority. Unplayable variants are discarded. Each instrument selects a connected voicing path across the source form to reduce hand movement, then reuses it through the arrangement.
Accompaniment uses independently authored two-bar jazz pattern families: Charleston and shifted Charleston, forward-motion syncopation, anticipated heavy beats, syncopated riffs, quarters, dotted spacing, shifting motifs, call-response and anticipated beat four. Acoustic distributes a single figure between bass and upper chord, normally 1–3 total attacks per full bar, without mandatory beat-one bass. Separate attacks are quarters/eighths and never overlap; a combined bass-plus-upper chord is allowed only when both attack and sustain together for more than one beat. Electric has distinct patterns with normally 0–2 attacks per bar, stabs, sustained chords and space. The source melody and harmonic rhythm guide complete-pattern selection. Half-beat upper/electric anticipations may establish the next written chord; bass stays on current harmony. Explicit cross-bar holds use ties, never automatic merging of repeated attacks. Canonical patterns repeat at corresponding form positions with documented harmonic/pickup exceptions, and changing only tempo leaves the music unchanged.
Accompaniment uses independently authored two-bar jazz pattern families, chosen against each song's original melody and harmonic rhythm. Acoustic repeats two patterns (A–B); electric repeats four (A–B–C–D). Occasional source-phrase-ending fills recur at the same positions in every iteration; both instruments allow at most one per eight full measures, including the repeat join. Acoustic normally has 1–3 attack positions per full bar with quarter/eighth bass and upper-chord notes that may overlap or sound together. Electric has distinct patterns with normally 0–2 attacks per bar, stabs, sustains and space. Cross-bar holds use ties; repeated attacks remain attacks. Pattern phase follows source-form position, including partial entries and odd-length forms. Meter changes use compatible cycles, with explicit diagnostics if fewer distinct patterns are available. Tempo changes do not reshuffle the accompaniment.
Each instrument has 24 distinct two-bar patterns in 4/4, 24 in 3/4, and 24 for each of the 3+2 and 2+3 groupings in 5/4. Written additive grouping takes precedence. Otherwise, the original chord changes and melody choose one consistent 5/4 grouping per continuous meter section, shared as source analysis by the independently generated parts. Explicit meter matching prevents treating 6/8 as 3/4. The learned solo model and playback count-in are unchanged by this accompaniment expansion.
Both accompaniment parts balance movement with sounding register and a soft cost for mean fret positions above nine; comfortable shifts can win over staying high. Acoustic uses a register center of MIDI 53 and electric uses 60. Compact spans and preferred chord families remain higher priorities, including for the final sustained chord. Electric melody's supporting chords use the same balance after the existing melody-string and foreground priorities; the original melody pitches and rhythm remain fixed. Voicing preferences do not reseed accompaniment rhythms.
The first new acoustic accompaniment attack in every measure includes bass, either alone or with the upper chord in a combined sustain longer than one beat. This can be a delayed entrance; no beat-one attack is required. Pattern fitting exchanges upper/bass assignments when needed, without inserting attacks. Tied carries and silent bars do not require a new bass attack.
The first acoustic attack for each new chord includes bass, alone or paired with the upper chord. A mid-measure change follows this rule; an unchanged chord crossing a barline does not create a bass obligation. Delayed entrances remain delayed. A qualified half-beat anticipated chord includes its bass at the anticipated entrance. Bass and upper notes share one engraved part, with ties preserving overlaps and independent releases.
Each song has six canonical roles: electric melody, electric accompaniment, electric solo, acoustic accompaniment, acoustic solo, and vocal melody. Matching source-form positions reuse the same musical events and TAB whenever a role returns. Final-turnaround omission changes only affected harmony; source-defined pickups/handoffs and the three-bar instrumental closing sustain remain explicit exceptions.
**Registered Solo 1 is notated exactly in the acoustic part and Solo 2 exactly in the electric part.** Unregistered song/slot pairs use deterministic prediction from one pooled statistical model trained on both slots. There is no learned acoustic/electric style distinction. Prediction uses up to four recent pitch/rhythm events, source-melody features, harmony, and metrical position; it does not plan complete themes or guarantee long-range phrase development. The current corpus is four authored solos from Autumn Leaves and All of Me.
+18 -6
View File
@@ -131,17 +131,27 @@ The active voicing selector compares complete paths with dynamic programming ove
Silent spans retain the preceding hand position. A continuous accompaniment repeat includes the last-to-first transition. Source-harmony variants, such as relocated pickup harmony, are solved between fixed canonical neighbors without mutating a shared previous-grip state; request order cannot change them. Per-span composition reports include candidate counts, selected family, bass/string family availability, selected positions, bass degree and movement, plus `fret_span`, `span_group` and `span_candidates` (primary/fallback counts before dominance reduction). These are diagnostic sidecar fields; the score/runtime format is unchanged.
`comping_rhythm.py` selects complete source-only figures from version 4 of `composition/accompaniment.json`. Entries contain stable IDs, a family, exact one/two-bar lengths, and timed events labeled bass, upper, combined, or electric. The ten 4/4 families have statement/answer variants, with additional held-answer/space variants and explicit 3/4, 5/4 and short-bar alternatives. No independent bass/upper Cartesian product or unconstrained onset enumeration remains. Role pools have different complete two-bar onset signatures; clipped partial figures and N.C. passages can coincide. Unknown measure lengths fail explicitly rather than being proportionally stretched.
`comping_rhythm.py` selects recurring source-only figures from version 6 of `composition/accompaniment.json`. The normal acoustic cycle has two two-bar slots; electric has four. Each template and anticipation mode is evaluated at every occurrence of its slot before retrieval limits apply. Up to eight diverse candidates per slot enter bounded complete-cycle comparison. Distinct slots differ in attack/duration relationships, not just IDs, bass/upper assignments or anticipation flags. Repetition is not penalized. The source melody and harmonic rhythm affect selection; title, iteration, generated solo and tempo do not. `groove_cycles` reports slot identities, anticipation modes, aggregate cost, compared cycles and any reduction caused by vocabulary or form length.
Normal acoustic figures use 1–3 total attacks per full bar, interleaving bass and upper chords without a downbeat-bass quota. Ordinary events last a quarter or eighth and must release before the next event. Only a single combined event lasting more than one quarter-note beat can sound bass and upper together; its components start/release together. This is checked again on complete sounding tie chains. Electric figures normally have 0–2 attacks per bar and mix stabs, sustains and space. A figure may cross its internal barline: flat sounding events are selected and voiced first, then projected into notation with exact ties. Repeated attacks remain attacks. Figures do not extend beyond their selected two-bar span.
Meter-specific full patterns declare `meter` and, for 5/4, `grouping: [3, 2]` or `[2, 3]`; `meter: partial` marks the bounded one-bar pickup vocabulary. The loader checks nominal lengths, grouping, quarter/eighth arithmetic, lane overlap and distinct acoustic/electric onset patterns. Each role has 24 distinct attack/duration figures per selectable pool: 4/4 (11 families), 3/4 (12 families), and each 5/4 grouping (12 families). The 4/4 pool removes one duplicate acoustic figure, develops another previously duplicated answer, and adds three electric variants. Patterns are independently authored from general principles, with research URLs and provenance in the resource, rather than transcriptions. The counts describe available choices, not required musical activity.
The acoustic `first_attack: bass_or_combined` policy applies separately to every measure, including pickups. After removing unavailable events under missing harmony and participation limits, fitting exchanges an upper-first event with the earliest later bass event in that measure, or assigns it to bass when no exchange is possible. Timing and attack count stay intact apart from existing harmonic clipping. Reassigned bass loses any anticipation permission and binds to current harmony. Anticipations and releases are resolved afterward, and invalid figures are rejected. Delayed entrances remain delayed; a combined first attack must last more than one beat. A tied carry is not an attack and a silent measure remains silent. Pattern reports record `layer_assignment` changes, and complete sounding-event validation checks bass-first again after projection. The three-bar ending remains one combined chord attack.
`grouping_for` is a pure source-only decision. It honors written 3+2 or 2+3, otherwise compares actual chord changes at the secondary boundary, then melody entrances following at least an eighth-note rest, then other melody attacks; equal evidence defaults to 3+2. An unchanged chord marking is not a change. The selection stays fixed through a continuous meter section, and each role derives it independently from the same source. Fills cannot borrow another grouping. No accents or playback swing changes are inferred. `groove_cycles` additionally reports grouping, its evidence and the eligible pattern count. Equal-duration compound meters (such as 6/8) do not borrow the 3/4 vocabulary; unsupported comping meters fail explicitly.
Direct events release at changed harmony. Eligible upper/electric events also offer a half-beat anticipation: the upcoming written chord must begin exactly half a beat after the attack, the event must continue into that chord, and it must release by the following change or protected participation boundary. The selected grip belongs to that upcoming chord. Bass and combined acoustic events cannot anticipate. Direct alternatives remain available. Missing harmony/N.C. stays silent; incompatible short durations reject a complete candidate rather than introducing unexplained subdivisions.
Accompaniment dependency keys and choice streams include only full-pattern meters and partial lengths present in that source, plus the applicable instrument policy. Adding unused-meter patterns therefore does not reseed an existing song; unrelated role policies remain excluded. Research prose is excluded from musical seeds. This schema migration changes the accompaniment choice-key domain once; it does not promise identical previous 4/4 tie-break winners. Source melodies, registered solos, notation format and playback count-in are unchanged.
Named activity, melody-support, harmonic-support and articulation contributions evaluate complete figures. A bounded dynamic program selects the form path, with a modest preference against immediate exact repetition. Family representatives survive the candidate cutoff. Source entrances, movement and space affect selection, while no generated role does. Effective final turnarounds and pickup masks revalidate the whole affected figure between fixed canonical neighbors. Reports identify the chosen pattern, its start measure, projected events, continuations, anticipation target times, score contributions and source-context exceptions. Tempo affects metadata/cache identity but never musical choice.
The cycle begins at the first full source-body measure. A partial-form entry first fits the incoming harmony, then slices the sounded figure instead of restarting it; this prevents a previously clipped chord hold from reappearing as a new attack. An odd final bar uses the appropriate half-pattern. Meter changes restart a compatible cycle. Unsupported durations fail explicitly, and limited vocabularies report a shorter cycle. Optional final-harmony changes rebind the same selected pattern. Canonical grips are fixed wherever the chord and interval start remain unchanged, so a new turnaround cannot move neighboring unaffected voicings.
Acoustic events use temporary bass/upper notation lanes for projection; then `acoustic.combine` converts their sounding intervals to one rhythmic voice before assembly. It splits notes at attack/release boundaries and ties only continuing pitches; repeated attacks stay separate. Staff and TAB share the result, with no competing rests or opposing layer stems. This preserves exact sounding durations instead of truncating overlaps. The older truncating `prepare` helper serves only the noncanonical legacy/test path. The product build audits canonical accompaniment reuse across arrangement iterations, allowing only source-derived changes in harmony, pickups and role handoffs. The `.roles` sidecars report pattern IDs, exact projected layer events, continuations, anticipation targets and exceptions. Their `selection_cost` records the complete figure’s local contributions, repeated on each projected bar for context; the form selector additionally applies its documented transition cost. Accompaniment dependency keys contain only that instrument's policy; electric melody consumes the shared electric voicing families. Solos retain their existing learned source-melody/harmony conditioning and registered exact performances.
Fills replace only a two-bar figure's ending bar, using a same-family variation that improves the existing source-fit objective. Eligible boundaries are source section endings, form endings, or a melody breath of at least one quarter-note beat at the block end. Carried events from the opening bar are protected. The largest improvements are retained with an eight-measure minimum separation around the whole cyclic form, equally for both roles; no fill quota applies. Fills do not change cycle phase and recur at identical source positions. Reports record the base slot, selected fill, source index and reason.
Normal acoustic patterns use 1–3 attack positions per full bar. Separate bass and upper events last quarters/eighths and may overlap; paired attacks can be short or sustained. Events cannot overlap within the same lane. First attacks of new harmonic spans include bass (`first_attack: bass_per_harmony`), including mid-bar changes; continued harmony does not reset at barlines or computational boundaries. Fitting can pair bass with an upper attack without changing its onset, with independent releases where necessary. A paired attack counts once in rhythm evaluation. The completed validator follows sounding tie chains; placement validates all simultaneously sounding strings, fret spans and anchors. The two lanes are merged into one notated part with tied fragments preserving their true durations.
Electric normally uses 0–2 attack positions per bar, with stabs, sustains and space. Direct events release at changed harmony. An eligible upper/electric template can anticipate a chord by half a beat when it continues into that chord; the grip comes from the upcoming harmony and releases by its following change. Acoustic anticipations also include bass on the new chord's first attack. Conflicting lane overlaps reject the candidate. Missing harmony remains silent. Protected participation limits retain the chosen figure, shortening only its affected releases and recording exceptions. Cross-bar sounding notes become ties, never automatic merging of repeated attacks.
All six production roles receive reuse audits. Musical comparisons include rests, pitch durations, ties and TAB while ignoring regenerated event IDs and global measure numbers. Source-defined pickup/participation changes and effective final-turnaround harmony are reported separately. The final three-bar hold remains instrumental.
Named activity, melody-support, harmonic-support and articulation contributions evaluate complete figures. Cycle selection aggregates them over all occurrences, retaining family diversity before the final bounded comparison. `groove_cycles.cost` is the selected base cycle's aggregate cost before optional fills. Reports identify the base slot, realized pattern, projected events, continuations, anticipation target times and source-context exceptions. Tempo affects metadata/cache identity but never musical choice.
Acoustic events use temporary bass/upper notation lanes for projection; then `acoustic.combine` converts their sounding intervals to one rhythmic voice before assembly. It splits notes at attack/release boundaries and ties only continuing pitches; repeated attacks stay separate. Staff and TAB share the result, with no competing rests or opposing layer stems. This preserves exact sounding durations instead of truncating overlaps. The older truncating `prepare` helper serves only the noncanonical legacy/test path. The product build audits canonical accompaniment reuse across arrangement iterations, allowing only source-derived changes in harmony, pickups and role handoffs. The `.roles` sidecars report pattern IDs, exact projected layer events, continuations, anticipation targets and exceptions. Their `selection_cost` records the complete figure’s local contributions, repeated on each projected bar for context; the cycle selector aggregates these by the number of measures in each occurrence, without an anti-repetition transition cost. Accompaniment dependency keys contain only that instrument's policy; electric melody consumes the shared electric voicing families. Solos retain their existing learned source-melody/harmony conditioning and registered exact performances.
Monophonic solos use strings 1–6 and frets 1–12, with no open strings or acoustic/electric string restrictions. Placement compares complete paths through legal positions, accounting for hand-region changes, travel time and string crossings. Ties retain their position. Chord placement retains its separate policy: frets 1–20, a sounding anchor on frets 1–12, primary spans of up to three consecutive positions and fallback spans of exactly four. The span is inclusive: 7–9 is three positions, 7–10 is four. Five positions are always invalid. The anchor must remain sounding, including after shorter notes release. Acoustic accompaniment additionally keeps its bass within frets 1–12 and requires an independent anchor in its upper three-note cluster, preserving the option of separate bass/upper attacks. These costs approximate physical feasibility; they do not assign fingers or guarantee comfortable playing.
@@ -149,6 +159,8 @@ Both accompaniment roles add `register_weight * abs(mean sounding MIDI - center)
Each selected accompaniment span reports `voicing_cost` contributions for movement, base grip effort, weighted register, weighted position and their combined effort. These describe the balanced tier, not the higher span/family priorities; repeat-join travel is assigned to the first canonical span. Both accompaniment roles keep `voicing` settings in concrete cache dependencies but exclude them from rhythmic choice seeds, including settings overrides. Acoustic rhythm selection can change under its new bass-first policy, but changing voicing weights alone never reshuffles rhythm. Each role still composes independently.
Recurring-groove verification on October 6, 2026: the full Python regression command passed 423 tests in 149.85 seconds. After the final partial-entry grip fix and one additional real-evaluator fill test, the focused accompaniment/arrangement command passed 59 tests in 21.64 seconds. The native test target passed in 30.58 seconds. The initial native rebuild took 42 seconds, including 7 seconds for independently generated roles and 33.62 seconds of engraving; accompaniment roles took 0.8–1.6 seconds each. The following ARM build reused all role/score caches and took 11 seconds. All four solo documents, both electric melodies and both vocals exactly matched the prechange baseline; native/ARM score assets matched byte for byte. The chosen base cycles differed by song and retained all two/four requested slots. Autumn Leaves selected the authored overlapping-quarter acoustic pattern. No optional fill improved these production winners, so no fills were forced; independently authored tests verify real source-fit selection of a fill. An acoustic score/TAB page was visually checked for overlap ties in a single part. Recordings were unchanged and no listening assessment was performed. Baseline snapshots stayed under `/tmp/`.
Shared-preference and bass-first verification on October 6, 2026: `python3 modulus.py python -m unittest discover -s tests -p 'test_*.py'` passed 408 tests in 61.35 seconds, and the native test target passed all 124 cases. Both product builds passed asset checks; 139 native/ARM score and page files matched byte for byte. All 273 acoustic measures containing new attacks started with bass, including 69 delayed entrances. Tied carries did not trigger new attacks. Both solos, vocals, electric accompaniment rhythms, and electric foreground melody pitches/timing/TAB matched the saved baseline exactly. Some All of Me melody support chords changed; Autumn Leaves' complete electric melody document did not. All of Me's canonical electric accompaniment mean fret dropped from 7.26 to 4.13 and movement cost from 57 to 48; Autumn Leaves stayed at 2.95 and 46. Recordings were unchanged. These are score checks and page inspections, not a listening assessment.
With no concurrent build, isolated cold role generation measured 0.56–0.57 seconds for acoustic accompaniment, 0.35–0.37 for electric accompaniment, and 0.26–0.29 for electric melody; cache hits took 0.02–0.04 seconds and matched clean regeneration for both songs. The normal two-worker native asset stage took about 14 seconds (4 composing, 9 engraving, 1 other); ARM reused the caches in about 4 seconds. Application compilation is separate from these asset-stage timings.
File diff suppressed because it is too large Load Diff
+23 -11
View File
@@ -7,6 +7,7 @@ import itertools
import generator.accompaniment
import generator.model
import generator.score
import generator.harmonic_time
@dataclasses.dataclass
class HarmonySpan:
@@ -138,18 +139,29 @@ def validate_comping(document: dict) -> None:
events = []
for (start, end), lanes in groups.items():
if len(lanes) == 2:
if end - start <= 1:
raise ValueError('combined acoustic sustain must exceed one beat')
if end - start < 1 and end - start != fractions.Fraction(1, 2):
raise ValueError('combined acoustic attacks require quarter/eighth notes or sustains')
elif end - start not in (fractions.Fraction(1, 2), 1):
raise ValueError('separate acoustic comping requires quarter/eighth notes')
events.append((start, end))
events.sort()
if any(a[1] > b[0] for a, b in zip(events, events[1:])):
raise ValueError('acoustic bass and upper must release together or remain separate')
cursor = fractions.Fraction()
for measure in document['staves'][0]['measures']:
length = generator.model.fraction(measure['duration'])
first = next((event for event in events if cursor <= event[0] < cursor + length), None)
if first and 'bass' not in groups[first]:
raise ValueError(f'measure {measure["number"]}: first acoustic attack must include bass')
cursor += length
# Distinct lanes can overlap. The physical validator still checks sounding
# strings and grip span, including staggered releases after voice merging.
for i, a in enumerate(events):
for b in events[i + 1:]:
if a[1] > b[0] and groups[a] & groups[b]:
raise ValueError('acoustic attacks overlap within the same lane')
spans = []
for span in generator.harmonic_time.read(document['staves'][0]):
if spans and generator.accompaniment.same_chord(spans[-1].chord, span.chord):
spans[-1] = dataclasses.replace(spans[-1], end=span.end)
else:
spans.append(span)
for span in spans:
if span.chord is None:
continue
first = next((event for event in events if span.start <= event[0] < span.end), None)
# A bounded anticipated bass can already be sounding into this harmony.
anticipated = any(span.start - fractions.Fraction(1, 2) <= a < span.start < b and 'bass' in groups[a, b] for a, b in events)
if first and not anticipated and not any('bass' in groups[e] for e in events if e[0] == first[0]):
raise ValueError(f'measure {span.source_measure}, beat {span.beat}: first acoustic attack for a chord must include bass')
+13 -11
View File
@@ -25,28 +25,30 @@ def harmonic_key(measure, previous):
return (key(active), tuple((generator.model.fraction(h['onset']), key(h)) for h in measure['harmonies']
if generator.model.fraction(h['onset']) > 0), measure['duration'], measure['time_signature'])
def audit(source, documents):
def audit(source, documents, *, role_documents=False):
"""Compare music, not generated IDs, at every reused form position."""
authored = generator.score.source_staff(source)['measures']
result = {}
for part, document in documents.items():
if part == 'vocal':
for name, document in documents.items():
part = name.split('_')[0] if role_documents else name
wanted = {'solo': 'slash', 'melody': 'vocal' if part == 'vocal' else 'melody'}.get(name.split('_')[-1], 'accompaniment') if role_documents else 'accompaniment'
if part == 'vocal' and not role_documents:
continue
sections = generator.arrange.form.full_arrangement_sections(authored, part)
entry_positions = {s.source_measure_index for s in sections if s.role == 'accompaniment'}
entry_positions = {s.source_measure_index for s in sections if s.role == wanted}
measures = document['staves'][0]['measures']
carried = generator.accompaniment.carried_chords(measures)
seen, exceptions, offset, reused = {}, [], 0, 0
for si, section in enumerate(sections):
following = sections[si + 1] if si + 1 < len(sections) else None
for j in range(section.measure_count):
if section.role != 'accompaniment':
if section.role != wanted:
continue
m = measures[offset + j]
if j == section.measure_count - 1 and following and following.role == 'melody' and generator.arrange.pickup.enters(following, authored):
if wanted == 'accompaniment' and j == section.measure_count - 1 and following and following.role == 'melody' and generator.arrange.pickup.enters(following, authored):
exceptions.append({'measure': m['number'], 'reason': 'protected melody pickup'})
continue
if j == section.measure_count - 1 and following and following.role == 'slash' and generator.score.body_start_measure_index(authored):
if wanted == 'accompaniment' and j == section.measure_count - 1 and following and following.role == 'slash' and generator.score.body_start_measure_index(authored):
exceptions.append({'measure': m['number'], 'reason': 'source-defined solo pickup allocation'})
continue
position = section.source_measure_index + j
@@ -60,16 +62,16 @@ def audit(source, documents):
boundary = following.role if following and block_start + 2 >= section.source_measure_index + section.measure_count and generator.score.body_start_measure_index(authored) and following.role in ('slash', 'melody') else ''
key = (position, context, boundary)
signature = tuple((v['id'], generator.model.fraction(n['onset']), generator.model.fraction(n['duration']),
tuple((generator.model.pitch_number(p), generator.model.fraction(p['duration']), p['tablature']['string'], p['tablature']['fret'], bool(p.get('tie_from')) if position not in entry_positions or generator.model.fraction(n['onset']) else False) for p in n['pitches']))
for v in m['voices'] for n in v['notes'] if n['pitches'])
tuple((generator.model.pitch_number(p), generator.model.fraction(p['duration']), p.get('tablature', {}).get('string'), p.get('tablature', {}).get('fret'), bool(p.get('tie_from')) if position not in entry_positions or generator.model.fraction(n['onset']) else False, bool(p.get('tie_to'))) for p in n['pitches']))
for v in m['voices'] for n in v['notes'])
if key in seen:
if signature != seen[key]:
raise ValueError(f'{part}, arrangement measure {m["number"]}: canonical accompaniment changed')
raise ValueError(f'{name}, arrangement measure {m["number"]}: canonical {wanted} changed')
reused += 1
else:
if any(k[0] == key[0] for k in seen):
exceptions.append({'measure': m['number'], 'reason': 'changed two-bar harmony, duration, or protected participation'})
seen[key] = signature
offset += section.measure_count
result[part] = {'reused_bars': reused, 'distinct_form_bars': len(seen), 'exceptions': exceptions}
result[name] = {'reused_bars': reused, 'distinct_form_bars': len(seen), 'exceptions': exceptions}
return result
+44 -11
View File
@@ -187,6 +187,9 @@ class Canonical:
self.policy = generator.vocabulary.role(part + '_accompaniment')
self.cache, self._reports, self._rhythm_reports = {}, {}, {}
self.measures = generator.score.source_staff(document)['measures']
# A held accompaniment chord may continue into the bar containing the
# source melody pickup. Keep its voice identity through that splice.
self.voice_id = self.measures[0]['voices'][0]['id']
self.melody = generator.comping_rhythm.melody_contexts(document)
self.first = generator.score.body_start_measure_index(self.measures)
sections = generator.arrange.form.full_arrangement_sections(self.measures, part)
@@ -199,15 +202,18 @@ class Canonical:
size = 1
if index + 1 < len(self.measures):
lengths = [generator.model.fraction(m['duration']) for m in self.measures[index:index + 2]]
if any(tuple(map(fractions.Fraction, p['lengths'])) == tuple(lengths) for p in generator.vocabulary.accompaniment(part)['patterns']):
if self.measures[index]['time_signature'] == self.measures[index + 1]['time_signature'] and any(tuple(map(fractions.Fraction, p['lengths'])) == tuple(lengths) for p in generator.vocabulary.accompaniment(part)['patterns']):
size = 2
self.groups.append(tuple((self.measures[i], carried[i], i) for i in range(index, index + size)))
index += size
layers = [self.candidates(group) for group in self.groups]
selected = generator.comping_rhythm.select(layers, seed)
choices, locations, self.groove_report = generator.comping_rhythm.choose_cycles(self.groups, part, self.melody, seed)
self.patterns = {g[0][2]: (p, a) for g, (p, a, _) in zip(self.groups, choices)}
self.locations = {g[0][2]: location for g, location in zip(self.groups, locations)}
selected = [r for _, _, r in choices]
self.block_rhythms = {group[0][2]: rhythm for group, rhythm in zip(self.groups, selected)}
prepared = [self.prepare(group, rhythm) for group, rhythm in zip(self.groups, selected)]
active = tuple(span for spans in prepared for span in spans if span.events)
self.canonical_spans = {group[0][2]: spans for group, spans in zip(self.groups, prepared)}
path, self.path_cost = self.select(active, close=self.closed)
self.base = dict(zip((s.key for s in active), path))
self.order = tuple((s.index, s.start, v) for s, v in zip(active, path))
@@ -264,6 +270,7 @@ class Canonical:
return select_grips(layers, costs, self.part, self.seed, left=left, right=right, close=close)
def __call__(self, source, prior, index, *, sources=None, bounds=None, stop=None):
group = next((g for g in self.groups if any(i == index for _, _, i in g)), ((source, prior, index),))
original_group = group
sources = sources or self.measures
carried = generator.accompaniment.carried_chords(sources)
low, high = bounds or (self.first, len(sources))
@@ -273,14 +280,38 @@ class Canonical:
limit = sum((generator.model.fraction(m['duration']) for m, _, i in group if i < stop[0]), fractions.Fraction()) + stop[1]
key = self.key(group, limit)
if key not in self.cache:
choices = self.candidates(group, limit)
start, end = group[0][2], group[-1][2]
# Preserve the canonical whole figure where still legal under changed
# effective harmony. Recompute only genuinely different source context.
original = self.block_rhythms.get(start)
matches = [r for r in choices if original and r.pattern_id == original.pattern_id]
rhythm = matches[0] if matches else generator.comping_rhythm.select((choices,), self.seed, self.block_rhythms.get(start - 2), self.block_rhythms.get(end + 1))[0]
offset = sum((generator.model.fraction(m['duration']) for m, _, i in original_group if i < start), fractions.Fraction())
if original_group[0][2] in self.patterns:
pattern, anticipate = self.patterns[original_group[0][2]]
if offset:
# Fit the incoming context before slicing. A raw template sustain
# may already have ended at a chord change; projecting it first
# would resurrect that note as an unintended new downbeat attack.
context = tuple((source if i == index else sources[i], prior if i == index else carried[i], i)
for _, _, i in original_group if i <= end)
incoming = generator.comping_rhythm.realize(pattern, context, self.part, self.melody, anticipate=anticipate, limit=offset + limit if limit is not None else None, boundary=True)
if incoming is None:
raise ValueError(f'source measure {source["number"]}: selected groove cannot fit incoming harmony')
pattern = {**pattern, 'events': [{'onset': str(e.start), 'duration': str(e.end - e.start),
'layer': e.lane, 'anticipate': e.harmonic_time is not None and e.lane in ('upper', 'electric')} for e in incoming.events]}
pattern = generator.comping_rhythm.project(pattern, offset, tuple(generator.model.fraction(m['duration']) for m, _, _ in group))
rhythm = generator.comping_rhythm.realize(pattern, group, self.part, self.melody, anticipate=anticipate, limit=limit, boundary=True)
if rhythm is None:
raise ValueError(f'source measure {source["number"]}: selected groove cannot fit protected boundary')
else:
# A source pickup outside the repeating body uses the existing partial
# vocabulary. It never selects or changes the body groove.
rhythm = min(self.candidates(group, limit), key=lambda r: (r.cost, generator.solo.policy.stable_rank(self.seed, r.signature)))
spans = self.prepare(group, rhythm)
# Keep every unchanged harmonic interval's canonical grip, even when a
# neighbor loses an optional turnaround or a section starts mid-pattern.
fixed = []
for span in spans:
original = next((s for s in self.canonical_spans.get(original_group[0][2], ()) if s.key in self.base
and s.start <= span.start + offset < s.end and semantic_key(s.chord) == semantic_key(span.chord)), None)
fixed.append(dataclasses.replace(span, options=(self.base[original.key],)) if original else span)
spans = tuple(fixed)
before = [v for i, _, v in self.order if i < start]
after = [v for i, _, v in self.order if i > end]
left = before[-1] if before else self.order[-1][2] if self.closed and self.order else ()
@@ -326,13 +357,14 @@ class Canonical:
self._reports[key] = rows
voices = []
for lane in ['upper', 'bass'] if self.part == 'acoustic' else ['electric']:
voice_id = lane if self.part == 'acoustic' else self.voice_id
notes, cursor = [], fractions.Fraction()
def emit(a, b, positions=(), chord=None):
previous = None
for duration, written in generator.accompaniment.written_durations(b - a):
pitches = generator.fretboard.chord_pitches(chord, positions) if positions else []
n = generator.model.make_event(f'comp_{self.part}_{index}_{lane}_{len(notes)}', a, duration, 'note' if pitches else 'rest', pitches, written)
n['voice'] = lane
n['voice'] = voice_id
if previous and pitches:
for old, new in zip(previous['pitches'], n['pitches']):
old['tie_to'], new['tie_from'] = new['id'], old['id']
@@ -346,12 +378,13 @@ class Canonical:
cursor = end
if cursor < length:
emit(cursor, length)
voices.append({'id': lane, 'notes': notes})
voices.append({'id': voice_id, 'notes': notes})
result = split_measures({'voices': voices}, inputs, self.part)
reports, cursor = [], fractions.Fraction()
for m, _, i in inputs:
duration = generator.model.fraction(m['duration'])
reports.append({'source_index': i, 'pattern_id': rhythm.pattern_id, 'pattern_start': index,
**next((self.locations[g[0][2]] for g in self.groups if any(j == i for _, _, j in g)), {}),
'families': dict(rhythm.families), 'contributions': dict(rhythm.contributions), 'selection_cost': rhythm.cost,
'exceptions': rhythm.exceptions, 'source_melody': [(str(a), str(b)) for a, b in self.melody[i]],
'events': [{'layer': e.lane, 'onset': str(max(e.start, cursor) - cursor), 'duration': str(min(e.end, cursor + duration) - max(e.start, cursor)),
+264 -45
View File
@@ -1,6 +1,7 @@
"""Complete, source-only comping figures. Time is in rational quarter notes."""
import dataclasses
import fractions
import itertools
import generator.accompaniment
import generator.model
@@ -8,6 +9,65 @@ import generator.solo.analysis
import generator.solo.policy
import generator.vocabulary
def meter_context(measure):
"""Nominal meter and explicit additive grouping, independent of bar length.
In particular, a three-quarter-note pickup or a 6/8 bar is not 3/4.
"""
groups = measure['meter']['groups']
if not groups or any(g['unit'] != 4 for g in groups):
raise ValueError(f"source measure {measure['number']}: no comping vocabulary for {measure['time_signature']}")
beats = tuple(b for g in groups for b in g['beats'])
total = sum(beats)
if total not in (3, 4, 5) or (len(beats) > 1 and beats not in ((3, 2), (2, 3))):
raise ValueError(f"source measure {measure['number']}: unsupported comping grouping {measure['time_signature']}")
return f'{total}/4', beats if len(beats) > 1 else ()
def grouping_for(run, melody):
"""Choose one source-only quintuple feel, shared by independent roles.
Written grouping wins. Otherwise compare actual chord changes, entrances
after an eighth-note breath, then all melodic attacks at the group boundary.
Counts are evidence, not inferred accents or performance dynamics.
"""
meter, explicit = meter_context(run[0][0][0])
if meter != '5/4':
return (), {'reason': 'simple_meter'}
if explicit:
return explicit, {'reason': 'written'}
evidence = {}
for boundary in (3, 2):
changes, entrances, attacks = 0, 0, 0
for group in run:
for m, prior, i in group:
changes += sum(a == boundary and c is not None for a, _, c in spans(m, prior))
context = melody[i]
for a, _ in context:
if a != boundary:
continue
attacks += 1
previous = max((b for x, b in context if x < a), default=fractions.Fraction())
entrances += a - previous >= fractions.Fraction(1, 2)
evidence[boundary] = (changes, entrances, attacks)
boundary = 2 if evidence[2] > evidence[3] else 3
return (boundary, 5 - boundary), {'reason': 'source' if evidence[2] != evidence[3] else 'default_3+2',
'boundary_evidence': {str(k): dict(zip(('chord_changes', 'melody_entrances', 'melody_attacks'), v)) for k, v in evidence.items()}}
def patterns_for(policy, measure, lengths, grouping=()):
meter, explicit = meter_context(measure)
grouping = explicit or grouping or ((3, 2) if meter == '5/4' else ())
# Untagged in-memory figures are allowed for independently authored unit
# tests; the resource loader requires metadata for every production entry.
return [p for p in policy['patterns'] if tuple(map(fractions.Fraction, p['lengths'])) == tuple(lengths)
and (p.get('meter') == 'partial' or (p.get('meter', meter) == meter and tuple(p.get('grouping', grouping)) == grouping))]
def relevant_policy(policy, measures):
"""An unused meter's vocabulary must not influence this role's decisions."""
meters = {meter_context(m)[0] for m in measures}
lengths = {generator.model.fraction(m['duration']) for m in measures}
return {**policy, 'patterns': [p for p in policy['patterns'] if p.get('meter') in meters
or (p.get('meter') == 'partial' and fractions.Fraction(p['lengths'][0]) in lengths)]}
@dataclasses.dataclass(frozen=True, order=True)
class Attack:
lane: str
@@ -72,23 +132,18 @@ def validate_events(events, part, lengths=None):
raise ValueError('comping event must have positive sounding duration')
if part == 'acoustic':
if e.lane == 'combined':
if duration <= 1:
raise ValueError('combined acoustic sustain must exceed one beat')
if duration < 1 and duration != fractions.Fraction(1, 2):
raise ValueError('combined acoustic attacks require quarter/eighth notes or sustains')
elif e.lane not in ('bass', 'upper') or duration not in (fractions.Fraction(1, 2), 1):
raise ValueError('ordinary acoustic attacks require separate quarter/eighth notes')
elif e.lane != 'electric':
raise ValueError('electric comping requires chord events')
if any(a.end > b.start for a, b in zip(ordered, ordered[1:])):
raise ValueError('separate comping events cannot overlap; use one combined sustain')
if part == 'acoustic' and lengths is not None:
cursor = fractions.Fraction()
for length in lengths:
first = next((e for e in ordered if cursor <= e.start < cursor + length), None)
if first and first.lane not in ('bass', 'combined'):
raise ValueError('first acoustic attack in each measure must include bass')
cursor += length
for i, a in enumerate(ordered):
for b in ordered[i + 1:]:
if a.end > b.start and (part != 'acoustic' or a.lane == b.lane or 'combined' in (a.lane, b.lane)):
raise ValueError('comping attacks cannot overlap on the same strings')
def fit(pattern, measure, prior, part, *, anticipate=False, limit=None):
def fit(pattern, measure, prior, part, *, anticipate=False, limit=None, initial_bass=True):
"""Bind a complete figure to written harmony, with bounded early arrivals."""
length = generator.model.fraction(measure['duration'])
limit = length if limit is None else min(length, limit)
@@ -105,19 +160,6 @@ def fit(pattern, measure, prior, part, *, anticipate=False, limit=None):
continue
available.append((dict(row), start, end, active))
lengths = tuple(map(fractions.Fraction, pattern.get('lengths', (length,))))
if part == 'acoustic' and generator.vocabulary.accompaniment(part)['first_attack'] == 'bass_or_combined':
cursor = fractions.Fraction()
for span in lengths:
local = [(row, start) for row, start, _, _ in available if cursor <= start < cursor + span]
if local and local[0][0]['layer'] == 'upper':
first, onset = local[0]
later = next(((row, start) for row, start in local[1:] if row['layer'] == 'bass'), None)
first['layer'] = 'bass'
first.pop('anticipate', None)
if later:
later[0]['layer'] = 'upper'
exceptions.append(('layer_assignment', f'bass first at {onset}' + (f'; upper exchanged with bass at {later[1]}' if later else '; no later bass to exchange')))
cursor += span
for row, start, end, active in available:
_, boundary, chord = active
target = None
@@ -127,6 +169,28 @@ def fit(pattern, measure, prior, part, *, anticipate=False, limit=None):
target, boundary = future[0], future[1]
end = min(end, boundary)
events.append(Attack(row['layer'], start, end, target))
if part == 'acoustic':
for a, b, chord in harmony:
if chord is None or (a == 0 and not initial_bass):
continue
local = [e for e in events if a <= (e.harmonic_time if e.harmonic_time is not None else e.start) < b]
if not local:
continue
first = min(local, key=lambda e: e.start)
if any(e.start == first.start and e.lane in ('bass', 'combined') for e in local):
continue
# Preserve the comping rhythm and upper voice. Add bass at the same
# attack, allowing different releases when the next bass enters early.
stop = min([first.end, first.start + 1, *(e.start for e in events if e.start > first.start and e.lane in ('bass', 'combined'))])
duration = next((d for d in (fractions.Fraction(1), fractions.Fraction(1, 2)) if d <= stop - first.start), None)
if duration is None:
return None
if first.end - first.start == duration:
events[events.index(first)] = dataclasses.replace(first, lane='combined')
else:
events.append(Attack('bass', first.start, first.start + duration, first.harmonic_time))
exceptions.append(('layer_assignment', f'bass included at first attack for harmony {a}: {first.start}'))
events.sort(key=lambda e: (e.start, e.lane))
try:
validate_events(events, part, lengths)
except ValueError:
@@ -143,8 +207,11 @@ def evaluate(events, part, melody, harmony, lengths):
sounding = any(e.start < cursor + length and e.end > cursor for e in events)
target = (1 if busy else 1.5) if part == 'electric' else (2 if busy else 2.5)
# Sustained occupancy and planned space are not missing attacks.
activity += .22 * max(0, abs(len(local) - target) - .75) if not (sounding and not local) else 0
for e in local:
activity += .22 * max(0, abs(len({e.start for e in local}) - target) - .75) if not (sounding and not local) else 0
# A bass paired with the upper chord is one rhythmic attack, regardless
# of whether unequal releases require two internal lane events.
attacks_by_time = {e.start: max((v for v in local if v.start == e.start), key=lambda v: v.end) for e in local}
for e in attacks_by_time.values():
gap = not any(a <= e.start < b for a, b in melody)
entrance = any(a == e.start and (i == 0 or a - melody[i - 1][1] >= fractions.Fraction(1, 2)) for i, (a, _) in enumerate(melody))
support += -.2 if entrance else -.12 if gap else .18 if busy and e.start in attacks else 0
@@ -156,13 +223,11 @@ def evaluate(events, part, melody, harmony, lengths):
divisor = max(1, len(lengths))
return tuple((k, v / divisor) for k, v in (('activity', activity), ('melody_support', support), ('harmonic_support', coverage), ('articulation', articulation)))
def candidates(measure, prior, part, melody, seed, *, lengths=None, limit=None):
def candidates(measure, prior, part, melody, seed, *, lengths=None, limit=None, grouping=()):
lengths = lengths or (generator.model.fraction(measure['duration']),)
policy = generator.vocabulary.accompaniment(part)
result = []
for pattern in policy['patterns']:
if tuple(map(fractions.Fraction, pattern['lengths'])) != tuple(lengths):
continue
for pattern in patterns_for(policy, measure, lengths, grouping):
for anticipate in (False, True):
fitted = fit(pattern, measure, prior, part, anticipate=anticipate, limit=limit)
if fitted is None:
@@ -191,18 +256,172 @@ def representatives(candidates, limit, seed):
retained.extend(c for c in ordered if c not in retained)
return tuple(retained[:limit])
def transition(previous, current):
if previous is None:
return 0.
return .18 * (previous.signature == current.signature) + .04 * (previous.families == current.families)
def project(pattern, offset, lengths):
"""Take a form-position slice, rather than restarting a pattern at an entry."""
end = offset + sum(lengths)
rows = []
for row in pattern['events']:
a = fractions.Fraction(row['onset'])
b = a + fractions.Fraction(row['duration'])
if a < end and b > offset:
rows.append({**row, 'onset': str(max(a, offset) - offset), 'duration': str(min(b, end) - max(a, offset))})
return {**pattern, 'lengths': list(map(str, lengths)), 'events': rows}
def select(layers, seed, left=None, right=None):
"""Compare complete two-bar paths, not independently chosen bar fragments."""
states = [(0., (), (), left)]
for i, layer in enumerate(layers):
following = []
for c in layer:
options = [(cost + c.cost + transition(previous, c), (*ties, generator.solo.policy.stable_rank(seed, (i, c.pattern_id, c.signature))), (*path, c), c) for cost, ties, path, previous in states]
following.append(min(options, key=lambda item: item[:2]))
states = following
return min(states, key=lambda item: (item[0] + (transition(item[3], right) if right else 0), item[1]))[2]
def realize(pattern, inputs, part, melody, *, anticipate=False, limit=None, boundary=False):
flat = flatten(inputs)
lengths = tuple(generator.model.fraction(m['duration']) for m, _, _ in inputs)
initial_bass = not generator.accompaniment.same_chord(inputs[0][1], generator.accompaniment.active_chord(inputs[0][0], fractions.Fraction(), inputs[0][1]))
fitted = fit(pattern, flat, None, part, anticipate=anticipate, limit=limit, initial_bass=initial_bass)
if fitted is None and boundary:
# Keep the selected onset/layers at a protected handoff, using the longest
# permitted short articulation that fits rather than another groove.
clipped = project(pattern, fractions.Fraction(), (min(sum(lengths), limit) if limit is not None else sum(lengths),))
rows = []
for row in clipped['events']:
a, duration = fractions.Fraction(row['onset']), fractions.Fraction(row['duration'])
_, b, _ = next(s for s in spans(flat, None) if s[0] <= a < s[1])
duration = min(duration, b - a)
if part == 'acoustic' and (row['layer'] != 'combined' or duration <= 1):
duration = next((d for d in (fractions.Fraction(1), fractions.Fraction(1, 2)) if d <= duration), fractions.Fraction())
if duration:
rows.append({**row, 'duration': str(duration)})
fitted = fit({**pattern, 'events': rows}, flat, None, part, anticipate=False, limit=limit, initial_bass=initial_bass)
if fitted is not None:
fitted = fitted[0], (*fitted[1], ('release', 'shortened articulation at source-defined boundary'))
if fitted is None:
return None
events, exceptions = fitted
context, cursor = [], fractions.Fraction()
for m, _, i in inputs:
context.extend((a + cursor, b + cursor) for a, b in melody[i])
cursor += generator.model.fraction(m['duration'])
return Rhythm(events, (('pattern', pattern['family']),), evaluate(events, part, context, spans(flat, None), lengths), exceptions,
pattern['id'] + ('.anticipated' if any(e.harmonic_time is not None for e in events) else ''))
def identity(pattern):
"""Layer swaps and anticipation permissions alone are not different rhythms."""
return tuple((fractions.Fraction(r['onset']), fractions.Fraction(r['duration'])) for r in pattern['events'])
def choose_cycles(groups, part, melody, seed):
"""Select a small recurring vocabulary against every occurrence in the form.
No local shortlist precedes whole-form evaluation. A pattern/mode is retained
as one decision even where a chord boundary changes its legal release.
"""
policy = generator.vocabulary.accompaniment(part)
settings = policy['groove']
selected, locations, reports = [], [], []
runs = []
for group in groups:
m = group[0][0]
key = (meter_context(m), generator.model.fraction(m['duration']))
if not runs or runs[-1][0] != key:
runs.append((key, []))
runs[-1][1].append(group)
for ((meter, _), length), run in runs:
grouping, grouping_evidence = grouping_for(run, melody)
patterns = patterns_for(policy, run[0][0][0], (length, length), grouping)
if not patterns:
patterns = patterns_for(policy, run[0][0][0], (length,), grouping)
requested = settings['cycle_patterns']
identities = {p['id']: identity(p) for p in patterns}
available = len(set(identities.values()))
cache = {}
for i, group in enumerate(run):
lengths = tuple(generator.model.fraction(m['duration']) for m, _, _ in group)
for p in patterns:
for anticipate in (False, True):
cache[i, p['id'], anticipate] = realize(project(p, fractions.Fraction(), lengths), group, part, melody, anticipate=anticipate)
winner, examined = None, 0
for width in range(min(requested, len(run), available), 0, -1):
layers = []
for slot in range(width):
unique = {}
for p in patterns:
for anticipate in (False, True):
values = [cache[i, p['id'], anticipate] for i in range(slot, len(run), width)]
if any(v is None for v in values):
continue
cost = sum(v.cost * len(run[i]) for i, v in zip(range(slot, len(run), width), values))
rank = (cost, generator.solo.policy.stable_rank(seed, (meter, slot, p['id'], anticipate)))
option = (rank, p, anticipate)
key = identities[p['id']]
if key not in unique or rank < unique[key][0]:
unique[key] = option
ordered = sorted(unique.values(), key=lambda o: o[0])
diverse, families = [], set()
for option in ordered:
if option[1]['family'] not in families:
diverse.append(option)
families.add(option[1]['family'])
diverse.extend(o for o in ordered if o not in diverse)
layers.append(diverse[:settings['slot_candidates']])
for cycle in itertools.product(*layers):
if len({identities[o[1]['id']] for o in cycle}) != width:
continue
examined += 1
rank = (sum(o[0][0] for o in cycle), tuple(o[0][1] for o in cycle))
if winner is None or rank < winner[0]:
winner = rank, cycle
if winner is not None:
break
if winner is None:
raise ValueError(f'source measure {run[0][0][0]["number"]}: no legal recurring {part} groove for {meter}; check harmonic releases and pattern vocabulary')
cycle = winner[1]
report = {'source_start': run[0][0][2], 'source_end': run[-1][-1][2], 'meter': meter,
'grouping': list(grouping), 'grouping_evidence': grouping_evidence, 'eligible_patterns': len(patterns),
'requested_patterns': requested, 'cycle_patterns': len(cycle), 'patterns': [o[1]['id'] for o in cycle],
'anticipation': [o[2] for o in cycle], 'cost': winner[0][0], 'complete_cycles_compared': examined,
'reduction': 'available distinct, legal patterns or source length' if len(cycle) < requested else None}
reports.append(report)
for i, group in enumerate(run):
_, p, anticipate = cycle[i % len(cycle)]
selected.append((p, anticipate, cache[i, p['id'], anticipate]))
locations.append({'cycle_start': run[0][0][2], 'cycle_slot': i % len(cycle), 'base_pattern': p['id'], 'fill': None})
add_fills(groups, selected, locations, part, melody, seed)
return selected, locations, reports
def add_fills(groups, selected, locations, part, melody, seed):
"""Fixed source-position answers; no iteration seed or generated lead input."""
policy = generator.vocabulary.accompaniment(part)
spacing = policy['groove']['fill_spacing_measures']
if not groups:
return
body_start, body_end = groups[0][0][2], groups[-1][-1][2] + 1
body_length = body_end - body_start
options = []
if body_length < spacing:
return
for i, (group, (base, anticipate, current)) in enumerate(zip(groups, selected)):
if len(group) != 2:
continue
last, _, index = group[-1]
length = generator.model.fraction(last['duration'])
reason = 'form_end' if i == len(groups) - 1 else 'section_end' if groups[i + 1][0][0].get('section_label') else None
if reason is None and max((min(b, length) for a, b in melody[index]), default=fractions.Fraction()) <= length - 1:
reason = 'melody_breath'
if reason is None:
continue
middle = generator.model.fraction(group[0][0]['duration'])
if any(e.start < middle < e.end for e in current.events):
continue
for variation in policy['patterns']:
if variation['family'] != base['family'] or variation['lengths'] != base['lengths'] or variation.get('meter') != base.get('meter') or variation.get('grouping') != base.get('grouping'):
continue
hybrid = {**base, 'id': base['id'] + '.fill.' + variation['id'], 'events':
[r for r in base['events'] if fractions.Fraction(r['onset']) < middle] +
[r for r in variation['events'] if fractions.Fraction(r['onset']) >= middle]}
candidate = realize(hybrid, group, part, melody, anticipate=anticipate)
if candidate is None or candidate.cost >= current.cost - 1e-9:
continue
if tuple(e for e in candidate.events if e.start < middle) != tuple(e for e in current.events if e.start < middle):
continue
gain = current.cost - candidate.cost
options.append((-gain, generator.solo.policy.stable_rank(seed, (index, hybrid['id'])), i, index, hybrid, anticipate, candidate, reason))
chosen = []
for _, _, i, index, pattern, anticipate, candidate, reason in sorted(options):
if any(min(abs(index - other), body_length - abs(index - other)) < spacing for other in chosen):
continue
selected[i] = pattern, anticipate, candidate
locations[i]['fill'] = {'pattern': pattern['id'], 'source_index': index, 'reason': reason}
chosen.append(index)
+14 -7
View File
@@ -98,11 +98,15 @@ def windows(document, part):
result.append((cursor, cursor + generator.arrange.ending.duration(source), part + ('_accompaniment' if part != 'vocal' else '_rest')))
return tuple(result)
def dependencies(role, resources):
def dependencies(role, resources, measures=None):
names = ['harmony', 'lead_ins'] if role.endswith('_solo') else []
accompaniment = {}
if role.endswith('_accompaniment'):
policy = generator.vocabulary.accompaniment(role.split('_')[0], resources)
accompaniment = {'accompaniment': generator.comping_rhythm.relevant_policy(policy, measures) if measures is not None else policy}
return {'roles': {'role': resources['roles']['roles'][role], 'difficulty': resources['roles']['difficulty']},
**({'voicing_families': generator.vocabulary.accompaniment('electric', resources)['voicing_families']} if role == 'electric_melody' else {}),
**({'accompaniment': generator.vocabulary.accompaniment(role.split('_')[0], resources)} if role.endswith('_accompaniment') else {}),
**accompaniment,
**{name: resources[name] for name in names}}
def _code_key(role):
@@ -193,7 +197,7 @@ def _generate_role(source, role, data_dir, settings=None, cache_dir=None):
registered = generator.solo.authored.lookup(source, role.removesuffix('_solo'), data_dir) if role.endswith('_solo') and not settings.get('predict', False) else None
algorithm = VERSION + '/' + generator.solo.learned.VERSION if role.endswith('_solo') else VERSION
inputs = {'version': algorithm, 'source': source.musical_key, 'role': role, 'settings': settings,
'vocabulary': dependencies(role, resources), 'catalogs': ({'authored': registered['sha256'], 'id': registered['id'], 'pitch_convention': registered['pitch_convention']} if registered else _resource_key(data_dir, role)), 'code': _code_key(role)}
'vocabulary': dependencies(role, resources, generator.score.source_staff(source.document())['measures']), 'catalogs': ({'authored': registered['sha256'], 'id': registered['id'], 'pitch_convention': registered['pitch_convention']} if registered else _resource_key(data_dir, role)), 'code': _code_key(role)}
# Source notation/lyrics also invalidate cached score representation, never the musical seed.
key = generator.vocabulary.fingerprint({'inputs': inputs, 'notation': source.document_json})
path = pathlib.Path(cache_dir) / role / (key + '.json') if cache_dir else None
@@ -219,6 +223,8 @@ def _generate_role(source, role, data_dir, settings=None, cache_dir=None):
# rhythm patterns. Removing this new field preserves the pre-policy seed.
choice_data = copy.deepcopy(choice_data)
choice_data['roles']['role'].pop('voicing', None)
for field in ('provenance', 'license', 'research'):
choice_data['accompaniment'].pop(field, None)
choice_settings = {k: v for k, v in settings.items() if k != 'voicing'}
seed = generator.vocabulary.fingerprint({'source': choice_source, 'role': role, 'settings': choice_settings, 'data': choice_data, 'version': algorithm})
details = {}
@@ -258,7 +264,7 @@ def _generate_role(source, role, data_dir, settings=None, cache_dir=None):
elif role.endswith('_accompaniment'):
canonical = generator.comping.Canonical(document, part, seed)
result = generator.arrange.generate._arrange_document(document, part, {}, owned={'accompaniment'}, accompaniment=canonical)
details = {'canonical_comping': canonical.report, 'accompaniment_rhythm': canonical.rhythm_report, 'rhythm_source': choice_source}
details = {'canonical_comping': canonical.report, 'accompaniment_rhythm': canonical.rhythm_report, 'groove_cycles': canonical.groove_report, 'rhythm_source': choice_source}
else:
result = generator.arrange.generate._arrange_document(document, part, {}, owned={'vocal' if part == 'vocal' else 'melody'})
allowed = tuple((a, b) for a, b, r in windows(document, part) if r == role)
@@ -367,7 +373,8 @@ def generate_tree(source_dir, output_dir, *, jobs=2, cache_dir=None, progress=Fa
raise ValueError(f'{path.name}/{role}: {error}') from error
for path, (original, source) in sources.items():
assembled = assemble(source, results[path])
reuse = generator.arrange.accompaniment.audit(original, assembled)
generator.arrange.accompaniment.audit(original, assembled)
reuse = generator.arrange.accompaniment.audit(original, {role: value['document'] for role, value in results[path].items()}, role_documents=True)
for part, document in assembled.items():
document['metadata'] = copy.deepcopy(original['metadata'])
document['metadata']['song_id'] = generator.score.slugify(path.stem)
@@ -377,7 +384,7 @@ def generate_tree(source_dir, output_dir, *, jobs=2, cache_dir=None, progress=Fa
(analysis_dir / (generator.score.slugify(path.stem) + '.txt')).write_text('\n'.join(results[path][r]['report'] for r in ROLES))
# Not a score file: diagnostics are deliberately outside the songs schema.
stable = {role: {k: v for k, v in value.items() if k not in ('seconds', 'composition_timings', 'cache_hit', 'observed_reads')} for role, value in results[path].items()}
for part, audit in reuse.items():
stable[part + '_accompaniment']['details']['reuse_audit'] = audit
for role, audit in reuse.items():
stable[role]['details']['reuse_audit'] = audit
(analysis_dir / (generator.score.slugify(path.stem) + '.roles')).write_text(json.dumps(stable, sort_keys=True, indent=2))
return len(sources) * 3
+24 -8
View File
@@ -23,14 +23,17 @@ def _load(contents):
result = {}
for name, text in contents:
value = json.loads(text)
if value.get('version') != (4 if name == 'accompaniment' else 1) or not value.get('provenance') or not value.get('license'):
if value.get('version') != (6 if name == 'accompaniment' else 1) or not value.get('provenance') or not value.get('license'):
raise ValueError(f'{name}: unsupported or unprovenanced production vocabulary')
if name == 'accompaniment':
signatures = {}
for part in ('electric', 'acoustic'):
policy = value['instruments'][part]
if part == 'acoustic' and policy.get('first_attack') != 'bass_or_combined':
raise ValueError('acoustic patterns require a bass-or-combined first attack policy')
if part == 'acoustic' and policy.get('first_attack') != 'bass_per_harmony':
raise ValueError('acoustic patterns require bass at the first attack of each harmony')
groove = policy['groove']
if any(type(groove[k]) is not int for k in ('cycle_patterns', 'slot_candidates', 'fill_spacing_measures')) or not 1 <= groove['cycle_patterns'] <= 4 or not 4 <= groove['slot_candidates'] <= 8 or groove['fill_spacing_measures'] < 8:
raise ValueError('invalid bounded accompaniment groove policy')
identifiers = set()
signatures[part] = set()
for pattern in policy['patterns']:
@@ -40,8 +43,17 @@ def _load(contents):
lengths = tuple(map(fractions.Fraction, pattern['lengths']))
if not 1 <= len(lengths) <= 2 or any(n <= 0 for n in lengths):
raise ValueError('patterns require one or two positive measure lengths')
meter = pattern.get('meter')
grouping = tuple(pattern.get('grouping', ()))
if meter == 'partial':
if len(lengths) != 1 or grouping:
raise ValueError('partial patterns require one measure and no grouping')
elif meter not in ('3/4', '4/4', '5/4') or lengths != (fractions.Fraction(meter[0]),) * 2:
raise ValueError('full patterns require an explicit supported meter and two complete bars')
if (meter == '5/4' and grouping not in ((3, 2), (2, 3))) or (meter != '5/4' and grouping):
raise ValueError('5/4 patterns require a 3+2 or 2+3 grouping')
# Validate musical events before any measure projection or ties.
previous = fractions.Fraction()
previous, releases = fractions.Fraction(), {}
onsets = []
for row in pattern['events']:
onset, duration = fractions.Fraction(row['onset']), fractions.Fraction(row['duration'])
@@ -51,18 +63,22 @@ def _load(contents):
raise ValueError('comping vocabulary uses a quarter/eighth grid')
if part == 'acoustic':
if row['layer'] == 'combined':
if duration <= 1:
raise ValueError('combined acoustic sustain must exceed one beat')
if duration < 1 and duration != fractions.Fraction(1, 2):
raise ValueError('combined acoustic attacks require quarter/eighth notes or sustains')
elif row['layer'] not in ('bass', 'upper') or duration not in (fractions.Fraction(1, 2), 1):
raise ValueError('acoustic short attacks require bass/upper quarter/eighth notes')
elif row['layer'] != 'electric':
raise ValueError('electric pattern must contain electric chord attacks')
if row.get('anticipate') and row['layer'] not in ('upper', 'electric'):
raise ValueError('only upper/electric chords can anticipate')
previous = onset + duration
lanes = ('bass', 'upper') if row['layer'] == 'combined' else (row['layer'],)
if any(releases.get(lane, fractions.Fraction()) > onset for lane in lanes):
raise ValueError('overlapping accompaniment attacks on the same strings')
releases.update({lane: onset + duration for lane in lanes})
previous = onset
onsets.append(onset)
if len(lengths) == 2:
signatures[part].add((lengths, tuple(onsets)))
signatures[part].add((meter, grouping, tuple(onsets)))
families = policy['voicing_families']
if not families.get('provenance'):
raise ValueError('guitar voicing families require provenance')
+221
View File
@@ -0,0 +1,221 @@
"""Recurring source-position grooves, fills and chord-first bass behavior."""
import concurrent.futures
import copy
import dataclasses
import fractions
import multiprocessing
import pathlib
import tempfile
import unittest
import unittest.mock
import generator.accompaniment
import generator.acoustic
import generator.arrange.accompaniment
import generator.arrange.form
import generator.comping
import generator.comping_rhythm
import generator.fretboard
import generator.harmony
import generator.model
import generator.production
import generator.score
import generator.vocabulary
import tests.solo_fixtures
def groups(source):
measures = source['staves'][0]['measures']
carried = generator.accompaniment.carried_chords(measures)
return [tuple((m, carried[i], i) for i, m in enumerate(measures[a:a + 2], a)) for a in range(0, len(measures), 2)]
def signature(measure):
return [(generator.model.fraction(n['onset']), generator.model.fraction(n['duration']),
[(generator.model.pitch_number(p), p.get('tablature'), bool(p.get('tie_from')), bool(p.get('tie_to'))) for p in n['pitches']])
for n in generator.score.measure_notes(measure)]
class RecurringGrooves(unittest.TestCase):
def test_cycles_repeat_distinct_patterns_for_arbitrary_lengths(self):
for count in (9, 18, 32):
source = tests.solo_fixtures.song(count=count, chords=('C7',))
for part, width in (('acoustic', 2), ('electric', 4)):
canonical = generator.comping.Canonical(source, part, 'cycle')
self.assertEqual(canonical.groove_report[0]['cycle_patterns'], width)
patterns = [p for p, _ in canonical.patterns.values()]
identities = [generator.comping_rhythm.identity(p) for p in patterns[:width]]
self.assertEqual(len(set(identities)), width)
for i, p in enumerate(patterns):
self.assertEqual(generator.comping_rhythm.identity(p), identities[i % width])
# The odd final measure is the first half of its cycle's figure.
if count % 2:
self.assertEqual(canonical.block_rhythms[count - 1].events,
tuple(e for e in canonical.block_rhythms[(len(patterns) - 1) % width * 2].events if e.start < 4))
def test_whole_form_evaluation_can_reject_locally_best_pattern(self):
source = tests.solo_fixtures.song(count=8, chords=('C7',))
resources = copy.deepcopy(generator.vocabulary.load())
policy = resources['accompaniment']['instruments']['electric']
policy['groove']['cycle_patterns'] = 1
policy['patterns'] = [p for p in policy['patterns'] if p['id'] in ('electric.charleston.statement', 'electric.quarters.statement')]
realize = generator.comping_rhythm.realize
def evaluated(pattern, inputs, *args, **kwargs):
candidate = realize(pattern, inputs, *args, **kwargs)
# Charleston wins the first two bars, but loses the full eight-bar span.
cost = (-1 if inputs[0][2] == 0 else 2) if 'charleston' in pattern['id'] else 0
return dataclasses.replace(candidate, contributions=(('test_context', cost),))
with generator.vocabulary.using(resources, 'electric_accompaniment'), unittest.mock.patch('generator.comping_rhythm.realize', side_effect=evaluated):
selected, _, report = generator.comping_rhythm.choose_cycles(groups(source), 'electric', generator.comping_rhythm.melody_contexts(source), 'coverage')
self.assertTrue(all(p['id'] == 'electric.quarters.statement' for p, _, _ in selected))
self.assertEqual(report[0]['cost'], 0)
def test_melody_and_progression_influence_selection_with_fixed_seed(self):
source = tests.solo_fixtures.song(count=16, chords=('C7', 'FM7'))
sparse = copy.deepcopy(source)
for m in sparse['staves'][0]['measures']:
m['voices'][0]['notes'] = [generator.model.make_event(m['id'] + '_held', fractions.Fraction(), fractions.Fraction(1, 2), 'note', [{'letter': 'c', 'alter': 0, 'octave': 4}])]
choices = [generator.comping_rhythm.choose_cycles(groups(s), 'electric', generator.comping_rhythm.melody_contexts(s), 'fixed')[2] for s in (source, sparse)]
self.assertNotEqual(choices[0][0]['patterns'], choices[1][0]['patterns'])
changed = copy.deepcopy(source)
for m in changed['staves'][0]['measures']:
m['harmonies'].append({**generator.harmony.parse_symbol('G7'), 'id': 'change', 'onset': generator.model.rational(fractions.Fraction(3, 2))})
other = generator.comping_rhythm.choose_cycles(groups(changed), 'electric', generator.comping_rhythm.melody_contexts(changed), 'fixed')[2]
self.assertNotEqual(choices[0][0]['cost'], other[0]['cost'])
def test_sparse_fill_replaces_only_second_bar_and_does_not_change_phase(self):
source = tests.solo_fixtures.song(count=32, chords=('C7',))
resources = copy.deepcopy(generator.vocabulary.load())
policy = resources['accompaniment']['instruments']['electric']
policy['groove']['cycle_patterns'] = 1
policy['patterns'] = [p for p in policy['patterns'] if p['id'] in ('electric.charleston.statement', 'electric.charleston.answer')]
# Section-ending evidence is source data. Deliberately put it at nearby
# positions to exercise spacing, including the cyclic end/start join.
for i in (2, 4, 12, 20, 30):
source['staves'][0]['measures'][i]['section_label'] = 'Source section'
realize = generator.comping_rhythm.realize
def evaluated(pattern, inputs, *args, **kwargs):
candidate = realize(pattern, inputs, *args, **kwargs)
cost = -1 if '.fill.' in pattern['id'] and pattern['id'].endswith('.answer') else 0 if pattern['id'].endswith('.statement') else 1
return dataclasses.replace(candidate, contributions=(('test_context', cost),))
with generator.vocabulary.using(resources, 'electric_accompaniment'), unittest.mock.patch('generator.comping_rhythm.realize', side_effect=evaluated):
selected, locations, _ = generator.comping_rhythm.choose_cycles(groups(source), 'electric', generator.comping_rhythm.melody_contexts(source), 'fills')
fills = [v['fill']['source_index'] for v in locations if v['fill']]
self.assertGreater(len(fills), 0)
self.assertLessEqual(len(fills), 4)
for i, a in enumerate(fills):
for b in fills[i + 1:]:
self.assertGreaterEqual(min(abs(a - b), 32 - abs(a - b)), 8)
original = policy['patterns'][0]
for pattern, _, _ in selected:
self.assertEqual([r for r in pattern['events'] if fractions.Fraction(r['onset']) < 4], [r for r in original['events'] if fractions.Fraction(r['onset']) < 4])
self.assertTrue(all(v['cycle_slot'] == 0 for v in locations))
def test_real_source_fit_can_select_a_fill_without_mocked_scores(self):
source = tests.solo_fixtures.song(count=16, chords=('C7',))
resources = copy.deepcopy(generator.vocabulary.load())
policy = resources['accompaniment']['instruments']['electric']
policy['groove']['cycle_patterns'] = 1
policy['patterns'] = [{'id': 'test.' + name, 'family': 'call_response', 'lengths': ['4', '4'], 'events': [
{'onset': '1/2', 'duration': '1/2', 'layer': 'electric'}, {'onset': onset, 'duration': '1/2', 'layer': 'electric'}]}
for name, onset in (('base', '11/2'), ('answer', '5'))]
# Most of the form favors the offbeat answer. At the final boundary the
# source enters on beat two after a breath, so the related on-beat answer
# supports that entrance. All contributions come from the real evaluator.
source['staves'][0]['measures'][-1]['voices'][0]['notes'].pop(0)
with generator.vocabulary.using(resources, 'electric_accompaniment'):
_, locations, reports = generator.comping_rhythm.choose_cycles(groups(source), 'electric', generator.comping_rhythm.melody_contexts(source), 'real-fill')
self.assertEqual(reports[0]['patterns'], ['test.base'])
self.assertEqual([r['fill']['source_index'] for r in locations if r['fill']], [15])
def test_meter_change_restarts_compatible_cycle_and_reports_reduction(self):
source = tests.solo_fixtures.song(count=8, chords=('C7',))
shorter = tests.solo_fixtures.song(meter=3, count=4, chords=('C7',))
source['staves'][0]['measures'][4:] = shorter['staves'][0]['measures']
canonical = generator.comping.Canonical(source, 'electric', 'meter')
self.assertEqual([r['meter'] for r in canonical.groove_report], ['4/4', '3/4'])
self.assertTrue(all(r['reduction'] for r in canonical.groove_report))
self.assertEqual(canonical.locations[4]['cycle_slot'], 0)
def test_partial_entry_uses_second_half_instead_of_restarting(self):
source = tests.solo_fixtures.song(count=10, chords=('C7',))
canonical = generator.comping.Canonical(source, 'electric', 'entry')
measures = source['staves'][0]['measures']
full = canonical(measures[3], measures[2]['harmonies'][0], 3)
sliced = canonical(measures[3], measures[2]['harmonies'][0], 3, bounds=(3, 10))
# The entry may attack a formerly carried chord; actual relative times
# and fret positions remain from the same second half of the figure.
def music(m):
return [(generator.model.fraction(n['onset']), generator.model.fraction(n['duration']), [(generator.model.pitch_number(p), p['tablature']) for p in n['pitches']]) for n in generator.score.measure_notes(m)]
self.assertEqual(music(full), music(sliced))
def test_final_harmony_change_keeps_unaffected_rhythm_and_grips(self):
source = tests.solo_fixtures.song(count=8, chords=('C7',))
canonical = generator.comping.Canonical(source, 'acoustic', 'final')
original = source['staves'][0]['measures']
changed = copy.deepcopy(original)
changed[-1]['harmonies'] = [{**generator.harmony.parse_symbol('FM7'), 'id': 'final', 'onset': generator.model.rational(0)}]
before = canonical(original[6], original[5]['harmonies'][0], 6)
after = canonical(changed[6], changed[5]['harmonies'][0], 6, sources=changed)
self.assertEqual(signature(before), signature(after))
self.assertEqual(canonical.rhythm_report[-1]['base_pattern'], canonical.locations[6]['base_pattern'])
def test_overlap_survives_single_voice_merge_and_tablature(self):
source = tests.solo_fixtures.song(count=2, chords=('C7',))
resources = copy.deepcopy(generator.vocabulary.load())
policy = resources['accompaniment']['instruments']['acoustic']
policy['patterns'] = [p for p in policy['patterns'] if p['id'] == 'acoustic.quarters.overlapping']
with generator.vocabulary.using(resources, 'acoustic_accompaniment'):
canonical = generator.comping.Canonical(source, 'acoustic', 'overlap')
source['staves'][0]['measures'] = [canonical(m, None, i) for i, m in enumerate(source['staves'][0]['measures'])]
source['staves'][0].update(arrangement_part_type='acoustic', tablature=True)
generator.acoustic.validate_comping(source)
generator.acoustic.combine(source)
generator.model.validate_document(source)
generator.fretboard.validate_document(source)
generator.acoustic.validate_comping(source)
first = source['staves'][0]['measures'][0]
self.assertEqual(len(first['voices']), 1)
overlap = next(n for n in generator.score.measure_notes(first) if generator.model.fraction(n['onset']) == fractions.Fraction(1, 2))
self.assertEqual(len(overlap['pitches']), 4)
self.assertTrue(any(p.get('tie_from') and p['tablature']['string'] >= 5 for p in overlap['pitches']))
def test_mid_measure_chord_change_bass_without_forced_barline_attack(self):
source = tests.solo_fixtures.song(count=2, chords=('C7',))
m = source['staves'][0]['measures'][0]
m['harmonies'].append({**generator.harmony.parse_symbol('FM7'), 'id': 'middle', 'onset': generator.model.rational(2)})
pattern = {'id': 'test', 'family': 'quarters', 'lengths': ['4'], 'events': [
{'onset': '1/2', 'duration': '1', 'layer': 'upper'}, {'onset': '5/2', 'duration': '1/2', 'layer': 'upper'}]}
events, _ = generator.comping_rhythm.fit(pattern, m, None, 'acoustic')
self.assertEqual([(e.start, e.lane) for e in events], [(fractions.Fraction(1, 2), 'combined'), (fractions.Fraction(5, 2), 'combined')])
# A computational boundary inside continuing harmony must not add bass.
inputs = ((source['staves'][0]['measures'][1], source['staves'][0]['measures'][1]['harmonies'][0], 1),)
r = generator.comping_rhythm.realize(pattern, inputs, 'acoustic', generator.comping_rhythm.melody_contexts(source))
self.assertTrue(all(e.lane == 'upper' for e in r.events))
def test_clean_cached_parallel_and_other_role_settings_keep_groove(self):
source = generator.production.SourceAnalysis.read(tests.solo_fixtures.song(count=10, chords=('C7', 'FM7')))
root = pathlib.Path('data').resolve()
roles = ('acoustic_accompaniment', 'electric_accompaniment')
clean = {r: generator.production.generate_role(source, r, root) for r in roles}
with tempfile.TemporaryDirectory() as directory:
for r in roles:
cold = generator.production.generate_role(source, r, root, cache_dir=directory)
warm = generator.production.generate_role(source, r, root, cache_dir=directory)
self.assertTrue(warm['cache_hit'])
self.assertEqual(clean[r]['document'], cold['document'])
self.assertEqual(clean[r]['document'], warm['document'])
generator.production.generate_role(source, roles[1], root, {'seed': 'different'}, directory)
self.assertEqual(clean[roles[0]]['document'], generator.production.generate_role(source, roles[0], root, cache_dir=directory)['document'])
with concurrent.futures.ProcessPoolExecutor(max_workers=2, mp_context=multiprocessing.get_context('spawn')) as pool:
futures = {r: pool.submit(generator.production._job, (source, r, root, {}, None)) for r in reversed(roles)}
for r, future in futures.items():
self.assertEqual(clean[r]['document'], future.result()['document'])
def test_reuse_audit_checks_rests_and_all_six_roles(self):
source = tests.solo_fixtures.song(count=2, chords=('C7',))
for role in generator.production.ROLES:
part = role.split('_')[0]
wanted = 'slash' if role.endswith('_solo') else 'vocal' if part == 'vocal' else 'melody' if role.endswith('_melody') else 'accompaniment'
section = generator.arrange.form.ArrangementSection('Repeated role', 0, 2, 1, wanted)
doc = copy.deepcopy(source)
doc['staves'][0]['measures'] *= 2
doc = copy.deepcopy(doc)
with unittest.mock.patch('generator.arrange.form.full_arrangement_sections', return_value=[section, section]):
result = generator.arrange.accompaniment.audit(source, {role: doc}, role_documents=True)
self.assertEqual(result[role]['reused_bars'], 2)
# Remove music only in the second occurrence; reuse must fail.
doc['staves'][0]['measures'][2] = copy.deepcopy(doc['staves'][0]['measures'][2])
n = doc['staves'][0]['measures'][2]['voices'][0]['notes'][0]
n.update(kind='rest', pitches=[])
with self.assertRaisesRegex(ValueError, 'canonical'):
generator.arrange.accompaniment.audit(source, {role: doc}, role_documents=True)
if __name__ == '__main__':
unittest.main()
+207
View File
@@ -0,0 +1,207 @@
"""Meter-specific vocabulary and source-derived quintuple grouping."""
import concurrent.futures
import copy
import fractions
import json
import multiprocessing
import pathlib
import tempfile
import unittest
import unittest.mock
import generator.accompaniment
import generator.comping
import generator.comping_rhythm
import generator.model
import generator.production
import generator.vocabulary
import tests.solo_fixtures
import tests.test_comping_cycles
F = fractions.Fraction
class CompingMeters(unittest.TestCase):
def source(self, beats=5, count=10, explicit=()):
source = tests.solo_fixtures.song(meter=beats, count=count, chords=('C7', 'FM7'))
if explicit:
for m in source['staves'][0]['measures']:
m['meter']['groups'][0]['beats'] = list(explicit)
m['time_signature'] = generator.model.meter_label(m['meter'])
source['metadata']['time_signature'] = source['staves'][0]['measures'][0]['time_signature']
return source
def test_balanced_pools_and_all_families_reach_joint_evaluation(self):
for beats, grouping in ((4, ()), (3, ()), (5, (3, 2)), (5, (2, 3))):
source = self.source(beats, 2, grouping)
inputs = tests.test_comping_cycles.groups(source)[0]
flat = generator.comping_rhythm.flatten(inputs)
for role in ('acoustic', 'electric'):
with self.subTest(beats=beats, grouping=grouping, role=role):
policy = generator.vocabulary.accompaniment(role)
pool = generator.comping_rhythm.patterns_for(policy, flat, (F(beats),) * 2, grouping)
self.assertEqual(len(pool), 24)
self.assertEqual(len({generator.comping_rhythm.identity(p) for p in pool}), 24)
self.assertEqual(len({p['family'] for p in pool}), 11 if beats == 4 else 12)
candidates = generator.comping_rhythm.candidates(flat, None, role, (), 'coverage', lengths=(F(beats),) * 2, grouping=grouping)
self.assertEqual({p['family'] for p in pool}, {dict(c.families)['pattern'] for c in candidates})
for p in pool:
fitted = generator.comping_rhythm.realize(p, inputs, role, generator.comping_rhythm.melody_contexts(source))
self.assertIsNotNone(fitted, p['id'])
generator.comping_rhythm.validate_events(fitted.events, role)
for start in (0, beats):
self.assertLessEqual(len({e.start for e in fitted.events if start <= e.start < start + beats}), 3 if role == 'acoustic' else 2)
if role == 'electric':
self.assertTrue(any(F(e['onset']) < beats < F(e['onset']) + F(e['duration']) for p in pool for e in p['events']))
def test_explicit_grouping_overrides_harmony_and_melody(self):
source = self.source(explicit=(2, 3))
for m in source['staves'][0]['measures']:
m['harmonies'].append({**generator.harmony.parse_symbol('G7'), 'id': 'secondary', 'onset': generator.model.rational(3)})
grouping, evidence = generator.comping_rhythm.grouping_for(tests.test_comping_cycles.groups(source), generator.comping_rhythm.melody_contexts(source))
self.assertEqual(grouping, (2, 3))
self.assertEqual(evidence['reason'], 'written')
def test_harmonic_boundary_then_melodic_entrances_then_attacks(self):
for boundary in (2, 3):
source = self.source()
for m in source['staves'][0]['measures']:
m['harmonies'].append({**generator.harmony.parse_symbol('G7'), 'id': 'secondary', 'onset': generator.model.rational(boundary)})
grouping, _ = generator.comping_rhythm.grouping_for(tests.test_comping_cycles.groups(source), generator.comping_rhythm.melody_contexts(source))
self.assertEqual(grouping, (boundary, 5 - boundary))
source = self.source()
# Competing secondary pulses have equal attack counts. Only one follows
# an actual rest; an uninterrupted reattack is weaker grouping evidence.
other = 5 - boundary
contexts = tuple(((F(other) - F(1, 2), F(other)), (F(other), F(other) + F(1, 2)), (F(boundary), F(boundary) + F(1, 2))) for _ in range(10))
grouping, _ = generator.comping_rhythm.grouping_for(tests.test_comping_cycles.groups(source), contexts)
self.assertEqual(grouping, (boundary, 5 - boundary))
contexts = tuple(((F(boundary), F(boundary + 1)),) for _ in range(10))
grouping, _ = generator.comping_rhythm.grouping_for(tests.test_comping_cycles.groups(source), contexts)
self.assertEqual(grouping, (boundary, 5 - boundary))
def test_same_chord_marking_is_not_a_harmonic_change(self):
source = self.source()
for m in source['staves'][0]['measures']:
m['harmonies'].append({**m['harmonies'][0], 'onset': generator.model.rational(2)})
grouping, evidence = generator.comping_rhythm.grouping_for(tests.test_comping_cycles.groups(source), generator.comping_rhythm.melody_contexts(source))
self.assertEqual(grouping, (3, 2))
self.assertEqual(evidence['reason'], 'default_3+2')
def test_not_just_equal_measure_length(self):
source = self.source(3)
m = source['staves'][0]['measures'][0]
m.update(meter={'groups': [{'beats': [6], 'unit': 8}]}, time_signature='6/8')
with self.assertRaisesRegex(ValueError, 'no comping vocabulary for 6/8'):
generator.comping.Canonical(source, 'electric', 'compound')
def test_schema_rejects_wrong_meter_lengths_and_missing_grouping(self):
root = pathlib.Path('data/composition')
contents = {p.stem: p.read_text() for p in root.glob('*.json')}
for change in ({'meter': '3/4'}, {'grouping': []}, {'grouping': [4, 1]}):
value = json.loads(contents['accompaniment'])
pattern = next(p for p in value['instruments']['electric']['patterns'] if p['meter'] == '5/4')
pattern.update(change)
with self.assertRaises(ValueError):
generator.vocabulary._load(tuple((name, json.dumps(value) if name == 'accompaniment' else text) for name, text in contents.items()))
def test_full_cycles_odd_forms_and_grouping_consistency(self):
for beats, grouping in ((4, ()), (3, ()), (5, (3, 2)), (5, (2, 3))):
source = self.source(beats, 9, grouping)
reports = []
for role, width in (('acoustic', 2), ('electric', 4)):
canonical = generator.comping.Canonical(source, role, 'cycles')
report = canonical.groove_report[0]
reports.append(report)
self.assertEqual(report['cycle_patterns'], width)
self.assertEqual(report['eligible_patterns'], 24)
self.assertIsNone(report['reduction'])
for location in canonical.locations.values():
slot = location['cycle_slot']
self.assertEqual(location['base_pattern'], report['patterns'][slot])
last = canonical.block_rhythms[8]
self.assertTrue(all(0 <= e.start < e.end <= beats for e in last.events))
self.assertEqual(reports[0]['grouping'], reports[1]['grouping'])
def test_grouping_changes_restart_and_fills_cannot_cross_groups(self):
source = self.source(count=18, explicit=(3, 2))
for m in source['staves'][0]['measures'][9:]:
m['meter']['groups'][0]['beats'] = [2, 3]
m['time_signature'] = '2+3/4'
for role in ('acoustic', 'electric'):
canonical = generator.comping.Canonical(source, role, 'change')
self.assertEqual([r['grouping'] for r in canonical.groove_report], [[3, 2], [2, 3]])
self.assertEqual(canonical.locations[9]['cycle_slot'], 0)
for pattern, _ in canonical.patterns.values():
suffix = '.' + '+'.join(map(str, pattern['grouping'])) + '.'
if '.fill.' in pattern['id']:
self.assertTrue(all(suffix in p for p in pattern['id'].split('.fill.')))
def test_every_fill_candidate_uses_its_base_grouping(self):
source = self.source(count=16, explicit=(2, 3))
checked = []
realize = generator.comping_rhythm.realize
def inspect(pattern, *args, **kwargs):
if '.fill.' in pattern['id']:
checked.append(pattern['id'])
self.assertTrue(all('.2+3.' in p for p in pattern['id'].split('.fill.')))
return realize(pattern, *args, **kwargs)
with unittest.mock.patch.object(generator.comping_rhythm, 'realize', side_effect=inspect):
generator.comping.Canonical(source, 'electric', 'fill-group')
self.assertTrue(checked)
def test_partial_pickup_and_mid_pattern_entry_keep_meter_and_phase(self):
for beats in (3, 5):
source = self.source(beats)
m = copy.deepcopy(source['staves'][0]['measures'][0])
m['duration'] = generator.model.rational(F(3, 2))
candidates = generator.comping_rhythm.candidates(m, None, 'electric', (), 'pickup')
self.assertTrue(candidates)
self.assertTrue(all(0 <= e.start < e.end <= F(3, 2) for c in candidates for e in c.events))
for role in ('acoustic', 'electric'):
canonical = generator.comping.Canonical(source, role, 'entry')
measures = source['staves'][0]['measures']
carried = generator.accompaniment.carried_chords(measures)
full = canonical(measures[3], carried[3], 3)
partial = canonical(measures[3], carried[3], 3, bounds=(3, 10))
# Rest engraving can split differently at a sliced boundary. Sounded
# onsets, releases, voicings and tie relationships must remain equal.
self.assertEqual([n for n in tests.test_comping_cycles.signature(full) if n[-1]], [n for n in tests.test_comping_cycles.signature(partial) if n[-1]])
def test_unused_meter_and_other_role_data_do_not_reseed(self):
resources = generator.vocabulary.load()
measures = self.source(4)['staves'][0]['measures']
baseline = generator.production.dependencies('acoustic_accompaniment', resources, measures)
changed = copy.deepcopy(resources)
for role, policy in changed['accompaniment']['instruments'].items():
for p in policy['patterns']:
if role == 'electric' or p['meter'] in ('3/4', '5/4'):
p['events'] = []
self.assertEqual(baseline, generator.production.dependencies('acoustic_accompaniment', changed, measures))
changed['accompaniment']['instruments']['acoustic']['patterns'][0]['events'] = []
self.assertNotEqual(baseline, generator.production.dependencies('acoustic_accompaniment', changed, measures))
def test_chord_changes_anticipations_and_cross_bar_ties(self):
for beats, grouping in ((4, ()), (3, ()), (5, (3, 2)), (5, (2, 3))):
source = self.source(beats, 2, grouping)
for m in source['staves'][0]['measures']:
m['harmonies'].append({**generator.harmony.parse_symbol('G7'), 'id': 'middle', 'onset': generator.model.rational(beats - 1)})
inputs = tests.test_comping_cycles.groups(source)[0]
for role in ('acoustic', 'electric'):
pool = generator.comping_rhythm.patterns_for(generator.vocabulary.accompaniment(role), inputs[0][0], (F(beats),) * 2, grouping)
anticipated = False
for p in pool:
result = generator.comping_rhythm.realize(p, inputs, role, generator.comping_rhythm.melody_contexts(source), anticipate=True)
self.assertIsNotNone(result, p['id'])
generator.comping_rhythm.validate_events(result.events, role)
for e in result.events:
if e.harmonic_time is not None:
anticipated = True
self.assertEqual(e.harmonic_time - e.start, F(1, 2))
self.assertTrue(anticipated)
def test_roles_are_deterministic_alone_cached_and_parallel(self):
for meter in (3, 5):
source = generator.production.SourceAnalysis.read(self.source(meter, 9))
roles = ('acoustic_accompaniment', 'electric_accompaniment')
with tempfile.TemporaryDirectory() as cache:
clean = {r: generator.production.generate_role(source, r, pathlib.Path('data')) for r in roles}
for role in reversed(roles):
cold = generator.production.generate_role(source, role, pathlib.Path('data'), cache_dir=cache)
warm = generator.production.generate_role(source, role, pathlib.Path('data'), cache_dir=cache)
self.assertEqual(clean[role]['document'], cold['document'])
self.assertEqual(cold['document'], warm['document'])
self.assertTrue(warm['cache_hit'])
with concurrent.futures.ProcessPoolExecutor(max_workers=2, mp_context=multiprocessing.get_context('spawn')) as pool:
jobs = {r: pool.submit(generator.production._job, (source, r, pathlib.Path('data'), {}, None)) for r in reversed(roles)}
for role, job in jobs.items():
self.assertEqual(clean[role]['document'], job.result()['document'])
if __name__ == '__main__':
unittest.main()
+15 -14
View File
@@ -53,15 +53,17 @@ class LayeredComping(unittest.TestCase):
self.assertTrue(any(c.events[0].start > 0 for c in choices['acoustic']))
self.assertTrue(all(c.events[0].lane in ('bass', 'combined') for c in choices['acoustic'] if c.events))
self.assertTrue(any(e.end - e.start > 1 for c in choices['electric'] for e in c.events))
def test_separate_short_attacks_and_only_synchronized_long_combinations(self):
def test_separate_lanes_can_overlap_and_short_combined_attacks_are_valid(self):
F = fractions.Fraction
attack = generator.comping_rhythm.Attack
generator.comping_rhythm.validate_events((attack('bass', F(0), F(1)), attack('upper', F(1), F(3, 2))), 'acoustic')
generator.comping_rhythm.validate_events((attack('combined', F(0), F(3, 2)),), 'acoustic')
for events in ((attack('combined', F(0), F(1)),), (attack('bass', F(0), F(1)), attack('upper', F(1, 2), F(1))), (attack('bass', F(0), F(2)),)):
generator.comping_rhythm.validate_events((attack('combined', F(0), F(1)),), 'acoustic')
generator.comping_rhythm.validate_events((attack('bass', F(0), F(1)), attack('upper', F(1, 2), F(3, 2))), 'acoustic')
for events in ((attack('combined', F(0), F(1)), attack('upper', F(1, 2), F(1))), (attack('bass', F(0), F(1)), attack('bass', F(1, 2), F(1))), (attack('bass', F(0), F(2)),)):
with self.assertRaises(ValueError):
generator.comping_rhythm.validate_events(events, 'acoustic')
def test_bass_first_reassigns_layers_without_moving_or_adding_attacks(self):
def test_bass_first_keeps_pattern_timing_and_does_not_reset_at_barlines(self):
source = tests.solo_fixtures.song(count=2, chords=('C7',))
flat = generator.comping_rhythm.flatten(tuple((m, None, i) for i, m in enumerate(source['staves'][0]['measures'])))
patterns = self.resources['accompaniment']['instruments']['acoustic']['patterns']
@@ -71,10 +73,11 @@ class LayeredComping(unittest.TestCase):
continue
events, _ = generator.comping_rhythm.fit(pattern, flat, None, 'acoustic')
self.assertEqual([(e.start, e.end - e.start) for e in events], [(fractions.Fraction(r['onset']), fractions.Fraction(r['duration'])) for r in pattern['events']])
for start in (0, 4):
local = [e for e in events if start <= e.start < start + 4]
if local:
self.assertIn(local[0].lane, ('bass', 'combined'))
if events:
self.assertIn(events[0].lane, ('bass', 'combined'))
pattern = {'lengths': ['4', '4'], 'events': [{'onset': '0', 'duration': '1', 'layer': 'bass'}, {'onset': '4', 'duration': '1', 'layer': 'upper'}]}
events, _ = generator.comping_rhythm.fit(pattern, flat, None, 'acoustic')
self.assertEqual(events[-1].lane, 'upper')
self.assertEqual(before, patterns)
def test_missing_harmony_reassigns_first_survivor_before_anticipation(self):
m = copy.deepcopy(self.source['staves'][0]['measures'][0])
@@ -86,24 +89,22 @@ class LayeredComping(unittest.TestCase):
{'onset': '3/2', 'duration': '1', 'layer': 'upper', 'anticipate': True},
{'onset': '7/2', 'duration': '1/2', 'layer': 'bass'}]}
events, changes = generator.comping_rhythm.fit(pattern, m, None, 'acoustic', anticipate=True)
self.assertEqual([(e.lane, e.start, e.end, e.harmonic_time) for e in events], [('bass', fractions.Fraction(3, 2), 2, None), ('upper', fractions.Fraction(7, 2), 4, None)])
self.assertEqual([(e.lane, e.start, e.end, e.harmonic_time) for e in events], [('combined', fractions.Fraction(3, 2), fractions.Fraction(5, 2), 2), ('bass', fractions.Fraction(7, 2), 4, None)])
self.assertIn('layer_assignment', {kind for kind, _ in changes})
limited, _ = generator.comping_rhythm.fit(pattern, m, None, 'acoustic', anticipate=True, limit=fractions.Fraction(2))
self.assertEqual(limited, (events[0],))
self.assertEqual([(e.lane, e.start, e.end, e.harmonic_time) for e in limited], [('combined', fractions.Fraction(3, 2), 2, None)])
def test_combined_first_attack_and_carried_hold_do_not_require_reattacks(self):
attack, F = generator.comping_rhythm.Attack, fractions.Fraction
valid = (attack('combined', F(3), F(5)), attack('bass', F(6), F(7)))
generator.comping_rhythm.validate_events(valid, 'acoustic', (4, 4))
generator.comping_rhythm.validate_events(valid[:1], 'acoustic', (4, 4))
invalid = (valid[0], attack('upper', F(6), F(7)))
with self.assertRaisesRegex(ValueError, 'first acoustic attack'):
generator.comping_rhythm.validate_events(invalid, 'acoustic', (4, 4))
generator.comping_rhythm.validate_events((valid[0], attack('upper', F(6), F(7))), 'acoustic', (4, 4))
def test_completed_score_rejects_upper_first_even_after_a_rest(self):
source = tests.solo_fixtures.song(count=2, chords=('C7',))
pattern = {'id': 'test.delayed', 'family': 'call_response', 'lengths': ['4', '4'], 'events': [
{'onset': '1', 'duration': '1/2', 'layer': 'upper'}, {'onset': '2', 'duration': '1', 'layer': 'bass'},
{'onset': '5', 'duration': '1', 'layer': 'combined'}]}
# The final combined event must be longer than one beat.
# A long combined event remains available alongside short overlaps.
pattern['events'][-1]['duration'] = '2'
resources = copy.deepcopy(self.resources)
resources['accompaniment']['instruments']['acoustic']['patterns'] = [pattern]
@@ -135,7 +136,7 @@ class LayeredComping(unittest.TestCase):
for c in anticipated:
for e in c.events:
if e.harmonic_time is not None:
self.assertIn(e.lane, ('upper', 'electric'))
self.assertIn(e.lane, ('bass', 'combined', 'upper', 'electric'))
self.assertEqual(e.harmonic_time - e.start, fractions.Fraction(1, 2))
self.assertLess(e.harmonic_time, e.end)
self.assertTrue(any(all(e.harmonic_time is None for e in c.events) for c in candidates))