diff --git a/README.md b/README.md index 45aab52..0caafc2 100644 --- a/README.md +++ b/README.md @@ -93,19 +93,17 @@ Artifacts are published to the khodo generic package registry's required `latest Uncompressed MusicXML 3.1/4.0 scores exported from Soundslice or MuseScore live in `data/`. Generated score JSON is not committed. -`data/songs` supplies the original melody and harmony. The 21 scale patterns, 2,954 relative interval paths, meter-specific rhythm catalogs, and versioned rules in `data/composition` supply production vocabulary. Shared operators enumerate notes, rests, complete routes and typed harmonic gestures before contextual selection. Production does not read target solos or previous output. Explicit development fitting is separate and cannot automatically publish its coefficients. +`data/songs` supplies the original melody and harmony. The 21 scale patterns, 2,954 relative interval paths, meter-specific rhythm catalogs, and general rules in `data/composition` supply musical vocabulary. Production never reads reference solos, fitted parameters, or previous music. -Six roles generate independently from immutable source analysis: acoustic accompaniment/solo; electric accompaniment/melody/solo; vocal melody. A bounded process pool composes roles in parallel, with fully keyed per-role caches for incremental builds. `--composition-jobs 1` provides the serial equivalent; the default is two. There is no joint acoustic/electric selection, generated-solo-aware backing, or reuse of generated melody grips. Assembly respects source-defined role windows and the returning vocal pickup. +Six roles generate independently: acoustic accompaniment/solo; electric accompaniment/melody/solo; vocal melody. A process pool and per-role caches support parallel and incremental builds. Canonical accompaniment repeats at corresponding source-form positions and does not react to generated solos. Original vocal and electric head content remain unchanged. -Each solo retains an introduced realized theme and uses ordered relationships from the source head. The current selector ranks complete musical actions with bounded backtracking; it is not a whole-phrase global optimizer. Both solos can use all six strings at frets 1–12 without open strings, with overlapping soft registers. Ordinary movement stays mapped; approaches, colors and anticipations require typed timing, harmony and resolution contracts. Pending targets survive internal boundaries. No dynamics, accents or humanization have been added. +The solo composer creates complete source-related melodic phrases before selection. A realized theme develops through returns, answers, register changes, arpeggio motion and complete scale routes. Phrase destinations, rhythm, rests and holds are coordinated. Both guitars use strings 1–6 and frets 1–12 without open strings. Colors, lead-ins, anticipations and suspensions need explicit harmonic evidence. No dynamics, accents or humanization are added. -Each accompaniment is a source-derived canonical part, reused unchanged at corresponding form positions. Sparse shells/dyads and compact voicings retain essential chord information, with deliberate releases and varied rhythms. Neither backing changes in response to generated solos. Source vocals and electric melody retain their established behavior. +`--export-solo-xml` (CMake `VAMP_EXPORT_SOLO_XML=ON`) writes `generated_assets/solo_xml/*-solo1.xml` for acoustic and `*-solo2.xml` for electric, with actual notes and TAB. These exports do not recompose music. `python -m generator.compose --song ... --role acoustic_solo --data-dir data --output ... --solo-xml ...` builds one independent role. -Optional solo exports are available with `-DVAMP_EXPORT_SOLO_XML=ON` in CMake (or `python -m generator --export-solo-xml ...`). They appear in `generated_assets/solo_xml/`: `*-solo1.xml` is acoustic and `*-solo2.xml` is electric. These contain the actual completed production notes, guitar octave transposition, and TAB positions; enabling export does not recompose music. The adjacent manifest records source and algorithm identities. A single-role build can add `--solo-xml build/example-solo1.xml`. +Optional comparison lives in `python -m tests.solo_compare`, outside production. Supply `--generated-dir`, explicit `--reference ROLE=PATH`, `--form-start ROLE=MEASURE`, `--reference-pitch guitar-written`, and `--output build/solo_comparison/review`; add `--assets` and `--baseline-dir` for consistent audio comparisons. References are analysis-only. The supplied targets sound one octave below encoded pitches; All of Me aligns at measure 1 and Autumn Leaves at measure 2, preserving the lead-in. Closeness is reported using actual notes and timing; matching statistics or technical validity do not establish musical preference. -Reference comparison runs separately through `python -m audition.solo_compare`; `--require-exact` returns a failure for any sounding-event or rest difference. For the supplied targets, use `--reference-pitch guitar-written`: they sound one octave below encoded pitches. The separate `audition.fit_selection` command can diagnose target constructibility and fit shared, whitelisted theory coefficients during development. It never stores target phrases in production data. Exact reproduction remains an unmet acceptance condition unless all four exported comparisons pass; constructibility and passing technical tests do not establish it. - -See [Independent Musical Generation](TECHNICAL_GUIDE.md#independent-musical-generation) for the dependency graph, data inventory/provenance, algorithms, constraints, single-role CLI, cache behavior and optional audition/reference commands. The four supplied scores and current production exports are compared in `build/solo_reference_comparison/index.html`; a compact previous-output snapshot is retained there. See [the current repair report](docs/composer/implementation_report.md). Comparison outputs never become production inputs. +See [Independent Musical Generation](TECHNICAL_GUIDE.md#independent-musical-generation) for the active pipeline and limitations. Keep just one compact baseline and the latest comparison under `build/solo_comparison/`. To inspect generated assets without building the full package, run: @@ -113,14 +111,14 @@ To inspect generated assets without building the full package, run: ./modulus.py python -m generator --data-dir data --output-dir /tmp/product_vamp_generated --static-assets-dir package/assets --check ``` -The `--check` path compares generation invariants against `data/expected/performance_summary.json`. +The `--check` path compares generation invariants against `tests/fixtures/performance_summary.json`. Modulus owns the pinned build-only dependencies in its optional `assets` group, selected by `tooling.python.groups` in `modulus.json`. Configure prepares `build/build_common/python`; `./modulus.py python` and CMake use that environment and the exported Modulus modules. Builds never install Python dependencies. After dependency pins change, run configure again. C++-only libraries do not opt in, and Python is not packaged for the Pi. CMake builds engrave with two Python workers and share a disposable page cache under `build/build_common/engraving_cache`. Native and ARM packaging reuse unchanged pages; CI also retains this cache between runs. Cached musical roles are reused independently; music, playback, and catalog validation still run for every generation. Set `VAMP_ENGRAVING_JOBS` (1-16) to adjust concurrency, or set `VAMP_ENGRAVING_CACHE_DIR` to an empty CMake value to disable caching. Direct generator runs can opt in with `--jobs 2 --engraving-cache build/build_common/engraving_cache --progress`. Identical notation bodies share layout and body rasterization even when titles or score identities differ; each score retains its own header, pages, music, and catalog identity. A `cold namespace` means no cache directory exists for the current renderer fingerprint. It does not mean that every score must be rendered independently: repeated bodies can still be reused within that build, including with persistent caching disabled. Progress separates full-score hits, body hits, newly rasterized bodies, and composed pages. -The build-time pipeline is Soundslice/MuseScore -> `.xml` -> musical model and rhythm plans -> part generation -> JSON, engraved PNG pages, generated_tone audio, Pico title artwork, and a read-only catalog database. Sources in `data/songs/` retain the seven-section performance form. Songs generate Acoustic, Electric, and Vocal parts in their authored key, without key variants or a key-selection menu. Measure numbers run continuously across the complete arrangement, starting at 1 in the lead-in; they do not restart with a new section or chorus. Pickups occupy the preceding measure, and exported excerpts retain their arrangement measure labels. +The build-time pipeline is Soundslice/MuseScore -> `.xml` -> musical model and rhythm plans -> part generation -> JSON, engraved PNG pages, generated_tone audio, Pico title artwork, and a read-only catalog database. Sources in `data/songs/` retain the seven-section performance form. Songs generate Acoustic, Electric, and Vocal parts in their authored key, without key variants or a key-selection menu. Performance measure numbers run continuously across the complete arrangement, starting at 1 in the lead-in; they do not restart with a new section or chorus. Pickups occupy the preceding measure. Standalone comparison XMLs use local measure numbers and retain the original arrangement numbers in `manifest.solo`. All generated monophonic guitar material uses frets 1–12 without open strings. Accompaniment uses all legal compact positions; a chord can extend to fret 20 while retaining at least one sounding anchor at fret 12 or below and the existing three-fret span restriction. Electric source-melody harmonization retains its existing placement policy. Exact source pitches and ties are preserved; unplayable or unsupported input fails explicitly rather than being silently octave-folded. diff --git a/TECHNICAL_GUIDE.md b/TECHNICAL_GUIDE.md index 2c84ca8..4e02d45 100644 --- a/TECHNICAL_GUIDE.md +++ b/TECHNICAL_GUIDE.md @@ -184,29 +184,44 @@ The analyzer requires a declared tonic and mode in the song's MusicXML key. A on Selection first preserves every required pitch. Recognized ii-V pairs, resolving dominants, and leading-tone diminished resolutions establish local destinations; the declared key provides the default context elsewhere. Minor cadences prefer harmonic minor and major cadences prefer the destination's major collection when compatible. The arrival chord is evaluated afresh, so a plain D-minor tonic need not retain a preceding cadence's raised seventh. Among equally appropriate choices, a deterministic dynamic program minimizes collection changes, then contextual pitch differences. Equivalent modes share a pitch collection rather than forcing new boundaries. This is a conservative harmonic heuristic, not a claim that ambiguous harmony has only one interpretation. -Only the soloing part receives compact `scale_spans`, with absolute rational start/end offsets, labels, and matched status. `N.C.` and solo boundaries stop annotations. The composer consumes the structured scale choices directly and checks every sounding note across its entire duration. It may select another permitted collection to preserve an exact melody quotation, but never silently changes that quotation's notes. A harmony with no exact compatible collection, invalid scale data, or missing tonal metadata fails generation before publication. The standalone scale-analysis utility can still report unmatched chords without composing them. +Only the soloing part receives compact `scale_spans`, with absolute rational start/end offsets, labels, and matched status. `N.C.` and solo boundaries stop annotations. The composer consumes the structured scale choices directly and checks every sounding note across its entire duration. Source features are adapted within those choices; original source melodies retain their own notes. A harmony with no exact compatible collection, invalid scale data, or missing tonal metadata fails generation before publication. The standalone scale-analysis utility can still report unmatched chords without composing them. Scale labels appear above chord symbols at collection changes and system starts. Detailed parent collections, source and arranged positions, chord-root modes, degrees, chord-tone targets, and selection reasons are written to `generated_assets/analysis/.txt`. These reports are published transactionally with the build output but excluded from both installation and Docker runtime assets. The C++ runtime only displays the generated pages; it does not load the scale library or run harmonic analysis. ### Independent Musical Generation -Production composes six roles independently from the original source and relevant resources in `data/`: acoustic accompaniment/solo, electric accompaniment/melody/solo, and vocal melody. `production.SourceAnalysis` provides immutable source harmony, melody, meter, tempo, form and participation windows. Assembly applies those windows and protects the returning vocal pickup without inspecting generated solos to compose another part. +`production.generate_role` composes each role from immutable original-song content and relevant `data/` resources. Separate role seeds and keyed caches preserve standalone, order-independent and parallel behavior. The ordinary default is two spawned workers (`--composition-jobs`); a single worker has identical musical semantics. Composition reads are audited; comparison files, other roles' concrete outputs and previous scores are prohibited inputs. -`production.generate_role` runs in a bounded spawn process pool, default two workers (`VAMP_COMPOSITION_JOBS` / `--composition-jobs`). Role-specific keyed caches support incremental rebuilds. Serial and parallel modes use the same deterministic musical choices; worker completion order is not an input. Actual composition reads are audited. Optional targets, prior scores and other roles' caches are excluded. +The solo path is `composer.compose → construction.Builder → phrases.Phrases → placement → validate → notation/export`. Source pitch/rhythm relationships seed one realized idea. The composer coordinates whole rhythmic groups, written-chord arpeggios, complete scale routes, deliberate rests and harmonic arrivals into a complete phrase candidate. It considers at most 24 alternatives per incoming history, retaining four histories at common source-derived horizons (up to eight measures). These are bounded search settings, not a library of 24 licks. Boundaries may remain open; source breaths inform them without forcing a cadence at every bar. Introductory identity and current development remain distinct. -The active path is `composer.compose → construction.Builder → operators.Operators + selection.Selector → placement → validation → notation/export`. Complete action candidates include scalar/arpeggiated routes, heard-theme returns/answers, rests, colors, approaches and anticipations. Ordinary actions need no chord-tone endpoint. Pending contracts retain exact future targets through internal boundaries. The search ranks actions by cost per quarter with bounded repair; reporting groups are not evidence of global phrase optimization. See the [active-path audit](docs/composer/active_path_audit.md) for budgets and limitations. +Rhythm choices are purpose-aware. Arrival values come from authored arrival cells. Fast subdivision groups have a difficulty budget rather than being selected uniformly from every catalog entry. Sequence indexing retains octave crossings. Catalog families contribute representatives, not probabilities proportional to XML file count. Source notes are never changed by generated rhythm transformations. -Both solo roles use strings 1–6, frets 1–12, no open strings, with soft overlapping register preferences. Exact timing, source transposition, ties, harmonic interpretation and runtime representation remain unchanged. The 21 scale patterns govern ordinary movement. Exceptions need actual typed preparation/target/duration evidence. A tonic/predominant natural minor sixth may be a bounded stable color; mapped-upper enclosures must verify their upper neighbor's collection membership. No chord is rewritten to excuse an ornament. +The complete-phrase evaluator reports continuity, contextual repetition, rhythmic interruption, broad activity feasibility, harmonic exposure, soft register and declared return fidelity. Histories covering the same span use duration-weighted phrase totals. The final report also contains an independent whole-stream diagnostic; it is labeled separately from the selection total. No reference-fitting schema, target-loss term, constructor bonus or optional training dependency remains. This evaluator is an inspectable engineering preference, not a model of listener enjoyment or a guarantee of globally optimal melody. -The authoritative evaluator reports validity, normalized components, weights and the actual selection total. It does not read reference events or compute runtime reference similarity. Development-only shared-weight fitting is explicit and bounded. Production catalogs have a checksummed non-executable JSON cache, separate from concrete role caches. Pure chord, expression, rhythm-group and legal-position analysis is memoized. +Both solo roles allow all six strings at frets 1–12 without opens. Overlapping register preferences never impose octave copies or string quotas. Ordinary notes remain in the mapped collection throughout their full sounding interval. Typed color, approach, enclosure, anticipation and suspension rules validate real targets, preparation and exposure. Tied fragments reconstruct one sounding event; reattacks remain distinct. The 21 scale patterns and authoritative written chord semantics are unchanged. Unsupported written harmony fails rather than silently substituting another chord. -Accompaniment remains source-only canonical sparse comping, reused at corresponding form positions. It does not respond to generated lead notes or borrow their grips. No new dynamics, velocity shaping, random timing or instrument effects are introduced. +Accompaniment remains source-derived and canonical across iterations, with independently chosen sparse voicings and rhythms. It neither reads nor reacts to solo notes. Source-defined participation masks keep accompaniment off the physical guitar while it solos. Source vocals, electric melody, pickup timing and runtime representation are preserved. -The resource inventory is documented in [data migration](docs/composer/data_migration.md). Scales, 2,954 relative paths and 3,538 meter-specific rhythms remain useful; the catalogs are not uniformly sampled. New resources store general musical purposes and target-relative rules, not reference phrases or bar recipes. The current user-authorized reference analysis informed these general rules; provenance states that explicitly. +Production exports `*-solo1.xml` (acoustic), `*-solo2.xml` (electric), and `manifest.solo`. Optional `python -m tests.solo_compare` accepts references only after completed generation; it cannot affect candidate selection. It preserves annotations and raw encoding and separately applies the confirmed guitar-written octave convention. No exact-output reproduction is promised. Compare whole pitches, onsets, releases, rests and phrase relationships; attack counts alone are insufficient. The previous fitting experiments and `audition/` and `docs/` directories are removed. -Production exports `*-solo1.xml` (acoustic) and `*-solo2.xml` (electric) plus `manifest.solo`. Optional `audition.solo_compare` and `audition.theory_review` accept explicit reference paths after generation. The four supplied scores use the separately confirmed guitar-written octave convention in this adapter only. Autumn Leaves aligns at measure 2 / first Gm7; preceding material is retained. References are untouched and missing files never change normal generation. +Source songs, scales and useful rhythm/sequence catalogs remain in `data/`. Versioned general rules live in `data/composition`; optional references stay in `build/`, and compatibility expectations live in `tests/fixtures`. Keep only the current comparison and one compact baseline under `build/solo_comparison`. Cached catalogs are checksummed derivatives of production data; a clean build recreates them. Legal TAB is a feasibility approximation, not a finger-assignment or ergonomic guarantee. -One review folder, `build/solo_reference_comparison/`, holds the previous four XMLs, the new four, the complete reference comparison and audio. Historical audition controllers and obsolete generated experiment trees were removed. [Commands](docs/composer/commands.md), [active path](docs/composer/active_path_audit.md), [evaluation contract](docs/composer/evaluation_contract.md), and [implementation evidence](docs/composer/implementation_report.md) describe the current behavior and limits. Technical validity and measured similarity do not establish listener preference. +Current resource and code migration: + +| Resource | Disposition and live consumer | +| --- | --- | +| `data/songs`, `data/scales` | Keep authoritative source music and the 21 collections; importer and scale mapper. | +| `data/sequences`, `data/rhythms` | Keep 2,954 relative paths and 3,538 meter-indexed snippets; complete routes and representative rhythm groups, with immutable catalog caching. | +| `data/composition/solo.json` | Simplify to authored groups/cells and bounded phrase search. Remove per-action budgets and redundant single-note groups. Consumer: `phrases`, `construction`, `rhythm`. | +| `data/composition/roles.json` | Keep independent role/register/difficulty policy; remove fitted selection coefficients. Consumer: `production`, `profiles`, `vocabulary`. | +| `data/composition/harmony.json`, `lead_ins.json`, `rhythms.json` | Keep general harmonic treatments, target-relative formulas, rhythm and sparse-comping vocabulary; preserve their original provenance. | +| `data/expected/performance_summary.json` | Move unchanged to `tests/fixtures`; validation-only form/vocal compatibility, never composition. | +| Old `solo/operators.py`, action-frontier search and learned selector | Replace with complete phrase construction/selection; remove unused path and register-cost helpers. No legacy fallback. | +| `audition/`, `docs/`, historical build trials/training packages | Remove. Keep one optional read-only `tests/solo_compare.py`, current comparison, and compact baseline. | + +The new constructor and search policy are independently authored. Existing general harmony rules acknowledge prior reference-guided review in their provenance; they contain no reference phrases or fitted parameters. Target files remain outside production and do not supply musical seeds. The optional comparison reports pitch-sequence edit distance, exact timed-attack distance, and duration of aligned pitch agreement separately, without a combined optimization score. + +The current bounded constructor still uses heuristics: source features are short, phrase horizons are chosen before surface realization, and only a small number of histories survive. “Extend” develops a longer continuation; it does not rewrite a previously committed phrase. Prepared suspensions and anticipations require an eligible editable boundary and may remain unselected. Supported validation vocabulary does not imply every treatment is a construction operation. Technical validity and these comparisons cannot establish that the resulting melody is preferred by a listener. ### Engraving Boundary @@ -246,7 +261,7 @@ The six compact source files were replaced, not retained as a second authoring f References: [MusicXML durations](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/duration/), [tuplets](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/tuplet/), [degree alterations](https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/degree-alter/), and [SMuFL accidentals](https://smufl.formats.music/latest/tables/standard-accidentals-12-edo.html). -Generator checks compare the generated performance summary against `data/expected/performance_summary.json`. The summary stores compatibility invariants such as section layout, chord-label placement, absence of uncomposed solo slashes, and vocal timing without committing the generated score files themselves. Refresh that file only when an intentional arrangement-logic change should become the new baseline. +Generator checks compare the generated performance summary against `tests/fixtures/performance_summary.json`. The summary stores compatibility invariants such as section layout, chord-label placement, absence of uncomposed solo slashes, and vocal timing without committing the generated score files themselves. Refresh that file only when an intentional arrangement-logic change should become the new baseline. ## Pico Runtime diff --git a/audition/__init__.py b/audition/__init__.py deleted file mode 100644 index f048df3..0000000 --- a/audition/__init__.py +++ /dev/null @@ -1 +0,0 @@ -"""Optional, downstream audition tools. Never imported by production composers.""" diff --git a/audition/benchmark.py b/audition/benchmark.py deleted file mode 100644 index c3515c5..0000000 --- a/audition/benchmark.py +++ /dev/null @@ -1,44 +0,0 @@ -"""Isolated production-role benchmark. No target discovery or audition capture.""" -from __future__ import annotations -import argparse -import cProfile -import hashlib -import json -import os -from pathlib import Path -import platform -import pstats -import time -from generator import production, parts, musicxml, vocabulary - - -def main(): - p=argparse.ArgumentParser(description=__doc__) - p.add_argument('--source',type=Path,required=True) - p.add_argument('--data-dir',type=Path,default=Path('data')) - p.add_argument('--role',choices=production.ROLES,required=True) - p.add_argument('--cache',type=Path) - p.add_argument('--output',type=Path,required=True) - p.add_argument('--profile',action='store_true') - args=p.parse_args();args.output.mkdir(parents=True,exist_ok=True) - start=time.monotonic() - source=production.SourceAnalysis.read(parts.prepare_document(musicxml.load_musicxml(args.source))) - analysis_seconds=time.monotonic()-start - profile=cProfile.Profile() if args.profile else None - if profile:profile.enable() - start=time.monotonic() - result=production.generate_role(source,args.role,args.data_dir,cache_dir=args.cache) - seconds=time.monotonic()-start - if profile: - profile.disable();profile.dump_stats(args.output/'role.prof') - with (args.output/'profile.txt').open('w') as out:pstats.Stats(profile,stream=out).sort_stats('cumulative').print_stats(70) - evidence={'source':str(args.source),'canonical_source':source.musical_key,'role':args.role, - 'platform':platform.platform(),'processor':platform.processor(),'cpu_count':os.cpu_count(),'workers':1, - 'cache_hit':result['cache_hit'],'analysis_seconds':analysis_seconds,'role_seconds':seconds, - 'cprofile_enabled':bool(profile),'musical_document_hash':vocabulary.fingerprint(result['document']), - 'input_trace':result['trace'],'observed_reads':result.get('observed_reads',[])} - (args.output/'result.json').write_text(json.dumps(result,sort_keys=True,indent=2)) - (args.output/'benchmark.json').write_text(json.dumps(evidence,sort_keys=True,indent=2)) - print(json.dumps({k:v for k,v in evidence.items() if k!='input_trace'},indent=2)) - -if __name__=='__main__':main() diff --git a/audition/dependency_check.py b/audition/dependency_check.py deleted file mode 100644 index e8f254d..0000000 --- a/audition/dependency_check.py +++ /dev/null @@ -1,42 +0,0 @@ -"""Explicit reproducibility exercise with authored synthetic input; no targets needed.""" -from __future__ import annotations -import argparse,concurrent.futures,contextlib,copy,json,multiprocessing,tempfile -from pathlib import Path -from generator import production,vocabulary -from tests.solo_fixtures import song - -def main(): - p=argparse.ArgumentParser(description=__doc__);p.add_argument('--output',type=Path,required=True);a=p.parse_args();a.output.mkdir(parents=True,exist_ok=True) - data=Path('data').resolve();source=production.SourceAnalysis.read(song(count=1,chords=('C7',))) - results={};evidence={'scope':'One-bar independently authored source, real production catalogs, all six roles; not a whole-song musical quality test.','canonical_source':source.musical_key,'cases':{}} - def collect(label,values): - hashes={role:vocabulary.fingerprint(value['document']) for role,value in values.items()} - evidence['cases'][label]=hashes - evidence.setdefault('reads',{})[label]={r:v.get('observed_reads',[]) for r,v in values.items()} - if results:assert hashes==results['base'],label - else:results['base']=hashes - with tempfile.TemporaryDirectory() as temp: - cache=Path(temp)/'cache' - collect('alone_cold',{r:production.generate_role(source,r,data,cache_dir=cache) for r in production.ROLES}) - collect('reverse_warm',{r:production.generate_role(source,r,data,cache_dir=cache) for r in reversed(production.ROLES)}) - with concurrent.futures.ProcessPoolExecutor(max_workers=2,mp_context=multiprocessing.get_context('spawn')) as pool: - futures={r:pool.submit(production._job,(source,r,data,{},None)) for r in reversed(production.ROLES)} - collect('parallel_cold',{r:f.result() for r,f in futures.items()}) - with contextlib.chdir(temp): - refs=Path('build');refs.mkdir() - for name in ('all-of-me','autumn-leaves'): - for i in (1,2):(refs/f'{name}-solo-{i}.xml').write_text('invalid optional content') - collect('targets_present_cold',{r:production.generate_role(source,r,data) for r in production.ROLES}) - for path in refs.glob('*.xml'):path.unlink() - collect('targets_absent_cold',{r:production.generate_role(source,r,data) for r in production.ROLES}) - # Exercise an operational musical setting, not merely a different cache - # key. A stream seed alone need not change a deterministic winner. - changed=production.generate_role(source,'electric_solo',data,{'register_center':72}) - changed_hash=vocabulary.fingerprint(changed['document']) - assert changed_hash!=results['base']['electric_solo'],'electric setting was inert' - evidence['changed_electric_register_hash']=changed_hash - collect('after_electric_setting',{r:production.generate_role(source,r,data,cache_dir=cache) for r in production.ROLES}) - evidence['passed']=True - (a.output/'dependency.json').write_text(json.dumps(evidence,indent=2)) - print(json.dumps({'passed':True,'hashes':results['base']},indent=2)) -if __name__=='__main__':main() diff --git a/audition/fit_selection.py b/audition/fit_selection.py deleted file mode 100644 index f500dcd..0000000 --- a/audition/fit_selection.py +++ /dev/null @@ -1,146 +0,0 @@ -"""Explicit development-only shared-rule derivation and fitting. - -Targets are read only by this command. Production imports neither this module -nor its artifacts. A derivation uses the exact same enumerated Actions as the -normal composer; accepting injected pitches is not a construction proof. -""" -from __future__ import annotations -import argparse -import bisect -import dataclasses -from fractions import Fraction as F -import hashlib -import heapq -import json -from pathlib import Path -import time - -from audition.reference_review import read -from audition.solo_compare import events -from generator import accompaniment, musicxml, parts, production, resources, score, vocabulary -from generator.arrange import form, generate -from generator.scales import selection, timeline -from generator.solo import analysis, construction, harmonic, profiles - - -def build(source_path,role,data_dir,cache_dir): - """Read original song/form and production data; never reference harmony.""" - document=parts.prepare_document(musicxml.load_musicxml(source_path)) - a=analysis.analyze(document);part=role.split('_')[0] - source=score.source_staff(document)['measures'] - sections=form.full_arrangement_sections(source,part);si=3 if part=='acoustic' else 4 - section=sections[si] - effective=generate.section_source(source,section,sections[si+1]) - measures=effective[section.source_measure_index:section.source_measure_index+section.measure_count] - initial=accompaniment.carried_chords(effective)[section.source_measure_index] - from generator.solo.arrangement import regions - first,_,offset,_=regions(document)[si] - start,end=next((a-offset,b-offset) for a,b,r in production.windows(document,part) if r==role) - if start<0: - template=generate._arrange_document(document,part,{},owned={'slash'}) - measures=[template['staves'][0]['measures'][first-1],*measures] - initial=accompaniment.carried_chords(template['staves'][0]['measures'])[first-1] - sc,seq,rh=resources.load(data_dir,cache_dir=cache_dir) - choices=selection.select(timeline.read({'measures':measures},initial=initial),timeline.Solo(F(),end-start,role),sc) - choices=tuple(dataclasses.replace(c,span=dataclasses.replace(c.span,start=c.span.start+start,end=c.span.end+start)) for c in choices) - h=harmonic.Harmony(choices,profiles.for_part(part,a)) - return construction.Builder(a,measures,h,seq,rh,end,'development',start=start),document - - -def canonical(notes): - return tuple((n.start,n.end,n.pitch.number()) for n in notes) - - -def derive(builder,target): - """Stateful target-constrained coverage probe; NEVER production success. - - Separate histories carry actual pending contracts. This bounded diagnostic - can report exhaustion, not prove a phrase impossible. No notes are injected. - """ - encoded=tuple((F(n['start']),F(n['end']),int(n['pitch'])) for n in events(target)) - starts=tuple(n[0] for n in encoded);encoded_set=set(encoded) - queue=[builder.start];chart={builder.start:[construction.State()]};queued={builder.start} - furthest=builder.start;last=construction.State();cuts=0 - while queue: - at=heapq.heappop(queue);states=chart.pop(at);furthest=max(furthest,at) - unique={builder.identity(s):s for s in states};states=list(unique.values()) - cuts+=max(0,len(states)-16);states=states[:16] - if at==builder.end: - for state in states: - checked=builder.harmony.evaluate(state.notes,expressions=tuple(c for a in state.actions for c in a.contracts),complete=True,until=at) - if checked.valid and canonical(state.notes)==encoded: - return state.actions,{'complete':True,'covered_until':str(at),'actions':len(state.actions), - 'status':'statefully_constructible','diagnostic_only':True,'pruned_histories':cuts} - left=bisect.bisect_left(starts,at) - def matches(action): - if any((c.target.start,c.target.end,c.target.pitch.number()) not in encoded_set for c in action.contracts):return False - expected=encoded[left:bisect.bisect_left(starts,action.end)] - return not (left and encoded[left-1][1]>at) and canonical(action.notes)==expected - first=encoded[left] if left=furthest),None) - return last.actions,{'complete':False,'covered_until':str(furthest), - 'status':'coverage_search_exhausted' if cuts else 'not_proposed_or_rejected_by_rule', - 'next_event':tuple(map(str,next_event)) if next_event else None, - 'pruned_histories':cuts,'diagnostic_only':True,'action_coverage':dict(builder.operators.coverage)} - - -def fit(examples,passes,on_checkpoint=None,resources=None): - # Imported only by this explicitly requested development operation. - from audition.structured_learning import fit_complete - return fit_complete(examples,passes,on_checkpoint,resources) - - -def main(): - p=argparse.ArgumentParser(description=__doc__) - p.add_argument('operation',choices=('explain','fit')) - p.add_argument('--source',action='append',required=True,metavar='CASE=PATH') - p.add_argument('--reference',action='append',required=True,metavar='CASE=PATH') - p.add_argument('--role',action='append',required=True,metavar='CASE=acoustic_solo|electric_solo') - p.add_argument('--form-start',action='append',default=[],metavar='CASE=MEASURE') - p.add_argument('--data-dir',type=Path,default=Path('data')) - p.add_argument('--cache-dir',type=Path) - p.add_argument('--output',type=Path,required=True) - p.add_argument('--passes',type=int,default=8) - p.add_argument('--solver-path',type=Path,help='Development-only pinned solver installation directory') - args=p.parse_args() - if args.solver_path: - import sys - sys.path.insert(0,str(args.solver_path.resolve())) - if not 1<=args.passes<=200:p.error('passes must be 1–200') - pairs=lambda values:dict(v.split('=',1) for v in values) - sources,refs,roles=map(pairs,(args.source,args.reference,args.role));starts=pairs(args.form_start) - if sources.keys()!=refs.keys() or sources.keys()!=roles.keys():p.error('source/reference/role cases must match') - args.output.mkdir(parents=True,exist_ok=True);results={};examples=[] - data=vocabulary.load(args.data_dir/'composition');began=time.monotonic() - for name in sorted(sources): - with vocabulary.using(data,roles[name]): - b,_=build(sources[name],roles[name],args.data_dir,args.cache_dir) - target=read(refs[name],name,int(starts[name]) if name in starts else None,'guitar-written') - path,result=derive(b,target) - results[name]={**result,'reference_sha256':target['sha256'],'source_sha256':hashlib.sha256(Path(sources[name]).read_bytes()).hexdigest(), - 'derivation':[{'kind':a.kind,'start':str(a.start),'end':str(a.end),'events':[(str(n.start),str(n.end),n.pitch.number()) for n in a.notes],'rules':[c.rule for c in a.contracts]} for a in path]} - if args.operation=='fit' and result['complete']:examples.append((name,roles[name],b,path)) - print(name,result['status'],result['covered_until'],flush=True) - result={'schema':'shared-phrase-learning-2','development_only':True,'coverage_seconds':time.monotonic()-began,'cases':results} - if args.operation=='fit': - if not all(r['complete'] for r in results.values()):result['fit_status']='blocked_by_construction_coverage' - else: - def checkpoint(value): - path=args.output/'checkpoint.json';temporary=path.with_suffix('.tmp') - temporary.write_text(json.dumps(value,indent=2)+'\n');temporary.replace(path) - fitting_start=time.monotonic() - result['fit']=fit(examples,args.passes,checkpoint,data) - result['fit_seconds']=time.monotonic()-fitting_start - result['total_seconds']=time.monotonic()-began - (args.output/'coverage.json').write_text(json.dumps(result,indent=2)+'\n') - return 0 if all(r['complete'] for r in results.values()) and result.get('fit',{'complete':True})['complete'] else 2 - -if __name__=='__main__':raise SystemExit(main()) diff --git a/audition/phrase_candidates.py b/audition/phrase_candidates.py deleted file mode 100644 index 38f19e6..0000000 --- a/audition/phrase_candidates.py +++ /dev/null @@ -1,74 +0,0 @@ -"""Export source-only complete candidates; references are not accepted here.""" -import argparse -import copy -import dataclasses -from fractions import Fraction as F -import json -from pathlib import Path -import time -import cProfile - -from audition.fit_selection import build -from generator import model, musicxml_writer, score, vocabulary -from generator.solo import notation, placement - - -def export(builder,document,output): - output.mkdir(parents=True,exist_ok=True) - source=score.source_staff(document)['measures'] - start_index=score.body_start_measure_index(source) - if builder.start<0:start_index-=1 - length=F();selected=[] - for m in source[start_index:]: - if length>=builder.end-builder.start:break - selected.append(m);length+=model.fraction(m['duration']) - if length!=builder.end-builder.start:raise ValueError('Diagnostic export requires complete source measures') - records=[] - for index,state in enumerate(builder.completed,1): - played=placement.place(state.notes,builder.analysis.tempo,profile=builder.harmony.profile) - if played is None:raise ValueError('Completed candidate is unplayable') - shifted=tuple(dataclasses.replace(n,start=n.start-builder.start,end=n.end-builder.start) for n in played[1]) - part=copy.deepcopy(document);staff=part['staves'][0] - part['staves']=[staff];staff['measures']=notation.measures(selected,shifted,'candidate') - staff['tuplets']=notation.tuplets(staff['measures']);staff['tablature']=True - for key in ('scale_spans','octave_shifts','slurs','wedges'):staff.pop(key,None) - path=output/f'candidate-{index}.xml' - musicxml_writer.write_musicxml(path,part,tablature=True) - records.append({'file':path.name,'automatic_winner':index==1,'evaluation':dataclasses.asdict(state.evaluation), - 'home':[(str(n.start),str(n.end),n.pitch.number()) for n in state.home], - 'events':[(str(n.start),str(n.end),n.pitch.number()) for n in state.notes], - 'actions':[{'kind':a.kind,'start':str(a.start),'end':str(a.end),'contracts':[c.rule for c in a.contracts]} for a in state.actions]}) - value={'source_only':True,'automatic_winner':'candidate-1.xml','start':str(builder.start),'end':str(builder.end), - 'candidates':records,'frontiers':builder.frontiers,'timings':dict(builder.timings), - 'note':'Diagnostic candidates, not publication or evidence of exact reproduction.'} - (output/'candidates.json').write_text(json.dumps(value,indent=2,default=str)+'\n') - return value - - -def main(): - p=argparse.ArgumentParser(description=__doc__) - p.add_argument('--source',type=Path,required=True);p.add_argument('--role',choices=('acoustic_solo','electric_solo'),required=True) - p.add_argument('--data-dir',type=Path,default=Path('data'));p.add_argument('--output',type=Path,required=True) - p.add_argument('--cache-dir',type=Path);p.add_argument('--bars',type=int,default=4) - p.add_argument('--profile',action='store_true') - args=p.parse_args() - if args.bars<1:p.error('--bars must be positive') - with vocabulary.using(vocabulary.load(args.data_dir/'composition'),args.role): - b,document=build(args.source,args.role,args.data_dir,args.cache_dir) - ends=[x-b.analysis.body_offset for x in b.analysis.boundaries if x-b.analysis.body_offset>0] - b.end=min(b.end,ends[min(args.bars-1,len(ends)-1)]);b.operators.end=b.end - started=time.monotonic();seen=set() - def progress(stage,at,state,proposed,retained): - if stage=='frontier' and at.denominator==1 and at not in seen: - seen.add(at);print(f'frontier {at}: {len(proposed)} histories, {len(retained)} retained; {time.monotonic()-started:.1f}s',flush=True) - b.observe=progress - args.output.mkdir(parents=True,exist_ok=True) - profiler=cProfile.Profile() if args.profile else None - if profiler:profiler.enable() - try:b.compose() - finally: - if profiler:profiler.disable();profiler.dump_stats(args.output/'search.prof') - export(b,document,args.output) - print(json.dumps({'completed':len(b.completed),'seconds':b.timings['action_search_seconds'],'winner':str(b.winner_evaluation.total)})) - -if __name__=='__main__':main() diff --git a/audition/reference_review.py b/audition/reference_review.py deleted file mode 100644 index f5fb52b..0000000 --- a/audition/reference_review.py +++ /dev/null @@ -1,143 +0,0 @@ -"""Read optional reference MusicXML literally, without repairing it or composing.""" -from __future__ import annotations -import argparse -import hashlib -import json -from pathlib import Path -from fractions import Fraction as F -import xml.etree.ElementTree as ET - - -def read(path, role, form_start=None, pitch_convention='encoded'): - if pitch_convention not in ('encoded','guitar-written'): - raise ValueError('Unknown reference pitch convention: '+pitch_convention) - raw=Path(path).read_bytes();tree=ET.fromstring(raw) - result={'path':str(Path(path).resolve()),'sha256':hashlib.sha256(raw).hexdigest(),'validation_role':role, - 'pitch_convention':pitch_convention,'part_names':[n.text for n in tree.findall('./part-list/score-part/part-name')], - 'measures':[],'notes':[],'harmonies':[],'annotations':[],'discrepancies':[]} - divisions=F(1);absolute=F();nominal=F(4);pending={};active=None;transpose=F() - for index,m in enumerate(tree.findall('./part/measure')): - cursor=F();extent=F();last_start=F();last_duration=F() - if (value:=m.findtext('attributes/divisions')) is not None:divisions=F(value) - meter=m.find('attributes/time') - if meter is not None and meter.findtext('beats'): - nominal=sum((F(v) for v in meter.findtext('beats').split('+')),F())*4/F(meter.findtext('beat-type')) - for item in m: - offset=F(item.findtext('offset','0'))/divisions - if item.tag=='attributes' and item.find('transpose') is not None: - transpose=F(item.findtext('transpose/chromatic','0'))+12*F(item.findtext('transpose/octave-change','0')) - elif item.tag=='backup':cursor-=F(item.findtext('duration'))/divisions - elif item.tag=='forward':cursor+=F(item.findtext('duration'))/divisions;extent=max(extent,cursor) - elif item.tag=='harmony': - active={'time':str(absolute+cursor+offset),'measure':index+1,'offset':str(cursor+offset), - 'root':item.findtext('root/root-step'),'alter':item.findtext('root/root-alter','0'), - 'kind':item.findtext('kind'),'display':item.find('kind').get('text') if item.find('kind') is not None else None, - 'bass':ET.tostring(item.find('bass'),encoding='unicode') if item.find('bass') is not None else None, - 'degrees':[{'value':d.findtext('degree-value'),'alter':d.findtext('degree-alter'),'type':d.findtext('degree-type')} for d in item.findall('degree')]} - result['harmonies'].append(active) - elif item.tag=='direction': - for words in item.findall('direction-type/words'): - result['annotations'].append({'time':str(absolute+cursor+offset),'measure':index+1,'text':''.join(words.itertext()),'interpretation':'descriptive; no production permission'}) - elif item.tag=='note': - duration=F(item.findtext('duration','0'))/divisions - start=last_start if item.find('chord') is not None else cursor - end=start+duration;voice=item.findtext('voice','1');p=item.find('pitch') - encoded={'step':p.findtext('step'),'alter':p.findtext('alter','0'),'octave':int(p.findtext('octave'))} if p is not None else None - kinds={t.get('type') for t in item.findall('tie')} | {t.get('type') for t in item.findall('notations/tied')} - event={'start':str(absolute+start),'end':str(absolute+end),'measure':index+1,'offset':str(start),'pitch':encoded,'rest':item.find('rest') is not None, - 'written_type':item.findtext('type'),'dots':len(item.findall('dot')),'voice':voice,'ties':sorted(kinds), - 'sounding_transposition':str(-12 if pitch_convention=='guitar-written' else transpose), - 'tablature':{'string':int(item.findtext('notations/technical/string')),'fret':int(item.findtext('notations/technical/fret'))} if item.find('notations/technical/string') is not None and item.find('notations/technical/fret') is not None else None, - 'time_modification':ET.tostring(item.find('time-modification'),encoding='unicode') if item.find('time-modification') is not None else None, - 'tuplets':[dict(t.attrib) for t in item.findall('notations/tuplet')],'grace':item.find('grace') is not None,'new_attack':'stop' not in kinds and p is not None} - key=(voice,json.dumps(encoded,sort_keys=True)) - if 'stop' in kinds: - old=pending.get(key) - if old is None or result['notes'][old]['end']!=event['start']: - result['discrepancies'].append({'measure':index+1,'issue':'unmatched/noncontiguous tie; retained literally'}) - else:event['continuation_of']=old - if 'start' not in kinds:pending.pop(key,None) - if 'start' in kinds:pending[key]=len(result['notes']) - result['notes'].append(event) - for container in ('notations/technical/fingering','notations/technical/other-technical','notations/other-notation'): - for label in item.findall(container): - text=''.join(label.itertext()).strip() - if text:result['annotations'].append({'time':str(absolute+start),'measure':index+1,'text':text, - 'container':container,'interpretation':'Original note annotation; container does not establish musical permission or guitar fingering.'}) - if item.find('chord') is None and item.find('grace') is None:cursor=end - last_start,last_duration=start,duration;extent=max(extent,end,cursor) - if extent!=nominal and m.get('implicit')!='yes':result['discrepancies'].append({'measure':index+1,'issue':f'encoded span {extent} differs from nominal {nominal}; no padding applied'}) - result['measures'].append({'number':m.get('number'),'start':str(absolute),'duration':str(extent),'implicit':m.get('implicit'),'initial_harmony_known':any(F(h['time'])<=absolute for h in result['harmonies'])}) - absolute+=extent - if pending:result['discrepancies'].append({'issue':'unterminated ties retained'}) - result['harmonies'].sort(key=lambda h:F(h['time'])) - # Do not run the strict score importer over these references: annotations, - # unusual timing and high encoded pitches are evidence, not repair requests. - from generator import harmony, model, fretboard - def semantic(h): - return {'root':{'letter':h['root'].lower(),'alter':model.rational(F(h['alter']))},'kind':h['kind'], - 'degrees':[{'value':int(d['value']),'alter':model.rational(F(d['alter'])),'type':d['type']} for d in h['degrees']]} - for event in result['notes']: - if not event['pitch']:continue - p=event['pitch'];number=12*(p['octave']+1)+model.NATURAL[p['step'].lower()]+F(p['alter']) - event['encoded_midi']=str(number) - event['sounding_midi']=str(number+F(event['sounding_transposition'])) - event['legal_positions_at_encoded_pitch']=len(fretboard.single_positions_for_pitch(number)) - event['legal_positions_at_sounding_pitch']=len(fretboard.single_positions_for_pitch(F(event['sounding_midi']))) - event['harmonic_spans']=[] - start,end=F(event['start']),F(event['end']) - changes=[F(h['time']) for h in result['harmonies'] if startt),None) - label['current_harmony']=current[-1] if current else None - label['following_harmony']=future - label['events']=[n for n in result['notes'] if F(n['start'])<=t0:high=min(high,math.floor(limit/slope)) - elif slope<0:low=max(low,math.ceil(limit/slope)) - elif limit<0:low,high=1,0;break - if low<=high: - value=low if low>0 else high if high<0 else 0 - choices.append((abs(value),key,value)) - if not choices:break - _,key,value=min(choices);simple[role]={key:value} - if set(simple)=={r['role'] for r in rows}: - for weights in simple.values():selection.validate_weights(weights) - return {'status':'solved','weights':simple,'bound':bound, - 'nonzero':sum(len(w) for w in simple.values()),'l1':sum(abs(v) for w in simple.values() for v in w.values()), - 'method':'exhaustive zero/one-coefficient integer intervals; minimum support and L1 within that support proven'} - names=sorted({(r['role'],k) for r in rows for k in r['delta']}) - variables={n:z3.Int('w_'+str(i)) for i,n in enumerate(names)} - limits=[v>=-bound for v in variables.values()]+[v<=bound for v in variables.values()] - rational=lambda v:z3.RealVal(str(F(v))) - constraints=[z3.Sum([variables[(r['role'],k)]*rational(v) for k,v in r['delta'].items()])+rational(r['prior'])<=-1 for r in rows] - check=z3.Solver();check.set(rlimit=2000000);check.add(*limits) - for i,c in enumerate(constraints):check.assert_and_track(c,'comparison_'+str(i)) - status=check.check() - if status==z3.unknown:return {'status':'solver_unknown','stage':'feasibility','reason':check.reason_unknown()} - if status==z3.unsat: - return {'status':'incompatible_with_current_features_and_bounds','bound':bound, - 'conflicts':[str(x) for x in check.unsat_core()], - 'identical_feature_pairs':[i for i,r in enumerate(rows) if not r['delta'] and r['prior']>=0]} - opt=z3.Optimize();opt.add(*limits,*constraints) - # Deterministic solver work budget. Exhaustion yields no proposed model; - # elapsed wall time never chooses a musical winner. - opt.set(rlimit=2000000) - opt.minimize(z3.Sum([z3.If(v==0,0,1) for v in variables.values()])) - opt.minimize(z3.Sum([z3.If(v<0,-v,v) for v in variables.values()])) - if opt.check()!=z3.sat:return {'status':'solver_unknown','reason':opt.reason_unknown()} - model=opt.model();weights={r:{} for r,_ in names} - for (role,key),v in variables.items(): - value=model.eval(v,model_completion=True).as_long() - if value:weights[role][key]=value - for w in weights.values():selection.validate_weights(w) - return {'status':'solved','weights':weights,'bound':bound, - 'nonzero':sum(len(w) for w in weights.values()),'l1':sum(abs(v) for w in weights.values() for v in w.values())} - - -def sounded(actions): - from generator.solo.types import sounding_notes - return tuple((n.start,n.end,n.pitch.number()) for n in sounding_notes(tuple(n for a in actions for n in a.notes))) - - -class CoverageTrace: - """Read-only observer. Targets diagnose cuts, never supply search decisions.""" - def __init__(self,gold): - self.gold=sounded(gold);self.records=[] - def matches(self,notes,end): - # These are committed whole actions, not a truncated playback preview. - # A quarter-note attack cannot reproduce a half-note target merely because - # its first quarter sounds the same. Future actions do not reopen it. - from generator.solo.types import sounding_notes - actual=tuple((n.start,n.end,n.pitch.number()) for n in sounding_notes(notes)) - expected=tuple((a,b,p) for a,b,p in self.gold if a0 else 'b'*-int(F(h['alter']))) - if h['kind']=='other': - # The explicit displayed quality is evidence; never substitute a target - # chord for a missing source chord. Preserve any parse error in the report. - return harmony.parse_symbol(root+(h['display'] or '')) - return {'root':{'letter':h['root'].lower(),'alter':model.rational(F(h['alter']))},'kind':h['kind'], - 'degrees':[{'value':int(d['value']),'alter':model.rational(F(d['alter'])),'type':d['type']} for d in h['degrees']]} - - -def members(h): - c=chord(h) - if not c:return {} - root=model.NATURAL[c['root']['letter']]+model.alteration(c['root']['alter']) - return {int(root+harmony.degree_semitones(d,a))%12:str(d) for d,a in harmony.degrees(c).items()} - - -def explain(review, source=None): - """Observed relationships and hypotheses, not retrospective permissions.""" - notes=solo_compare.events(review);offset=F(review['alignment']['encoded_offset'] or 0) - hs=[{**h,'time':str(F(h['time'])-offset)} for h in review['harmonies']] - output=[] - for i,n in enumerate(notes): - start,end=F(n['start']),F(n['end']);p=int(F(n['pitch'])) - active=next((h for h in reversed(hs) if F(h['time'])<=start),None) - future=next((h for h in hs if F(h['time'])>start),None) - relation=[];uncertainty=[] - try: - cm=members(active);fm=members(future) - except (ValueError,KeyError) as e:cm=fm={};uncertainty.append(str(e)) - if p%12 in cm:relation.append('chord member '+cm[p%12]) - else:relation.append('non-basic pitch: needs collection/color/motion analysis') - if future and start0 else 'turn') - for target in notes[i+1:i+4]: - t=F(target['start']);q=int(F(target['pitch'])) - if t-start>F(3,2):break - if abs(q-p)==1 and end<=t and t-end<=F(1,2): - relation.append('semitone approach to '+target['pitch']+' at '+target['start']);break - if end-start>=2:relation.append('sustained event; '+('breath follows' if i+1end else 'continuation follows')) - label=[a['text'] for a in review['annotations'] if start<=F(a['time'])-offset1], - 'source_four_event_relationships':source_matches,'source_relationship_limit':'Exact relative four-event evidence only; absence does not disprove thematic influence.', - 'interpretation':'Descriptive hypotheses; no label waives production validation. No output from this command is loaded by production.'} - - -def rule_review(review, choices, offset): - """Read-only recognition under production policies; never run composition. - - Recognition deliberately need not cover every reference note. Failed or - ambiguous relationships remain failures, rather than changing a policy. - """ - import dataclasses - from generator import vocabulary - from generator.solo import types, harmonic, expressions - choices=tuple(dataclasses.replace(c,span=dataclasses.replace(c.span,start=c.span.start-offset,end=c.span.end-offset)) for c in choices) - h=harmonic.Harmony(choices);ns=[] - for i,n in enumerate(solo_compare.events(review)): - p=n['spelling'];ns.append(types.Note(F(n['start']),F(n['end']),types.Pitch(p['step'].lower(),F(p['alter']),p['octave']-1),event_id=str(i))) - contracts=[];rejections=collections.defaultdict(list) - def accept(c): - if expressions.validate(tuple(ns),(*contracts,c),h)[1]:return False - contracts.append(c);return True - for n in ns: - for boundary in (s.start for s in h.spans if n.start0 else 'b'*-int(F(raw['alter'])))+str(raw['octave']-1) - lines.append(f"| {n['measure']} | {n['start']} | {pitch} / {F(n['end'])-F(n['start'])} | {'; '.join(n['relationships'])} | {'; '.join(n['annotation']).replace(chr(10),' / ')} |") - lines.extend(['',f"Relative four-event returns: {analysis['relative_four_event_returns']}",'',f"Exact relative source relationships: {analysis['source_four_event_relationships']}",'', - 'General rule recognition against ORIGINAL source harmony: '+str(recognized['statuses']), - 'Recognized policies: '+str(recognized['rules']),'',recognized['limit'],'', - '| Onset | Pitch | Collection-valid throughout | Recognition | Policy / source harmony |','|---|---|---|---|---|']) - for n in recognized['events']: - failures='; '.join(r['rule']+': '+r['rejection'].split(': ',1)[-1]+' (expected '+r['expected_spelling']+')' for r in n['candidate_rejections']) - lines.append(f"| {n['start']} | {n['pitch']} | {n['mapped_throughout']} | {n['status']} | {n['rule'] or ''} / {', '.join(n['harmonies'])} {failures} |") - (args.output/'theory.json').write_text(json.dumps(results,indent=2)) - (args.output/'theory.md').write_text('\n'.join(lines)+'\n') - print('Reviewed every sounding event in',len(results),'complete reference solos; no composer invoked.') - - -if __name__=='__main__':main() diff --git a/audition/train_completed.py b/audition/train_completed.py deleted file mode 100644 index d562efb..0000000 --- a/audition/train_completed.py +++ /dev/null @@ -1,144 +0,0 @@ -"""Fit shared ranking from completed ordinary generation, never prefix roll-ins. - -This development command reads explicit targets and saved complete alternatives. -It writes a *proposal*, not a released model. A staged data copy may be passed to -the ordinary generator for a separate target-free cold reproduction test. -""" -import argparse -import copy -from fractions import Fraction as F -import hashlib -import json -from pathlib import Path -import shutil -import sys - -from audition.fit_selection import build,derive -from audition.reference_review import read -from audition.structured_learning import vector,constraint,solve,sounded,CoverageTrace -from generator import production,score,vocabulary -from generator.solo import construction,selection,policy - - -def evaluation_vector(value): - if value['version']!=selection.VERSION:raise ValueError('Completed output uses another evaluator version') - span=F(value['denominators']['span']) - features={k:F(v)*span for k,v in value['raw'].items() if not k.startswith('prior/') and F(v)} - selection.validate_weights(dict.fromkeys(features,0)) - prior=sum(F(v)*span*selection.DEFAULT[k.removeprefix('prior/')] for k,v in value['raw'].items() if k.startswith('prior/')) - return features,prior - - -def training_context(data): - """Changing learned weights is allowed between rounds; musical rules are not.""" - musical=copy.deepcopy(data) - musical['roles'].pop('provenance',None) - for role in musical['roles']['roles'].values():role.pop('selection_weights',None) - return {'evaluator':selection.VERSION,'algorithm':policy.DEFAULT.composition_version, - 'fixed_priors':selection.DEFAULT,'musical_data':vocabulary.fingerprint(musical)} - - -def previous_constraints(evidence,cases,context): - if evidence.get('schema')!='complete-candidate-training-1' or evidence.get('context')!=context: - raise ValueError('Previous round has incompatible evaluation or musical data') - earlier=evidence['training_dependency'] - if earlier.keys()!=cases.keys():raise ValueError('Previous round has different training cases') - for name in cases: - for key in ('source_sha256','target_sha256','role'): - if cases[name][key]!=earlier[name][key]:raise ValueError('Previous round changed '+name+'/'+key) - rows=[] - for row in evidence['constraints']: - if row['case'] not in cases or row['role']!=cases[row['case']]['role']: - raise ValueError('Previous constraint has a mismatched case/role') - selection.validate_weights(dict.fromkeys(row['delta'],0)) - rows.append({**row,'delta':{k:F(v) for k,v in row['delta'].items()},'prior':F(row['prior'])}) - return rows - - -def main(): - p=argparse.ArgumentParser(description=__doc__) - for name in ('source','reference','role'):p.add_argument('--'+name,action='append',required=True,metavar='CASE=VALUE') - p.add_argument('--form-start',action='append',default=[]) - p.add_argument('--data-dir',type=Path,default=Path('data')) - p.add_argument('--completed-dir',type=Path,required=True,help='Completed ordinary assets directory containing analysis/*.roles') - p.add_argument('--output',type=Path,required=True);p.add_argument('--cache-dir',type=Path) - p.add_argument('--solver-path',type=Path,required=True) - p.add_argument('--stage-data',action='store_true',help='Write an isolated data copy for ordinary inference; does not publish it') - p.add_argument('--previous-round',type=Path,action='append',default=[],help='Accumulate complete-output constraints from a compatible training.json; never a production input') - args=p.parse_args();sys.path.insert(0,str(args.solver_path.resolve())) - pairs=lambda values:dict(v.split('=',1) for v in values) - sources,refs,roles=map(pairs,(args.source,args.reference,args.role));starts=pairs(args.form_start) - if sources.keys()!=refs.keys() or sources.keys()!=roles.keys():p.error('Cases must agree') - args.output.mkdir(parents=True,exist_ok=True);rows=[];cases={};contexts=[];data=vocabulary.load(args.data_dir/'composition') - for name in sorted(sources): - with vocabulary.using(data,roles[name]): - b,document=build(sources[name],roles[name],args.data_dir,args.cache_dir) - target=read(refs[name],name,int(starts[name]) if name in starts else None,'guitar-written') - gold,coverage=derive(b,target) - if not coverage['complete']:raise ValueError(name+': '+coverage['status']) - s=construction.State() - for a in gold:s=b.advance(s,None,a) - positive=vector(b,gold,s.home) - contexts.append((name,roles[name],b,gold)) - path=args.completed_dir/'analysis'/(score.slugify(Path(sources[name]).stem)+'.roles') - generated=json.loads(path.read_text())[roles[name]] - if generated['trace']['source']!=production.SourceAnalysis.read(document).musical_key: - raise ValueError(name+': completed candidates belong to different canonical source music') - if generated['trace']['role']!=roles[name]:raise ValueError(name+': wrong completed role') - if generated['trace']['vocabulary']!=vocabulary.fingerprint(production.dependencies(roles[name],data)): - raise ValueError(name+': completed candidates use different production data/settings') - candidates=generated['details']['selection_trace']['completed_candidates'] - negatives=0;exact=False - for c in candidates: - stream=tuple((F(a),F(z),int(n)) for a,z,n in c['events']) - same=stream==sounded(gold) - if c['rank']==1:exact=same - if not same:rows.append(constraint(roles[name],positive,evaluation_vector(c['evaluation']),name));negatives+=1 - cases[name]={'source':sources[name],'reference':refs[name],'role':roles[name], - 'source_sha256':hashlib.sha256(Path(sources[name]).read_bytes()).hexdigest(), - 'target_sha256':target['sha256'],'completed_artifact_sha256':hashlib.sha256(path.read_bytes()).hexdigest(), - 'coverage':coverage,'ordinary_winner_exact':exact,'complete_negative_candidates':negatives} - print(name,exact,negatives,flush=True) - context=training_context(data);parents=[] - for path in args.previous_round: - raw=path.read_bytes();rows.extend(previous_constraints(json.loads(raw),cases,context)) - parents.append({'path':str(path),'sha256':hashlib.sha256(raw).hexdigest()}) - unique={} - for row in rows: - key=(row['role'],row['case'],tuple(sorted(row['delta'].items())),row['prior']) - unique[key]=row - rows=[unique[key] for key in sorted(unique)] - result=solve(rows) - evidence={'schema':'complete-candidate-training-1','training_dependency':cases,'solver':result, - 'context':context,'previous_rounds':parents, - 'constraints':[{**r,'delta':{k:str(v) for k,v in r['delta'].items()},'prior':str(r['prior'])} for r in rows], - 'released':False,'acceptance':'Exact four full outputs from ordinary cold target-free generation, plus transfer; not satisfied by this training solve.'} - if result['status']=='solved': - # A zero here proves this proposed model cannot reproduce the full output. - # A nonzero result proves neither full retention nor exact generation. - # This is read-only diagnosis, not prefix fitting or candidate injection. - cuts={} - for name,role,b,gold in contexts: - with vocabulary.using(data,role): - b.selector.weights=result['weights'].get(role,{}) - trace=CoverageTrace(gold);b.observe=trace - retained=b.candidates(b.start,construction.State());b.observe=None - count=sum(trace.matches(a.notes,a.end) for _,a in retained) - cuts[name]={'matching_first_successors':count,'observed_failures':trace.records, - 'full_reproduction_already_excluded':count==0} - evidence['post_fit_cutoff_diagnosis']=cuts - (args.output/'training.json').write_text(json.dumps(evidence,indent=2)+'\n') - if result['status']=='solved': - (args.output/'proposed-weights.json').write_text(json.dumps(result['weights'],indent=2)+'\n') - if args.stage_data: - staged=args.output/'data' - if staged.exists():raise ValueError('Refusing to overwrite a previous staged data proposal') - shutil.copytree(args.data_dir,staged) - path=staged/'composition'/'roles.json';value=json.loads(path.read_text()) - for role,weights in result['weights'].items():value['roles'][role]['selection_weights']=weights - value['provenance']='Shared bounded coefficients learned from explicit authored training examples; see adjacent development training.json. No runtime target reads or stored phrases.' - path.write_text(json.dumps(value,indent=2)+'\n') - print(json.dumps(result,indent=2)) - return 0 if result['status']=='solved' else 2 - -if __name__=='__main__':raise SystemExit(main()) diff --git a/audition/transfer_review.py b/audition/transfer_review.py deleted file mode 100644 index a3f2894..0000000 --- a/audition/transfer_review.py +++ /dev/null @@ -1,32 +0,0 @@ -"""Optional evaluation of independent source excerpts; never a production import.""" -from __future__ import annotations -import argparse,json,time -from pathlib import Path -from generator import musicxml,parts,production,vocabulary,score,musicxml_writer - -def main(): - p=argparse.ArgumentParser(description=__doc__) - p.add_argument('--sources',type=Path,default=Path('tests/evaluation_sources')) - p.add_argument('--data-dir',type=Path,default=Path('data')) - p.add_argument('--output',type=Path,required=True) - args=p.parse_args();args.output.mkdir(parents=True,exist_ok=True) - report={'scope':'Four-bar source excerpts, not complete standards; read adjacent provenance. No target solo used.', 'cases':[]} - for path in sorted(args.sources.glob('*.xml')): - source=production.SourceAnalysis.read(parts.prepare_document(musicxml.load_musicxml(path))) - roles={};case={'source':str(path),'canonical_source':source.musical_key,'roles':{}} - for role in production.ROLES: - start=time.monotonic() - try: - value=production.generate_role(source,role,args.data_dir) - roles[role]=value;case['roles'][role]={'valid':True,'seconds':time.monotonic()-start,'hash':vocabulary.fingerprint(value['document'])} - except ValueError as error: - case['roles'][role]={'valid':False,'seconds':time.monotonic()-start,'diagnostic':str(error)} - print(path.stem,role,case['roles'][role],flush=True) - if len(roles)==len(production.ROLES): - for part,document in production.assemble(source,roles).items(): - score.write_json(args.output/(path.stem+'_'+part+'.json'),document) - musicxml_writer.write_musicxml(args.output/(path.stem+'_'+part+'.xml'),document) - report['cases'].append(case) - (args.output/'report.json').write_text(json.dumps(report,indent=2)) - return int(any(not r['valid'] for c in report['cases'] for r in c['roles'].values())) -if __name__=='__main__':raise SystemExit(main()) diff --git a/cmake/zero.cmake b/cmake/zero.cmake index cb78f32..aa6d850 100644 --- a/cmake/zero.cmake +++ b/cmake/zero.cmake @@ -136,7 +136,7 @@ configure_file(package/manifest.json.in ${VAMP_MANIFEST} @ONLY) file(GLOB_RECURSE VAMP_GENERATOR_INPUTS CONFIGURE_DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/data/*.xml - ${CMAKE_CURRENT_SOURCE_DIR}/data/expected/*.json + ${CMAKE_CURRENT_SOURCE_DIR}/tests/fixtures/performance_summary.json ${CMAKE_CURRENT_SOURCE_DIR}/data/composition/*.json ${CMAKE_CURRENT_SOURCE_DIR}/generator/*.py ${CMAKE_CURRENT_SOURCE_DIR}/generator/**/*.py diff --git a/conanfile.py b/conanfile.py index 625f24e..a9e9215 100644 --- a/conanfile.py +++ b/conanfile.py @@ -29,7 +29,6 @@ class product_vamp_conan(conan.ConanFile): "modulus.py", "modulus.json", "cmake/*.cmake", - "data/expected/*.json", "data/songs/*.xml", "data/rhythms/**/*.xml", "data/scales/**/*.xml", diff --git a/data/composition/roles.json b/data/composition/roles.json index 29179b5..dda0f14 100644 --- a/data/composition/roles.json +++ b/data/composition/roles.json @@ -5,18 +5,11 @@ "roles": { "acoustic_solo": { "seed": "acoustic", - "register_center": 57, - "selection_weights": { - "duration/1": -1, - "space/after_arrival": -2 - } + "register_center": 57 }, "electric_solo": { "seed": "electric", - "register_center": 62, - "selection_weights": { - "metric/eighth/1_2": -1 - } + "register_center": 62 }, "acoustic_accompaniment": { "seed": "acoustic_comp", diff --git a/data/composition/solo.json b/data/composition/solo.json index 8a1db97..4a7e377 100644 --- a/data/composition/solo.json +++ b/data/composition/solo.json @@ -1,6 +1,6 @@ { "version": 1, - "provenance": "General musical operators authored from chord-tone, scale-motion, rhythmic grouping and target-resolution principles. Shared ranking may be fitted in explicitly invoked development commands; no stored reference phrases or song-specific prescriptions are permitted.", + "provenance": "Independently authored general melodic phrase, rhythm and search policy. No reference-derived patterns or fitted parameters.", "license": "MIT", "cells": [ { @@ -171,21 +171,6 @@ "answer" ] }, - { - "id": "balanced", - "values": [ - "1", - "1/2", - "1/2" - ], - "purpose": [ - "arpeggio", - "scalar", - "develop", - "answer", - "neighbor" - ] - }, { "id": "offbeat_turn", "values": [ @@ -224,19 +209,6 @@ "neighbor" ] }, - { - "id": "triplet_motion", - "values": [ - "1/3", - "1/3", - "1/3" - ], - "purpose": [ - "scalar", - "develop" - ], - "difficulty": "triplet" - }, { "id": "sixteenth_turn", "values": [ @@ -250,61 +222,11 @@ "neighbor" ], "difficulty": "sixteenth" - }, - { - "id": "single_step", - "values": [ - "1/2" - ], - "purpose": [ - "arpeggio", - "scalar", - "develop", - "answer", - "neighbor" - ] - }, - { - "id": "quarter_step", - "values": [ - "1" - ], - "purpose": [ - "arpeggio", - "scalar", - "develop", - "answer", - "neighbor" - ] } ], - "operators": { - "durations": [ - "1/4", - "1/2", - "1", - "3/2", - "2", - "5/2", - "3", - "7/2", - "4" - ], - "approach_holds": [ - "1/2", - "1", - "3/2", - "2", - "5/2", - "3", - "1/3" - ], - "per_state": 24, - "connection_preparations": [ - "1/2", - "1" - ], - "histories": 16, - "completed": 8 - } + "search": { + "alternatives": 24, + "histories": 4 + }, + "subdivision_groups": 1 } diff --git a/docs/composer/active_path_audit.md b/docs/composer/active_path_audit.md deleted file mode 100644 index 9940361..0000000 --- a/docs/composer/active_path_audit.md +++ /dev/null @@ -1,64 +0,0 @@ -# Active selector audit - -Starting revision: `ef62af892bce168e2e9846dc6035b414f55d8959`. -Previous results are preserved in [baseline_ef62af8.md](baseline_ef62af8.md). -The current candidate baseline is under `build/solo_reference_comparison/structured/baseline`; -previously published assets were not replaced. - -Before: `production.generate_role → composer.compose → Builder.compose → greedy action cost per quarter → occasional dead-end backtrack → reporting phrases → placement → validation/export`. - -After: `production.generate_role → composer.compose → Builder.compose → common-time multi-history frontiers → complete sounded-event evaluation → whole-solo winner → placement → validation/export`. - -| Finding | Reproduced evidence | Change | -|---|---|---| -| Immediate commitment | DFS continued the first cheapest action; alternative histories were visited only after failure | Multiple histories survive to equal-time checkpoints; whole completed candidates compete | -| Segmentation-dependent objective | Prefix total summed individually normalized action rates | One canonical event-stream evaluation; notation ties merge and rest partitions disappear | -| Constructor-family bonuses | Acoustic `kind/arpeggio`, `kind/answer`; electric `kind/scalar`, `kind/enclosure` | Removed from data and rejected by fitted-feature schema | -| False arpeggio label | Constant scale-degree stride ±2 | Renamed `scale_thirds`; actual arpeggios walk members of the written chord | -| Premature home identity | First four eligible recent notes | Home comes from the completed opening source window and survives separately from the last-eight-note context | -| Prefix fitting | Early-update perceptron optimized first mismatch | Replaced by shared bounded integer constraints from complete outputs | -| Coverage proof lost obligations | Constrained derivation merged histories solely by time | Stateful diagnostic retains pending contracts and validates the complete resulting stream | -| Repeated expensive work | One-bar cProfile spent most cumulative time in action enumeration/validation | Cached note/route calculations; indexed sequence licensing; deferred full validation with replacement of rejected proposals | - -Budget: 16 histories, 24 valid successors per history, eight completed exports. -Retrieval ranks within equal-endpoint buckets, then round-robins endpoints. It does -not compare variable-duration actions by their individual cost rates. Frontier -scores cover the same heard span. A selected action may extend beyond a checkpoint; -its actual unsplit notes, route and obligations remain stored. Computational -quarter-note checkpoints and four-source-measure home selection are not forced -cadences. Final phrase reporting does not insert notes or endings. - -Each destructive frontier records proposed/distinct/retained counts. A read-only -development observer can compare those cuts with a training example; its return -value cannot change selection. Absence from the final eight is not proof of -unrepresentability. The ordinary build never installs that observer or imports -a training/reference reader. - -This is bounded phrase/chorus comparison, not globally optimal search. Early cuts -can still lose a desirable complete phrase. Opening diversity is retained in -addition to recent rhythmic diversity; it does not guarantee eight aesthetically -different or convincing tunes. - -## Reproduction failure observed in the current candidate - -The optional `structured/exact-opening-pruning.json` probe observes the ordinary -first successor cutoff without changing decisions. All Me acoustic has one exact -compatible action among 9,853 and retains none; electric has two and retains none. -Autumn acoustic has two among 8,492 and retains none; electric has nine and retains -none. Every case retains 24 successors. All four therefore fail at retrieval, -before complete-output selection. The earlier `opening-pruning.json` compared -clipped audible prefixes and counted a shortened hold as surviving. That weaker -diagnostic was corrected: committed attacks must match their complete durations. -Separate stateful constrained derivation reaches all four complete targets with -no diagnostic history pruning; that establishes construction coverage only. - -Full generation also confirms a ranking failure: all four complete targets score -worse than their selected outputs under the default evaluator. Exact totals and -contribution differences are in `evaluation-diagnosis-all.json`. Eight surviving -complete alternatives are not proof that the necessary alternatives survived. - -The first full regression run also exposed optional-color lookahead parsing an -explicit `N.C.` span as a chord. `eligible_color` now rejects missing/no-collection -harmonic spans before parsing. The current production sources' affected contexts -are recorded separately so a build started before that repair is not silently -described as a build from identical code. diff --git a/docs/composer/baseline_0e68f53.md b/docs/composer/baseline_0e68f53.md deleted file mode 100644 index bd0c7cf..0000000 --- a/docs/composer/baseline_0e68f53.md +++ /dev/null @@ -1,84 +0,0 @@ -# Complete-phrase solo rewrite - -Starting revision: `68df7292ee85918a0cdbd0965139ccb2ceecb130`. This revision replaces the active solo constructor. Source import, harmony, exact timing, typed expression validation, independent roles, guitar placement, notation, playback and atomic asset publication remain in use. - -The selected direction is a new melody over the source changes, with optional head references. The source head no longer supplies a compulsory rhythmic prefix or protected initial hold. The four supplied solos remain outside production and unchanged; exact reproduction or listener preference is not claimed. - -## Construction and simplification - -The normal path is `production.generate_role → composer.compose → construction.Builder → rhythm.Rhythms/routes.Routes → scoring.evaluate → placement → validation → notation/export`. - -Each proposal chooses a phrase purpose, rhythm groups, direction, harmonic destination, arrival timing, sounding duration and optional treatment together. Complete pitch paths realize those choices. Open continuations need no chord-tone stop; settling, completion and handoff have actual destination contracts. Anticipations, approaches, enclosures, connections, colors, blues treatments and prepared suspensions enter before route selection and must validate as complete relationships. - -An introduced realized phrase supplies the home idea. Development, answer, deliberate return and optional head reference are distinct reachable actions. A return protects a short heard rhythm and directional signature with harmonic adaptation; ordinary phrases have no compulsory three-note prefix. This signature is not a perceptual guarantee of recognition. - -One versioned evaluator reports purpose, incoming continuity, contextual repetition and broad register costs. Physical travel selects fingering, not the winning tune. One-phrase lookahead averages normalized completed-phrase scores across variable durations; it remains a bounded heuristic, not a whole-chorus optimum. Route construction has its own explicitly separate heuristic. - -The solo package decreases from **6,232 to 1,729 Python lines**, and from 30 to 19 modules. Thirteen obsolete constructors/helpers were removed and two focused modules added. Old plan/draft/paraphrase API tests were retired; retained musical and representation contracts were migrated. Removed tests are not counted as passing tests. No legacy production fallback remains. - -## Inputs and data - -The 21 scales, 2,954 sequence paths and 3,538 rhythm snippets remain intact. `solo.json` contains independently authored parameterized actions, nine atomic rhythm groups and migrated authored cells. `grammar.json` and `melodic_vocabulary.json` were retired after their consumers migrated. Accompaniment fields and resources were not changed. - -References: `build/all-of-me-solo-1.xml`, `all-of-me-solo-2.xml`, `autumn-leaves-solo-1.xml`, `autumn-leaves-solo-2.xml`. Solo 1 maps to acoustic; Solo 2 to electric. The optional reader applies the confirmed guitar-written octave convention and preserves raw evidence. Autumn Leaves aligns at the first Gm7 in reference measure 2; its preceding measure remains in the comparison. Four Autumn electric reference events extend beyond the source-defined vocal allocation. Neither the target nor the vocal timing is changed to force equality. - -All six strings remain available on frets 1–12 without open strings. Ties retain one sounding event and one position. No performance-expression changes were introduced. The existing independently generated accompaniment, electric melody and vocals are intentionally preserved by this solo rewrite. - -## Verification and review - -The final Python command was the repository Python runner followed by `-m unittest discover -s tests -p 'test_*.py'`: **435 tests passed in 218.783 seconds**. During implementation, repeat-build tests caught profiling timings leaking into stable diagnostics. Timings now live only in the separate build profile; the final suite includes those repeat-build checks. Earlier interrupted/stale-import runs are not counted as passing verification. - -The explicit dependency command passed all six roles on independently authored input: alone/cold, reverse/warm, parallel/cold, invalid optional targets present, targets removed, and after an electric-only setting change. Actual reads and canonical document hashes are retained. This synthetic test is not a whole-song quality assessment. - -The optional transfer command passed all 18 role builds on evaluation-only four-bar excerpts from After You've Gone, Hindustan and Indiana. Their provenance remains under `tests/evaluation_sources/`. This demonstrates limited transfer; it does not establish arbitrary meter, form or chord support. - -The comparison page includes full generated, previous and reference solo audio, common-backing mixes and opening excerpts. The same neutral renderer and constant comparison gains are used. Comparison reads happen only after generation. No ear-based audition or listener approval is claimed. - -## Evidence locations - -- [Active path and findings](active_path_audit.md) -- [Resource/code migration and provenance](data_migration.md) -- [Evaluation contract and search limitation](evaluation_contract.md) -- [Capability matrix](musical_capability_matrix.md) -- [Independent-input verification](dependency_proof.md) -- [Reproduction commands](commands.md) - -`build/solo_reference_comparison/` is the sole comparison location. `previous/` retains the immediate pre-rewrite output and input hashes; `generated/` contains the replacement. The complete operation/pruning trace and local/next-phrase evaluations are in the manifest. `analysis/selected_phrases.md` provides readable selected events, destinations, holds, expression evidence and alternative scores. - -## Remaining boundaries - -The reference solos are outside-review examples, not a numerical loss. The replacement output remains new music, not those four scores. Counts and matching attacks do not establish greater musical quality. - -The search retains a small frontier and limited lookahead. It can choose another complete phrase, but does not perform arbitrary two-sided revision of committed music. Some exact thematic returns may still sound too weak; the selected rhythm and contour contract is less strict than exact intervals. Acceptable candidates can share a zero musical penalty, leaving the stable role key to break ties. - -Unsupported written harmony produces an explicit diagnostic. Optional expression rules do not repair ambiguous source chords or authorize arbitrary chromatic notes. Legal TAB and coarse tempo limits are not full biomechanical certification. The next useful listening check is the four complete comparisons: whether their openings establish memorable ideas and whether later returns remain recognizable through the freer continuations. - -## Published assets and measured cost - -Full assets are published at `build/build_x86_debug/generated_assets/`; the four exported solos are in `solo_xml/`. All twelve musical role documents match between cold staged generation and cached publication. The eight non-solo documents are exactly equal to the preserved baseline. Every exported solo sounding event and TAB position matches the selected stream. All committed holds and typed harmonic contracts passed validation. All six source/reference file hashes are unchanged. - -| Solo | New attacks | Sounding MIDI range | Occupancy | -|---|---:|---|---:| -| all_of_me/acoustic | 117 | 47–67 | 0.781 | -| all_of_me/electric | 139 | 52–72 | 0.824 | -| autumn_leaves/acoustic | 153 | 47–67 | 0.859 | -| autumn_leaves/electric | 109 | 52–70 | 0.732 | - -These descriptive measurements are not targets or proof of musical improvement. None of the four generated solos exactly matches its reference. All four normalized references have legal pitch/position paths within the existing guitar constraints; the Autumn electric vocal-allocation conflict remains explicit. - -The native integration test `ctest --test-dir build/build_x86_debug -R '^product_vamp$' --output-on-failure` passed in **30.45 seconds**, with required localhost access. The optional comparison adapter suite passed **8 tests in 0.018 seconds** after the audio-directory update. All **36 comparison WAVs** passed PCM clipping checks, and all **50 page links** resolved. The signal review is not an ear-based audition. - -The isolated final staged asset build took **107.15 seconds** wall time, including **91.31 seconds** in the performance-part stage. All twelve role caches missed. Two composition workers ran in separate processes; no tests or audition rendering overlapped. Engraving was warm and reused all six scores. The **27.57-second** normal CMake publication reused all twelve role caches and six engraving results. - -| Cold solo role | Role seconds | Construction seconds | Route seconds | -|---|---:|---:|---:| -| all-of-me/electric_solo | 40.82 | 26.27 | 21.47 | -| all-of-me/acoustic_solo | 44.38 | 29.96 | 24.90 | -| autumn_leaves/electric_solo | 35.56 | 33.09 | 28.31 | -| autumn_leaves/acoustic_solo | 44.04 | 41.71 | 35.80 | - -Worker times overlap. Role time includes resource setup, notation and validation beyond construction. Route time is included in construction, not an additional cost. Hardware/process evidence: 12 logical CPUs, Linux x86-64, repository Python 3.13; `build-profile.json` contains actual reads, worker settings and stage times. The first worker use still spends time loading the full catalogs. Route search remains the dominant construction cost. - -An isolated one-process All of Me acoustic build took **39.80 seconds** without cProfile. The diagnostic run before the performance-only optimization took 163.09 seconds **with cProfile enabled**, so those times are not a valid speedup ratio. Profiling exposed repeated pitch hashing/stringification and harmony lookups. Memoization and hoisted identical calculations preserved the complete musical document hash `2b6b35011e353d166d287a8bfff6074723e62559b5e64ff978084623e1520e67`, also matching final production. The rewrite changes music and search structure; its timing is not presented as a semantics-preserving speedup over the old composer. - -Cleanup removed **151.2 MiB** of superseded comparison audio, stages, probes and temporary rendering tones during this rewrite. The immediate baseline, four user references, build toolchains, caches, published assets and useful verification evidence remain. Current audio is consolidated under `audio/` and linked from one `index.html`; `cleanup.json` records removed paths. diff --git a/docs/composer/baseline_ef62af8.md b/docs/composer/baseline_ef62af8.md deleted file mode 100644 index c2db789..0000000 --- a/docs/composer/baseline_ef62af8.md +++ /dev/null @@ -1,76 +0,0 @@ -# Exact-target composer repair — release status - -**Not released. Exact production reproduction of the four targets has not been achieved.** Valid derivations exist for all four complete event streams, but a target-constrained derivation is not an unrestricted production winner. The existing published assets remain intact. The previous release's report is [archived here](baseline_0e68f53.md); its tests and performance numbers do not apply to this rewrite. - -Starting revision: `0e68f5364bb0196ab1b7087ecc186f5d7446c42b`. Preserved baseline: `build/solo_reference_comparison/repair/baseline/`. Its `inputs.json` records source/data/reference hashes and initial cache state. The sources remain `data/songs/all-of-me.xml` and `data/songs/autumn_leaves.xml`; no archive source has replaced either. - -## Acceptance and target identity - -The exact target files are `build/all-of-me-solo-1.xml`, `build/all-of-me-solo-2.xml`, `build/autumn-leaves-solo-1.xml`, and `build/autumn-leaves-solo-2.xml`. Solo 1 is acoustic; Solo 2 is electric. They are untouched. The optional adapter applies the confirmed guitar-written octave convention and merges ties while preserving repeated attacks. Autumn aligns at the first Gm7 in measure 2, including the entire preceding measure at negative chorus-relative times. All of Me aligns at measure 1. - -The latest approved development scope allows target-guided general-rule work and shared-coefficient fitting, but prohibits stored tunes, per-song recipes and runtime target reads. No fitted model is promoted automatically. Optional development tools and their artifacts remain outside production imports and cache identities. - -The baseline exact comparison returned failure: All of Me acoustic/electric had 0/3 exact matching attacks; Autumn acoustic/electric had 0/0. Differing onset positions were 205, 201, 211 and 178 respectively, and rests differed in all four. These are exact-event diagnostics, not musical-quality scores. - -## Changes in this working tree - -The hash-preselected phrase constructor and separate route solver are replaced by enumerable joint musical actions, one interpretable rational evaluator and bounded deterministic backtracking. Actions include ordinary notes/rests, complete degree routes, realized-theme returns/answers, colors, approaches, enclosures and anticipations. Pending treatments carry their actual future targets and deadlines across actions. Whole-line guitar placement follows selection; it cannot change the tune or inspect another generated part. - -General expressive rules now cover delayed exact-target approaches, separated enclosures, reattacked anticipations and anticipated colors. Color octave enumeration no longer loses alternating F-sharp registers through rounding. The base scale vocabulary remains the same 21 collections; exceptions have typed, bounded contracts. Full target endings and source-derived pickup allocations are supported. Electric may overlap the returning vocal pickup as authorized; source vocal events remain unchanged. - -Performance changes remove redundant parent catalog parsing, add content-addressed non-executable catalog JSON, memoize pure chord/color/spelling checks, and avoid building full rational contribution reports for thousands of rejected candidates. The lightweight ranking total and detailed evaluation are equal and tested. Existing independent process workers remain available. - -The development fitter uses only shared whitelisted musical features and integer updates. An initial trial exposed two defects: unordered source-interval evidence and per-attack rewards that favored subdivisions. Ordered source relationships and duration-normalized rewards repaired them. A later trace exposed indistinguishable pending targets; their relative delay, step and hold now enter selection explicitly. Interrupted diagnostic trials are not reported as completed fitting or benchmark results. - -## Evidence and limitations - -The staged build completed with `--check`: six performance scores, complete solo XML, notation/TAB, playback assets, and independent accompaniment for both current sources. It is under `build/solo_reference_comparison/repair/staged`, not the published asset directory. The exact gate returned exit code 1: - -| Output | Generated / target attacks | Exactly matching attacks | Differing onset positions | Exact result | -|---|---:|---:|---:|---| -| All of Me acoustic | 118 / 137 | 2 | 189 | Fail | -| All of Me electric | 128 / 141 | 0 | 232 | Fail | -| Autumn Leaves acoustic | 106 / 121 | 1 | 177 | Fail | -| Autumn Leaves electric | 143 / 115 | 5 | 195 | Fail | - -Rests differ in every case. These results are not close reproduction, and attack counts are not a substitute for matching the requested music. The full comparison page is `build/solo_reference_comparison/repair/comparison/index.html`. It includes untouched references, the baseline and staged XML, complete solo-only and fixed-backing WAVs, opening excerpts, and exact note differences. Rendering used one neutral treatment and constant shared gains; no ear-based audition was performed. - -The actual selected events and TAB were checked against all four reparsed exports: all agree. Thus these discrepancies arise in composition/selection, not from an exporter shifting the selected melody. Source vocal and electric head documents are unchanged from the baseline. Accompaniment documents changed with the shared difficulty configuration and the source-derived participation allocation; they are still independently generated. Source and reference SHA-256 values, and the previously published solo exports, remain unchanged. Evidence: `repair/asset-verification.json`. - -The fitting trial was stopped after **40 completed passes of its configured 200-pass maximum**, without convergence. Every pass still had four incorrect rollouts; the best had only eight matching prefix attacks in total across the four cases. This is an interrupted diagnostic trial, not a completed 200-pass result and not a proof that every shared model is infeasible. Its best coefficients remain development-only under `repair/fitting`; none were copied to production data. - -A further normal-production check of that best checkpoint completed All of Me acoustic: **5 matching attacks, 223 differing onset positions, and different rests**; it also failed the exact gate. The other fitted-model jobs were interrupted without completed exports. This is not a four-case fitted-model release test. Its completed XML and comparison are in `repair/fitted` and `repair/fitted-comparison`; the interruption is recorded in `repair/fitted-build-status.json`. The complete four-case results above use the unfitted production defaults. - -The default selection trace explains an important failure. All of Me acoustic selected 29 arpeggio actions and six returns; electric selected 32 scalar actions, nine returns and one ordinary tone. No standalone rest won in either output. Many legal expressive candidates were proposed and rejected by score. The role's action-kind preferences overwhelm much of the vocabulary, and the local policy has no complete-phrase destination comparison. Adding legal actions has not repaired that selection limitation. - -### Commands actually validated - -All commands use `build/build_common/python/bin/python3 build/build_tools/python_runner.py build/build_common/modulus` as their prefix. Exact invocations are in [commands.md](commands.md). - -- `-m unittest discover -s tests -p 'test_*.py'`: **446 tests passed in 348.905 seconds** (`repair/verified-tests.log`). -- `-m unittest tests.test_solo_comparison`: **9 tests passed in 0.083 seconds** after removing an invalid optional report link (`repair/comparison-tests.log`). -- `-m audition.dependency_check --output .../dependency-final`: passed all six roles, including an operational electric register change that changed that role's document and left the other roles intact. -- Full staged `-m generator ... --check --jobs 2 --composition-jobs 2 --export-solo-xml`: passed. The first attempt exposed an unfillable final quarter-beat; a tempo-gated sixteenth note/rest primitive and a regression test repaired that failure before the successful build. -- `-m audition.solo_compare ... --require-exact`: failed all four outputs, as shown above. This is the release-blocking musical result, not a passing check. -- Full-score selected-event/TAB verification and source/reference/published-baseline hash checks: passed (`repair/asset-verification.json`). - -The staged asset run took **430.402 seconds** with two composition workers and warm generic catalog/non-solo caches. Tests and development fitting were running concurrently, so that time is not an isolated speed benchmark. Solo role times were 156.117, 228.292, 182.301 and 174.160 seconds in that run. No cold-generation speedup is claimed from these numbers. The full recorded stage timings and actual read traces are in `repair/build-profile.json`. - -After other generation/rendering jobs stopped, `-m audition.benchmark --source data/songs/all-of-me.xml --role acoustic_solo --cache .../performance/cache --output .../performance/cold` took **134.722 seconds** for the role, plus 0.046 seconds for source analysis, with one worker and an empty concrete/catalog cache. Repeating it with `--output .../performance/warm` took **2.068 seconds** with a role-cache hit. Hardware/runtime: 12 reported CPUs, Linux 7.2.3 x86-64, repository Python environment; cProfile disabled. Cold, warm and parallel-full-build documents have the same hash `adcc372315aa10cae8a956cc5002700859c81cdcf4eeaa59b4ae6f8b41161baf`. Evidence: `repair/performance/summary.json` and the two benchmark records. - -The requested cold-generation reduction is **not achieved**. Warm memoization avoids recomposition, but the wider action enumeration is still expensive. The historical baseline's different music, algorithm and benchmark conditions do not support a controlled speedup ratio. Candidate construction/evaluation remains the dominant recorded stage; no further beam expansion is justified by these results. - -- [Active path, reproduced findings and search limits](active_path_audit.md) -- [Data migration and provenance](data_migration.md) -- [Authoritative evaluation and fitting contract](evaluation_contract.md) -- [Capabilities and unimplemented behavior](musical_capability_matrix.md) -- [Independent-input checks](dependency_proof.md) -- [Reproduction and exact-release commands](commands.md) - -Detailed winning/runner-up contributions, selected note groups and proposal/valid/retained/selected counts are in `build/solo_reference_comparison/repair/decisions.md`. Superseded diagnostic logs are consolidated in `repair/diagnostic-history.zip`; current evidence, the baseline and targets remain intact. - -The search is a local action policy, not a global complete-phrase optimizer. Dedicated suspension/blues constructors and two-sided phrase revision are not present in this replacement; validators remain. Source influence and the short realized home theme are limited models of melodic identity. A catalog containing thousands of paths does not mean all those paths currently compete as complete musical ideas. No new held-out source-standard evaluation set was added. Existing synthetic transposition/meter/harmony tests do not establish generalization to an unseen standard or subjective musical quality. - -The earliest demonstrated remaining failure is selection: legal target-compatible actions lose even at the opening, and later action choices collapse toward a few favored families. The next justified investigation is whether a shared context model can distinguish those conflicting choices over complete phrases, before adding more rules, expanding the beam, or running another long fit. The finite trial here does not settle that question. There is no automatic fallback to reference notes or previous output. - -Technical completion, exact target equality and listener preference are separate. No ear-based audition is claimed. The release remains blocked unless all four unrestricted exported comparisons pass; a valid or stylistically plausible mismatch is not substituted for that requirement. diff --git a/docs/composer/commands.md b/docs/composer/commands.md deleted file mode 100644 index 0002e1a..0000000 --- a/docs/composer/commands.md +++ /dev/null @@ -1,106 +0,0 @@ -# Reproduction commands and release gate - -Prefix from the repository root: - -```sh -build/build_common/python/bin/python3 build/build_tools/python_runner.py build/build_common/modulus -``` - -The commands below go after that prefix. They write only the consolidated repair directory. Existing published assets and the baseline remain separate. - -```sh --m unittest discover -s tests -p 'test_*.py' --m audition.dependency_check --output build/solo_reference_comparison/structured/dependency -``` - -Development-only stateful derivation / complete-output fitting (use `explain` instead of `fit` for coverage alone). Install the optional solver with repository Python and `-m pip install --target build/development_packages -r audition/requirements-training.txt`. Production does not load that directory: - -```sh --m audition.fit_selection fit --passes 8 --solver-path build/development_packages \ - --source all_me_a=data/songs/all-of-me.xml --source all_me_e=data/songs/all-of-me.xml \ - --source autumn_a=data/songs/autumn_leaves.xml --source autumn_e=data/songs/autumn_leaves.xml \ - --reference all_me_a=build/all-of-me-solo-1.xml --reference all_me_e=build/all-of-me-solo-2.xml \ - --reference autumn_a=build/autumn-leaves-solo-1.xml --reference autumn_e=build/autumn-leaves-solo-2.xml \ - --role all_me_a=acoustic_solo --role all_me_e=electric_solo \ - --role autumn_a=acoustic_solo --role autumn_e=electric_solo \ - --form-start all_me_a=1 --form-start all_me_e=1 --form-start autumn_a=2 --form-start autumn_e=2 \ - --cache-dir build/solo_reference_comparison/structured/cache --output build/solo_reference_comparison/structured/fitting -``` - -Production stage (no reference arguments): - -```sh --m generator --data-dir data --output-dir build/solo_reference_comparison/structured/staged \ - --static-assets-dir package/assets --check --jobs 2 --composition-jobs 2 \ - --composition-cache build/solo_reference_comparison/structured/verified-cache \ - --engraving-cache build/build_common/engraving_cache --export-solo-xml --trace-level full \ - --profile-output build/solo_reference_comparison/structured/build-profile.json --progress -``` - -Exact gate after generation: - -```sh --m audition.solo_compare --generated-dir build/solo_reference_comparison/structured/staged/solo_xml \ - --reference all_of_me/acoustic=build/all-of-me-solo-1.xml --reference all_of_me/electric=build/all-of-me-solo-2.xml \ - --reference autumn_leaves/acoustic=build/autumn-leaves-solo-1.xml --reference autumn_leaves/electric=build/autumn-leaves-solo-2.xml \ - --form-start all_of_me/acoustic=1 --form-start all_of_me/electric=1 \ - --form-start autumn_leaves/acoustic=2 --form-start autumn_leaves/electric=2 \ - --reference-pitch guitar-written \ - --baseline-dir build/solo_reference_comparison/structured/baseline/solo_xml \ - --assets build/solo_reference_comparison/structured/staged \ - --output build/solo_reference_comparison/structured/comparison --require-exact -``` - -`--require-exact` returns 1 on any sounding-event/rest mismatch; it does not reinterpret a similar style as success. Add `--assets ` for consistent neutral rendering. A failed fit or exact gate is not permission to copy targets, promote a target-constrained derivation, or publish a mismatched stage. Normal CMake publication is intentionally deferred until the approved acceptance gate passes. - -Individual roles remain available through `-m generator.compose --song ... --role acoustic_solo --data-dir data --cache-dir ... --output ... --solo-xml ...`. Use `-m audition.benchmark --source ... --role ... --cache ... --output ...` for an isolated measurement after other builds/tests/fits stop. Profiling changes measured runtime; do not compare a cProfile run to an unprofiled build as a speedup ratio. - -Source-only four-bar completed alternatives (the XML marked candidate 1 is the automatic winner): - -```sh --m audition.phrase_candidates --source data/songs/all-of-me.xml --role acoustic_solo \ - --bars 4 --cache-dir build/solo_reference_comparison/structured/cache \ - --output build/solo_reference_comparison/structured/four-bar -``` - -No solver output or constrained derivation may replace an ordinary production score. -The exact four-complete-output gate remains mandatory. Reference absence proves a -runtime boundary only; learned coefficients still depend on their training data. - -A completed ordinary build can supply the negative candidates for one learning -round without repeating inference just to reconstruct its report. Use -`-m audition.train_completed` with the same `--source`, `--reference`, `--role`, -and `--form-start` case arguments, plus `--completed-dir `, -`--solver-path build/development_packages`, and `--output `. -`--stage-data` creates a separate production-data copy with the proposed shared -weights. It does not edit `data/` or published assets. Feed that copy to the normal -`-m generator --data-dir ...` command for a new cold target-free build, then run -the complete four-output exact gate. A solver that separates eight saved -candidates has not demonstrated exact generation. - -For later complete-output rounds, add `--previous-round ` -to retain prior counterexamples. Incompatible source/example identities, -evaluation versions or fixed musical policy are rejected. Only learned weights -may change between compatible rounds. Do not fit only the latest eight outputs -and silently forget the earlier failures. - -The first proposed model's ordinary trial uses the same production command with -`--data-dir .../round-1/data --output-dir .../learned-stage`, separate -`--composition-cache .../learned-cache`, `--composition-jobs 4`, and -`--profile-output .../learned-build-profile.json`. This process receives no -reference arguments. Comparison and learning are separate later commands. - -The completed second fit used `--data-dir .../round-1/data`, -`--completed-dir .../learned-stage`, -`--previous-round .../round-1/evidence/training.json`, and -`--output .../round-2`, with the same four source/reference/role arguments. -It accumulated 64 full-output inequalities. Its successful solver status is -not a release: the acoustic openings still fail the strict cutoff diagnosis, -and no third complete ordinary rollout was performed. - -Final focused tests were run with `PYTHONPATH=build/development_packages` and -`-m unittest tests.test_joint_actions tests.test_structured_selection` (35 passed). -The latest dependency check uses `--output .../dependency-cached`. -The read-only staged verification script also accepts -`--stage learned-stage --data-dir .../round-1/data --prefix learned-`. -All comparison artifacts are linked from `structured/index.html`. diff --git a/docs/composer/data_migration.md b/docs/composer/data_migration.md deleted file mode 100644 index 448e01a..0000000 --- a/docs/composer/data_migration.md +++ /dev/null @@ -1,52 +0,0 @@ -# Data and consumer migration - -The latest approved scope permits target-guided diagnosis of general rules and fitting shared theory coefficients. Production target reads, copied tunes, per-song recipes and hidden phrase banks remain prohibited. Policy provenance records this development influence; these four targets are not an unseen evaluation set. A development weight artifact is not automatically a production resource. The first table records the inherited migration at the starting revision; the selector-repair section below identifies this iteration's changes. - -| Path | Action | Live consumer / reason | -|---|---|---| -| `data/songs/` | Keep untouched | Authoritative source melody, harmony, meter and form | -| `data/scales/` | Keep 21 patterns | Ordinary mapped movement; no added scale family | -| `data/sequences/` | Keep 2,954 paths | `Operators` licenses exact repeated degree strides; other paths remain available but are not all active choices | -| `data/rhythms/` | Keep 3,538 snippets | Whole coherent meter-specific representatives; file count does not determine frequency | -| `composition/rhythms.json` | Keep | Authored melodic cells and canonical sparse comping | -| `composition/solo.json` | Migrate | Keep cells/groups; remove hash-selection menus and dead scorer settings; add action durations and search budgets | -| `composition/harmony.json` | Extend | Stable/resolving colors; mapped half-diminished natural 13 only before a functional dominant | -| `composition/lead_ins.json` | Extend | Whole approaches, separated enclosures, complete subdivisions, delayed exact-target resolution | -| `composition/roles.json` | Migrate solo fields | Soft registers and shared action weights replace unused menus; comping fields retained | -| `data/expected/` | Keep validation-only | Arrangement compatibility, never selection | -| Four XML files in `build/` | Keep untouched | Explicit development derivation/fitting and downstream comparison | -| `build/.../repair/fitting/` | Development artifact | Not loaded by production; failed fits are not promoted | -| `/_resources/` | Add derivative | Checksummed JSON production catalogs keyed by content and decoder code; no song/role/reference result | - -`routes.py` and `Rhythms.realize` are retired. `scoring.py` retains the evaluation record; `selection.py` is the sole scorer. `operators.py` owns enumerable actions. There is no legacy fallback. Deleted-API tests were replaced by active-constructor tests, not counted as passing removed tests. - -The formulas are general relative intervals and durations, not extracted phrases. Supporting discussion: [Berklee tension and resolution](https://www.berklee.edu/berklee-today/summer-2014/tension-and-resolution), [available tensions](https://college.berklee.edu/berklee-today-55), [Locrian natural six](https://www.jazz-guitar-licks.com/pages/guitar-scales-modes/modes-of-the-harmonic-minor-scale/the-locrian-natural-6-mode.html). No example melody from these sources was imported. Numerical bounds are project choices, not universal jazz rules. Authored resources retain MIT licensing. - -Cold-build independence cannot establish provenance alone; review policy/code diffs too. No source score, target score, tuning, fret limit, runtime score schema, velocity or playback feel was changed. - -## Selector repair at ef62af8 - -No scale, sequence, rhythm, harmony, lead-in, source-song or accompaniment vocabulary -was added or removed in this step. The live solo budget now names `histories=16`, -`per_state=24`, and `completed=8`; the obsolete `backtracks` setting is removed. -Role-level constructor-family weights were removed. Existing duration/metrical -preferences remain. Both roles still use all six strings and frets 1–12. - -`Operators` now distinguishes scale thirds from written-chord arpeggios. Sequence -licensing is indexed once. Harmonic interval permissions, note objects and color -construction are memoized without changing their underlying theory rules. -Expensive validation can be deferred past cheap retrieval, but a candidate must -pass before it advances. The ordinary `Operators.at` diagnostic still returns -fully validated actions. - -The old prefix-perceptron implementation was removed from `audition/fit_selection.py`. -Its replacement is `audition/structured_learning.py`, with the pinned optional -solver in `audition/requirements-training.txt`. Neither is imported by production. -`audition/phrase_candidates.py` exports ordinary completed alternatives without -accepting any target-file arguments. Failed development models remain under -`build/solo_reference_comparison/structured`; none is automatically copied to data. - -`audition/train_completed.py` can form complete-output ranking constraints from -ordinary finished candidate reports. It verifies the source, role, vocabulary, -and evaluation version, and records explicit training dependencies. Its optional -staged data copy is for a subsequent ordinary cold build, not automatic promotion. diff --git a/docs/composer/dependency_proof.md b/docs/composer/dependency_proof.md deleted file mode 100644 index a2c2faf..0000000 --- a/docs/composer/dependency_proof.md +++ /dev/null @@ -1,37 +0,0 @@ -# Independent inputs and reproducibility - -Production consumes canonical source music, relevant production data, role settings and `phrase-frontiers-2`. References, training outputs and prior published parts are not runtime musical inputs. Any coefficients learned by the separate development commands explicitly depend on their training examples; that is not complete independence from those examples. No learned proposal has been promoted. - -`input_audit.composition` records actual reads and rejects paths outside production data, generator code, Python, the current role's own concrete cache and the checksummed immutable `_resources` catalog cache. Other roles' concrete caches are prohibited. The optional training/comparison modules are not imported by production. - -## Current exercised boundary - -`audition.dependency_check` passed for an independently authored one-bar source with real production catalogs and all six roles: - -- Each role alone with cold caches, then reversed order with warm caches. -- Two spawned processes with cold caches. -- Deliberately invalid optional targets present, then absent, with cold generation. -- An electric-only register change that changed the electric output, followed by unchanged hashes for the original settings of every role. - -Evidence and actual read paths are in `build/solo_reference_comparison/structured/dependency-cached/dependency.json`. All six cases equal the preceding `dependency-final/dependency.json` and `dependency/dependency.json` runs. Canonical musical document hashes: - -- `acoustic_accompaniment`: `45658c437f6a19991246a4dcf2966d0de90ace8bbb71deae7899614ad12dc6da`. -- `electric_accompaniment`: `89bbf33108dfc428e59a141414059aa9ec8a23d6efd00e842adbeba32913d7b8`. -- `electric_melody`: `b8d18ef6fd65b204bd2fc2b69e660068890cb30aea6db827c3f3b506284ea2a1`. -- `vocal_melody`: `e3348117b9c63771cfa0439f276bdadf0d26dd6e560c9676c97e12a4a713737b`. -- `acoustic_solo`: `84328ecb944f52ab5cdd16b143546831116d9c83edefbe307e16af967e785d8e`. -- `electric_solo`: `c9741203d6d4a9df7f4d3a83b0bca72744c657bd49bc41a80842fb25ec468a07`. - -Changed electric configuration: `146b3efa50acd7db14f40d61637bbb55f1feb456abfa7f29750ca1133218743c`. - -This establishes the exercised dependency boundary, not whole-song aesthetic quality or exact reproduction. It is not a four-full-song cold/warm timing benchmark. The ordinary regression suite also exercises source-melody changes, irrelevant title/ID changes, role settings, tuplets and canonical accompaniment reuse. - -The two complete production builds provide an additional whole-song check: -changing the shared solo coefficient tables changed both solos but left all eight -non-solo role documents identical across both songs. Canonical hashes are in -`structured/full-song-role-independence.json`. This does not replace the standalone -role tests or establish exact reproduction; both full models failed that gate. - -New model feature schemas prohibit song names, titles, file paths, hashes, absolute bar indices, arbitrary predicates and event-sequence banks. This bounds the induction representation but cannot by itself prove that every possible fitted model generalizes. Training provenance and held-out evaluation remain necessary. - -The earlier local-policy dependency and full-song benchmark results belong to the archived baseline report. They must not be quoted as current whole-song equivalence evidence. diff --git a/docs/composer/evaluation_contract.md b/docs/composer/evaluation_contract.md deleted file mode 100644 index bc8106e..0000000 --- a/docs/composer/evaluation_contract.md +++ /dev/null @@ -1,107 +0,0 @@ -# Canonical phrase evaluation and shared learning - -`Selector.complete(actions, home, start, end)` is authoritative. It concatenates -selected sounding events, merges only explicit ties, reconstructs silence, and -calls the versioned `phrase-selection-2` evaluator. Its exact rational total is -the sum of its reported contributions. Constructor labels earn no reward. -The same sounded music and home identity have the same complete evaluation -regardless of action partitioning. Repeated attacks remain different from ties. - -The record exposes raw normalized values, weights, contributions, total, span and -attack count. Common-time frontiers score only the heard portion; the remaining -route is preserved in search state. Pending future treatment features are not a -claim that a resolution has already sounded. Completed alternatives cover the -same allocation. Retrieval costs are explicitly heuristics, not published -whole-phrase scores. Expensive validation runs before a retained successor may -advance; rejected proposals are replaced from their endpoint bucket. - -General priors concern large leaps, a soft register corridor, oscillation, -repeated attacks, interruption, ordered source relationships, realized-theme -recurrence, sustained chord-tone arrival, and directed motion. Learned features -may describe duration, relative metrical placement, chord quality/relative pitch, -relative motion, register band, source rhythm, theme recurrence, silence context, -and bounded pending obligations. Silence has no fabricated register. Missing -harmony covers only the actual unharmonized interval, not the rest of a phrase. - -## Development learning - -`audition.fit_selection fit` uses completed ordinary source-only rollouts. -`audition.structured_learning` adds shared within-role ranking inequalities and -uses pinned development-only `z3-solver==4.13.3.0`. Coefficients are integers within -±1024. Objectives minimize nonzero coefficients then total absolute magnitude; -solver work uses a deterministic resource budget. Exhaustion returns no model. -An unsatisfiable core means incompatibility within this schema/bound, not a proof -that no generic musical model could reproduce the examples. - -`audition.train_completed --previous-round` retains prior complete-output -constraints. It verifies source/example hashes, role, evaluator, semantic -algorithm, fixed priors and production data with learned weights excluded. -Only changing weights or their provenance permits reuse; changing the underlying -musical conditions requires a new, explicitly identified training experiment. -Duplicate inequalities are removed. No previous generated part becomes a -production input: these are development-only learning records. - -Before combinatorial optimization, the fitter checks exact integer intervals for -zero or one nonzero coefficient per independent role. If all roles admit these, -enumerating every allowed feature proves the minimum support and its minimum L1 -solution directly. This repaired a measured solver-budget exhaustion on a small -feasible problem; it changes the solver strategy, not the acceptance gate. -The post-fit first-successor diagnostic can disprove full reproduction when no -compatible opening survives. Surviving that check cannot prove reproduction. -Compatibility requires the complete committed attack duration. A clipped match -to the first portion of a target hold does not count as surviving reproduction. - -Only demonstrated identical-feature conflicts may request a new conjunction of -two or three whitelisted conditions. Conditions are chord quality, root-relative -degree, relative motion, metrical category, duration, and ordered source evidence. -No source ID, title, path, hash, absolute measure, target sequence, arbitrary -predicate, or per-song branch is admitted. Constructor-family weights are rejected. -Conjunction semantics have independent positive, negative and transposition tests. -Any actually induced rule is recorded as training-dependent; no untested learned -rule or solver result is promoted automatically. - -The training examples may affect coefficients and induced rules. That is an -explicit training dependency. Production target-file absence does not prove -independence of learned knowledge from those examples. Runtime composition uses -plain validated coefficient data, not Z3, target paths or a phrase-retrieval bank. - -The previous prefix fitter is retired. Target-constrained derivation remains only -a stateful coverage probe, never a generated winner. A completed training iteration -is not the release gate: separate cold ordinary role builds and full exported -sounding-event/rest equality are still mandatory for all four authored targets. - -## Exact performance shortcuts - -`Selector.extend` updates sufficient statistics by replacing the affected tail -and seam, retaining enough preceding context for source interval pairs, -oscillation and a complete possible home-theme match. A newly established home -or explicit tied continuation takes the complete-evaluation path. Pending-target -features are recalculated for the new horizon. Regression tests compare every -incremental record against canonical complete evaluation. - -Retrieval caches internal route contributions but recalculates incoming pitch -relationships and the meaning of a leading rest. Weights are immutable validated -mappings; changing the mapping invalidates weighted caches. Past-frontier concrete -candidates are evicted after all incoming histories have been considered; selected -events remain in immutable search histories. These cache lifetimes are not musical -inputs or wall-clock search budgets. - -Theme signature equality short-circuits on the first actual differing feature; -home signatures are cached while retaining their immutable home objects. The -comparison is exactly equivalent to normalized onset/duration/relative-pitch -tuple equality. It does not approximate thematic similarity or change scores. - -Unconditional retrieval combines cached unary, transition, gap and pending-target -costs directly instead of allocating full feature reports for every rejected -action. Tests compare the result with the authoritative feature calculation -across all action families, incoming histories and weight changes. Learned joint -conditions and explicit tied fragments take the complete feature path; no missing -contextual term or altered tied duration is silently dropped by the shortcut. - -Repeated endpoint retrieval is memoized only for the same immutable action -catalog, exact frontier, last three pitches/release, incoming sustained/rest -classification, pending contracts, weights and budget. Full home-aware candidate -evaluation still runs with the real history. Weight changes invalidate the cache; -retaining the catalog object prevents object-ID reuse. Tests distinguish pending -targets and incoming hold durations, and a complete four-bar rerun preserves all -events, candidate evaluations and frontier records. diff --git a/docs/composer/implementation_report.md b/docs/composer/implementation_report.md deleted file mode 100644 index 6dd4351..0000000 --- a/docs/composer/implementation_report.md +++ /dev/null @@ -1,125 +0,0 @@ -# Complete-phrase selection and shared-rule learning - -**Exact reproduction is not achieved. The published baseline is unchanged.** The four complete authored solos remain the acceptance criterion. Legal construction, matching prefixes, learned rankings and passing tests do not satisfy it. - -Starting revision: `ef62af892bce168e2e9846dc6035b414f55d8959`. The preceding report is [archived](baseline_ef62af8.md). Current evidence is consolidated in `build/solo_reference_comparison/structured/`; `baseline/` preserves the previous complete comparison output. Saved output is never a production input. - -## Active implementation - -The production path is `production.generate_role → solo.composer.compose → construction.Builder → placement → validation/export`. The former builder committed the cheapest next action and backtracked only after failure. It now retains histories through common exact-time frontiers and selects among completed event streams. Limits are 16 histories, 24 successors per history and eight completed exports. A route, sounding tail and pending harmonic obligation survive a computational frontier. - -`Selector.complete` is the authoritative `phrase-selection-2` evaluator. It merges explicit ties, reconstructs silence and reports the contributions actually used for selection. Action labels no longer earn numerical rewards. `scale_thirds` means alternate scale-degree walks; `arpeggio` walks written chord members. A completed opening owns the home identity, separately from the eight-note recent-history buffer. - -A controlled test retains a locally more expensive opening and selects it when it produces the better completed phrase. Tests also cover partition invariance, pending continuation, arpeggios, home identity, deferred validation and exact incremental scoring. These establish selector behavior, not aesthetic quality or global optimality. - -No source song, scale family, rhythm catalog, accompaniment vocabulary, fret domain, runtime schema or performance expression changed. Production data changes remove action-family bonuses and replace the backtrack budget. Missing/no-collection harmony now rejects optional colors before parsing `N.C.`. Fast retrieval uses canonical evaluation for tied fragments and learned contextual rules. - -## Complete default-build result - -The staged build passed `--check` and produced six arrangement scores, complete solos and independent accompaniment for both songs. It is in `structured/staged`, not the published asset directory. - -| Output | Generated / target attacks | Matching pitch/onset/release attacks | Full exact result | -|---|---:|---:|---| -| All of Me acoustic | 116 / 137 | 0 | Fail | -| All of Me electric | 333 / 141 | 2 | Fail | -| Autumn Leaves acoustic | 164 / 121 | 1 | Fail | -| Autumn Leaves electric | 311 / 115 | 6 | Fail | - -Rests differ in every case. These are substantial failures, especially the electric parts' continuous activity. Counts are diagnostics, not substitute acceptance thresholds. The exact comparison command returned **1**. - -The [comparison page](../../build/solo_reference_comparison/structured/comparison/index.html) includes baseline, candidate and unchanged reference XML; complete solo-only and fixed-backing WAVs; opening excerpts; final independent backing; and exact differences. Rendering uses consistent neutral treatment and constant gains. **No ear-based audition was performed.** - -Reparsed exports agree with all four selected event streams. Every TAB position is legal; ties retain position. Harmonic validation reports no unresolved treatments. All 32 completed-candidate totals equal a fresh canonical evaluation (`asset-verification.json`, `evaluation-recheck.json`). The mismatch is in composition/search/selection, not export. - -## Failure diagnosis - -All four targets are playable on strings 1–6, frets 1–12 without open strings under the confirmed guitar-written convention. All of Me aligns at measure 1; Autumn Leaves at the first Gm7 in measure 2, retaining the preceding measure at negative form-relative time. Sources and references are unchanged. - -Stateful constrained derivation reaches all four complete streams with valid harmonic contracts and zero diagnostic history cuts. This demonstrates coverage only. The ordinary first successor cutoff drops every whole-action-compatible target opening: - -| Case | Proposed actions | Exact compatible openings | Retained | -|---|---:|---:|---:| -| All of Me acoustic | 9,853 | 1 | 0 | -| All of Me electric | 9,853 | 2 | 0 | -| Autumn Leaves acoustic | 8,492 | 2 | 0 | -| Autumn Leaves electric | 8,492 | 9 | 0 | - -Evidence: `exact-opening-pruning.json`. An earlier clipped-prefix diagnostic counted a shortened Autumn acoustic hold as surviving; `opening-pruning.json` records that weaker historical observation. The observer now checks complete committed events, with a regression preventing this mistake. It never supplies candidates or changes scores. - -Final ranking is also insufficient. Complete targets would lose under default weights even if retained: - -| Case | Selected total | Complete target total | -|---|---:|---:| -| All of Me acoustic | −2787/64 | 945/64 | -| All of Me electric | −4551/128 | 1287/128 | -| Autumn Leaves acoustic | −1420/33 | 14/11 | -| Autumn Leaves electric | −420/11 | 239/22 | - -Lower wins. `evaluation-diagnosis-all.json` records component differences. Source interval-pair resemblance, directed motion, register cost and duration preferences still favor continuous movement. The current features do not establish the longer ordered relationships needed to specify these authored tunes. This is not a proof that every generic model must fail. - -The source-only four-bar diagnostic exports eight completed alternatives and identifies the automatic winner. Seven first-bar realizations differ, but all share the first four attacks. Their complete events, home identities, contracts, actions, frontier counts and scores are in `completed-phrases/candidates.json`. Eight retained alternatives do not by themselves demonstrate useful opening diversity. - -## Development learning - -The prefix perceptron is removed. The replacement forms exact rational inequalities between complete ordinary outputs and complete examples. It learns shared bounded integer coefficients per role. Allowed features are relative musical categories and bounded two/three-condition rules, not song names, hashes, absolute measure indices, event arrays or per-song recipes. New conjunctions require demonstrated feature collisions. Production does not import the solver or read examples. - -The optional dependency is pinned to `z3-solver==4.13.3.0`. Exact zero/one-coefficient interval enumeration handles sparse cases; integer optimization handles the remainder. Feasibility and optimization have deterministic work limits. Incompatibility or exhaustion yields a diagnosis, not a substituted tune. - -The first four-example round compared each target with eight saved complete alternatives. It found acoustic `duration/1 = 9` and electric `metric/eighth/1_2 = 15`. These sparse coefficients separate the sampled alternatives; **they are not a learned jazz grammar or evidence of reproduction**. They remain in `structured/round-1/data`, with explicit source/target/artifact hashes and training provenance. Nothing was promoted to production `data/`. - -Successive rounds can accumulate full-output counterexamples using `--previous-round`. Source/example identities, evaluator, fixed priors and musical policies must agree; only learned weights may differ. Incompatible histories are rejected and duplicate inequalities removed. Later fitting cannot silently forget earlier failed complete outputs. No target-prefix update is used. - -A fresh ordinary build of that proposal completed under `learned-stage`, using four workers and no target arguments. It passed technical checks but failed the exact gate in all four cases: - -| Learned output | Generated / target attacks | Exact matching attacks | Exact result | -|---|---:|---:|---| -| All of Me acoustic | 214 / 137 | 0 | Fail | -| All of Me electric | 244 / 141 | 0 | Fail | -| Autumn Leaves acoustic | 315 / 121 | 0 | Fail | -| Autumn Leaves electric | 280 / 115 | 0 | Fail | - -The [learned-model comparison](../../build/solo_reference_comparison/structured/learned-comparison/index.html) contains the complete exports, consistent auditions and exact differences. All 32 saved learned-candidate scores and four selected event/TAB streams passed independent rechecking. All eight complete non-solo role documents remained byte-canonically identical to the default stage despite the changed solo weights (`full-song-role-independence.json`). No inference about musical preference follows from that technical result. - -The second fit accumulated **64 complete-output inequalities** from both ordinary builds. It found acoustic `space/after_arrival = -37` and electric `duration/1_4 = 19`. Both acoustic targets are still excluded at the first retrieval cutoff. The two electric openings survive that check, which proves neither full retention nor reproduction. This proposal was not staged or published, and no third full rollout is claimed. The result demonstrates how a sparse fit can separate sampled full outputs through coarse rhythmic statistics without recovering the ordered melody. Repeating the same fit or widening the beam is not a demonstrated remedy. Evidence is in `round-2/training.json`. - -Learning explicitly depends on the authored examples. Runtime target-free is **not training independence**. These repeatedly studied examples are not held-out data. Transfer to separate standards has not been demonstrated here and is not substituted for exact reproduction. - -## Validation - -Commands use `build/build_common/python/bin/python3 build/build_tools/python_runner.py build/build_common/modulus`; [commands.md](commands.md) records arguments. - -- `-m unittest discover -s tests -p 'test_*.py'`: **458 run: 456 passed, two optional solver skips, 1,139.453 seconds**. A preceding run had one `N.C.` lookahead error; it was repaired before the full rerun. -- With `PYTHONPATH=build/development_packages`, `-m unittest tests.test_joint_actions tests.test_structured_selection`: **35 passed in 24.256 seconds** (`successor-cache-tests.log`). This covers later exact shortcuts, solver changes, accumulated-round compatibility, strict diagnostic matching and incoming-state cache invalidation. The full suite was not rerun after every performance edit. -- `-m audition.dependency_check --output .../dependency-cached`: passed all six roles alone/cold, reversed/warm, two-process/cold, with invalid targets present/absent, and after an operational electric-only register change. All case hashes match the preceding `dependency-final` and `dependency` runs. Actual reads are recorded. -- Full `-m generator ... --check --composition-jobs 2 --export-solo-xml`: passed for the default stage. The separate learned-data run with `--composition-jobs 4` also passed. Actual read traces contain no reference or other part's concrete output. -- `-m audition.solo_compare ... --require-exact`: failed all four in **both** full builds, preventing a success claim. -- `structured/verify_completed.py`, through the Python wrapper: passed full selected-event, TAB, tie and canonical-score checks for both stages (64 completed candidates total). -- `git diff --check`: passed at the latest review. - -The dependency exercise uses a synthetic one-bar source and real catalogs, not four complete-song cold/warm repetitions. [dependency_proof.md](dependency_proof.md) gives hashes and scope. - -## Performance and cleanup - -Hardware reports Intel Core i7-1355U, 12 logical CPUs, Linux 7.2.3 x86-64. The two-worker default stage took **4,832.443 seconds**: 4,817.592 in performance generation, 1.418 playback compilation and 12.655 engraving. Solo role times were 2,126.676 / 2,711.550 seconds for All of Me and 1,958.730 / 2,101.732 for Autumn Leaves. Other roles took about 0.2–0.5 seconds each. Tests/diagnostics ran concurrently; this is not an isolated benchmark. - -This is a substantial regression from the prior one-action policy. Wider comparison is not a generation speedup. A 20-second live sample found retrieval, state advancement and frontier scoring dominant; engraving is not the bottleneck. - -Pure-operation caching, incremental evaluation, short-circuit theme equality and direct retrieval costs remove repeated work. All eight four-bar candidates and every frontier were identical across 227.751-, 126.421- and 92.633-second runs with different concurrent loads; these do **not** establish a controlled whole-build speedup. A paired 9,853-action microbenchmark measured median 0.274 seconds with full feature reports versus 0.125 with direct costs, every cost equal. That claim is limited to warm-cache retrieval. - -Successor-ranking memoization also preserves all eight candidates and every frontier. Its four-bar run took 154.177 seconds while the four learned workers ran, with 90 cache hits and 645 misses. The different load prohibits interpreting that elapsed time as a cache speedup or slowdown. The cache distinguishes catalog identity, the three incoming pitches, release time, sustained/rest context, pending obligations, weights and budget; complete home-aware evaluation still occurs per history. It removes repeated ranking work without changing search breadth. - -The learned four-worker full build took **2,007.666 seconds**, including 1,994 seconds of performance generation and approximately 13 seconds of engraving. Solo role times were 1,708.084 / 1,920.536 seconds for All of Me and 1,989.394 / 1,945.428 for Autumn Leaves. Its music, weights, optimizations, worker count and concurrent load differ from the default run. The shorter wall time cannot be attributed to parallelism alone. Cold iteration remains expensive; no arbitrary performance target has been met or promised. - -Default-build workers started before the final performance fixes. `frozen-inputs.json` and `post-freeze-repair.json` preserve that distinction. The no-chord guard does not apply to these sources; equality and export checks cover the shortcuts. The learned build has its own `learned-frozen-inputs.json` and predates only the final successor-cache optimization, recorded in `learned-post-freeze-repair.json`. Neither full run is presented as a cold benchmark of every final code byte. - -Incomplete task-local stages and superseded caches totaling about 12.8 MB were removed; `cleanup.json` records paths. References, sources, baseline, completed comparisons and useful verified caches remain. Temporary work stays in the consolidated ignored build directory. - -## Remaining boundary - -The selector repair and learning mechanism are implemented. **The exact musical objective remains unmet.** No learned parameters or staged assets have been promoted. Independent parts, harmonic/physical safeguards and runtime representation remain intact. - -The next bottleneck is specific: complete-phrase discrimination and early retrieval disagree, and the current event features permit degenerate rhythmic policies to beat the examples. More beam width, more unconstrained vocabulary, another prefix fit or hidden target retrieval would not establish the requested solution. Any next model change must face accumulated complete ordinary counterexamples and the four full-output target-free gate. - -There is also a narrower theme-construction limitation: `related()` transposes or inverts the entire realized opening through strictly mapped pitches. It does not adapt the opening's expressive contracts. A legal chromatic/color-bearing home therefore need not have a legal generated return under this constructor. All four default outputs selected zero returns/answers. This is a construction/representation limitation, not evidence that the scorer has learned to avoid unmusical returns. Full operation counters and winner/runner-up contributions are in `structured/decisions.md`. - -See [active_path_audit.md](active_path_audit.md), [data_migration.md](data_migration.md), [evaluation_contract.md](evaluation_contract.md), and [musical_capability_matrix.md](musical_capability_matrix.md). diff --git a/docs/composer/musical_capability_matrix.md b/docs/composer/musical_capability_matrix.md deleted file mode 100644 index 2c1e5d2..0000000 --- a/docs/composer/musical_capability_matrix.md +++ /dev/null @@ -1,27 +0,0 @@ -# Capabilities and release limits - -| Behavior | Current status | -|---|---| -| Ordinary scale motion, rests and held chord tones | Enumerable and validated | -| Complete scalar/scale-third groups | Reachable; exact scale indices, whole rhythm groups | -| Actual chord arpeggios | Successive written basic chord members, distinct from scale thirds | -| Heard-theme return and inverted answer | Constructors and synthetic reachability retained; a long expressive home may have no legal literal adaptation. No required count | -| Approaches, enclosures, prepared chromatic connections | Reachable joint actions; target, time and register validated | -| Anticipation with hold or deliberate reattack | Reachable and separately validated | -| Stable/resolving colors and anticipated colors | Reachable; complete sounding exposure checked | -| Delayed semitone resolution | Pending across actions; exact target and bounded deadline required | -| Prepared suspension and named blues inflection | Validators retained; no dedicated new action constructor yet | -| Optional full head quotation | No new constructor; source influence currently uses ordered interval relationships and rhythm evidence | -| Multiple complete histories | Active bounded common-time search; controlled locally-cheaper-opening counterfactual passes | -| Home identity | Completed opening candidate retained separately from recent-note retrieval context | -| Two-sided join revision | No edit/rewrite pass in this selector repair | -| All six strings, frets 1–12, no opens | Shared physical domain and whole-line fingering | -| Source-only independent accompaniment | Existing canonical sparse comping retained | -| Exact construction coverage for all four targets | Demonstrated by constrained derivation plus final harmonic validation; not a production winner | -| Exact unrestricted reproduction of all four targets | Not achieved; all four lose complete-duration-compatible openings at the first successor cutoff, and the full default exports fail exact comparison | -| Shared development learning | Complete-output constraints, bounded integer coefficients, accumulated compatible counterexamples and limited contextual-rule schema; a first four-example fit separated saved negatives but did not establish reproduction | -| Training-independent knowledge | Not claimed: approved fitting/induction depends explicitly on the examples | - -A complete route is indivisible, but many independent single attacks remain available. This flexibility removes former coverage restrictions without proving melodic coherence. Source relationships are richer than an unordered interval set but remain a limited identity model. Register and fingering are separate; TAB legality is not biomechanical certification. - -Unsupported written chords still fail explicitly. No source harmony is manufactured from a target. Electric may continue over the returning vocal pickup under the latest authorized allocation; vocal timing is unchanged. No dynamics, velocity shaping or new playback feel was introduced. diff --git a/generator/__main__.py b/generator/__main__.py index 3cf0656..4bba50a 100644 --- a/generator/__main__.py +++ b/generator/__main__.py @@ -104,7 +104,7 @@ def generate_assets(args: argparse.Namespace) -> None: export_tree(staged) generator.score.validate_generated_tree(staged) if args.check: - generator.summary.validate_expected_performance_summary(data_dir / 'expected' / 'performance_summary.json', staged / 'songs') + generator.summary.validate_expected_performance_summary(pathlib.Path(__file__).resolve().parents[1] / 'tests' / 'fixtures' / 'performance_summary.json', staged / 'songs') report('compiling playback') generator.playback.compile_tree(staged) report('engraving pages') diff --git a/generator/solo/composer.py b/generator/solo/composer.py index 17ab9a5..35138f7 100644 --- a/generator/solo/composer.py +++ b/generator/solo/composer.py @@ -24,9 +24,10 @@ def compose(analysis,measures,part,end,*,scales,sequences,rhythms,initial=None,e if placed is None:raise construction.SearchExhausted('selected melody has no complete legal guitar path') by_id={n.event_id:n for n in placed[1]} phrases=tuple(dataclasses.replace(p,notes=tuple(by_id[n.event_id] for n in p.notes)) for p in phrases) - result=types.Solo(part,choices,phrases,analysis.tempo,end,{'construction':'complete_phrase_frontiers','version':policy.DEFAULT.composition_version, + result=types.Solo(part,choices,phrases,analysis.tempo,end,{'construction':'source_melodic_phrases','version':policy.DEFAULT.composition_version, 'construction_seconds':time.monotonic()-began,**builder.timings,'operation_coverage':{k:dict(v) for k,v in builder.coverage.items()}, - 'frontiers':builder.frontiers,'completed_candidates':[{'rank':i+1,'evaluation':dataclasses.asdict(s.evaluation),'events':[(str(n.start),str(n.end),n.pitch.number()) for n in s.notes]} for i,s in enumerate(builder.completed)],'rejections':dict(builder.failures),'decisions':builder.decisions,'expressions_enabled':expressions_enabled},profile,start) + 'whole_stream_diagnostic':dataclasses.asdict(builder.winner_evaluation), + 'frontiers':builder.frontiers,'completed_candidates':[{'rank':i+1,'evaluation':{'selection_total': str(s.total / (end-start)), 'phrases': [dataclasses.asdict(p.evaluation) for p in s.phrases]},'events':[(str(n.start),str(n.end),n.pitch.number()) for n in s.notes]} for i,s in enumerate(builder.completed)],'rejections':dict(builder.failures),'decisions':builder.decisions,'expressions_enabled':expressions_enabled},profile,start) validate(result) return result diff --git a/generator/solo/construction.py b/generator/solo/construction.py index 18e3888..36fd130 100644 --- a/generator/solo/construction.py +++ b/generator/solo/construction.py @@ -1,268 +1,83 @@ -"""Bounded search of joint musical actions; one normal production path. - -Selection compares several histories at common exact-time frontiers. Actions -are indivisible; crossing a reporting window does not restart their contracts. -""" -from __future__ import annotations -from collections import Counter +"""Bounded comparison of completed source-related phrases.""" +import collections import dataclasses -from fractions import Fraction as F +import fractions import time -import heapq -import itertools -import functools +import generator.solo.phrases +import generator.solo.selection +import generator.solo.sequences +import generator.solo.policy +import generator.vocabulary -from generator import vocabulary -from generator.solo import types, groups, operators, selection, sequences, scoring -class SearchExhausted(ValueError):pass +class SearchExhausted(ValueError): + pass + @dataclasses.dataclass(frozen=True) class State: - total: F = F() - recent: tuple = () + phrases: tuple = () home: tuple = () - last_kind: str = '' - parent: object = None - action: object = None - evaluation: object = None - tie: tuple = () - pending: tuple = () - alternatives: tuple = () - actions: tuple = () - notes: tuple = () - full_evaluation: object = None + total: fractions.Fraction = fractions.Fraction() + + @property + def notes(self): + return tuple(n for p in self.phrases for n in p.notes) - @functools.cached_property - def music_key(self):return tuple((n.start,n.end,n.pitch.number()) for n in self.notes) - @functools.cached_property - def path_key(self):return tuple(a.key for a in self.actions) class Builder: - def __init__(self,analysis,measures,harmony,sequences,rhythms,end,seed,enabled=True,*,start=F()): - self.analysis=analysis;self.harmony=harmony;self.start=start;self.end=end - self.operators=operators.Operators(harmony,analysis,end,start,enabled,sequences,rhythms) - self.selector=selection.Selector(harmony,analysis,harmony.profile.reference or 60) - self.rules=vocabulary.data('solo')['operators'];self.decisions=[];self.failures=Counter();self.coverage={};self.timings=Counter() - self.observe=None # Optional read-only diagnostic; its return is ignored. - self.successor_cache={};self.retrieval_at=None;self.retrieval_weights=None + def __init__(self,analysis,measures,harmony,sequences,rhythms,end,seed,enabled=True,*,start=fractions.Fraction()): + self.analysis,self.harmony = analysis,harmony + self.start,self.end,self.seed = start,end,seed + self.phrases = generator.solo.phrases.Phrases(analysis,harmony,sequences,rhythms,start,end,seed,enabled) + self.rules = generator.vocabulary.data('solo')['search'] + self.timings = {}; self.frontiers = []; self.coverage = collections.defaultdict(collections.Counter); self.decisions = [] + self.failures = self.phrases.rejections - def candidates(self,at,state): - candidates=self.operators.proposals(at) - if state.home and state.last_kind!='return': - candidates+=self.operators.related(at,state.home,'return')+self.operators.related(at,state.home,'answer') - # Retrieval never compares cost rates of differently sized actions. Each - # endpoint has its own ranking; round-robin the endpoints before cutoff. - context=(tuple(n.pitch.number() for n in state.recent[-3:]),state.recent[-1].end if state.recent else None) - settled=state.recent[-1].end-state.recent[-1].start>=F(3,2) if state.recent else None - if at!=self.retrieval_at or self.retrieval_weights is not self.selector.weights: - self.successor_cache.clear();self.retrieval_at=at;self.retrieval_weights=self.selector.weights - # Cache only endpoint retrieval, whose sufficient incoming state is three - # pitches, release, rest function and pending contracts. Home-specific - # actions change the immutable catalog identity; full evaluation below - # still uses the actual complete home and recent history on every call. - key=(id(candidates),context,settled,state.pending,self.rules['per_state']) - cached=self.successor_cache.get(key) - if cached is not None and cached[0] is candidates: - self.timings['retrieval_cache_hits']+=1;retained=cached[1] - else: - self.timings['retrieval_cache_misses']+=1;buckets={} - for order,a in enumerate(candidates): - if not self.pending_compatible(state,a):continue - cost=self.selector.fast_cost(a,state.recent,context=context) - buckets.setdefault(a.end,[]).append((cost,order,a)) - ranked=[] - for end,rows in sorted(buckets.items()): - heapq.heapify(rows);ranked.append(rows) - retained=[] - while ranked and len(retained)64:self.successor_cache.pop(next(iter(self.successor_cache))) - for _,a in retained:self.operators.coverage['retained:'+a.kind]+=1 - if self.observe:self.observe('successors',at,state,tuple(candidates),tuple(a for _,a in retained)) - return tuple((self.selector.evaluate(a,state.recent,state.home,state.last_kind),a) for _,a in retained) - - def pending_compatible(self,state,action): - for contract in state.pending: - target=contract.target - if action.start<=target.startaction.end) - elif obligations: - heard={n.event_id:n for c in state.pending for n in c.events if n.startself.start] - opening_end=boundaries[min(3,len(boundaries)-1)] if boundaries else self.end - if not home and action.end>=opening_end and not pending and len(notes)>=3: - # The same sounded opening has the same home identity regardless of - # whether its last generating action extends beyond this source window. - home=tuple(n for n in notes if n.startmax(F(),self.start)),self.end) - opening=tuple(itertools.takewhile(lambda n:n.startuntil else n for n in a.notes if n.start=until:ready.append(state) - else: - agenda.setdefault(cursor,[]).append(state) - if cursor not in queued:heapq.heappush(queue,cursor);queued.add(cursor) - while queue: - cursor=heapq.heappop(queue) - partial=self.retain(agenda.pop(cursor),cursor) - for state in partial: - for evaluation,action in self.candidates(cursor,state): - next_state=self.advance(state,evaluation,action) - if next_state is None:continue - if action.end>=until:ready.append(next_state) - else: - agenda.setdefault(action.end,[]).append(next_state) - if action.end not in queued:heapq.heappush(queue,action.end);queued.add(action.end) - # This exact-time frontier will never be expanded again. Keep chosen - # actions in immutable histories, not tens of thousands of rejected - # concrete actions from obsolete frontiers (especially across workers). - self.operators.cache.clear();self.operators.validity_cache.clear() - self.selector.static.clear();self.selector.cost_static.clear() - if not ready:raise SearchExhausted(f'No continuation at {until} within deterministic search budget') - scored=[] - for state in ready: - evaluation=self.frontier_score(state,until) - scored.append(dataclasses.replace(state,total=evaluation.total,evaluation=evaluation)) - states=self.retain(scored,until);at=until - completed=tuple(s for s in states if not s.pending) - if not completed:raise SearchExhausted('No complete phrase within deterministic frontier budget; not a proof of musical impossibility') - completed=tuple(sorted(completed,key=self.rank)) - self.completed=(completed[0],*sequences.diverse(completed[1:],self.rank,self.diversity,self.rules['completed']-1)) - final=completed[0] - self.timings['action_search_seconds']=time.monotonic()-began - self.timings['frontiers']=len(self.frontiers) - path=[];state=final - while state.action:path.append(state);state=state.parent - path.reverse();phrases=[];pending=[] - self.selected_actions=final.actions - boundaries=[x-self.analysis.body_offset for x in self.analysis.boundaries] - checkpoints=set(boundaries[4::4]) - for state in path: - action=state.action;pending.append(state) - self.operators.coverage['selected:'+action.kind]+=1 - self.decisions.append({'start':str(action.start),'end':str(action.end),'selected':action.kind, - 'evaluation':dataclasses.asdict(state.evaluation),'prefix_total':state.total}) - # Reporting partitions never invent a cadence or split an active action. - boundary=any(action.start=2 - ending='handoff' if end==self.end else 'settle' if held else 'open' - anchor=None - if held: - region=self.harmony.at(last.start) - anchor=types.Anchor(f'gesture_{index}:arrival',last.start,last.start,last.pitch, - frozenset({last.pitch.number()%12}),self.harmony.profile.register,last.end-last.start, - 'terminal' if ending=='handoff' else 'intermediate',region.chord,False,last.end,ending) - evaluation=self.selector.complete(actions,states[-1].home,start=start,end=end) - direction='up' if len(ns)>1 and ns[-1].pitch.number()>ns[0].pitch.number() else 'down' - rhythm=tuple(g for a in actions for g in groups.describe(tuple((n.start-a.start,n.end-a.start) for n in a.notes),origin=a.start,source=a.kind)) - return types.Phrase(f'gesture_{index}',start,end,ns,actions[0].kind,direction,ending,anchor, - tuple(c for a in actions for c in a.contracts),rhythm,'home',evaluation.total,evaluation,tuple(a.source or a.kind for a in actions)) + for phrase,introduced in self.candidates(start,end,state): + total = state.total+phrase.evaluation.total*(end-start) + successors.append(State((*state.phrases,phrase),state.home or (introduced if start>=0 and len(introduced)>=3 else ()),total)) + if not successors: + raise SearchExhausted(f'No valid complete phrase at {start}–{end}; failures: {dict(self.failures)}') + successors.sort(key=self.rank) + states = tuple(generator.solo.sequences.diverse(successors,self.rank, + lambda s:(s.phrases[-1].purpose,s.phrases[-1].direction,s.phrases[-1].ending),self.rules['histories'])) + for state in states: + self.coverage[state.phrases[-1].purpose]['retained_histories'] += 1 + self.frontiers.append({'start':str(start),'end':str(end),'completed':len(successors),'retained':len(states),'best':str(min(s.total for s in states))}) + start = end + self.completed = tuple(sorted(states,key=self.rank)) + winner = self.completed[0] + self.timings['phrase_search_seconds'] = time.monotonic()-began + self.winner_evaluation = generator.solo.selection.evaluate(winner.notes,self.start,self.end,self.harmony, + home=winner.home,contracts=tuple(c for p in winner.phrases for c in p.expressions),center=self.phrases.center) + for k,v in self.phrases.proposals.items(): + self.coverage[k]['proposed'] = v + for p in winner.phrases: + self.coverage[p.purpose]['selected'] += 1 + self.decisions = [{'start':str(p.start),'end':str(p.end),'purpose':p.purpose,'ending':p.ending,'evaluation':dataclasses.asdict(p.evaluation)} for p in winner.phrases] + return winner.phrases diff --git a/generator/solo/operators.py b/generator/solo/operators.py deleted file mode 100644 index 0c444b6..0000000 --- a/generator/solo/operators.py +++ /dev/null @@ -1,326 +0,0 @@ -"""Shared, enumerable musical actions for composition and derivation checking. - -An action owns its complete rhythm and path. A rest is an action; an ordinary -attack is not an implied cadence. Compound actions retain approach targets and -whole tuplet groups. This module has no training/reference-file interface. -""" -from __future__ import annotations -import dataclasses -import functools -from fractions import Fraction as F -from collections import Counter - -from generator import harmony as chords, vocabulary -from generator.solo import types, expressions, groups, rhythm - - -@dataclasses.dataclass(frozen=True) -class Action: - start: F - end: F - notes: tuple - kind: str - contracts: tuple = () - source: str = '' - - @functools.cached_property - def key(self): - return (self.start,self.end,self.kind,tuple((n.start,n.end,n.pitch.number()) for n in self.notes), - tuple((c.rule,str(c.boundary),c.target.start,c.target.end,c.target.pitch.number()) for c in self.contracts)) - - -class Operators: - def __init__(self,harmony,analysis,end,start=F(),enabled=True,sequences=None,rhythms=None): - self.harmony=harmony;self.analysis=analysis;self.start=start;self.end=end;self.enabled=enabled - self.rules=vocabulary.data('solo')['operators'];self.cache={};self.pitch_cache={};self.coverage=Counter() - self.ordinary_cache={};self.validity_cache={} - self.sequences=sequences - self.licensed=frozenset(s.steps for values in sequences.paths.values() for s in values) if sequences else None - self.patterns=[] - if rhythms is not None: - library=rhythm.Rhythms(rhythms,analysis.tempo) - # Purposeful curated groups are peers; exhaustive catalogs contribute - # representatives, not a probability proportional to XML file counts. - candidates=library.groups - for meter,values in library.catalog.items(): - if any(b-a==F(meter.split('/')[0])*4/int(meter.split('/')[1]) for a,b in zip(analysis.boundaries,analysis.boundaries[1:])): - candidates+=list(values) - seen=set() - for name,pattern,length,_ in candidates: - if 2<=len(pattern)<=4 and length<=4 and (pattern,length) not in seen: - seen.add((pattern,length));self.patterns.append((name,pattern,length)) - - def pitches(self,time): - span=self.harmony.at(time) - if not span or not span.scale:return () - if span.scale not in self.pitch_cache: - self.pitch_cache[span.scale]=tuple(p for i in range(-28,29) - if self.harmony.profile.permits((p:=types.scale_pitch(span.scale,i)).number())) - return self.pitch_cache[span.scale] - - @functools.lru_cache(maxsize=32768) - def note(self,start,end,pitch,role): - # Identity follows exact musical content, not an output filename or hash. - return types.Note(start,end,pitch,role=role,event_id=f'{start}:{end}:{pitch.number()}') - - def degree_pitches(self,chord,rule): - spelling=expressions.degree_pitch(chord,rule['degree'],rule['alter'],60) - # Enumerate octaves, not repeated rounding from C-octave centers. Halfway - # pitches (e.g. F#) otherwise disappear because round uses even ties. - return tuple(p for octave in range(1,6) - if self.harmony.profile.permits((p:=dataclasses.replace(spelling,octave=octave)).number())) - - def _accepted(self,action,validate=True): - self.coverage['proposed:'+action.kind]+=1 - if action.end>self.end or action.end<=action.start: - self.coverage['rejected_by_allocation:'+action.kind]+=1;return False - if any(not self.harmony.profile.permits(n.pitch.number()) for n in action.notes): - self.coverage['rejected_by_physical_domain:'+action.kind]+=1;return False - if not groups.valid(tuple((n.start-action.start,n.end-action.start) for n in action.notes)): - self.coverage['rejected_by_rhythm:'+action.kind]+=1;return False - self.coverage['feasible:'+action.kind]+=1 - return self.is_valid(action) if validate else True - - def is_valid(self,action): - key=action.key - if key in self.validity_cache:return self.validity_cache[key] - if action.contracts: - valid=self.harmony.evaluate(action.notes,expressions=action.contracts,complete=False,until=action.end).valid - else: - valid=True - for n in action.notes: - note_key=(n.start,n.end,n.pitch.number()) - if note_key not in self.ordinary_cache:self.ordinary_cache[note_key]=self.harmony.evaluate((n,),complete=True,until=n.end).valid - if not self.ordinary_cache[note_key]:valid=False;break - self.coverage[('valid:' if valid else 'rejected_by_rule:')+action.kind]+=1 - self.validity_cache[key]=valid - if len(self.validity_cache)>8192:self.validity_cache.pop(next(iter(self.validity_cache))) - return valid - - @functools.lru_cache(maxsize=8192) - def _color(self,n,span): - result=[] - for rule in vocabulary.data('harmony')['colors']: - if rule.get('resolve') or not expressions.eligible_color(rule,span,self.harmony):continue - expected=expressions.degree_pitch(expressions.parsed_chord(span.chord),rule['degree'],rule['alter'],n.pitch.number()) - if expected.number()!=n.pitch.number():continue - c=types.ExpressionContract(n.event_id+':'+rule['id'],'color',rule['id'],(n,),n,span.chord) - result.append(Action(n.start,n.end,(n,),'color',(c,),rule['id'])) - return tuple(result) - - def proposals(self,time): - return self.at(time,validate=False) - - def at(self,time,eligible=None,first=None,*,validate=True): - """Unpruned legal actions. Incoming-state ranking is a separate operation.""" - if eligible is None and first is None and (time,validate) in self.cache:return self.cache[(time,validate)] - span=self.harmony.at(time);values=[] - if not span or not span.scale: - end=min(self.end,span.end if span else self.end) - result=(Action(time,end,(),'space'),) - if eligible is not None:return tuple(a for a in result if eligible(a)) - self.cache[(time,validate)]=result;return result - def add(action): - if eligible is not None and not eligible(action):return - if self._accepted(action,validate=validate):values.append(action) - def entry(a,b,pitch): - return first is None or first==(a,b,pitch.number()) - durations=tuple(map(F,self.rules['durations'])) - if self.analysis.tempo>vocabulary.data('roles')['difficulty']['sixteenth_max_bpm']: - durations=tuple(d for d in durations if d>=F(1,2)) - for duration in durations: - if time+duration>self.end:continue - if (time*(4 if duration==F(1,4) else 2)).denominator==1: - add(Action(time,time+duration,(),'rest')) - for pitch in self.pitches(time): - if not entry(time,time+duration,pitch):continue - n=self.note(time,time+duration,pitch,'motion') - if pitch.number()%12 in self.harmony.permitted(time,n.end): - add(Action(time,n.end,(n,),'tone')) - if self.enabled: - delayed=vocabulary.data('lead_ins').get('delayed_approach') - if delayed: - for delay in map(F,delayed['arrivals']): - arrival=time+delay - for hold in map(F,delayed['holds']): - if arrival+hold>self.end:continue - region=self.harmony.at(arrival) - if not region:continue - for p in self.pitches(arrival): - if p.number()%12 not in self.harmony.members(arrival,arrival+hold):continue - target=self.note(arrival,arrival+hold,p,'arrival') - for step in delayed['steps']: - n=self.note(time,time+F(delayed['max_note']),expressions.chromatic(p,-step),'delayed_approach') - if not entry(n.start,n.end,n.pitch):continue - if n.pitch.number()%12 in self.harmony.permitted(n.start,n.end):continue - c=types.ExpressionContract(n.event_id+':delayed','delayed_approach',delayed['id'],(n,),target,region.chord) - add(Action(time,n.end,(n,),'delayed_approach',(c,),delayed['id'])) - # Optional colors retain functional degree identity independently of - # the reference/exporter's enharmonic notation. No scale is widened. - for rule in vocabulary.data('harmony')['colors']: - if not expressions.eligible_color(rule,span,self.harmony):continue - for p in self.degree_pitches(expressions.parsed_chord(span.chord),rule): - if not self.harmony.profile.permits(p.number()):continue - for duration in durations: - if duration>F(rule['max_beats']) or time+duration>self.end:continue - if not entry(time,time+duration,p):continue - n=self.note(time,time+duration,p,'color') - if not rule.get('resolve'): - c=types.ExpressionContract(n.event_id+':'+rule['id'],'color',rule['id'],(n,),n,span.chord) - add(Action(time,n.end,(n,),'color',(c,),rule['id'])) - else: - for gap in (F(),F(1,2),F(1)): - for q in self.pitches(n.end+gap): - if not 0self.end:continue - c=types.ExpressionContract(n.event_id+':'+rule['id'],'color',rule['id'],(n,),target,target_span.chord) - add(Action(time,n.end,(n,),'color_resolution',(c,),rule['id'])) - for boundary in (s.start for s in self.harmony.spans if timevocabulary.data('roles')['difficulty'][rule['difficulty']+'_max_bpm']:continue - # A connection's preceding event is explicit in the action. It cannot - # obtain a fictitious preparation from a label or from a future note. - if rule['kind']=='connection': - for preparation in map(F,self.rules['connection_preparations']): - if first is not None and (first[0]!=time or first[1]!=time+preparation):continue - arrival=time+preparation+sum(map(F,rule['durations']),F())+F(rule['gap']) - for hold in map(F,self.rules['approach_holds']): - if arrival+hold>self.end:continue - for p in self.pitches(arrival): - region=self.harmony.at(arrival) - target=self.note(arrival,arrival+hold,p,'arrival') - cursor=time+preparation;notes=[] - for step,length in zip(rule['semitones'],map(F,rule['durations'])): - notes.append(self.note(cursor,cursor+length,expressions.chromatic(p,step),'connection'));cursor+=length - for prep_pitch in self.pitches(time): - if not entry(time,time+preparation,prep_pitch) or not 0F(vocabulary.data('harmony')['anticipation_max']):continue - anticipation=types.ExpressionContract(target.event_id+':early','anticipation','early_chord_tone',(target,),target,upcoming.chord,boundary=upcoming.start) - approach=dataclasses.replace(c,chord=upcoming.chord,boundary=upcoming.start) - add(Action(time,target.end,(*notes,target),rule['kind'],(approach,anticipation),rule['id'])) - # Complete directional groups use exact scale indexing at each harmonic - # time. A changed collection retains actual pitches, never degree labels. - lengths_list=[(F(1,2),)*2,(F(1,2),)*4,(F(1),F(1,2),F(1,2)),(F(1,2),F(1),F(1,2))] - difficulty=vocabulary.data('roles')['difficulty'] - if self.analysis.tempo<=difficulty['triplet_max_bpm']:lengths_list.append((F(1,3),)*3) - if self.analysis.tempo<=difficulty['sixteenth_max_bpm']:lengths_list.extend(((F(1,4),)*4,(F(1,4),)*2)) - patterns=list(self.patterns) - for lengths in lengths_list: - cursor=F();pattern=[] - for d in lengths:pattern.append((cursor,cursor+d));cursor+=d - patterns.append(('complete_subdivision',tuple(pattern),cursor)) - for name,pattern,length in patterns: - if time+length>self.end:continue - if first is not None and (time+pattern[0][0],time+pattern[0][1])!=first[:2]:continue - for stride in (-2,-1,1,2): - # Catalog licenses relative paths, but never sets family probability. - if self.licensed is not None and (stride,)*(len(pattern)-1) not in self.licensed:continue - for initial in self.pitches(time): - if not entry(time+pattern[0][0],time+pattern[0][1],initial):continue - cursor=time;p=initial;notes=[] - for index,(a,b) in enumerate(pattern): - cursor=time+a - region=self.harmony.at(cursor) - if not region or not region.scale:break - if index: - try:p=types.scale_pitch(region.scale,types.scale_index(region.scale,p.number())+stride) - except ValueError:break - notes.append(self.note(cursor,time+b,p,'scalar' if abs(stride)==1 else 'scale_thirds')) - if len(notes)==len(pattern):add(Action(time,time+length,tuple(notes),'scalar' if abs(stride)==1 else 'scale_thirds',source=f'{name}:degree_stride:{stride}')) - # A real arpeggio walks successive members of the WRITTEN chord, not - # alternate degrees of its selected seven-note collection. - for direction in (-1,1): - for initial in self.pitches(time): - if initial.number()%12 not in span.roles.basic:continue - if not entry(time+pattern[0][0],time+pattern[0][1],initial):continue - p=initial;notes=[] - for index,(a,b) in enumerate(pattern): - region=self.harmony.at(time+a) - if not region or not region.scale:break - if index: - choices=[q for q in self.pitches(time+a) if q.number()%12 in region.roles.basic and (q.number()-p.number())*direction>0] - if not choices:break - p=min(choices,key=lambda q:abs(q.number()-p.number())) - notes.append(self.note(time+a,time+b,p,'arpeggio')) - if len(notes)==len(pattern):add(Action(time,time+length,tuple(notes),'arpeggio',source=f'{name}:written_chord:{direction}')) - # Equivalent musical actions need one stable derivation, not extra votes. - unique={} - for a in values:unique.setdefault(a.key,a) - result=tuple(unique[k] for k in sorted(unique)) - if eligible is not None or first is not None:return result - if len(self.cache)>=32:self.cache.pop(next(iter(self.cache))) - if len(self.ordinary_cache)>8192:self.ordinary_cache.clear() - self.cache[(time,validate)]=result - return result - - @functools.lru_cache(maxsize=512) - def related(self,time,home,kind): - """Construct a heard-theme return or answer as a whole group, never a prefix.""" - if len(home)<3:return () - pattern=tuple((n.start-home[0].start,n.end-home[0].start) for n in home) - if not groups.valid(pattern):return () - values=[] - for p in self.pitches(time): - notes=[] - for n,(a,b) in zip(home,pattern): - number=p.number()+(n.pitch.number()-home[0].pitch.number())*(1 if kind=='return' else -1) - q=next((q for q in self.pitches(time+a) if q.number()==number),None) - if q is None:break - notes.append(self.note(time+a,time+b,q,kind)) - if len(notes)==len(home): - action=Action(time,time+pattern[-1][1],tuple(notes),kind,source='realized_theme') - if self._accepted(action):values.append(action) - return tuple(values) diff --git a/generator/solo/phrases.py b/generator/solo/phrases.py new file mode 100644 index 0000000..bebb0e0 --- /dev/null +++ b/generator/solo/phrases.py @@ -0,0 +1,383 @@ +"""Source-related melodic sentences realized as whole pitch/rhythm routes. + +Original song analysis and general vocabulary are the only musical inputs. +""" +import collections +import dataclasses +import fractions +import functools +import generator.model +import generator.solo.expressions +import generator.solo.groups +import generator.solo.policy +import generator.solo.rhythm +import generator.solo.selection +import generator.solo.types +import generator.vocabulary + + +@dataclasses.dataclass(frozen=True) +class Idea: + rhythm: tuple + intervals: tuple + source: str + + @property + def length(self): + return self.rhythm[-1][1] + + +def idea(notes, source): + first = notes[0] + return Idea(tuple((n.start-first.start,n.end-first.start) for n in notes), + tuple(n.pitch.number()-first.pitch.number() for n in notes),source) + + +def source_ideas(analysis): + found = {} + for i in range(len(analysis.melody)): + for count in (3,4,5,6): + notes = analysis.melody[i:i+count] + if len(notes)!=count or not 2<=notes[-1].end-notes[0].start<=6: + continue + value = idea(notes,'head:'+str(notes[0].start)) + if not generator.solo.groups.valid(value.rhythm): + continue + key = (value.rhythm,value.intervals) + old,number = found.get(key,(value,0)); found[key] = (old,number+1) + def rank(row): + value,count = row + intervals = [b-a for a,b in zip(value.intervals,value.intervals[1:])] + turns = sum(a*b<0 for a,b in zip(intervals,intervals[1:])) + durations = len({b-a for a,b in value.rhythm}) + return (-min(count,3),-durations,-min(turns,2),-min(7,max(value.intervals)-min(value.intervals)),value.source) + return tuple(v for v,_ in sorted(found.values(),key=rank)[:12]) + + +class Phrases: + def __init__(self,analysis,harmony,sequences,rhythms,start,end,seed,enabled): + self.analysis,self.harmony = analysis,harmony + self.start,self.end,self.seed,self.enabled = start,end,seed,enabled + self.sequences = sequences + self.ideas = source_ideas(analysis) + library = generator.solo.rhythm.Rhythms(rhythms,analysis.tempo) + self.patterns = tuple(library.groups) + self.arrivals = tuple(sorted({(cell.pattern[-1][1]-cell.pattern[-1][0],cell.duration-cell.pattern[-1][1]) + for cell in generator.vocabulary.rhythm_cells(analysis.tempo,'arrival') if cell.pattern and cell.pattern[-1][1]-cell.pattern[-1][0]>=1})) + # Catalog families contribute one peer per rhythmic function, not one + # vote per file. In particular the large 5/4 family cannot dominate 4/4. + representatives = {} + lengths = {b-a for a,b in zip(analysis.boundaries,analysis.boundaries[1:])} + for values in library.catalog.values(): + for name,pattern,length,purposes in values: + if length not in lengths or not 2<=len(pattern)<=6: + continue + descriptor = (any(a[1]1) + representatives.setdefault(descriptor,(name,pattern,length,purposes)) + self.patterns += tuple(representatives.values()) + self.center = harmony.profile.reference or 60 + self.rejections = collections.Counter() + self.proposals = collections.Counter() + + def choose(self,values,domain,key): + return min(values,key=lambda x:generator.solo.policy.stable_rank(self.seed+domain,(key,x))) + + def horizon(self,start): + region = self.harmony.at(start) + if not region or not region.scale: + return min(self.end,region.end if region else self.end) + gap = next((s.start for s in self.harmony.spans if startn.end) for i,n in enumerate(self.analysis.melody)) + return (abs(boundaries.index(end)+1-4)-2*int(cadence)-int(breath),end) + return min(gap,min(candidates,key=cost)) + + @functools.lru_cache(maxsize=8192) + def pitches(self,start,end): + span = self.harmony.at(start) + if not span or not span.scale: + return () + permitted = self.harmony.permitted(start,end) + return tuple(p for index in range(-28,29) if self.harmony.profile.permits((p:=generator.solo.types.scale_pitch(span.scale,index)).number()) and p.number()%12 in permitted) + + @staticmethod + def note(start,end,pitch,role): + return generator.solo.types.Note(start,end,pitch,role=role,event_id=f'{start}:{end}:{pitch.number()}') + + @functools.lru_cache(maxsize=16384) + def options(self,start,end,near,structural=False): + result = [] + span = self.harmony.at(start) + if not span or not span.scale: + return () + for p in self.pitches(start,end): + if abs(p.number()-near)>12: + continue + note = self.note(start,end,p,'arrival' if structural else 'motion') + if self.harmony.evaluate((note,),complete=True).valid: + if not structural or p.number()%12 in self.harmony.members(start,end): + result.append((note,())) + if self.enabled and end-start>=1: + for rule in generator.vocabulary.data('harmony')['colors']: + if rule.get('resolve') or not generator.solo.expressions.eligible_color(rule,span,self.harmony): + continue + pitch = generator.solo.expressions.degree_pitch(generator.solo.expressions.parsed_chord(span.chord),rule['degree'],rule['alter'],p.number()) + if pitch.number()!=p.number(): + continue + color = generator.solo.types.ExpressionContract(note.event_id+':'+rule['id'],'color',rule['id'],(note,),note,span.chord) + if self.harmony.evaluate((note,),expressions=(color,),complete=True).valid: + result.append((dataclasses.replace(note,role='color'),(color,))) + return tuple(result) + + def scale_step(self,scale,number,step): + """Continue from sounding pitch, never transfer a degree between maps.""" + try: + index = generator.solo.types.scale_index(scale,number) + except ValueError: + # The preceding attack may belong to the previous collection. Its + # pitch stays untouched; enter the new one in the requested direction. + if not step: + return None + entries = [generator.solo.types.scale_pitch(scale,i) for i in range(-28,29)] + entries = [p for p in entries if (p.number()-number)*step>0] + if not entries: + return None + nearest = min(entries,key=lambda p:abs(p.number()-number)) + index = generator.solo.types.scale_index(scale,nearest.number()) + step -= 1 if step>0 else -1 + return generator.solo.types.scale_pitch(scale,index+step) + + def route(self,rhythm,contour,incoming,center,mode='develop',*,degree_path=False): + """DP compares completed paths; no single route step is committed alone.""" + if not rhythm: + return None + initial = incoming.pitch.number() if incoming else center + if incoming and mode in ('connect','arpeggio'): + direction = 1 if contour[-1]>=contour[0] else -1 + center = initial+direction*2 + options = self.options(*rhythm[0],center) + if mode=='arpeggio': + options = tuple(v for v in options if v[0].pitch.number()%12 in self.harmony.at(rhythm[0][0]).roles.basic) + starts = sorted(options,key=lambda v:(abs(v[0].pitch.number()-center)+max(0,abs(v[0].pitch.number()-initial)-5),v[0].pitch.number()))[:3] + states = [(0,(n,),c,n.pitch.number()) for n,c in starts] + for index,(a,b) in enumerate(rhythm[1:],1): + next_states = [] + for cost,notes,contracts,origin in states: + wanted = origin+contour[min(index,len(contour)-1)] + expected = contour[min(index,len(contour)-1)]-contour[min(index-1,len(contour)-1)] + if degree_path: + span = self.harmony.at(a) + target = self.scale_step(span.scale,notes[-1].pitch.number(),expected) if span and span.scale else None + if target is None: + continue + wanted = target.number() + expected = wanted-notes[-1].pitch.number() + for note,support in self.options(a,b,wanted): + if degree_path and note.pitch.number()!=wanted: + continue + delta = note.pitch.number()-notes[-1].pitch.number() + if mode=='arpeggio' and (note.pitch.number()%12 not in self.harmony.at(a).roles.basic or delta==0): + continue + error = abs(note.pitch.number()-wanted)+3*int(delta==0 and expected!=0)+max(0,abs(delta)-7)*2 + if b-a>=1 and note.pitch.number()%12 not in self.harmony.at(a).roles.basic and not support: + error += 2 + next_states.append((cost+error,(*notes,note),(*contracts,*support),origin)) + if not next_states: + return None + states = sorted(next_states,key=lambda s:(s[0],tuple(n.pitch.number() for n in s[1])))[:8] + for _,notes,contracts,_ in sorted(states,key=lambda s:(s[0],tuple(n.pitch.number() for n in s[1]))): + if self.harmony.evaluate(notes,expressions=contracts,complete=True).valid: + return notes,contracts + return None + + def contour(self,count,direction,mode,motif,time): + if mode in ('statement','return','develop','answer') and motif: + values = motif.intervals + sign = -1 if mode=='answer' else 1 + # Explicit reduction/extension; never cycle a route to fit its rhythm. + return tuple(sign*values[i] if i0)==(direction>0) and sum(a*b<0 for a,b in zip(p.steps,p.steps[1:]))<=1] + span = self.harmony.at(time) + if paths and span and span.scale: + path = min(paths,key=lambda p:(abs(abs(p.offsets()[-1])-(count-1)),p.rank())) + if mode=='connect': + return path.offsets() + base = generator.solo.types.scale_pitch(span.scale,0) + return tuple(generator.solo.types.scale_pitch(span.scale,i).number()-base.number() for i in path.offsets()) + if mode=='connect': + return tuple(direction*i for i in range(count)) + return tuple(direction*i*(3 if mode=='arpeggio' else 2) for i in range(count)) + + def make(self,start,end,incoming,home,history,variant): + if not any(s.scale and s.startstart for s in self.harmony.spans): + self.proposals['space'] += 1 + evaluation = generator.solo.selection.evaluate((),start,end,self.harmony) + return generator.solo.types.Phrase('space:'+str(start),start,end,(),'space','level','open',evaluation=evaluation),() + motif = idea(home,'realized_theme') if home else self.choose(self.ideas,'source',variant) if self.ideas else None + purpose = 'statement' if not home else self.choose(('return','develop','answer','extend'),'purpose',(start,variant)) + if purpose=='return' and history and history[-1].purpose=='return': + purpose = 'develop' + self.proposals[purpose] += 1 + center = self.center+self.choose((-5,0,5),'register',(start,variant)) + if incoming: + center = max(incoming.pitch.number()-7,min(incoming.pitch.number()+7,center)) + direction = self.choose((-1,1),'direction',(start,variant)) + hold,release = self.choose(self.arrivals,'arrival',(start,variant)) + close = end==self.end or self.choose((True,True,False),'closure',(start,variant)) + arrival = max(start,end-release-hold) if close and end-start>=4 else end + notes,contracts,sources = [],[],[] + cursor = start + if start>=0 and purpose!='return': + cursor += self.choose((fractions.Fraction(),fractions.Fraction(1,2),fractions.Fraction(1)),'entrance',(start,variant)) + signature_notes = (); group_index = 0; fast_groups = 0 + while cursor=min(remaining,1) + and bool(set(purposes)&({'continue','connect','develop','answer','scalar','arpeggio'}|({'entrance'} if group_index==0 else set()))) + and (not fast or fast_groups=3: + signature_notes = tuple((*notes,*ns)) + fast_groups += int(any((b-a).denominator not in (1,2) for a,b in pattern)) + notes.extend(ns); contracts.extend(cs); sources.append(source) + cursor += length; group_index += 1 + anchor = None; ending = 'open' + if arrivalstart for s in self.harmony.spans): + return None + if self.enabled and variant%3 and notes: + notes,contracts = self.expression(tuple(notes),tuple(contracts),signature_notes,start,variant) + checked = self.harmony.evaluate(notes,expressions=contracts,until=end,complete=True) + if not checked.valid: + self.rejections[checked.error] += 1; return None + evaluation = generator.solo.selection.evaluate(notes,start,end,self.harmony,incoming=(incoming,) if incoming else (), + home=home,purpose=purpose,contracts=contracts,center=self.center) + key = generator.solo.policy.stable_rank(self.seed,(start,end,variant))[:12] + pattern = tuple((n.start-start,n.end-start) for n in notes) + try: + groups = generator.solo.groups.describe(pattern,origin=start,source=purpose) + except ValueError: + self.rejections['rhythm_group'] += 1; return None + phrase = generator.solo.types.Phrase(key,start,end,tuple(notes),purpose,'up' if direction>0 else 'down',ending, + anchor,tuple(contracts),groups,'home' if home else key,float(evaluation.total),evaluation,tuple(sources)) + return phrase,signature_notes + + def expression(self,notes,contracts,protected,start,variant): + protected_ids = {n.event_id for n in protected} + used = {n.event_id for c in contracts for n in (*c.events,c.target)} + targets = [n for n in notes if n.event_id not in used|protected_ids and n.end-n.start>=1 and n.pitch.number()%12 in self.harmony.members(n.start,n.end)] + if not targets: + return notes,contracts + target = self.choose(targets,'expression_target',(start,variant)) + previous = next((n for n in reversed(notes) if n.end<=target.start),None) + region = self.harmony.at(target.start) + editable = previous and previous.event_id not in used|protected_ids and previous.end==target.start + if editable and region.start==target.start and target is not notes[-1] and target.end-target.start>=fractions.Fraction(3,2): + for rule in generator.vocabulary.data('harmony').get('suspensions',[]): + if target.pitch.number()-previous.pitch.number() not in rule['steps']: + continue + held = dataclasses.replace(previous,end=target.start+fractions.Fraction(1,2),event_id=previous.event_id+':held',role='suspension') + resolved = dataclasses.replace(target,start=held.end,event_id=target.event_id+':resolved') + c = generator.solo.types.ExpressionContract(held.event_id,'suspension',rule['id'],(held,),resolved,region.chord,boundary=region.start) + revised = tuple(held if n is previous else resolved if n is target else n for n in notes) + if self.harmony.evaluate(revised,expressions=(*contracts,c),complete=True).valid: + return revised,(*contracts,c) + # Anticipation changes the actual rhythm before phrase selection. The + # original committed arrival interval remains sounding in its entirety. + if editable and region.start==target.start and previous.end-previous.start>=1: + early = dataclasses.replace(target,start=target.start-fractions.Fraction(1,2),event_id=target.event_id+':early') + before = dataclasses.replace(previous,end=early.start,event_id=previous.event_id+':short') + contract = generator.solo.types.ExpressionContract(early.event_id,'anticipation','early_chord_tone',(early,),early,region.chord,boundary=region.start) + revised = tuple(early if n is target else before if n is previous else n for n in notes) + if self.harmony.evaluate(revised,expressions=(*contracts,contract),complete=True).valid: + return revised,(*contracts,contract) + # Altered colors resolve to an existing destination; the rule, not the + # annotation or accidental, authorizes their complete exposed interval. + if editable and variant%2 and previous.end-previous.start<=1: + span = self.harmony.at(previous.start) + for rule in generator.vocabulary.data('harmony')['colors']: + if not rule.get('resolve') or not generator.solo.expressions.eligible_color(rule,span,self.harmony): + continue + pitch = generator.solo.expressions.degree_pitch(generator.solo.expressions.parsed_chord(span.chord),rule['degree'],rule['alter'],target.pitch.number()) + color = dataclasses.replace(previous,pitch=pitch,event_id=previous.event_id+':color',role='color') + c = generator.solo.types.ExpressionContract(color.event_id,'color',rule['id'],(color,),target,region.chord) + revised = tuple(color if n is previous else n for n in notes) + if self.harmony.profile.permits(pitch.number()) and self.harmony.evaluate(revised,expressions=(*contracts,c),complete=True).valid: + return revised,(*contracts,c) + rules = [r for r in generator.vocabulary.data('lead_ins')['patterns'] if r['kind'] in ('approach','enclosure') and (not r.get('difficulty') or self.analysis.tempo<=generator.vocabulary.data('roles')['difficulty'][r['difficulty']+'_max_bpm'])] + rules = sorted(rules,key=lambda r:generator.solo.policy.stable_rank(self.seed+'expression',(start,variant,r['id']))) + for rule in rules: + gaps = tuple(map(fractions.Fraction,rule.get('internal_gaps',[]))) + cursor = target.start-sum(map(fractions.Fraction,rule['durations']),fractions.Fraction())-sum(gaps,fractions.Fraction())-fractions.Fraction(rule['gap']) + affected = [n for n in notes if n.startcursor] + if cursor7 or before.end==gesture[0].start and before.pitch==gesture[0].pitch): + continue + revised = tuple(sorted((*[n for n in notes if n not in affected],*gesture),key=lambda n:n.start)) + contract = generator.solo.types.ExpressionContract(target.event_id+':'+rule['id'],rule['kind'],rule['id'],tuple(gesture),target,self.harmony.at(target.start).chord) + if self.harmony.evaluate(revised,expressions=(*contracts,contract),complete=True).valid: + return revised,(*contracts,contract) + return notes,contracts diff --git a/generator/solo/policy.py b/generator/solo/policy.py index 021edf9..a82039f 100644 --- a/generator/solo/policy.py +++ b/generator/solo/policy.py @@ -4,7 +4,7 @@ import dataclasses, hashlib class Policy: long_note: float = 2. stopping_note: float = 1. - composition_version: str = 'phrase-frontiers-2' + composition_version: str = 'melodic-phrases-1' DEFAULT=Policy() def stable_rank(seed, value): return hashlib.sha256((seed + repr(value)).encode()).hexdigest() diff --git a/generator/solo/profiles.py b/generator/solo/profiles.py index 241328c..8cb128b 100644 --- a/generator/solo/profiles.py +++ b/generator/solo/profiles.py @@ -25,24 +25,9 @@ class Instrument: def permits(self, number): return any(p.string_number in self.strings for p in generator.fretboard.single_positions_for_pitch(F(number))) - def cost(self, number): - return .5 * min(1., max(0, abs(number - self.reference) - 7) / 12) if self.reference is not None else 0. - - UNRESTRICTED = Instrument() -def weighted_center(notes): - values = sorted((n.pitch.number(), n.end - n.start) for n in notes) - total = sum((d for _, d in values), F()) - elapsed = F() - for number, duration in values: - elapsed += duration - if elapsed >= total / 2: - return number - return 60 - - def for_part(part, analysis=None): if part not in ('acoustic', 'electric'): raise ValueError('unknown solo instrument: ' + part) diff --git a/generator/solo/report.py b/generator/solo/report.py index 7374db6..3c5fe73 100644 --- a/generator/solo/report.py +++ b/generator/solo/report.py @@ -36,7 +36,7 @@ def structure(solo): def describe(solo,scales,measure_offset=0,time_offset=0,*,detail=None): detail=detail or structure(solo) - lines=[f'Independent joint-action solo: {solo.part}', 'Times are exact quarter beats from the chorus start; negative times are allocated lead-ins. Pitches sound one octave below guitar notation.', + lines=[f'Independent melodic-phrase solo: {solo.part}', 'Times are exact quarter beats from the chorus start; negative times are allocated lead-ins. Pitches sound one octave below guitar notation.', 'Approximate fingering costs are not a biomechanical guarantee.',json.dumps(detail['phrasing'])] for p in solo.phrases: lines.append(f'{p.id}: {p.start}–{p.end}; {p.purpose}, {p.direction}, {p.ending}; theme {p.theme}; score {p.score:.4f}') diff --git a/generator/solo/rhythm.py b/generator/solo/rhythm.py index f13d2e9..0ddbd56 100644 --- a/generator/solo/rhythm.py +++ b/generator/solo/rhythm.py @@ -17,7 +17,7 @@ class Rhythms: self.groups.append((item['id'],tuple(pattern),cursor,tuple(item['purpose']))) for cell in vocabulary.rhythm_cells(tempo): # Imported authored cells remain whole. They are never sliced to fit. - if groups.valid(cell.pattern):self.groups.append((cell.id,cell.pattern,cell.duration,('develop','answer') if cell.purpose=='answer' else ('arpeggio','scalar','develop','neighbor'))) + if groups.valid(cell.pattern):self.groups.append((cell.id,cell.pattern,cell.duration,(cell.purpose,))) # Meter-indexed catalog representatives: file counts do not set style frequency. self.catalog={} for meter,items in library.snippets.items(): @@ -30,4 +30,3 @@ class Rhythms: descriptor=(len(pattern),pattern[0][0]>0,any(a[1]1024: - raise ValueError('Unsupported fitted feature (only shared musical categories): '+key) - if any(c not in 'abcdefghijklmnopqrstuvwxyz_0123456789/-=&' for c in key): - raise ValueError('Noncanonical fitted feature: '+key) - return dict(weights) +VERSION = 'melodic-phrases-1' +WEIGHTS = {'continuity': 2, 'repetition': 3, 'interruption': 2, 'activity': 2, + 'harmony': 2, 'register': 1, 'identity': 3} def signature(notes): - if not notes:return () - first=notes[0] - return tuple((n.start-first.start,n.end-n.start,n.pitch.number()-first.pitch.number()) for n in notes) + notes = generator.solo.types.sounding_notes(notes) + if not notes: + return () + return tuple((n.start-notes[0].start, n.end-n.start, n.pitch.number()-notes[0].pitch.number()) for n in notes) -def matches_signature(notes,expected): - """Exact signature equality, stopping at the first differing feature.""" - if len(notes)!=len(expected):return False - if not notes:return True - first=notes[0];pitch=first.pitch.number() - return all(n.pitch.number()-pitch==relative and n.end-n.start==duration and n.start-first.start==onset - for n,(onset,duration,relative) in zip(notes,expected)) - - -class Selector: - @property - def weights(self):return self._weights - @weights.setter - def weights(self,value):self._weights=MappingProxyType(validate_weights(value)) - - def __init__(self,harmony,analysis,center,weights=None): - self.harmony=harmony;self.analysis=analysis;self.center=center - self.weights=validate_weights(weights if weights is not None else vocabulary.role().get('selection_weights',{})) - self.source_intervals=Counter(b.pitch.number()-a.pitch.number() for a,b in zip(analysis.melody,analysis.melody[1:])) - self.source_pairs=Counter((b.pitch.number()-a.pitch.number(),c.pitch.number()-b.pitch.number()) - for a,b,c in zip(analysis.melody,analysis.melody[1:],analysis.melody[2:])) - self.source_rhythms=Counter(n.end-n.start for n in analysis.melody) - self.static={} - self.home_signatures={} - self.cost_static={};self.cost_weights=None - - def contexts(self,notes,recent=()): - history=list(recent[-2:]) - for n in notes: - region=self.harmony.at(n.start) - if region and region.chord: - chord=expressions.parsed_chord(region.chord) - root=int(model.NATURAL[chord['root']['letter']]+model.alteration(chord['root']['alter'])) - delta=n.pitch.number()-history[-1].pitch.number() if history else None - previous=history[-1].pitch.number()-history[-2].pitch.number() if len(history)>1 else None - metric='barline' if n.start+self.analysis.body_offset in self.analysis.boundaries else 'beat' if n.start%1==0 else 'eighth' if n.start%1==F(1,2) else 'subdivision' - context={'quality':expressions.quality(chord),'degree':str((n.pitch.number()-root)%12), - 'metric':metric,'duration':str(n.end-n.start).replace('/','_'), - 'source_pair':'yes' if (previous,delta) in self.source_pairs else 'no'} - if delta is not None:context['motion']=str(max(-12,min(12,delta))) - yield context,int((n.end-n.start)*12) - history.append(n) - - def base(self,action): - if action.key in self.static:return self.static[action.key] - values=Counter();prior=Counter();ticks=int((action.end-action.start)*12) - for contract in action.contracts: - if contract.target.start>=action.end and contract.events: - # Pending destinations must be visible to selection. Identical heard - # attacks with different future obligations are different actions. - last=contract.events[-1];target=contract.target - values[f'obligation/step/{max(-12,min(12,target.pitch.number()-last.pitch.number()))}']+=ticks - values['obligation/delay/'+str(target.start-last.start).replace('/','_')]+=ticks - values['obligation/hold/'+str(target.end-target.start).replace('/','_')]+=ticks - for n in action.notes: - f,p=self.note_base(n.start,n.end,n.pitch.number()) - values.update(f);prior.update(p) - self.static[action.key]=(values,prior) - if len(self.static)>8192:self.static.pop(next(iter(self.static))) - return values,prior - - @functools.lru_cache(maxsize=32768) - def note_base(self,start,end,number): - values=Counter();prior=Counter() - duration=end-start;length=int(duration*12) - label=str(duration).replace('/','_') - values['duration/'+label]+=length - beat=(start%1) - metric='barline' if start+self.analysis.body_offset in self.analysis.boundaries else 'beat' if beat==0 else 'eighth' if beat==F(1,2) else 'subdivision' - values['metric/'+metric+'/'+label]+=length - band=max(-4,min(4,(number-self.center+6)//12)) - values[f'register/{band}']+=length - region=self.harmony.at(start) - if not region or not region.chord:return values,prior - chord=expressions.parsed_chord(region.chord) - root=int(model.NATURAL[chord['root']['letter']]+model.alteration(chord['root']['alter'])) - degree=(number-root)%12 - values[f'harmony/{expressions.quality(chord)}/{degree}/{label}']+=length - prior['register_distance']+=length*max(0,abs(number-self.center)-7) - if duration>=2 and number%12 in self.harmony.members(start,end): - prior['sustained_arrival']+=length - values['source/rhythm_match']+=length if duration in self.source_rhythms else 0 - return values,prior - - def features(self,action,recent=(),home=(),previous_kind=''): - if any(n.continuation for n in action.notes):action=dataclasses.replace(action,notes=types.sounding_notes(action.notes)) - static,base=self.base(action);values=Counter(static);prior=Counter(base) - # Silence is measured as intervals in the sounded stream, independent of - # the number of rest actions used to encode it. Missing harmony is distinct. - cursor=action.start;before=recent[-1] if recent else None - for n in (*action.notes,None): - until=n.start if n else action.end - if until>cursor: - f,p=self.space_features(cursor,until,None if before is None else before.end-before.start>=F(3,2)) - values.update(f);prior.update(p) - if n:cursor=n.end;before=n - previous=recent[-1] if recent else None - intervals=[b.pitch.number()-a.pitch.number() for a,b in zip(recent,recent[1:])] - for n in action.notes: - length=int((n.end-n.start)*12) - if previous: - delta=n.pitch.number()-previous.pitch.number() - token=max(-12,min(12,delta));label=str(n.end-n.start).replace('/','_') - values[f'motion/{token}/{label}']+=length - if intervals: - # Direction changes are contextual, not a blanket leap prohibition. - sign=lambda x: 'up' if x>0 else 'down' if x<0 else 'repeat' - values[f'continuation/{sign(intervals[-1])}/{sign(delta)}']+=length - if len(intervals)>=2 and intervals[-2]==-intervals[-1]==delta and delta:prior['oscillation']+=length - prior['excessive_leap']+=length*max(0,abs(delta)-7) - prior['repeated_attack']+=length if delta==0 and n.start==previous.end else 0 - # Ordered relationships, not the set of available intervals. Weight - # by sounding duration so subdivisions cannot multiply a reward. - if intervals and (intervals[-1],delta) in self.source_pairs:prior['source_relation']+=length - if 00:prior['directed_motion']+=length - intervals.append(delta) - previous=n - if len(home)>=3: - key=id(home) - if key not in self.home_signatures: - self.home_signatures[key]=(home,signature(home)) - if len(self.home_signatures)>32:self.home_signatures.pop(next(iter(self.home_signatures))) - expected=self.home_signatures[key][1] - joined=(*recent,*action.notes) - for i in range(max(0,len(recent)-len(home)+1),len(joined)-len(home)+1): - part=joined[i:i+len(home)] - if part[0].start>home[-1].end and matches_signature(part,expected): - values['theme/realized_identity']+=12 - prior['theme_relation']+=12 - joint=[(key,rule_conditions(key)) for key in self.weights if key.startswith('joint/')] - if joint: - for context,ticks in self.contexts(action.notes,recent): - for key,conditions in joint: - if all(context.get(k)==v for k,v in conditions.items()):values[key]+=ticks - return values,prior - - def complete(self,actions,home=(),*,start=None,end=None): - """The sole authoritative score. Ties/rests/partitions are canonicalized.""" - from generator.solo.operators import Action - actions=tuple(actions) - first=start if start is not None else actions[0].start - last=end if end is not None else actions[-1].end - notes=types.sounding_notes(tuple(n for a in actions for n in a.notes)) - contracts=tuple(dict.fromkeys(c for a in actions for c in a.contracts)) - return self.evaluate(Action(first,last,notes,'sounded_phrase',contracts),home=home) - - def extend(self,previous,old_notes,action,home=(),*,old_home=(),start=F(),contracts=()): - """Exact sufficient-statistic update, checked against complete() in tests. - - Reevaluate the seam and the tail containing any possible new home match. - Previously completed events are not rescored on every search expansion. - """ - from generator.solo.operators import Action - if previous is None or home!=old_home or any(n.continuation for n in action.notes): - notes=types.sounding_notes((*old_notes,*action.notes)) - return self.evaluate(Action(start,action.end,notes,'sounded_phrase',contracts),home=home) - span=F(previous.denominators['span']) - raw=Counter({k:v*span for k,v in previous.raw.items() if not k.startswith('obligation/')}) - index=max(0,len(old_notes)-max(3,len(home)+2)) - tail=old_notes[index:];context=old_notes[max(0,index-3):index] - begin=tail[0].start if index else start - before=Action(begin,action.start,tail,'sounded_phrase') - after=Action(begin,action.end,(*tail,*action.notes),'sounded_phrase') - for sign,part in ((-1,before),(1,after)): - f,p=self.features(part,context,home) - for k,v in f.items(): - if not k.startswith('obligation/'):raw[k]+=sign*v - for k,v in p.items():raw['prior/'+k]+=sign*v - obligations,_=self.base(Action(start,action.end,(),'sounded_phrase',contracts)) - raw.update({k:v for k,v in obligations.items() if k.startswith('obligation/')}) - weights={'prior/'+k:v for k,v in DEFAULT.items()}|self.weights - duration=action.end-start - raw={k:F(v)/duration for k,v in raw.items() if v} - contributions={k:v*weights.get(k,0) for k,v in raw.items()} - return scoring.Evaluation(VERSION,True,raw,weights,contributions,sum(contributions.values(),F()), - {'span':str(duration),'ticks_per_quarter':12,'attacks':len(old_notes)+len(action.notes)}) - - def evaluate(self,action,recent=(),home=(),previous_kind=''): - features,prior=self.features(action,recent,home,previous_kind) - contributions={'prior/'+k:v*DEFAULT[k] for k,v in prior.items() if v} - contributions.update({k:v*self.weights.get(k,0) for k,v in features.items() if v}) - weights={'prior/'+k:v for k,v in DEFAULT.items()}|self.weights - raw={'prior/'+k:v for k,v in prior.items() if v}|{k:v for k,v in features.items() if v} - span=action.end-action.start - contributions={k:F(v)/span for k,v in contributions.items()} - raw={k:F(v)/span for k,v in raw.items()} - total=sum(contributions.values(),F()) - return scoring.Evaluation(VERSION,True,raw,weights,contributions,total, - {'span':str(action.end-action.start),'ticks_per_quarter':12,'attacks':len(types.sounding_notes(action.notes))}) - - def cost(self,action,recent=(),home=(),previous_kind=''): - """Identical selection total without serializing every rejected breakdown.""" - features,prior=self.features(action,recent,home,previous_kind) - numerator=sum(v*DEFAULT[k] for k,v in prior.items()) - numerator+=sum(v*self.weights.get(k,0) for k,v in features.items()) - return F(numerator)/(action.end-action.start) - - @functools.lru_cache(maxsize=32768) - def transition_cost(self,delta,preceding,earlier,length,label,contiguous): - token=max(-12,min(12,delta)) - sign=lambda x:'up' if x>0 else 'down' if x<0 else 'repeat' - cost=length*self.weights.get(f'motion/{token}/{label}',0) - cost+=length*DEFAULT['excessive_leap']*max(0,abs(delta)-7) - if delta==0 and contiguous:cost+=length*DEFAULT['repeated_attack'] - if preceding is not None: - cost+=length*self.weights.get(f'continuation/{sign(preceding)}/{sign(delta)}',0) - if (preceding,delta) in self.source_pairs:cost+=length*DEFAULT['source_relation'] - if 00:cost+=length*DEFAULT['directed_motion'] - if earlier is not None and earlier==-preceding==delta and delta:cost+=length*DEFAULT['oscillation'] - return cost - - @functools.lru_cache(maxsize=32768) - def seam_cost(self,prefix,context,contiguous): - def total(initial): - pitches=list(initial);value=0 - for index,(number,length,label,touch) in enumerate(prefix): - if pitches: - value+=self.transition_cost(number-pitches[-1],pitches[-1]-pitches[-2] if len(pitches)>1 else None, - pitches[-2]-pitches[-3] if len(pitches)>2 else None,length,label,contiguous if index==0 and initial else touch) - pitches.append(number) - return value - return total(context)-total(()) - - @functools.lru_cache(maxsize=32768) - def unary_cost(self,start,end,number): - features,prior=self.note_base(start,end,number) - return sum(v*DEFAULT[k] for k,v in prior.items())+sum(v*self.weights.get(k,0) for k,v in features.items()) - - def intrinsic_cost(self,action): - """Exact ordinary retrieval cost without constructing discarded reports. - - Unary, transition, silence and pending-target contributions are identical - to features(action). Contextual learned conjunctions use the full path. - """ - notes=types.sounding_notes(action.notes) if any(n.continuation for n in action.notes) else action.notes - cost=0;history=[];cursor=action.start;settled=None;previous=None - for n in notes: - if n.start>cursor:cost+=self.gap_cost(cursor,n.start,settled) - number=n.pitch.number();length=int((n.end-n.start)*12);label=str(n.end-n.start).replace('/','_') - cost+=self.unary_cost(n.start,n.end,number) - if history: - cost+=self.transition_cost(number-history[-1],history[-1]-history[-2] if len(history)>1 else None, - history[-2]-history[-3] if len(history)>2 else None,length,label,n.start==previous.end) - history.append(number);previous=n;cursor=n.end;settled=n.end-n.start>=F(3,2) - if action.end>cursor:cost+=self.gap_cost(cursor,action.end,settled) - ticks=int((action.end-action.start)*12) - for contract in action.contracts: - if contract.target.start>=action.end and contract.events: - last=contract.events[-1];target=contract.target - cost+=ticks*self.weights.get(f'obligation/step/{max(-12,min(12,target.pitch.number()-last.pitch.number()))}',0) - cost+=ticks*self.weights.get('obligation/delay/'+str(target.start-last.start).replace('/','_'),0) - cost+=ticks*self.weights.get('obligation/hold/'+str(target.end-target.start).replace('/','_'),0) - return cost - - def fast_cost(self,action,recent=(),context=None): - """Unnormalized endpoint-bucket retrieval, with cached internal routes. - - Only the first three transitions depend on incoming history. Final ranking - always uses complete(); this heuristic does not compare unequal spans. - """ - # Learned conjunctions include incoming context; do not use a shortcut - # that forgets it. This small fallback is explicit and profileable. - if any(n.continuation for n in action.notes) or any(k.startswith('joint/') for k in self.weights): - f,p=self.features(action,recent) - return sum(v*DEFAULT[k] for k,v in p.items())+sum(v*self.weights.get(k,0) for k,v in f.items()) - if self.cost_weights is not self.weights: - self.cost_static.clear();self.transition_cost.cache_clear();self.seam_cost.cache_clear();self.gap_cost.cache_clear();self.unary_cost.cache_clear();self.cost_weights=self.weights - key=id(action) - if key not in self.cost_static: - value=self.intrinsic_cost(action) - prefix=tuple((n.pitch.number(),int((n.end-n.start)*12),str(n.end-n.start).replace('/','_'), - i>0 and action.notes[i-1].end==n.start) for i,n in enumerate(action.notes[:3])) - self.cost_static[key]=(action,value,prefix) - if len(self.cost_static)>32768:self.cost_static.pop(next(iter(self.cost_static))) - _,value,prefix=self.cost_static[key] - if context is None:context=(tuple(n.pitch.number() for n in recent[-3:]),recent[-1].end if recent else None) - if context[0] and prefix:value+=self.seam_cost(prefix,context[0],context[1]==action.notes[0].start) - gap_end=action.notes[0].start if action.notes else action.end - if recent and gap_end>action.start: - # Incoming sustained arrivals change the role of a leading rest, just - # as incoming pitches change the first intervals. Do not cache it as a - # fresh silent entrance in every action. - value+=self.gap_cost(action.start,gap_end,recent[-1].end-recent[-1].start>=F(3,2))-self.gap_cost(action.start,gap_end,None) - return value - - @functools.lru_cache(maxsize=8192) - def space_features(self,start,end,settled): - values=Counter();prior=Counter();delayed=0 - boundaries=sorted({start,end,*[s.start for s in self.harmony.spans if startstart for s in harmony.spans) + activity = max(0,density-7)+(2 if active and not notes else 0) + if notes and sounding0)!=(b[2]>0) for a,b in zip(signature(home),signature(sample)))/len(home) + register = sum(float(n.end-n.start)*max(0,abs(n.pitch.number()-center)-10) for n in notes)/max(1,float(sounding)) + values = dict(continuity=continuity,repetition=repetition,interruption=interruption, + activity=activity,harmony=checked.exposure,register=register,identity=identity) + raw = {k:fractions.Fraction(str(v)).limit_denominator(1000000) for k,v in values.items()} + contributions = {k:v*WEIGHTS[k] for k,v in raw.items()} + return generator.solo.scoring.Evaluation(VERSION,checked.valid,raw,dict(WEIGHTS),contributions, + sum(contributions.values(),fractions.Fraction()),{'span':str(duration),'attacks':len(notes),'sounding':str(sounding)}) diff --git a/generator/solo/sequences.py b/generator/solo/sequences.py index 4460ef8..37f0af2 100644 --- a/generator/solo/sequences.py +++ b/generator/solo/sequences.py @@ -5,8 +5,6 @@ import pathlib import types import functools -import generator.solo.policy - import generator.model import generator.musicxml @@ -24,30 +22,6 @@ class Sequence: offsets = self.offsets() return max(map(abs, self.steps)), max(offsets) - min(offsets), sum(a * b < 0 for a, b in zip(self.steps, self.steps[1:])), self.id -@dataclasses.dataclass(frozen=True) -class Path: - steps: tuple[int, ...] - sources: tuple[str, ...] - @functools.lru_cache(maxsize=32768) - def offsets(self): - result = [0] - for step in self.steps: - result.append(result[-1] + step) - return tuple(result) - -@functools.lru_cache(maxsize=32768) -def family(steps): - if not steps or all(s == 0 for s in steps): - return 'repeated' - turns = sum(a * b < 0 for a, b in zip(steps, steps[1:])) - if not turns: - return 'rising' if sum(steps) > 0 else 'falling' - if max(map(abs, steps)) >= 4: - return 'leap_return' - if sum(steps) == 0 and max(map(abs, steps)) == 1: - return 'neighbor' - return ('arch' if steps[0] > 0 else 'valley') if turns == 1 else 'wave' - def diverse(items, rank, category, limit): """Round-robin eligible musical families before a destructive cutoff.""" groups = {} @@ -70,7 +44,6 @@ def diverse(items, rank, category, limit): @dataclasses.dataclass(frozen=True, init=False) class Library: paths: types.MappingProxyType - chains: dict def __init__(self, sequences: tuple[Sequence, ...]): paths, identities = {}, set() for sequence in sequences: @@ -79,7 +52,6 @@ class Library: identities.add(sequence.id) paths.setdefault(len(sequence.steps) + 1, []).append(sequence) object.__setattr__(self, 'paths', types.MappingProxyType({count: tuple(sorted(items, key=lambda s: s.rank())) for count, items in paths.items()})) - object.__setattr__(self, 'chains', {}) def connecting(self, count: int, displacement: int | None = None) -> tuple[Sequence, ...]: return tuple(s for s in self.paths.get(count, ()) if displacement is None or s.offsets()[-1] == displacement) diff --git a/generator/vocabulary.py b/generator/vocabulary.py index f7aa63f..8542a5e 100644 --- a/generator/vocabulary.py +++ b/generator/vocabulary.py @@ -32,8 +32,7 @@ def _load(contents): cells=result['rhythms']['cells']+result['solo']['cells'] if len({c['id'] for c in cells})!=len(cells):raise ValueError('Duplicate rhythm identity') rules=result['solo'] - if any(type(rules['operators'][k]) is not int or rules['operators'][k]<=0 for k in ('histories','completed','per_state')):raise ValueError('Invalid deterministic search budget') - if any(F(d)<=0 for d in rules['operators']['durations']):raise ValueError('Invalid action duration') + if any(type(rules['search'][k]) is not int or rules['search'][k]<=0 for k in ('histories','alternatives')):raise ValueError('Invalid deterministic search budget') for group in rules['groups']: if not group['values'] or any(F(v.removeprefix('r:'))<=0 for v in group['values']):raise ValueError('Invalid rhythm group') return result diff --git a/tests/evaluation_sources/README.md b/tests/evaluation_sources/README.md index c007f97..b0d5d54 100644 --- a/tests/evaluation_sources/README.md +++ b/tests/evaluation_sources/README.md @@ -6,4 +6,4 @@ The manually transcribed first four chorus measures retain the vocal rhythm, inc St. Louis Blues was considered, but the institutional original-score download was unavailable (403). Hindustan supplies the third independent source instead; no target arrangement was used to reconstruct missing music. -Run `python -m audition.transfer_review --output build/transfer_review` through the repository Python runner. This runs the normal independent generators; no tuning/selection result flows back into production. +Evaluate these inputs using the ordinary `python -m generator.compose --song ... --role ... --data-dir data --output ...` command. They are transfer checks, not sources of production vocabulary. diff --git a/data/expected/performance_summary.json b/tests/fixtures/performance_summary.json similarity index 100% rename from data/expected/performance_summary.json rename to tests/fixtures/performance_summary.json diff --git a/audition/solo_compare.py b/tests/solo_compare.py similarity index 53% rename from audition/solo_compare.py rename to tests/solo_compare.py index 054cda8..2b53462 100644 --- a/audition/solo_compare.py +++ b/tests/solo_compare.py @@ -1,5 +1,137 @@ -"""Explicit post-generation comparison. Never imports or runs a composer.""" +"""Read optional reference MusicXML literally, without repairing it or composing.""" from __future__ import annotations +import argparse +import hashlib +import json +from pathlib import Path +from fractions import Fraction as F +import xml.etree.ElementTree as ET + + +def read(path, role, form_start=None, pitch_convention='encoded'): + if pitch_convention not in ('encoded','guitar-written'): + raise ValueError('Unknown reference pitch convention: '+pitch_convention) + raw=Path(path).read_bytes();tree=ET.fromstring(raw) + result={'path':str(Path(path).resolve()),'sha256':hashlib.sha256(raw).hexdigest(),'validation_role':role, + 'pitch_convention':pitch_convention,'part_names':[n.text for n in tree.findall('./part-list/score-part/part-name')], + 'measures':[],'notes':[],'harmonies':[],'annotations':[],'discrepancies':[]} + divisions=F(1);absolute=F();nominal=F(4);pending={};active=None;transpose=F() + for index,m in enumerate(tree.findall('./part/measure')): + cursor=F();extent=F();last_start=F();last_duration=F() + if (value:=m.findtext('attributes/divisions')) is not None:divisions=F(value) + meter=m.find('attributes/time') + if meter is not None and meter.findtext('beats'): + nominal=sum((F(v) for v in meter.findtext('beats').split('+')),F())*4/F(meter.findtext('beat-type')) + for item in m: + offset=F(item.findtext('offset','0'))/divisions + if item.tag=='attributes' and item.find('transpose') is not None: + transpose=F(item.findtext('transpose/chromatic','0'))+12*F(item.findtext('transpose/octave-change','0')) + elif item.tag=='backup':cursor-=F(item.findtext('duration'))/divisions + elif item.tag=='forward':cursor+=F(item.findtext('duration'))/divisions;extent=max(extent,cursor) + elif item.tag=='harmony': + active={'time':str(absolute+cursor+offset),'measure':index+1,'offset':str(cursor+offset), + 'root':item.findtext('root/root-step'),'alter':item.findtext('root/root-alter','0'), + 'kind':item.findtext('kind'),'display':item.find('kind').get('text') if item.find('kind') is not None else None, + 'bass':ET.tostring(item.find('bass'),encoding='unicode') if item.find('bass') is not None else None, + 'degrees':[{'value':d.findtext('degree-value'),'alter':d.findtext('degree-alter'),'type':d.findtext('degree-type')} for d in item.findall('degree')]} + result['harmonies'].append(active) + elif item.tag=='direction': + for words in item.findall('direction-type/words'): + result['annotations'].append({'time':str(absolute+cursor+offset),'measure':index+1,'text':''.join(words.itertext()),'interpretation':'descriptive; no production permission'}) + elif item.tag=='note': + duration=F(item.findtext('duration','0'))/divisions + start=last_start if item.find('chord') is not None else cursor + end=start+duration;voice=item.findtext('voice','1');p=item.find('pitch') + encoded={'step':p.findtext('step'),'alter':p.findtext('alter','0'),'octave':int(p.findtext('octave'))} if p is not None else None + kinds={t.get('type') for t in item.findall('tie')} | {t.get('type') for t in item.findall('notations/tied')} + event={'start':str(absolute+start),'end':str(absolute+end),'measure':index+1,'offset':str(start),'pitch':encoded,'rest':item.find('rest') is not None, + 'written_type':item.findtext('type'),'dots':len(item.findall('dot')),'voice':voice,'ties':sorted(kinds), + 'sounding_transposition':str(-12 if pitch_convention=='guitar-written' else transpose), + 'tablature':{'string':int(item.findtext('notations/technical/string')),'fret':int(item.findtext('notations/technical/fret'))} if item.find('notations/technical/string') is not None and item.find('notations/technical/fret') is not None else None, + 'time_modification':ET.tostring(item.find('time-modification'),encoding='unicode') if item.find('time-modification') is not None else None, + 'tuplets':[dict(t.attrib) for t in item.findall('notations/tuplet')],'grace':item.find('grace') is not None,'new_attack':'stop' not in kinds and p is not None} + key=(voice,json.dumps(encoded,sort_keys=True)) + if 'stop' in kinds: + old=pending.get(key) + if old is None or result['notes'][old]['end']!=event['start']: + result['discrepancies'].append({'measure':index+1,'issue':'unmatched/noncontiguous tie; retained literally'}) + else:event['continuation_of']=old + if 'start' not in kinds:pending.pop(key,None) + if 'start' in kinds:pending[key]=len(result['notes']) + result['notes'].append(event) + for container in ('notations/technical/fingering','notations/technical/other-technical','notations/other-notation'): + for label in item.findall(container): + text=''.join(label.itertext()).strip() + if text:result['annotations'].append({'time':str(absolute+start),'measure':index+1,'text':text, + 'container':container,'interpretation':'Original note annotation; container does not establish musical permission or guitar fingering.'}) + if item.find('chord') is None and item.find('grace') is None:cursor=end + last_start,last_duration=start,duration;extent=max(extent,end,cursor) + if extent!=nominal and m.get('implicit')!='yes':result['discrepancies'].append({'measure':index+1,'issue':f'encoded span {extent} differs from nominal {nominal}; no padding applied'}) + result['measures'].append({'number':m.get('number'),'start':str(absolute),'duration':str(extent),'implicit':m.get('implicit'),'initial_harmony_known':any(F(h['time'])<=absolute for h in result['harmonies'])}) + absolute+=extent + if pending:result['discrepancies'].append({'issue':'unterminated ties retained'}) + result['harmonies'].sort(key=lambda h:F(h['time'])) + # Do not run the strict score importer over these references: annotations, + # unusual timing and high encoded pitches are evidence, not repair requests. + from generator import harmony, model, fretboard + def semantic(h): + return {'root':{'letter':h['root'].lower(),'alter':model.rational(F(h['alter']))},'kind':h['kind'], + 'degrees':[{'value':int(d['value']),'alter':model.rational(F(d['alter'])),'type':d['type']} for d in h['degrees']]} + for event in result['notes']: + if not event['pitch']:continue + p=event['pitch'];number=12*(p['octave']+1)+model.NATURAL[p['step'].lower()]+F(p['alter']) + event['encoded_midi']=str(number) + event['sounding_midi']=str(number+F(event['sounding_transposition'])) + event['legal_positions_at_encoded_pitch']=len(fretboard.single_positions_for_pitch(number)) + event['legal_positions_at_sounding_pitch']=len(fretboard.single_positions_for_pitch(F(event['sounding_midi']))) + event['harmonic_spans']=[] + start,end=F(event['start']),F(event['end']) + changes=[F(h['time']) for h in result['harmonies'] if startt),None) + label['current_harmony']=current[-1] if current else None + label['following_harmony']=future + label['events']=[n for n in result['notes'] if F(n['start'])<=tPrevious generated MusicXML

') + rows.append('') + for key in ('pitch_sequence_edit_distance','timed_attack_edit_distance','same_sounding_pitch_beats','same_pitch_class_beats','either_sounding_beats'): + rows.append('') + rows.append('
Diagnostic against referenceCurrentBaseline
'+key.replace('_',' ')+''+str(r['relationships'][key])+''+str(r.get('baseline_relationships',{}).get(key,'—'))+'

Lower edit distance means fewer insertions, deletions or substitutions on that axis. Sounding agreement is measured in quarter-note beats, without octave shifting or fitted alignment. These do not measure musical quality.

') selected=collections.Counter(c['kind'] for c in r['production_phrases'].get('expressions',[])) rows.append('

Selected, validated production treatments: '+html.escape(str(dict(selected)))+'. Ordinary mapped material remains separate.

') if r.get('reference_events_after_production_end'): diff --git a/tests/test_joint_actions.py b/tests/test_joint_actions.py deleted file mode 100644 index 689c4d4..0000000 --- a/tests/test_joint_actions.py +++ /dev/null @@ -1,233 +0,0 @@ -"""Independently authored operator, fitting-boundary and cache regressions.""" -import copy -import dataclasses -from fractions import Fraction as F -from pathlib import Path -import tempfile -import unittest -import contextlib -import io -from unittest.mock import patch - -from generator import resources, vocabulary -from generator.solo import expressions, harmonic, operators, selection, types -from tests import solo_fixtures as fixture, rhythm_fixtures, scale_fixtures -from tests.test_phrase_composer import builder - -class Operators(unittest.TestCase): - def test_supported_sixteenth_tail_can_finish_the_allocation(self): - b=builder(('CM7',),count=1) - actions=b.operators.at(F(15,4)) - self.assertTrue(any(a.kind=='tone' and a.end==4 for a in actions)) - # A source/role cutoff does not require a new long cadence. - b.start=F(15,4) - phrases=b.compose() - self.assertEqual(phrases[-1].end,F(4)) - self.assertLessEqual(sum(len(p.notes) for p in phrases),1) - - def test_source_relation_uses_order_and_does_not_reward_subdivision(self): - b=builder(('CM7',),count=1) - slow=operators.Action(F(),F(4),tuple(fixture.note(i,i+1,p,event_id=str(i)) for i,p in enumerate(('c','d','e','f'))),'scalar') - fast=dataclasses.replace(slow,end=F(2),notes=tuple(dataclasses.replace(n,start=n.start/2,end=n.end/2) for n in slow.notes)) - self.assertEqual(b.selector.evaluate(slow).raw['prior/source_relation'],b.selector.evaluate(fast).raw['prior/source_relation']) - changed=dataclasses.replace(b.analysis,melody=tuple(fixture.note(i,i+1,p,event_id=str(i)) for i,p in enumerate(('e','f','g','a')))) - other=selection.Selector(b.harmony,changed,b.selector.center,{}) - self.assertNotEqual(b.selector.evaluate(slow).raw.get('prior/source_relation',0),other.evaluate(slow).raw.get('prior/source_relation',0)) - - @unittest.skipUnless(__import__('importlib.util',fromlist=['find_spec']).find_spec('z3'),'development-only pinned solver not installed') - def test_sparse_integer_intervals_are_exact_and_role_separable(self): - from audition.structured_learning import solve - rows=[{'role':'acoustic_solo','delta':{'duration/1':F(2,3)},'prior':F(1,2),'case':'independent'}, - {'role':'electric_solo','delta':{'duration/1_2':F(-1,2)},'prior':F(1,4),'case':'independent'}] - result=solve(rows) - self.assertEqual(result['weights'],{'acoustic_solo':{'duration/1':-3},'electric_solo':{'duration/1_2':3}}) - self.assertEqual(result,solve(list(reversed(rows)))) - impossible=[*rows,{'role':'acoustic_solo','delta':{'duration/1':F(-2,3)},'prior':F(1,2),'case':'opposite'}] - self.assertEqual(solve(impossible)['status'],'incompatible_with_current_features_and_bounds') - - def test_full_rollout_uses_the_requested_data_and_role_context(self): - import copy - from generator import vocabulary - from audition.fit_selection import fit - resources=copy.deepcopy(vocabulary.current()['data']) - resources['roles']['difficulty']['triplet_max_bpm']=123 - b=builder(('CM7',),count=1) - action=operators.Action(F(),F(4),(fixture.note(0,4,'c'),),'tone') - def proposed(at,**kwargs): - self.assertEqual(vocabulary.current()['role'],'acoustic_solo') - self.assertEqual(vocabulary.data('roles')['difficulty']['triplet_max_bpm'],123) - return (action,) if at==0 else () - with patch.object(b.operators,'at',side_effect=proposed),contextlib.redirect_stdout(io.StringIO()): - result=fit([('synthetic','acoustic_solo',b,(action,))],1,resources=resources) - self.assertTrue(result['complete']) - - def test_complete_training_rounds_accumulate_only_compatible_evidence(self): - from audition.train_completed import training_context,previous_constraints - data=copy.deepcopy(vocabulary.current()['data']) - context=training_context(data) - data['roles']['roles']['electric_solo']['selection_weights']={'duration/1':7} - data['roles']['provenance']='Explicit development trial' - self.assertEqual(context,training_context(data)) - cases={'example':{'role':'electric_solo','source_sha256':'source','target_sha256':'example'}} - old={'schema':'complete-candidate-training-1','context':context,'training_dependency':cases, - 'constraints':[{'case':'example','role':'electric_solo','delta':{'duration/1':'2/3'},'prior':'1/2'}]} - row=previous_constraints(old,cases,context)[0] - self.assertEqual(row['delta']['duration/1'],F(2,3));self.assertEqual(row['prior'],F(1,2)) - changed=copy.deepcopy(cases);changed['example']['target_sha256']='different' - with self.assertRaises(ValueError):previous_constraints(old,changed,context) - data['roles']['roles']['electric_solo']['register_center']+=12 - with self.assertRaises(ValueError):previous_constraints(old,cases,training_context(data)) - bad=copy.deepcopy(old);bad['constraints'][0]['delta']={'song/example':1} - with self.assertRaises(ValueError):previous_constraints(bad,cases,context) - - def test_coverage_diagnostic_does_not_accept_a_shortened_target_hold(self): - from audition.structured_learning import CoverageTrace - from generator.solo.construction import State - held=fixture.note(0,2,'e') - target=operators.Action(F(),F(2),(held,),'tone') - trace=CoverageTrace((target,)) - self.assertFalse(trace.matches((dataclasses.replace(held,end=F(1)),),F(1))) - self.assertTrue(trace.matches((held,),F(2))) - # A computational cut through an already committed hold is different - # from selecting a shorter attack and promising to repair it afterward. - s=State(action=target,notes=(held,)) - trace('frontier',F(1),None,(s,),(s,)) - self.assertFalse(trace.records) - - @unittest.skipUnless(__import__('importlib.util',fromlist=['find_spec']).find_spec('z3'),'development-only pinned solver not installed') - def test_shared_fitter_learns_an_independent_synthetic_decision(self): - from audition.fit_selection import fit - b=builder(('CM7',),count=1) - low=operators.Action(F(),F(4),(fixture.note(0,4,'c',event_id='low'),),'tone') - high=operators.Action(F(),F(4),(fixture.note(0,4,'e',event_id='high'),),'tone') - b.selector.weights={} - with patch.object(b.operators,'at',side_effect=lambda at,**kwargs: (low,high) if at==0 else ()),contextlib.redirect_stdout(io.StringIO()): - result=fit([('synthetic','acoustic_solo',b,(high,))],10) - self.assertTrue(result['complete']) - b.selector.weights=result['weights']['acoustic_solo'] - self.assertLess(b.selector.evaluate(high).total,b.selector.evaluate(low).total) - selection.validate_weights(b.selector.weights) - - @unittest.skipUnless(__import__('importlib.util',fromlist=['find_spec']).find_spec('z3'),'development-only pinned solver not installed') - def test_fitting_normalization_covers_pickup_plus_whole_note(self): - from audition.fit_selection import fit - b=builder(('CM7',),count=2);b.end=F(17,4) - low=operators.Action(F(),b.end,(fixture.note(0,F(1,4),'c',event_id='lead'),fixture.note(F(1,4),b.end,'e',event_id='low')),'anticipation') - high=dataclasses.replace(low,notes=(low.notes[0],fixture.note(F(1,4),b.end,'g',event_id='high'))) - b.selector.weights={} - with patch.object(b.operators,'at',side_effect=lambda at,**kwargs: (low,high) if at==0 else ()),contextlib.redirect_stdout(io.StringIO()): - result=fit([('synthetic','acoustic_solo',b,(high,))],10) - self.assertTrue(result['complete']) - selection.validate_weights(result['weights']['acoustic_solo']) - - def test_anticipated_reattack_is_not_an_invented_tie(self): - b=builder(('C7','FM7'),count=2) - actions=b.operators.at(F(7,2)) - a=next(a for a in actions if a.kind=='anticipation' and a.source=='early_reattack') - self.assertEqual(len(a.notes),2) - self.assertEqual(a.notes[0].end,a.notes[1].start) - self.assertEqual(a.notes[0].pitch,a.notes[1].pitch) - self.assertFalse(a.notes[1].continuation) - self.assertTrue(b.harmony.evaluate(a.notes,expressions=a.contracts).valid) - self.assertFalse(b.harmony.evaluate(a.notes[:1],expressions=a.contracts).valid) - - def test_open_motion_and_all_color_octaves_are_constructed(self): - b=builder(('C7','FM7'),count=2) - actions=b.operators.at(F()) - self.assertTrue(any(len(a.notes)==1 and a.notes[0].pitch.number()%12==2 and a.end==F(1,2) for a in actions)) - h=builder(('E7','Am7'),count=2) - values=h.operators.at(F()) - self.assertTrue(any(a.kind=='color' and len(a.notes)==1 and a.notes[0].pitch.number()==66 for a in values)) - self.assertTrue(any(a.kind=='rest' for a in values)) - - def test_anticipation_color_has_actual_future_resolution(self): - h=harmonic.Harmony(fixture.choices(('Dm7','G7','CM7'))) - held=fixture.note(F(7,2),F(11,2),'c',alter=1,event_id='held') - target=fixture.note(F(11,2),7,'d',event_id='target') - c=types.ExpressionContract('anticipated','anticipated_color','lydian_color',(held,),target,'G7',boundary=F(4)) - self.assertTrue(h.evaluate((held,target),expressions=(c,)).valid) - self.assertFalse(h.evaluate((held,),expressions=(c,)).valid) - changed=dataclasses.replace(target,pitch=dataclasses.replace(target.pitch,octave=5)) - self.assertFalse(h.evaluate((held,changed),expressions=(c,)).valid) - self.assertFalse(h.evaluate((held,target)).valid) - - def test_delayed_approach_is_pending_and_expires(self): - h=harmonic.Harmony(fixture.choices(('Dm7',))) - approach=fixture.note(0,F(1,2),'c',alter=1,event_id='approach') - target=fixture.note(3,F(7,2),'d',event_id='target') - c=types.ExpressionContract('delayed','delayed_approach','bounded_semitone_resolution',(approach,),target,'Dm7') - r=h.evaluate((approach,),expressions=(c,),until=F(1,2),complete=False) - self.assertTrue(r.valid,r.error);self.assertEqual(r.pending,(c,)) - self.assertFalse(h.evaluate((approach,),expressions=(c,),until=4,complete=True).valid) - self.assertTrue(h.evaluate((approach,target),expressions=(c,)).valid) - - def test_distinct_pending_targets_are_visible_to_the_selector(self): - b=builder(('Dm7',),count=2) - actions=[a for a in b.operators.at(F()) if a.kind=='delayed_approach'] - first=actions[0] - second=next(a for a in actions if a.notes==first.notes and a.contracts[0].target.start!=first.contracts[0].target.start) - left,_=b.selector.features(first);right,_=b.selector.features(second) - self.assertNotEqual(left,right) - selection.validate_weights(dict.fromkeys(left,1)) - - def test_missed_pending_target_is_rejected_before_expensive_validation(self): - from generator.solo.construction import State - b=builder(('Dm7',),count=2) - approach=fixture.note(0,F(1,2),'c',alter=1,event_id='approach') - target=fixture.note(3,4,'d',event_id='target') - c=types.ExpressionContract('delayed','delayed_approach','bounded_semitone_resolution',(approach,),target,'Dm7') - state=State(pending=(c,)) - wrong=operators.Action(F(3),F(4),(fixture.note(3,4,'e',event_id='wrong'),),'tone') - with patch.object(b.harmony,'evaluate',side_effect=AssertionError('unnecessary full check')): - self.assertIsNone(b.advance(state,None,wrong)) - good=dataclasses.replace(wrong,notes=(target,)) - self.assertIsNotNone(b.advance(state,None,good)) - - def test_functional_degree_does_not_depend_on_enharmonic_label(self): - h=harmonic.Harmony(fixture.choices(('E7','Am7'))) - n=fixture.note(0,2,'g',event_id='sharp9') - c=types.ExpressionContract('color','color','dominant_sharp_ninth',(n,),n,'E7') - self.assertTrue(h.evaluate((n,),expressions=(c,)).valid) - wrong=dataclasses.replace(n,pitch=dataclasses.replace(n.pitch,alter=F(1))) - self.assertFalse(h.evaluate((wrong,),expressions=(dataclasses.replace(c,events=(wrong,),target=wrong),)).valid) - - def test_weight_contributions_and_target_free_features(self): - b=builder(count=1) - a=next(a for a in b.operators.at(F()) if a.kind=='tone') - v=b.selector.evaluate(a) - self.assertEqual(v.total,sum(v.contributions.values())) - self.assertEqual(v.total,b.selector.cost(a)) - key='duration/'+str(a.end-a.start).replace('/','_');b.selector.weights={key:3} - changed=b.selector.evaluate(a) - self.assertEqual(changed.total,b.selector.cost(a)) - self.assertEqual(changed.total-v.total,3*v.raw[key]) - with self.assertRaises(ValueError):selection.validate_weights({'song/all_of_me':1}) - - def test_derivation_cannot_inject_an_illegal_reference_note(self): - from audition.fit_selection import derive - b=builder(('CM7',),count=1) - target={'alignment':{'encoded_offset':'0'},'sounding_events':[{'start':'0','end':'4','sounding_midi':'99','pitch':{},'measure':1}]} - path,result=derive(b,target) - self.assertFalse(result['complete']);self.assertFalse(path) - -class CompiledResources(unittest.TestCase): - def test_compiled_and_xml_libraries_have_identical_contents(self): - original=(scale_fixtures.library(),fixture.library(),rhythm_fixtures.library()) - import json - encoded=json.loads(json.dumps(resources.encode(original),default=str)) - restored=resources.decode(encoded) - self.assertEqual(original[0],restored[0]) - self.assertEqual(original[1].paths,restored[1].paths) - self.assertEqual(original[2].snippets,restored[2].snippets) - - def test_warm_cache_avoids_xml_import_and_corruption_recompiles(self): - original=(scale_fixtures.library(),fixture.library(),rhythm_fixtures.library()) - with tempfile.TemporaryDirectory() as root: - with patch('generator.resources.scales.load_library',return_value=original[0]) as sc,patch('generator.resources.sequences.load_library',return_value=original[1]),patch('generator.resources.rhythm.load_library',return_value=original[2]): - resources._load.cache_clear();resources.load(Path(root),{'fixture':'1'},root) - self.assertEqual(sc.call_count,1) - resources._load.cache_clear();resources.load(Path(root),{'fixture':'1'},root) - self.assertEqual(sc.call_count,1) - next((Path(root)/'_resources').glob('*.json')).write_text('{}') - resources._load.cache_clear();resources.load(Path(root),{'fixture':'1'},root) - self.assertEqual(sc.call_count,2) diff --git a/tests/test_music_model.py b/tests/test_music_model.py index c3794be..d70b4ab 100644 --- a/tests/test_music_model.py +++ b/tests/test_music_model.py @@ -124,7 +124,9 @@ class MusicModelTests(unittest.TestCase): self.assertEqual((output / 'last-good').read_text(), 'preserved') (data / 'songs' / 'bad.xml').write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()), encoding='utf-8') args.check = True - with self.assertRaisesRegex(ValueError, 'summary'): + # --check uses the validation-only repository fixture. An alternate + # source tree fails its form contract before atomic publication. + with self.assertRaisesRegex(ValueError, 'performance generator drift'): generator.__main__.generate_assets(args) self.assertEqual((output / 'last-good').read_text(), 'preserved') def test_instrument_transposition_is_converted_once(self): diff --git a/tests/test_phrase_composer.py b/tests/test_phrase_composer.py index fd70da0..781e9fd 100644 --- a/tests/test_phrase_composer.py +++ b/tests/test_phrase_composer.py @@ -1,96 +1,165 @@ -"""General, independently authored capabilities of the active action search.""" -import dataclasses, unittest -from fractions import Fraction as F +"""Independent musical and selection contracts for complete phrase generation.""" +import dataclasses +import fractions +import unittest +from unittest.mock import patch from generator import model, vocabulary from generator.scales import selection, timeline -from generator.solo import analysis, construction, harmonic, profiles, types, placement, operators +from generator.solo import analysis, construction, harmonic, profiles, types, placement +from generator.solo import selection as evaluation from tests import solo_fixtures as fixtures, rhythm_fixtures, scale_fixtures def builder(symbols=('C7','FM7','Dm7','G7'),seed='independent',count=8): source=fixtures.song(count=count,chords=symbols);a=analysis.analyze(source);ms=source['staves'][0]['measures'] end=sum(model.fraction(m['duration']) for m in ms) - cs=selection.select(timeline.read(source['staves'][0]),timeline.Solo(F(),end,'test'),scale_fixtures.library()) + cs=selection.select(timeline.read(source['staves'][0]),timeline.Solo(0,end,'test'),scale_fixtures.library()) return construction.Builder(a,ms,harmonic.Harmony(cs,profiles.Instrument(reference=60)),fixtures.library(),rhythm_fixtures.library(),end,seed) + class PhraseConstruction(unittest.TestCase): - def test_candidates_include_open_motion_arrivals_and_joint_expression(self): - b=builder(count=2);values=b.operators.at(F()) - self.assertTrue(any(a.kind=='tone' and a.end==F(1,2) for a in values)) - self.assertTrue(any(len(a.notes)==1 and a.end>=2 for a in values)) - self.assertTrue(any(a.contracts for a in values)) - for a in values: - checked=b.harmony.evaluate(a.notes,until=a.end,complete=False,expressions=a.contracts) - self.assertTrue(checked.valid,checked.error) - evaluation=b.selector.evaluate(a) - self.assertEqual(evaluation.total,sum(evaluation.contributions.values())) - self.assertTrue(all(a.start<=n.start' + actual=self.review(root,'actual.xml',xml(pitched(24),leadin=prelude),form_start=2,pitch_convention='guitar-written') + with patch.object(score,'load_json',return_value=document),patch.object(solo_compare,'render_events',return_value=array.array('f',[0]*10)) as render,patch.object(playback,'write_pcm'): + solo_compare.audition_pair(actual,actual,'fixture/acoustic',root,Path(root),{}) + # Backing's preceding bar starts at the same audition clock as the lead. + self.assertEqual(render.call_args_list[0].args[0][0][0],0) + self.assertTrue(any(start>=4 for start,end,pitch in render.call_args_list[0].args[0])) + def test_triplet_time_and_annotations_are_literal(self): with tempfile.TemporaryDirectory() as root: triplet=pitched(2).replace('quarter','eighth32') diff --git a/tests/test_structured_selection.py b/tests/test_structured_selection.py deleted file mode 100644 index 821dc2e..0000000 --- a/tests/test_structured_selection.py +++ /dev/null @@ -1,208 +0,0 @@ -"""Independent counterfactuals: no authored target needed.""" -import dataclasses -from fractions import Fraction as F -from unittest.mock import patch -import unittest -from generator.solo import construction, operators, selection -from tests.solo_fixtures import note -from tests.test_phrase_composer import builder - - -class CompleteSelection(unittest.TestCase): - def test_fast_signature_comparison_is_exact(self): - home=tuple(note(F(i,3),F(i+1,3),('c','e','g')[i%3]) for i in range(12)) - variants=[(),home,home[:1],tuple(dataclasses.replace(n,start=n.start+5,end=n.end+5) for n in home)] - for i in range(len(home)): - for field in ('start','end'): - values=list(home);values[i]=dataclasses.replace(values[i],**{field:getattr(values[i],field)+F(1,12)}) - variants.append(tuple(values)) - values=list(home);values[i]=dataclasses.replace(values[i],pitch=dataclasses.replace(values[i].pitch,octave=5)) - variants.append(tuple(values)) - expected=selection.signature(home) - for notes in variants:self.assertEqual(selection.matches_signature(notes,expected),selection.signature(notes)==expected) - - def test_locally_cheaper_opening_can_lose_complete_phrase(self): - b=builder(('CM7',),count=1) - # C is locally cheaper. Its only continuation leaps to high C, whereas - # the more costly E opening reaches nearby G. All notes are chord tones. - low=operators.Action(F(),F(2),(note(0,2,'c',octave=4),),'tone') - high=operators.Action(F(),F(2),(note(0,2,'e',octave=4),),'tone') - bad=operators.Action(F(2),F(4),(note(2,4,'c',octave=5),),'tone') - good=operators.Action(F(2),F(4),(note(2,4,'g',octave=4),),'tone') - b.selector.weights={'harmony/major/4/2':1} - self.assertLess(b.selector.cost(low),b.selector.cost(high)) - def options(at,state): - xs=(low,high) if at==0 else (bad,) if state.notes[-1].pitch.number()==60 else (good,) - return tuple((b.selector.evaluate(a),a) for a in xs) - with patch.object(b,'candidates',side_effect=options):b.compose() - self.assertEqual(b.selected_actions,(high,good)) - self.assertEqual(len(b.completed),2) - - def test_constructor_labels_and_segmentation_do_not_change_score(self): - b=builder(('CM7',),count=1) - ns=(note(0,1,'c'),note(1,2,'e'),note(3,4,'g')) - whole=operators.Action(F(),F(4),ns,'arpeggio') - split=(operators.Action(F(),F(2),ns[:2],'scalar'),operators.Action(F(2),F(5,2),(),'rest'), - operators.Action(F(5,2),F(3),(),'rest'),operators.Action(F(3),F(4),ns[2:],'answer')) - self.assertEqual(b.selector.complete((whole,)),b.selector.complete(split)) - self.assertEqual(b.selector.evaluate(whole),b.selector.evaluate(dataclasses.replace(whole,kind='enclosure'))) - held=note(0,4,'e') - tied=(operators.Action(F(),F(2),(dataclasses.replace(held,end=F(2)),),'tone'), - operators.Action(F(2),F(4),(dataclasses.replace(held,start=F(2),continuation=True),),'tone')) - self.assertEqual(b.selector.complete((operators.Action(F(),F(4),(held,),'tone'),)),b.selector.complete(tied)) - - def test_home_belongs_to_complete_opening_not_recent_buffer(self): - b=builder(('CM7',),count=4);state=construction.State() - for i in range(16): - a=operators.Action(F(i),F(i+1),(note(i,i+1,('c','e','g','b')[i%4]),),'tone') - state=b.advance(state,None,a) - if i<15:self.assertFalse(state.home) - self.assertEqual(len(state.home),16);self.assertEqual(len(state.recent),8) - self.assertEqual(state.home[0].start,0) - - def test_true_arpeggio_uses_written_chord_members(self): - b=builder(('C7',),count=1) - values=b.operators.at(F()) - arps=[a for a in values if a.kind=='arpeggio'] - thirds=[a for a in values if a.kind=='scale_thirds'] - self.assertTrue(arps);self.assertTrue(thirds) - for a in arps: - self.assertTrue(all(n.pitch.number()%12 in b.harmony.at(n.start).roles.basic for n in a.notes)) - self.assertTrue(any(any(n.pitch.number()%12 not in b.harmony.at(n.start).roles.basic for n in a.notes) for a in thirds)) - - def test_optional_color_lookahead_rejects_missing_and_no_chord_harmony(self): - from generator.solo import expressions - from generator import vocabulary - b=builder(('CM7',),count=1) - absent=dataclasses.replace(b.harmony.at(F()),scale=None,chord='N.C.') - for rule in vocabulary.data('harmony')['colors']: - self.assertFalse(expressions.eligible_color(rule,None,b.harmony)) - self.assertFalse(expressions.eligible_color(rule,absent,b.harmony)) - - def test_training_features_cannot_encode_role_song_lookup(self): - for key in ('kind/arpeggio','song/all_of_me','target/pitch_sequence','source/hash'): - with self.assertRaises(ValueError):selection.validate_weights({key:1}) - with self.assertRaises(ValueError):selection.validate_weights({'duration/1':1025}) - from audition.structured_learning import constraint - row=constraint('acoustic_solo',({'duration/1':12},F(3)),({'duration/1':12},F(2)),'test') - self.assertFalse(row['delta']);self.assertEqual(row['prior'],1) - - def test_contextual_rules_are_bounded_transposable_and_operational(self): - from audition.structured_learning import contextual_candidates - b=builder(('C7',),count=1);d=builder(('D7',),count=1) - ns=(note(0,1,'d'),note(1,2,'e')) - moved=(note(0,1,'e'),note(1,2,'f',alter=1)) - key='joint/degree=4&motion=2&quality=dominant' - selection.validate_weights({key:-3}) - left=contextual_candidates(b.selector,ns);right=contextual_candidates(d.selector,moved) - self.assertEqual(left[key],12);self.assertEqual(left[key],right[key]) - self.assertNotIn(key,contextual_candidates(b.selector,(ns[0],note(1,2,'f')))) - a=operators.Action(F(),F(2),ns,'tone');plain=b.selector.evaluate(a) - b.selector.weights={key:-3};changed=b.selector.evaluate(a) - self.assertEqual(changed.total-plain.total,-18) - for forbidden in ('joint/degree=4&metric=beat&motion=2&quality=dominant', - 'joint/degree=4&song=all_of_me','joint/absolute_bar=4&quality=dominant'): - with self.assertRaises(ValueError):selection.validate_weights({forbidden:1}) - - def test_open_route_survives_equal_time_frontier_without_extra_attack(self): - b=builder(('CM7',),count=1) - n=note(0,4,'e',event_id='held') - a=operators.Action(F(),F(4),(n,),'tone') - with patch.object(b.operators,'at',return_value=(a,)):b.compose() - self.assertEqual(b.completed[0].notes,(n,)) - self.assertEqual(b.selected_actions,(a,)) - self.assertEqual(b.completed[0].evaluation,b.selector.complete((a,))) - - def test_deferred_validation_preserves_eager_successor_selection(self): - b=builder(('C7',),count=1) - state=construction.State(recent=(note(-1,0,'g'),)) - lazy=b.candidates(F(),state) - with patch.object(b.operators,'proposals',side_effect=lambda at:b.operators.at(at)): - eager=b.candidates(F(),state) - self.assertEqual(tuple(a.key for _,a in lazy),tuple(a.key for _,a in eager)) - - def test_incremental_evaluation_equals_complete_event_evaluation(self): - b=builder(('CM7','Dm7','G7','CM7'),count=8) - actions=[];notes=();previous=None;home=();old_home=();cursor=F() - for i in range(18): - gap=F(1,2) if i%4==2 else F() - length=(F(1,2),F(1),F(3,2))[i%3] - ns=() if i%7==6 else (note(cursor+gap,cursor+gap+length,('c','d','e','g')[i%4]),) - a=operators.Action(cursor,cursor+gap+length,ns,'arbitrary_label') - if i==5:home=notes - contracts=() - fast=b.selector.extend(previous,notes,a,home,old_home=old_home,contracts=contracts) - actions.append(a) - full=b.selector.complete(actions,home) - self.assertEqual(fast,full,(i,fast.raw,full.raw)) - previous=fast;notes=(*notes,*ns);old_home=home;cursor=a.end - - def test_cached_retrieval_retains_incoming_rhythm_and_pitch_context(self): - b=builder(('C7',),count=1) - incoming=(note(-3,-2,'c'),note(-2,0,'e')) - b.selector.weights={'space/after_arrival':-2,'motion/3/1':2} - values=b.operators.at(F()) - selected=[a for a in values if a.kind in ('rest','scalar','arpeggio')][:80] - for a in selected: - self.assertEqual(b.selector.fast_cost(a,incoming),b.selector.cost(a,incoming)*(a.end-a.start),a.key) - - def test_intrinsic_retrieval_matches_reports_for_every_action_family(self): - from collections import Counter - b=builder(('C7','FM7'),count=2) - incoming=(note(0,1,'c'),note(1,F(7,2),'e')) - b.selector.weights={'duration/1':4,'harmony/dominant/4/1_2':-3,'obligation/step/-1':-2, - 'space/after_arrival':-1,'metric/eighth/1_2':-1,'source/rhythm_match':2, - 'register/1':1,'continuation/up/up':-1,'motion/2/1_2':3} - checked=Counter() - for a in b.operators.proposals(F(7,2)): - if checked[a.kind]>=32:continue - checked[a.kind]+=1 - self.assertEqual(b.selector.fast_cost(a,incoming),b.selector.cost(a,incoming)*(a.end-a.start),a.key) - self.assertGreaterEqual(len(checked),9) - b.selector.weights={'duration/1':-3} - a=operators.Action(F(7,2),F(9,2),(note(F(7,2),F(9,2),'e'),),'tone') - self.assertEqual(b.selector.fast_cost(a,incoming),b.selector.cost(a,incoming)*(a.end-a.start)) - - def test_cached_retrieval_treats_tied_fragments_as_one_sounding_event(self): - b=builder(('CM7',),count=1) - incoming=(note(-2,-1,'c'),note(-1,0,'d')) - whole=operators.Action(F(),F(3),(note(0,2,'e'),note(2,3,'g')),'tone') - tied=dataclasses.replace(whole,notes=(note(0,1,'e'),note(1,2,'e',continuation=True),whole.notes[-1])) - expected=b.selector.cost(whole,incoming)*3 - self.assertEqual(b.selector.fast_cost(whole,incoming),expected) - self.assertEqual(b.selector.fast_cost(tied,incoming),expected) - - def test_successor_cache_preserves_context_and_invalidates_changed_weights(self): - b=builder(('C7',),count=1) - state=construction.State(recent=(note(-3,-2,'c'),note(-2,-1,'e'),note(-1,0,'g'))) - with patch.object(b.selector,'fast_cost',wraps=b.selector.fast_cost) as score: - first=b.candidates(F(),state);calls=score.call_count - self.assertGreater(calls,0) - self.assertEqual(first,b.candidates(F(),state));self.assertEqual(score.call_count,calls) - b.selector.weights={'duration/1':9} - changed=b.candidates(F(),state);self.assertGreater(score.call_count,calls) - b.successor_cache.clear() - self.assertEqual(changed,b.candidates(F(),state)) - before=score.call_count - # Same pitches and final release; a longer incoming hold changes how - # leading space is classified, so it is a distinct retrieval context. - held=dataclasses.replace(state,recent=(note(-4,-3,'c'),note(-3,-2,'e'),note(-2,0,'g'))) - b.candidates(F(),held);self.assertGreater(score.call_count,before) - - def test_successor_cache_keeps_pending_destinations_distinct(self): - from generator.solo import types - b=builder(('C7',),count=1) - incoming=(note(-1,0,'d',alter=1),) - state=construction.State(recent=incoming) - target=next(a.notes[0] for a in b.operators.at(F()) if a.kind=='tone' and a.end==1 and a.notes[0].pitch.number()==64) - contract=types.ExpressionContract('pending','delayed_approach','bounded_semitone_resolution',incoming,target,'C7') - with patch.object(b.selector,'fast_cost',wraps=b.selector.fast_cost) as score: - b.candidates(F(),state);calls=score.call_count - pending=dataclasses.replace(state,pending=(contract,)) - values=b.candidates(F(),pending) - self.assertTrue(values);self.assertGreater(score.call_count,calls) - self.assertTrue(all(b.pending_compatible(pending,a) for _,a in values)) - b.successor_cache.clear() - self.assertEqual(values,b.candidates(F(),pending)) - -if __name__=='__main__':unittest.main()