khodo/product-vamp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Cleanup

Browse Source
This commit is contained in:
khodowany
2026-10-04 18:48:02 -05:00
parent 68df7292ee
commit 0e68f5364b
58 changed files with 1416 additions and 7567 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/composer/implementation_report.md
+56 -66
View File
@@ -1,94 +1,84 @@
# Reference-focused composer repair
# Complete-phrase solo rewrite
Starting revision: `f234dd6`. The goal of this revision is to make the active composer construct the musical relationships seen in the four supplied solos, using general rules and original song inputs. It does not import their notes or optimize candidate similarity to them. Exact reconstruction from general theory alone is not claimed.
Starting revision: `68df7292ee85918a0cdbd0965139ccb2ceecb130`. This revision replaces the active solo constructor. Source import, harmony, exact timing, typed expression validation, independent roles, guitar placement, notation, playback and atomic asset publication remain in use.
## Actual changes
The selected direction is a new melody over the source changes, with optional head references. The source head no longer supplies a compulsory rhythmic prefix or protected initial hold. The four supplied solos remain outside production and unchanged; exact reproduction or listener preference is not claimed.
The active `themes → gestures → routes` path now chooses harmonic arrival timing, target pitch, sounding duration and release before pitch realization. It can construct an approach into an anticipated chord-tone target with both obligations present. Color destinations are available before route filling. Several compatible treatments can survive within one phrase; the local decoration stage preserves existing contracts instead of replacing them.
## Construction and simplification
On-time/delayed arrival pitches and their uninterrupted tails remain fixed through route filling and refinement. Other cell endings stay provisional. Internal long values can be reopened into eighth-note motion where the selected thematic contract permits it; exact rhythm identities and committed closing holds remain protected. Four-member chord traversals are available alongside the existing mapped paths.
The normal path is `production.generate_role → composer.compose → construction.Builder → rhythm.Rhythms/routes.Routes → scoring.evaluate → placement → validation → notation/export`.
Two general enclosure formulas use an actually mapped whole-tone upper neighbor. A natural sixth on tonic/predominant minor harmony can be a bounded stable color rather than always requiring immediate neighbor resolution. These rules retain written harmony, omissions, scale restrictions on ordinary notes and physical validation. No new scale families or performance effects were added.
Each proposal chooses a phrase purpose, rhythm groups, direction, harmonic destination, arrival timing, sounding duration and optional treatment together. Complete pitch paths realize those choices. Open continuations need no chord-tone stop; settling, completion and handoff have actual destination contracts. Anticipations, approaches, enclosures, connections, colors, blues treatments and prepared suspensions enter before route selection and must validate as complete relationships.
Reference inspection now includes every one of the 514 sounding events across the four complete scores. Original labels in technical-fingering and other-notation containers are retained. The review checks recognized treatments against **original source harmony**, preserves missing initial harmony, and keeps hypotheses separate from validated rules. Remaining unexplained events are reported instead of automatically authorized.
An introduced realized phrase supplies the home idea. Development, answer, deliberate return and optional head reference are distinct reachable actions. A return protects a short heard rhythm and directional signature with harmonic adaptation; ordinary phrases have no compulsory three-note prefix. This signature is not a perceptual guarantee of recognition.
## One comparison location
One versioned evaluator reports purpose, incoming continuity, contextual repetition and broad register costs. Physical travel selects fingering, not the winning tune. One-phrase lookahead averages normalized completed-phrase scores across variable durations; it remains a bounded heuristic, not a whole-chorus optimum. Route construction has its own explicitly separate heuristic.
`build/solo_reference_comparison/` contains the compact previous four XMLs/manifest, current output, full reference comparison/audio, complete theory review and verification logs. Start with `index.html`; `analysis/theory.md` supplies the all-event analysis. Solo 1 means acoustic; Solo 2 means electric.
The solo package decreases from **6,232 to 1,729 Python lines**, and from 30 to 19 modules. Thirteen obsolete constructors/helpers were removed and two focused modules added. Old plan/draft/paraphrase API tests were retired; retained musical and representation contracts were migrated. Removed tests are not counted as passing tests. No legacy production fallback remains.
The references are `build/all-of-me-solo-1.xml`, `all-of-me-solo-2.xml`, `autumn-leaves-solo-1.xml`, and `autumn-leaves-solo-2.xml`. Source songs remain authoritative in `data/songs/`. All six input hashes were recorded before work. The optional adapter applies the separately confirmed guitar-written minus-one-octave convention; production import is unchanged. Autumn Leaves aligns at measure 2 / first Gm7 and retains the preceding measure. Four Autumn electric reference events conflict with the production vocal cutoff; neither input is rewritten.
## Inputs and data
Cleanup removed 1.82 GiB of obsolete generated comparison trees/logs and five superseded tracked audition files. Exact removed paths and sizes are recorded in `cleanup.json`. Build toolchains, reusable composition/engraving caches, source songs and the four references were preserved. The old multi-experiment documentation was replaced with current commands and contracts.
The 21 scales, 2,954 sequence paths and 3,538 rhythm snippets remain intact. `solo.json` contains independently authored parameterized actions, nine atomic rhythm groups and migrated authored cells. `grammar.json` and `melodic_vocabulary.json` were retired after their consumers migrated. Accompaniment fields and resources were not changed.
## Evidence and boundaries
References: `build/all-of-me-solo-1.xml`, `all-of-me-solo-2.xml`, `autumn-leaves-solo-1.xml`, `autumn-leaves-solo-2.xml`. Solo 1 maps to acoustic; Solo 2 to electric. The optional reader applies the confirmed guitar-written octave convention and preserves raw evidence. Autumn Leaves aligns at the first Gm7 in reference measure 2; its preceding measure remains in the comparison. Four Autumn electric reference events extend beyond the source-defined vocal allocation. Neither the target nor the vocal timing is changed to force equality.
- [Active path and verified defects](active_path_audit.md)
- [Resource provenance, consumers and cleanup](data_migration.md)
- [Authoritative evaluation](evaluation_contract.md)
- [Musical capability and limits](musical_capability_matrix.md)
- [Independent input contract](dependency_proof.md)
All six strings remain available on frets 1–12 without open strings. Ties retain one sounding event and one position. No performance-expression changes were introduced. The existing independently generated accompaniment, electric melody and vocals are intentionally preserved by this solo rewrite.
## Verification and review
The final Python command was the repository Python runner followed by `-m unittest discover -s tests -p 'test_*.py'`: **435 tests passed in 218.783 seconds**. During implementation, repeat-build tests caught profiling timings leaking into stable diagnostics. Timings now live only in the separate build profile; the final suite includes those repeat-build checks. Earlier interrupted/stale-import runs are not counted as passing verification.
The explicit dependency command passed all six roles on independently authored input: alone/cold, reverse/warm, parallel/cold, invalid optional targets present, targets removed, and after an electric-only setting change. Actual reads and canonical document hashes are retained. This synthetic test is not a whole-song quality assessment.
The optional transfer command passed all 18 role builds on evaluation-only four-bar excerpts from After You've Gone, Hindustan and Indiana. Their provenance remains under `tests/evaluation_sources/`. This demonstrates limited transfer; it does not establish arbitrary meter, form or chord support.
The comparison page includes full generated, previous and reference solo audio, common-backing mixes and opening excerpts. The same neutral renderer and constant comparison gains are used. Comparison reads happen only after generation. No ear-based audition or listener approval is claimed.
## Evidence locations
- [Active path and findings](active_path_audit.md)
- [Resource/code migration and provenance](data_migration.md)
- [Evaluation contract and search limitation](evaluation_contract.md)
- [Capability matrix](musical_capability_matrix.md)
- [Independent-input verification](dependency_proof.md)
- [Reproduction commands](commands.md)
The six-role independent-input exercise passed with real catalogs: standalone, reversed, cold/warm, two-process, optional targets present/deleted, and after an electric-only setting change. Current hashes and actual read traces are in `dependency/dependency.json`. New proposal and route cache tests compare actual uncached/cached results and caller provenance. This does not establish perceptual quality.
`build/solo_reference_comparison/` is the sole comparison location. `previous/` retains the immediate pre-rewrite output and input hashes; `generated/` contains the replacement. The complete operation/pruning trace and local/next-phrase evaluations are in the manifest. `analysis/selected_phrases.md` provides readable selected events, destinations, holds, expression evidence and alternative scores.
The final score weights were preserved. Construction coverage is reported separately from selection under `selection_trace.joint_gestures`. A supported relationship may never be selected; matching a reference attack is a different fact again. Complete per-phrase evaluation totals remain inspectable.
## Remaining boundaries
The first diagnostic profile exposed repeated approach/rhythm/fretboard work. Bounded caches now reuse exact immutable constraints and restore each caller's provenance. No search beam was widened and no machine-speed timeout selects music. Full build costs and final test outcomes are recorded below; concurrent checks are not treated as an isolated benchmark.
The reference solos are outside-review examples, not a numerical loss. The replacement output remains new music, not those four scores. Counts and matching attacks do not establish greater musical quality.
This remains a bounded rule-based composer. Some expressive reference relationships are unsupported or not recognized by the conservative optional parser; this is not proof those notes are musically wrong. Guitar placement is a legal-path check, not full ergonomic certification. No ear-based audition or listener approval is claimed.
The search retains a small frontier and limited lookahead. It can choose another complete phrase, but does not perform arbitrary two-sided revision of committed music. Some exact thematic returns may still sound too weak; the selected rhythm and contour contract is less strict than exact intervals. Acceptable candidates can share a zero musical penalty, leaving the stable role key to break ties.
## Completed checks
Unsupported written harmony produces an explicit diagnostic. Optional expression rules do not repair ambiguous source chords or authorize arbitrary chromatic notes. Legal TAB and coarse tempo limits are not full biomechanical certification. The next useful listening check is the four complete comparisons: whether their openings establish memorable ideas and whether later returns remain recognizable through the freer continuations.
- Full Python suite: **576 tests passed in 1700.428s**. Command: repository Python runner plus `-m unittest discover -s tests -p 'test_*.py'`; full log in `build/solo_reference_comparison/tests.log`. This overlapped the full production build.
- Focused joint-arrival/evaluator/comparison tests: **33 passed**; eight comparison-adapter tests also passed after the presentation update.
- Six-role real-catalog dependency exercise: **passed**, including two spawned processes and reference presence/absence; evidence in `dependency/dependency.json`.
- `git diff --check`: passed at the pre-publication checkpoint.
## Published assets and measured cost
The comparison table aggregates timing-only retained-unit labels (such as diminution) into their direct construction family. The manifest preserves the original raw diagnostic labels. This reporting normalization does not select or edit music.
Full assets are published at `build/build_x86_debug/generated_assets/`; the four exported solos are in `solo_xml/`. All twelve musical role documents match between cold staged generation and cached publication. The eight non-solo documents are exactly equal to the preserved baseline. Every exported solo sounding event and TAB position matches the selected stream. All committed holds and typed harmonic contracts passed validation. All six source/reference file hashes are unchanged.
## Published results and comparison
| Solo | New attacks | Sounding MIDI range | Occupancy |
|---|---:|---|---:|
| all_of_me/acoustic | 117 | 47–67 | 0.781 |
| all_of_me/electric | 139 | 52–72 | 0.824 |
| autumn_leaves/acoustic | 153 | 47–67 | 0.859 |
| autumn_leaves/electric | 109 | 52–70 | 0.732 |
Normal production assets are published at `build/build_x86_debug/generated_assets/`. The four MusicXML exports in its `solo_xml/` directory are byte-identical to the copies in `build/solo_reference_comparison/generated/`.
These descriptive measurements are not targets or proof of musical improvement. None of the four generated solos exactly matches its reference. All four normalized references have legal pitch/position paths within the existing guitar constraints; the Autumn electric vocal-allocation conflict remains explicit.
| Solo | New attacks | Sounding MIDI range | Exact matching reference attacks | Differing onset positions |
|---|---:|---|---:|---:|
| All of Me acoustic | 104 | 50–69 | 1 | 186 |
| All of Me electric | 100 | 51–67 | 3 | 187 |
| Autumn Leaves acoustic | 106 | 43–62 | 3 | 182 |
| Autumn Leaves electric | 97 | 55–70 | 2 | 170 |
The native integration test `ctest --test-dir build/build_x86_debug -R '^product_vamp$' --output-on-failure` passed in **30.45 seconds**, with required localhost access. The optional comparison adapter suite passed **8 tests in 0.018 seconds** after the audio-directory update. All **36 comparison WAVs** passed PCM clipping checks, and all **50 page links** resolved. The signal review is not an ear-based audition.
**None is an exact reference match.** These counts are descriptive, not evidence that the new music is closer perceptually. The earlier outputs had 0/3/0/1 exact matching attacks and 202/191/179/182 differing onset positions respectively. Different attack counts change these comparisons; they must not become a production optimization target.
The isolated final staged asset build took **107.15 seconds** wall time, including **91.31 seconds** in the performance-part stage. All twelve role caches missed. Two composition workers ran in separate processes; no tests or audition rendering overlapped. Engraving was warm and reused all six scores. The **27.57-second** normal CMake publication reused all twelve role caches and six engraving results.
The new winners contain 9/12/13/13 validated expressive contracts respectively. All of Me electric selects one anticipation; the other three winners select none. Early-arrival candidates are constructed, validate and survive pruning in the traces, so their infrequent final selection is now visible separately from lack of capability. No quota was imposed to conceal that result.
| Cold solo role | Role seconds | Construction seconds | Route seconds |
|---|---:|---:|---:|
| all-of-me/electric_solo | 40.82 | 26.27 | 21.47 |
| all-of-me/acoustic_solo | 44.38 | 29.96 | 24.90 |
| autumn_leaves/electric_solo | 35.56 | 33.09 | 28.31 |
| autumn_leaves/acoustic_solo | 44.04 | 41.71 | 35.80 |
All of Me acoustic contains a complete ascending A3–A#3–B3 connection at beats 12–13, and a mapped A3–F#3 approach to G3 at 64–65. All of Me electric sustains B3 as a valid minor ninth over its Am7 context at beats 28–31. These are selected events, not inserted examples or borrowed reference notes. `analysis/selected_phrases.md` records actual themes, full opening rhythms, every selected treatment, hold completion and alternative score breakdowns. Alternatives shown over equal spans are not falsely presented as the complete solo's globally ranked runners-up.
Worker times overlap. Role time includes resource setup, notation and validation beyond construction. Route time is included in construction, not an additional cost. Hardware/process evidence: 12 logical CPUs, Linux x86-64, repository Python 3.13; `build-profile.json` contains actual reads, worker settings and stage times. The first worker use still spends time loading the full catalogs. Route search remains the dominant construction cost.
The first All of Me acoustic theme still begins with a 4.5-beat source-derived rhythmic hold. This and the small number of selected anticipations demonstrate remaining distance from the supplied examples' phrase construction. The next narrow investigation is the interaction between retained source-rhythm contracts and complete-phrase selection: inspect valid arrival alternatives that lose, and source features that prevent editing a provisional opening. Do not respond by adding an enclosure quota, changing accompaniment, fitting reference counts or claiming that more vocabulary alone solved the problem.
An isolated one-process All of Me acoustic build took **39.80 seconds** without cProfile. The diagnostic run before the performance-only optimization took 163.09 seconds **with cProfile enabled**, so those times are not a valid speedup ratio. Profiling exposed repeated pitch hashing/stringification and harmony lookups. Memoization and hoisted identical calculations preserved the complete musical document hash `2b6b35011e353d166d287a8bfff6074723e62559b5e64ff978084623e1520e67`, also matching final production. The rewrite changes music and search structure; its timing is not presented as a semantics-preserving speedup over the old composer.
## Final verification
- All **12 role documents** agree between the cold staged build and the warm normal publication, including notes, timing and TAB. Evidence: `verification.json`.
- All eight non-solo role documents (accompaniment, electric source melody and vocal for both songs) match the previous release exactly as parsed JSON.
- All four MusicXML sounding streams match their selected production role documents. Full-duration harmonic validity, ending completion, and evaluation-contribution sums pass.
- The two original song files and four supplied references remain SHA-256 identical to their preserved inputs.
- Native command `ctest --test-dir build/build_x86_debug -R '^product_vamp$' --output-on-failure` passed in **33.54s**, with the required localhost socket access.
- The single comparison page contains 24 WAVs: complete generated/reference solos, identical-backing mixes, and opening excerpts. All passed PCM clipping checks. All 36 local page links exist. No subjective listening verdict is claimed.
## Measured cost
Hardware: Intel Core i7-1355U, 12 logical CPUs; two composition workers. Cold solo generation plus full staged asset build took **2249.06s (37.5 minutes)**. Regression and dependency checks overlapped part of that run, so this is not an isolated speed comparison with the earlier release.
| Cold solo role | Seconds |
|---|---:|
| All of Me acoustic | 1041.56 |
| All of Me electric | 850.48 |
| Autumn Leaves acoustic | 1328.58 |
| Autumn Leaves electric | 1170.46 |
Worker times overlap. Four All of Me non-solo caches were warm; the four Autumn Leaves non-solo roles regenerated in under one second each. Engraving rendered 90 new pages, reused two scores and took 16.65s wall time. Stage/input/read evidence is in `build-profile.json` and `build-inputs.json`.
The isolated warm CMake publication took **31.49s**, including about eight seconds of CMake/configuration overhead; asset generation reported 23s. All twelve role caches and all six engraving results were reused. Musical equality was verified, not assumed from cache-hit flags.
An isolated 30-repeat structural-proposal microbenchmark returned exactly equal results: **0.4422s** without the proposal cache versus **0.02193s** with it (shared rhythm/position caches already warm). This is a repeated-calculation measurement, **not** a claimed whole-composer speedup. Cold search is still expensive; candidate construction/validation and repeated thematic alternatives remain performance work.
After verification, duplicate full asset stages, two interrupted stages, temporary tone samples and the binary diagnostic profile were removed. The comparison retains the four current scores, compact previous scores, references through links, useful audio and verification evidence in one directory. No previous output is a generation input.
Cleanup removed **151.2 MiB** of superseded comparison audio, stages, probes and temporary rendering tones during this rewrite. The immediate baseline, four user references, build toolchains, caches, published assets and useful verification evidence remain. Current audio is consolidated under `audio/` and linked from one `index.html`; `cleanup.json` records removed paths.