Sepearate Generation

TECHNICAL_GUIDE.md

@@ -188,33 +188,75 @@ Only the soloing part receives compact `scale_spans`, with absolute rational sta
 
 Scale labels appear above chord symbols at collection changes and system starts. Detailed parent collections, source and arranged positions, chord-root modes, degrees, chord-tone targets, and selection reasons are written to `generated_assets/analysis/<song>.txt`. These reports are published transactionally with the build output but excluded from both installation and Docker runtime assets. The C++ runtime only displays the generated pages; it does not load the scale library or run harmonic analysis.
 
-### Phrase Composition
+### Independent Musical Generation
 
-The musical arrangement has six reusable parts: acoustic accompaniment/solo, electric accompaniment/melody/solo, and vocal melody. `arrange.accompaniment.install` builds each accompaniment once and copies its events at matching authored form positions, retaining cycle phase in lead-in/ending excerpts. It preserves the actual guitar pickup's event identities, ties and tuplets. Changed effective harmony is revoiced locally; a structural audit rejects unexplained differences among repeated bars. `solo.arrangement.apply` installs only the soloist's notes and validates the unchanged supporting part against the solo scale map. No solo-dependent accompaniment planning, voicing, or dynamics remains in production.
+The arrangement has six independently generated roles: acoustic accompaniment and solo; electric accompaniment, melody and solo; vocal melody. `generator.production.SourceAnalysis` contains a canonical immutable copy of the original song. Its harmony, melody, form, meter, tempo, and source-defined role windows are the only shared musical context. A role never reads another role's candidate pool, chosen notes, attacks, register, or grips. Assembly unions their reserved windows; it does not compose, reselect, or rewrite music. The original vocal pickup reserves time before solo composition.
 
-Solo `profiles.Instrument` carries hard string limits, the sounding register, and a melody-derived preferred center. The electric reference is the prepared concert melody plus twelve semitones; acoustic is twelve semitones below electric. This reproduces the existing electric-melody sounding transformation without using the highest harmonization tone. The soft center has a four-semitone deadband, and excursions remain possible. Concrete search and placement caches include the profile.
+`generator.production.generate_role` is the production entry point for each role. The ordinary build schedules these jobs in a bounded **spawn process pool**, default two workers. `--composition-jobs 1` is the serial reference; values through 16 are accepted. CMake exposes `VAMP_COMPOSITION_JOBS` separately from `VAMP_ENGRAVING_JOBS`. More workers can increase memory use and need not be faster on a small machine.
 
-`gesture_rhythm.revise` compares up to eight alternatives in each of two editable windows, retaining the same pitch sequence and attack count. Feature contracts, complete-duration harmony checks, anchor checks and required tails all remain binding. `Phrase.before_rhythm` preserves the exact pre-revision candidate for comparison. Anchor `effect` distinguishes local settling, cadential arrival and handoff; local settling carries a small next-destination preference into the following phrase. An enclosure may reuse a contiguous mapped upper note through an explicit approach segment; extending that note does not waive its ordinary duration-sensitive harmonic policy.
+Composition caches contain fully keyed role results under `build/build_common/composition_cache`. Keys cover canonical source content, role settings, relevant production data, algorithm version, and implementation files. Role-specific content hashes and named decision domains determine equivalent-choice tie breaks; output names, title, request order, references, previous scores, and wall-clock budgets do not determine musical choices. Concrete route caches include incoming state and endpoints. Rhythm-window indexing and per-worker parsed catalogs remove repeated full-catalog scans. Cache corruption is rejected, and clean generation remains available without a cache. Unchanged songs/roles can reuse their results; rendering and assembly remain validated. Trace records accompany completed output in `analysis/<song>.roles`; elapsed timings are progress output, not part of deterministic published scores.
 
-`generator/solo/` is build-only. The active path is `arrangement.apply` → `composer.compose_pair` → `leads.thematic_options` → `themes.phrase_candidates` → `realization.checked_phrase`. The song's actual melody supplies a selected rhythmic, contour, opening, or interval seed. Each instrument introduces its own realized solo theme; later development refers to that heard identity, not each successive phrase of the head. Close paraphrase remains available through `mode='paraphrase'` for comparisons. The older cell-level planner/refiner is not the production theme constructor.
+The active solo construction is `composer.compose` → `leads.thematic_options` → `themes.phrase_candidates` → `realization.checked_phrase`. Source-feature retrieval now covers the complete head, including characteristic rhythm, rests, interval relationships and turns, rather than only its first eight bars. One realized theme drives later development and selected clear returns. The acoustic and electric policies independently prefer different source-feature treatments and soft, overlapping register trajectories. Pairwise uniqueness is diagnostic, never a selection constraint. All six strings are eligible for either solo; sounding notes require frets 1–12 without open strings. Placement compares complete hand-region paths, time-aware shifts and crossings, retaining ties in the same position. Electric melody/vocal transformations and the separate chord anchor policy remain unchanged.
 
-`phrasing.py` proposes four-, five-, six-, and eight-bar endpoints from actual measure boundaries and harmony, with explicit missing-harmony interruptions and a trimmed final handoff. Two promising endpoint alternatives compete through realization from each incoming history. A soft preference aims for approximately six substantial closes per 32 bars (five to seven is the initial range). Ending alternatives choose a chord-valid pitch, arrival, two-to-four-beat sounding interval, and optional release together before constructing the approach. Intermediate search units do not close the thought. A held target must remain one sounding event for the promised interval; repeated attacks or touching and abandoning it do not fulfill the contract. Cross-bar holds must remain valid across every chord and mapped collection they overlap.
+Complete phrase alternatives combine an identifying feature, continuation, harmonic destination, exact rhythm and sounding tail. Structural notes come from the written chord; mapped scale routes connect them with octave-aware indices. Complete paths retain their destination across cells, barlines and scale changes. The search remains bounded, with diverse rhythm/endpoint representatives before expensive realization and a limited replan on failure. An exhausted search reports its failed window; there is no generic fallback tune. Quarter/eighth movement, delayed entrances, offbeat/dotted values, rests, cross-bar holds and selected triplets or sixteenths come from catalogs plus independently authored general rhythm cells. Faster cells obey tempo/difficulty limits. Sustained endings remain binding, but there are no chorus quotas for attacks, closes, returns, pitch distribution or strings. Occupancy is diagnostic.
 
-Theme appearances distinguish clear returns, developed appearances, and freer continuation. A clear return preserves an identifying melodic relationship with its timing; development can preserve melodic intervals while altering rhythm, or a characteristic rhythm while recomposing pitches. A stored opening signature from the realized introduction supplies clear returns, including its rhythm and rests; development may use its smaller three-note core. Ordinary generated events outside that declared signature remain editable. A uniform quarter-note rhythm alone is not sufficient identity for a rhythmic development. Approximately two clear returns are preferred, independently of phrase closures. Once a feature contract is fulfilled, additional copying is not rewarded.
+Typed expression policies distinguish optional colors, semitone approaches from either side, both enclosure orders, short chromatic connections, delayed resolutions, anticipations and two conservative blues inflections. They are local alternatives to a direct approach around existing destinations, with complete timing and pitch contracts. Optional colors are conditioned on chord quality/function, duration and actual continuation; altered dominant ninths/thirteenths require a resolving dominant. A natural minor sixth needs tonic/predominant context and stepwise resolution. These policies do not rewrite the chord or force the backing to voice a color. Explicit alterations/omissions take precedence. Written extensions remain distinct from optional colors.
 
-`sequences.py` validates relative paths in `data/sequences` and chains them into complete routes using octave-aware scale indices. `themes.py` combines these with exact rhythms from `data/rhythms`, including rhythmic alteration, diminution, displacement already present in catalog windows, and sustained endings. Concrete routes retain their complete entry, pitch path, timing, and destination. Up to eight rhythm alternatives and six destinations are retrieved; sixteen joint rhythm/start/destination jobs preserve coverage before expensive realization, expanding to thirty-two during a bounded replan. Eight intermediate, phrase, and chorus alternatives are retained. Cache keys include the actual musical state used in construction; history-dependent ranking occurs after shared concrete retrieval.
+Ordinary material still belongs to every mapped collection it overlaps. Only a validated expression can authorize an exception. Validation checks actual event IDs, exact pitch/octave, harmonic time, every overlapped chord/scale, preparation where required, resolution deadline, guitar placement, protected identity and terminal hold. An annotation or role label alone authorizes nothing. Ties merge before duration treatment; repeated attacks do not satisfy a hold. Missing harmony yields explicit space, unsupported chords fail visibly, and the 21 collections are not claimed to cover every possible chord. Expression selection is a bounded local pass over a valid theme, not a global search over all possible tensions. It may select no decoration.
 
-A rolling sixteen-bar history measures exact and transposed pitch/rhythm recurrence, repeated oscillations, and cadence formulas. Recent reuse receives more pressure than distant reuse, and deliberate returns receive a discount rather than immunity. Ties collapse to one attack. Register goals offer wider acoustic development and lower electric answers after high passages. These are musical preferences, not range or string quotas. Phrase activity retains a broad preference of four to six attacks per four quarter-note beats, including its held ending; individual closing cells have no attack quota. Occupancy is diagnostic. Explicit active, sustained-arrival, and space spans prevent a silent candidate from satisfying a moving passage.
+`generator.comping.Canonical` constructs each accompaniment independently from source harmony, original melody attacks, form position and static policy. It considers dyads, shells and compact voicings, preserving defining written degrees and explicit bass requirements; complex written extensions can require more tones. Acoustic carries primary bass responsibility. Electric supplies bass at essential source-defined positions. Context selects releases, delayed shells, held figures or space; there is no compulsory four-bar loop. The canonical form is reused exactly when that instrument accompanies, including behind solos. It neither borrows electric melody grips nor follows generated solo gaps. Role windows prevent self-accompaniment. Ensemble overlap is reported downstream and never feeds selection.
 
-`leads.elaborate` may replace a complete internal span of shorter notes, as well as an editable portion of a hold. Feature contracts and protected sounding tails are rechecked against the complete result. Exact source quotations remain immutable. The plain theme is a complete playable line, and elaboration is optional. Harmonic validity, exact timing, scale membership, monophony, required holds, and guitar placement remain hard constraints. A bounded replan may reconsider eligible ending alternatives; exhausted search reports its failed passage without claiming mathematical incompatibility or falling back to generic material.
+No accents, dynamics, velocity shaping, new swing interpretation or random timing have been added. Runtime JSON, notation/TAB semantics, playback synthesis and transactional publication are unchanged. Source melody and vocal compatibility remain checked separately from generated music.
 
-Acoustic/electric selection compares complete alternatives, preserving different related themes and discouraging unintended long copies. Placement uses sounding pitches on strings 2–6 for acoustic solos and 1–4 for electric solos, frets 1–12 without open strings; written guitar pitch is one octave above sound exactly once. Ties retain string and fret. The first declared source tempo, or 120 quarter notes/minute, informs movement costs; runtime tempo remains adjustable. No out-of-collection chromatic substitutions, accents, dynamics, velocity shaping, or swing changes are part of this composer.
+#### Data inventory and consumers
 
-`arrangement.py` analyzes each song once and replaces only the two solo-section windows after normal head/accompaniment placement. Electric backs Acoustic Solo; acoustic backs Electric Solo. Neither guitarist accompanies itself. Supporting parts retain the canonical accompaniment chosen from the original song, including its exact rhythm, pitches and fingerings. The electric canonical part may reuse a corresponding melody grip only when it satisfies the written chord requirements and contains chord tones. Solo installation validates the existing backing against the mapped collections and audits repeated form positions; it does not regenerate backing from foreground activity. The complete vocal part remains unchanged, and its pickup cutoff is reserved before composing the electric ending.
+| Resource | Action and purpose | Active consumer |
+|---|---|---|
+| `data/songs/*.xml` (2 current sources) | Keep original songs; no reference-derived substitute sources | Source analysis, form, each role |
+| `data/scales/` (21 patterns) | Keep major, melodic-minor and harmonic-minor modes | Scale mapper, strict route validation |
+| `data/sequences/` (2,954 relative paths) | Keep combinatorial intervals; they are routes, not complete curated tunes | Endpoint-indexed solo routes |
+| `data/rhythms/` (3,538 snippets) | Keep all meters: 34 in 2/4, 146 in 3/4, 630 in 4/4, 2,728 in 5/4; retrieval is meter-specific, never weighted by file count | Solo rhythm windows; legacy rhythm regression fixtures |
+| `data/composition/roles.json` | Add versioned role preferences, register, bass and moderate difficulty policy | Independent role choices, placement preferences, fast-cell eligibility |
+| `data/composition/grammar.json` | Add transposable feature-preservation contracts and hold/release alternatives | Theme development and ending construction |
+| `data/composition/harmony.json` | Add explicit optional tension/function/duration/resolution rules | Expression candidates and validation |
+| `data/composition/lead_ins.json` | Add generic target-relative formulas and blues resolutions | Complete expression candidates and validation |
+| `data/composition/rhythms.json` | Add purpose-bearing melodic cells and sparse comping figures | Solo joint realization, canonical comping |
+| `data/lead_phrases/original_lead_theme.json` | Migrate unchanged to `audition/controls/`; remove from production data/build dependencies | Optional historical audition controls only |
+| `data/expected/performance_summary.json` | Keep validation-only form/harmony/vocal invariants; never used in selection | Post-generation compatibility check |
+| `build/*solo-[12].xml`, old generated assets, audition snapshots | Keep untouched, outside all musical input traces and caches | Explicit optional review only |
 
-`notation.py` converts placed attack spans into ordinary score events, rests, and tied continuations across notation/bar boundaries. Engraving, playback and C++ consumption retain their existing formats. `report.py` appends motif/source IDs, transformation contracts, activity and landmark plans, target windows, sounding pitches, TAB decisions and search diagnostics. Generation still stages and validates the complete output before publication. Listening and playing remain necessary to accept the musical result.
+The five added JSON resources carry stable IDs, version 1, independent authorship provenance and MIT licensing. They were written from the musical brief and general principles, without extracting target notes, rhythm fingerprints, statistics or fitted weights. Existing catalog provenance is unchanged. No songs or useful combinatorial families were deleted. Legacy cell composition utilities remain for regression/explicit historical comparison; they are not a fallback production path.
 
-`python -m generator.solo.audition --output build/solo_foreground_auditions --capture before` preserves the current implementation, and `--capture after` captures its successor. Existing snapshots are never overwritten. `--render` exports full solos, opening eight bars, and closing four bars as MusicXML and WAV, with the final canonical backing held constant. The published arrangement reuses that same backing instead of recomposing around the foreground. It also exports whole-phrase register transfers of the baseline, the selected theme before rhythmic revision, the new plain theme and unaccompanied plain/elaborated full solos. Source hashes, implementation/policy fingerprints, tempo, and shared mix gain are recorded. Reports distinguish completed sustained endings from ordinary long notes, show retained theme features, recurrence counts, movement/arrival activity, local register distribution, and forced versus optional first-string use. These diagnostics establish behavior; listening is still needed to judge musical quality.
+#### Commands and validation boundary
+
+```sh
+# One role; no other role is composed. A JSON settings file can override its policy.
+./modulus.py python -m generator.compose --song data/songs/autumn_leaves.xml \
+  --role acoustic_solo --data-dir data --cache-dir build/build_common/composition_cache \
+  --output build/acoustic-role.json
+
+# Full production build, bounded processes and persistent role/page caches.
+./modulus.py python -m generator --data-dir data --output-dir build/build_x86_debug/generated_assets \
+  --composition-jobs 2 --composition-cache build/build_common/composition_cache \
+  --jobs 2 --engraving-cache build/build_common/engraving_cache --check --progress
+
+# Auditions read completed output and an optional preserved baseline; never compose.
+./modulus.py python -m audition.production_review \
+  --assets build/independent_generation_comparison/after \
+  --baseline build/independent_generation_comparison/baseline \
+  --output build/independent_generation_comparison/auditions
+
+# Optional reference review: role association is explicit, not inferred from "Vocal".
+./modulus.py python -m audition.reference_review \
+  --reference 'autumn_leaves/acoustic=build/autumn-leaves-solo-1.xml' \
+  --form-start autumn_leaves/acoustic=2 --output build/reference-review.json
+```
+
+The actual four optional paths are `build/all-of-me-solo-1.xml`, `build/all-of-me-solo-2.xml`, `build/autumn-leaves-solo-1.xml`, and `build/autumn-leaves-solo-2.xml`. Solo 1 maps to acoustic; Solo 2 maps to electric for review only. All of Me has 32 notated measures. Autumn Leaves has 33, with its first Gm7 at measure 2; its initial material is retained without invented harmony. The reviewer records encoded pitches, literal annotations, rational offsets, carried harmony, tuplets, ties, sounding events and uncertain annotation relationships. No guitar-octave interpretation or repairs are applied. Encoded pitches outside the legal solo domain remain recorded as such. This optional command is never run by the build or required tests.
+
+Auditions export full solos plus opening/ending excerpts, solo stems, a common fixed backing for comparisons and the final assembled backing. Plain themes and selected pre-expression themes are included. Source controls are explicitly placed in a legal guitar register for audition only. Neutral rendering and gains are shared; score overlap diagnostics are not claims of perceived masking or musical improvement. Required tests use independent synthetic musical fixtures and cover serial/parallel/order equality, role settings, clean/cache equality, reference isolation, exact timing, ordinary scale motion, typed exceptions, preserved holds, notation and placement. Listening and playing are still necessary to judge the result.
+
+Current boundaries: source meter vocabulary is 2/4–5/4; unsupported meters or unrepresentable notation fail explicitly. The bounded composer cannot prove impossibility when it exhausts its budget. Expression vocabulary and local editing are deliberately finite, and the source-derived seven-section arrangement remains the existing product form. Independent parts can still collide or sound similar; diagnostics cannot automatically reselect either part. Playback remains one neutral mono tone profile, not a simulation of acoustic/electric timbre.
 
 ### Engraving Boundary