khodo/product-vamp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rhythm Selection & Cleanup
ci / checks (push) Successful in 1h16m10s
Details

Browse Source
This commit is contained in:
khodowany
2026-09-26 02:50:22 -05:00
parent 4d6348d354
commit 9fd5abd075
91 changed files with 4692 additions and 2484 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitea/workflows/ci.yml
+1 -1
View File
@@ -112,7 +112,7 @@ jobs:
uses: actions/cache/restore@v4
with:
path: build/build_common/engraving-cache/${{ steps.renderer.outputs.fingerprint }}
key: product-vamp-pages-${{ steps.renderer.outputs.fingerprint }}-${{ hashFiles('generator/**/*.py', 'generator/*.py', 'data/practice/*.xml', 'data/songs/*.xml') }}
key: product-vamp-pages-${{ steps.renderer.outputs.fingerprint }}-${{ hashFiles('generator/**/*.py', 'generator/*.py', 'data/practice/*.xml', 'data/songs/*.xml', 'data/rhythms/**/*.xml') }}
restore-keys: product-vamp-pages-${{ steps.renderer.outputs.fingerprint }}-
- name: Build native release
run: python3 modulus.py build --target x86_64_gcc12_release
README.md
+14 -8
View File
@@ -68,7 +68,7 @@ build/packages/product-vamp-arm_64_gcc12_release.tar.gz
build/packages/product-vamp-arm_64_gcc12_release.tar.gz.sha256
```
The tarball contains the Zero Docker archive, Pi install scripts, generated catalog database, pages, audio, fonts, and Pico UF2 firmware. Complete score JSON is stored inside `catalog.db`; individual JSON files remain build intermediates and are not deployed. Builds replace the same target-specific archive without a release label. The manifest and `vamp-zero --build-info` identify the source commit.
The tarball contains the Zero Docker archive, Pi install scripts, generated catalog database, pages, audio, fonts, Pico title artwork, and Pico UF2 firmware. Complete score JSON is stored inside `catalog.db`; individual JSON files remain build intermediates and are not deployed. Builds replace the same target-specific archive without a release label. The manifest and `vamp-zero --build-info` identify the source commit.
Run `sudo zero/install-zero.sh` from an extracted bundle on the Zero. Every installation resets all Product Vamp data and settings, including the control Wi-Fi profile, to the new bundle defaults. No previous database rows or configuration are migrated. Unrelated OS data and networks are untouched. Installing through the control Wi-Fi can disconnect your session; use a local console or an independent connection.
@@ -88,7 +88,7 @@ Artifacts are published to the khodo generic package registry's required `latest
Uncompressed MusicXML 3.1/4.0 scores exported from Soundslice or MuseScore live in `data/`. Generated score JSON is not committed.
Only `data/practice` and `data/songs` are active inputs. `data/rhythms` is reserved for future work and is ignored by generation, catalog tests, and packaging.
`data/practice` and `data/songs` supply the authored music. `data/rhythms/<meter>/<series>/` supplies the accompaniment rhythm library for 2/4, 3/4, 4/4, and 5/4. Rhythm snippets are build inputs, not separate catalog entries or runtime files. Generation loads the library once and shares it across parts.
To inspect generated assets without building the full package, run:
@@ -101,7 +101,13 @@ The pinned build-only dependencies are in `generator/requirements.txt`. Configur
CMake builds engrave with two Python workers and share a disposable page cache under `build/build_common/engraving-cache`. Native and ARM packaging reuse unchanged pages; CI also retains this cache between runs. Music, playback, and catalog validation still run for every generation. Set `VAMP_ENGRAVING_JOBS` (1-16) to adjust concurrency, or set `VAMP_ENGRAVING_CACHE_DIR` to an empty CMake value to disable caching. Direct generator runs can opt in with `--jobs 2 --engraving-cache build/build_common/engraving-cache --progress`.
The build-time pipeline is Soundslice/MuseScore -> `.xml` -> musical model -> part generation -> JSON, engraved PNG pages, generated-tone audio, and a read-only catalog database. Sources in `data/songs/` retain the seven-section performance form. Four-digit numbered sources in `data/practice/` (currently `0001.xml` through `0366.xml`; `0000.xml` is also valid) produce one pass each of **acoustic accompaniment**, **electric melody**, and **vocal melody**. Both categories use the authored musical key, with no key variants or key-selection menu. Duplicate titles never merge exercises.
Identical notation bodies share layout and body rasterization even when titles or exercise identities differ; each score retains its own header, pages, music, and catalog identity. A `cold namespace` means no cache directory exists for the current renderer fingerprint. It does not mean that every score must be rendered independently: repeated bodies can still be reused within that build, including with persistent caching disabled. Progress separates full-score hits, body hits, newly rasterized bodies, and composed pages.
The build-time pipeline is Soundslice/MuseScore -> `.xml` -> musical model and rhythm plans -> part generation -> JSON, engraved PNG pages, generated-tone audio, Pico title artwork, and a read-only catalog database. Sources in `data/songs/` retain the seven-section performance form. Four-digit numbered sources in `data/practice/` (currently `0001.xml` through `0366.xml`; `0000.xml` is also valid) produce one pass each of **acoustic accompaniment**, **electric melody**, and **vocal melody**. Both categories use the authored musical key, with no key variants or key-selection menu. Duplicate titles never merge exercises.
Accompaniment uses deterministic four-bar phrase cycles: a two-bar motif repeats with a one-beat variation in the fourth bar when a suitable candidate exists. Acoustic provides the foundation; electric accompaniment responds to the melody and acoustic attacks. Selection considers harmonic coverage, density, collisions, and available space, and restarts at section or meter changes. Rhythm snippets supply attacks, rests, and written durations; structured harmony supplies chord tones. N.C. and absent harmony remain silent. Authored melody, solo slashes, and the seven-section form retain their roles. Unsupported accompaniment meters and mid-measure meter changes fail explicitly.
There are no musical genre presets, genre defaults, or genre metadata. Obsolete `vamp:style` hints are rejected; visual styles, fonts, colors, and barline notation remain supported. Phrase cycles are written into the generated score and played once in reading order.
Guitar parts share a theory-based voicing and proximity search over standard tuning, frets 0-20, and a maximum four-fret span excluding open strings. They render standard notation plus generated TAB. Practice sources provide a monophonic melody and optional chord symbols; the vocal part adds deterministic scat syllables without changing rhythm or written register. Acoustic, electric, and vocal parts all sound one octave below their notation, for both songs and practice. Explicit MusicXML instrument transposition is honored once; an unplayable passage fails rather than being shifted to fit.
@@ -109,7 +115,7 @@ Write all music in reading order: repeat bars, numbered endings, navigation jump
The main menu's **Playback** toggle plays the selected part once at the menu BPM, suppresses the metronome, and returns to the part menu after natural completion. Manual page turns do not seek or restart audio. Playback is off by default. Python generates shared PCM tones and exact beat timelines; C++/Sonus only loads, schedules, and mixes them. Audio sample sustain loops hold individual notes for their written duration; they never repeat musical passages. No SoundFonts or runtime synthesizer are required. Practice navigation is Exercise -> Part -> Score, with the filename number retained alongside an authored title.
Code style is managed by Modulus. Run `./modulus.py format` to format C++ and Python, or `./modulus.py format --check` to check formatting and prohibited aliases. The project-local Neovim integration routes Conform formatting through the same container tools, including format on save and existing LazyVim toggles. Run configure first when the image or exported tooling needs updating. Use `tooling.editor.clang_format` for C++ overrides and `tooling.format.exclude` for project-relative exclusion globs; do not edit generated `.clang-format` files.
Code style is managed by Modulus, using compact two-space formatting. Python imports modules by their full names and qualifies imported symbols; only `from __future__ import annotations` is exempt, and imports have no aliases. Run `./modulus.py format` to format C++ and Python, or `./modulus.py format --check` to check formatting and prohibited aliases. The project-local Neovim integration routes Conform formatting through the same container tools, including format on save and existing LazyVim toggles. Run configure first when the image or exported tooling needs updating. Use `tooling.editor.clang_format` for C++ overrides and `tooling.format.exclude` for project-relative exclusion globs; do not edit generated `.clang-format` files.
Use American English in project-owned identifiers, filenames, messages, and documentation, such as `catalog`, `color`, and `canceled`. Preserve authored music and external API names.
@@ -117,17 +123,17 @@ See [Score Generation](TECHNICAL_GUIDE.md#score-generation) for supported MusicX
Build-time engraving is implemented in Python in `generator/engraving`, using the bundled Petaluma OTF fonts and SMuFL metadata. MuseScore is an optional authoring tool, not a build dependency or rendering backend. The managed image and CI install the pinned Python requirements automatically; direct host builds use `pip install -r generator/requirements.txt`. Exact musical timing is retained through layout, and dense measures can continue across systems without shrinking the notation. Missing glyphs, unrepresentable page geometry, and failed notation audits stop generation before replacing existing assets.
Petaluma is from Steinberg under the SIL Open Font License. Runtime packages include PNG score pages, `PetalumaScript.otf` for UI text, and `OFL.txt`; the music font and engraver stay build-only.
Petaluma is from Steinberg under the SIL Open Font License. Petaluma supplies music glyphs, and Petaluma Script supplies all visible text on score pages and both device interfaces. The Zero uses packaged `PetalumaScript.otf` for dynamic UI text; the Pico displays prerendered Petaluma Script artwork. Runtime packages retain `OFL.txt`; the music font and engraver stay build-only.
Font generation requires Pillow's Raqm layout support to preserve kerning. On Debian/Ubuntu, install `libfribidi0` alongside the pinned Python requirements. The build image and CI install both. See [Pillow's installation requirements](https://pillow.readthedocs.io/en/stable/installation/basic-installation.html).
Pico artwork generation requires Pillow's Raqm layout support to preserve kerning. On Debian/Ubuntu, install `libfribidi0` alongside the pinned Python requirements. The build image and CI install both. See [Pillow's installation requirements](https://pillow.readthedocs.io/en/stable/installation/basic-installation.html).
Native and Pico builds generate their bitmap font implementation under `generated/vamp_style/font.cpp` in the build directory. The generated implementation is not checked in. The build image and CI install the pinned generator dependency automatically. For a direct build, first run:
Pico firmware embeds a small set of fixed labels, part names, digits, and the page separator, generated as four-bit coverage sprites under `generated/pico/static.p4`. Score generation prerenders titles into content-addressed `pico/titles/*.mask` assets referenced by the catalog and served by the Zero when the selection changes. Pico blends those pixels and composes page counts from digit sprites; it does not load fonts, shape text, or rasterize glyphs. All Pico text shaping and artwork generation run on the build host. The build image and CI install the pinned dependencies automatically. For a direct build, first run:
```sh
python3 -m pip install -r generator/requirements.txt
```
To check a generated atlas, run `python3 generator/generate_pico_font.py --output build/build_x86_debug/generated/vamp_style/font.cpp --check`. Changing the source OTF or generator rebuilds the atlas automatically.
To inspect the fixed artwork independently, run `python3 -m generator.pico_artwork --static-output /tmp/product-vamp-pico/static.p4`. Changing the source OTF or artwork generator rebuilds the firmware sprites and generated title assets. These generated files are not authored music and are not checked in.
## License
TECHNICAL_GUIDE.md
+34 -26
View File
@@ -8,9 +8,9 @@ Product Vamp follows the Modulus product layout used by the khodo repos:
- `include/vamp_pico` and `src/vamp_pico`: Pi Pico 2 W C++17 firmware.
- `include/vamp_protocol` and `src/vamp_protocol`: shared route and page-command contracts used by both devices.
- `cmake/pico.cmake` and `cmake/zero.cmake`: device-specific build definitions selected by the root `CMakeLists.txt`.
- `include/vamp_style.hpp`, `include/vamp_style/font.hpp`, and `src/vamp_style.cpp`: shared color palette and Petaluma font interfaces. Bitmap font data is generated in the build directory.
- `data`: authored MusicXML for performance songs and practice exercises, plus the generator regression summary.
- `generator`: build-time Python part generation, guitar placement, scat lyrics, engraving, audio compilation, and catalog construction.
- `include/vamp_style.hpp` and `src/vamp_style.cpp`: shared visual palette. `include/vamp_pico/artwork.hpp` defines the Pico coverage-image and sprite interfaces; fixed artwork is generated in the build directory.
- `data`: authored MusicXML for performance songs, practice exercises, and accompaniment rhythms, plus the generator regression summary.
- `generator`: build-time Python rhythm planning, part generation, guitar placement, scat lyrics, engraving, audio compilation, Pico artwork, and catalog construction.
- `package`: generic bundle inputs, scripts, config, static font assets, and device install files.
## Zero Runtime
@@ -24,6 +24,7 @@ The API is plain HTTP on the private AP:
- `GET /api/status`
- `GET /api/health`
- `GET /api/remote/display`
- `GET /api/remote/title`
- `POST /api/remote/heartbeat`
- `POST /api/remote/next`
- `POST /api/remote/previous`
@@ -53,16 +54,22 @@ The build runs `python3 -m generator` against `data/` and writes build-only asse
Practice sources live directly in `data/practice/` with four-digit ASCII filenames, currently `0001.xml` through `0366.xml`; `0000.xml` is also valid. For example, `data/practice/0001.xml` generates `practice/0001/{acoustic,electric,vocal}.json` and corresponding page directories. Songs stay in `data/songs/`. The filename number determines exercise identity and ordering; MusicXML titles determine labels, falling back to the number. Root-level XML, other practice filenames, and nested practice directories are rejected. Adding, renaming, or removing a source requires no registry edit. Regeneration replaces the output tree and catalog, removing old three-digit paths and IDs without aliases or migration. The current 366 exercises produce 1,098 practice parts, plus six performance parts.
The pipeline is Soundslice/MuseScore -> uncompressed `.xml` -> musical model -> part generation -> semantic JSON, audio, and engraved pages -> packaged SQLite catalog. Export MusicXML 3.1 or 4.0, `score-partwise`, with one part and one staff. The musical model retains independent voices and per-tone durations/ties; automatic practice generation additionally requires a single monophonic melody and optional structured harmonies. Packaged music describes each generated part, not an archival copy of the XML.
The pipeline is Soundslice/MuseScore -> uncompressed `.xml` -> musical model and rhythm plans -> part generation -> semantic JSON, audio, engraved pages, and Pico title artwork -> packaged SQLite catalog and assets. Export MusicXML 3.1 or 4.0, `score-partwise`, with one part and one staff. The musical model retains independent voices and per-tone durations/ties; automatic practice generation additionally requires a single monophonic melody and optional structured harmonies. Packaged music describes each generated part, not an archival copy of the XML.
### Rhythm Sources
`data/rhythms/<meter>/<series>/` is an independent source collection, excluded from application generation and packaging. Each series contains `4^N` four-digit numbered XML files for an N/4 meter: 16 in 2/4, 64 in 3/4, 256 in 4/4, and 1,024 in 5/4, totaling 2,720 files across both series. Numbering starts at `0001.xml` in every directory.
`data/rhythms/<meter>/<series>/` is an active build input for accompaniment, using meter directories `2-4`, `3-4`, `4-4`, and `5-4`. Each series contains `4^N` four-digit numbered XML files for an N/4 meter: 16 in 2/4, 64 in 3/4, 256 in 4/4, and 1,024 in 5/4, totaling 2,720 files across both series. Numbering starts at `0001.xml` in every directory. Snippets do not create menu entries and their XML is not deployed.
- Series `1` uses two measures. Read the file number minus one as `2*N` binary digits, most significant first: `0` is a quarter rest and `1` is a C5 quarter note.
- Series `2` uses one measure. Read the file number minus one as `N` base-4 digits, most significant first: `0` is a quarter note, `1` a quarter rest, `2` two eighth notes, and `3` an eighth rest followed by an eighth note.
- All pitched notes are C5, with a Vocal part, treble clef, C major, and tempo 120. Quarter-only measures use divisions 4; measures containing eighths use divisions 8. Consecutive pitched eighths beam together, interrupted by rests, quarter notes, and barlines. In 4/4, beams also break at the half-bar; the other meters use whole-measure groups. Isolated eighth notes have no beam.
`generator.rhythm.load_library()` parses the library once per asset generation. Immutable snippets retain exact event onsets, durations, written values, rests, beams, and continuations. Their placeholder pitches and tempo do not transpose chords or select playback BPM. A snippet must contain one or two complete sequential bars in one supported meter and voice, without grace notes, tuplets, time modifications, tremolo, or meter changes.
`generator.accompaniment` selects a two-bar motif, either a complete two-bar snippet or a pair of one-bar snippets, and realizes a four-bar `A, B, A, B'` cycle. The fourth bar varies one beat when a suitable candidate exists. The cycle repeats through the section, restarting at section or meter boundaries; a final partial cycle uses only the required bars. Acoustic establishes the foundation, while electric accompaniment is planned against both the authored melody and that acoustic plan. Selection ranks harmonic coverage and density, then attack collisions, occupied melody time, and pulse, with source identity breaking ties deterministically. It does not enumerate every pair of one-bar snippets or introduce randomness.
Pickups use the corresponding tail of a full-meter rhythm bar. Harmonic state carries across bars until changed or canceled by N.C.; a snippet attack sounds only when a chord is active. A held generated chord stops before a different harmony or N.C., leaving silence until the next selected attack. Melody attacks influence selection but do not directly delete acoustic attacks. Missing rhythm material, unsupported accompaniment meters, and mid-measure meter changes produce errors. The broader import and engraving model still accepts the meters described below.
### MusicXML Contract
| Area | Accepted notation |
@@ -74,7 +81,7 @@ The pipeline is Soundslice/MuseScore -> uncompressed `.xml` -> musical model ->
| Small notes | Grace notes with zero metric duration, cue notes, and measured single/paired tremolos. |
| Staff state | G/F/C, percussion, and no-clef staves; octave clefs and octave-shift spans; major/minor/modal keys or custom seven-letter alterations; timed key, clef, and meter changes. Authored TAB and technical fingering instructions are rejected; TAB is generated from pitched guitar parts. |
| Meter | Simple, compound, additive, mixed-denominator, and unmetered timelines. A partial measure must be marked `implicit="yes"`; an initial implicit measure is the arrangement pickup. Leading rests are retained. |
| Harmony | Structured roots, kinds, inversions, slash basses, and degree additions/alterations/subtractions. Harmony events have their own offsets, including inside held notes. Their positions never split or shorten notes. |
| Harmony | Structured roots, kinds, inversions, slash basses, and degree additions/alterations/subtractions. Harmony events have their own offsets, including inside held notes. Their positions never split or shorten authored melody notes. |
| Text | Title, composer, part name, multiple lyric verses with syllabification, elisions, and extenders, rehearsal labels at measure starts, and Vamp directives. |
| Tempo | Metronome directions and positive quarter-note BPM from `<sound tempo="...">`, either within a direction or directly in a measure. Playback/navigation attributes on `sound` are rejected. |
| Reading order | All measures are written out. Repeat bars, numbered endings, navigation jumps, and pass-specific ties are rejected. Ordinary barlines and contiguous ties remain supported. |
@@ -87,17 +94,9 @@ Chord labels are generated from structured harmony using `M`, `m`, `aug`, `dim`,
Standard MusicXML kinds and explicit degrees remain authoritative. For Soundslice compatibility, a text-only `<kind text="mM7">other</kind>` (also `m7b5`, `7b9`, and other recognized chord spellings) is normalized into a standard kind and degrees. Complete symbols such as `(Fm7)` or `(Bb7)` supply their own root; an exporter placeholder C is accepted only for text-only `other` harmonies. A conflicting non-placeholder root, conflicting bass, contradictory structured chord, or unknown spelling is an error with source/measure context. Raw exporter text is not retained as an override that could become stale during transposition. This compatibility path does not require editing the XML or weakening structured-harmony validation.
### Vamp Metadata
### Vamp Directives
In MuseScore's score properties, set `vamp:style` to a supported player style. It exports as standard MusicXML miscellaneous metadata:
```xml
<miscellaneous>
<miscellaneous-field name="vamp:style">Medium Swing</miscellaneous-field>
</miscellaneous>
```
The default is Medium Swing; `Med.`, `Med. Swing`, and `Medium` remain style aliases. Keys, modes, keyless notation, and local changes are preserved. There is no transposition reference or `vamp:tonic` property.
Musical genres and player-style presets are removed. There is no default genre, alias table, generated `style` metadata, or genre selection. The strict importer rejects every obsolete `vamp:style` miscellaneous-field hint, including an empty value, instead of discarding it. The XML writer does not emit that field. Keys, modes, keyless notation, and local changes remain preserved; there is no transposition reference or `vamp:tonic` property. Visual styling, font attributes, engraving `Style`, presentation fingerprints, and ordinary `<bar-style>` notation are unrelated and remain supported.
Parenthesized chord symbols mark optional turnarounds, including Soundslice's text-only `(Fm7)` and `(Bb7)` exports. Alternatively, place staff text `vamp:optional-chord` at the same onset as exactly one harmony; this text may be hidden. Earlier arrangement sections retain those chords and their parentheses; the final ending omits them. The normalized harmony stores `parenthesized` display metadata separately from the generator-only `chord_optional` flag. Orphaned/duplicate markers and unknown `vamp:` properties are errors; consumed directives never enter runtime JSON.
@@ -114,22 +113,22 @@ Ordinary ties still link adjacent matching tones across measures and systems. Tu
A newly generated bundle has three top-level fields:
- `music`: metadata and a single staff containing measures, independent voice lanes, timed harmonies/attributes/tempos, instruments, tuplets, and octave spans.
- `presentation`: engraver name, style checksum, fixed width/height, and ordered relative PNG paths with SHA-256 checksums.
- `presentation`: engraver name, style checksum, fixed width/height, ordered relative PNG paths with SHA-256 checksums, and the `title_image` content identifier for Pico artwork.
- `playback`: availability, exact performed beat duration/events, shared PCM sample descriptors, and a generated release envelope. Unavailable interpretations carry a reason instead of an approximate timeline.
There are no authored-key/selected-key metadata pairs. The score's key metadata describes its actual music, and menus display that key alongside its part name. Instrument normalization transforms explicit key cancellations with the same interval as the notes.
There are no authored-key/selected-key metadata pairs or musical genre/style fields. The score's key metadata describes its actual music, and menus display that key alongside its part name. Instrument normalization transforms explicit key cancellations with the same interval as the notes. Presentation's `style_sha256` identifies visual engraving settings and fonts, not a musical genre.
Measure durations, event onsets/durations, harmony offsets, span times, and pitch alterations use `{ "numerator": N, "denominator": D }`. Times are in quarter-note units and alterations in semitones. There is no tick grid, display-duration surrogate, or runtime flattening into one voice. Written note types, dots, and time modifications remain distinct from metric duration. Every event owns a voice; every chord tone has its own ID, written duration, duration, and reciprocal tie links. Tuplets reference events and belong to one voice.
### Shared Part Generation
`parts.py` owns shared melody-copying and harmonic-rhythm construction. `arrange/generate.py` supplies the existing seven-section song form; `practice/generate.py` supplies a single source pass. `pitch.py` resolves source instrument transposition and the shared written-octave convention. The former generic twelve-key transposer and independent practice/arrangement fretboard implementations are removed.
`parts.py` owns shared source validation, melody-copying, measure construction, rests, and solo slashes. `rhythm.py` loads snippets; `accompaniment.py` plans phrase cycles and realizes their chord events. `arrange/generate.py` supplies the existing seven-section song form; `practice/generate.py` supplies a single source pass. `pitch.py` resolves source instrument transposition and the shared written-octave convention. The former generic twelve-key transposer and independent practice/arrangement fretboard implementations are removed.
Practice acoustic accompaniment is derived from structured harmony roots, kinds, degrees, inversions, and slash basses using the shared accompaniment rhythm. A missing harmony or N.C. produces rests, not guessed chords. Electric retains the exact melody, spelling, rhythm, ties, and tuplets; vocal retains those same authored notes and receives scat lyrics. Song vocal lyrics are unchanged. Practice optional-turnaround harmonies are played once, without creating extra sections.
Accompaniment attacks at each bar start and authored harmony onset, sustaining until the next harmony onset or bar end. Harmony carries into subsequent bars until changed or explicitly canceled by N.C.; melody attacks never suppress accompaniment. This same rule applies to electric accompaniment sections in songs. Song acoustic rhythm and solo-slash patterns are unchanged.
Acoustic accompaniment in songs and practice uses the planned foundation. Electric accompaniment sections in songs use its response plan; electric melody sections still copy the melody. Song sections remain Rhythm Lead-in, Vocal Melody, Guitar Melody, Guitar Solo, Guitar Solo, Vocal Melody, and Ending, with each part retaining its melody, accompaniment, rest, or solo-slash role. Optional turnarounds are removed before planning the final ending. The generated phrase cycles contain explicit events in reading order, not runtime loops or repeat signs.
Each source is parsed, validated, and normalized once for all three parts. Part builders treat the prepared source as read-only and own their generated notes and spans. Source-path checks reject colliding song slugs before writing parts: for example, `one-two.xml` and `one_two.xml` would both produce `one-two-acoustic.json` and cannot coexist.
Each source is parsed, validated, and normalized once for all three parts. The batch generator computes shared accompaniment plans before building those parts. Public `arrange_document`, `practice_document`, `generate_performance_scores`, and `generate_practice_scores` calls require a keyword `rhythms` argument holding a `generator.rhythm.Library`; internal part builders receive precomputed plans. Builders treat the prepared source and plans as read-only and own their generated notes and spans. Source-path checks reject colliding song slugs before writing parts: for example, `one-two.xml` and `one_two.xml` would both produce `one-two-acoustic.json` and cannot coexist.
All three parts, for both songs and practice, use octave-transposing notation: written notes sound one octave lower. This includes the vocal part for the singer's range. Unmarked source notation uses that convention by default; copied melodies retain their written pitches and octave numbers. For example, written C4 sounds C3 (A-string fret 3), and written C#6 sounds C#5, within the 0-20 fret range. Generated accompaniment is notated using the same convention.
@@ -161,10 +160,12 @@ At runtime the shared on-demand score loader serves both display and audio. A ba
### Engraving Boundary
`generator/engraving/` is a custom Python engraver, not a wrapper around MuseScore or another notation engine. `layout_score(document, style, fonts)` produces immutable layout pages with exact source intervals, notation anchors and drawing primitives. `rasterize_page(page, fonts)` draws one page at four-times resolution and downsamples it. `engrave_tree(root, fonts)` retains the asset-pipeline entry point. `generator/musicxml_writer.py` remains a round-trip/export utility and is not involved in engraving.
`generator/engraving/` is a custom Python engraver, not a wrapper around MuseScore or another notation engine. `generator.engraving.layout.layout_score(document, style, fonts)` produces immutable layout pages with exact source intervals, notation anchors and drawing primitives. `generator.engraving.raster.rasterize_page(page, fonts)` draws one page at four-times resolution and downsamples it. `generator.engraving.pipeline.engrave_tree(root, fonts)` is the asset-pipeline entry point. `fonts.Fonts`, `types.Style`, and `types.EngravingError` live in their respective engraving modules; page constants and `check_page` live in `validation`. Package initializers do not re-export these APIs. `generator/musicxml_writer.py` remains a round-trip/export utility and is not involved in engraving.
Engraving supports a bounded process pool (`VAMP_ENGRAVING_JOBS`, default 2) and a build-only content-addressed page cache (`VAMP_ENGRAVING_CACHE_DIR`, default `build/build_common/engraving-cache`). Keys include the complete musical document, engraving implementation, fonts/metadata, style, Python and rendering dependency versions. Playback is deliberately excluded from page keys and is always rebuilt independently. Cache hits are copied into the current staged output and checked for checksums, PNG decoding, dimensions, blank pages and clipped ink. Missing or corrupt entries are regenerated. All semantic and asset validation, database construction and transactional publication remain in place. The cache is not installed and can be deleted at any time when builds are idle. CI saves it after the native build and reuses it for ARM packaging; generation prints stage timing and per-worker progress instead of remaining silent during engraving.
Reuse has two levels. A full-score hit restores complete page images for the same musical document and metadata. A notation-body group shares system layout and body pixels among documents with identical staff content, while retaining each document's title, subtitle, pagination, footer, and complete semantic bundle. Body keys include the rendering context and actual page-band geometry; incompatible placement cannot reuse those pixels. Each header/footer is composed separately, and pages whose drawing primitives cannot be separated safely use full-page rasterization. This reuse never merges exercise IDs, catalog records, or authored music.
`notation.py` interprets staff state, exact pitch positions and shared accidental state; `notes.py` and `attributes.py` construct symbols; `spans.py` handles beams, ties, tuplets, octave shifts and lyric continuations; `layout.py` handles spacing and pagination. `fonts.py` reads bundled OTF outlines and SMuFL metadata, caching metrics and outlines for one generation run. `raster.py` uses FontTools, aggdraw and Pillow only as low-level font and drawing libraries. All dependencies are pinned in `generator/requirements.txt` and remain build-only. Standard glyph names and their upstream notice are vendored alongside the fonts; no system music fonts are consulted.
Voices retain separate stems and collision offsets; chord tones retain their own written durations and per-tone ties. Pitched and percussion material use their own staff positions. Unknown exact microtonal alterations use a rational semitone annotation rather than a rounded accidental; an unavailable explicit SMuFL glyph is an error. The broader model/engraver supports this notation, while automatic guitar and practice generation enforce the source contracts above.
@@ -179,9 +180,11 @@ Pages are 976x484 pixels with 12-pixel staff spacing, 28-pixel horizontal margin
### Validation And Publication
Generation validates sources and complete staged output before replacing the previous asset directory. Replacement is always complete; deleted sources cannot leave stale JSON, pages, or audio. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalog. Runtime packages contain `catalog.db` with embedded semantic JSON, PNG pages, generated PCM WAVs, and the OTF UI text font, not MusicXML, MuseScore, or Python dependencies. Python test discovery runs model/import/export tests; the normal asset build additionally qualifies and audits every catalog score.
Generation validates sources and complete staged output before replacing the previous asset directory. Replacement is always complete; deleted sources cannot leave stale JSON, pages, audio, or Pico title assets. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalog. Runtime packages contain `catalog.db` with embedded semantic JSON and title-image identifiers, PNG pages, generated PCM WAVs, Pico title images, and the OTF UI text font, not MusicXML, MuseScore, or Python dependencies. Python test discovery runs model/import/export tests; the normal asset build additionally qualifies and audits every catalog score.
The page cache uses `<renderer fingerprint>/<score fingerprint>/` directories, without a manual cache revision. `python3 -m generator --renderer-fingerprint` reports the same identity used by rendering and CI: source bytes, fonts, style, Python implementation/version, and rendering-library versions. Python compiler/build timestamps are excluded. CI keys append all generator and active XML inputs and restore earlier snapshots only within the same renderer directory; only that directory is saved after a successful build. A renderer or environment change intentionally starts cold. Progress reports the resolved directory and reused, missing, and rejected counts. Obsolete cache directories are not migrated and can be removed when builds are idle.
The page cache uses `<renderer fingerprint>/fullscore/<score fingerprint>/` and `<renderer fingerprint>/notation/<band fingerprint>/` directories, without a manual cache revision. `python3 -m generator --renderer-fingerprint` reports the same identity used by rendering and CI: source bytes, fonts, visual style, Python implementation/version, and rendering-library versions. Python compiler/build timestamps are excluded. CI keys append all generator and active XML inputs, including rhythms, and restore earlier snapshots only within the same renderer directory; only that directory is saved after a successful build.
`cold namespace` means that renderer's persistent directory does not yet exist. A renderer or environment change starts a new namespace; deleting a namespace has the same effect. It does not disable reuse within the current run: shared notation is still laid out once per group and its body pixels can be reused as other titles are composed, even without a persistent cache. Conversely, a warm directory does not guarantee that a particular score is cached. Progress reports full-score hits, body hits, newly rasterized body bands, composed pages, missing/rejected entries, and phase timings. Worker phase timings are summed and may exceed wall time. Obsolete namespaces are not migrated and can be removed when builds are idle.
Diagnostics identify the file, part, measure, and unsupported element or invalid relationship. Examples include `part P1, measure 4: unsupported musical element <slur>`, `tie stop has no contiguous matching start`, and `optional-chord marker must identify exactly one harmony`.
@@ -201,8 +204,13 @@ The display is rotated 90 degrees clockwise into a 320x240 landscape UI. The top
- Yellow: retrying or remote request failed.
- Green: connected to the Zero API.
The body stays blank until a performance or practice score is active, then it shows the active title, selected part, and page count. Touch controls are tap for next, double tap for previous, and tap-and-hold for select.
The Pico text renderer uses an antialiased bitmap atlas generated from `package/assets/fonts/petaluma/PetalumaScript.otf`. The shared font CMake rule generates `generated/vamp_style/font.cpp` separately in each native/Pico build directory, using `generator/generate_pico_font.py --output <path>`. Font and generator changes trigger regeneration. The generated atlas includes ASCII kerning pairs so display text such as `VAMP` uses the same spacing as the source font. The pinned Pillow build dependency is declared in `generator/requirements.txt`.
The body shows a fixed Product Vamp/status label when no score is active, then the active title, selected part, and page count. Touch controls are tap for next, double tap for previous, and tap-and-hold for select. All visible text uses Petaluma Script, matching score-page text and the Zero UI.
`generator.pico_artwork` shapes text on the build host with the bundled `PetalumaScript.otf` and Pillow's Raqm support. Fixed labels (`VAMP`, connection/status messages, and `PAGE`), digits, and `/` become four-bit coverage sprites. The Pico CMake rule writes `generated/pico/static.p4` and embeds it through `generated/pico/artwork.S`. The Zero does not link this sprite atlas; Luma renders its dynamic UI text from the packaged OTF.
Score generation renders titles as 296x64 four-bit coverage images under `pico/titles/<sha256>.mask`. Each file is 9,480 bytes: an eight-byte `VPTI` header with little-endian width/height, followed by packed coverage. Identical image bytes share an asset without merging score identities. The generated `presentation.title_image` identifier is also stored in catalog metadata; no separate lookup manifest is required. These assets are generated and validated before staged output is published. Remote status includes `title_image=<sha256>`; `POST /api/remote/title` with `{"id":"<sha256>"}` serves validated bytes only for catalog-registered titles. Pico fetches a changed title, validates its fixed format and bounds, rejects stale responses, and retains the selected image. Part labels and page counts use fixed sprites without reshaping or refetching title text. Commands take priority over image requests, and failed title requests display `UNAVAILABLE` until a retry succeeds.
Pico performs coverage blending and sprite placement only: no OTF parsing, font shaping, glyph rasterization, or general text renderer runs in firmware. Missing glyphs and invalid source text are build errors. Font or generator changes rebuild both static firmware sprites and title artwork. Direct generation uses `python3 -m generator.pico_artwork --static-output <path>` for firmware sprites and `python3 -m generator.pico_artwork --assets <generated-assets>` for title assets. Python, Pillow/Raqm, and FontTools remain build-only dependencies.
The root `CMakeLists.txt` is also the Pico firmware entrypoint when configured with `-DVAMP_PICO_FIRMWARE_PROJECT=ON`. That mode returns before the Zero/Linux targets are declared, so the Pico build can use the embedded ARM toolchain cleanly.
cmake/font.cmake
-21
View File
@@ -1,21 +0,0 @@
find_package(Python3 COMPONENTS Interpreter REQUIRED)
execute_process(
COMMAND ${Python3_EXECUTABLE} -c "from PIL import features; assert features.check_feature('raqm')"
RESULT_VARIABLE VAMP_PILLOW_STATUS
ERROR_QUIET
)
if(NOT VAMP_PILLOW_STATUS EQUAL 0)
message(FATAL_ERROR "Font generation requires Pillow with Raqm support. Install libfribidi0 on Debian/Ubuntu, then run: ${Python3_EXECUTABLE} -m pip install -r ${CMAKE_CURRENT_SOURCE_DIR}/generator/requirements.txt")
endif()
set(VAMP_FONT_SOURCE ${CMAKE_CURRENT_BINARY_DIR}/generated/vamp_style/font.cpp)
add_custom_command(
OUTPUT ${VAMP_FONT_SOURCE}
COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/generator/generate_pico_font.py
--output ${VAMP_FONT_SOURCE}
DEPENDS
${CMAKE_CURRENT_SOURCE_DIR}/generator/generate_pico_font.py
${CMAKE_CURRENT_SOURCE_DIR}/generator/requirements.txt
${CMAKE_CURRENT_SOURCE_DIR}/package/assets/fonts/petaluma/PetalumaScript.otf
VERBATIM
)
cmake/pico.cmake
+2 -2
View File
@@ -36,7 +36,7 @@ set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
pico_sdk_init()
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/font.cmake)
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/pico_artwork.cmake)
set(VAMP_MODULUS_INCLUDE_DIR "" CACHE PATH "Path to the Modulus public include directory")
set(VAMP_RESOLVED_MODULUS_INCLUDE_DIR "${VAMP_MODULUS_INCLUDE_DIR}")
@@ -76,7 +76,6 @@ endif()
add_executable(product-vamp-pico2w
src/vamp_style.cpp
${VAMP_FONT_SOURCE}
src/vamp_pico/display.cpp
src/vamp_pico/http_client.cpp
src/vamp_pico/input.cpp
@@ -86,6 +85,7 @@ add_executable(product-vamp-pico2w
src/vamp_protocol/page_control.cpp
src/vamp_protocol/gesture.cpp
)
vamp_add_pico_artwork(product-vamp-pico2w)
target_include_directories(product-vamp-pico2w PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/include
${VAMP_RESOLVED_MODULUS_INCLUDE_DIR}
cmake/pico_artwork.cmake
+28
View File
@@ -0,0 +1,28 @@
find_package(Python3 COMPONENTS Interpreter REQUIRED)
execute_process(
COMMAND ${Python3_EXECUTABLE} -c "import PIL.features; import fontTools.ttLib; assert PIL.features.check_feature('raqm')"
RESULT_VARIABLE VAMP_PILLOW_STATUS
ERROR_QUIET
)
if(NOT VAMP_PILLOW_STATUS EQUAL 0)
message(FATAL_ERROR "Pico artwork generation requires Pillow with Raqm and fontTools. Install libfribidi0 on Debian/Ubuntu, then run: ${Python3_EXECUTABLE} -m pip install -r ${CMAKE_CURRENT_SOURCE_DIR}/generator/requirements.txt")
endif()
# Zero uses packaged font files; only the Pico embeds this fixed sprite atlas.
function(vamp_add_pico_artwork target)
set(VAMP_PICO_ARTWORK_BINARY ${CMAKE_CURRENT_BINARY_DIR}/generated/pico/static.p4)
set(VAMP_PICO_ARTWORK_SOURCE ${CMAKE_CURRENT_BINARY_DIR}/generated/pico/artwork.S)
add_custom_command(
OUTPUT ${VAMP_PICO_ARTWORK_BINARY}
COMMAND ${Python3_EXECUTABLE} -m generator.pico_artwork --static-output ${VAMP_PICO_ARTWORK_BINARY}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
DEPENDS
${CMAKE_CURRENT_SOURCE_DIR}/generator/pico_artwork.py
${CMAKE_CURRENT_SOURCE_DIR}/generator/requirements.txt
${CMAKE_CURRENT_SOURCE_DIR}/package/assets/fonts/petaluma/PetalumaScript.otf
VERBATIM
)
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/src/vamp_pico/artwork.S.in ${VAMP_PICO_ARTWORK_SOURCE} @ONLY)
set_source_files_properties(${VAMP_PICO_ARTWORK_SOURCE} PROPERTIES OBJECT_DEPENDS ${VAMP_PICO_ARTWORK_BINARY})
target_sources(${target} PRIVATE ${VAMP_PICO_ARTWORK_SOURCE})
endfunction()
cmake/zero.cmake
+4 -5
View File
@@ -8,6 +8,7 @@ set(CMAKE_CXX_EXTENSIONS OFF)
find_package(modulus CONFIG REQUIRED COMPONENTS boost_json openssl raylib)
find_package(Threads REQUIRED)
find_package(Python3 COMPONENTS Interpreter REQUIRED)
find_package(ligo CONFIG REQUIRED)
find_package(noto CONFIG REQUIRED)
find_package(luma CONFIG REQUIRED)
@@ -34,7 +35,6 @@ cmake_pop_check_state()
if(NOT VAMP_DEPENDENCY_APIS_AVAILABLE)
message(FATAL_ERROR "Product Vamp requires current Sonus sequence and Luma uncached-texture APIs. Run ./modulus.py configure --update after publishing the updated dependencies, or create their Conan packages locally before configuring --no-remote. See CMake's configure log for the compiler diagnostic.")
endif()
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/font.cmake)
execute_process(
COMMAND git rev-parse --short=12 HEAD
@@ -49,7 +49,6 @@ endif()
set(VAMP_STYLE_SOURCES
src/vamp_style.cpp
${VAMP_FONT_SOURCE}
)
add_library(vamp-style STATIC ${VAMP_STYLE_SOURCES})
@@ -140,7 +139,6 @@ file(GLOB_RECURSE VAMP_GENERATOR_INPUTS CONFIGURE_DEPENDS
${CMAKE_CURRENT_SOURCE_DIR}/package/assets/fonts/*
${CMAKE_CURRENT_SOURCE_DIR}/package/assets/fonts/**/*
)
list(FILTER VAMP_GENERATOR_INPUTS EXCLUDE REGEX "/data/rhythms/")
set(VAMP_GENERATED_ASSETS_DIR ${CMAKE_CURRENT_BINARY_DIR}/generated-assets)
set(VAMP_GENERATED_ASSETS_STAMP ${CMAKE_CURRENT_BINARY_DIR}/generated-assets.stamp)
set(VAMP_ENGRAVING_JOBS 2 CACHE STRING "Maximum parallel Python engraving workers (1-16)")
@@ -237,8 +235,8 @@ if(VAMP_BUILD_PICO_FIRMWARE)
file(GLOB VAMP_PICO_SOURCES CONFIGURE_DEPENDS
${CMAKE_CURRENT_SOURCE_DIR}/CMakeLists.txt
${CMAKE_CURRENT_SOURCE_DIR}/cmake/pico.cmake
${CMAKE_CURRENT_SOURCE_DIR}/cmake/font.cmake
${CMAKE_CURRENT_SOURCE_DIR}/generator/generate_pico_font.py
${CMAKE_CURRENT_SOURCE_DIR}/cmake/pico_artwork.cmake
${CMAKE_CURRENT_SOURCE_DIR}/generator/pico_artwork.py
${CMAKE_CURRENT_SOURCE_DIR}/generator/requirements.txt
${CMAKE_CURRENT_SOURCE_DIR}/package/assets/fonts/petaluma/PetalumaScript.otf
${CMAKE_CURRENT_SOURCE_DIR}/cmake/vamp_options.cmake
@@ -250,6 +248,7 @@ if(VAMP_BUILD_PICO_FIRMWARE)
${CMAKE_CURRENT_SOURCE_DIR}/include/vamp_protocol/*.hpp
${CMAKE_CURRENT_SOURCE_DIR}/src/vamp_style.cpp
${CMAKE_CURRENT_SOURCE_DIR}/src/vamp_pico/*.cpp
${CMAKE_CURRENT_SOURCE_DIR}/src/vamp_pico/*.S.in
${CMAKE_CURRENT_SOURCE_DIR}/src/vamp_protocol/*.cpp
)
add_custom_command(
conanfile.py
+2
View File
@@ -32,6 +32,7 @@ class product_vamp_conan(conan.ConanFile):
"data/expected/*.json",
"data/practice/*.xml",
"data/songs/*.xml",
"data/rhythms/**/*.xml",
"tests/fixtures/*",
"tests/fixtures/**/*",
"generator/*.py",
@@ -39,6 +40,7 @@ class product_vamp_conan(conan.ConanFile):
"generator/requirements.txt",
"include/*.hpp",
"include/vamp_style/*.hpp",
"src/vamp_pico/*.S.in",
"include/vamp_pico/*.h",
"include/vamp_pico/capi/*.h",
"include/vamp_pico/*.hpp",
generator/__main__.py
+41 -33
View File
@@ -1,24 +1,28 @@
from __future__ import annotations
import argparse
import pathlib
import shutil
import tempfile
from pathlib import Path
from time import monotonic
import time
from generator.arrange.generate import generate_performance_scores
from generator.practice.generate import generate_practice_scores
from generator.score import count_json_files, validate_generated_tree
from generator.summary import validate_expected_performance_summary, validate_practice_parts
from generator.parts import source_paths
from generator.engraving import Fonts, Style, engrave_tree
from generator.engraving.cache import renderer_fingerprint
from generator.playback import compile_tree
from generator.catalog import build_catalog
import generator.arrange.generate
import generator.catalog
import generator.engraving.cache
import generator.engraving.fonts
import generator.engraving.pipeline
import generator.engraving.types
import generator.parts
import generator.pico_artwork
import generator.playback
import generator.practice.generate
import generator.rhythm
import generator.score
import generator.summary
RUNTIME_FONT_FILES = ("PetalumaScript.otf", "OFL.txt")
def copy_runtime_fonts(static_assets_dir: Path, output_dir: Path) -> None:
def copy_runtime_fonts(static_assets_dir: pathlib.Path, output_dir: pathlib.Path) -> None:
source = static_assets_dir / "fonts" / "petaluma"
destination = output_dir / "fonts" / "petaluma"
destination.mkdir(parents=True, exist_ok=True)
@@ -27,12 +31,12 @@ def copy_runtime_fonts(static_assets_dir: Path, output_dir: Path) -> None:
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Generate Product Vamp package score assets.")
parser.add_argument("--data-dir", type=Path)
parser.add_argument("--output-dir", type=Path)
parser.add_argument("--data-dir", type=pathlib.Path)
parser.add_argument("--output-dir", type=pathlib.Path)
parser.add_argument("--renderer-fingerprint", action="store_true", help="Print the engraving cache identity without generating assets.")
parser.add_argument("--static-assets-dir", type=Path, default=Path("package/assets"))
parser.add_argument("--static-assets-dir", type=pathlib.Path, default=pathlib.Path("package/assets"))
parser.add_argument("--check", action="store_true")
parser.add_argument("--engraving-cache", type=Path)
parser.add_argument("--engraving-cache", type=pathlib.Path)
parser.add_argument("--jobs", type=int, default=1, choices=range(1, 17), metavar="1-16")
parser.add_argument("--progress", action="store_true")
args = parser.parse_args()
@@ -49,11 +53,11 @@ def generate_assets(args: argparse.Namespace) -> None:
if cache_dir and (cache_dir.is_relative_to(output_dir) or output_dir.is_relative_to(cache_dir)):
raise ValueError('engraving cache and generated assets must not overlap')
progress = getattr(args, 'progress', False)
started = monotonic()
started = time.monotonic()
def report(message):
if progress:
print(f'Assets [{monotonic() - started:.0f}s]: {message}', flush=True)
print(f'Assets [{time.monotonic() - started:.0f}s]: {message}', flush=True)
report('checking source paths')
for source in sorted(data_dir.rglob('*.xml')):
@@ -61,29 +65,33 @@ def generate_assets(args: argparse.Namespace) -> None:
continue
if source.parent not in (data_dir / 'practice', data_dir / 'songs'):
raise ValueError(f'{source}: MusicXML sources belong directly under data/practice or data/songs')
source_paths(data_dir / 'songs')
source_paths(data_dir / 'practice', practice=True)
generator.parts.source_paths(data_dir / 'songs')
generator.parts.source_paths(data_dir / 'practice', practice=True)
report('loading rhythm snippets')
rhythms = generator.rhythm.load_library(data_dir / 'rhythms')
output_dir.parent.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix='.vamp-assets-', dir=output_dir.parent) as temporary:
staged = Path(temporary) / 'assets'
staged = pathlib.Path(temporary) / 'assets'
staged.mkdir(parents=True, exist_ok=True)
copy_runtime_fonts(static_assets_dir, staged)
report('generating performance and practice parts')
performance_count = generate_performance_scores(data_dir / 'songs', staged / 'songs')
practice_count = generate_practice_scores(data_dir / 'practice', staged / 'practice')
validate_generated_tree(staged)
performance_count = generator.arrange.generate.generate_performance_scores(data_dir / 'songs', staged / 'songs', rhythms=rhythms)
practice_count = generator.practice.generate.generate_practice_scores(data_dir / 'practice', staged / 'practice', rhythms=rhythms)
generator.score.validate_generated_tree(staged)
if args.check:
validate_expected_performance_summary(data_dir / 'expected' / 'performance_summary.json', staged / 'songs')
validate_practice_parts(staged / 'practice')
generator.summary.validate_expected_performance_summary(data_dir / 'expected' / 'performance_summary.json', staged / 'songs')
generator.summary.validate_practice_parts(staged / 'practice')
report('compiling playback')
compile_tree(staged)
generator.playback.compile_tree(staged)
report('engraving pages')
engrave_tree(staged, static_assets_dir / 'fonts' / 'petaluma', cache_dir=cache_dir, jobs=getattr(args, 'jobs', 1), progress=progress)
validated_pages = generator.engraving.pipeline.engrave_tree(staged, static_assets_dir / 'fonts' / 'petaluma', cache_dir=cache_dir, jobs=getattr(args, 'jobs', 1), progress=progress)
report('rendering Pico titles')
generator.pico_artwork.generate_titles(staged, static_assets_dir / 'fonts' / 'petaluma')
report('validating score, page, and audio assets')
validate_generated_tree(staged)
generator.score.validate_generated_tree(staged, validated_pages)
report('building catalog database')
build_catalog(staged)
backup = Path(temporary) / 'previous'
generator.catalog.build_catalog(staged)
backup = pathlib.Path(temporary) / 'previous'
if output_dir.exists():
output_dir.rename(backup)
try:
@@ -94,14 +102,14 @@ def generate_assets(args: argparse.Namespace) -> None:
raise
report('publishing complete')
print(f'Generated Product Vamp assets: {performance_count} performance scores, '
f'{practice_count} practice parts, {count_json_files(output_dir)} total JSON files.')
f'{practice_count} practice parts, {generator.score.count_json_files(output_dir)} total JSON files.')
if args.check:
print('Generator check passed.')
def main() -> int:
args = parse_args()
if args.renderer_fingerprint:
print(renderer_fingerprint(Fonts(args.static_assets_dir / 'fonts' / 'petaluma'), Style()))
print(generator.engraving.cache.renderer_fingerprint(generator.engraving.fonts.Fonts(args.static_assets_dir / 'fonts' / 'petaluma'), generator.engraving.types.Style()))
else:
generate_assets(args)
return 0
generator/accompaniment.py
+221
View File
@@ -0,0 +1,221 @@
from __future__ import annotations
import copy
import dataclasses
import fractions
import generator.model
import generator.rhythm
@dataclasses.dataclass(frozen=True)
class Context:
measure: dict
previous: dict | None
offset: fractions.Fraction
intervals: tuple[tuple[fractions.Fraction, fractions.Fraction], ...]
attacks: frozenset[fractions.Fraction]
occupied: tuple[tuple[fractions.Fraction, fractions.Fraction], ...]
@dataclasses.dataclass(frozen=True)
class Plan:
acoustic: tuple[generator.rhythm.Bar, ...]
electric: tuple[generator.rhythm.Bar, ...]
def active_chord(measure: dict, onset: fractions.Fraction, previous: dict | None) -> dict | None:
active = previous
for chord in measure['harmonies']:
if generator.model.fraction(chord['onset']) > onset:
break
active = chord
return active if active and active['kind'] != 'none' else None
def carried_chords(measures: list[dict], initial: dict | None = None) -> list[dict | None]:
result = []
for measure in measures:
result.append(initial)
if measure['harmonies']:
initial = measure['harmonies'][-1]
return result
def context(measure: dict, previous: dict | None) -> Context:
length = generator.model.fraction(measure['duration'])
nominal = generator.model.meter_length(measure['meter'])
offset = nominal - length if measure.get('implicit') and length < nominal else fractions.Fraction()
boundaries = sorted({fractions.Fraction(), length} | {generator.model.fraction(h['onset']) for h in measure['harmonies']})
intervals = tuple((a, b) for a, b in zip(boundaries, boundaries[1:]) if b > a and active_chord(measure, a, previous))
attacks, occupied = set(), []
for voice in measure['voices']:
for note in voice['notes']:
pitches = [p for p in note['pitches'] if not p.get('cue')]
if not pitches or note.get('cue'):
continue
start = generator.model.fraction(note['onset'])
end = start + generator.model.fraction(note['duration'])
if not all('tie_from' in p for p in pitches):
attacks.add(start)
occupied.append((start, end))
return Context(measure, previous, offset, intervals, frozenset(attacks), tuple(occupied))
def sounding_attacks(bar: generator.rhythm.Bar, item: Context) -> tuple[fractions.Fraction, ...]:
return tuple(t - item.offset for t in bar.attacks() if any(a <= t - item.offset < b for a, b in item.intervals))
def sounding_spans(bar: generator.rhythm.Bar, item: Context) -> tuple[tuple[fractions.Fraction, fractions.Fraction], ...]:
return tuple((max(event.onset - item.offset, a), min(event.onset + event.duration - item.offset, b)) for event in bar.events if event.sounding and event.onset + event.duration > item.offset for a, b in item.intervals if a <= max(event.onset - item.offset, 0) < b)
def rank(bar: generator.rhythm.Bar, contexts: list[tuple[Context, tuple[fractions.Fraction, ...]]], electric: bool) -> tuple:
missing_bars, uncovered, density, collisions, occupied, pulse = 0, 0, fractions.Fraction(), 0, 0, 0
for item, acoustic in contexts:
attacks = sounding_attacks(bar, item)
spans = sounding_spans(bar, item)
missing_bars += bool(item.intervals) and not spans and not electric
uncovered += sum(not any(start < b and end > a for start, end in spans) for a, b in item.intervals)
duration = sum((b - a for a, b in item.intervals), fractions.Fraction())
density += abs(len(attacks) - duration / (4 if electric else 2))
collisions += sum((t in item.attacks) + (t in acoustic) for t in attacks)
occupied += sum(any(a <= t < b for a, b in item.occupied) for t in attacks)
pulse -= sum(2 if t == 0 else 1 if t.denominator == 1 else 0 for t in attacks)
return missing_bars, uncovered, density, collisions, occupied, pulse
def add_ranks(left: tuple, right: tuple) -> tuple:
return tuple(a + b for a, b in zip(left, right))
def cycle(snippets: tuple[generator.rhythm.Snippet, ...], contexts: list[Context], acoustic: list[generator.rhythm.Bar] | None = None) -> tuple[generator.rhythm.Bar, ...]:
electric = acoustic is not None
bars = {}
for snippet in snippets:
for index, bar in enumerate(snippet.bars):
bars.setdefault(bar, f'{snippet.id}:{index}')
phases = [[(item, sounding_attacks(acoustic[i], item) if acoustic else ()) for i, item in enumerate(contexts) if i % 2 == phase] for phase in range(2)]
scores = [{bar: rank(bar, phase, electric) for bar in bars} for phase in phases]
candidates = [(add_ranks(scores[0][s.bars[0]], scores[1][s.bars[1]]), s.id, s.bars) for s in snippets if len(s.bars) == 2]
single = [s for s in snippets if len(s.bars) == 1]
if single:
pair = tuple(min(single, key=lambda s: (scores[phase][s.bars[0]], s.id)) for phase in range(2))
candidates.append((add_ranks(scores[0][pair[0].bars[0]], scores[1][pair[1].bars[0]]), '|'.join(s.id for s in pair), tuple(s.bars[0] for s in pair)))
score, _, pair = min(candidates, key=lambda candidate: candidate[:2])
if score[0]:
failing = next(item.measure for i, item in enumerate(contexts) if item.intervals and not sounding_spans(pair[i % 2], item))
raise ValueError(f'measure {failing["number"]}: no rhythm motif provides acoustic accompaniment in every harmonically active bar')
fourth = [(item, sounding_attacks(acoustic[i], item) if acoustic else ()) for i, item in enumerate(contexts) if i % 4 == 3]
second = pair[1]
alternatives = [bar for bar in bars if sum(bar.beat(n) != second.beat(n) for n in range(int(second.length))) == 1]
variation = min(alternatives, key=lambda bar: (rank(bar, fourth, electric), bars[bar])) if alternatives and fourth else second
if rank(variation, fourth, electric)[0]:
variation = second
result = (pair[0], second, pair[0], variation)
if electric and any(item.intervals for item in contexts) and not any(sounding_spans(result[i % 4], item) for i, item in enumerate(contexts)):
raise ValueError(f'measure {contexts[0].measure["number"]}: rhythm library cannot provide an audible electric response')
return result
def plan_section(measures: list[dict], library: generator.rhythm.Library, *, initial: dict | None = None, electric: bool = True) -> Plan:
previous = carried_chords(measures, initial)
acoustic, responses = [], []
start = 0
while start < len(measures):
meter = measures[start]['time_signature']
snippets = library.for_meter(meter)
end = start + 1
while end < len(measures) and measures[end]['time_signature'] == meter and not measures[end].get('section_label'):
end += 1
segment = measures[start:end]
for measure in segment:
if any('meter' in change and generator.model.fraction(change['onset']) != 0 for change in measure.get('attributes', [])):
raise ValueError(f'measure {measure["number"]}: accompaniment does not support mid-measure meter changes')
contexts = [context(m, previous[start + i]) for i, m in enumerate(segment)]
foundation = cycle(snippets, contexts)
chosen = [foundation[i % 4] for i in range(len(segment))]
response = cycle(snippets, contexts, chosen) if electric else foundation
acoustic.extend(chosen)
responses.extend(response[i % 4] for i in range(len(segment)))
start = end
return Plan(tuple(acoustic), tuple(responses))
def same_chord(left: dict | None, right: dict | None) -> bool:
return left is not None and right is not None and all(left.get(k) == right.get(k) for k in ('root', 'kind', 'degrees', 'bass', 'inversion'))
def normalize_beams(events: list[dict]) -> None:
for level in sorted({beam['level'] for event in events for beam in event.get('beams', [])}):
groups, group = [], []
for event in events:
beam = next((b for b in event.get('beams', []) if b['level'] == level), None)
if not beam or beam['type'] in ('begin', 'forward hook', 'backward hook'):
groups.append(group)
group = []
if beam:
group.append((event, beam))
if beam['type'] in ('end', 'forward hook', 'backward hook'):
groups.append(group)
group = []
groups.append(group)
for group in groups:
for index, (event, beam) in enumerate(group):
if len(group) > 1:
beam['type'] = 'begin' if index == 0 else 'end' if index == len(group) - 1 else 'continue'
elif level > 1 and any(b['level'] == level - 1 for b in event['beams']):
beam['type'] = 'backward hook' if beam['type'] in ('end', 'backward hook') else 'forward hook'
else:
event['beams'].remove(beam)
for event in events:
if not event.get('beams'):
event.pop('beams', None)
def realize(measure: dict, bar: generator.rhythm.Bar, previous: dict | None) -> list[dict]:
item = context(measure, previous)
length = generator.model.fraction(measure['duration'])
events = []
for source in bar.events:
start, end = max(fractions.Fraction(), source.onset - item.offset), min(length, source.onset + source.duration - item.offset)
if start >= end:
continue
chord = active_chord(measure, start, previous) if source.sounding else None
continuation = source.continuation and source.onset - item.offset == start
preceding = previous
if continuation:
for change in measure['harmonies']:
if generator.model.fraction(change['onset']) >= start:
break
preceding = change
if not same_chord(chord, preceding):
chord = None
sounding_end = end
for change in measure['harmonies']:
time = generator.model.fraction(change['onset'])
if chord and start < time < sounding_end and not same_chord(chord, active_chord(measure, time, previous)):
sounding_end = time
written = {'type': source.written_type, 'dots': source.dots} if end - start == source.duration and sounding_end == end else None
event = generator.model.make_event(f'{measure["id"]}-rhythm-{len(events)}', start, sounding_end - start, 'note' if chord else 'rest', written=written)
if chord:
event['_harmony'] = chord
if continuation:
event['_rhythm_continuation'] = True
if source.beams and written and chord:
event['beams'] = [{'level': level, 'type': kind} for level, kind in source.beams]
events.append(event)
if sounding_end < end:
events.append(generator.model.make_event(f'{measure["id"]}-rhythm-{len(events)}', sounding_end, end - sounding_end, 'rest'))
normalize_beams(events)
return events
def connect_continuations(document: dict) -> bool:
previous = None
changed = False
for measure in document['staves'][0]['measures']:
if measure.get('section_label'):
previous = None
for voice in measure['voices']:
for note in voice['notes']:
continuation = note.pop('_rhythm_continuation', False)
if continuation and previous and previous['pitches'] and note['pitches']:
note['pitches'] = copy.deepcopy(previous['pitches'])
for index, tone in enumerate(note['pitches']):
source = previous['pitches'][index]
tone.update(id=f'{note["id"]}-p{index + 1}', duration=copy.deepcopy(note['duration']), written=copy.deepcopy(note['written']), tie_from=source['id'])
tone.pop('tie_to', None)
source['tie_to'] = tone['id']
changed = True
elif continuation:
note.update(kind='rest', pitches=[])
note.pop('beams', None)
changed = True
previous = note
return changed
generator/arrange/form.py
+8 -8
View File
@@ -1,17 +1,17 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Literal
import dataclasses
import typing
from generator.score import body_start_measure_index
import generator.score
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class ArrangementSection:
label: str
source_measure_index: int
measure_count: int
section_index: int
role: Literal["rhythm", "accompaniment", "melody", "vocal", "rest", "slash"]
role: typing.Literal["accompaniment", "melody", "vocal", "rest", "slash"]
def phrase_aware_edge_measure_count(body_measure_count: int) -> int:
if body_measure_count >= 8:
@@ -20,9 +20,9 @@ def phrase_aware_edge_measure_count(body_measure_count: int) -> int:
return 4
return body_measure_count
def section_role(part_type: str, section_index: int) -> Literal["rhythm", "accompaniment", "melody", "vocal", "rest", "slash"]:
def section_role(part_type: str, section_index: int) -> typing.Literal["accompaniment", "melody", "vocal", "rest", "slash"]:
if part_type == "acoustic":
return "slash" if section_index == 3 else "rhythm"
return "slash" if section_index == 3 else "accompaniment"
if part_type == "electric":
if section_index == 4:
return "slash"
@@ -36,7 +36,7 @@ def section_role(part_type: str, section_index: int) -> Literal["rhythm", "accom
def full_arrangement_sections(measures: list[dict], part_type: str) -> list[ArrangementSection]:
if not measures:
return []
body_start = body_start_measure_index(measures)
body_start = generator.score.body_start_measure_index(measures)
if body_start >= len(measures):
return []
body_measure_count = len(measures) - body_start
generator/arrange/generate.py
+77 -48
View File
@@ -1,104 +1,133 @@
from __future__ import annotations
import copy
from fractions import Fraction
from pathlib import Path
import fractions
import pathlib
from generator.arrange.form import full_arrangement_sections
from generator.guitar import place_guitar
from generator.parts import PART_TYPES, build_measure, prepare_document, propagated_chords, source_paths
from generator.model import fraction, key_label, make_event, meter_label, rational
from generator.musicxml import load_musicxml
from generator.score import copy_source_voices, make_arranged_document, reidentify, slugify, source_staff, write_json
import generator.accompaniment
import generator.arrange.form
import generator.guitar
import generator.model
import generator.musicxml
import generator.parts
import generator.rhythm
import generator.score
FINAL_SECTION_INDEX = 6
def arrange_document(document: dict, part_type: str) -> dict:
return _arrange_document(prepare_document(document), part_type)
def arrange_document(document: dict, part_type: str, *, rhythms: generator.rhythm.Library) -> dict:
document = generator.parts.prepare_document(document)
plans = plan_arrangement(document, rhythms) if part_type != 'vocal' else {}
return _arrange_document(document, part_type, plans)
def _arrange_document(document: dict, part_type: str) -> dict:
if part_type not in PART_TYPES:
def section_source(source: list[dict], final: bool) -> list[dict]:
result = copy.deepcopy(source)
for measure in result:
measure['harmonies'] = [h for h in measure['harmonies'] if not final or not h.get('chord_optional')]
for chord in measure['harmonies']:
chord.pop('chord_optional', None)
return result
def plan_arrangement(document: dict, rhythms: generator.rhythm.Library) -> dict[int, generator.accompaniment.Plan]:
source = generator.score.source_staff(document)['measures']
result = {}
for section in generator.arrange.form.full_arrangement_sections(source, 'acoustic'):
if section.role == 'slash':
continue
measures = section_source(source, section.section_index == FINAL_SECTION_INDEX)
previous = generator.accompaniment.carried_chords(measures)
start = section.source_measure_index
result[section.section_index] = generator.accompaniment.plan_section(measures[start:start + section.measure_count], rhythms, initial=previous[start], electric=section.section_index in (1, 5, 6))
return result
def _arrange_document(document: dict, part_type: str, plans: dict[int, generator.accompaniment.Plan]) -> dict:
if part_type not in generator.parts.PART_TYPES:
raise ValueError(f'unknown arrangement part: {part_type}')
staff = source_staff(document)
staff = generator.score.source_staff(document)
source = staff['measures']
sections = full_arrangement_sections(source, part_type)
sections = generator.arrange.form.full_arrangement_sections(source, part_type)
output, tuplets, octave_shifts = [], [], []
absolute = Fraction()
absolute = fractions.Fraction()
source_starts = []
source_time = Fraction()
source_time = fractions.Fraction()
for m in source:
source_starts.append(source_time)
source_time += fraction(m['duration'])
source_time += generator.model.fraction(m['duration'])
for section in sections:
previous = propagated_chords(source, include_optional=section.section_index != FINAL_SECTION_INDEX)
section_measures = section_source(source, section.section_index == FINAL_SECTION_INDEX)
previous = generator.accompaniment.carried_chords(section_measures)
segment = []
for si in range(section.source_measure_index, section.source_measure_index + section.measure_count):
m = copy.deepcopy(source[si])
m['harmonies'] = [h for h in m['harmonies'] if section.section_index != FINAL_SECTION_INDEX or not h.get('chord_optional')]
for h in m['harmonies']:
h.pop('chord_optional', None)
result = build_measure(m, f'measure-{len(output) + len(segment) + 1}', section.role, part_type, previous[si], section.label if not segment else '')
m = section_measures[si]
rhythm = None
if section.role == 'accompaniment':
plan = plans[section.section_index]
rhythm = (plan.acoustic if part_type == 'acoustic' else plan.electric)[len(segment)]
result = generator.parts.build_measure(m, f'measure-{len(output) + len(segment) + 1}', section.role, part_type, previous[si], section.label if not segment else '', rhythm)
segment.append(result)
identity_segment = segment
segment_length = sum((fraction(m['duration']) for m in segment), Fraction())
segment_length = sum((generator.model.fraction(m['duration']) for m in segment), fractions.Fraction())
if section.role in ('vocal', 'melody'):
start = source_starts[section.source_measure_index]
for shift in staff.get('octave_shifts', []):
low, high = max(start, fraction(shift['start'])), min(start + segment_length, fraction(shift['end']))
low, high = max(start, generator.model.fraction(shift['start'])), min(start + segment_length, generator.model.fraction(shift['end']))
if low < high:
octave_shifts.append({**shift, 'id': f'section-{section.section_index}-{shift["id"]}', 'start': rational(absolute + low - start), 'end': rational(absolute + high - start)})
octave_shifts.append({**shift, 'id': f'section-{section.section_index}-{shift["id"]}', 'start': generator.model.rational(absolute + low - start), 'end': generator.model.rational(absolute + high - start)})
if part_type == 'vocal' and section.section_index in (1, 5) and section.source_measure_index == 1 and source[0].get('implicit') and output:
pickup = build_measure(source[0], source[0]['id'], 'vocal', part_type, None)
pickup = generator.parts.build_measure(source[0], source[0]['id'], 'vocal', part_type, None)
destination = output[-1]
pad = fraction(destination['duration']) - fraction(pickup['duration'])
pad = generator.model.fraction(destination['duration']) - generator.model.fraction(pickup['duration'])
if pad < 0:
raise ValueError('pickup is longer than the preceding arrangement measure')
voices = copy_source_voices(pickup, strip_lyrics=False)
voices = generator.score.copy_source_voices(pickup, strip_lyrics=False)
for voice in voices:
for n in voice['notes']:
n['onset'] = rational(fraction(n['onset']) + pad)
n['onset'] = generator.model.rational(generator.model.fraction(n['onset']) + pad)
if pad:
voice['notes'].insert(0, make_event(f'pickup-rest-{voice["id"]}', Fraction(), pad, 'rest') | {'voice': voice['id']})
voice['notes'].insert(0, generator.model.make_event(f'pickup-rest-{voice["id"]}', fractions.Fraction(), pad, 'rest') | {'voice': voice['id']})
destination['voices'] = voices
identity_segment = [destination, *segment]
if pickup['harmonies']:
destination['harmonies'] = [h for h in destination['harmonies'] if fraction(h['onset']) < pad]
destination['harmonies'] = [h for h in destination['harmonies'] if generator.model.fraction(h['onset']) < pad]
for h in pickup['harmonies']:
h['onset'] = rational(fraction(h['onset']) + pad)
h['onset'] = generator.model.rational(generator.model.fraction(h['onset']) + pad)
destination['harmonies'].append(h)
state = {k: copy.deepcopy(pickup[k]) for k in ('key', 'clef', 'meter', 'transpose')}
if pad:
destination['attributes'].append({'onset': rational(pad), **state})
destination['attributes'].append({'onset': generator.model.rational(pad), **state})
else:
destination.update(state)
destination['key_signature'] = key_label(state['key'])
destination['time_signature'] = meter_label(state['meter'])
destination['key_signature'] = generator.model.key_label(state['key'])
destination['time_signature'] = generator.model.meter_label(state['meter'])
for change in pickup.get('attributes', []):
destination['attributes'].append({**change, 'onset': rational(fraction(change['onset']) + pad)})
destination['attributes'].append({**change, 'onset': generator.model.rational(generator.model.fraction(change['onset']) + pad)})
for shift in staff.get('octave_shifts', []):
low, high = fraction(shift['start']), min(fraction(pickup['duration']), fraction(shift['end']))
low, high = generator.model.fraction(shift['start']), min(generator.model.fraction(pickup['duration']), generator.model.fraction(shift['end']))
if low < high:
offset = absolute - fraction(pickup['duration'])
octave_shifts.append({**shift, 'id': f'pickup-{section.section_index}-{shift["id"]}', 'start': rational(offset + low), 'end': rational(offset + high)})
tuplets.extend(reidentify(identity_segment, staff.get('tuplets', [])))
offset = absolute - generator.model.fraction(pickup['duration'])
octave_shifts.append({**shift, 'id': f'pickup-{section.section_index}-{shift["id"]}', 'start': generator.model.rational(offset + low), 'end': generator.model.rational(offset + high)})
tuplets.extend(generator.score.reidentify(identity_segment, staff.get('tuplets', [])))
output.extend(segment)
absolute += segment_length
result = make_arranged_document(document, staff, staff_id='staff-arranged-' + part_type, staff_name=part_type.capitalize(), role='vocal' if part_type == 'vocal' else 'guitar', arrangement_part_type=part_type, measures=output, tuplets=tuplets, octave_shifts=octave_shifts)
result = generator.score.make_arranged_document(document, staff, staff_id='staff-arranged-' + part_type, staff_name=part_type.capitalize(), role='vocal' if part_type == 'vocal' else 'guitar', arrangement_part_type=part_type, measures=output, tuplets=tuplets, octave_shifts=octave_shifts)
result['metadata'].setdefault('tempo', 124)
if part_type != 'vocal':
place_guitar(result)
generator.guitar.place_guitar(result)
if generator.accompaniment.connect_continuations(result):
generator.guitar.place_guitar(result)
return result
def generate_performance_scores(source_dir: Path, output_dir: Path) -> int:
def generate_performance_scores(source_dir: pathlib.Path, output_dir: pathlib.Path, *, rhythms: generator.rhythm.Library) -> int:
count = 0
for path in source_paths(source_dir):
for path in generator.parts.source_paths(source_dir):
try:
document = prepare_document(load_musicxml(path))
document = generator.parts.prepare_document(generator.musicxml.load_musicxml(path))
plans = plan_arrangement(document, rhythms)
except ValueError as error:
raise ValueError(f'{path}: {error}') from error
for part in PART_TYPES:
for part in generator.parts.PART_TYPES:
try:
write_json(output_dir / f'{slugify(path.stem)}-{part}.json', _arrange_document(document, part))
generator.score.write_json(output_dir / f'{generator.score.slugify(path.stem)}-{part}.json', _arrange_document(document, part, plans))
except ValueError as error:
raise ValueError(f'{path}, part {part}: {error}') from error
count += 1
generator/catalog.py
+14 -9
View File
@@ -2,9 +2,10 @@ from __future__ import annotations
import argparse
import json
import pathlib
import re
import sqlite3
import tempfile
from pathlib import Path
APPLICATION_ID = 0x56414D50
COMMON_COLUMNS = """
@@ -15,10 +16,11 @@ COMMON_COLUMNS = """
source_path TEXT NOT NULL UNIQUE CHECK(length(source_path) > 0),
source_json TEXT NOT NULL CHECK(json_valid(source_json)),
measure_count INTEGER NOT NULL CHECK(measure_count > 0),
total_pages INTEGER NOT NULL CHECK(total_pages BETWEEN 1 AND 10000)
total_pages INTEGER NOT NULL CHECK(total_pages BETWEEN 1 AND 10000),
title_image TEXT NOT NULL CHECK(length(title_image) = 64 AND title_image NOT GLOB '*[^0-9a-f]*')
"""
def metadata(path: Path, root: Path) -> dict:
def metadata(path: pathlib.Path, root: pathlib.Path) -> dict:
if path.is_symlink() or not path.resolve().is_relative_to(root.resolve()):
raise ValueError('score escapes the asset root')
text = path.read_text(encoding='utf-8')
@@ -33,14 +35,17 @@ def metadata(path: Path, root: Path) -> dict:
if not isinstance(meta[name], str):
raise ValueError(f'{name} must be a string')
key = meta['key_signature']
title_image = presentation.get('title_image')
if not isinstance(title_image, str) or not re.fullmatch('[0-9a-f]{64}', title_image):
raise ValueError('presentation requires a generated Pico title image; regenerate assets')
if not isinstance(staff['name'], str):
raise ValueError('staff name must be a string')
relative = path.relative_to(root)
return dict(id=path.stem, title=meta['title'], composer=meta['composer'], key_signature=key,
source_path=relative.as_posix(), source_json=text,
measure_count=len(staff['measures']), total_pages=len(presentation['pages']), part=staff['name'])
measure_count=len(staff['measures']), total_pages=len(presentation['pages']), title_image=title_image, part=staff['name'])
def populate(connection: sqlite3.Connection, root: Path) -> None:
def populate(connection: sqlite3.Connection, root: pathlib.Path) -> None:
if not root.is_dir():
raise ValueError(f'{root}: score asset directory not found')
connection.executescript(f'''
@@ -81,13 +86,13 @@ def populate(connection: sqlite3.Connection, root: Path) -> None:
if connection.execute('PRAGMA integrity_check').fetchall() != [('ok',)]:
raise ValueError('catalog integrity check failed')
def build_catalog(root: Path, destination: Path | None = None) -> Path:
def build_catalog(root: pathlib.Path, destination: pathlib.Path | None = None) -> pathlib.Path:
if not root.is_dir():
raise ValueError(f'{root}: score asset directory not found')
destination = destination or root / 'catalog.db'
destination.parent.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix='.vamp-catalog-', dir=destination.parent) as temporary:
staged = Path(temporary) / 'catalog.db'
staged = pathlib.Path(temporary) / 'catalog.db'
connection = sqlite3.connect(staged)
try:
populate(connection, root)
@@ -98,7 +103,7 @@ def build_catalog(root: Path, destination: Path | None = None) -> Path:
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Compile generated score bundles into a read-only runtime catalog.')
parser.add_argument('assets', type=Path)
parser.add_argument('destination', type=Path, nargs='?')
parser.add_argument('assets', type=pathlib.Path)
parser.add_argument('destination', type=pathlib.Path, nargs='?')
arguments = parser.parse_args()
build_catalog(arguments.assets, arguments.destination)
generator/engraving/__init__.py
+1 -108
View File
@@ -1,108 +1 @@
from __future__ import annotations
from concurrent.futures import ProcessPoolExecutor
import hashlib
import json
import multiprocessing
from pathlib import Path
import shutil
from PIL import Image, ImageChops
from generator.engraving.fonts import Fonts
from generator.engraving.cache import renderer_fingerprint, renderer_namespace, restore_pages, score_fingerprint, store_pages
from generator.engraving.layout import layout_score
from generator.engraving.raster import rasterize_page
from generator.engraving.types import EngravingError, Style
from generator.model import validate_document
ENGINE_NAME = 'Vamp Python'
PAGE_WIDTH = 976
PAGE_HEIGHT = 484
def check_page(path: Path) -> None:
with Image.open(path) as image:
image.load()
if image.format != 'PNG' or image.size != (PAGE_WIDTH, PAGE_HEIGHT):
raise EngravingError(f'{path.name}: expected a {PAGE_WIDTH}x{PAGE_HEIGHT} PNG page')
white = Image.new('RGBA', image.size, 'white')
white.alpha_composite(image.convert('RGBA'))
ink = ImageChops.difference(white.convert('RGB'), Image.new('RGB', image.size, 'white')).convert('L').point(lambda x: 255 if x > 32 else 0)
bounds = ink.getbbox()
if not bounds:
raise EngravingError(f'{path.name}: engraver produced a blank page')
if bounds[0] < 4 or bounds[1] < 4 or bounds[2] > PAGE_WIDTH - 4 or bounds[3] > PAGE_HEIGHT - 4:
raise EngravingError(f'{path.name}: notation reaches the page edge; refusing potentially clipped output')
def engrave_scores(paths: list[Path], root: Path, fonts: Path, cache_dir: Path | None, progress: bool, worker: int, renderer: str) -> tuple[int, int, int, int]:
family = Fonts(fonts)
style = Style()
fingerprint = family.fingerprint(style)
reused, missing, rejected = 0, 0, 0
for index, path in enumerate(paths, 1):
document = json.loads(path.read_text(encoding='utf-8'))
try:
validate_document(document)
destination = path.with_suffix('.pages')
if destination.exists():
shutil.rmtree(destination)
entry = cache_dir / score_fingerprint(document, renderer) if cache_dir else None
present = entry is not None and (entry.exists() or entry.is_symlink())
pages = restore_pages(entry, destination) if entry else None
if pages is None:
if entry:
rejected += int(present)
missing += int(not present)
if destination.exists():
shutil.rmtree(destination)
destination.mkdir()
layout = layout_score(document, style, family)
pages = []
for page in layout.pages:
name = f'{page.number:04}.png'
target = destination / name
rasterize_page(page, family).save(target, format='PNG')
check_page(target)
pages.append({'path': name, 'sha256': hashlib.sha256(target.read_bytes()).hexdigest()})
if entry:
store_pages(entry, destination, pages)
else:
reused += 1
entries = [{**page, 'path': destination.name + '/' + page['path']} for page in pages]
music = {k: v for k, v in document.items() if k != 'playback'}
bundle = {'music': music, 'presentation': {'engine': ENGINE_NAME, 'style_sha256': fingerprint, 'width': PAGE_WIDTH, 'height': PAGE_HEIGHT, 'pages': entries}}
if 'playback' in document:
bundle['playback'] = document['playback']
path.write_text(json.dumps(bundle, indent=2, ensure_ascii=False) + '\n', encoding='utf-8')
if progress and (index % 50 == 0 or index == len(paths)):
print(f'Engraving worker {worker}: {index}/{len(paths)} scores, {reused} reused, {missing} missing, {rejected} rejected', flush=True)
except ValueError as error:
raise EngravingError(f'{path.relative_to(root)}: {error}') from error
return len(paths), reused, missing, rejected
def engrave_tree(root: Path, fonts: Path, *, cache_dir: Path | None = None, jobs: int = 1, progress: bool = False) -> None:
if type(jobs) is not int or not 1 <= jobs <= 16:
raise ValueError('engraving jobs must be between 1 and 16')
if cache_dir and (cache_dir.resolve().is_relative_to(root.resolve()) or root.resolve().is_relative_to(cache_dir.resolve())):
raise ValueError('engraving cache and generated assets must not overlap')
paths = sorted(root.rglob('*.json'))
if not paths:
return
renderer = renderer_fingerprint(Fonts(fonts), Style()) if cache_dir else ''
if cache_dir:
cache_dir = renderer_namespace(cache_dir.resolve(), renderer)
if progress:
state = 'warm namespace' if cache_dir.is_dir() else 'cold namespace'
print(f'Engraving cache enabled: {cache_dir} ({state}; renderer {renderer})', flush=True)
elif progress:
print('Engraving cache disabled', flush=True)
jobs = min(jobs, len(paths))
if jobs == 1:
results = [engrave_scores(paths, root, fonts, cache_dir, progress, 1, renderer)]
else:
with ProcessPoolExecutor(max_workers=jobs, mp_context=multiprocessing.get_context('spawn')) as pool:
futures = [pool.submit(engrave_scores, paths[index::jobs], root, fonts, cache_dir, progress, index + 1, renderer) for index in range(jobs)]
results = [future.result() for future in futures]
if progress:
total, reused, missing, rejected = (sum(row[index] for row in results) for index in range(4))
print(f'Engraving complete: {total} scores, {reused} reused, {missing} missing, {rejected} rejected', flush=True)
"""Score engraving implementation; import the owning modules explicitly."""
generator/engraving/attributes.py
+25 -26
View File
@@ -1,13 +1,13 @@
from __future__ import annotations
from fractions import Fraction
import fractions
from generator.model import LETTERS, TYPE_LENGTHS, alteration, fraction, key_from_fifths
from generator.engraving.fonts import Fonts
from generator.engraving.notation import ScoreIndex, accidental
from generator.engraving.types import Primitive, bounds, line
import generator.engraving.fonts
import generator.engraving.notation
import generator.engraving.types
import generator.model
def staff_attributes(index: ScoreIndex, state: dict, time: Fraction, fonts: Fonts, space: float, source: str, fields: tuple[str, ...]) -> list[Primitive]:
def staff_attributes(index: generator.engraving.notation.ScoreIndex, state: dict, time: fractions.Fraction, fonts: generator.engraving.fonts.Fonts, space: float, source: str, fields: tuple[str, ...]) -> list[generator.engraving.types.Primitive]:
result = []
x = 0
references = (source,)
@@ -22,18 +22,18 @@ def staff_attributes(index: ScoreIndex, state: dict, time: Fraction, fonts: Font
if octave := clef.get('octave_change', 0):
label = str(abs(octave) * 7 + 1)
result.append(fonts.text(label, (glyph.box.left + glyph.box.right) / 2, glyph.box.top - 2 if octave > 0 else glyph.box.bottom + 14, 14, 'clef-octave', references, True))
x = bounds(result).right + space
x = generator.engraving.types.bounds(result).right + space
if 'key' in fields and clef['sign'] != 'percussion':
key = state['key']
steps = key.get('steps')
if steps is None:
order = 'fcgdaeb' if key.get('fifths', 0) >= 0 else 'beadgcf'
steps = [{'letter': c, 'alter': key['alterations'][LETTERS.index(c)]} for c in order if alteration(key['alterations'][LETTERS.index(c)])]
steps = [{'letter': c, 'alter': key['alterations'][generator.model.LETTERS.index(c)]} for c in order if generator.model.alteration(key['alterations'][generator.model.LETTERS.index(c)])]
cancellation = key.get('cancel', 0)
if cancellation:
order = 'fcgdaeb' if cancellation > 0 else 'beadgcf'
old = key_from_fifths(cancellation)['alterations']
steps = [{'letter': c, 'alter': 0} for c in order if alteration(old[LETTERS.index(c)]) and alteration(old[LETTERS.index(c)]) != alteration(key['alterations'][LETTERS.index(c)])] + steps
old = generator.model.key_from_fifths(cancellation)['alterations']
steps = [{'letter': c, 'alter': 0} for c in order if generator.model.alteration(old[generator.model.LETTERS.index(c)]) and generator.model.alteration(old[generator.model.LETTERS.index(c)]) != generator.model.alteration(key['alterations'][generator.model.LETTERS.index(c)])] + steps
for step in steps:
pitch = {'letter': step['letter'], 'octave': step.get('octave', 5), 'id': source}
position = index.pitch_position(pitch, time)
@@ -42,9 +42,9 @@ def staff_attributes(index: ScoreIndex, state: dict, time: Fraction, fonts: Font
position -= 7
while position < 2:
position += 7
items = accidental(fonts, alteration(step['alter']), {'name': step.get('accidental', '')}, space, references)
items = generator.engraving.notation.accidental(fonts, generator.model.alteration(step['alter']), {'name': step.get('accidental', '')}, space, references)
result.extend(p.moved(x, -position * space / 2) for p in items)
x += bounds(items).right - bounds(items).left + 4
x += generator.engraving.types.bounds(items).right - generator.engraving.types.bounds(items).left + 4
if steps:
x += space / 2
if 'meter' in fields:
@@ -67,8 +67,8 @@ def staff_attributes(index: ScoreIndex, state: dict, time: Fraction, fonts: Font
items = []
cursor = 0
if y == -space and meter.get('symbol') in ('note', 'dotted-note'):
unit = Fraction(4, group['unit'])
kind = next((name for name, length in TYPE_LENGTHS.items() if length == unit), None)
unit = fractions.Fraction(4, group['unit'])
kind = next((name for name, length in generator.model.TYPE_LENGTHS.items() if length == unit), None)
glyph_name = {'whole': 'noteWhole', 'half': 'noteHalfUp', 'quarter': 'noteQuarterUp', 'eighth': 'note8thUp'}.get(kind, 'note' + str(kind) + 'Up')
if kind and glyph_name in fonts.names:
glyph = fonts.glyph(glyph_name, 0, y + space, space * 0.55, 'meter', references)
@@ -82,36 +82,35 @@ def staff_attributes(index: ScoreIndex, state: dict, time: Fraction, fonts: Font
items.append(glyph)
cursor = glyph.box.right + 1
groups.append(items)
width = max(bounds(items).right for items in groups)
width = max(generator.engraving.types.bounds(items).right for items in groups)
for items in groups:
result.extend(p.moved(x + (width - bounds(items).right) / 2, 0) for p in items)
result.extend(p.moved(x + (width - generator.engraving.types.bounds(items).right) / 2, 0) for p in items)
x += width + 6
if 'staff_details' in fields and state['staff_details'].get('capo'):
result.append(fonts.text(f'Capo {state["staff_details"]["capo"]}', x, -space * (lines + 1), 16, 'staff-details', references))
return result
def barline(style: str, x: float, top: float, source: str) -> list[Primitive]:
def barline(style: str, x: float, top: float, source: str) -> list[generator.engraving.types.Primitive]:
if style == 'none':
return []
references = (source,)
if style in ('dashed', 'dotted'):
dash, gap = (5, 4) if style == 'dashed' else (1, 4)
return [line('barline', references, ((x, y), (x, min(y + dash, 0)))) for y in range(round(top), 0, dash + gap)]
return [generator.engraving.types.line('barline', references, ((x, y), (x, min(y + dash, 0)))) for y in range(round(top), 0, dash + gap)]
if style in ('tick', 'short'):
return [line('barline', references, ((x, top - 4 if style == 'tick' else top * 0.75), (x, top + 5 if style == 'tick' else top * 0.25)))]
return [generator.engraving.types.line('barline', references, ((x, top - 4 if style == 'tick' else top * 0.75), (x, top + 5 if style == 'tick' else top * 0.25)))]
weights = {'regular': [1.4], 'heavy': [4], 'light-light': [1.4, 1.4], 'light-heavy': [1.4, 4], 'heavy-light': [4, 1.4], 'heavy-heavy': [4, 4]}[style]
return [line('barline', references, ((x - (len(weights) - 1 - i) * 5, top), (x - (len(weights) - 1 - i) * 5, 0)), width) for i, width in enumerate(weights)]
return [generator.engraving.types.line('barline', references, ((x - (len(weights) - 1 - i) * 5, top), (x - (len(weights) - 1 - i) * 5, 0)), width) for i, width in enumerate(weights)]
def tempo(value: dict, fonts: Fonts, x: float, y: float, space: float, source: str) -> list[Primitive]:
from generator.engraving.notation import flags, head_glyph
def tempo(value: dict, fonts: generator.engraving.fonts.Fonts, x: float, y: float, space: float, source: str) -> list[generator.engraving.types.Primitive]:
written = value['written']
references = (source,)
glyph = fonts.glyph(head_glyph(written, None), x, y, space * 0.7, 'tempo', references)
glyph = fonts.glyph(generator.engraving.notation.head_glyph(written, None), x, y, space * 0.7, 'tempo', references)
items = [glyph]
stem_x = glyph.box.right - 0.5
if written['type'] not in ('whole', 'breve', 'long', 'maxima'):
items.append(line('tempo', references, ((stem_x, y), (stem_x, y - 23)), 1))
if flags(written):
items.append(generator.engraving.types.line('tempo', references, ((stem_x, y), (stem_x, y - 23)), 1))
if generator.engraving.notation.flags(written):
name = 'flag' + ('8th' if written['type'] == 'eighth' else written['type']) + 'Up'
items.append(fonts.glyph(name, stem_x, y - 23, space * 0.7, 'tempo', references))
cursor = max(p.box.right for p in items) + 3
@@ -119,5 +118,5 @@ def tempo(value: dict, fonts: Fonts, x: float, y: float, space: float, source: s
dot = fonts.glyph('augmentationDot', cursor, y - 3, space * 0.7, 'tempo', references)
items.append(dot)
cursor = dot.box.right + 3
items.append(fonts.text(' = ' + str(fraction(value['bpm'])), cursor, y + 3, 16, 'tempo', references))
items.append(fonts.text(' = ' + str(generator.model.fraction(value['bpm'])), cursor, y + 3, 16, 'tempo', references))
return items
generator/engraving/audit.py
+45 -41
View File
@@ -1,48 +1,52 @@
from __future__ import annotations
from fractions import Fraction
import fractions
from generator.model import TYPE_LENGTHS, fraction
from generator import harmony
from generator.engraving.fonts import Fonts
from generator.engraving.notation import ScoreIndex, head_glyph
from generator.engraving.types import EngravingError, LayoutDocument
import generator.engraving.fonts
import generator.engraving.notation
import generator.engraving.types
import generator.harmony
import generator.model
def audit_layout(document: dict, layout: LayoutDocument, fonts: Fonts) -> None:
index = ScoreIndex(document)
def audit_primitives(primitives, width, height, fonts):
for item in primitives:
if not item.box.finite():
raise generator.engraving.types.EngravingError(f'{item.sources}: non-finite or inverted primitive geometry')
if item.box.left < 4 or item.box.top < 4 or item.box.right > width - 4 or item.box.bottom > height - 4:
raise generator.engraving.types.EngravingError(f'{item.sources}, {item.role}: primitive outside page bounds')
if item.kind == 'glyph':
actual = fonts.glyph_box(item.value, item.size).moved(*item.points[0])
if any(abs(a - b) > 0.001 for a, b in zip((actual.left, actual.top, actual.right, actual.bottom), (item.box.left, item.box.top, item.box.right, item.box.bottom))):
raise generator.engraving.types.EngravingError(f'{item.sources}: glyph bounds disagree with its actual outline')
def audit_layout(document: dict, layout: generator.engraving.types.LayoutDocument, fonts: generator.engraving.fonts.Fonts) -> None:
index = generator.engraving.notation.ScoreIndex(document)
fragments, primitives = {}, {}
previous_end = Fraction()
previous_end = fractions.Fraction()
for page in layout.pages:
audit_primitives(page.primitives, page.width, page.height, fonts)
for item in page.primitives:
if not item.box.finite():
raise EngravingError(f'{item.sources}: non-finite or inverted primitive geometry')
if item.box.left < 4 or item.box.top < 4 or item.box.right > page.width - 4 or item.box.bottom > page.height - 4:
raise EngravingError(f'{item.sources}, {item.role}: primitive outside page bounds')
if item.kind == 'glyph':
actual = fonts.glyph_box(item.value, item.size).moved(*item.points[0])
if any(abs(a - b) > 0.001 for a, b in zip((actual.left, actual.top, actual.right, actual.bottom), (item.box.left, item.box.top, item.box.right, item.box.bottom))):
raise EngravingError(f'{item.sources}: glyph bounds disagree with its actual outline')
for source in item.sources:
primitives.setdefault(source, []).append(item)
for system in page.systems:
if system.start != previous_end or system.end <= system.start:
raise EngravingError('layout systems do not partition the score timeline')
raise generator.engraving.types.EngravingError('layout systems do not partition the score timeline')
previous_end = system.end
baseline = max(p.points[0][1] for p in system.primitives if p.role == 'staff')
for anchor in system.anchors:
if anchor.fragment.kind == 'tone':
expected = index.pitch_position(index.tones[anchor.source][1], anchor.time)
if abs(anchor.y - (baseline - expected * layout.style.space / 2)) > 0.01:
raise EngravingError(f'{anchor.source}: displayed staff position changed pitch')
raise generator.engraving.types.EngravingError(f'{anchor.source}: displayed staff position changed pitch')
role = 'head' if anchor.fragment.kind == 'tone' else anchor.fragment.kind
candidates = [p for p in system.primitives if p.role == role and anchor.source in p.sources and abs(p.points[0][0] - anchor.x) < 0.01]
if not candidates:
raise EngravingError(f'{anchor.source}: visible anchor has no corresponding drawing primitive')
raise generator.engraving.types.EngravingError(f'{anchor.source}: visible anchor has no corresponding drawing primitive')
if role == 'head' and candidates[0].kind == 'glyph':
event, tone = index.tones[anchor.source]
expected_glyph = head_glyph({'type': anchor.fragment.written_type}, tone.get('notehead', event.value.get('notehead')))
expected_glyph = generator.engraving.notation.head_glyph({'type': anchor.fragment.written_type}, tone.get('notehead', event.value.get('notehead')))
if candidates[0].value != expected_glyph:
raise EngravingError(f'{anchor.source}: notehead changed written duration or shape')
raise generator.engraving.types.EngravingError(f'{anchor.source}: notehead changed written duration or shape')
if index.staff.get('tablature') and anchor.fragment.kind == 'tone':
tone = index.tones[anchor.source][1]
tab = tone['tablature']
@@ -51,7 +55,7 @@ def audit_layout(document: dict, layout: LayoutDocument, fonts: Fonts) -> None:
expected_x = min(a.x for a in system.anchors if a.event == anchor.event and a.time == anchor.time)
tabs = [p for p in system.primitives if p.role == 'tab' and anchor.source in p.sources]
if len(tabs) != 1 or tabs[0].value != str(tab['fret']) or abs(tabs[0].points[0][1] - expected_y) > 0.01 or abs(tabs[0].points[0][0] - expected_x) > 0.01:
raise EngravingError(f'{anchor.source}: tablature changed position or rhythmic alignment')
raise generator.engraving.types.EngravingError(f'{anchor.source}: tablature changed position or rhythmic alignment')
for fragment in system.fragments:
fragments.setdefault(fragment.source, []).append(fragment)
# Staff, ledger, stem, beam and tie contacts are intentional. Distinct
@@ -64,57 +68,57 @@ def audit_layout(document: dict, layout: LayoutDocument, fonts: Fonts) -> None:
if item.role != other.role and {item.role, other.role} - {'head', 'tab', 'accidental'}:
continue
if item.box.overlaps(other.box, 0.1):
raise EngravingError(f'{item.sources} and {other.sources}: colliding {item.role}/{other.role}')
raise generator.engraving.types.EngravingError(f'{item.sources} and {other.sources}: colliding {item.role}/{other.role}')
if previous_end != index.end:
raise EngravingError('layout omits the end of the score')
raise generator.engraving.types.EngravingError('layout omits the end of the score')
for event in index.events:
note = event.value
for source in note['pitches'] or [note]:
identifier = source['id']
pieces = sorted(fragments.get(identifier, []), key=lambda f: f.start)
if not pieces:
raise EngravingError(f'{identifier}: missing notation fragments')
raise generator.engraving.types.EngravingError(f'{identifier}: missing notation fragments')
cursor = event.start
for fragment in pieces:
expected_visible = not note.get('hidden') and note['kind'] != 'spacer' and source.get('print', {}).get('print-object') != 'no'
if isinstance(source.get('notehead'), dict) and source['notehead'].get('name') == 'none':
expected_visible = False
if fragment.visible != expected_visible:
raise EngravingError(f'{identifier}: notation visibility changed')
raise generator.engraving.types.EngravingError(f'{identifier}: notation visibility changed')
if fragment.start != cursor:
raise EngravingError(f'{identifier}: fragments overlap or omit metric time')
raise generator.engraving.types.EngravingError(f'{identifier}: fragments overlap or omit metric time')
cursor = fragment.end
expected = TYPE_LENGTHS[fragment.written_type] * (2 - Fraction(1, 2 ** fragment.dots)) * fragment.ratio
expected = generator.model.TYPE_LENGTHS[fragment.written_type] * (2 - fractions.Fraction(1, 2 ** fragment.dots)) * fragment.ratio
if not note.get('grace') and not note.get('measure_rest') and fragment.length() != expected:
raise EngravingError(f'{identifier}: written fragment duration is inexact')
raise generator.engraving.types.EngravingError(f'{identifier}: written fragment duration is inexact')
if fragment.visible and not any(p.role in ('head', 'tab', 'rest', 'slash') for p in primitives.get(identifier, [])):
raise EngravingError(f'{identifier}: visible notation has no drawing primitive')
if cursor != event.start + fraction(source['duration']):
raise EngravingError(f'{identifier}: rendered duration changed')
raise generator.engraving.types.EngravingError(f'{identifier}: visible notation has no drawing primitive')
if cursor != event.start + generator.model.fraction(source['duration']):
raise generator.engraving.types.EngravingError(f'{identifier}: rendered duration changed')
if any(f.visible for f in pieces) and identifier in index.tones and index.accidentals[identifier]:
if not any(p.role == 'accidental' for p in primitives.get(identifier, [])):
raise EngravingError(f'{identifier}: required accidental was omitted')
raise generator.engraving.types.EngravingError(f'{identifier}: required accidental was omitted')
if identifier in index.tones and source.get('tie_to') and any(f.visible for f in pieces):
if not any(p.role == 'tie' for p in primitives.get(identifier, [])):
raise EngravingError(f'{identifier}: tie was omitted')
raise generator.engraving.types.EngravingError(f'{identifier}: tie was omitted')
if not note.get('hidden'):
for verse in note.get('lyrics', []):
text = ''.join(s['text'] for s in verse['syllables'] if s['kind'] in ('text', 'elision'))
identifier = note['id'] + ':lyric:' + verse['number']
if text and not any(p.role == 'lyric' and p.value == text for p in primitives.get(identifier, [])):
raise EngravingError(f'{identifier}: lyric text was omitted or changed')
raise generator.engraving.types.EngravingError(f'{identifier}: lyric text was omitted or changed')
for _, _, measure in index.measures:
for value in measure['harmonies']:
label = harmony.symbol(value)
label = generator.harmony.symbol(value)
if not any(p.role == 'harmony' and p.value == label for p in primitives.get(value['id'], [])):
raise EngravingError(f'{value["id"]}: harmony was omitted')
raise generator.engraving.types.EngravingError(f'{value["id"]}: harmony was omitted')
for i, _ in enumerate(measure.get('tempos', [])):
if not any(p.role == 'tempo' for p in primitives.get(measure['id'] + f':tempo:{i}', [])):
raise EngravingError(f'{measure["id"]}: tempo was omitted')
raise generator.engraving.types.EngravingError(f'{measure["id"]}: tempo was omitted')
for span in index.staff.get('tuplets', []):
if span.get('show_number', 'actual') != 'none' or span.get('bracket', 'auto') != 'no':
if not any(p.role.startswith('tuplet-') for p in primitives.get(span['id'], [])):
raise EngravingError(f'{span["id"]}: tuplet was omitted')
raise generator.engraving.types.EngravingError(f'{span["id"]}: tuplet was omitted')
for span in index.staff.get('octave_shifts', []):
if not any(p.role == 'octave-shift' for p in primitives.get(span['id'], [])):
raise EngravingError(f'{span["id"]}: octave shift was omitted')
raise generator.engraving.types.EngravingError(f'{span["id"]}: octave shift was omitted')
generator/engraving/cache.py
+79 -16
View File
@@ -1,19 +1,26 @@
from __future__ import annotations
import dataclasses
import hashlib
import importlib.metadata
import io
import json
from pathlib import Path
import shutil
import pathlib
import platform
import re
import shutil
import tempfile
from PIL import features
import PIL.Image
import PIL.ImageChops
import PIL.features
import generator.engraving.validation
def renderer_identity(fonts, style) -> dict:
root = Path(__file__).resolve().parents[1]
root = pathlib.Path(__file__).resolve().parents[1]
versions = {name: importlib.metadata.version(name) for name in ('Pillow', 'fonttools', 'aggdraw')}
versions.update({name: features.version(name) for name in features.get_supported()})
versions.update({name: PIL.features.version(name) for name in PIL.features.get_supported()})
sources = [*root.joinpath('engraving').glob('*.py'), root / 'model.py', root / 'harmony.py', root / 'requirements.txt']
return {'fonts_style': fonts.fingerprint(style), 'python': {'implementation': platform.python_implementation(), 'version': platform.python_version()}, 'dependencies': versions,
'sources': {path.relative_to(root).as_posix(): hashlib.sha256(path.read_bytes()).hexdigest() for path in sorted(sources)}}
@@ -21,7 +28,7 @@ def renderer_identity(fonts, style) -> dict:
def renderer_fingerprint(fonts, style) -> str:
return hashlib.sha256(json.dumps(renderer_identity(fonts, style), sort_keys=True).encode()).hexdigest()
def renderer_namespace(cache: Path, renderer: str) -> Path:
def renderer_namespace(cache: pathlib.Path, renderer: str) -> pathlib.Path:
namespace = cache / renderer
if namespace.parent.is_symlink() or namespace.is_symlink():
raise ValueError('engraving cache namespaces must not be symlinks')
@@ -31,10 +38,19 @@ def score_fingerprint(document: dict, renderer: str) -> str:
music = {key: value for key, value in document.items() if key != 'playback'}
return hashlib.sha256(json.dumps([renderer, music], sort_keys=True, ensure_ascii=False).encode()).hexdigest()
def restore_pages(entry: Path, destination: Path) -> list[dict] | None:
from generator.engraving import check_page
def notation_fingerprint(document: dict, renderer: str) -> str:
return hashlib.sha256(json.dumps([renderer, document['staves']], sort_keys=True, ensure_ascii=False).encode()).hexdigest()
def band_fingerprint(notation: str, page, band) -> str:
context = [notation, page.width, page.height, page.supersampling, dataclasses.asdict(band)]
return hashlib.sha256(json.dumps(context, sort_keys=True, ensure_ascii=False).encode()).hexdigest()
def has_symlink(path: pathlib.Path) -> bool:
return any(parent.is_symlink() for parent in (path, *path.parents))
def restore_pages(entry: pathlib.Path, destination: pathlib.Path, validated_pages: set[str] | None = None) -> list[dict] | None:
try:
if entry.is_symlink():
if has_symlink(entry) or has_symlink(destination):
return None
manifest = entry / 'pages.json'
if manifest.is_symlink():
@@ -44,27 +60,29 @@ def restore_pages(entry: Path, destination: Path) -> list[dict] | None:
return None
destination.mkdir(parents=True, exist_ok=True)
for index, page in enumerate(pages, 1):
if not isinstance(page, dict) or set(page) != {'path', 'sha256'} or page['path'] != f'{index:04}.png':
if not isinstance(page, dict) or set(page) != {'path', 'sha256'} or page['path'] != f'{index:04}.png' or not isinstance(page['sha256'], str) or not re.fullmatch('[0-9a-f]{64}', page['sha256']):
return None
path = entry / page['path']
if path.is_symlink():
if path.is_symlink() or not path.is_file():
return None
target = destination / page['path']
shutil.copyfile(path, target)
if hashlib.sha256(target.read_bytes()).hexdigest() != page['sha256']:
if target.is_symlink():
return None
check_page(target)
shutil.copyfile(path, target)
generator.engraving.validation.check_page(target, validated_pages, expected_sha256=page['sha256'])
return pages
except (OSError, ValueError, TypeError):
return None
def store_pages(entry: Path, destination: Path, pages: list[dict]) -> None:
def store_pages(entry: pathlib.Path, destination: pathlib.Path, pages: list[dict]) -> None:
if has_symlink(entry.parent):
raise ValueError('engraving cache directories must not be symlinks')
entry.parent.mkdir(parents=True, exist_ok=True)
if entry.is_symlink() or entry.is_file():
entry.unlink(missing_ok=True)
entry.mkdir(exist_ok=True)
with tempfile.TemporaryDirectory(prefix='.engraving-', dir=entry.parent) as temporary:
staged = Path(temporary)
staged = pathlib.Path(temporary)
for page in pages:
shutil.copyfile(destination / page['path'], staged / page['path'])
(staged / 'pages.json').write_text(json.dumps(pages), encoding='utf-8')
@@ -74,3 +92,48 @@ def store_pages(entry: Path, destination: Path, pages: list[dict]) -> None:
if target.is_dir() and not target.is_symlink():
shutil.rmtree(target, ignore_errors=True)
(staged / name).replace(target)
def restore_band(entry: pathlib.Path, key: str, width: int, height: int) -> PIL.Image.Image | None:
try:
manifest, asset = entry / 'band.json', entry / 'band.png'
if has_symlink(entry) or manifest.is_symlink() or asset.is_symlink():
return None
metadata = json.loads(manifest.read_text(encoding='utf-8'))
if not isinstance(metadata, dict) or set(metadata) != {'context', 'width', 'height', 'sha256'}:
return None
if metadata['context'] != key or type(metadata['width']) is not int or type(metadata['height']) is not int or (metadata['width'], metadata['height']) != (width, height):
return None
digest = metadata['sha256']
if not isinstance(digest, str) or not re.fullmatch('[0-9a-f]{64}', digest):
return None
data = asset.read_bytes()
if hashlib.sha256(data).hexdigest() != digest:
return None
with PIL.Image.open(io.BytesIO(data)) as image:
if image.format != 'PNG' or image.size != (width, height) or image.mode != 'RGB':
return None
image.load()
ink = PIL.ImageChops.difference(image, PIL.Image.new('RGB', image.size, 'white')).getbbox()
if not ink or ink[0] < 3 or ink[1] < 3 or ink[2] > width - 3 or ink[3] > height - 3:
return None
return image.copy()
except (OSError, ValueError, TypeError):
return None
def store_band(entry: pathlib.Path, key: str, image: PIL.Image.Image) -> None:
if has_symlink(entry.parent):
raise ValueError('engraving cache directories must not be symlinks')
entry.parent.mkdir(parents=True, exist_ok=True)
if entry.is_symlink() or entry.is_file():
entry.unlink()
entry.mkdir(exist_ok=True)
with tempfile.TemporaryDirectory(prefix='.engraving-', dir=entry.parent) as temporary:
staged = pathlib.Path(temporary)
image.save(staged / 'band.png', format='PNG')
metadata = {'context': key, 'width': image.width, 'height': image.height, 'sha256': hashlib.sha256((staged / 'band.png').read_bytes()).hexdigest()}
(staged / 'band.json').write_text(json.dumps(metadata), encoding='utf-8')
for name in ('band.png', 'band.json'):
target = entry / name
if target.is_dir() and not target.is_symlink():
shutil.rmtree(target)
(staged / name).replace(target)
generator/engraving/fonts.py
+26 -26
View File
@@ -1,27 +1,27 @@
from __future__ import annotations
from dataclasses import asdict
import dataclasses
import hashlib
import json
from pathlib import Path
import pathlib
from fontTools.pens.boundsPen import BoundsPen
from fontTools.pens.recordingPen import DecomposingRecordingPen
from fontTools.ttLib import TTFont
from PIL import ImageFont
import PIL.ImageFont
import fontTools.pens.boundsPen
import fontTools.pens.recordingPen
import fontTools.ttLib
from generator.engraving.types import Box, EngravingError, Primitive, Style
import generator.engraving.types
class Fonts:
def __init__(self, root: Path):
def __init__(self, root: pathlib.Path):
self.root = root
try:
self.music = TTFont(root / 'Petaluma.otf')
self.script = TTFont(root / 'PetalumaScript.otf')
self.music = fontTools.ttLib.TTFont(root / 'Petaluma.otf')
self.script = fontTools.ttLib.TTFont(root / 'PetalumaScript.otf')
self.metadata = json.loads((root / 'petaluma_metadata.json').read_text())
self.names = json.loads((root / 'smufl_glyphnames.json').read_text())
except (OSError, ValueError) as error:
raise EngravingError(f'cannot load bundled engraving fonts: {error}') from error
raise generator.engraving.types.EngravingError(f'cannot load bundled engraving fonts: {error}') from error
self.names.update(self.metadata.get('optionalGlyphs', {}))
self.cmap = self.music.getBestCmap()
self.text_cmap = self.script.getBestCmap()
@@ -30,37 +30,37 @@ class Fonts:
self.metrics = {}
self.outlines = {}
self.text_fonts = {}
def fingerprint(self, style: Style) -> str:
digest = hashlib.sha256(json.dumps(asdict(style), sort_keys=True).encode())
def fingerprint(self, style: generator.engraving.types.Style) -> str:
digest = hashlib.sha256(json.dumps(dataclasses.asdict(style), sort_keys=True).encode())
for name in ('Petaluma.otf', 'PetalumaScript.otf', 'petaluma_metadata.json', 'smufl_glyphnames.json'):
digest.update(name.encode())
digest.update((self.root / name).read_bytes())
return digest.hexdigest()
def glyph_name(self, name: str) -> str:
if name not in self.names:
raise EngravingError(f'unknown SMuFL glyph {name!r}')
raise generator.engraving.types.EngravingError(f'unknown SMuFL glyph {name!r}')
code = int(self.names[name]['codepoint'].removeprefix('U+'), 16)
if code not in self.cmap:
raise EngravingError(f'Petaluma is missing requested SMuFL glyph {name!r}')
raise generator.engraving.types.EngravingError(f'Petaluma is missing requested SMuFL glyph {name!r}')
return self.cmap[code]
def glyph_box(self, name: str, size: float) -> Box:
def glyph_box(self, name: str, size: float) -> generator.engraving.types.Box:
if name not in self.metrics:
pen = BoundsPen(self.glyphs)
pen = fontTools.pens.boundsPen.BoundsPen(self.glyphs)
self.glyphs[self.glyph_name(name)].draw(pen)
if pen.bounds is None:
raise EngravingError(f'empty SMuFL glyph {name!r}')
raise generator.engraving.types.EngravingError(f'empty SMuFL glyph {name!r}')
self.metrics[name] = pen.bounds
left, bottom, right, top = self.metrics[name]
scale = size / self.units
return Box(left * scale, -top * scale, right * scale, -bottom * scale)
return generator.engraving.types.Box(left * scale, -top * scale, right * scale, -bottom * scale)
def outline(self, name: str):
if name not in self.outlines:
pen = DecomposingRecordingPen(self.glyphs)
pen = fontTools.pens.recordingPen.DecomposingRecordingPen(self.glyphs)
self.glyphs[self.glyph_name(name)].draw(pen)
self.outlines[name] = pen
return self.outlines[name]
def glyph(self, name: str, x: float, y: float, size: float, role: str, sources: tuple[str, ...]) -> Primitive:
return Primitive('glyph', role, sources, self.glyph_box(name, size).moved(x, y), ((x, y),), name, size)
def glyph(self, name: str, x: float, y: float, size: float, role: str, sources: tuple[str, ...]) -> generator.engraving.types.Primitive:
return generator.engraving.types.Primitive('glyph', role, sources, self.glyph_box(name, size).moved(x, y), ((x, y),), name, size)
def anchor(self, name: str, anchor: str, size: float) -> tuple[float, float]:
value = self.metadata.get('glyphsWithAnchors', {}).get(name, {}).get(anchor)
if value is None:
@@ -70,14 +70,14 @@ class Fonts:
def text_font(self, size: float):
pixels = max(1, round(size))
if pixels not in self.text_fonts:
self.text_fonts[pixels] = ImageFont.truetype(str(self.root / 'PetalumaScript.otf'), pixels)
self.text_fonts[pixels] = PIL.ImageFont.truetype(str(self.root / 'PetalumaScript.otf'), pixels)
return self.text_fonts[pixels]
def text(self, value: str, x: float, y: float, size: float, role: str, sources: tuple[str, ...], center: bool = False) -> Primitive:
def text(self, value: str, x: float, y: float, size: float, role: str, sources: tuple[str, ...], center: bool = False) -> generator.engraving.types.Primitive:
missing = [c for c in value if ord(c) not in self.text_cmap and not c.isspace()]
if missing:
raise EngravingError(f'Petaluma Script is missing text glyph {missing[0]!r}')
raise generator.engraving.types.EngravingError(f'Petaluma Script is missing text glyph {missing[0]!r}')
font = self.text_font(size * 4)
left, top, right, bottom = font.getbbox(value, anchor='ls')
offset = (left + right) / 8 if center else 0
x -= offset
return Primitive('text', role, sources, Box(x + left / 4, y + top / 4, x + right / 4, y + bottom / 4), ((x, y),), value, size)
return generator.engraving.types.Primitive('text', role, sources, generator.engraving.types.Box(x + left / 4, y + top / 4, x + right / 4, y + bottom / 4), ((x, y),), value, size)
generator/engraving/layout.py
+81 -61
View File
@@ -1,33 +1,35 @@
from __future__ import annotations
from dataclasses import replace
from bisect import bisect_left, bisect_right
from fractions import Fraction
import bisect
import collections
import dataclasses
import fractions
import math
from generator import harmony
from generator.engraving import tablature
from generator.model import validate_document
from generator.engraving.attributes import barline, staff_attributes, tempo
from generator.engraving.fonts import Fonts
from generator.engraving.notation import ScoreIndex
from generator.engraving.notes import draw_event
from generator.engraving.spans import draw_beams, draw_lyric_spans, draw_spans, place_text_lanes
from generator.engraving.types import Anchor, EngravingError, Fragment, LayoutDocument, Page, Primitive, Style, System, bounds, line
import generator.engraving.attributes
import generator.engraving.audit
import generator.engraving.fonts
import generator.engraving.notation
import generator.engraving.notes
import generator.engraving.spans
import generator.engraving.tablature
import generator.engraving.types
import generator.harmony
import generator.model
def lyric_text(verse: dict) -> str:
return ''.join(s['text'] for s in verse['syllables'] if s['kind'] in ('text', 'elision'))
def column(index: ScoreIndex, time: Fraction, start: Fraction, end: Fraction, fonts: Fonts, style: Style) -> tuple[list[Primitive], list[Anchor], list[Fragment]]:
def column(index: generator.engraving.notation.ScoreIndex, time: fractions.Fraction, start: fractions.Fraction, end: fractions.Fraction, fonts: generator.engraving.fonts.Fonts, style: generator.engraving.types.Style) -> tuple[list[generator.engraving.types.Primitive], list[generator.engraving.types.Anchor], list[generator.engraving.types.Fragment]]:
items, anchors, fragments = [], [], []
events = index.events_at(time, include_sustained=time == start)
grace_width = 0
occupied = []
for event in events:
try:
drawn, placed, pieces = draw_event(index, event, start, end, fonts, style)
drawn, placed, pieces = generator.engraving.notes.draw_event(index, event, start, end, fonts, style)
except ValueError as error:
raise EngravingError(f'measure {event.measure["number"]}, voice {event.voice}, {event.value["id"]}: {error}') from error
raise generator.engraving.types.EngravingError(f'measure {event.measure["number"]}, voice {event.voice}, {event.value["id"]}: {error}') from error
x = grace_width
collidable = [p for p in drawn if p.role in ('head', 'tab', 'rest', 'slash', 'accidental', 'dot')]
while any(p.box.moved(x, 0).overlaps(q.box, 2) for p in collidable for q in occupied):
@@ -38,7 +40,7 @@ def column(index: ScoreIndex, time: Fraction, start: Fraction, end: Fraction, fo
grace_width = max((p.box.right for p in drawn), default=grace_width) + 9
occupied.extend(p for p in drawn if p.role in ('head', 'tab', 'rest', 'slash', 'accidental', 'dot'))
items.extend(drawn)
items.extend(tablature.column(index, event, placed, fonts, style))
items.extend(generator.engraving.tablature.column(index, event, placed, fonts, style))
anchors.extend(placed)
fragments.extend(pieces)
if event.start == time and not event.value.get('hidden'):
@@ -51,47 +53,61 @@ def column(index: ScoreIndex, time: Fraction, start: Fraction, end: Fraction, fo
measure_start, _, measure = index.measure_at(time)
for owner, _, item in index.harmonies_by_onset.get(time, []):
if owner is measure:
items.append(fonts.text(harmony.symbol(item), 0, 0, 21, 'harmony', (item['id'],)))
items.append(fonts.text(generator.harmony.symbol(item), 0, 0, 21, 'harmony', (item['id'],)))
for owner, i, value in index.tempos_by_onset.get(time, []):
if owner is measure:
items.extend(tempo(value, fonts, 0, 0, style.space, measure['id'] + f':tempo:{i}'))
items.extend(generator.engraving.attributes.tempo(value, fonts, 0, 0, style.space, measure['id'] + f':tempo:{i}'))
if time == measure_start:
items.append(fonts.text(str(measure['number']), 0, -style.space * (index.state(time)['staff_details']['lines'] + 0.5), 12, 'measure-number', (measure['id'],)))
if label := measure.get('section_label'):
items.append(fonts.text(label, 0, 0, 19, 'section', (measure['id'] + ':section',)))
return items, anchors, fragments
def make_system(index: ScoreIndex, start: Fraction, end: Fraction, fonts: Fonts, style: Style, justify: bool = False) -> System:
def column_metrics(index, time, start, end, fonts, style, columns):
events = index.events_at(time, include_sustained=time == start)
finish = min(end, max((event.end for event in events), default=time))
key = (time, start, finish)
if columns is not None and key in columns:
columns.move_to_end(key)
return columns[key]
drawn, placed, pieces = column(index, time, start, end, fonts, style)
result = drawn, placed, pieces, generator.engraving.types.bounds(drawn)
if columns is not None:
columns[key] = result
if len(columns) > 256:
columns.popitem(last=False)
return result
def make_system(index: generator.engraving.notation.ScoreIndex, start: fractions.Fraction, end: fractions.Fraction, fonts: generator.engraving.fonts.Fonts, style: generator.engraving.types.Style, justify: bool = False, *, columns=None) -> generator.engraving.types.System:
items, anchors, fragments = [], [], []
state = index.state(start)
measure = index.measure_at(start)[2]
header = staff_attributes(index, state, start, fonts, style.space, measure['id'] + ':initial', ('clef', 'key', 'meter', 'staff_details'))
header = generator.engraving.attributes.staff_attributes(index, state, start, fonts, style.space, measure['id'] + ':initial', ('clef', 'key', 'meter', 'staff_details'))
items.extend(header)
cursor = max((p.box.right for p in header), default=0) + 22
content_left = cursor - 9
times = [start] + index.column_times[bisect_right(index.column_times, start):bisect_left(index.column_times, end)]
times = [start] + index.column_times[bisect.bisect_right(index.column_times, start):bisect.bisect_left(index.column_times, end)]
placements = []
groups = []
for ti, time in enumerate(times):
item_start, anchor_start = len(items), len(anchors)
drawn, placed, pieces = column(index, time, start, end, fonts, style)
drawn, placed, pieces, box = column_metrics(index, time, start, end, fonts, style, columns)
leading = []
measure_start, _, m = index.measure_at(time)
if time == measure_start and time > start:
previous = index.measure_at(time, before=True)[2]
leading.extend(barline(previous.get('barlines', {}).get('right', 'regular'), 0, -style.space * (index.state(time)['staff_details']['lines'] - 1), previous['id']))
leading.extend(generator.engraving.attributes.barline(previous.get('barlines', {}).get('right', 'regular'), 0, -style.space * (index.state(time)['staff_details']['lines'] - 1), previous['id']))
old_state = index.state(time, before=True)
fields = tuple(k for k in ('clef', 'key', 'meter', 'staff_details') if old_state[k] != index.state(time)[k])
leading.extend(p.moved(12, 0) for p in staff_attributes(index, index.state(time), time, fonts, style.space, m['id'] + ':initial', fields))
leading.extend(p.moved(12, 0) for p in generator.engraving.attributes.staff_attributes(index, index.state(time), time, fonts, style.space, m['id'] + ':initial', fields))
if time > start:
for owner, ci, change in index.attributes_by_onset.get(time, []):
extra = staff_attributes(index, index.state(time), time, fonts, style.space, owner['id'] + f':attributes:{ci}', tuple(change))
extra = generator.engraving.attributes.staff_attributes(index, index.state(time), time, fonts, style.space, owner['id'] + f':attributes:{ci}', tuple(change))
leading.extend(p.moved(max((p.box.right for p in leading), default=0) + 10, 0) for p in extra)
if leading:
width = bounds(leading).right - bounds(leading).left + 14
items.extend(p.moved(cursor - bounds(leading).left, 0) for p in leading)
width = generator.engraving.types.bounds(leading).right - generator.engraving.types.bounds(leading).left + 14
items.extend(p.moved(cursor - generator.engraving.types.bounds(leading).left, 0) for p in leading)
cursor += width
box = bounds(drawn)
x = cursor - min(0, box.left)
placements.append((time, x))
items.extend(p.moved(x, 0) for p in drawn)
@@ -114,10 +130,10 @@ def make_system(index: ScoreIndex, start: Fraction, end: Fraction, fonts: Fonts,
last_measure = index.measure_at(end, before=True)[2]
tab_items = [p for p in items if p.role.startswith('tab')]
items = [p for p in items if not p.role.startswith('tab')]
items = draw_beams(index, anchors, items, start, end, content_left, right, style)
items = draw_spans(index, anchors, items, start, end, content_left, right, fonts, style)
items = place_text_lanes(items, fonts, style)
items = draw_lyric_spans(index, items, anchors, start, end, content_left, right, fonts)
items = generator.engraving.spans.draw_beams(index, anchors, items, start, end, content_left, right, style)
items = generator.engraving.spans.draw_spans(index, anchors, items, start, end, content_left, right, fonts, style)
items = generator.engraving.spans.place_text_lanes(items, fonts, style)
items = generator.engraving.spans.draw_lyric_spans(index, items, anchors, start, end, content_left, right, fonts)
changes = [(0, state['staff_details']['lines'])]
for time, x in placements:
lines = index.state(time)['staff_details']['lines']
@@ -128,18 +144,18 @@ def make_system(index: ScoreIndex, start: Fraction, end: Fraction, fonts: Fonts,
x2 = changes[i + 1][0] if i + 1 < len(changes) else width - 1
for number in range(lines):
y = -number * style.space
staff_lines.append(line('staff', (), ((x1, y), (x2, y)), style.space * 0.13))
staff_lines.append(generator.engraving.types.line('staff', (), ((x1, y), (x2, y)), style.space * 0.13))
items = staff_lines + items
if end in index.boundaries:
items.extend(barline(last_measure.get('barlines', {}).get('right', 'regular'), width - 1, -style.space * (index.state(end, before=True)['staff_details']['lines'] - 1), last_measure['id']))
items.extend(generator.engraving.attributes.barline(last_measure.get('barlines', {}).get('right', 'regular'), width - 1, -style.space * (index.state(end, before=True)['staff_details']['lines'] - 1), last_measure['id']))
if start in index.boundaries:
items.extend(barline(measure.get('barlines', {}).get('left', 'none'), 0, -style.space * (state['staff_details']['lines'] - 1), measure['id']))
items.extend(generator.engraving.attributes.barline(measure.get('barlines', {}).get('left', 'none'), 0, -style.space * (state['staff_details']['lines'] - 1), measure['id']))
else:
items.append(fonts.text(str(measure['number']) + ' (cont.)', 0, min(p.box.top for p in items) - 6, 12, 'continuation', (measure['id'],)))
items = tablature.staff(index, items + tab_items, start, end, content_left, width - 1, fonts, style)
return System(start, end, tuple(items), tuple(anchors), tuple(fragments), bounds(items))
items = generator.engraving.tablature.staff(index, items + tab_items, start, end, content_left, width - 1, fonts, style)
return generator.engraving.types.System(start, end, tuple(items), tuple(anchors), tuple(fragments), generator.engraving.types.bounds(items))
def paginate(systems: list[System], document: dict, fonts: Fonts, style: Style) -> tuple[Page, ...]:
def paginate(systems: list[generator.engraving.types.System], document: dict, fonts: generator.engraving.fonts.Fonts, style: generator.engraving.types.Style) -> tuple[generator.engraving.types.Page, ...]:
pages = []
placed = []
header = []
@@ -154,10 +170,10 @@ def paginate(systems: list[System], document: dict, fonts: Fonts, style: Style)
size -= 1
item = fonts.text(title, style.margin_x, style.margin_y + size, size, 'title', ())
if item.box.right > style.width - style.margin_x:
raise EngravingError('score title cannot fit the page without clipping')
raise generator.engraving.types.EngravingError('score title cannot fit the page without clipping')
result = [item]
if number == 1:
subtitle = ' - '.join(str(value) for value in (document['metadata'].get('composer'), document['staves'][0].get('name'), document['metadata'].get('style')) if value)
subtitle = ' - '.join(str(value) for value in (document['metadata'].get('composer'), document['staves'][0].get('name')) if value)
if subtitle:
result.append(fonts.text(subtitle, style.margin_x, item.box.bottom + 22, 16, 'subtitle', ()))
return result
@@ -166,61 +182,65 @@ def paginate(systems: list[System], document: dict, fonts: Fonts, style: Style)
height = system.box.bottom - system.box.top
if first:
header = page_header(len(pages) + 1)
cursor = bounds(header).bottom + 24
cursor = generator.engraving.types.bounds(header).bottom + 24
first = False
if cursor + height > style.height - style.margin_y - 18:
if not placed:
raise EngravingError(f'at musical offset {system.start}: simultaneous notation needs {height:.1f}px vertically; fixed-size page cannot fit it')
raise generator.engraving.types.EngravingError(f'at musical offset {system.start}: simultaneous notation needs {height:.1f}px vertically; fixed-size page cannot fit it')
number = len(pages) + 1
footer = fonts.text(str(number), style.width / 2, style.height - style.margin_y, 12, 'page-number', (), True)
pages.append(Page(number, style.width, style.height, style.supersampling, tuple(placed), tuple(header + [p for s in placed for p in s.primitives] + [footer])))
pages.append(generator.engraving.types.Page(number, style.width, style.height, style.supersampling, tuple(placed), tuple(header + [p for s in placed for p in s.primitives] + [footer])))
placed = []
header = page_header(len(pages) + 1)
cursor = bounds(header).bottom + 24
cursor = generator.engraving.types.bounds(header).bottom + 24
if cursor + height > style.height - style.margin_y - 18:
raise EngravingError(f'at musical offset {system.start}: simultaneous notation is too tall for a page')
raise generator.engraving.types.EngravingError(f'at musical offset {system.start}: simultaneous notation is too tall for a page')
dx, dy = style.margin_x + 1, cursor - system.box.top
moved = replace(system, primitives=tuple(p.moved(dx, dy) for p in system.primitives), anchors=tuple(a.moved(dx, dy) for a in system.anchors), box=system.box.moved(dx, dy))
moved = dataclasses.replace(system, primitives=tuple(p.moved(dx, dy) for p in system.primitives), anchors=tuple(a.moved(dx, dy) for a in system.anchors), box=system.box.moved(dx, dy))
placed.append(moved)
cursor += height + style.system_gap
if placed:
number = len(pages) + 1
footer = fonts.text(str(number), style.width / 2, style.height - style.margin_y, 12, 'page-number', (), True)
pages.append(Page(number, style.width, style.height, style.supersampling, tuple(placed), tuple(header + [p for s in placed for p in s.primitives] + [footer])))
pages.append(generator.engraving.types.Page(number, style.width, style.height, style.supersampling, tuple(placed), tuple(header + [p for s in placed for p in s.primitives] + [footer])))
return tuple(pages)
def layout_score(document: dict, style: Style, fonts: Fonts) -> LayoutDocument:
validate_document(document)
index = ScoreIndex(document)
def layout_systems(document: dict, style: generator.engraving.types.Style, fonts: generator.engraving.fonts.Fonts) -> tuple[generator.engraving.types.System, ...]:
generator.model.validate_document(document)
index = generator.engraving.notation.ScoreIndex(document)
if not index.measures:
raise EngravingError('score has no measures')
raise generator.engraving.types.EngravingError('score has no measures')
systems = []
start = Fraction()
start = fractions.Fraction()
available = style.width - style.margin_x * 2
while start < index.end:
candidates = index.boundary_times[bisect_right(index.boundary_times, start):]
columns = collections.OrderedDict()
candidates = index.boundary_times[bisect.bisect_right(index.boundary_times, start):]
selected = None
# Reserve the same title/credit and footer bands when considering a split,
# so a wide pitch range can break vertically as well as horizontally.
height_limit = style.height - style.margin_y * 2 - 110
for end in candidates:
trial = make_system(index, start, end, fonts, style)
trial = make_system(index, start, end, fonts, style, columns=columns)
if trial.box.right - trial.box.left > available or trial.box.bottom - trial.box.top > height_limit:
break
selected = trial
if selected is None:
measure_end = candidates[0]
for end in index.column_times[bisect_right(index.column_times, start):bisect_left(index.column_times, measure_end)]:
trial = make_system(index, start, end, fonts, style)
for end in index.column_times[bisect.bisect_right(index.column_times, start):bisect.bisect_left(index.column_times, measure_end)]:
trial = make_system(index, start, end, fonts, style, columns=columns)
if trial.box.right - trial.box.left <= available and trial.box.bottom - trial.box.top <= height_limit:
selected = trial
else:
break
if selected is None:
raise EngravingError(f'at musical offset {start}: irreducible simultaneous content exceeds the page width or height')
systems.append(make_system(index, start, selected.end, fonts, style, justify=selected.end < index.end))
raise generator.engraving.types.EngravingError(f'at musical offset {start}: irreducible simultaneous content exceeds the page width or height')
systems.append(make_system(index, start, selected.end, fonts, style, justify=True, columns=columns) if selected.end < index.end else selected)
start = selected.end
layout = LayoutDocument(paginate(systems, document, fonts, style), style, fonts.fingerprint(style))
from generator.engraving.audit import audit_layout
audit_layout(document, layout, fonts)
return tuple(systems)
def layout_score(document: dict, style: generator.engraving.types.Style, fonts: generator.engraving.fonts.Fonts) -> generator.engraving.types.LayoutDocument:
systems = layout_systems(document, style, fonts)
layout = generator.engraving.types.LayoutDocument(paginate(systems, document, fonts, style), style, fonts.fingerprint(style))
generator.engraving.audit.audit_layout(document, layout, fonts)
return layout
generator/engraving/notation.py
+36 -36
View File
@@ -1,24 +1,24 @@
from __future__ import annotations
from dataclasses import dataclass
from bisect import bisect_left, bisect_right
from fractions import Fraction
import bisect
import dataclasses
import fractions
from generator.model import LETTERS, NOTE_TYPES, alteration, fraction, written_for
from generator.engraving.fonts import Fonts
from generator.engraving.types import EngravingError, Primitive, bounds
import generator.engraving.fonts
import generator.engraving.types
import generator.model
ACCIDENTAL_GLYPHS = {Fraction(0): 'accidentalNatural', Fraction(1): 'accidentalSharp', Fraction(-1): 'accidentalFlat', Fraction(2): 'accidentalDoubleSharp', Fraction(-2): 'accidentalDoubleFlat', Fraction(3): 'accidentalTripleSharp', Fraction(-3): 'accidentalTripleFlat', Fraction(1, 2): 'accidentalQuarterToneSharpStein', Fraction(-1, 2): 'accidentalQuarterToneFlatStein', Fraction(3, 2): 'accidentalThreeQuarterTonesSharpStein', Fraction(-3, 2): 'accidentalThreeQuarterTonesFlatZimmermann'}
ACCIDENTAL_GLYPHS = {fractions.Fraction(0): 'accidentalNatural', fractions.Fraction(1): 'accidentalSharp', fractions.Fraction(-1): 'accidentalFlat', fractions.Fraction(2): 'accidentalDoubleSharp', fractions.Fraction(-2): 'accidentalDoubleFlat', fractions.Fraction(3): 'accidentalTripleSharp', fractions.Fraction(-3): 'accidentalTripleFlat', fractions.Fraction(1, 2): 'accidentalQuarterToneSharpStein', fractions.Fraction(-1, 2): 'accidentalQuarterToneFlatStein', fractions.Fraction(3, 2): 'accidentalThreeQuarterTonesSharpStein', fractions.Fraction(-3, 2): 'accidentalThreeQuarterTonesFlatZimmermann'}
HEADS = {'x': 'X', 'diamond': 'Diamond', 'triangle': 'TriangleUp', 'inverted triangle': 'TriangleDown', 'circle-x': 'CircleX', 'cross': 'Plus', 'slash': 'Slash', 'normal': '', 'do': 'TriangleUp', 're': 'Moon', 'mi': 'Diamond', 'fa': 'TriangleRight', 'fa up': 'TriangleLeft', 'so': 'Round', 'la': 'Square', 'ti': 'TriangleRoundDown', 'rectangle': 'Square', 'square': 'Square', 'arrow down': 'LargeArrowDown', 'arrow up': 'LargeArrowUp'}
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class Event:
measure: dict
voice: str
lane: int
lanes: int
start: Fraction
end: Fraction
start: fractions.Fraction
end: fractions.Fraction
value: dict
class ScoreIndex:
@@ -32,20 +32,20 @@ class ScoreIndex:
self.tempos_by_onset = {}
self.by_id = {}
self.tones = {}
self.boundaries = {Fraction()}
self.columns = {Fraction()}
self.boundaries = {fractions.Fraction()}
self.columns = {fractions.Fraction()}
self._states = {}
self._beam_groups = None
time = Fraction()
time = fractions.Fraction()
for measure in self.staff['measures']:
end = time + fraction(measure['duration'])
end = time + generator.model.fraction(measure['duration'])
self.measures.append((time, end, measure))
self.boundaries.add(end)
self.columns.update((time, end))
for lane, voice in enumerate(measure['voices']):
for note in voice['notes']:
onset = time + fraction(note['onset'])
event = Event(measure, voice['id'], lane, len(measure['voices']), onset, onset + fraction(note['duration']), note)
onset = time + generator.model.fraction(note['onset'])
event = Event(measure, voice['id'], lane, len(measure['voices']), onset, onset + generator.model.fraction(note['duration']), note)
self.events.append(event)
self.by_id[note['id']] = event
self.columns.add(onset)
@@ -54,7 +54,7 @@ class ScoreIndex:
for field in ('attributes', 'harmonies', 'tempos'):
bucket = {'attributes': self.attributes_by_onset, 'harmonies': self.harmonies_by_onset, 'tempos': self.tempos_by_onset}[field]
for index, item in enumerate(measure.get(field, [])):
onset = time + fraction(item['onset'])
onset = time + generator.model.fraction(item['onset'])
self.columns.add(onset)
bucket.setdefault(onset, []).append((measure, index, item))
time = end
@@ -72,15 +72,15 @@ class ScoreIndex:
self.boundary_times = sorted(self.boundaries)
self.lyric_lanes = list(dict.fromkeys((e.voice, verse['number']) for e in self.events for verse in e.value.get('lyrics', [])))
self.accidentals = self.accidental_state()
def measure_at(self, time: Fraction, before: bool = False) -> tuple:
position = bisect_left(self.measure_starts, time) if before else bisect_right(self.measure_starts, time)
def measure_at(self, time: fractions.Fraction, before: bool = False) -> tuple:
position = bisect.bisect_left(self.measure_starts, time) if before else bisect.bisect_right(self.measure_starts, time)
return self.measures[max(0, position - 1)]
def events_at(self, time: Fraction, include_sustained: bool = False) -> list[Event]:
def events_at(self, time: fractions.Fraction, include_sustained: bool = False) -> list[Event]:
attacks = self.events_by_onset.get(time, [])
if not include_sustained:
return attacks
sustained = []
position = bisect_left(self.event_starts, time) - 1
position = bisect.bisect_left(self.event_starts, time) - 1
# Prefix maxima let system continuations stop searching once every earlier note has ended.
while position >= 0 and self.event_max_ends[position] > time:
event = self.events[position]
@@ -88,18 +88,18 @@ class ScoreIndex:
sustained.append(event)
position -= 1
return list(reversed(sustained)) + attacks
def state(self, time: Fraction, before: bool = False) -> dict:
def state(self, time: fractions.Fraction, before: bool = False) -> dict:
if (time, before) in self._states:
return self._states[time, before]
start, _, measure = self.measure_at(time, before)
state = {key: measure[key] for key in ('key', 'meter', 'clef', 'transpose', 'staff_details')}
for change in measure.get('attributes', []):
onset = start + fraction(change['onset'])
onset = start + generator.model.fraction(change['onset'])
if onset < time or (onset == time and not before):
state.update({key: value for key, value in change.items() if key != 'onset'})
self._states[time, before] = state
return state
def pitch_position(self, tone: dict, time: Fraction) -> float:
def pitch_position(self, tone: dict, time: fractions.Fraction) -> float:
state = self.state(time)
clef = state['clef']
pitch = tone.get('unpitched', tone)
@@ -107,14 +107,14 @@ class ScoreIndex:
reference = {'G': 4 * 7 + 4, 'F': 3 * 7 + 3, 'C': 4 * 7}.get(sign, 4 * 7 + 4)
clef_line = clef.get('line', 2) if sign in ('G', 'F', 'C') else 2
octave = 0 if 'unpitched' in tone else clef.get('octave_change', 0)
shift = sum(s['octaves'] for s in self.staff.get('octave_shifts', []) if fraction(s['start']) <= time < fraction(s['end'])) if 'unpitched' not in tone else 0
return pitch['octave'] * 7 + LETTERS.index(pitch['letter']) + 7 * (shift - octave) - reference + 2 * (clef_line - 1)
shift = sum(s['octaves'] for s in self.staff.get('octave_shifts', []) if generator.model.fraction(s['start']) <= time < generator.model.fraction(s['end'])) if 'unpitched' not in tone else 0
return pitch['octave'] * 7 + generator.model.LETTERS.index(pitch['letter']) + 7 * (shift - octave) - reference + 2 * (clef_line - 1)
def accidental_state(self) -> dict[str, bool]:
result = {}
for start, end, measure in self.measures:
memory = {}
key = None
onsets = self.onsets[bisect_left(self.onsets, start):bisect_left(self.onsets, end)]
onsets = self.onsets[bisect.bisect_left(self.onsets, start):bisect.bisect_left(self.onsets, end)]
for onset in onsets:
state = self.state(onset)
if key != state['key']:
@@ -130,11 +130,11 @@ class ScoreIndex:
result[tone['id']] = False
continue
identity = (tone['letter'], tone['octave'])
value = alteration(tone['alter'])
signature = Fraction() if key.get('custom') else alteration(key['alterations'][LETTERS.index(tone['letter'])])
value = generator.model.alteration(tone['alter'])
signature = fractions.Fraction() if key.get('custom') else generator.model.alteration(key['alterations'][generator.model.LETTERS.index(tone['letter'])])
for step in key.get('steps', []):
if step['letter'] == tone['letter'] and step.get('octave', tone['octave']) == tone['octave']:
signature = alteration(step['alter'])
signature = generator.model.alteration(step['alter'])
result[tone['id']] = bool(tone.get('accidental_display')) or (not tone.get('tie_from') and memory.get(identity, signature) != value)
simultaneous.setdefault(identity, set()).add(value)
for identity, values in simultaneous.items():
@@ -146,11 +146,11 @@ class ScoreIndex:
result[tone['id']] = True
return result
def fragment_written(written: dict, original_start: Fraction, original_end: Fraction, start: Fraction, end: Fraction) -> dict:
return written if (original_start, original_end) == (start, end) else written_for(end - start)
def fragment_written(written: dict, original_start: fractions.Fraction, original_end: fractions.Fraction, start: fractions.Fraction, end: fractions.Fraction) -> dict:
return written if (original_start, original_end) == (start, end) else generator.model.written_for(end - start)
def flags(written: dict) -> int:
return max(0, NOTE_TYPES.index(written['type']) - NOTE_TYPES.index('quarter'))
return max(0, generator.model.NOTE_TYPES.index(written['type']) - generator.model.NOTE_TYPES.index('quarter'))
def head_glyph(written: dict, head: dict | str | None) -> str:
head = {'name': head} if isinstance(head, str) else head or {'name': 'normal'}
@@ -170,13 +170,13 @@ def head_glyph(written: dict, head: dict | str | None) -> str:
if name in ('circled', 'circle dot'):
return 'noteheadCircled' + suffix
if name not in HEADS:
raise EngravingError(f'no SMuFL mapping for notehead {name!r}; specify its smufl glyph name')
raise generator.engraving.types.EngravingError(f'no SMuFL mapping for notehead {name!r}; specify its smufl glyph name')
shape = HEADS[name]
if shape in ('Moon', 'Round', 'Square', 'TriangleRight', 'TriangleLeft', 'TriangleRoundDown'):
suffix = 'Black' if suffix == 'Black' else 'White'
return 'notehead' + shape + suffix
def accidental(fonts: Fonts, value: Fraction, display: dict, size: float, sources: tuple[str, ...]) -> list[Primitive]:
def accidental(fonts: generator.engraving.fonts.Fonts, value: fractions.Fraction, display: dict, size: float, sources: tuple[str, ...]) -> list[generator.engraving.types.Primitive]:
glyph = display.get('smufl') or ACCIDENTAL_GLYPHS.get(value)
special = {'natural-sharp': ('accidentalNatural', 'accidentalSharp'), 'natural-flat': ('accidentalNatural', 'accidentalFlat')}
if display.get('name') in special and not display.get('smufl'):
@@ -192,7 +192,7 @@ def accidental(fonts: Fonts, value: Fraction, display: dict, size: float, source
sign = '+' if value > 0 else ''
items = [fonts.text(f'({sign}{value} st)', 0, 0, size * 1.15, 'accidental', sources)]
if display.get('parentheses') == 'yes' or display.get('bracket') == 'yes':
box = bounds(items)
box = generator.engraving.types.bounds(items)
left, right = ('[', ']') if display.get('bracket') == 'yes' else ('(', ')')
items.extend((fonts.text(left, box.left - 7, size / 2, size * 1.8, 'accidental', sources), fonts.text(right, box.right + 2, size / 2, size * 1.8, 'accidental', sources)))
return items
generator/engraving/notes.py
+35 -35
View File
@@ -1,13 +1,14 @@
from __future__ import annotations
from fractions import Fraction
import dataclasses
import fractions
from generator.model import alteration, fraction
from generator.engraving.fonts import Fonts
from generator.engraving.notation import Event, ScoreIndex, accidental, flags, fragment_written, head_glyph
from generator.engraving.types import Anchor, Fragment, Primitive, Style, bounds, line
import generator.engraving.fonts
import generator.engraving.notation
import generator.engraving.types
import generator.model
def dots(fonts: Fonts, count: int, x: float, y: float, size: float, sources: tuple[str, ...]) -> list[Primitive]:
def dots(fonts: generator.engraving.fonts.Fonts, count: int, x: float, y: float, size: float, sources: tuple[str, ...]) -> list[generator.engraving.types.Primitive]:
result = []
for i in range(count):
result.append(fonts.glyph('augmentationDot', x + i * size * 0.6, y, size, 'dot', sources))
@@ -16,13 +17,13 @@ def dots(fonts: Fonts, count: int, x: float, y: float, size: float, sources: tup
def rest_glyph(kind: str) -> str:
return {'maxima': 'restMaxima', 'long': 'restLonga', 'breve': 'restDoubleWhole', 'whole': 'restWhole', 'half': 'restHalf', 'quarter': 'restQuarter', 'eighth': 'rest8th'}.get(kind, 'rest' + kind)
def time_modification(written: dict, items: list[Primitive], fonts: Fonts, sources: tuple[str, ...]) -> None:
ratio = fraction(written.get('ratio', {'numerator': 1, 'denominator': 1}))
def time_modification(written: dict, items: list[generator.engraving.types.Primitive], fonts: generator.engraving.fonts.Fonts, sources: tuple[str, ...]) -> None:
ratio = generator.model.fraction(written.get('ratio', {'numerator': 1, 'denominator': 1}))
if ratio != 1 and items:
box = bounds(items)
box = generator.engraving.types.bounds(items)
items.append(fonts.text(f'{ratio.denominator}:{ratio.numerator}', (box.left + box.right) / 2, box.top - 6, 13, 'time-modification', sources, True))
def draw_event(index: ScoreIndex, event: Event, start: Fraction, end: Fraction, fonts: Fonts, style: Style) -> tuple[list[Primitive], list[Anchor], list[Fragment]]:
def draw_event(index: generator.engraving.notation.ScoreIndex, event: generator.engraving.notation.Event, start: fractions.Fraction, end: fractions.Fraction, fonts: generator.engraving.fonts.Fonts, style: generator.engraving.types.Style) -> tuple[list[generator.engraving.types.Primitive], list[generator.engraving.types.Anchor], list[generator.engraving.types.Fragment]]:
note = event.value
onset = max(start, event.start)
finish = min(end, event.end)
@@ -33,17 +34,17 @@ def draw_event(index: ScoreIndex, event: Event, start: Fraction, end: Fraction,
state = index.state(onset)
lines = state['staff_details']['lines']
if not note['pitches']:
written = fragment_written(note['written'], event.start, event.end, onset, finish)
written = generator.engraving.notation.fragment_written(note['written'], event.start, event.end, onset, finish)
y = -style.space * (lines - 1) / 2
if position := note.get('rest_position'):
y = -index.pitch_position(position, onset) * style.space / 2
elif event.lanes > 1:
y += (-1 if event.lane % 2 == 0 else 1) * (event.lane // 2 + 1) * style.space * 3
kind = 'slash' if note.get('notehead') == 'slash' else 'rest'
fragment = Fragment(note['id'], note['id'], onset, finish, written['type'], written.get('dots', 0), fraction(written.get('ratio', {'numerator': 1, 'denominator': 1})), visible, kind, None)
fragment = generator.engraving.types.Fragment(note['id'], note['id'], onset, finish, written['type'], written.get('dots', 0), generator.model.fraction(written.get('ratio', {'numerator': 1, 'denominator': 1})), visible, kind, None)
fragments.append(fragment)
if visible:
glyph = head_glyph(written, 'slash') if kind == 'slash' else rest_glyph('whole' if note.get('measure_rest') and onset == event.start and finish == event.end else written['type'])
glyph = generator.engraving.notation.head_glyph(written, 'slash') if kind == 'slash' else rest_glyph('whole' if note.get('measure_rest') and onset == event.start and finish == event.end else written['type'])
if written['type'] == 'whole' and not note.get('rest_position') and event.lanes == 1:
y -= style.space
items.append(fonts.glyph(glyph, 0, y, size, kind, references))
@@ -51,20 +52,20 @@ def draw_event(index: ScoreIndex, event: Event, start: Fraction, end: Fraction,
if kind == 'slash' and written['type'] not in ('whole', 'breve', 'long', 'maxima'):
stem_x = items[0].box.right
stem_y = y - 3 * size
items.append(line('stem', references, ((stem_x, y), (stem_x, stem_y))))
if flags(written):
items.append(generator.engraving.types.line('stem', references, ((stem_x, y), (stem_x, stem_y))))
if generator.engraving.notation.flags(written):
flag = 'flag' + ('8th' if written['type'] == 'eighth' else written['type']) + 'Up'
items.append(fonts.glyph(flag, stem_x, stem_y, size, 'flag', references))
anchors.append(Anchor(note['id'], note['id'], event.voice, onset, finish, 0, y, 0, y - size * 3, -1, 0, fragment))
anchors.append(generator.engraving.types.Anchor(note['id'], note['id'], event.voice, onset, finish, 0, y, 0, y - size * 3, -1, 0, fragment))
if not note.get('tuplet_ids') or written != note['written']:
time_modification(written, items, fonts, references)
return items, anchors, fragments
groups = {}
for tone in note['pitches']:
tone_end = event.start + fraction(tone['duration'])
tone_end = event.start + generator.model.fraction(tone['duration'])
if tone_end <= start and not note.get('grace'):
continue
written = fragment_written(tone['written'], event.start, tone_end, onset, min(end, tone_end))
written = generator.engraving.notation.fragment_written(tone['written'], event.start, tone_end, onset, min(end, tone_end))
stem = tone.get('stem', note.get('stem', 'auto'))
identity = (written['type'], written.get('dots', 0), str(written.get('ratio')), stem)
groups.setdefault(identity, []).append((tone, written, tone_end))
@@ -82,7 +83,7 @@ def draw_event(index: ScoreIndex, event: Event, start: Fraction, end: Fraction,
for position, tone, written, tone_end in positioned:
sources = (note['id'], tone['id'])
tone_size = size * (0.7 if tone.get('cue') and not note.get('cue') else 1)
glyph = head_glyph(written, tone.get('notehead', note.get('notehead')))
glyph = generator.engraving.notation.head_glyph(written, tone.get('notehead', note.get('notehead')))
displayed = visible and tone.get('print', {}).get('print-object') != 'no' and bool(glyph)
y = -position * style.space / 2
if previous is not None and abs(position - previous) < 2:
@@ -91,13 +92,13 @@ def draw_event(index: ScoreIndex, event: Event, start: Fraction, end: Fraction,
displaced = False
x = group_x + (tone_size * 1.2 * -direction if displaced else 0)
previous = position
fragment = Fragment(tone['id'], note['id'], onset, min(end, tone_end), written['type'], written.get('dots', 0), fraction(written.get('ratio', {'numerator': 1, 'denominator': 1})), displayed, 'tone', position)
fragment = generator.engraving.types.Fragment(tone['id'], note['id'], onset, min(end, tone_end), written['type'], written.get('dots', 0), generator.model.fraction(written.get('ratio', {'numerator': 1, 'denominator': 1})), displayed, 'tone', position)
fragments.append(fragment)
if not displayed:
continue
if written['type'] in ('long', 'maxima') and tone.get('notehead', {}).get('name', 'normal') == 'normal':
width = tone_size * (2.8 if written['type'] == 'maxima' else 1.4)
head = line('head', sources, ((x, y - tone_size / 2), (x + width, y - tone_size / 2), (x + width, y + tone_size / 2), (x, y + tone_size / 2), (x, y - tone_size / 2)), 2)
head = generator.engraving.types.line('head', sources, ((x, y - tone_size / 2), (x + width, y - tone_size / 2), (x + width, y + tone_size / 2), (x, y + tone_size / 2), (x, y - tone_size / 2)), 2)
else:
head = fonts.glyph(glyph, x, y, tone_size, 'head', sources)
group_items.append(head)
@@ -105,15 +106,15 @@ def draw_event(index: ScoreIndex, event: Event, start: Fraction, end: Fraction,
group_items.extend((fonts.text('(', head.box.left - 6, y + 5, 18, 'head-parentheses', sources), fonts.text(')', head.box.right + 2, y + 5, 18, 'head-parentheses', sources)))
ledger = -2
while position <= ledger:
group_items.append(line('ledger', sources, ((head.box.left - 5, -ledger * style.space / 2), (head.box.right + 5, -ledger * style.space / 2)), 1.8))
group_items.append(generator.engraving.types.line('ledger', sources, ((head.box.left - 5, -ledger * style.space / 2), (head.box.right + 5, -ledger * style.space / 2)), 1.8))
ledger -= 2
ledger = lines * 2
while position >= ledger:
group_items.append(line('ledger', sources, ((head.box.left - 5, -ledger * style.space / 2), (head.box.right + 5, -ledger * style.space / 2)), 1.8))
group_items.append(generator.engraving.types.line('ledger', sources, ((head.box.left - 5, -ledger * style.space / 2), (head.box.right + 5, -ledger * style.space / 2)), 1.8))
ledger += 2
if 'alter' in tone and index.accidentals[tone['id']] and onset == event.start:
accidental_items = accidental(fonts, alteration(tone['alter']), tone.get('accidental_display', {}), tone_size, sources)
box = bounds(accidental_items)
accidental_items = generator.engraving.notation.accidental(fonts, generator.model.alteration(tone['alter']), tone.get('accidental_display', {}), tone_size, sources)
box = generator.engraving.types.bounds(accidental_items)
dx = min(group_x, head.box.left) - 5 - box.right
candidate = box.moved(dx, y)
while any(candidate.overlaps(other, 2) for other in accidental_boxes):
@@ -128,32 +129,31 @@ def draw_event(index: ScoreIndex, event: Event, start: Fraction, end: Fraction,
stem_x = x + fonts.anchor(glyph, 'stemUpSE' if direction == -1 else 'stemDownNW', tone_size)[0]
else:
stem_x = head.box.right - 0.6 if direction == -1 else head.box.left + 0.6
stem_y = y + direction * max(3.5 * size, (flags(written) + 1) * size * 0.75)
group_anchors.append(Anchor(tone['id'], note['id'], event.voice, onset, min(end, tone_end), x, y, stem_x, stem_y, direction, flags(written), fragment))
stem_y = y + direction * max(3.5 * size, (generator.engraving.notation.flags(written) + 1) * size * 0.75)
group_anchors.append(generator.engraving.types.Anchor(tone['id'], note['id'], event.voice, onset, min(end, tone_end), x, y, stem_x, stem_y, direction, generator.engraving.notation.flags(written), fragment))
if group_anchors:
stem_y = min(a.stem_y for a in group_anchors) if direction == -1 else max(a.stem_y for a in group_anchors)
anchor = min(group_anchors, key=lambda a: a.y) if direction == 1 else max(group_anchors, key=lambda a: a.y)
written = members[0][1]
if stem != 'none' and written['type'] not in ('whole', 'breve'):
group_items.append(line('stem', references, ((anchor.stem_x, anchor.y), (anchor.stem_x, stem_y)), size * 0.12))
group_items.append(generator.engraving.types.line('stem', references, ((anchor.stem_x, anchor.y), (anchor.stem_x, stem_y)), size * 0.12))
if stem == 'double':
other_x = anchor.x if direction == -1 else anchor.x + size * 1.3
other_y = anchor.y - direction * size * 3.5
group_items.append(line('stem', references, ((other_x, anchor.y), (other_x, other_y)), size * 0.12))
if flags(written):
group_items.append(generator.engraving.types.line('stem', references, ((other_x, anchor.y), (other_x, other_y)), size * 0.12))
if generator.engraving.notation.flags(written):
flag = 'flag' + ('8th' if written['type'] == 'eighth' else written['type']) + ('Down' if direction == -1 else 'Up')
group_items.append(fonts.glyph(flag, other_x, other_y, size, 'flag', references))
if flags(written):
if generator.engraving.notation.flags(written):
flag = 'flag' + ('8th' if written['type'] == 'eighth' else written['type']) + ('Up' if direction == -1 else 'Down')
group_items.append(fonts.glyph(flag, anchor.stem_x, stem_y, size, 'flag', references))
if note.get('grace', {}).get('slash'):
group_items.append(line('grace-slash', references, ((anchor.stem_x - 5, stem_y + 14), (anchor.stem_x + 9, stem_y + 4)), 1.5))
group_items.append(generator.engraving.types.line('grace-slash', references, ((anchor.stem_x - 5, stem_y + 14), (anchor.stem_x + 9, stem_y + 4)), 1.5))
if note.get('tremolo', {}).get('type') == 'single':
for mark in range(note['tremolo']['marks']):
ty = stem_y - direction * (10 + mark * 5)
group_items.append(line('tremolo', references, ((anchor.stem_x - 6, ty + 2), (anchor.stem_x + 6, ty - 2)), 3))
from dataclasses import replace
anchors.extend(replace(a, stem_x=anchor.stem_x, stem_y=stem_y) for a in group_anchors)
group_items.append(generator.engraving.types.line('tremolo', references, ((anchor.stem_x - 6, ty + 2), (anchor.stem_x + 6, ty - 2)), 3))
anchors.extend(dataclasses.replace(a, stem_x=anchor.stem_x, stem_y=stem_y) for a in group_anchors)
if not note.get('tuplet_ids') or members[0][1] != members[0][0]['written']:
time_modification(members[0][1], group_items, fonts, references)
items.extend(group_items)
generator/engraving/pipeline.py
+261
View File
@@ -0,0 +1,261 @@
from __future__ import annotations
import collections
import concurrent.futures
import dataclasses
import hashlib
import json
import multiprocessing
import pathlib
import shutil
import tempfile
import time
import PIL.Image
import generator.engraving.audit
import generator.engraving.cache
import generator.engraving.fonts
import generator.engraving.layout
import generator.engraving.raster
import generator.engraving.types
import generator.engraving.validation
import generator.model
@dataclasses.dataclass
class GroupResult:
scores: list = dataclasses.field(default_factory=list)
fullscore: int = 0
bodyhits: int = 0
bodies: int = 0
renderedpages: int = 0
rejected: int = 0
timings: dict = dataclasses.field(default_factory=lambda: dict.fromkeys(('layout', 'pagination', 'raster', 'cache'), 0.0))
class BodyBands:
def __init__(self, root, persistent, result, limit=16 * 1024 * 1024):
self.root, self.persistent, self.result, self.limit = root, persistent, result, limit
self.images = collections.OrderedDict()
self.size = 0
self.known = {}
self.root.mkdir(parents=True, exist_ok=True)
def get(self, notation, page, band, fonts):
key = generator.engraving.cache.band_fingerprint(notation, page, band)
if key in self.images:
self.images.move_to_end(key)
self.result.bodyhits += 1
return self.images[key]
started = time.monotonic()
entry = self.root / (key + '.rgb')
image = None
if key in self.known:
if entry.is_symlink() or entry.stat().st_size != page.width * (band.bottom - band.top) * 3:
raise generator.engraving.types.EngravingError('temporary notation band was corrupted')
data = entry.read_bytes()
if hashlib.sha256(data).hexdigest() != self.known[key]:
raise generator.engraving.types.EngravingError('temporary notation band was corrupted')
image = PIL.Image.frombytes('RGB', (page.width, band.bottom - band.top), data)
elif self.persistent:
cached = self.persistent / 'notation' / key
image = generator.engraving.cache.restore_band(cached, key, page.width, band.bottom - band.top)
self.result.rejected += int(image is None and (cached.exists() or cached.is_symlink()))
self.result.timings['cache'] += time.monotonic() - started
if image is None:
started = time.monotonic()
image = generator.engraving.raster.rasterize_band(page, band.primitives, band.top, band.bottom, fonts)
self.result.timings['raster'] += time.monotonic() - started
self.result.bodies += 1
if self.persistent:
started = time.monotonic()
generator.engraving.cache.store_band(self.persistent / 'notation' / key, key, image)
self.result.timings['cache'] += time.monotonic() - started
else:
self.result.bodyhits += 1
if key not in self.known:
started = time.monotonic()
# Spill finalized pixels, so eviction never repeats rasterization or PNG decoding.
data = image.tobytes()
entry.write_bytes(data)
self.result.timings['cache'] += time.monotonic() - started
self.known[key] = hashlib.sha256(data).hexdigest()
size = image.width * image.height * 3
while self.images and self.size + size > self.limit:
_, old = self.images.popitem(last=False)
self.size -= old.width * old.height * 3
old.close()
self.images[key] = image
self.size += size
return image
def close(self):
for image in self.images.values():
image.close()
self.images.clear()
self.size = 0
def clear_pages(destination):
if destination.is_symlink() or destination.is_file():
destination.unlink()
elif destination.exists():
shutil.rmtree(destination)
def write_bundle(path, document, pages, fingerprint):
entries = [{**page, 'path': path.with_suffix('.pages').name + '/' + page['path']} for page in pages]
music = {key: value for key, value in document.items() if key != 'playback'}
bundle = {'music': music, 'presentation': {'engine': generator.engraving.validation.ENGINE_NAME, 'style_sha256': fingerprint,
'width': generator.engraving.validation.PAGE_WIDTH, 'height': generator.engraving.validation.PAGE_HEIGHT, 'pages': entries}}
if 'playback' in document:
bundle['playback'] = document['playback']
path.write_text(json.dumps(bundle, indent=2, ensure_ascii=False) + '\n', encoding='utf-8')
def engrave_group(notation, paths, root, font_path, cache_dir, fingerprint, progress=False):
family = generator.engraving.fonts.Fonts(font_path)
style = generator.engraving.types.Style()
systems = None
scores = {}
audited = set()
result = GroupResult()
with tempfile.TemporaryDirectory(prefix='vamp-notation-') as temporary:
bodies = BodyBands(pathlib.Path(temporary), cache_dir, result)
try:
for path, score_key in paths:
try:
document = json.loads(path.read_text(encoding='utf-8'))
destination = path.with_suffix('.pages')
clear_pages(destination)
destination.mkdir()
if score_key in scores:
source, pages = scores[score_key]
for page in pages:
shutil.copyfile(source / page['path'], destination / page['path'])
result.fullscore += 1
else:
if systems is None:
started = time.monotonic()
systems = generator.engraving.layout.layout_systems(document, style, family)
result.timings['layout'] += time.monotonic() - started
started = time.monotonic()
pages_layout = generator.engraving.layout.paginate(systems, document, family, style)
placement = tuple((page.width, page.height, tuple((system.start, system.end, system.box) for system in page.systems)) for page in pages_layout)
if placement not in audited:
layout = generator.engraving.types.LayoutDocument(pages_layout, style, fingerprint)
generator.engraving.audit.audit_layout(document, layout, family)
audited.add(placement)
else:
for page in pages_layout:
decoration = tuple(p for p in page.primitives if p.role in ('title', 'subtitle', 'page-number'))
generator.engraving.audit.audit_primitives(decoration, page.width, page.height, family)
result.timings['pagination'] += time.monotonic() - started
pages = []
for page in pages_layout:
bands = generator.engraving.raster.page_bands(page)
if bands is None:
started = time.monotonic()
image = generator.engraving.raster.rasterize_page(page, family)
else:
body = bodies.get(notation, page, bands[1], family)
started = time.monotonic()
image = generator.engraving.raster.compose_page(page, family, bands, body)
target = destination / f'{page.number:04}.png'
with image:
image.save(target, format='PNG')
result.timings['raster'] += time.monotonic() - started
result.renderedpages += 1
pages.append({'path': target.name, 'sha256': hashlib.sha256(target.read_bytes()).hexdigest()})
scores[score_key] = (destination, pages)
write_bundle(path, document, pages, fingerprint)
result.scores.append((path, score_key, pages))
if progress and len(result.scores) % 50 == 0:
print(f'Engraving body {notation[:12]}: {len(result.scores)}/{len(paths)} scores, fullscore {result.fullscore}, bodyhits {result.bodyhits}, bodies {result.bodies}, renderedpages {result.renderedpages}', flush=True)
except ValueError as error:
raise generator.engraving.types.EngravingError(f'{path.relative_to(root)}: {error}') from error
finally:
bodies.close()
return result
def engrave_tree(root: pathlib.Path, fonts: pathlib.Path, *, cache_dir: pathlib.Path | None = None, jobs: int = 1, progress: bool = False) -> set[str]:
if type(jobs) is not int or not 1 <= jobs <= 16:
raise ValueError('engraving jobs must be between 1 and 16')
if cache_dir and (cache_dir.resolve().is_relative_to(root.resolve()) or root.resolve().is_relative_to(cache_dir.resolve())):
raise ValueError('engraving cache and generated assets must not overlap')
started = time.monotonic()
paths = sorted(root.rglob('*.json'))
validated_pages = set()
if not paths:
return validated_pages
family, style = generator.engraving.fonts.Fonts(fonts), generator.engraving.types.Style()
fingerprint = family.fingerprint(style)
renderer = generator.engraving.cache.renderer_fingerprint(family, style)
if cache_dir:
cache_dir = generator.engraving.cache.renderer_namespace(cache_dir, renderer)
if progress:
state = 'warm namespace' if cache_dir.is_dir() else 'cold namespace'
print(f'Engraving cache enabled: {cache_dir} ({state}; renderer {renderer})', flush=True)
elif progress:
print('Engraving cache disabled', flush=True)
totals = GroupResult()
missing, rejected, completed = 0, 0, 0
groups = {}
for path in paths:
try:
document = json.loads(path.read_text(encoding='utf-8'))
generator.model.validate_document(document)
score_key = generator.engraving.cache.score_fingerprint(document, renderer)
destination = path.with_suffix('.pages')
clear_pages(destination)
entry = cache_dir / 'fullscore' / score_key if cache_dir else None
present = entry is not None and (entry.exists() or entry.is_symlink())
pages = generator.engraving.cache.restore_pages(entry, destination, validated_pages) if entry else None
if pages is None:
rejected += int(present)
missing += int(entry is not None and not present)
notation = generator.engraving.cache.notation_fingerprint(document, renderer)
groups.setdefault(notation, []).append((path, score_key))
else:
write_bundle(path, document, pages, fingerprint)
totals.fullscore += 1
completed += 1
except ValueError as error:
raise generator.engraving.types.EngravingError(f'{path.relative_to(root)}: {error}') from error
prepare_time = time.monotonic() - started
if progress:
print(f'Engraving grouped: {len(paths)} scores, {len(groups)} notation bodies, {totals.fullscore} fullscore hits', flush=True)
validation_time = 0.0
def finish(result):
nonlocal completed, validation_time
started = time.monotonic()
for path, score_key, pages in result.scores:
destination = path.with_suffix('.pages')
for page in pages:
generator.engraving.validation.check_page(destination / page['path'], validated_pages, expected_sha256=page['sha256'])
validation_time += time.monotonic() - started
if cache_dir:
started = time.monotonic()
stored = set()
for path, score_key, pages in result.scores:
if score_key not in stored:
generator.engraving.cache.store_pages(cache_dir / 'fullscore' / score_key, path.with_suffix('.pages'), pages)
stored.add(score_key)
result.timings['cache'] += time.monotonic() - started
for field in ('fullscore', 'bodyhits', 'bodies', 'renderedpages', 'rejected'):
setattr(totals, field, getattr(totals, field) + getattr(result, field))
for phase, elapsed in result.timings.items():
totals.timings[phase] += elapsed
completed += len(result.scores)
if progress:
print(f'Engraving: {completed}/{len(paths)} scores, fullscore {totals.fullscore}, bodyhits {totals.bodyhits}, bodies {totals.bodies}, renderedpages {totals.renderedpages}', flush=True)
jobs = min(jobs, len(groups))
if jobs <= 1:
for notation, group in groups.items():
finish(engrave_group(notation, group, root, fonts, cache_dir, fingerprint, progress))
else:
with concurrent.futures.ProcessPoolExecutor(max_workers=jobs, mp_context=multiprocessing.get_context('spawn')) as pool:
futures = [pool.submit(engrave_group, notation, group, root, fonts, cache_dir, fingerprint, progress) for notation, group in groups.items()]
for future in concurrent.futures.as_completed(futures):
finish(future.result())
if progress:
print(f'Engraving complete: {len(paths)} scores, {totals.fullscore} reused, {missing} missing, {rejected} rejected; fullscore {totals.fullscore}, bodyhits {totals.bodyhits}, bodies {totals.bodies}, renderedpages {totals.renderedpages}, rejected bodies {totals.rejected}', flush=True)
phases = {'prepare': prepare_time, **totals.timings, 'validation': validation_time}
print('Engraving timings: ' + ', '.join(f'{phase} {elapsed:.3f}s' for phase, elapsed in phases.items()) + f'; wall {time.monotonic() - started:.3f}s (worker phases summed)', flush=True)
return validated_pages
generator/engraving/raster.py
+55 -13
View File
@@ -1,13 +1,17 @@
from __future__ import annotations
import dataclasses
import math
import PIL.Image
import PIL.ImageDraw
import aggdraw
from fontTools.pens.basePen import BasePen
from PIL import Image, ImageDraw
import fontTools.pens.basePen
from generator.engraving.fonts import Fonts
from generator.engraving.types import Page
import generator.engraving.fonts
import generator.engraving.types
class OutlinePen(BasePen):
class OutlinePen(fontTools.pens.basePen.BasePen):
def __init__(self, glyphs, scale: float, x: float, y: float):
super().__init__(glyphs)
self.path = aggdraw.Path()
@@ -25,12 +29,15 @@ class OutlinePen(BasePen):
def _endPath(self):
pass
def rasterize_page(page: Page, fonts: Fonts) -> Image.Image:
def rasterize_page(page: generator.engraving.types.Page, fonts: generator.engraving.fonts.Fonts) -> PIL.Image.Image:
return rasterize_band(page, page.primitives, 0, page.height, fonts)
def rasterize_band(page: generator.engraving.types.Page, primitives: tuple[generator.engraving.types.Primitive, ...], top: int, bottom: int, fonts: generator.engraving.fonts.Fonts) -> PIL.Image.Image:
scale = page.supersampling
image = Image.new('RGB', (page.width * scale, page.height * scale), 'white')
image = PIL.Image.new('RGB', (page.width * scale, (bottom - top) * scale), 'white')
context = aggdraw.Draw(image)
for item in page.primitives:
points = [(x * scale, y * scale) for x, y in item.points]
for item in primitives:
points = [(x * scale, y * scale - top * scale) for x, y in item.points]
if item.kind == 'text':
continue
if item.kind == 'glyph':
@@ -52,8 +59,43 @@ def rasterize_page(page: Page, fonts: Fonts) -> Image.Image:
else:
context.path(path, aggdraw.Pen('black', item.thickness * scale))
context.flush()
drawing = ImageDraw.Draw(image)
for item in page.primitives:
drawing = PIL.ImageDraw.Draw(image)
for item in primitives:
if item.kind == 'text':
drawing.text(tuple(v * scale for v in item.points[0]), item.value, font=fonts.text_font(item.size * scale), anchor='ls', fill='black')
return image.resize((page.width, page.height), Image.Resampling.LANCZOS)
x, y = item.points[0]
drawing.text((x * scale, y * scale - top * scale), item.value, font=fonts.text_font(item.size * scale), anchor='ls', fill='black')
return image.resize((page.width, bottom - top), PIL.Image.Resampling.LANCZOS)
@dataclasses.dataclass(frozen=True)
class Band:
top: int
bottom: int
primitives: tuple[generator.engraving.types.Primitive, ...]
def page_bands(page: generator.engraving.types.Page) -> tuple[Band, Band, Band] | None:
body = tuple(p for system in page.systems for p in system.primitives)
header = tuple(p for p in page.primitives if p.role in ('title', 'subtitle'))
footer = tuple(p for p in page.primitives if p.role == 'page-number')
if not body or not header or not footer or page.primitives != header + body + footer:
return None
bands = []
for primitives in (header, body, footer):
box = generator.engraving.types.bounds(primitives)
if not box.finite() or box.left < 4 or box.right > page.width - 4 or box.top < 4 or box.bottom > page.height - 4:
return None
# Six output pixels cover raster antialiasing and the three-pixel Lanczos
# support. Integer crop origins preserve the full-page sampling phase.
bands.append(Band(max(0, math.floor(box.top) - 6), min(page.height, math.ceil(box.bottom) + 6), primitives))
if bands[0].bottom > bands[1].top or bands[1].bottom > bands[2].top:
return None
return tuple(bands)
def compose_page(page: generator.engraving.types.Page, fonts: generator.engraving.fonts.Fonts, bands: tuple[Band, Band, Band], body: PIL.Image.Image) -> PIL.Image.Image:
image = PIL.Image.new('RGB', (page.width, page.height), 'white')
if body.mode != 'RGB' or body.size != (page.width, bands[1].bottom - bands[1].top):
raise generator.engraving.types.EngravingError('notation band has incompatible geometry')
image.paste(body, (0, bands[1].top))
for band in (bands[0], bands[2]):
with rasterize_band(page, band.primitives, band.top, band.bottom, fonts) as decoration:
image.paste(decoration, (0, band.top))
return image
generator/engraving/spans.py
+34 -34
View File
@@ -1,13 +1,13 @@
from __future__ import annotations
from fractions import Fraction
import fractions
from generator.model import beat_lengths, fraction
from generator.engraving.fonts import Fonts
from generator.engraving.notation import ScoreIndex, flags
from generator.engraving.types import Anchor, Primitive, Style, bounds, curve, line, polygon
import generator.engraving.fonts
import generator.engraving.notation
import generator.engraving.types
import generator.model
def beam_groups(index: ScoreIndex) -> tuple[tuple[str, ...], ...]:
def beam_groups(index: generator.engraving.notation.ScoreIndex) -> tuple[tuple[str, ...], ...]:
if index._beam_groups is not None:
return index._beam_groups
groups = []
@@ -18,17 +18,17 @@ def beam_groups(index: ScoreIndex) -> tuple[tuple[str, ...], ...]:
note = event.value
primary = next((b['type'] for b in note.get('beams', []) if b['level'] == 1), None)
measure_start = next(start for start, _, measure in index.measures if measure['id'] == event.measure['id'])
beats = beat_lengths(index.state(event.start)['meter'])
beats = generator.model.beat_lengths(index.state(event.start)['meter'])
offset = event.start - measure_start
beat = 0
edge = Fraction()
edge = fractions.Fraction()
for duration in beats:
edge += duration
if offset < edge:
break
beat += 1
key = (event.measure['id'], beat, tuple(note.get('tuplet_ids', [])), bool(note.get('grace')))
eligible = note['kind'] == 'note' and flags(note['written']) and note.get('stem') != 'none' and not note.get('hidden')
eligible = note['kind'] == 'note' and generator.engraving.notation.flags(note['written']) and note.get('stem') != 'none' and not note.get('hidden')
if not eligible or primary == 'begin' or (primary is None and key != previous_key):
if current:
groups.append(current)
@@ -44,7 +44,7 @@ def beam_groups(index: ScoreIndex) -> tuple[tuple[str, ...], ...]:
index._beam_groups = tuple(tuple(g) for g in groups if len(g) > 1 or index.by_id[g[0]].value.get('beams'))
return index._beam_groups
def draw_beams(index: ScoreIndex, anchors: list[Anchor], items: list[Primitive], start: Fraction, end: Fraction, left: float, right: float, style: Style) -> list[Primitive]:
def draw_beams(index: generator.engraving.notation.ScoreIndex, anchors: list[generator.engraving.types.Anchor], items: list[generator.engraving.types.Primitive], start: fractions.Fraction, end: fractions.Fraction, left: float, right: float, style: generator.engraving.types.Style) -> list[generator.engraving.types.Primitive]:
additions = []
removed = set()
for group in beam_groups(index):
@@ -65,7 +65,7 @@ def draw_beams(index: ScoreIndex, anchors: list[Anchor], items: list[Primitive],
x1 = left if index.by_id[group[0]].start < start else first.stem_x
x2 = right if index.by_id[group[-1]].start >= end else last.stem_x
for anchor in selected:
additions.append(line('beam-stem', (anchor.event,), ((anchor.stem_x, anchor.stem_y), (anchor.stem_x, beam_y)), 1.4))
additions.append(generator.engraving.types.line('beam-stem', (anchor.event,), ((anchor.stem_x, anchor.stem_y), (anchor.stem_x, beam_y)), 1.4))
for level in range(1, max(a.flags for a in selected) + 1):
y = beam_y - direction * (level - 1) * style.space * 0.75
for i, anchor in enumerate(selected):
@@ -88,18 +88,18 @@ def draw_beams(index: ScoreIndex, anchors: list[Anchor], items: list[Primitive],
b = a + (8 if next_anchor else -8)
else:
continue
additions.append(polygon('beam', sources, ((a, y), (b, y), (b, y - direction * style.space * 0.5), (a, y - direction * style.space * 0.5))))
additions.append(generator.engraving.types.polygon('beam', sources, ((a, y), (b, y), (b, y - direction * style.space * 0.5), (a, y - direction * style.space * 0.5))))
if selected and x1 < selected[0].stem_x and selected[0].flags >= level:
additions.append(line('beam', sources, ((x1, y), (selected[0].stem_x, y)), style.space * 0.5))
additions.append(generator.engraving.types.line('beam', sources, ((x1, y), (selected[0].stem_x, y)), style.space * 0.5))
return [p for p in items if not (p.role == 'flag' and any(s in removed for s in p.sources))] + additions
def draw_spans(index: ScoreIndex, anchors: list[Anchor], items: list[Primitive], start: Fraction, end: Fraction, left: float, right: float, fonts: Fonts, style: Style) -> list[Primitive]:
def draw_spans(index: generator.engraving.notation.ScoreIndex, anchors: list[generator.engraving.types.Anchor], items: list[generator.engraving.types.Primitive], start: fractions.Fraction, end: fractions.Fraction, left: float, right: float, fonts: generator.engraving.fonts.Fonts, style: generator.engraving.types.Style) -> list[generator.engraving.types.Primitive]:
result = []
by_tone = {a.source: a for a in anchors}
by_event = {a.event: a for a in anchors}
for identifier, (event, tone) in index.tones.items():
target = tone.get('tie_to')
tone_end = event.start + fraction(tone['duration'])
tone_end = event.start + generator.model.fraction(tone['duration'])
anchor = by_tone.get(identifier)
if anchor and anchor.fragment.end < tone_end:
target = identifier
@@ -118,11 +118,11 @@ def draw_spans(index: ScoreIndex, anchors: list[Anchor], items: list[Primitive],
y1 = anchor.y + sign * 8 if anchor else reference.y + sign * 8
y2 = destination.y + sign * 8 if destination else y1
if x2 > x1:
result.append(curve('tie', (identifier, target), ((x1, y1), (x1 + (x2 - x1) / 3, y1 + sign * 9), (x2 - (x2 - x1) / 3, y2 + sign * 9), (x2, y2)), 1.6))
result.append(generator.engraving.types.curve('tie', (identifier, target), ((x1, y1), (x1 + (x2 - x1) / 3, y1 + sign * 9), (x2 - (x2 - x1) / 3, y2 + sign * 9), (x2, y2)), 1.6))
if anchor and anchor.fragment.start > event.start:
y = anchor.y - anchor.direction * 8
if anchor.x - 2 > left:
result.append(curve('tie', (identifier,), ((left, y), (left + 5, y - anchor.direction * 9), (anchor.x - 7, y - anchor.direction * 9), (anchor.x - 2, y)), 1.6))
result.append(generator.engraving.types.curve('tie', (identifier,), ((left, y), (left + 5, y - anchor.direction * 9), (anchor.x - 7, y - anchor.direction * 9), (anchor.x - 2, y)), 1.6))
top = min(-style.space * 5, min((p.box.top for p in items if p.role not in ('harmony', 'lyric', 'section', 'tempo')), default=0))
tuplet_lanes = []
for span in index.staff.get('tuplets', []):
@@ -158,33 +158,33 @@ def draw_spans(index: ScoreIndex, anchors: list[Anchor], items: list[Primitive],
if q <= p:
continue
if span.get('line_shape') == 'curved':
result.append(curve('tuplet-bracket', references, ((p, y + 3), (p + (q - p) / 3, y - 4), (q - (q - p) / 3, y - 4), (q, y + 3)), 1.3))
result.append(generator.engraving.types.curve('tuplet-bracket', references, ((p, y + 3), (p + (q - p) / 3, y - 4), (q - (q - p) / 3, y - 4), (q, y + 3)), 1.3))
else:
result.append(line('tuplet-bracket', references, ((p, y), (q, y)), 1.3))
result.append(generator.engraving.types.line('tuplet-bracket', references, ((p, y), (q, y)), 1.3))
if a.start >= start:
result.append(line('tuplet-bracket', references, ((x1, y), (x1, y + 5)), 1.3))
result.append(generator.engraving.types.line('tuplet-bracket', references, ((x1, y), (x1, y + 5)), 1.3))
if b.end <= end:
result.append(line('tuplet-bracket', references, ((x2, y), (x2, y + 5)), 1.3))
outer_top = min(top, bounds(result).top if result else top)
result.append(generator.engraving.types.line('tuplet-bracket', references, ((x2, y), (x2, y + 5)), 1.3))
outer_top = min(top, generator.engraving.types.bounds(result).top if result else top)
for span in index.staff.get('octave_shifts', []):
a, b = fraction(span['start']), fraction(span['end'])
a, b = generator.model.fraction(span['start']), generator.model.fraction(span['end'])
if b <= start or a >= end:
continue
members = [n for n in anchors if a <= n.time < b]
x1 = min((n.x for n in members), default=left) if a >= start else left
x2 = max((n.x + style.space * 2 for n in members), default=right) if b <= end else right
above = span['octaves'] < 0
y = outer_top - 18 if above else max(bounds(items + result).bottom, 0) + 24
y = outer_top - 18 if above else max(generator.engraving.types.bounds(items + result).bottom, 0) + 24
label = str(abs(span['octaves']) * 7 + 1) + ('va' if above else 'vb')
text = fonts.text(label, x1, y, 16, 'octave-shift', (span['id'],))
result.append(text)
outer_top = min(outer_top, text.box.top - 4)
cursor = text.box.right + 4
while cursor < x2:
result.append(line('octave-shift', (span['id'],), ((cursor, y - 5), (min(cursor + 6, x2), y - 5)), 1))
result.append(generator.engraving.types.line('octave-shift', (span['id'],), ((cursor, y - 5), (min(cursor + 6, x2), y - 5)), 1))
cursor += 10
if b <= end:
result.append(line('octave-shift', (span['id'],), ((x2, y - 5), (x2, y + (2 if above else -12))), 1))
result.append(generator.engraving.types.line('octave-shift', (span['id'],), ((x2, y - 5), (x2, y + (2 if above else -12))), 1))
for event in index.events:
tremolo = event.value.get('tremolo', {})
if tremolo.get('type') != 'start':
@@ -199,18 +199,18 @@ def draw_spans(index: ScoreIndex, anchors: list[Anchor], items: list[Primitive],
x1, x2 = a.stem_x + 4 if a else left, b.stem_x - 4 if b else right
y = anchor.stem_y - anchor.direction * 10
for mark in range(tremolo['marks']):
result.append(line('tremolo', (event.value['id'], partner.value['id']), ((x1, y + mark * 6), (x2, y + mark * 6)), 3.5))
result.append(generator.engraving.types.line('tremolo', (event.value['id'], partner.value['id']), ((x1, y + mark * 6), (x2, y + mark * 6)), 3.5))
return items + result
def place_text_lanes(items: list[Primitive], fonts: Fonts, style: Style) -> list[Primitive]:
def place_text_lanes(items: list[generator.engraving.types.Primitive], fonts: generator.engraving.fonts.Fonts, style: generator.engraving.types.Style) -> list[generator.engraving.types.Primitive]:
music = [p for p in items if p.role not in ('harmony', 'lyric', 'section', 'tempo')]
box = bounds(music)
box = generator.engraving.types.bounds(music)
result = list(music)
top = box.top - 12
for role in ('harmony', 'tempo', 'section'):
lane = [p for p in items if p.role == role]
if lane:
extent = bounds(lane)
extent = generator.engraving.types.bounds(lane)
result.extend(p.moved(0, top - extent.bottom) for p in lane)
top -= extent.bottom - extent.top + 10
lyrics = [p for p in items if p.role == 'lyric']
@@ -218,7 +218,7 @@ def place_text_lanes(items: list[Primitive], fonts: Fonts, style: Style) -> list
result.extend(p.moved(0, max(box.bottom, 0) + 20) for p in lyrics)
return result
def draw_lyric_spans(index: ScoreIndex, items: list[Primitive], anchors: list[Anchor], start: Fraction, end: Fraction, left: float, right: float, fonts: Fonts) -> list[Primitive]:
def draw_lyric_spans(index: generator.engraving.notation.ScoreIndex, items: list[generator.engraving.types.Primitive], anchors: list[generator.engraving.types.Anchor], start: fractions.Fraction, end: fractions.Fraction, left: float, right: float, fonts: generator.engraving.fonts.Fonts) -> list[generator.engraving.types.Primitive]:
result = list(items)
labels = {p.sources[0]: p for p in items if p.role == 'lyric'}
positions = {a.event: a for a in anchors}
@@ -232,7 +232,7 @@ def draw_lyric_spans(index: ScoreIndex, items: list[Primitive], anchors: list[An
if lyric_baseline is not None:
break
if lyric_baseline is None:
lyric_baseline = max(0, bounds(items).bottom) + 20
lyric_baseline = max(0, generator.engraving.types.bounds(items).bottom) + 20
for voice, number in index.lyric_lanes:
verses = [(e, v) for e in index.events if e.voice == voice and not e.value.get('hidden') for v in e.value.get('lyrics', []) if v['number'] == number]
for i, (event, verse) in enumerate(verses):
@@ -245,7 +245,7 @@ def draw_lyric_spans(index: ScoreIndex, items: list[Primitive], anchors: list[An
continue
target = following[0].start if following else max(e.end for e in index.events if e.voice == voice)
if 'extend_to' in verse:
target = fraction(verse['extend_to'])
target = generator.model.fraction(verse['extend_to'])
if target <= start or event.start >= end:
continue
identifier = event.value['id'] + ':lyric:' + number
@@ -262,7 +262,7 @@ def draw_lyric_spans(index: ScoreIndex, items: list[Primitive], anchors: list[An
if x2 <= x1:
continue
if extend in ('start', 'continue'):
result.append(line('lyric-extension', (identifier,), ((x1, y + 3), (x2, y + 3)), 1))
result.append(generator.engraving.types.line('lyric-extension', (identifier,), ((x1, y + 3), (x2, y + 3)), 1))
elif following:
count = max(1, int((x2 - x1) / 90))
for dash in range(count):
generator/engraving/tablature.py
+9 -9
View File
@@ -1,8 +1,8 @@
from __future__ import annotations
from generator.model import fraction
from generator.engraving.notes import rest_glyph
from generator.engraving.types import bounds, curve, line
import generator.engraving.notes
import generator.engraving.types
import generator.model
def column(index, event, anchors, fonts, style):
if not index.staff.get('tablature') or not anchors:
@@ -11,7 +11,7 @@ def column(index, event, anchors, fonts, style):
sources = (event.value['id'],)
if not event.value['pitches']:
anchor = anchors[0]
glyph = 'noteheadSlashHorizontalEnds' if anchor.fragment.kind == 'slash' else rest_glyph(anchor.fragment.written_type)
glyph = 'noteheadSlashHorizontalEnds' if anchor.fragment.kind == 'slash' else generator.engraving.notes.rest_glyph(anchor.fragment.written_type)
return [fonts.glyph(glyph, x, -2 * style.space, style.space, 'tab-' + anchor.fragment.kind, sources)]
result = []
for anchor in anchors:
@@ -26,7 +26,7 @@ def staff(index, items, start, end, left, right, fonts, style):
numbers = [p for p in items if p.role.startswith('tab')]
if not index.staff.get('tablature'):
return items
offset = bounds(notation).bottom + 32 + 5 * style.space
offset = generator.engraving.types.bounds(notation).bottom + 32 + 5 * style.space
numbers = [p.moved(0, offset) for p in numbers]
drawing = list(numbers)
drawing.append(fonts.text('TAB', 0, offset - 2 * style.space, 15, 'tab-label', ()))
@@ -36,14 +36,14 @@ def staff(index, items, start, end, left, right, fonts, style):
cursor = 0
for a, b in cuts + [(right, right)]:
if a > cursor:
drawing.append(line('tab-staff', (), ((cursor, y), (a, y)), style.space * 0.13))
drawing.append(generator.engraving.types.line('tab-staff', (), ((cursor, y), (a, y)), style.space * 0.13))
cursor = max(cursor, b)
for bar in (p for p in notation if p.role == 'barline'):
drawing.append(line('tab-barline', bar.sources, tuple((x, offset + y * 5 / 4) for x, y in bar.points), bar.thickness))
drawing.append(generator.engraving.types.line('tab-barline', bar.sources, tuple((x, offset + y * 5 / 4) for x, y in bar.points), bar.thickness))
by_source = {source: p for p in numbers if p.role == 'tab' for source in p.sources[1:]}
for identifier, (event, tone) in index.tones.items():
target = tone.get('tie_to')
finish = event.start + fraction(tone['duration'])
finish = event.start + generator.model.fraction(tone['duration'])
if not target and not (event.start < start < finish or event.start < end < finish):
continue
if target:
@@ -55,5 +55,5 @@ def staff(index, items, start, end, left, right, fonts, style):
x2 = last.box.left - 2 if last else right
y = offset - (6 - tone['tablature']['string']) * style.space + 8
if x2 > x1:
drawing.append(curve('tab-tie', (identifier,), ((x1, y), (x1 + (x2 - x1) / 3, y + 5), (x2 - (x2 - x1) / 3, y + 5), (x2, y))))
drawing.append(generator.engraving.types.curve('tab-tie', (identifier,), ((x1, y), (x1 + (x2 - x1) / 3, y + 5), (x2 - (x2 - x1) / 3, y + 5), (x2, y))))
return notation + drawing
generator/engraving/types.py
+22 -22
View File
@@ -1,13 +1,13 @@
from __future__ import annotations
from dataclasses import dataclass, replace
from fractions import Fraction
import dataclasses
import fractions
import math
class EngravingError(ValueError):
pass
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class Style:
width: int = 976
height: int = 484
@@ -18,7 +18,7 @@ class Style:
column_gap: float = 10
system_gap: float = 26
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class Box:
left: float
top: float
@@ -31,7 +31,7 @@ class Box:
def finite(self) -> bool:
return all(math.isfinite(v) for v in (self.left, self.top, self.right, self.bottom)) and self.left <= self.right and self.top <= self.bottom
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class Primitive:
kind: str
role: str
@@ -42,30 +42,30 @@ class Primitive:
size: float = 0
thickness: float = 0
def moved(self, x: float, y: float) -> Primitive:
return replace(self, box=self.box.moved(x, y), points=tuple((a + x, b + y) for a, b in self.points))
return dataclasses.replace(self, box=self.box.moved(x, y), points=tuple((a + x, b + y) for a, b in self.points))
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class Fragment:
source: str
event: str
start: Fraction
end: Fraction
start: fractions.Fraction
end: fractions.Fraction
written_type: str
dots: int
ratio: Fraction
ratio: fractions.Fraction
visible: bool
kind: str
position: float | None
def length(self) -> Fraction:
def length(self) -> fractions.Fraction:
return self.end - self.start
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class Anchor:
source: str
event: str
voice: str
time: Fraction
end: Fraction
time: fractions.Fraction
end: fractions.Fraction
x: float
y: float
stem_x: float
@@ -74,18 +74,18 @@ class Anchor:
flags: int
fragment: Fragment
def moved(self, x: float, y: float) -> Anchor:
return replace(self, x=self.x + x, y=self.y + y, stem_x=self.stem_x + x, stem_y=self.stem_y + y)
return dataclasses.replace(self, x=self.x + x, y=self.y + y, stem_x=self.stem_x + x, stem_y=self.stem_y + y)
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class System:
start: Fraction
end: Fraction
start: fractions.Fraction
end: fractions.Fraction
primitives: tuple[Primitive, ...]
anchors: tuple[Anchor, ...]
fragments: tuple[Fragment, ...]
box: Box
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class Page:
number: int
width: int
@@ -94,7 +94,7 @@ class Page:
systems: tuple[System, ...]
primitives: tuple[Primitive, ...]
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class LayoutDocument:
pages: tuple[Page, ...]
style: Style
@@ -111,7 +111,7 @@ def line(role: str, sources: tuple[str, ...], points: tuple[tuple[float, float],
return Primitive('line', role, sources, box, points, thickness=thickness)
def curve(role: str, sources: tuple[str, ...], points: tuple[tuple[float, float], ...], thickness: float = 1.6) -> Primitive:
return replace(line(role, sources, points, thickness), kind='curve')
return dataclasses.replace(line(role, sources, points, thickness), kind='curve')
def polygon(role: str, sources: tuple[str, ...], points: tuple[tuple[float, float], ...]) -> Primitive:
return replace(line(role, sources, points, 0), kind='polygon')
return dataclasses.replace(line(role, sources, points, 0), kind='polygon')
generator/engraving/validation.py
+43
View File
@@ -0,0 +1,43 @@
from __future__ import annotations
import hashlib
import io
import pathlib
import re
import PIL.Image
import PIL.ImageChops
import generator.engraving.types
ENGINE_NAME = 'Vamp Python'
PAGE_WIDTH = 976
PAGE_HEIGHT = 484
def check_content(data: bytes, name: str) -> None:
with PIL.Image.open(io.BytesIO(data)) as image:
if image.format != 'PNG' or image.size != (PAGE_WIDTH, PAGE_HEIGHT):
raise generator.engraving.types.EngravingError(f'{name}: expected a {PAGE_WIDTH}x{PAGE_HEIGHT} PNG page')
image.load()
white = PIL.Image.new('RGBA', image.size, 'white')
white.alpha_composite(image.convert('RGBA'))
ink = PIL.ImageChops.difference(white.convert('RGB'), PIL.Image.new('RGB', image.size, 'white')).convert('L').point(lambda x: 255 if x > 32 else 0)
bounds = ink.getbbox()
if not bounds:
raise generator.engraving.types.EngravingError(f'{name}: engraver produced a blank page')
if bounds[0] < 4 or bounds[1] < 4 or bounds[2] > PAGE_WIDTH - 4 or bounds[3] > PAGE_HEIGHT - 4:
raise generator.engraving.types.EngravingError(f'{name}: notation reaches the page edge; refusing potentially clipped output')
def check_page(path: pathlib.Path, validated_pages: set[str] | None = None, *, expected_sha256: str | None = None) -> str:
# Trust content, never a filename or a caller-supplied checksum alone.
if path.is_symlink() or not path.is_file():
raise generator.engraving.types.EngravingError(f'{path.name}: expected a regular PNG file')
data = path.read_bytes()
digest = hashlib.sha256(data).hexdigest()
if expected_sha256 is not None and (not isinstance(expected_sha256, str) or not re.fullmatch('[0-9a-f]{64}', expected_sha256) or digest != expected_sha256):
raise generator.engraving.types.EngravingError(f'{path.name}: score page checksum mismatch')
if validated_pages is None or digest not in validated_pages:
check_content(data, path.name)
if validated_pages is not None:
validated_pages.add(digest)
return digest
generator/generate_pico_font.py
-192
View File
@@ -1,192 +0,0 @@
#!/usr/bin/env python3
import argparse
import difflib
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont, features
REPO = Path(__file__).resolve().parents[1]
FONT_PATH = REPO / "package/assets/fonts/petaluma/PetalumaScript.otf"
FONT_SIZE = 32
FONT_HEIGHT = 36
FONT_MAX_WIDTH = 28
FIRST_CODEPOINT = 32
LAST_CODEPOINT = 126
def cpp_char_literal(value: str) -> str:
if value == "'":
return "'\\''"
if value == "\\":
return "'\\\\'"
return f"'{value}'"
def render_glyph(font: ImageFont.FreeTypeFont, value: str, y_offset: int) -> tuple[list[int], int, int]:
image = Image.new("L", (FONT_MAX_WIDTH, FONT_HEIGHT), 0)
draw = ImageDraw.Draw(image)
bbox = font.getbbox(value)
x = max(0, -bbox[0])
draw.text((x, y_offset), value, font=font, fill=255)
pixels = [min(15, max(0, round(pixel / 17))) for pixel in image.get_flattened_data()]
visible = image.getbbox()
width = 0 if visible is None else min(FONT_MAX_WIDTH, visible[2])
advance = min(255, max(0, round(font.getlength(value))))
return pixels, width, advance
def render_alpha_rows(values: list[int]) -> list[str]:
lines = []
for row in range(FONT_HEIGHT):
start = row * FONT_MAX_WIDTH
row_values = values[start: start + FONT_MAX_WIDTH]
lines.append(" " + ", ".join(f"0x{value:02x}" for value in row_values) + ",")
return lines
def pair_adjustments(font: ImageFont.FreeTypeFont) -> list[tuple[int, int]]:
chars = [chr(value) for value in range(FIRST_CODEPOINT, LAST_CODEPOINT + 1)]
adjustments = []
for left in chars:
left_width = font.getlength(left)
for right in chars:
adjustment = round(font.getlength(left + right) - left_width - font.getlength(right))
if adjustment == 0:
continue
key = (ord(left) << 8) | ord(right)
adjustments.append((key, int(adjustment)))
return sorted(adjustments)
def generate() -> str:
if not features.check_feature("raqm"):
raise RuntimeError("Font generation requires Pillow with Raqm support; install libfribidi0 on Debian/Ubuntu.")
font = ImageFont.truetype(str(FONT_PATH), FONT_SIZE, layout_engine=ImageFont.Layout.RAQM)
min_top = min(font.getbbox(chr(value))[1] for value in range(FIRST_CODEPOINT, LAST_CODEPOINT + 1))
y_offset = max(0, -min_top)
adjustments = pair_adjustments(font)
lines = [
"#include <vamp_style/font.hpp>",
"",
"#include <algorithm>",
"#include <array>",
"#include <cstdint>",
"#include <string_view>",
"",
"namespace {",
"",
"struct pair_adjustment {",
" std::uint16_t key{};",
" std::int8_t adjustment{};",
"};",
"",
f"constexpr auto pair_adjustments = std::array<pair_adjustment, {len(adjustments)}>{{{{",
]
for key, adjustment in adjustments:
lines.append(f" {{0x{key:04x}, {adjustment}}},")
lines.extend(
[
"}};",
"",
"auto pair_key(std::char_t left, std::char_t right) -> std::uint16_t {",
" auto l = static_cast<std::uint16_t>(static_cast<std::uint8_t>(left));",
" auto r = static_cast<std::uint16_t>(static_cast<std::uint8_t>(right));",
" return static_cast<std::uint16_t>((l << 8U) | r);",
"}",
"",
"auto empty_glyph() -> vamp_style::font_glyph {",
" return {",
" .alpha = {},",
" .width = 0,",
" .advance = 8,",
" };",
"}",
"",
"}",
"",
"auto vamp_style::ui_font_glyph(std::char_t value) -> vamp_style::font_glyph {",
" switch (value) {",
]
)
for codepoint in range(FIRST_CODEPOINT, LAST_CODEPOINT + 1):
value = chr(codepoint)
alpha, width, advance = render_glyph(font, value, y_offset)
lines.extend(
[
f" case {cpp_char_literal(value)}:",
" return {",
" .alpha = {",
]
)
lines.extend(render_alpha_rows(alpha))
lines.extend(
[
" },",
f" .width = {width},",
f" .advance = {advance},",
" };",
]
)
lines.extend(
[
" default:",
" return empty_glyph();",
" }",
"}",
"",
"auto vamp_style::ui_font_pair_adjust(std::char_t left, std::char_t right) -> std::int32_t {",
" auto key = pair_key(left, right);",
" auto found = std::lower_bound(pair_adjustments.begin(), pair_adjustments.end(), key, [](pair_adjustment const& item, std::uint16_t value) {",
" return item.key < value;",
" });",
" if (found == pair_adjustments.end() || found->key != key) {",
" return 0;",
" }",
" return found->adjustment;",
"}",
"",
"auto vamp_style::ui_text_width(std::string_view text, std::int32_t scale) -> std::int32_t {",
" if (text.empty() || scale <= 0) {",
" return 0;",
" }",
" auto width = std::int32_t{};",
" auto previous = std::char_t{};",
" for (auto ch : text) {",
" if (previous != std::char_t{}) {",
" width += vamp_style::ui_font_pair_adjust(previous, ch) * scale;",
" }",
" width += static_cast<std::int32_t>(vamp_style::ui_font_glyph(ch).advance) * scale;",
" previous = ch;",
" }",
" return std::max(std::int32_t{}, width);",
"}",
"",
]
)
return "\n".join(lines)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--check", action="store_true")
args = parser.parse_args()
generated = generate()
if args.check:
if not args.output.is_file():
parser.error(f"generated font is missing: {args.output}")
current = args.output.read_text(encoding="utf-8")
if current == generated:
print("pico font is up to date")
return 0
diff = difflib.unified_diff(
current.splitlines(),
generated.splitlines(),
fromfile=str(args.output),
tofile=f"{args.output} (generated)",
lineterm="",
)
print("\n".join(diff))
return 1
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(generated, encoding="utf-8")
print(f"generated {args.output}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
generator/guitar.py
+40 -39
View File
@@ -1,13 +1,14 @@
from __future__ import annotations
from fractions import Fraction
from dataclasses import dataclass
from functools import lru_cache
from itertools import groupby
from typing import NamedTuple
import dataclasses
import fractions
import functools
import itertools
import typing
from generator import harmony, pitch
from generator.model import NATURAL, alteration, fraction, make_event, pitch_number
import generator.harmony
import generator.model
import generator.pitch
STANDARD_TUNING = ((6, 40), (5, 45), (4, 50), (3, 55), (2, 59), (1, 64))
MIN_FRET = 0
@@ -15,7 +16,7 @@ MAX_FRET = 20
MAX_FRET_SPAN = 4
WRITTEN_GUITAR_OCTAVE_OFFSET = 12
class TabPosition(NamedTuple):
class TabPosition(typing.NamedTuple):
string_number: int
fret: int
midi: int
@@ -24,7 +25,7 @@ def within_fret_span(positions) -> bool:
fretted = [position.fret for position in positions if position.fret > 0]
return not fretted or max(fretted) - min(fretted) <= MAX_FRET_SPAN
def positions_for_pitch(midi: Fraction) -> tuple[TabPosition, ...]:
def positions_for_pitch(midi: fractions.Fraction) -> tuple[TabPosition, ...]:
if midi.denominator != 1:
return ()
return tuple(TabPosition(string, int(midi - open_midi), int(midi)) for string, open_midi in STANDARD_TUNING if MIN_FRET <= midi - open_midi <= MAX_FRET)
@@ -35,20 +36,20 @@ def validate_tuning(state: dict, location: str) -> None:
raise ValueError(f'{location}: guitar generation does not support capo configurations')
tunings = details.get('tunings', [])
expected = {7 - string: midi for string, midi in STANDARD_TUNING}
if tunings and (len(tunings) != 6 or {t['line']: pitch_number(t) for t in tunings} != expected):
if tunings and (len(tunings) != 6 or {t['line']: generator.model.pitch_number(t) for t in tunings} != expected):
raise ValueError(f'{location}: guitar generation requires standard six-string tuning')
def chord_requirements(definition: dict) -> tuple[tuple[int, ...], tuple[int, ...], int | None]:
degrees = harmony.degrees(definition)
degrees = generator.harmony.degrees(definition)
root = definition['root']
root_pc = NATURAL[root['letter']] + alteration(root['alter'])
root_pc = generator.model.NATURAL[root['letter']] + generator.model.alteration(root['alter'])
explicit = {d['value'] for d in definition.get('degrees', []) if d['type'] != 'subtract'}
optional = {d for d, a in degrees.items() if not a and d not in explicit and (d == 5 and len(degrees) > 3 or d in (9, 11) and d < max(degrees))}
classes = {d: (root_pc + harmony.degree_semitones(d, a)) % 12 for d, a in degrees.items()}
classes = {d: (root_pc + generator.harmony.degree_semitones(d, a)) % 12 for d, a in degrees.items()}
required = {value for d, value in classes.items() if d not in optional}
allowed = set(classes.values())
bass = definition.get('bass')
bass_pc = (NATURAL[bass['letter']] + alteration(bass['alter'])) % 12 if bass else None
bass_pc = (generator.model.NATURAL[bass['letter']] + generator.model.alteration(bass['alter'])) % 12 if bass else None
if bass_pc is None and 'inversion' in definition:
ordered = sorted(degrees)
inversion = definition['inversion']
@@ -60,7 +61,7 @@ def chord_requirements(definition: dict) -> tuple[tuple[int, ...], tuple[int, ..
required.add(bass_pc)
return tuple(sorted(required)), tuple(sorted(allowed)), bass_pc
@lru_cache(maxsize=256)
@functools.lru_cache(maxsize=256)
def chord_positions(required: tuple[int, ...], allowed: tuple[int, ...], bass: int | None, prune: bool = True) -> tuple[tuple[TabPosition, ...], ...]:
result = []
options = [tuple(TabPosition(string, fret, open_midi + fret) for fret in range(MIN_FRET, MAX_FRET + 1) if (open_midi + fret) % 12 in allowed) for string, open_midi in STANDARD_TUNING]
@@ -92,26 +93,26 @@ def chord_positions(required: tuple[int, ...], allowed: tuple[int, ...], bass: i
def chord_pitches(definition: dict, positions: tuple[TabPosition, ...]) -> list[dict]:
spellings = {}
for degree, alter in harmony.degrees(definition).items():
value = pitch.shift_pitch(definition['root'], degree - 1, harmony.degree_semitones(degree, alter))
spellings[(NATURAL[value['letter']] + alteration(value['alter'])) % 12] = value
for degree, alter in generator.harmony.degrees(definition).items():
value = generator.pitch.shift_pitch(definition['root'], degree - 1, generator.harmony.degree_semitones(degree, alter))
spellings[(generator.model.NATURAL[value['letter']] + generator.model.alteration(value['alter'])) % 12] = value
if bass := definition.get('bass'):
spellings[(NATURAL[bass['letter']] + alteration(bass['alter'])) % 12] = bass
spellings[(generator.model.NATURAL[bass['letter']] + generator.model.alteration(bass['alter'])) % 12] = bass
result = []
for position in sorted(positions, key=lambda p: p.midi):
value = spellings[position.midi % 12].copy()
value['octave'] = int((position.midi + WRITTEN_GUITAR_OCTAVE_OFFSET - NATURAL[value['letter']] - alteration(value['alter'])) // 12 - 1)
value['octave'] = int((position.midi + WRITTEN_GUITAR_OCTAVE_OFFSET - generator.model.NATURAL[value['letter']] - generator.model.alteration(value['alter'])) // 12 - 1)
value['tablature'] = {'string': position.string_number, 'fret': position.fret}
result.append(value)
return result
@lru_cache(maxsize=16384)
@functools.lru_cache(maxsize=16384)
def movement(previous: tuple[TabPosition, ...], current: tuple[TabPosition, ...]) -> int:
if not previous or not current:
return 0
return sum(abs(a - b) for a, b in zip(position_center(previous), position_center(current)))
@lru_cache(maxsize=4096)
@functools.lru_cache(maxsize=4096)
def position_center(positions: tuple[TabPosition, ...]) -> tuple[int, int]:
frets = [p.fret for p in positions if p.fret] or [0]
strings = [p.string_number for p in positions]
@@ -131,7 +132,7 @@ def extend_positions(held, pending):
if position.string_number not in occupied and within_fret_span((*held.values(), position)):
yield from extend_positions({**held, identifier: position}, pending[1:])
@dataclass
@dataclasses.dataclass
class Placement:
cost: tuple[int, int, int]
active: tuple
@@ -169,13 +170,13 @@ def nearest_positions(states: dict) -> list[list[tuple | None]]:
relax(x, y, x, y + 1)
return grid
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class GuitarTone:
pitch: dict
end: Fraction
end: fractions.Fraction
positions: tuple[TabPosition, ...]
@dataclass(frozen=True)
@dataclasses.dataclass(frozen=True)
class AttackCandidate:
active: tuple
choices: tuple
@@ -208,38 +209,38 @@ def attack_candidates(notes: list[dict], tones: dict[str, GuitarTone], active: d
def guitar_events(document: dict) -> tuple[list, dict[str, GuitarTone]]:
events, tones = [], {}
absolute = Fraction()
absolute = fractions.Fraction()
for measure in document['staves'][0]['measures']:
changes = sorted(measure.get('attributes', []), key=lambda a: fraction(a['onset']))
changes = sorted(measure.get('attributes', []), key=lambda a: generator.model.fraction(a['onset']))
state = {k: measure[k] for k in ('transpose', 'staff_details')}
for change in [state, *changes]:
state.update({k: change[k] for k in state if k in change})
validate_tuning(state, measure['id'])
for voice in measure['voices']:
for order, note in enumerate(voice['notes']):
onset = fraction(note['onset'])
onset = generator.model.fraction(note['onset'])
state = {k: measure[k] for k in ('transpose', 'staff_details')}
for change in changes:
if fraction(change['onset']) <= onset:
if generator.model.fraction(change['onset']) <= onset:
state.update({k: change[k] for k in state if k in change})
for tone in note['pitches']:
sounding = pitch.sounding_pitch(tone, state)
sounding = generator.pitch.sounding_pitch(tone, state)
options = positions_for_pitch(sounding)
if not options:
raise ValueError(f'{measure["id"]}, event {note["id"]}: sounding pitch {sounding} has no string/fret in {MIN_FRET}-{MAX_FRET}')
tones[tone['id']] = GuitarTone(tone, absolute + onset + fraction(tone['duration']), options)
tones[tone['id']] = GuitarTone(tone, absolute + onset + generator.model.fraction(tone['duration']), options)
if '_harmony' in note:
for string, _ in STANDARD_TUNING:
tones[f'{note["id"]}-s{string}'] = GuitarTone({}, absolute + onset + fraction(note['duration']), ())
tones[f'{note["id"]}-s{string}'] = GuitarTone({}, absolute + onset + generator.model.fraction(note['duration']), ())
if note['pitches'] or '_harmony' in note:
events.append((absolute + onset, not bool(note.get('grace')), order if note.get('grace') else -1, note))
absolute += fraction(measure['duration'])
absolute += generator.model.fraction(measure['duration'])
return sorted(events, key=lambda e: e[:3]), tones
def retain_tone(tone: GuitarTone, onset: Fraction) -> bool:
def retain_tone(tone: GuitarTone, onset: fractions.Fraction) -> bool:
return tone.end > onset or tone.end == onset and bool(tone.pitch.get('tie_to'))
def advance_positions(notes: list[dict], onset: Fraction, tones: dict[str, GuitarTone], states: dict, preserve_strings: bool) -> dict:
def advance_positions(notes: list[dict], onset: fractions.Fraction, tones: dict[str, GuitarTone], states: dict, preserve_strings: bool) -> dict:
continued = {p['tie_from'] for n in notes for p in n['pitches'] if 'tie_from' in p}
held_ids = {i for previous in states.values() for i, _ in previous.active if retain_tone(tones[i], onset)}
result = {}
@@ -269,17 +270,17 @@ def advance_positions(notes: list[dict], onset: Fraction, tones: dict[str, Guita
def place_guitar(document: dict) -> None:
"""Choose a minimum-cost path, retaining held strings in each search state."""
events, tones = guitar_events(document)
groups = [(key[0], [item[3] for item in values]) for key, values in groupby(events, key=lambda e: e[:3])]
groups = [(key[0], [item[3] for item in values]) for key, values in itertools.groupby(events, key=lambda e: e[:3])]
states = {(): Placement((0, 0, 0), (), None, ())}
for index, (onset, notes) in enumerate(groups):
next_onset = groups[index + 1][0] if index + 1 < len(groups) else None
preserve_strings = next_onset is not None and any('_harmony' in n and onset + fraction(n['duration']) > next_onset for n in notes)
preserve_strings = next_onset is not None and any('_harmony' in n and onset + generator.model.fraction(n['duration']) > next_onset for n in notes)
states = advance_positions(notes, onset, tones, states, preserve_strings)
selected = min(states.values(), key=lambda s: (s.cost, s.active))
while selected.previous is not None:
for note, choices in selected.choices:
if definition := note.pop('_harmony', None):
note['pitches'] = make_event(note['id'], fraction(note['onset']), fraction(note['duration']), 'note', chord_pitches(definition, choices))['pitches']
note['pitches'] = generator.model.make_event(note['id'], generator.model.fraction(note['onset']), generator.model.fraction(note['duration']), 'note', chord_pitches(definition, choices))['pitches']
else:
for tone, position in choices:
tone['tablature'] = {'string': position.string_number, 'fret': position.fret}
generator/harmony.py
+12 -12
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
import re
from generator.model import NATURAL, alteration, parse_pitch_name, pitch_name, rational
import generator.model
# Semantic degrees are independent of guitar voicing omissions.
KINDS = {
@@ -49,7 +49,7 @@ def degrees(harmony: dict) -> dict[int, int]:
raise ValueError(f'unsupported harmony kind: {kind}')
result = KINDS[kind].copy()
for change in harmony.get('degrees', []):
degree, alter, operation = change['value'], alteration(change['alter']), change['type']
degree, alter, operation = change['value'], generator.model.alteration(change['alter']), change['type']
if type(degree) is not int or degree < 1:
raise ValueError('invalid harmony degree')
if operation == 'subtract':
@@ -79,10 +79,10 @@ def pitch_classes(harmony: dict) -> tuple[int, ...]:
if harmony['kind'] == 'none':
return ()
root = harmony['root']
root_pc = NATURAL[root['letter']] + alteration(root['alter'])
root_pc = generator.model.NATURAL[root['letter']] + generator.model.alteration(root['alter'])
values = [(root_pc + degree_semitones(d, a)) % 12 for d, a in degrees(harmony).items()]
if bass := harmony.get('bass'):
values.append((NATURAL[bass['letter']] + alteration(bass['alter'])) % 12)
values.append((generator.model.NATURAL[bass['letter']] + generator.model.alteration(bass['alter'])) % 12)
return tuple(dict.fromkeys(values))
def chord_suffix(harmony: dict) -> str:
@@ -98,7 +98,7 @@ def chord_suffix(harmony: dict) -> str:
return suffix
suffix = SUFFIXES.get(kind, kind)
for change in sorted(harmony.get('degrees', []), key=lambda change: change['value']):
amount = alteration(change['alter'])
amount = generator.model.alteration(change['alter'])
sign = '#' * max(int(amount), 0) + 'b' * max(int(-amount), 0) if amount.denominator == 1 else f'({amount})'
suffix += ('no' if change['type'] == 'subtract' else ('add' if change['type'] == 'add' and not sign else '') + sign) + str(change['value'])
return suffix
@@ -107,9 +107,9 @@ def symbol(harmony: dict) -> str:
if harmony['kind'] == 'none':
return 'N.C.'
suffix = chord_suffix(harmony)
result = pitch_name(harmony['root']) + suffix
result = generator.model.pitch_name(harmony['root']) + suffix
if bass := harmony.get('bass'):
result += '/' + pitch_name(bass)
result += '/' + generator.model.pitch_name(bass)
return '(' + result + ')' if harmony.get('parenthesized') else result
def parse_symbol(text: str) -> dict:
@@ -137,11 +137,11 @@ def parse_symbol(text: str) -> dict:
operation = 'subtract' if operation == 'no' else 'alter' if degree in degrees(result) and operation != 'add' else 'add'
result['degrees'].append({'value': degree, 'alter': accidental.count('#') - accidental.count('b'), 'type': operation})
remainder = remainder[modifier.end():]
result['root'] = parse_pitch_name(root)
result['root'] = generator.model.parse_pitch_name(root)
for change in result['degrees']:
change['alter'] = rational(change['alter'])
change['alter'] = generator.model.rational(change['alter'])
if bass:
result['bass'] = parse_pitch_name(bass)
result['bass'] = generator.model.parse_pitch_name(bass)
if parenthesized:
result['parenthesized'] = True
degrees(result)
@@ -152,12 +152,12 @@ def normalize_text(harmony: dict, text: str) -> None:
parenthesized = text.startswith('(') and text.endswith(')')
content = text[1:-1] if parenthesized else text
try:
displayed = parse_symbol(pitch_name(harmony['root']) + content)
displayed = parse_symbol(generator.model.pitch_name(harmony['root']) + content)
except ValueError:
displayed = parse_symbol(content)
text_only = harmony['kind'] == 'other' and not harmony['degrees']
if displayed['root'] != harmony['root']:
if not text_only or harmony['root'] != parse_pitch_name('C'):
if not text_only or harmony['root'] != generator.model.parse_pitch_name('C'):
raise ValueError(f'contradictory harmony root in {text!r}')
if 'bass' in displayed and 'bass' in harmony and displayed['bass'] != harmony['bass']:
raise ValueError(f'contradictory harmony bass in {text!r}')
generator/model.py
+32 -32
View File
@@ -1,56 +1,56 @@
from __future__ import annotations
import copy
from fractions import Fraction
from typing import Any
import fractions
import typing
MAX_INTEGER = (1 << 63) - 1
NOTE_TYPES = ('maxima', 'long', 'breve', 'whole', 'half', 'quarter', 'eighth', '16th', '32nd', '64th', '128th', '256th', '512th', '1024th')
TYPE_LENGTHS = {name: Fraction(32, 2 ** index) for index, name in enumerate(NOTE_TYPES)}
TYPE_LENGTHS = {name: fractions.Fraction(32, 2 ** index) for index, name in enumerate(NOTE_TYPES)}
NATURAL = {'c': 0, 'd': 2, 'e': 4, 'f': 5, 'g': 7, 'a': 9, 'b': 11}
LETTERS = 'cdefgab'
MODE_STEPS = {'major': 0, 'ionian': 0, 'dorian': 1, 'phrygian': 2, 'lydian': 3, 'mixolydian': 4, 'minor': 5, 'aeolian': 5, 'locrian': 6}
ACCIDENTALS = {'natural': Fraction(0), 'sharp': Fraction(1), 'flat': Fraction(-1), 'double-sharp': Fraction(2), 'sharp-sharp': Fraction(2), 'flat-flat': Fraction(-2), 'triple-sharp': Fraction(3), 'triple-flat': Fraction(-3), 'natural-sharp': Fraction(1), 'natural-flat': Fraction(-1), 'quarter-sharp': Fraction(1, 2), 'quarter-flat': Fraction(-1, 2), 'three-quarters-sharp': Fraction(3, 2), 'three-quarters-flat': Fraction(-3, 2)}
ACCIDENTALS = {'natural': fractions.Fraction(0), 'sharp': fractions.Fraction(1), 'flat': fractions.Fraction(-1), 'double-sharp': fractions.Fraction(2), 'sharp-sharp': fractions.Fraction(2), 'flat-flat': fractions.Fraction(-2), 'triple-sharp': fractions.Fraction(3), 'triple-flat': fractions.Fraction(-3), 'natural-sharp': fractions.Fraction(1), 'natural-flat': fractions.Fraction(-1), 'quarter-sharp': fractions.Fraction(1, 2), 'quarter-flat': fractions.Fraction(-1, 2), 'three-quarters-sharp': fractions.Fraction(3, 2), 'three-quarters-flat': fractions.Fraction(-3, 2)}
def accidental_name(value: Fraction) -> str:
def accidental_name(value: fractions.Fraction) -> str:
for name, amount in ACCIDENTALS.items():
if amount == value:
return name
return 'other'
def rational(value: Fraction | int) -> dict[str, int]:
value = Fraction(value)
def rational(value: fractions.Fraction | int) -> dict[str, int]:
value = fractions.Fraction(value)
if abs(value.numerator) > MAX_INTEGER or value.denominator > MAX_INTEGER:
raise ValueError('musical time exceeds the supported 64-bit rational range')
return {'numerator': value.numerator, 'denominator': value.denominator}
def fraction(value: Any) -> Fraction:
def fraction(value: typing.Any) -> fractions.Fraction:
if not isinstance(value, dict) or set(value) != {'numerator', 'denominator'}:
raise ValueError('musical time must be a numerator/denominator object')
n, d = value['numerator'], value['denominator']
if type(n) is not int or type(d) is not int or d <= 0 or abs(n) > MAX_INTEGER or d > MAX_INTEGER:
raise ValueError('invalid musical time')
result = Fraction(n, d)
result = fractions.Fraction(n, d)
if rational(result) != value:
raise ValueError('musical time must be reduced')
return result
def written_length(written: dict) -> Fraction:
def written_length(written: dict) -> fractions.Fraction:
kind, dots = written['type'], written.get('dots', 0)
if kind not in TYPE_LENGTHS or type(dots) is not int or not 0 <= dots <= 16:
raise ValueError('invalid written duration or augmentation-dot count (maximum 16)')
return TYPE_LENGTHS[kind] * (2 - Fraction(1, 2 ** dots))
return TYPE_LENGTHS[kind] * (2 - fractions.Fraction(1, 2 ** dots))
def written_for(length: Fraction) -> dict:
def written_for(length: fractions.Fraction) -> dict:
for name, base in TYPE_LENGTHS.items():
for dots in range(17):
if length == base * (2 - Fraction(1, 2 ** dots)):
if length == base * (2 - fractions.Fraction(1, 2 ** dots)):
return {'type': name, 'dots': dots}
# Generated rational spans need an explicit time modification, not rounded ticks.
base = next((name for name, size in TYPE_LENGTHS.items() if size <= length), '1024th')
return {'type': base, 'dots': 0, 'ratio': rational(length / TYPE_LENGTHS[base])}
def metric_length(written: dict) -> Fraction:
def metric_length(written: dict) -> fractions.Fraction:
ratio = fraction(written.get('ratio', rational(1)))
if ratio <= 0:
raise ValueError('written duration ratio must be positive')
@@ -58,10 +58,10 @@ def metric_length(written: dict) -> Fraction:
written_length(written['normal'])
return written_length(written) * ratio
def alteration(value: int | dict) -> Fraction:
return Fraction(value) if type(value) is int else fraction(value)
def alteration(value: int | dict) -> fractions.Fraction:
return fractions.Fraction(value) if type(value) is int else fraction(value)
def pitch_number(pitch: dict) -> Fraction:
def pitch_number(pitch: dict) -> fractions.Fraction:
letter = pitch['letter'].lower()
alter, octave = alteration(pitch.get('alter', 0)), pitch['octave']
if letter not in NATURAL or type(octave) is not int or not -4 <= octave <= 16:
@@ -98,24 +98,24 @@ def key_label(key: dict) -> str:
label = pitch_name(key['tonic'])
return label if key['mode'] in ('major', 'ionian') else label + ' ' + key['mode']
def meter_length(meter: dict) -> Fraction:
return sum((Fraction(sum(group['beats']) * 4, group['unit']) for group in meter.get('groups', [])), Fraction())
def meter_length(meter: dict) -> fractions.Fraction:
return sum((fractions.Fraction(sum(group['beats']) * 4, group['unit']) for group in meter.get('groups', [])), fractions.Fraction())
def meter_label(meter: dict) -> str:
if meter.get('unmetered'):
return 'senza misura'
return '+'.join('+'.join(map(str, group['beats'])) + '/' + str(group['unit']) for group in meter['groups'])
def beat_lengths(meter: dict) -> list[Fraction]:
def beat_lengths(meter: dict) -> list[fractions.Fraction]:
result = []
for group in meter.get('groups', []):
beats, unit = group['beats'], group['unit']
if len(beats) > 1:
result.extend(Fraction(b * 4, unit) for b in beats)
result.extend(fractions.Fraction(b * 4, unit) for b in beats)
elif beats[0] > 3 and beats[0] % 3 == 0 and unit >= 8:
result.extend([Fraction(12, unit)] * (beats[0] // 3))
result.extend([fractions.Fraction(12, unit)] * (beats[0] // 3))
else:
result.extend([Fraction(4, unit)] * beats[0])
result.extend([fractions.Fraction(4, unit)] * beats[0])
return result
def validate_meter(meter: dict) -> None:
@@ -128,7 +128,7 @@ def validate_meter(meter: dict) -> None:
if any(type(b) is not int or not 0 < b <= 1024 for b in group['beats']):
raise ValueError('meter count must be between 1 and 1024')
def make_event(identifier: str, onset: Fraction, duration: Fraction, kind: str, pitches: list[dict] | None = None, written: dict | None = None) -> dict:
def make_event(identifier: str, onset: fractions.Fraction, duration: fractions.Fraction, kind: str, pitches: list[dict] | None = None, written: dict | None = None) -> dict:
if duration <= 0:
raise ValueError('generated event duration must be positive')
notation = written or written_for(duration)
@@ -155,9 +155,9 @@ def validate_document(document: dict) -> None:
raise ValueError('score instruments require unique nonempty IDs')
ids: set[str] = set()
events: dict[str, dict] = {}
tones: dict[str, tuple[dict, Fraction, Fraction, str]] = {}
tones: dict[str, tuple[dict, fractions.Fraction, fractions.Fraction, str]] = {}
ownership: dict[str, str] = {}
absolute = Fraction()
absolute = fractions.Fraction()
for measure in staves[0]['measures']:
if {'repeats', 'endings', 'initial_attributes'} & measure.keys():
raise ValueError('repeat notation is unsupported; regenerate scores from music written out in reading order')
@@ -167,7 +167,7 @@ def validate_document(document: dict) -> None:
voices = measure.get('voices', [])
if not voices or len({v['id'] for v in voices}) != len(voices) or any(not v['id'] for v in voices):
raise ValueError('measure requires uniquely identified voice lanes')
previous_ends = {v['id']: Fraction() for v in voices}
previous_ends = {v['id']: fractions.Fraction() for v in voices}
for voice, note in ((v, n) for v in voices for n in v['notes']):
if any({'sound_ties', 'printed_ties'} & pitch.keys() for pitch in note['pitches']):
raise ValueError('pass-specific tie metadata is unsupported; regenerate scores')
@@ -206,17 +206,17 @@ def validate_document(document: dict) -> None:
else:
pitch_number(p)
if staves[0].get('tablature'):
from generator.guitar import MAX_FRET, MIN_FRET, STANDARD_TUNING
from generator.pitch import sounding_pitch
import generator.guitar
import generator.pitch
tab = p.get('tablature', {})
string, fret = tab.get('string'), tab.get('fret')
if type(string) is not int or type(fret) is not int or string not in dict(STANDARD_TUNING) or not MIN_FRET <= fret <= MAX_FRET:
if type(string) is not int or type(fret) is not int or string not in dict(generator.guitar.STANDARD_TUNING) or not generator.guitar.MIN_FRET <= fret <= generator.guitar.MAX_FRET:
raise ValueError(f'{p["id"]}: invalid generated tablature position')
state = dict(measure)
for change in measure.get('attributes', []):
if fraction(change['onset']) <= onset:
state.update(change)
if dict(STANDARD_TUNING)[string] + fret != sounding_pitch(p, state):
if dict(generator.guitar.STANDARD_TUNING)[string] + fret != generator.pitch.sounding_pitch(p, state):
raise ValueError(f'{p["id"]}: tablature disagrees with sounding pitch')
pd = fraction(p['duration'])
if pd < 0 or (pd == 0 and not note.get('grace')) or pd > duration or not p['id'] or p['id'] in tones:
@@ -269,7 +269,7 @@ def validate_document(document: dict) -> None:
span_ids.add(span['id'])
for note in events.values():
for verse in note.get('lyrics', []):
start = tones[note['pitches'][0]['id']][1] if note['pitches'] else Fraction()
start = tones[note['pitches'][0]['id']][1] if note['pitches'] else fractions.Fraction()
if 'extend_to' in verse and not start < fraction(verse['extend_to']) <= absolute:
raise ValueError('lyric extension endpoint is outside the score')
if any(t not in span_ids for t in note.get('tuplet_ids', [])):
generator/musicxml.py
+68 -74
View File
@@ -1,14 +1,13 @@
from __future__ import annotations
import copy
from fractions import Fraction
from pathlib import Path
import fractions
import pathlib
from defusedxml import ElementTree
import defusedxml.ElementTree
from generator import harmony
from generator.model import (ACCIDENTALS, TYPE_LENGTHS, alteration, fraction, key_from_fifths, key_label, meter_label, meter_length, pitch_number, rational, validate_document, written_for, written_length, )
from generator.styles import canonical_style
import generator.harmony
import generator.model
class MusicXMLError(ValueError):
pass
@@ -18,11 +17,11 @@ LAYOUT = {'default-x', 'default-y', 'relative-x', 'relative-y', 'color', 'font-f
IGNORED_TOP = {'defaults', 'credit', 'work', 'movement-number', 'movement-title', 'identification', 'part-list'}
class Importer:
def __init__(self, path: Path) -> None:
def __init__(self, path: pathlib.Path) -> None:
self.path = path
self.location = str(path)
self.divisions = Fraction(1)
self.key = key_from_fifths(0)
self.divisions = fractions.Fraction(1)
self.key = generator.model.key_from_fifths(0)
self.meter = {'groups': [{'beats': [4], 'unit': 4}], 'symbol': 'normal'}
self.clef = {'sign': 'G', 'line': 2, 'octave_change': 0}
self.transpose = {'diatonic': 0, 'chromatic': 0, 'octave_change': 0}
@@ -36,7 +35,7 @@ class Importer:
self.octaves: list[dict] = []
self.voice_id = '1'
self.staff_details = {'lines': 5, 'tunings': []}
self.absolute = Fraction()
self.absolute = fractions.Fraction()
self.tremolo_starts = {}
def fail(self, text: str):
raise MusicXMLError(f'{self.location}: {text}')
@@ -48,16 +47,16 @@ class Importer:
for name in node.attrib:
if name not in LAYOUT and name not in allowed:
self.fail(f'unsupported {node.tag} attribute {name!r}')
def number(self, value: str | None, *, integer: bool = False, positive: bool = False) -> Fraction | int:
def number(self, value: str | None, *, integer: bool = False, positive: bool = False) -> fractions.Fraction | int:
try:
result = Fraction(value or '')
result = fractions.Fraction(value or '')
if integer and result.denominator != 1 or positive and result <= 0:
raise ValueError()
rational(result)
generator.model.rational(result)
return int(result) if integer else result
except (ValueError, ZeroDivisionError, OverflowError):
self.fail(f'invalid {"integer" if integer else "number"}: {value!r}')
def offset(self, node, cursor: Fraction) -> Fraction:
def offset(self, node, cursor: fractions.Fraction) -> fractions.Fraction:
return cursor + self.number(node.findtext('offset', '0')) / self.divisions
def identity(self, node) -> None:
voice = node.findtext('voice', '1' if node.tag in ('note', 'forward') else self.voice_id)
@@ -69,9 +68,9 @@ class Importer:
if not voice.strip():
self.fail('empty voice identifier')
self.voice_id = voice
def attributes(self, node, onset: Fraction) -> dict:
def attributes(self, node, onset: fractions.Fraction) -> dict:
self.children(node, {'divisions', 'key', 'time', 'clef', 'staves', 'transpose', 'instruments', 'staff-details'})
result = {'onset': rational(onset)}
result = {'onset': generator.model.rational(onset)}
for item in node:
if item.get('number', '1') != '1':
self.fail('attributes for multiple staves are not supported')
@@ -84,9 +83,9 @@ class Importer:
self.attrs(item, {'number', 'print-object'})
self.children(item, {'cancel', 'fifths', 'mode', 'key-step', 'key-alter', 'key-accidental', 'key-octave'})
if item.find('fifths') is not None:
self.key = key_from_fifths(self.number(item.findtext('fifths'), integer=True), item.findtext('mode', 'major'))
self.key = generator.model.key_from_fifths(self.number(item.findtext('fifths'), integer=True), item.findtext('mode', 'major'))
else:
self.key = {'fifths': 0, 'mode': 'none', 'tonic': None, 'alterations': [rational(0) for _ in range(7)], 'custom': True, 'steps': []}
self.key = {'fifths': 0, 'mode': 'none', 'tonic': None, 'alterations': [generator.model.rational(0) for _ in range(7)], 'custom': True, 'steps': []}
pending = None
for field in item:
if field.tag == 'key-step':
@@ -96,7 +95,7 @@ class Importer:
elif field.tag == 'key-alter':
if pending is None:
self.fail('key alteration without a step')
amount = rational(self.number(field.text))
amount = generator.model.rational(self.number(field.text))
self.key['alterations']['cdefgab'.index(pending)] = amount
self.key['steps'].append({'letter': pending, 'alter': amount})
pending = None
@@ -155,7 +154,7 @@ class Importer:
self.staff_details = {'lines': self.number(item.findtext('staff-lines', '5'), integer=True, positive=True), 'tunings': [], 'capo': self.number(item.findtext('capo', '0'), integer=True)}
for tuning in item.findall('staff-tuning'):
self.children(tuning, {'tuning-step', 'tuning-alter', 'tuning-octave'})
self.staff_details['tunings'].append({'line': self.number(tuning.get('line'), integer=True, positive=True), 'letter': tuning.findtext('tuning-step', '').lower(), 'alter': rational(self.number(tuning.findtext('tuning-alter', '0'))), 'octave': self.number(tuning.findtext('tuning-octave'), integer=True)})
self.staff_details['tunings'].append({'line': self.number(tuning.get('line'), integer=True, positive=True), 'letter': tuning.findtext('tuning-step', '').lower(), 'alter': generator.model.rational(self.number(tuning.findtext('tuning-alter', '0'))), 'octave': self.number(tuning.findtext('tuning-octave'), integer=True)})
result['staff_details'] = copy.deepcopy(self.staff_details)
elif item.tag == 'transpose':
self.children(item, {'diatonic', 'chromatic', 'octave-change'})
@@ -169,7 +168,7 @@ class Importer:
self.transpose['diatonic'] = (1 if semitones >= 0 else -1) * (7 * (magnitude // 12) + (0, 0, 1, 2, 2, 3, 3, 4, 5, 5, 6, 6)[magnitude % 12])
result['transpose'] = copy.deepcopy(self.transpose)
return result
def read_harmony(self, node, cursor: Fraction) -> dict:
def read_harmony(self, node, cursor: fractions.Fraction) -> dict:
self.attrs(node, {'print-object', 'print-frame', 'type'})
self.children(node, {'root', 'kind', 'degree', 'bass', 'inversion', 'offset', 'staff'})
if len(node.findall('kind')) != 1 or len(node.findall('root')) > 1:
@@ -177,7 +176,7 @@ class Importer:
self.identity(node)
kind = node.find('kind')
self.harmony_sequence += 1
result = {'id': f'harmony-{self.harmony_sequence}', 'onset': rational(self.offset(node, cursor)), 'kind': kind.text, 'degrees': []}
result = {'id': f'harmony-{self.harmony_sequence}', 'onset': generator.model.rational(self.offset(node, cursor)), 'kind': kind.text, 'degrees': []}
self.attrs(kind, {'text', 'use-symbols', 'stack-degrees', 'parentheses-degrees', 'bracket-degrees'})
for tag in ('root', 'bass'):
item = node.find(tag)
@@ -189,7 +188,7 @@ class Importer:
step = item.findtext(tag + '-step', '').lower()
if len(step) != 1 or step not in 'cdefgab':
self.fail('invalid harmony root/bass')
result[tag] = {'letter': step, 'alter': rational(self.number(item.findtext(tag + '-alter', '0')))}
result[tag] = {'letter': step, 'alter': generator.model.rational(self.number(item.findtext(tag + '-alter', '0')))}
for degree in node.findall('degree'):
self.children(degree, {'degree-value', 'degree-alter', 'degree-type'})
value = self.number(degree.findtext('degree-value'), integer=True, positive=True)
@@ -197,15 +196,15 @@ class Importer:
operation = degree.findtext('degree-type')
# MusicXML alterations are relative to the kind; additions use dominant intervals.
if operation == 'alter':
degree_alter += harmony.KINDS.get(result['kind'], {}).get(value, 0)
degree_alter += generator.harmony.KINDS.get(result['kind'], {}).get(value, 0)
elif operation == 'add' and (value - 1) % 7 == 6:
degree_alter -= 1
elif operation == 'subtract' and degree_alter:
self.fail('cannot subtract a degree different from the one in the chord kind')
result['degrees'].append({'value': value, 'alter': rational(degree_alter), 'type': operation})
result['degrees'].append({'value': value, 'alter': generator.model.rational(degree_alter), 'type': operation})
if kind.get('text') and result['kind'] != 'none':
harmony.normalize_text(result, kind.get('text'))
chord_degrees = harmony.degrees(result)
generator.harmony.normalize_text(result, kind.get('text'))
chord_degrees = generator.harmony.degrees(result)
if node.find('inversion') is not None:
inversion = self.number(node.findtext('inversion'), integer=True)
result['inversion'] = inversion
@@ -216,12 +215,11 @@ class Importer:
root = result['root']
letter_index = 'cdefgab'.index(root['letter']) + degree - 1
bass_letter = 'cdefgab'[letter_index % 7]
from generator.model import NATURAL
target = NATURAL[root['letter']] + alteration(root['alter']) + harmony.degree_semitones(degree, chord_degrees[degree])
result['bass'] = {'letter': bass_letter, 'alter': rational(target - (12 * (letter_index // 7) + NATURAL[bass_letter]))}
result['symbol'] = harmony.symbol(result)
target = generator.model.NATURAL[root['letter']] + generator.model.alteration(root['alter']) + generator.harmony.degree_semitones(degree, chord_degrees[degree])
result['bass'] = {'letter': bass_letter, 'alter': generator.model.rational(target - (12 * (letter_index // 7) + generator.model.NATURAL[bass_letter]))}
result['symbol'] = generator.harmony.symbol(result)
return result
def direction(self, node, cursor: Fraction, measure: dict, optional: list[Fraction]) -> None:
def direction(self, node, cursor: fractions.Fraction, measure: dict, optional: list[fractions.Fraction]) -> None:
self.attrs(node, {'print-object', 'directive', 'system'})
self.children(node, {'direction-type', 'offset', 'staff', 'voice', 'sound'})
onset = self.offset(node, cursor)
@@ -245,7 +243,7 @@ class Importer:
size = self.number(item.get('size', '8'), integer=True)
if size not in (8, 15, 22):
self.fail('octave shifts support 8, 15, or 22')
position = rational(self.absolute + onset)
position = generator.model.rational(self.absolute + onset)
if action in ('up', 'down'):
if number in self.open_octaves:
self.fail('duplicate octave-shift start')
@@ -261,20 +259,20 @@ class Importer:
elif item.tag == 'metronome':
self.children(item, {'beat-unit', 'beat-unit-dot', 'per-minute'})
written = {'type': item.findtext('beat-unit'), 'dots': len(item.findall('beat-unit-dot'))}
written_length(written)
generator.model.written_length(written)
bpm = self.number(item.findtext('per-minute'), positive=True)
measure.setdefault('tempos', []).append({'onset': rational(onset), 'written': written, 'bpm': rational(bpm)})
measure.setdefault('tempos', []).append({'onset': generator.model.rational(onset), 'written': written, 'bpm': generator.model.rational(bpm)})
sound = node.find('sound')
if sound is not None:
self.sound(sound, onset, measure)
def sound(self, node, onset: Fraction, measure: dict) -> None:
def sound(self, node, onset: fractions.Fraction, measure: dict) -> None:
if {'dacapo', 'dalsegno', 'segno', 'coda', 'tocoda', 'fine', 'forward-repeat', 'time-only'} & node.attrib.keys():
self.fail('navigation jumps are unsupported; write the music out in reading order')
self.attrs(node, {'tempo'})
self.children(node, set())
if node.get('tempo') and not any(fraction(t['onset']) == onset for t in measure.get('tempos', [])):
measure.setdefault('tempos', []).append({'onset': rational(onset), 'written': {'type': 'quarter', 'dots': 0}, 'bpm': rational(self.number(node.get('tempo'), positive=True))})
def read_note(self, node, cursor: Fraction, previous: dict | None) -> tuple[dict, Fraction, bool]:
if node.get('tempo') and not any(generator.model.fraction(t['onset']) == onset for t in measure.get('tempos', [])):
measure.setdefault('tempos', []).append({'onset': generator.model.rational(onset), 'written': {'type': 'quarter', 'dots': 0}, 'bpm': generator.model.rational(self.number(node.get('tempo'), positive=True))})
def read_note(self, node, cursor: fractions.Fraction, previous: dict | None) -> tuple[dict, fractions.Fraction, bool]:
self.attrs(node, {'print-object', 'print-dot', 'print-spacing', 'default-x', 'dynamics', 'end-dynamics'})
if node.get('dynamics') or node.get('end-dynamics'):
self.fail('note dynamics are not supported')
@@ -288,34 +286,34 @@ class Importer:
voice = self.voice_id
is_chord = node.find('chord') is not None
grace = node.find('grace')
duration = Fraction() if grace is not None else self.number(node.findtext('duration'), positive=True) / self.divisions
onset = fraction(previous['onset']) if is_chord and previous else cursor
if is_chord and (previous is None or previous['voice'] != voice or previous['kind'] != 'note' or duration > fraction(previous['duration'])):
duration = fractions.Fraction() if grace is not None else self.number(node.findtext('duration'), positive=True) / self.divisions
onset = generator.model.fraction(previous['onset']) if is_chord and previous else cursor
if is_chord and (previous is None or previous['voice'] != voice or previous['kind'] != 'note' or duration > generator.model.fraction(previous['duration'])):
self.fail('invalid chord tone or chord tone longer than its anchor')
if is_chord and (grace is not None) != ('grace' in previous):
self.fail('chord tones must share grace/regular timing')
written = {'type': node.findtext('type'), 'dots': len(node.findall('dot'))}
if written['type'] is None:
written = {'type': 'whole', 'dots': 0} if node.find('rest') is not None and node.find('rest').get('measure') == 'yes' else written_for(duration)
written_length(written)
written = {'type': 'whole', 'dots': 0} if node.find('rest') is not None and node.find('rest').get('measure') == 'yes' else generator.model.written_for(duration)
generator.model.written_length(written)
modulation = node.find('time-modification')
if modulation is not None:
self.children(modulation, {'actual-notes', 'normal-notes', 'normal-type', 'normal-dot'})
actual = self.number(modulation.findtext('actual-notes'), integer=True, positive=True)
normal = self.number(modulation.findtext('normal-notes'), integer=True, positive=True)
written['ratio'] = rational(Fraction(normal, actual))
written['ratio'] = generator.model.rational(fractions.Fraction(normal, actual))
if modulation.find('normal-type') is not None:
written['normal'] = {'type': modulation.findtext('normal-type'), 'dots': len(modulation.findall('normal-dot'))}
written_length(written['normal'])
generator.model.written_length(written['normal'])
elif modulation.findall('normal-dot'):
self.fail('<normal-dot> requires <normal-type>')
measure_rest = node.find('rest') is not None and node.find('rest').get('measure') == 'yes'
if grace is None and not measure_rest:
expected = written_length(written) * fraction(written.get('ratio', rational(1)))
expected = generator.model.written_length(written) * generator.model.fraction(written.get('ratio', generator.model.rational(1)))
if duration != expected:
self.fail(f'<note> duration {duration} disagrees with written duration/time-modification {expected}')
self.sequence += 1
note = {'id': f'note-{self.sequence}', 'voice': voice, 'kind': 'rest' if node.find('rest') is not None else 'note', 'onset': rational(onset), 'duration': rational(duration), 'written': written, 'hidden': node.get('print-object') == 'no', 'pitches': []}
note = {'id': f'note-{self.sequence}', 'voice': voice, 'kind': 'rest' if node.find('rest') is not None else 'note', 'onset': generator.model.rational(onset), 'duration': generator.model.rational(duration), 'written': written, 'hidden': node.get('print-object') == 'no', 'pitches': []}
if is_chord:
note = previous
if grace is not None and not is_chord:
@@ -324,7 +322,7 @@ class Importer:
for name in ('steal-time-previous', 'steal-time-following', 'make-time'):
if name in grace.attrib:
value = self.number(grace.get(name), positive=True)
note['grace'][name] = rational(value / self.divisions if name == 'make-time' else value)
note['grace'][name] = generator.model.rational(value / self.divisions if name == 'make-time' else value)
if node.find('cue') is not None and not is_chord:
note['cue'] = True
if node.find('rest') is not None:
@@ -335,12 +333,12 @@ class Importer:
note['rest_position'] = {'letter': node.findtext('rest/display-step').lower(), 'octave': self.number(node.findtext('rest/display-octave'), integer=True)}
else:
pitch_node = node.find('pitch')
p = {'id': f'{note["id"]}-p{len(note["pitches"]) + 1}', 'duration': rational(duration), 'written': written}
p = {'id': f'{note["id"]}-p{len(note["pitches"]) + 1}', 'duration': generator.model.rational(duration), 'written': written}
if pitch_node is not None:
self.children(pitch_node, {'step', 'alter', 'octave'})
p.update({'letter': pitch_node.findtext('step', '').lower(), 'alter': rational(self.number(pitch_node.findtext('alter', '0'))), 'octave': self.number(pitch_node.findtext('octave'), integer=True)})
pitch_number(p)
value = (voice, p['letter'], alteration(p['alter']), p['octave'])
p.update({'letter': pitch_node.findtext('step', '').lower(), 'alter': generator.model.rational(self.number(pitch_node.findtext('alter', '0'))), 'octave': self.number(pitch_node.findtext('octave'), integer=True)})
generator.model.pitch_number(p)
value = (voice, p['letter'], generator.model.alteration(p['alter']), p['octave'])
elif node.find('unpitched') is not None:
self.children(node.find('unpitched'), {'display-step', 'display-octave'})
p['unpitched'] = {'letter': node.findtext('unpitched/display-step', 'B').lower(), 'octave': self.number(node.findtext('unpitched/display-octave', '4'), integer=True)}
@@ -362,7 +360,7 @@ class Importer:
accidental = node.find('accidental')
if accidental is not None:
self.attrs(accidental, {'cautionary', 'editorial', 'parentheses', 'bracket', 'size', 'smufl'})
if 'unpitched' in p or accidental.text in ACCIDENTALS and ACCIDENTALS[accidental.text] != alteration(p['alter']):
if 'unpitched' in p or accidental.text in generator.model.ACCIDENTALS and generator.model.ACCIDENTALS[accidental.text] != generator.model.alteration(p['alter']):
self.fail('contradictory accidental')
p['accidental_display'] = {'name': accidental.text, **{k: v for k, v in accidental.attrib.items() if k not in LAYOUT}}
if node.find('notehead') is not None:
@@ -384,7 +382,7 @@ class Importer:
self.fail('unsupported tie notation')
if 'stop' in ties:
source = self.open_ties.pop(value, None)
if source is None or source['_absolute'] + fraction(source['duration']) != p['_absolute']:
if source is None or source['_absolute'] + generator.model.fraction(source['duration']) != p['_absolute']:
self.fail('tie stop has no contiguous matching start')
source['tie_to'], p['tie_from'] = p['id'], source['id']
if 'start' in ties:
@@ -491,7 +489,7 @@ class Importer:
self.active_tuplets.pop(number)['end'] = note['id']
return note, cursor if is_chord else cursor + duration, is_chord
def read(self) -> dict:
root = ElementTree.parse(self.path, forbid_entities=True, forbid_external=True).getroot()
root = defusedxml.ElementTree.parse(self.path, forbid_entities=True, forbid_external=True).getroot()
if root.tag != 'score-partwise' or root.get('version') not in ('3.1', '4.0'):
self.fail('expected uncompressed score-partwise MusicXML 3.1 or 4.0')
self.children(root, IGNORED_TOP | {'part'})
@@ -519,21 +517,18 @@ class Importer:
instruments.append(instrument)
if any(item.get('id') not in {i['id'] for i in instruments} for item in definitions[0].findall('midi-instrument')):
self.fail('MIDI definition refers to an unknown score instrument')
fields = {}
for item in root.findall('identification/miscellaneous/miscellaneous-field'):
name = item.get('name', '')
if name.startswith('vamp:'):
if name != 'vamp:style' or name in fields:
self.fail(f'unknown or duplicate Vamp property {name!r}')
fields[name] = item.text or ''
metadata = {'title': root.findtext('work/work-title') or root.findtext('movement-title') or self.path.stem, 'composer': ' / '.join(c.text for c in root.findall('identification/creator') if c.get('type') == 'composer' and c.text), 'style': canonical_style(fields.get('vamp:style'))}
self.fail(f'unsupported Vamp property {name!r}')
metadata = {'title': root.findtext('work/work-title') or root.findtext('movement-title') or self.path.stem, 'composer': ' / '.join(c.text for c in root.findall('identification/creator') if c.get('type') == 'composer' and c.text)}
measures = []
self.children(parts[0], {'measure'})
for index, bar in enumerate(parts[0]):
self.location = f'{self.path}: part {parts[0].get("id")}, measure {bar.get("number", str(index + 1))}'
self.children(bar, {'attributes', 'note', 'direction', 'sound', 'harmony', 'forward', 'backup', 'barline', 'print'})
self.attrs(bar, {'number', 'implicit', 'non-controlling', 'width', 'text'})
cursor, extent = Fraction(), Fraction()
cursor, extent = fractions.Fraction(), fractions.Fraction()
voices = {}
previous = None
optional = []
@@ -567,10 +562,9 @@ class Importer:
self.children(item, {'duration', 'voice', 'staff'})
self.identity(item)
duration = self.number(item.findtext('duration'), positive=True) / self.divisions
from generator.model import make_event
if item.find('voice') is not None:
self.sequence += 1
voices.setdefault(self.voice_id, []).append(make_event(f'spacer-{self.sequence}', cursor, duration, 'spacer') | {'voice': self.voice_id})
voices.setdefault(self.voice_id, []).append(generator.model.make_event(f'spacer-{self.sequence}', cursor, duration, 'spacer') | {'voice': self.voice_id})
cursor += duration
extent = max(extent, cursor)
previous = None
@@ -597,22 +591,22 @@ class Importer:
cursor = extent
if measure.pop('_right_barline_onset', cursor) != cursor:
self.fail('right barline must occur at the end of its measure')
expected = meter_length(measure['meter'])
expected = generator.model.meter_length(measure['meter'])
meter_changed = any('meter' in a for a in measure['attributes'])
if expected and cursor != expected and not measure['implicit'] and not meter_changed:
self.fail(f'measure has {cursor} quarter notes, expected {expected}; mark a partial measure implicit')
measure['duration'] = rational(cursor)
measure['key_signature'], measure['time_signature'] = key_label(measure['key']), meter_label(measure['meter'])
measure['voices'] = [{'id': voice, 'notes': sorted(events, key=lambda n: fraction(n['onset']))} for voice, events in voices.items()]
measure['harmonies'].sort(key=lambda h: fraction(h['onset']))
measure['duration'] = generator.model.rational(cursor)
measure['key_signature'], measure['time_signature'] = generator.model.key_label(measure['key']), generator.model.meter_label(measure['meter'])
measure['voices'] = [{'id': voice, 'notes': sorted(events, key=lambda n: generator.model.fraction(n['onset']))} for voice, events in voices.items()]
measure['harmonies'].sort(key=lambda h: generator.model.fraction(h['onset']))
used = set()
for onset in optional:
matches = [h for h in measure['harmonies'] if fraction(h['onset']) == onset]
matches = [h for h in measure['harmonies'] if generator.model.fraction(h['onset']) == onset]
if len(matches) != 1 or onset in used:
self.fail('optional-chord marker must identify exactly one harmony and occur only once')
matches[0]['chord_optional'] = True
matches[0]['parenthesized'] = True
matches[0]['symbol'] = harmony.symbol(matches[0])
matches[0]['symbol'] = generator.harmony.symbol(matches[0])
used.add(onset)
measures.append(measure)
self.absolute += cursor
@@ -627,11 +621,11 @@ class Importer:
metadata.update({'key_signature': measures[0]['key_signature'], 'time_signature': measures[0]['time_signature'], 'key': copy.deepcopy(measures[0]['key'])})
result = {'metadata': metadata, 'staves': [{'id': 'staff-source', 'name': definitions[0].findtext('part-name', 'Source'), 'role': 'vocal', 'clef': copy.deepcopy(measures[0]['clef']), 'tuplets': self.tuplets, 'octave_shifts': self.octaves, 'measures': measures}]}
result['staves'][0]['instruments'] = instruments
validate_document(result)
generator.model.validate_document(result)
return result
def load_musicxml(path: Path) -> dict:
importer = Importer(Path(path))
def load_musicxml(path: pathlib.Path) -> dict:
importer = Importer(pathlib.Path(path))
try:
return importer.read()
except MusicXMLError:
generator/musicxml_writer.py
+44 -46
View File
@@ -1,21 +1,21 @@
from __future__ import annotations
from fractions import Fraction
from math import lcm
from pathlib import Path
from xml.etree import ElementTree
import fractions
import math
import pathlib
import xml.etree.ElementTree
from generator import harmony
from generator.model import alteration, fraction, validate_document
import generator.harmony
import generator.model
def element(parent, tag, content=None, **attributes):
node = ElementTree.SubElement(parent, tag, {k.replace('_', '-'): str(v) for k, v in attributes.items() if v is not None})
node = xml.etree.ElementTree.SubElement(parent, tag, {k.replace('_', '-'): str(v) for k, v in attributes.items() if v is not None})
if content is not None:
node.text = str(content)
return node
def decimal(value) -> str:
value = alteration(value) if isinstance(value, dict) else Fraction(value)
value = generator.model.alteration(value) if isinstance(value, dict) else fractions.Fraction(value)
denominator = value.denominator
for prime in (2, 5):
while denominator % prime == 0:
@@ -37,7 +37,7 @@ def write_attributes(parent, state: dict, divisions=None) -> None:
if 'cancel' in key:
element(item, 'cancel', key['cancel'])
if key.get('custom'):
steps = key.get('steps', [{'letter': letter, 'alter': amount} for letter, amount in zip('cdefgab', key['alterations']) if alteration(amount)])
steps = key.get('steps', [{'letter': letter, 'alter': amount} for letter, amount in zip('cdefgab', key['alterations']) if generator.model.alteration(amount)])
for step in steps:
element(item, 'key-step', step['letter'].upper())
element(item, 'key-alter', decimal(step['alter']))
@@ -85,7 +85,7 @@ def write_harmony(parent, value: dict, divisions: int) -> None:
item = element(node, 'root')
element(item, 'root-step', root['letter'].upper())
element(item, 'root-alter', decimal(root['alter']))
element(node, 'kind', value['kind'], text=harmony.symbol(value) if value.get('parenthesized') else None)
element(node, 'kind', value['kind'], text=generator.harmony.symbol(value) if value.get('parenthesized') else None)
if 'inversion' in value:
element(node, 'inversion', value['inversion'])
if 'bass' in value:
@@ -94,20 +94,20 @@ def write_harmony(parent, value: dict, divisions: int) -> None:
element(item, 'bass-alter', decimal(value['bass']['alter']))
for degree in value['degrees']:
item = element(node, 'degree')
amount = alteration(degree['alter'])
amount = generator.model.alteration(degree['alter'])
if degree['type'] == 'alter':
amount -= harmony.KINDS[value['kind']].get(degree['value'], 0)
amount -= generator.harmony.KINDS[value['kind']].get(degree['value'], 0)
elif degree['type'] == 'add' and (degree['value'] - 1) % 7 == 6:
amount += 1
element(item, 'degree-value', degree['value'])
element(item, 'degree-alter', decimal(amount))
element(item, 'degree-type', degree['type'])
element(node, 'offset', int(fraction(value['onset']) * divisions))
element(node, 'offset', int(generator.model.fraction(value['onset']) * divisions))
def write_note(parent, event: dict, voice: str, divisions: int, tuplets: list[dict]) -> None:
if event['kind'] == 'spacer':
item = element(parent, 'forward')
element(item, 'duration', int(fraction(event['duration']) * divisions))
element(item, 'duration', int(generator.model.fraction(event['duration']) * divisions))
element(item, 'voice', voice)
return
pitches = event['pitches'] or [None]
@@ -117,7 +117,7 @@ def write_note(parent, event: dict, voice: str, divisions: int, tuplets: list[di
printing.update(pitch.get('print', {}))
node = element(parent, 'note', **printing)
if grace := event.get('grace'):
element(node, 'grace', **{k: ('yes' if v else 'no') if isinstance(v, bool) else decimal(fraction(v) * divisions if k == 'make-time' else v) for k, v in grace.items()})
element(node, 'grace', **{k: ('yes' if v else 'no') if isinstance(v, bool) else decimal(generator.model.fraction(v) * divisions if k == 'make-time' else v) for k, v in grace.items()})
if (pitch or event).get('cue', event.get('cue')):
element(node, 'cue')
if index:
@@ -137,7 +137,7 @@ def write_note(parent, event: dict, voice: str, divisions: int, tuplets: list[di
element(item, 'alter', decimal(pitch['alter']) if pitch else '0')
element(item, 'octave', pitch['octave'] if pitch else 4)
if not event.get('grace'):
element(node, 'duration', int(fraction(pitch['duration'] if pitch else event['duration']) * divisions))
element(node, 'duration', int(generator.model.fraction(pitch['duration'] if pitch else event['duration']) * divisions))
for field, action in (('tie_from', 'stop'), ('tie_to', 'start')):
if pitch and pitch.get(field):
element(node, 'tie', type=action)
@@ -151,7 +151,7 @@ def write_note(parent, event: dict, voice: str, divisions: int, tuplets: list[di
if pitch and (display := pitch.get('accidental_display')):
element(node, 'accidental', display['name'], **{k: v for k, v in display.items() if k != 'name'})
if 'ratio' in written:
ratio = fraction(written['ratio'])
ratio = generator.model.fraction(written['ratio'])
item = element(node, 'time-modification')
element(item, 'actual-notes', ratio.denominator)
element(item, 'normal-notes', ratio.numerator)
@@ -204,15 +204,13 @@ def write_barline(bar, measure, location):
node = element(bar, 'barline', location=location)
element(node, 'bar-style', measure.get('barlines', {}).get(location, 'regular'))
def musicxml_tree(document: dict) -> ElementTree.ElementTree:
validate_document(document)
root = ElementTree.Element('score-partwise', {'version': '4.0'})
def musicxml_tree(document: dict) -> xml.etree.ElementTree.ElementTree:
generator.model.validate_document(document)
root = xml.etree.ElementTree.Element('score-partwise', {'version': '4.0'})
metadata, staff = document['metadata'], document['staves'][0]
element(element(root, 'work'), 'work-title', metadata['title'])
identification = element(root, 'identification')
element(identification, 'creator', metadata.get('composer', ''), type='composer')
miscellaneous = element(identification, 'miscellaneous')
element(miscellaneous, 'miscellaneous-field', metadata.get('style', 'Medium Swing'), name='vamp:style')
defaults = element(root, 'defaults')
element(defaults, 'music-font', font_family='Petaluma')
element(defaults, 'word-font', font_family='Petaluma Script')
@@ -240,23 +238,23 @@ def musicxml_tree(document: dict) -> ElementTree.ElementTree:
for tag, value in instrument['midi'].items():
element(item, tag, value)
part = element(root, 'part', id='P1')
absolute = Fraction()
absolute = fractions.Fraction()
previous_state = {}
for index, measure in enumerate(staff['measures']):
bar = element(part, 'measure', number=index + 1, implicit='yes' if measure['implicit'] else 'no')
write_barline(bar, measure, 'left')
times = [fraction(measure['duration'])]
times.extend(fraction(n[field]) for v in measure['voices'] for n in v['notes'] for field in ('onset', 'duration'))
times.extend(fraction(p['duration']) for v in measure['voices'] for n in v['notes'] for p in n['pitches'])
times.extend(fraction(item['onset']) for field in ('attributes', 'harmonies', 'tempos') for item in measure.get(field, []))
times.extend(fraction(s[field]) - absolute for s in staff.get('octave_shifts', []) for field in ('start', 'end'))
divisions = lcm(*(t.denominator for t in times))
times = [generator.model.fraction(measure['duration'])]
times.extend(generator.model.fraction(n[field]) for v in measure['voices'] for n in v['notes'] for field in ('onset', 'duration'))
times.extend(generator.model.fraction(p['duration']) for v in measure['voices'] for n in v['notes'] for p in n['pitches'])
times.extend(generator.model.fraction(item['onset']) for field in ('attributes', 'harmonies', 'tempos') for item in measure.get(field, []))
times.extend(generator.model.fraction(s[field]) - absolute for s in staff.get('octave_shifts', []) for field in ('start', 'end'))
divisions = math.lcm(*(t.denominator for t in times))
state = {k: measure[k] for k in ('key', 'meter', 'clef', 'transpose', 'staff_details') if k in measure}
write_attributes(bar, {k: v for k, v in state.items() if previous_state.get(k) != v}, divisions)
previous_state.update(state)
cursor = Fraction()
cursor = fractions.Fraction()
for change in measure.get('attributes', []):
onset = fraction(change['onset'])
onset = generator.model.fraction(change['onset'])
if onset != cursor:
element(element(bar, 'forward' if onset > cursor else 'backup'), 'duration', int(abs(onset - cursor) * divisions))
write_attributes(bar, change)
@@ -269,7 +267,7 @@ def musicxml_tree(document: dict) -> ElementTree.ElementTree:
if value.get('chord_optional'):
direction = element(bar, 'direction', print_object='no')
element(element(direction, 'direction-type'), 'words', 'vamp:optional-chord')
element(direction, 'offset', int(fraction(value['onset']) * divisions))
element(direction, 'offset', int(generator.model.fraction(value['onset']) * divisions))
if measure.get('section_label'):
element(element(element(bar, 'direction'), 'direction-type'), 'rehearsal', measure['section_label'], halign='left')
for tempo in measure.get('tempos', []):
@@ -279,35 +277,35 @@ def musicxml_tree(document: dict) -> ElementTree.ElementTree:
for _ in range(tempo['written'].get('dots', 0)):
element(metronome, 'beat-unit-dot')
element(metronome, 'per-minute', decimal(tempo['bpm']))
element(direction, 'offset', int(fraction(tempo['onset']) * divisions))
element(direction, 'offset', int(generator.model.fraction(tempo['onset']) * divisions))
for si, span in enumerate(staff.get('octave_shifts', [])):
for field in ('start', 'end'):
onset = fraction(span[field]) - absolute
inside = 0 <= onset < fraction(measure['duration']) if field == 'start' else 0 < onset <= fraction(measure['duration'])
onset = generator.model.fraction(span[field]) - absolute
inside = 0 <= onset < generator.model.fraction(measure['duration']) if field == 'start' else 0 < onset <= generator.model.fraction(measure['duration'])
if not inside:
continue
direction = element(bar, 'direction')
element(element(direction, 'direction-type'), 'octave-shift', type='stop' if field == 'end' else 'up' if span['octaves'] > 0 else 'down', size=abs(span['octaves']) * 7 + 1, number=si + 1)
element(direction, 'offset', int(onset * divisions))
cursor = Fraction()
cursor = fractions.Fraction()
for voice in measure['voices']:
if cursor:
element(element(bar, 'backup'), 'duration', int(cursor * divisions))
cursor = Fraction()
cursor = fractions.Fraction()
for event in voice['notes']:
onset = fraction(event['onset'])
onset = generator.model.fraction(event['onset'])
if onset > cursor:
element(element(bar, 'forward'), 'duration', int((onset - cursor) * divisions))
write_note(bar, event, voice['id'], divisions, staff.get('tuplets', []))
cursor = onset + fraction(event['duration'])
if cursor < fraction(measure['duration']):
element(element(bar, 'forward'), 'duration', int((fraction(measure['duration']) - cursor) * divisions))
cursor = fraction(measure['duration'])
cursor = onset + generator.model.fraction(event['duration'])
if cursor < generator.model.fraction(measure['duration']):
element(element(bar, 'forward'), 'duration', int((generator.model.fraction(measure['duration']) - cursor) * divisions))
cursor = generator.model.fraction(measure['duration'])
write_barline(bar, measure, 'right')
absolute += fraction(measure['duration'])
return ElementTree.ElementTree(root)
absolute += generator.model.fraction(measure['duration'])
return xml.etree.ElementTree.ElementTree(root)
def write_musicxml(path: Path, document: dict) -> None:
def write_musicxml(path: pathlib.Path, document: dict) -> None:
tree = musicxml_tree(document)
ElementTree.indent(tree, space=' ')
xml.etree.ElementTree.indent(tree, space=' ')
tree.write(path, encoding='utf-8', xml_declaration=True)
generator/parts.py
+34 -75
View File
@@ -1,16 +1,18 @@
from __future__ import annotations
from fractions import Fraction
from pathlib import Path
import fractions
import pathlib
from generator.guitar import validate_tuning
from generator.model import beat_lengths, fraction, make_event
from generator.pitch import concert_document, shift_pitch
from generator.score import copy_source_voices, make_measure, slugify, source_staff
import generator.accompaniment
import generator.guitar
import generator.model
import generator.pitch
import generator.rhythm
import generator.score
PART_TYPES = ('acoustic', 'electric', 'vocal')
def source_paths(directory: Path, *, practice: bool = False) -> list[Path]:
def source_paths(directory: pathlib.Path, *, practice: bool = False) -> list[pathlib.Path]:
if not directory.is_dir():
raise ValueError(f'{"practice" if practice else "performance"} source directory not found: {directory}')
paths = sorted(directory.rglob('*.xml'))
@@ -21,7 +23,7 @@ def source_paths(directory: Path, *, practice: bool = False) -> list[Path]:
raise ValueError(f'{path}: practice sources require a four-digit filename directly under data/practice')
elif path.parent != directory:
raise ValueError(f'{path}: performance sources belong directly under data/songs')
identity = path.stem if practice else slugify(path.stem)
identity = path.stem if practice else generator.score.slugify(path.stem)
if identity in identities:
raise ValueError(f'{identities[identity]} and {path}: conflicting output path {identity}-acoustic.json')
identities[identity] = path
@@ -29,81 +31,38 @@ def source_paths(directory: Path, *, practice: bool = False) -> list[Path]:
def prepare_document(document: dict, *, practice: bool = False) -> dict:
validate_source(document, practice=practice)
return concert_document(document, octave_transposing=True)
def propagated_chords(measures: list[dict], *, include_optional: bool = True) -> list[dict | None]:
result = []
current = None
for measure in measures:
result.append(current)
for h in measure.get('harmonies', []):
if include_optional or not h.get('chord_optional'):
current = h
return result
def active_chord(measure: dict, onset: Fraction, previous: dict | None) -> dict | None:
active = previous
for h in measure['harmonies']:
if fraction(h['onset']) > onset:
break
active = h
return active if active and active['kind'] != 'none' else None
return generator.pitch.concert_document(document, octave_transposing=True)
def generated_voice(measure: dict, role: str, previous: dict | None) -> list[dict]:
length = fraction(measure['duration'])
length = generator.model.fraction(measure['duration'])
if role == 'rest':
return [make_event('rest', Fraction(), length, 'rest', written={'type': 'whole', 'dots': 0}) | {'measure_rest': True}]
common_time = measure['time_signature'] == '4/4' and not any('meter' in a for a in measure.get('attributes', []))
if role == 'rhythm' and common_time:
attacks = [(Fraction(n, 2), Fraction(1, 2)) for n in (0, 3, 4, 5, 7)]
elif role == 'accompaniment':
boundaries = sorted({Fraction(), length} | {fraction(h['onset']) for h in measure['harmonies'] if fraction(h['onset']) < length})
attacks = [(start, end - start) for start, end in zip(boundaries, boundaries[1:])]
else:
attacks = []
states = [(Fraction(), measure['meter'])]
states.extend((fraction(a['onset']), a['meter']) for a in measure.get('attributes', []) if 'meter' in a)
for index, (onset, meter) in enumerate(states):
end = states[index + 1][0] if index + 1 < len(states) else length
beats = beat_lengths(meter)
beat_index = 0
while onset < end:
beat = min(beats[beat_index % len(beats)] if beats else end - onset, end - onset)
duration = beat if role == 'slash' or not beats else min(beat / 2, Fraction(1, 2))
attacks.append((onset, duration))
onset += beat
beat_index += 1
return [generator.model.make_event('rest', fractions.Fraction(), length, 'rest', written={'type': 'whole', 'dots': 0}) | {'measure_rest': True}]
if role != 'slash':
raise ValueError(f'{role} requires a preplanned rhythm')
events = []
cursor = Fraction()
for onset, duration in attacks:
if onset >= length:
break
if onset > cursor:
events.append(make_event(f'rest-{len(events)}', cursor, onset - cursor, 'rest'))
duration = min(duration, length - onset)
chord = active_chord(measure, onset, previous)
if role == 'slash':
event = make_event(f'slash-{len(events)}', onset, duration, 'note')
states = [(fractions.Fraction(), measure['meter'])]
states.extend((generator.model.fraction(a['onset']), a['meter']) for a in measure.get('attributes', []) if 'meter' in a)
for index, (onset, meter) in enumerate(states):
end = states[index + 1][0] if index + 1 < len(states) else length
beats = generator.model.beat_lengths(meter)
beat_index = 0
while onset < end:
duration = min(beats[beat_index % len(beats)] if beats else end - onset, end - onset)
event = generator.model.make_event(f'slash-{len(events)}', onset, duration, 'note')
event['notehead'] = 'slash'
elif not chord:
event = make_event(f'rest-{len(events)}', onset, duration, 'rest')
else:
event = make_event(f'chord-{len(events)}', onset, duration, 'note')
event['_harmony'] = chord
events.append(event)
cursor = onset + duration
if cursor < length:
events.append(make_event(f'rest-{len(events)}', cursor, length - cursor, 'rest'))
events.append(event)
onset += duration
beat_index += 1
return events
def validate_source(document: dict, *, practice: bool = False) -> None:
voices = set()
for measure in source_staff(document)['measures']:
for measure in generator.score.source_staff(document)['measures']:
for state in [measure, *measure.get('attributes', [])]:
if state.get('clef', {}).get('sign') in ('TAB', 'percussion'):
raise ValueError(f'{measure["id"]}: parts require a pitched, non-TAB source')
if 'staff_details' in state:
validate_tuning(state, measure['id'])
generator.guitar.validate_tuning(state, measure['id'])
for voice in measure['voices']:
voices.add(voice['id'])
for note in voice['notes']:
@@ -114,13 +73,13 @@ def validate_source(document: dict, *, practice: bool = False) -> None:
if practice and len(voices) > 1:
raise ValueError('practice requires one monophonic voice')
def build_measure(source: dict, identifier: str, role: str, part: str, previous: dict | None, label: str = '') -> dict:
voices = copy_source_voices(source, strip_lyrics=role == 'melody') if role in ('vocal', 'melody') else None
notes = [n for v in voices for n in v['notes']] if voices else generated_voice(source, role, previous)
def build_measure(source: dict, identifier: str, role: str, part: str, previous: dict | None, label: str = '', rhythm: generator.rhythm.Bar | None = None) -> dict:
voices = generator.score.copy_source_voices(source, strip_lyrics=role == 'melody') if role in ('vocal', 'melody') else None
notes = [n for v in voices for n in v['notes']] if voices else generator.accompaniment.realize(source, rhythm, previous) if rhythm is not None else generated_voice(source, role, previous)
if role in ('melody', 'vocal'):
for note in notes:
note['pitches'] = [shift_pitch(p, 7, 12) for p in note['pitches']]
result = make_measure(identifier, source, [] if voices else notes, section_label=label)
note['pitches'] = [generator.pitch.shift_pitch(p, 7, 12) for p in note['pitches']]
result = generator.score.make_measure(identifier, source, [] if voices else notes, section_label=label)
if voices:
result['voices'] = voices
result['transpose'] = {'diatonic': 0, 'chromatic': 0, 'octave_change': -1, 'explicit': True}
generator/pico_artwork.py
+172
View File
@@ -0,0 +1,172 @@
"""Build-only PetalumaScript sprites and content-addressed Pico title images."""
from __future__ import annotations
import argparse
import hashlib
import json
import pathlib
import struct
import PIL.Image
import PIL.ImageDraw
import PIL.ImageFont
import PIL.features
import fontTools.ttLib
TITLE_WIDTH = 296
TITLE_HEIGHT = 64
TITLE_HEADER = struct.pack('<4sHH', b'VPTI', TITLE_WIDTH, TITLE_HEIGHT)
TITLE_BYTES = len(TITLE_HEADER) + TITLE_WIDTH * TITLE_HEIGHT // 2
MAX_TITLES = 8192
LABELS = ('VAMP', 'CONNECTING', 'OFFLINE', 'BUSY', 'READY', 'LOADING', 'WIFI', 'ERROR', 'PAGE', 'Acoustic', 'Electric', 'Vocal', 'UNAVAILABLE')
SPRITES = LABELS + tuple('0123456789/')
DIGIT_WIDTH = 22
DEFAULT_FONT_ROOT = pathlib.Path(__file__).resolve().parents[1] / 'package/assets/fonts/petaluma'
class Renderer:
def __init__(self, font_path: pathlib.Path):
if not PIL.features.check_feature('raqm'):
raise RuntimeError('Pico artwork requires Pillow with Raqm support')
self.font_path = font_path
with fontTools.ttLib.TTFont(font_path) as font:
self.characters = set(font.getBestCmap())
self.fonts = {}
def font(self, size: int) -> PIL.ImageFont.FreeTypeFont:
if size not in self.fonts:
self.fonts[size] = PIL.ImageFont.truetype(str(self.font_path), size, layout_engine=PIL.ImageFont.Layout.RAQM)
return self.fonts[size]
def validate(self, text: str) -> None:
if not isinstance(text, str) or not text.strip() or len(text) > 200:
raise ValueError('Pico title and part must contain 1 to 200 characters')
if any(ord(character) < 32 or ord(character) not in self.characters for character in text):
raise ValueError(f'Unsupported PetalumaScript text: {text!r}')
def line(self, text: str, width: int, height: int, size: int) -> PIL.Image.Image:
self.validate(text)
if width < 5 or height < 3:
raise ValueError('Pico text tile is too small')
font = self.font(size)
while size > 14:
box = font.getbbox(text)
if box[2] - box[0] <= width - 4 and box[3] - box[1] <= height - 2:
break
size -= 1
font = self.font(size)
if font.getbbox(text)[2] - font.getbbox(text)[0] > width - 4:
if font.getbbox('...')[2] - font.getbbox('...')[0] > width - 4:
raise ValueError('Pico text tile cannot fit an ellipsis')
original = text
while original and font.getbbox(original + '...')[2] - font.getbbox(original + '...')[0] > width - 4:
original = original[:-1]
text = original + '...'
box = font.getbbox(text)
if box[3] - box[1] > height - 2:
raise ValueError('Pico text tile is too short')
image = PIL.Image.new('L', (width, height))
PIL.ImageDraw.Draw(image).text(((width - box[2] + box[0]) // 2 - box[0],
(height - box[3] + box[1]) // 2 - box[1]), text, font=font, fill=255)
return image
def digit(self, text: str) -> PIL.Image.Image:
font = self.font(24)
box = font.getbbox(text)
if text not in '0123456789/' or len(text) != 1 or box[2] - box[0] > DIGIT_WIDTH - 4 or box[1] + 2 < 0 or box[3] + 2 > 28:
raise ValueError('Invalid Pico counter tile')
image = PIL.Image.new('L', (DIGIT_WIDTH, 28))
PIL.ImageDraw.Draw(image).text(((DIGIT_WIDTH - box[2] + box[0]) // 2 - box[0], 2), text, font=font, fill=255)
return image
def title(self, title: str) -> bytes:
self.validate(title)
for size in range(32, 15, -1):
lines = ['']
for word in title.split():
candidate = (lines[-1] + ' ' + word).strip()
box = self.font(size).getbbox(candidate)
if lines[-1] and box[2] - box[0] > TITLE_WIDTH - 4:
lines.append(word)
else:
lines[-1] = candidate
if len(lines) <= 2:
break
if len(lines) > 2:
lines = [lines[0], ' '.join(lines[1:])]
image = PIL.Image.new('L', (TITLE_WIDTH, TITLE_HEIGHT))
for index, line in enumerate(lines):
image.paste(self.line(line, TITLE_WIDTH, 32, size), (0, (TITLE_HEIGHT - len(lines) * 32) // 2 + index * 32))
return TITLE_HEADER + pack_coverage(image)
def pack_coverage(image: PIL.Image.Image) -> bytes:
pixels = image.tobytes()
if len(pixels) % 2:
pixels += b'\0'
levels = [(pixel + 8) // 17 for pixel in pixels]
return bytes((levels[index] << 4) | levels[index + 1] for index in range(0, len(levels), 2))
def generate_static(output: pathlib.Path, font_root: pathlib.Path = DEFAULT_FONT_ROOT) -> None:
renderer = Renderer(font_root / 'PetalumaScript.otf')
directory = bytearray(struct.pack('<4sHH', b'VPSA', len(SPRITES), 0))
payload = bytearray()
for text in SPRITES:
image = renderer.line(text, 176, 28, 24) if text in LABELS else renderer.digit(text)
if text in LABELS:
box = image.getbbox()
image = image.crop((max(0, box[0] - 1), 0, min(image.width, box[2] + 1), image.height))
directory.extend(struct.pack('<HHI', image.width, image.height, 8 + 8 * len(SPRITES) + len(payload)))
payload.extend(pack_coverage(image))
output.parent.mkdir(parents=True, exist_ok=True)
output.write_bytes(directory + payload)
def generate_titles(root: pathlib.Path, font_root: pathlib.Path = DEFAULT_FONT_ROOT) -> None:
"""Call after score generation, before catalog validation and atomic promotion.
Adds presentation.title_image to score bundles. The identifier is SHA-256 of
the complete fixed-size .mask file, shared across parts with the same title.
"""
renderer = Renderer(font_root / 'PetalumaScript.otf')
destination = root / 'pico' / 'titles'
destination.mkdir(parents=True, exist_ok=True)
titles = {}
for category in ('songs', 'practice'):
for source in sorted((root / category).rglob('*.json')):
document = json.loads(source.read_text(encoding='utf-8'))
title = document['music']['metadata']['title']
if title not in titles:
if len(titles) >= MAX_TITLES:
raise ValueError('Too many Pico titles')
image = renderer.title(title)
titles[title] = hashlib.sha256(image).hexdigest()
(destination / f'{titles[title]}.mask').write_bytes(image)
document['presentation']['title_image'] = titles[title]
source.write_text(json.dumps(document, ensure_ascii=False, separators=(',', ':')) + '\n', encoding='utf-8')
def validate_title(root: pathlib.Path, identifier: str) -> None:
if not isinstance(identifier, str) or len(identifier) != 64 or any(character not in '0123456789abcdef' for character in identifier):
raise ValueError('Invalid Pico title identifier; regenerate assets')
try:
asset_root = root.resolve(strict=True)
directory = (root / 'pico' / 'titles').resolve(strict=True)
source = (directory / f'{identifier}.mask').resolve(strict=True)
if not directory.is_relative_to(asset_root) or source.parent != directory:
raise ValueError('Pico title escapes the generated title directory')
if source.stat().st_size != TITLE_BYTES:
raise ValueError('Invalid Pico title size')
with source.open('rb') as stream:
image = stream.read(TITLE_BYTES + 1)
if len(image) != TITLE_BYTES or image[:len(TITLE_HEADER)] != TITLE_HEADER or hashlib.sha256(image).hexdigest() != identifier:
raise ValueError('Invalid Pico title header or digest')
except OSError as error:
raise ValueError(f'Pico title unavailable: {error}') from error
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--font-root', type=pathlib.Path, default=DEFAULT_FONT_ROOT)
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('--static-output', type=pathlib.Path)
group.add_argument('--assets', type=pathlib.Path)
arguments = parser.parse_args()
if arguments.static_output:
generate_static(arguments.static_output, arguments.font_root)
else:
generate_titles(arguments.assets, arguments.font_root)
if __name__ == '__main__':
main()
generator/pitch.py
+17 -18
View File
@@ -2,38 +2,37 @@ from __future__ import annotations
import copy
from generator import harmony
from generator.model import LETTERS, NATURAL, alteration, fraction, key_from_fifths, key_label, pitch_number, rational
import generator.harmony
import generator.model
def shift_pitch(pitch: dict, steps: int, semitones: int) -> dict:
output = copy.deepcopy(pitch)
if 'unpitched' in pitch:
return output
original_octave = pitch.get('octave', 4)
diatonic = original_octave * 7 + LETTERS.index(pitch['letter']) + steps
diatonic = original_octave * 7 + generator.model.LETTERS.index(pitch['letter']) + steps
octave, index = divmod(diatonic, 7)
output['letter'] = LETTERS[index]
output['letter'] = generator.model.LETTERS[index]
output['octave'] = octave
output['alter'] = rational(pitch_number({**pitch, 'octave': original_octave}) + semitones - (12 * (octave + 1) + NATURAL[output['letter']]))
output['alter'] = generator.model.rational(generator.model.pitch_number({**pitch, 'octave': original_octave}) + semitones - (12 * (octave + 1) + generator.model.NATURAL[output['letter']]))
if 'octave' not in pitch:
output.pop('octave')
if 'accidental_display' in output:
from generator.model import accidental_name
output['accidental_display']['name'] = accidental_name(alteration(output['alter']))
output['accidental_display']['name'] = generator.model.accidental_name(generator.model.alteration(output['alter']))
return output
def shift_key(key: dict, steps: int, semitones: int) -> dict:
tonic = shift_pitch(key['tonic'], steps, semitones) if key.get('tonic') else None
alterations = [0] * 7
for index, letter in enumerate(LETTERS):
for index, letter in enumerate(generator.model.LETTERS):
p = shift_pitch({'letter': letter, 'alter': key['alterations'][index]}, steps, semitones)
alterations[LETTERS.index(p['letter'])] = p['alter']
alterations[generator.model.LETTERS.index(p['letter'])] = p['alter']
result = {**copy.deepcopy(key), 'tonic': tonic, 'alterations': alterations}
if 'steps' in key:
result['steps'] = [shift_pitch(step, steps, semitones) for step in key['steps']]
if not key.get('custom') and tonic:
for fifths in range(-14, 15):
candidate = key_from_fifths(fifths, key['mode'])
candidate = generator.model.key_from_fifths(fifths, key['mode'])
if candidate['tonic'] == tonic and candidate['alterations'] == alterations:
result['fifths'] = fifths
break
@@ -41,7 +40,7 @@ def shift_key(key: dict, steps: int, semitones: int) -> dict:
result['custom'] = True
if 'cancel' in key:
# A cancellation references the old signature, not a count of naturals.
canceled = shift_key(key_from_fifths(key['cancel']), steps, semitones)
canceled = shift_key(generator.model.key_from_fifths(key['cancel']), steps, semitones)
if canceled.get('custom'):
raise ValueError('transposed key cancellation cannot be represented by fifths')
result['cancel'] = canceled['fifths']
@@ -59,7 +58,7 @@ def concert_document(document: dict, *, octave_transposing: bool = False) -> dic
def interval(onset):
state = dict(initial)
for change in changes:
if fraction(change['onset']) <= onset:
if generator.model.fraction(change['onset']) <= onset:
state.update({key: change[key] for key in initial if key in change})
transpose, clef = state['transpose'], state['clef']
steps = transpose['diatonic'] + 7 * transpose['octave_change']
@@ -69,17 +68,17 @@ def concert_document(document: dict, *, octave_transposing: bool = False) -> dic
return steps, semitones
measure['key'] = shift_key(measure['key'], *interval(0))
measure['key_signature'] = key_label(measure['key'])
measure['key_signature'] = generator.model.key_label(measure['key'])
for n in (n for v in measure['voices'] for n in v['notes']):
n['pitches'] = [shift_pitch(p, *interval(fraction(n['onset']))) for p in n['pitches']]
n['pitches'] = [shift_pitch(p, *interval(generator.model.fraction(n['onset']))) for p in n['pitches']]
for h in measure['harmonies']:
for field in ('root', 'bass'):
if field in h:
h[field] = shift_pitch(h[field], *interval(fraction(h['onset'])))
h['symbol'] = harmony.symbol(h)
h[field] = shift_pitch(h[field], *interval(generator.model.fraction(h['onset'])))
h['symbol'] = generator.harmony.symbol(h)
for change in measure.get('attributes', []):
if 'key' in change:
change['key'] = shift_key(change['key'], *interval(fraction(change['onset'])))
change['key'] = shift_key(change['key'], *interval(generator.model.fraction(change['onset'])))
if 'transpose' in change:
change['transpose'] = copy.deepcopy(zero)
measure['transpose'] = copy.deepcopy(zero)
@@ -92,4 +91,4 @@ def sounding_pitch(pitch: dict, state: dict):
if 'unpitched' in pitch:
raise ValueError('unpitched notes have no fretted or generated-tone interpretation')
transpose = state['transpose']
return pitch_number(pitch) + transpose['chromatic'] + 12 * transpose['octave_change']
return generator.model.pitch_number(pitch) + transpose['chromatic'] + 12 * transpose['octave_change']
generator/playback.py
+47 -47
View File
@@ -1,16 +1,16 @@
from __future__ import annotations
from array import array
from fractions import Fraction
import array
import fractions
import hashlib
import json
import math
from pathlib import Path
import pathlib
import sys
import wave
from generator.model import TYPE_LENGTHS, fraction, rational
from generator import pitch
import generator.model
import generator.pitch
SAMPLE_RATE = 48000
PROFILE = 'soft-struck'
@@ -19,10 +19,10 @@ MAX_EVENTS = 1000000
class PlaybackUnavailable(ValueError):
pass
def sounding_pitch(tone: dict, state: dict) -> Fraction:
def sounding_pitch(tone: dict, state: dict) -> fractions.Fraction:
if 'unpitched' in tone:
raise PlaybackUnavailable('unpitched notation has no generated-tone interpretation')
value = pitch.sounding_pitch(tone, state)
value = generator.pitch.sounding_pitch(tone, state)
frequency = 440 * 2 ** (float(value - 69) / 12)
if not 20 <= frequency <= 18000:
raise PlaybackUnavailable(f'sounding pitch {value} is outside the generated-tone range (20-18000 Hz)')
@@ -30,19 +30,19 @@ def sounding_pitch(tone: dict, state: dict) -> Fraction:
def compile_timeline(document: dict) -> dict:
events, notes, linked = [], {}, {}
absolute = Fraction()
absolute = fractions.Fraction()
for measure in document['staves'][0]['measures']:
for voice in measure['voices']:
state = {key: measure[key] for key in ('transpose', 'clef', 'staff_details')}
changes = iter(sorted(measure.get('attributes', []), key=lambda a: fraction(a['onset'])))
changes = iter(sorted(measure.get('attributes', []), key=lambda a: generator.model.fraction(a['onset'])))
change = next(changes, None)
for note in voice['notes']:
onset = fraction(note['onset'])
while change is not None and fraction(change['onset']) <= onset:
onset = generator.model.fraction(note['onset'])
while change is not None and generator.model.fraction(change['onset']) <= onset:
state.update({key: change[key] for key in state if key in change})
change = next(changes, None)
notes[note['id']] = (note, absolute + onset, dict(state), voice['id'])
absolute += fraction(measure['duration'])
absolute += generator.model.fraction(measure['duration'])
adjustments = {}
insertions = {}
lanes = {}
@@ -63,25 +63,25 @@ def compile_timeline(document: dict) -> dict:
raise PlaybackUnavailable('grace group mixes conflicting timing modes')
mode = next(iter(modes), 'steal-time-following')
if mode == 'make-time':
lengths = [fraction(g.get(mode, rational(0))) for g in specified]
total = sum(lengths, Fraction())
insertions[onset] = max(insertions.get(onset, Fraction()), total)
lengths = [generator.model.fraction(g.get(mode, generator.model.rational(0))) for g in specified]
total = sum(lengths, fractions.Fraction())
insertions[onset] = max(insertions.get(onset, fractions.Fraction()), total)
else:
donor = previous[0] if mode == 'steal-time-previous' and previous else note if mode == 'steal-time-following' else None
if donor is None:
raise PlaybackUnavailable('grace notes have no preceding duration to steal')
percentages = [fraction(g[mode]) if mode in g else Fraction(25, len(pending)) for g in specified]
percentages = [generator.model.fraction(g[mode]) if mode in g else fractions.Fraction(25, len(pending)) for g in specified]
if not 0 < sum(percentages) < 100:
raise PlaybackUnavailable('grace group must steal between zero and 100 percent')
lengths = [fraction(donor['duration']) * amount / 100 for amount in percentages]
total = sum(lengths, Fraction())
delay, extension = adjustments.get(donor['id'], (Fraction(), Fraction()))
lengths = [generator.model.fraction(donor['duration']) * amount / 100 for amount in percentages]
total = sum(lengths, fractions.Fraction())
delay, extension = adjustments.get(donor['id'], (fractions.Fraction(), fractions.Fraction()))
if mode == 'steal-time-previous':
cursor -= total
else:
delay += total
extension -= total
if fraction(donor['duration']) + extension <= 0 or any(fraction(p['duration']) + extension <= 0 for p in donor['pitches']):
if generator.model.fraction(donor['duration']) + extension <= 0 or any(generator.model.fraction(p['duration']) + extension <= 0 for p in donor['pitches']):
raise PlaybackUnavailable(f'{donor["id"]}: grace timing consumes a donor or chord tone')
adjustments[donor['id']] = (delay, extension)
for item, length in zip(pending, lengths):
@@ -95,60 +95,60 @@ def compile_timeline(document: dict) -> dict:
for identifier, (note, onset, state, voice) in notes.items():
if note.get('cue'):
continue
delay, extension = adjustments.get(identifier, (Fraction(), Fraction()))
performed = onset + delay + sum((length for time, length in insertions.items() if time < onset or time == onset and not note.get('grace')), Fraction())
delay, extension = adjustments.get(identifier, (fractions.Fraction(), fractions.Fraction()))
performed = onset + delay + sum((length for time, length in insertions.items() if time < onset or time == onset and not note.get('grace')), fractions.Fraction())
tremolo = note.get('tremolo')
if tremolo and tremolo['type'] == 'stop':
continue
for pitch in note['pitches']:
if pitch.get('cue'):
continue
length = fraction(pitch['duration']) + extension
length = generator.model.fraction(pitch['duration']) + extension
if not note.get('grace'):
length += sum((amount for time, amount in insertions.items() if onset < time < onset + fraction(pitch['duration'])), Fraction())
length += sum((amount for time, amount in insertions.items() if onset < time < onset + generator.model.fraction(pitch['duration'])), fractions.Fraction())
if length <= 0:
raise PlaybackUnavailable(f'{identifier}: grace timing consumes a chord tone')
value = sounding_pitch(pitch, state)
if pitch.get('tie_from'):
previous = linked.get(pitch['tie_from'])
if previous is None or fraction(previous['onset']) + fraction(previous['duration']) != performed or fraction(previous['pitch']) != value:
if previous is None or generator.model.fraction(previous['onset']) + generator.model.fraction(previous['duration']) != performed or generator.model.fraction(previous['pitch']) != value:
raise PlaybackUnavailable(f'{identifier}: tie has inconsistent performed timing or pitch')
previous['duration'] = rational(fraction(previous['duration']) + length)
previous['duration'] = generator.model.rational(generator.model.fraction(previous['duration']) + length)
linked[pitch['id']] = previous
continue
if tremolo and tremolo['marks']:
if pitch.get('tie_to'):
raise PlaybackUnavailable('tied measured tremolo is not yet qualified for playback')
written = pitch['written']
step = min(Fraction(1), TYPE_LENGTHS[written['type']]) * fraction(written.get('ratio', rational(1))) / 2 ** tremolo['marks']
step = min(fractions.Fraction(1), generator.model.TYPE_LENGTHS[written['type']]) * generator.model.fraction(written.get('ratio', generator.model.rational(1))) / 2 ** tremolo['marks']
if tremolo['type'] == 'start':
partner = notes[tremolo['partner']]
if partner[1] != onset + fraction(note['duration']) or len(partner[0]['pitches']) != len(note['pitches']):
if partner[1] != onset + generator.model.fraction(note['duration']) or len(partner[0]['pitches']) != len(note['pitches']):
raise PlaybackUnavailable('paired tremolo requires contiguous equal-size chords')
step *= 2
length += fraction(partner[0]['pitches'][note['pitches'].index(pitch)]['duration'])
cursor, attack = Fraction(), 0
length += generator.model.fraction(partner[0]['pitches'][note['pitches'].index(pitch)]['duration'])
cursor, attack = fractions.Fraction(), 0
while cursor < length:
amount = value
if tremolo['type'] == 'start' and attack % 2:
other = partner[0]['pitches'][note['pitches'].index(pitch)]
amount = sounding_pitch(other, partner[2])
events.append({'id': f'{pitch["id"]}-{attack}', 'pitch': rational(amount), 'onset': rational(performed + cursor), 'duration': rational(min(step, length - cursor))})
events.append({'id': f'{pitch["id"]}-{attack}', 'pitch': generator.model.rational(amount), 'onset': generator.model.rational(performed + cursor), 'duration': generator.model.rational(min(step, length - cursor))})
cursor += step
attack += 1
if len(events) > MAX_EVENTS:
raise PlaybackUnavailable('playback exceeds one million tone events')
else:
event = {'id': pitch['id'], 'pitch': rational(value), 'onset': rational(performed), 'duration': rational(length)}
event = {'id': pitch['id'], 'pitch': generator.model.rational(value), 'onset': generator.model.rational(performed), 'duration': generator.model.rational(length)}
events.append(event)
linked[pitch['id']] = event
if len(events) > MAX_EVENTS:
raise PlaybackUnavailable('playback exceeds one million tone events')
events.sort(key=lambda e: (fraction(e['onset']), e['id']))
return {'available': True, 'duration': rational(absolute + sum(insertions.values(), Fraction())), 'events': events}
events.sort(key=lambda e: (generator.model.fraction(e['onset']), e['id']))
return {'available': True, 'duration': generator.model.rational(absolute + sum(insertions.values(), fractions.Fraction())), 'events': events}
def write_pcm(path: Path, values) -> str:
samples = array('h', (round(max(-1, min(1, sample)) * 32767) for sample in values))
def write_pcm(path: pathlib.Path, values) -> str:
samples = array.array('h', (round(max(-1, min(1, sample)) * 32767) for sample in values))
if sys.byteorder != 'little':
samples.byteswap()
path.parent.mkdir(parents=True, exist_ok=True)
@@ -157,9 +157,9 @@ def write_pcm(path: Path, values) -> str:
output.writeframes(samples.tobytes())
return hashlib.sha256(path.read_bytes()).hexdigest()
def generate_tone(root: Path, pitch: Fraction) -> dict:
def generate_tone(root: pathlib.Path, pitch: fractions.Fraction) -> dict:
frequency = 440 * 2 ** (float(pitch - 69) / 12)
period = Fraction(frequency / SAMPLE_RATE).limit_denominator(SAMPLE_RATE)
period = fractions.Fraction(frequency / SAMPLE_RATE).limit_denominator(SAMPLE_RATE)
loop_length, cycles = period.denominator, period.numerator
attack = 12000
name = hashlib.sha256(f'{PROFILE}:{pitch}'.encode('ascii')).hexdigest()[:24]
@@ -174,7 +174,7 @@ def generate_tone(root: Path, pitch: Fraction) -> dict:
checksum = write_pcm(root / relative, (sample(i) for i in range(attack + loop_length)))
return {'id': name, 'path': relative, 'sha256': checksum, 'sample_rate': SAMPLE_RATE, 'frames': attack + loop_length, 'loop_start': attack, 'loop_end': attack + loop_length}
def compile_tree(root: Path) -> None:
def compile_tree(root: pathlib.Path) -> None:
tones = {}
release = {'path': 'audio/release.wav', 'sample_rate': SAMPLE_RATE, 'frames': 2400}
release['sha256'] = write_pcm(root / release['path'], ((1 - i / 2399) ** 2 for i in range(2400)))
@@ -184,8 +184,8 @@ def compile_tree(root: Path) -> None:
playback = compile_timeline(document)
used = {}
for event in playback['events']:
event['end'] = rational(fraction(event['onset']) + fraction(event.pop('duration')))
pitch = fraction(event.pop('pitch'))
event['end'] = generator.model.rational(generator.model.fraction(event['onset']) + generator.model.fraction(event.pop('duration')))
pitch = generator.model.fraction(event.pop('pitch'))
if pitch not in tones:
tones[pitch] = generate_tone(root, pitch)
tone = tones[pitch]
@@ -197,7 +197,7 @@ def compile_tree(root: Path) -> None:
document['playback'] = playback
path.write_text(json.dumps(document, indent=2, ensure_ascii=False) + '\n', encoding='utf-8')
def validate_assets(playback: dict, root: Path, cache: dict | None = None) -> None:
def validate_assets(playback: dict, root: pathlib.Path, cache: dict | None = None) -> None:
if not isinstance(playback, dict) or type(playback.get('available')) is not bool:
raise ValueError('playback manifest requires boolean availability')
if not playback['available']:
@@ -208,7 +208,7 @@ def validate_assets(playback: dict, root: Path, cache: dict | None = None) -> No
def check_sample(value):
name = value['path']
relative = Path(name)
relative = pathlib.Path(name)
if not name.startswith('audio/') or relative.is_absolute() or relative.suffix != '.wav' or any(p in ('', '.', '..') for p in name.split('/')) or '\\' in name or '\x00' in name:
raise ValueError('invalid generated audio path')
path = (root / relative).resolve()
@@ -225,7 +225,7 @@ def validate_assets(playback: dict, root: Path, cache: dict | None = None) -> No
pcm = audio.readframes(value['frames'])
if (audio.getnchannels(), audio.getsampwidth(), audio.getframerate(), audio.getnframes()) != (1, 2, SAMPLE_RATE, value['frames']) or len(pcm) != value['frames'] * 2:
raise ValueError('invalid or truncated generated PCM WAV')
samples = array('h')
samples = array.array('h')
samples.frombytes(pcm)
if sys.byteorder != 'little':
samples.byteswap()
@@ -234,7 +234,7 @@ def validate_assets(playback: dict, root: Path, cache: dict | None = None) -> No
cache[key] = samples
return samples
duration = fraction(playback['duration'])
duration = generator.model.fraction(playback['duration'])
if duration <= 0 or len(playback['events']) > MAX_EVENTS:
raise ValueError('invalid playback timeline length')
release = check_sample(playback['release'])
@@ -250,10 +250,10 @@ def validate_assets(playback: dict, root: Path, cache: dict | None = None) -> No
raise ValueError('invalid sample identity or loop')
check_sample(sample)
samples[sample['id']] = sample
previous = Fraction()
previous = fractions.Fraction()
identities = set()
for event in playback['events']:
onset, end = fraction(event['onset']), fraction(event['end'])
onset, end = generator.model.fraction(event['onset']), generator.model.fraction(event['end'])
if event['id'] in identities or event['sample'] not in samples or not previous <= onset < end <= duration:
raise ValueError('invalid playback event identity, reference or timing')
identities.add(event['id'])
generator/practice/generate.py
+32 -23
View File
@@ -1,44 +1,53 @@
from __future__ import annotations
from pathlib import Path
import pathlib
from generator.guitar import place_guitar
from generator.musicxml import load_musicxml
from generator.parts import PART_TYPES, build_measure, prepare_document, propagated_chords, source_paths
from generator.scat import add_scat
from generator.score import make_arranged_document, reidentify, source_staff, write_json
import generator.accompaniment
import generator.guitar
import generator.musicxml
import generator.parts
import generator.rhythm
import generator.scat
import generator.score
def practice_document(document: dict, part_type: str) -> dict:
return _practice_document(prepare_document(document, practice=True), part_type)
def practice_document(document: dict, part_type: str, *, rhythms: generator.rhythm.Library) -> dict:
document = generator.parts.prepare_document(document, practice=True)
plan = generator.accompaniment.plan_section(generator.score.source_staff(document)['measures'], rhythms, electric=False) if part_type == 'acoustic' else None
return _practice_document(document, part_type, plan)
def _practice_document(document: dict, part_type: str) -> dict:
if part_type not in PART_TYPES:
def _practice_document(document: dict, part_type: str, plan: generator.accompaniment.Plan | None) -> dict:
if part_type not in generator.parts.PART_TYPES:
raise ValueError(f'unknown practice part: {part_type}')
staff = source_staff(document)
previous = propagated_chords(staff['measures'])
staff = generator.score.source_staff(document)
previous = generator.accompaniment.carried_chords(staff['measures'])
role = {'acoustic': 'accompaniment', 'electric': 'melody', 'vocal': 'vocal'}[part_type]
measures = [build_measure(m, f'measure-{i + 1}', role, part_type, previous[i], m.get('section_label', '')) for i, m in enumerate(staff['measures'])]
if part_type == 'acoustic' and plan is None:
raise ValueError('acoustic practice requires a rhythm plan')
measures = [generator.parts.build_measure(m, f'measure-{i + 1}', role, part_type, previous[i], m.get('section_label', ''), plan.acoustic[i] if part_type == 'acoustic' else None) for i, m in enumerate(staff['measures'])]
for measure in measures:
for chord in measure['harmonies']:
chord.pop('chord_optional', None)
tuplets = reidentify(measures, staff.get('tuplets', []))
result = make_arranged_document(document, staff, staff_id='staff-practice-' + part_type, staff_name=part_type.capitalize(), role='vocal' if part_type == 'vocal' else 'guitar', arrangement_part_type=part_type, measures=measures, tuplets=tuplets, octave_shifts=staff.get('octave_shifts', []) if role in ('melody', 'vocal') else [])
tuplets = generator.score.reidentify(measures, staff.get('tuplets', []))
result = generator.score.make_arranged_document(document, staff, staff_id='staff-practice-' + part_type, staff_name=part_type.capitalize(), role='vocal' if part_type == 'vocal' else 'guitar', arrangement_part_type=part_type, measures=measures, tuplets=tuplets, octave_shifts=staff.get('octave_shifts', []) if role in ('melody', 'vocal') else [])
if part_type == 'vocal':
add_scat(result)
generator.scat.add_scat(result)
else:
place_guitar(result)
generator.guitar.place_guitar(result)
if generator.accompaniment.connect_continuations(result):
generator.guitar.place_guitar(result)
return result
def generate_practice_scores(source_dir: Path, output_dir: Path) -> int:
sources = source_paths(source_dir, practice=True)
def generate_practice_scores(source_dir: pathlib.Path, output_dir: pathlib.Path, *, rhythms: generator.rhythm.Library) -> int:
sources = generator.parts.source_paths(source_dir, practice=True)
for path in sources:
try:
document = prepare_document(load_musicxml(path), practice=True)
document = generator.parts.prepare_document(generator.musicxml.load_musicxml(path), practice=True)
plan = generator.accompaniment.plan_section(generator.score.source_staff(document)['measures'], rhythms, electric=False)
except ValueError as error:
raise ValueError(f'{path}: {error}') from error
for part in PART_TYPES:
for part in generator.parts.PART_TYPES:
try:
write_json(output_dir / path.stem / (part + '.json'), _practice_document(document, part))
generator.score.write_json(output_dir / path.stem / (part + '.json'), _practice_document(document, part, plan))
except ValueError as error:
raise ValueError(f'{path}, part {part}: {error}') from error
return len(sources) * len(PART_TYPES)
return len(sources) * len(generator.parts.PART_TYPES)
generator/rhythm.py
+95
View File
@@ -0,0 +1,95 @@
from __future__ import annotations
import dataclasses
import fractions
import pathlib
import types
import generator.model
import generator.musicxml
@dataclasses.dataclass(frozen=True)
class Event:
onset: fractions.Fraction
duration: fractions.Fraction
sounding: bool
written_type: str
dots: int = 0
beams: tuple[tuple[int, str], ...] = ()
continuation: bool = False
@dataclasses.dataclass(frozen=True)
class Bar:
events: tuple[Event, ...]
length: fractions.Fraction
def attacks(self) -> tuple[fractions.Fraction, ...]:
return tuple(e.onset for e in self.events if e.sounding and not e.continuation)
def beat(self, number: int) -> tuple:
return tuple((max(e.onset, number) - number, min(e.onset + e.duration, number + 1) - max(e.onset, number), e.sounding, e.continuation or e.onset < number) for e in self.events if e.onset < number + 1 and e.onset + e.duration > number)
@dataclasses.dataclass(frozen=True)
class Snippet:
id: str
meter: str
bars: tuple[Bar, ...]
@dataclasses.dataclass(frozen=True, init=False)
class Library:
snippets: types.MappingProxyType
def __init__(self, snippets: tuple[Snippet, ...]):
grouped = {}
identities = set()
for snippet in sorted(snippets, key=lambda s: s.id):
if snippet.id in identities:
raise ValueError(f'duplicate rhythm identity: {snippet.id}')
identities.add(snippet.id)
grouped.setdefault(snippet.meter, []).append(snippet)
object.__setattr__(self, 'snippets', types.MappingProxyType({meter: tuple(values) for meter, values in grouped.items()}))
def for_meter(self, meter: str) -> tuple[Snippet, ...]:
if meter not in ('2/4', '3/4', '4/4', '5/4') or meter not in self.snippets:
raise ValueError(f'accompaniment requires rhythm snippets for {meter}; supported meters are 2/4, 3/4, 4/4, and 5/4')
return self.snippets[meter]
def read_snippet(path: pathlib.Path, identifier: str) -> Snippet:
document = generator.musicxml.load_musicxml(path)
staff = document['staves'][0]
if len(staff['measures']) not in (1, 2) or staff.get('tuplets'):
raise ValueError(f'{path}: rhythm snippets require one or two bars without tuplets')
meter = staff['measures'][0]['time_signature']
bars = []
for measure in staff['measures']:
length = generator.model.fraction(measure['duration'])
if measure['time_signature'] != meter or length != generator.model.meter_length(measure['meter']):
raise ValueError(f'{path}, measure {measure["number"]}: rhythm bars must be complete and use one meter')
if len(measure['voices']) != 1 or any('meter' in a for a in measure.get('attributes', [])):
raise ValueError(f'{path}: rhythm snippets require one voice without meter changes')
events = []
cursor = fractions.Fraction()
for note in measure['voices'][0]['notes']:
onset, duration = generator.model.fraction(note['onset']), generator.model.fraction(note['duration'])
if onset != cursor or duration <= 0 or note.get('grace') or note['kind'] not in ('note', 'rest') or note.get('tremolo'):
raise ValueError(f'{path}, measure {measure["number"]}: rhythm events must fill the bar sequentially')
if 'ratio' in note['written']:
raise ValueError(f'{path}, measure {measure["number"]}: rhythm snippets do not support time modifications')
events.append(Event(onset, duration, note['kind'] == 'note', note['written']['type'], note['written'].get('dots', 0), tuple((b['level'], b['type']) for b in note.get('beams', [])), any('tie_from' in p for p in note['pitches'])))
cursor += duration
if cursor != length:
raise ValueError(f'{path}: rhythm events do not fill the bar')
bars.append(Bar(tuple(events), length))
return Snippet(identifier, meter, tuple(bars))
def load_library(root: pathlib.Path) -> Library:
if not root.is_dir():
raise ValueError(f'{root}: rhythm library not found')
snippets = []
for path in sorted(root.rglob('*.xml')):
relative = path.relative_to(root)
if len(relative.parts) != 3 or relative.parts[0] not in ('2-4', '3-4', '4-4', '5-4') or relative.parts[1] not in ('1', '2'):
raise ValueError(f'{path}: rhythm sources belong under meter/series')
snippet = read_snippet(path, relative.as_posix())
if snippet.meter != relative.parts[0].replace('-', '/'):
raise ValueError(f'{path}: rhythm meter disagrees with its directory')
snippets.append(snippet)
if not snippets:
raise ValueError(f'{root}: rhythm library is empty')
return Library(tuple(snippets))
generator/scat.py
+17 -17
View File
@@ -1,21 +1,21 @@
from __future__ import annotations
from fractions import Fraction
import fractions
from generator.model import beat_lengths, fraction, rational
import generator.model
def pulse(measure: dict, onset: Fraction) -> tuple[Fraction | None, bool]:
start, meter = Fraction(), measure['meter']
def pulse(measure: dict, onset: fractions.Fraction) -> tuple[fractions.Fraction | None, bool]:
start, meter = fractions.Fraction(), measure['meter']
for change in measure.get('attributes', []):
if fraction(change['onset']) <= onset and 'meter' in change:
start, meter = fraction(change['onset']), change['meter']
beats = beat_lengths(meter)
if generator.model.fraction(change['onset']) <= onset and 'meter' in change:
start, meter = generator.model.fraction(change['onset']), change['meter']
beats = generator.model.beat_lengths(meter)
if not beats:
return None, False
# Pickups occupy the final beats of their meter.
phase = onset - start
if measure.get('implicit') and not start:
phase += max(Fraction(), sum(beats) - fraction(measure['duration']))
phase += max(fractions.Fraction(), sum(beats) - generator.model.fraction(measure['duration']))
phase %= sum(beats)
for beat in beats:
if phase < beat:
@@ -23,7 +23,7 @@ def pulse(measure: dict, onset: Fraction) -> tuple[Fraction | None, bool]:
phase -= beat
raise ValueError('invalid meter pulse')
def syllable(duration: Fraction, beat: Fraction | None, onbeat: bool, final: bool, previous: str) -> str:
def syllable(duration: fractions.Fraction, beat: fractions.Fraction | None, onbeat: bool, final: bool, previous: str) -> str:
if duration >= (beat if beat is not None else 1):
return 'doo'
if final:
@@ -33,7 +33,7 @@ def syllable(duration: Fraction, beat: Fraction | None, onbeat: bool, final: boo
def add_scat(document: dict) -> None:
events, pitches = [], {}
absolute = Fraction()
absolute = fractions.Fraction()
boundaries = set()
for measure in document['staves'][0]['measures']:
if measure.get('section_label'):
@@ -42,34 +42,34 @@ def add_scat(document: dict) -> None:
for note in voice['notes']:
for field in ('lyrics', 'lyric', 'lyric_notation'):
note.pop(field, None)
onset = absolute + fraction(note['onset'])
onset = absolute + generator.model.fraction(note['onset'])
for tone in note['pitches']:
pitches[tone['id']] = (note, tone, onset)
events.append((note, measure, onset))
absolute += fraction(measure['duration'])
absolute += generator.model.fraction(measure['duration'])
attacks = [(n, m, t) for n, m, t in events if n['pitches'] and not n.get('cue') and not n['pitches'][0].get('cue') and not n['pitches'][0].get('tie_from')]
previous, previous_end = '', None
for index, (note, measure, onset) in enumerate(attacks):
tone = note['pitches'][0]
end = onset + fraction(tone['duration'])
end = onset + generator.model.fraction(tone['duration'])
chain = []
while target := tone.get('tie_to'):
continuation, tone, time = pitches[target]
chain.append(continuation)
end = time + fraction(tone['duration'])
end = time + generator.model.fraction(tone['duration'])
following = attacks[index + 1][2] if index + 1 < len(attacks) else None
final = following is None or following > end or any(onset < b <= following for b in boundaries)
if previous_end is None or onset > previous_end or onset in boundaries:
previous = ''
beat, onbeat = pulse(measure, fraction(note['onset']))
beat, onbeat = pulse(measure, generator.model.fraction(note['onset']))
text = syllable(end - onset, beat, onbeat, final, previous)
values = [{'kind': 'syllabic', 'text': 'single', 'attributes': {}}, {'kind': 'text', 'text': text, 'attributes': {}}]
verse = {'number': '1', 'syllables': values}
if chain:
values.append({'kind': 'extend', 'text': '', 'attributes': {'type': 'start'}})
verse['extend_to'] = rational(end)
verse['extend_to'] = generator.model.rational(end)
for position, continuation in enumerate(chain):
continuation['lyrics'] = [{'number': '1', 'syllables': [{'kind': 'extend', 'text': '', 'attributes': {'type': 'stop' if position == len(chain) - 1 else 'continue'}}], 'extend_to': rational(end)}]
continuation['lyrics'] = [{'number': '1', 'syllables': [{'kind': 'extend', 'text': '', 'attributes': {'type': 'stop' if position == len(chain) - 1 else 'continue'}}], 'extend_to': generator.model.rational(end)}]
note['lyrics'] = [verse]
note['lyric'] = text
note['lyric_notation'] = {'text': text, 'syllabic': 'single', 'extend': 'start' if chain else ''}
generator/score.py
+20 -19
View File
@@ -1,46 +1,45 @@
from __future__ import annotations
import copy
import hashlib
import json
import pathlib
import re
from pathlib import Path
from generator.model import validate_document
import generator.engraving.validation
import generator.model
import generator.pico_artwork
import generator.playback
def load_json(path: Path) -> dict:
def load_json(path: pathlib.Path) -> dict:
document = json.loads(path.read_text(encoding='utf-8'))
return validate_json(document, path)
def validate_json(document: dict, path: Path) -> dict:
def validate_json(document: dict, path: pathlib.Path, validated_pages: set[str] | None = None) -> dict:
if not isinstance(document, dict):
raise ValueError('expected a score object')
if 'music' in document:
from generator.engraving import ENGINE_NAME, PAGE_HEIGHT, PAGE_WIDTH, check_page
presentation = document.get('presentation', {})
if not isinstance(document['music'], dict) or not isinstance(presentation, dict) or not isinstance(presentation.get('pages'), list) or not presentation['pages']:
raise ValueError('invalid score bundle; regenerate score assets')
if (presentation.get('engine'), presentation.get('width'), presentation.get('height')) != (ENGINE_NAME, PAGE_WIDTH, PAGE_HEIGHT):
if (presentation.get('engine'), presentation.get('width'), presentation.get('height')) != (generator.engraving.validation.ENGINE_NAME, generator.engraving.validation.PAGE_WIDTH, generator.engraving.validation.PAGE_HEIGHT):
raise ValueError('unsupported score presentation; regenerate score assets')
for page in presentation['pages']:
if not isinstance(page, dict) or not isinstance(page.get('path'), str) or not isinstance(page.get('sha256'), str):
raise ValueError('invalid score page manifest')
name = page['path']
relative = Path(name)
relative = pathlib.Path(name)
if '\x00' in name or '\\' in name or relative.is_absolute() or any(part in ('', '.', '..') for part in name.split('/')) or relative.suffix != '.png':
raise ValueError('invalid score page path')
resolved = (path.parent / relative).resolve()
if not resolved.is_relative_to(path.parent.resolve()):
raise ValueError('score page escapes its asset directory')
if not re.fullmatch('[0-9a-f]{64}', page['sha256']) or hashlib.sha256(resolved.read_bytes()).hexdigest() != page['sha256']:
raise ValueError('score page checksum mismatch')
check_page(resolved)
generator.engraving.validation.check_page(resolved, validated_pages, expected_sha256=page['sha256'])
document = document['music']
validate_document(document)
generator.model.validate_document(document)
return document
def write_json(path: Path, document: dict) -> None:
validate_document(document)
def write_json(path: pathlib.Path, document: dict) -> None:
generator.model.validate_document(document)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(document, indent=2, ensure_ascii=False) + '\n', encoding='utf-8')
@@ -121,14 +120,16 @@ def reidentify(measures: list[dict], source_tuplets: list[dict]) -> list[dict]:
def make_arranged_document(source_document: dict, staff: dict, *, staff_id: str, staff_name: str, role: str, arrangement_part_type: str, measures: list[dict], tuplets=None, octave_shifts=None) -> dict:
return {'metadata': copy.deepcopy(source_document['metadata']), 'staves': [{'id': staff_id, 'name': staff_name, 'clef': copy.deepcopy(staff['clef']), 'role': role, 'arranged': True, 'arrangement_part_type': arrangement_part_type, 'measures': measures, 'tuplets': tuplets or [], 'octave_shifts': copy.deepcopy(octave_shifts or []), 'instruments': copy.deepcopy(staff.get('instruments', []))}]}
def count_json_files(root: Path) -> int:
def count_json_files(root: pathlib.Path) -> int:
return sum(1 for p in root.rglob('*.json') if p.is_file())
def validate_generated_tree(root: Path) -> None:
def validate_generated_tree(root: pathlib.Path, validated_pages: set[str] | None = None) -> None:
audio_cache = {}
validated_pages = set() if validated_pages is None else validated_pages
for path in root.rglob('*.json'):
document = json.loads(path.read_text(encoding='utf-8'))
validate_json(document, path)
validate_json(document, path, validated_pages)
if 'presentation' in document:
generator.pico_artwork.validate_title(root, document['presentation'].get('title_image'))
if 'playback' in document:
from generator.playback import validate_assets
validate_assets(document['playback'], root, audio_cache)
generator.playback.validate_assets(document['playback'], root, audio_cache)
generator/styles.py
-80
View File
@@ -1,80 +0,0 @@
from __future__ import annotations
DEFAULT_STYLE = "Medium Swing"
BUILT_IN_STYLES = (
# Jazz
"Afro 12/8",
"Ballad Double Time Feel",
"Ballad Even",
"Ballad Melodic",
"Ballad Swing",
"Blue Note",
"Bossa Nova",
"Doo Doo Cats",
"Double Time Swing",
"Even 8ths",
"Even 8ths Open",
"Even 16ths",
"Guitar Trio",
"Gypsy Jazz",
"Latin",
"Latin/Swing",
"Long Notes",
"Medium Swing",
"Medium Up Swing",
"Medium Up Swing 2",
"New Orleans Swing",
"Second Line",
"Slow Swing",
"Swing Two/Four",
"Trad Jazz",
"Up Tempo Swing",
"Up Tempo Swing 2",
# Latin
"Argentina: Tango",
"Brazil: Bossa Acoustic",
"Brazil: Bossa Electric",
"Brazil: Samba",
"Cuba: Bolero",
"Cuba: Cha Cha Cha",
"Cuba: Son Montuno 2-3",
"Cuba: Son Montuno 3-2",
# Pop
"Bluegrass",
"Country",
"Disco",
"Funk",
"Glam Funk",
"House",
"Reggae",
"RnB",
"Rock",
"Rock 12/8",
"Shuffle",
"Slow Rock",
"Smooth",
"Soul",
"Virtual Funk",
)
BUILT_IN_STYLE_SET = frozenset(BUILT_IN_STYLES)
STYLE_ALIASES = {
"Med.": DEFAULT_STYLE,
"Med. Swing": DEFAULT_STYLE,
"Medium": DEFAULT_STYLE,
}
def canonical_style(value: object) -> str:
if value is None:
return DEFAULT_STYLE
if not isinstance(value, str):
raise ValueError("style must be a string")
style = value.strip()
if not style:
return DEFAULT_STYLE
style = STYLE_ALIASES.get(style, style)
if style not in BUILT_IN_STYLE_SET:
raise ValueError(f"style {value!r} is not a supported player style")
return style
generator/summary.py
+19 -19
View File
@@ -1,39 +1,39 @@
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
import pathlib
import typing
from generator.score import load_json, measure_notes
from generator.parts import PART_TYPES
from generator.model import fraction, pitch_name
import generator.model
import generator.parts
import generator.score
def _arranged_staff(document: dict[str, Any]) -> dict[str, Any]:
def _arranged_staff(document: dict[str, typing.Any]) -> dict[str, typing.Any]:
staves = document.get("staves")
if not isinstance(staves, list) or not staves or not isinstance(staves[0], dict):
raise ValueError("generated score document must contain an arranged staff")
return staves[0]
def _measure_summary(staff: dict[str, Any], measure: dict[str, Any], measure_number: int) -> dict[str, Any]:
notes = measure_notes(measure)
summary: dict[str, Any] = {
def _measure_summary(staff: dict[str, typing.Any], measure: dict[str, typing.Any], measure_number: int) -> dict[str, typing.Any]:
notes = generator.score.measure_notes(measure)
summary: dict[str, typing.Any] = {
"measure": measure_number,
"section_label": str(measure.get("section_label", "")),
"harmonies": [{"onset": str(fraction(h["onset"])), "symbol": h["symbol"]} for h in measure["harmonies"]],
"duration": str(fraction(measure["duration"])),
"harmonies": [{"onset": str(generator.model.fraction(h["onset"])), "symbol": h["symbol"]} for h in measure["harmonies"]],
"duration": str(generator.model.fraction(measure["duration"])),
"slash_count": sum(1 for note in notes if note.get("kind", "note") == "note" and note.get("notehead") == "slash"),
}
if staff.get("arrangement_part_type") == "vocal":
summary["vocal_skeleton"] = [
f'{fraction(note["onset"])}+{fraction(note["duration"])} '
f'{generator.model.fraction(note["onset"])}+{generator.model.fraction(note["duration"])} '
+ note['written']['type'] + '.' * note['written'].get('dots', 0)
+ ' ' + note['kind'] + ' ' + ','.join(pitch_name(p) + str(p['octave']) for p in note['pitches'])
+ ' ' + note['kind'] + ' ' + ','.join(generator.model.pitch_name(p) + str(p['octave']) for p in note['pitches'])
+ ' ' + note.get('lyric', '')
for note in notes
]
return summary
def performance_document_summary(document: dict[str, Any]) -> dict[str, Any]:
def performance_document_summary(document: dict[str, typing.Any]) -> dict[str, typing.Any]:
metadata = document.get("metadata")
if not isinstance(metadata, dict):
metadata = {}
@@ -60,13 +60,13 @@ def performance_document_summary(document: dict[str, Any]) -> dict[str, Any]:
"measures": measure_summaries,
}
def performance_tree_summary(root: Path) -> dict[str, Any]:
def performance_tree_summary(root: pathlib.Path) -> dict[str, typing.Any]:
return {
path.name: performance_document_summary(load_json(path))
path.name: performance_document_summary(generator.score.load_json(path))
for path in sorted(root.glob("*.json"))
}
def validate_expected_performance_summary(expected_path: Path, generated_songs_dir: Path) -> None:
def validate_expected_performance_summary(expected_path: pathlib.Path, generated_songs_dir: pathlib.Path) -> None:
if not expected_path.exists():
raise ValueError(f"missing generator compatibility summary: {expected_path}")
with expected_path.open("r", encoding="utf-8") as stream:
@@ -85,10 +85,10 @@ def validate_expected_performance_summary(expected_path: Path, generated_songs_d
raise ValueError(f"performance generator drift in {name}; update generator logic or refresh the expected summary intentionally")
raise ValueError("performance generator drift")
def validate_practice_parts(generated_practice_dir: Path) -> None:
def validate_practice_parts(generated_practice_dir: pathlib.Path) -> None:
if not generated_practice_dir.exists():
return
expected_names = {part + '.json' for part in PART_TYPES}
expected_names = {part + '.json' for part in generator.parts.PART_TYPES}
for exercise_dir in sorted(path for path in generated_practice_dir.iterdir() if path.is_dir()):
json_names = {path.name for path in exercise_dir.glob("*.json")}
if json_names != expected_names:
include/vamp_pico/artwork.hpp
+137
View File
@@ -0,0 +1,137 @@
#pragma once
#include <algorithm>
#include <array>
#include <cstdint>
#include <charconv>
#include <string_view>
#include <vamp_protocol/page_control.hpp>
namespace vamp_pico {
constexpr auto digit_width = std::size_t{22};
constexpr auto max_pages = std::uint32_t{10000};
inline auto counter_text(std::uint32_t page, std::uint32_t pages) -> std::array<std::char_t, 12> {
auto text = std::array<std::char_t, 12>{};
if (pages == 0U || pages > max_pages || page == 0U || page > pages) {
return text;
}
auto first = std::to_chars(text.data(), text.data() + text.size() - 1U, page);
*first.ptr = '/';
std::to_chars(first.ptr + 1U, text.data() + text.size() - 1U, pages);
return text;
}
enum class sprite : std::size_t {
VAMP,
CONNECTING,
OFFLINE,
BUSY,
READY,
LOADING,
WIFI,
ERROR,
PAGE,
ACOUSTIC,
ELECTRIC,
VOCAL,
UNAVAILABLE,
DIGIT_0,
DIGIT_1,
DIGIT_2,
DIGIT_3,
DIGIT_4,
DIGIT_5,
DIGIT_6,
DIGIT_7,
DIGIT_8,
DIGIT_9,
SLASH,
COUNT,
};
struct coverage_image {
std::string_view pixels{};
std::size_t width{};
std::size_t height{};
auto at(std::size_t x, std::size_t y) const -> std::uint8_t {
if (x >= width || y >= height || (y * width + x) / 2U >= pixels.size()) {
return 0U;
}
auto index = y * width + x;
auto packed = static_cast<std::uint8_t>(pixels[index / 2U]);
return (index & 1U) == 0U ? packed >> 4U : packed & 15U;
}
};
inline auto static_sprite(std::string_view atlas, sprite value) -> coverage_image {
constexpr auto count = static_cast<std::size_t>(sprite::COUNT);
auto index = static_cast<std::size_t>(value);
if (atlas.size() < 8U + count * 8U || atlas.substr(0U, 4U) != "VPSA" ||
static_cast<std::uint8_t>(atlas[4]) != count || atlas[5] != 0 || atlas[6] != 0 || atlas[7] != 0 || index >= count) {
return {};
}
auto word = [&](std::size_t offset) -> std::size_t {
return static_cast<std::uint8_t>(atlas[offset]) | (static_cast<std::size_t>(static_cast<std::uint8_t>(atlas[offset + 1U])) << 8U);
};
auto entry = 8U + index * 8U;
auto width = word(entry);
auto height = word(entry + 2U);
auto offset = word(entry + 4U) | (word(entry + 6U) << 16U);
auto size = (width * height + 1U) / 2U;
if (width == 0U || width > 176U || height != 28U || offset < 8U + count * 8U || offset > atlas.size() || size > atlas.size() - offset) {
return {};
}
return {atlas.substr(offset, size), width, height};
}
class title_cache {
public:
auto select(std::string_view identifier) -> std::bool_t {
if (identifier == identifier_.data()) {
return false;
}
identifier_.fill(0);
if (vamp_protocol::valid_title_id(identifier)) {
std::copy(identifier.begin(), identifier.end(), identifier_.begin());
}
ready_ = false;
failed_ = false;
return true;
}
auto accept(std::string_view identifier, std::string_view bytes) -> std::bool_t {
if (identifier != identifier_.data() || !vamp_protocol::valid_title_id(identifier) || !vamp_protocol::valid_title_image(bytes)) {
return false;
}
std::copy(bytes.begin() + vamp_protocol::title_header_size, bytes.end(), pixels_.begin());
ready_ = true;
failed_ = false;
return true;
}
auto fail(std::string_view identifier) -> std::bool_t {
if (identifier != identifier_.data() || ready_) {
return false;
}
failed_ = true;
return true;
}
auto ready() const -> std::bool_t {
return ready_;
}
auto failed() const -> std::bool_t {
return failed_;
}
auto image() const -> coverage_image {
return ready_ ? coverage_image{{pixels_.data(), pixels_.size()}, vamp_protocol::title_width, vamp_protocol::title_height} : coverage_image{};
}
private:
std::array<std::char_t, vamp_protocol::title_id_size + 1U> identifier_{};
std::array<std::char_t, vamp_protocol::title_width * vamp_protocol::title_height / 2U> pixels_{};
std::bool_t ready_{};
std::bool_t failed_{};
};
}
include/vamp_pico/display.hpp
+23 -17
View File
@@ -3,7 +3,8 @@
#include <array>
#include <cstdint>
#include <modulus/types.hpp>
#include <vamp_pico/artwork.hpp>
#include <vamp_pico/http_client.hpp>
#include <vamp_style.hpp>
@@ -12,27 +13,32 @@ namespace vamp_pico {
constexpr auto display_width = std::int32_t{320};
constexpr auto display_height = std::int32_t{240};
constexpr auto color_black = std::uint16_t{0x0000};
constexpr auto color_white = std::uint16_t{0xffff};
constexpr auto color_gray = std::uint16_t{0x8410};
enum class link_status { CONNECTING,
OFFLINE,
CONNECTED,
ERROR };
class display {
public:
auto init() -> void;
auto fill(std::uint16_t color) -> void;
auto fill_rect(std::int32_t x, std::int32_t y, std::int32_t width, std::int32_t height, std::uint16_t color) -> void;
auto draw_text(std::int32_t x, std::int32_t y, std::char_t const *text, std::uint16_t color, std::int32_t scale = 2) -> void;
auto show_boot(std::char_t const *line1, std::char_t const *line2, std::uint16_t accent) -> void;
auto show_remote(vamp_style::color status, std::char_t const *title, std::char_t const *part, std::uint32_t page, std::uint32_t pages, std::bool_t active, std::bool_t busy = false) -> void;
auto show_remote(link_status link, remote_display_state const &state, std::bool_t busy = false) -> void;
auto accept_title(std::string_view identifier, std::string_view image) -> std::bool_t;
auto reject_title(std::string_view identifier) -> void;
auto title_ready() const -> std::bool_t;
// At most four scanlines per call, with an 8 ms minimum write interval.
auto poll(std::uint64_t now_ms) -> void;
private:
std::bool_t remote_valid_{};
vamp_style::color last_status_{};
std::bool_t last_active_{};
std::bool_t last_busy_{};
std::array<std::char_t, 96> last_title_{};
std::array<std::char_t, 48> last_part_{};
std::uint32_t last_page_{};
std::uint32_t last_pages_{};
auto dirty(std::size_t begin, std::size_t end) -> void;
title_cache title_{};
std::array<std::bool_t, display_height> dirty_{};
link_status link_{link_status::CONNECTING};
std::bool_t active_{};
std::bool_t busy_{};
sprite part_{sprite::COUNT};
std::uint32_t page_{};
std::uint32_t pages_{};
std::uint64_t last_write_ms_{};
std::size_t next_row_{};
};
}
include/vamp_pico/http_client.hpp
+6 -1
View File
@@ -19,6 +19,7 @@ struct remote_display_state {
std::char_t title[96]{};
std::char_t part[48]{};
std::char_t mode[24]{};
std::array<std::char_t, vamp_protocol::title_id_size + 1U> title_image{};
std::uint32_t page{};
std::uint32_t pages{};
};
@@ -56,6 +57,9 @@ public:
auto take_result() -> std::optional<http_result>;
// Discards the request/result and returns to IDLE. Call before Wi-Fi deinit.
auto cancel() -> void;
// View remains valid until start/cancel. Copy only after matching the selected ID.
auto title_image() const -> std::string_view;
auto current_kind() const -> request_kind;
private:
friend struct http_client_callbacks;
tcp_pcb *pcb_{};
@@ -66,8 +70,9 @@ private:
std::uint64_t timeout_ms_{};
std::array<std::char_t, 384> request_buffer_{};
std::size_t request_size_{};
std::array<std::char_t, 1024> response_buffer_{};
std::array<std::char_t, vamp_protocol::title_image_size + 1024U> response_buffer_{};
std::size_t response_size_{};
std::size_t title_offset_{};
};
}
include/vamp_pico/input.hpp
+7
View File
@@ -14,11 +14,13 @@ enum class request_kind {
DISPLAY,
COMMAND,
HEARTBEAT,
TITLE,
};
struct remote_request {
request_kind kind{request_kind::DISPLAY};
vamp_protocol::page_command command{vamp_protocol::page_command::NEXT};
std::array<std::char_t, vamp_protocol::title_id_size + 1U> title_id{};
};
enum class enqueue_result {
@@ -33,6 +35,8 @@ public:
auto next(std::uint64_t now_ms) -> std::optional<remote_request>;
auto complete(std::uint64_t now_ms) -> void;
auto queued() const -> std::size_t;
auto want_title(std::string_view identifier) -> void;
auto cancel_title() -> void;
private:
std::array<vamp_protocol::page_command, command_capacity> commands_{};
std::size_t head_{};
@@ -40,6 +44,9 @@ private:
std::bool_t in_flight_{};
std::bool_t completed_{};
std::uint64_t completed_ms_{};
std::array<std::char_t, vamp_protocol::title_id_size + 1U> title_id_{};
std::bool_t title_pending_{};
request_kind active_kind_{request_kind::DISPLAY};
};
}
include/vamp_protocol/page_control.hpp
+30
View File
@@ -1,6 +1,9 @@
#pragma once
#include <modulus/types.hpp>
#include <cstddef>
#include <cstdint>
#include <string_view>
namespace vamp_protocol {
@@ -12,6 +15,33 @@ constexpr std::char_t remote_previous_path[] = "/api/remote/previous";
constexpr std::char_t remote_select_path[] = "/api/remote/select";
constexpr std::char_t remote_heartbeat_path[] = "/api/remote/heartbeat";
constexpr std::char_t remote_display_path[] = "/api/remote/display";
constexpr std::char_t remote_title_path[] = "/api/remote/title";
constexpr auto title_width = std::size_t{296};
constexpr auto title_height = std::size_t{64};
constexpr auto title_header_size = std::size_t{8};
constexpr auto title_image_size = title_header_size + title_width * title_height / 2U;
constexpr auto title_id_size = std::size_t{64};
inline auto valid_title_id(std::string_view value) -> std::bool_t {
if (value.size() != title_id_size) {
return false;
}
for (auto character : value) {
if (!((character >= '0' && character <= '9') || (character >= 'a' && character <= 'f'))) {
return false;
}
}
return true;
}
// VPTI, little-endian uint16 dimensions, then row-major coverage, high nibble first.
inline auto valid_title_image(std::string_view bytes) -> std::bool_t {
return bytes.size() == title_image_size && bytes.substr(0U, 4U) == "VPTI" &&
static_cast<std::uint8_t>(bytes[4]) == (title_width & 255U) &&
static_cast<std::uint8_t>(bytes[5]) == (title_width >> 8U) &&
static_cast<std::uint8_t>(bytes[6]) == title_height && bytes[7] == 0;
}
enum class page_command {
PREVIOUS,
include/vamp_style.hpp
+3
View File
@@ -1,12 +1,15 @@
#pragma once
#include <cstdint>
#include <string_view>
#include <modulus/types.hpp>
namespace vamp_style {
constexpr auto ui_font_path = std::string_view{"/usr/share/product-vamp/fonts/petaluma/PetalumaScript.otf"};
struct color {
std::uint8_t r{};
std::uint8_t g{};
include/vamp_style/font.hpp
-26
View File
@@ -1,26 +0,0 @@
#pragma once
#include <array>
#include <cstdint>
#include <string_view>
#include <modulus/types.hpp>
namespace vamp_style {
constexpr auto ui_font_path = std::string_view{"/usr/share/product-vamp/fonts/petaluma/PetalumaScript.otf"};
constexpr auto ui_font_height = std::int32_t{36};
constexpr auto ui_font_max_width = std::int32_t{28};
struct font_glyph {
std::array<std::uint8_t, ui_font_height * ui_font_max_width> alpha{};
std::uint8_t width{};
std::uint8_t advance{};
};
auto ui_font_glyph(std::char_t value) -> vamp_style::font_glyph;
auto ui_font_pair_adjust(std::char_t left, std::char_t right) -> std::int32_t;
auto ui_text_width(std::string_view text, std::int32_t scale) -> std::int32_t;
}
include/vamp_zero/portal.hpp
+1
View File
@@ -37,5 +37,6 @@ private:
auto make_router(vamp_zero::control_state &state) -> ligo::service_router;
auto portal_status_json(vamp_zero::config const &config, vamp_zero::app_state const &state) -> boost::json::value;
auto validated_title_path(std::string const &assets_path, std::string const &identifier) -> std::expected<std::string, modulus::error>;
}
include/vamp_zero/types.hpp
+1
View File
@@ -40,6 +40,7 @@ struct library_part {
std::string source_path{};
std::size_t measure_count{};
std::size_t total_pages{};
std::string title_image{};
};
struct song_group {
src/vamp_pico/artwork.S.in
+8
View File
@@ -0,0 +1,8 @@
.section .rodata.vamp_pico_artwork,"a",%progbits
.balign 4
.global vamp_pico_artwork_begin
.global vamp_pico_artwork_end
vamp_pico_artwork_begin:
.incbin "@VAMP_PICO_ARTWORK_BINARY@"
vamp_pico_artwork_end:
.section .note.GNU-stack,"",%progbits
src/vamp_pico/display.cpp
+128 -155
View File
@@ -1,7 +1,5 @@
#include <algorithm>
#include <array>
#include <cinttypes>
#include <cstdio>
#include <cstdint>
#include <cstring>
@@ -10,7 +8,6 @@
#include <pico/stdlib.h>
#include <vamp_pico/display.hpp>
#include <vamp_style/font.hpp>
namespace {
@@ -73,53 +70,50 @@ auto set_window(std::int32_t x0, std::int32_t y0, std::int32_t x1, std::int32_t
command(0x2c);
}
auto same_color(vamp_style::color left, vamp_style::color right) -> std::bool_t {
return left.r == right.r && left.g == right.g && left.b == right.b && left.a == right.a;
extern "C" std::char_t const vamp_pico_artwork_begin[];
extern "C" std::char_t const vamp_pico_artwork_end[];
auto sprite_image(vamp_pico::sprite value) -> vamp_pico::coverage_image {
return vamp_pico::static_sprite({vamp_pico_artwork_begin,
static_cast<std::size_t>(vamp_pico_artwork_end - vamp_pico_artwork_begin)},
value);
}
template <std::size_t Count>
auto same_text(std::array<std::char_t, Count> const &cached, std::char_t const *text) -> std::bool_t {
auto const *value = text == nullptr ? "" : text;
return std::strncmp(cached.data(), value, Count) == 0;
auto channel(std::uint8_t fg, std::uint8_t bg, std::uint8_t alpha) -> std::uint8_t {
return static_cast<std::uint8_t>((fg * alpha + bg * (15U - alpha) + 7U) / 15U);
}
template <std::size_t Count>
auto copy_text(std::array<std::char_t, Count> &cached, std::char_t const *text) -> void {
auto const *value = text == nullptr ? "" : text;
auto count = std::min(Count - 1U, std::strlen(value));
std::memcpy(cached.data(), value, count);
cached[count] = '\0';
auto blend(vamp_style::color fg, std::uint8_t alpha) -> std::uint16_t {
auto bg = vamp_style::background;
return vamp_style::rgb565({channel(fg.r, bg.r, alpha), channel(fg.g, bg.g, alpha), channel(fg.b, bg.b, alpha), 255});
}
auto blend_channel(std::uint8_t foreground, std::uint8_t background, std::uint8_t alpha) -> std::uint8_t {
auto fg = static_cast<std::uint32_t>(foreground) * alpha;
auto bg = static_cast<std::uint32_t>(background) * (15U - alpha);
return static_cast<std::uint8_t>((fg + bg + 7U) / 15U);
}
auto blend(vamp_style::color foreground, vamp_style::color background, std::uint8_t alpha) -> vamp_style::color {
if (alpha >= 15U) {
return foreground;
}
return {
.r = blend_channel(foreground.r, background.r, alpha),
.g = blend_channel(foreground.g, background.g, alpha),
.b = blend_channel(foreground.b, background.b, alpha),
.a = 255,
};
}
auto draw_centered_text(vamp_pico::display &display, std::int32_t y, std::char_t const *text, std::uint16_t color, std::int32_t preferred_scale, std::int32_t max_width) -> void {
if (text == nullptr || text[0] == '\0') {
auto overlay(std::array<std::uint16_t, vamp_pico::display_width> &row, std::size_t y, vamp_pico::coverage_image image, std::size_t x, std::size_t top, vamp_style::color ink) -> void {
if (y < top || y - top >= image.height) {
return;
}
auto scale = preferred_scale;
while (scale > 1 && vamp_style::ui_text_width(text, scale) > max_width) {
--scale;
for (auto column = std::size_t{}; column < image.width && x + column < row.size(); ++column) {
row[x + column] = blend(ink, image.at(column, y - top));
}
auto width = vamp_style::ui_text_width(text, scale);
auto x = std::max<std::int32_t>(12, (vamp_pico::display_width - width) / 2);
display.draw_text(x, y, text, color, scale);
}
auto centered(std::array<std::uint16_t, vamp_pico::display_width> &row, std::size_t y, vamp_pico::sprite value, std::size_t top, vamp_style::color ink) -> void {
auto image = sprite_image(value);
overlay(row, y, image, (row.size() - image.width) / 2U, top, ink);
}
auto status_color(vamp_pico::link_status link) -> vamp_style::color {
switch (link) {
case vamp_pico::link_status::CONNECTING:
return vamp_style::connecting;
case vamp_pico::link_status::CONNECTED:
return vamp_style::connected;
case vamp_pico::link_status::ERROR:
return vamp_style::danger;
case vamp_pico::link_status::OFFLINE:
return vamp_style::retrying;
}
return vamp_style::retrying;
}
}
@@ -155,135 +149,114 @@ auto vamp_pico::display::init() -> void {
command(0x13);
command(0x29);
sleep_ms(20);
fill(vamp_pico::color_black);
remote_valid_ = false;
dirty(0U, display_height);
}
auto vamp_pico::display::fill(std::uint16_t color) -> void {
fill_rect(0, 0, vamp_pico::display_width, vamp_pico::display_height, color);
auto vamp_pico::display::dirty(std::size_t begin, std::size_t end) -> void {
std::fill(dirty_.begin() + begin, dirty_.begin() + end, true);
}
auto vamp_pico::display::fill_rect(std::int32_t x, std::int32_t y, std::int32_t width, std::int32_t height, std::uint16_t color) -> void {
if (width <= 0 || height <= 0) {
return;
auto vamp_pico::display::show_remote(link_status link, remote_display_state const &state, std::bool_t busy) -> void {
if (link != link_ || busy != busy_) {
dirty(0U, 24U);
dirty(144U, 172U);
}
auto x0 = std::clamp(x, std::int32_t{0}, vamp_pico::display_width - std::int32_t{1});
auto y0 = std::clamp(y, std::int32_t{0}, vamp_pico::display_height - std::int32_t{1});
auto x1 = std::clamp(x + width - std::int32_t{1}, std::int32_t{0}, vamp_pico::display_width - std::int32_t{1});
auto y1 = std::clamp(y + height - std::int32_t{1}, std::int32_t{0}, vamp_pico::display_height - std::int32_t{1});
if (x1 < x0 || y1 < y0) {
return;
auto title_changed = title_.select(state.active ? std::string_view{state.title_image.data()} : std::string_view{});
if (active_ != state.active || title_changed) {
dirty(56U, 120U);
dirty(144U, 172U);
}
set_window(x0, y0, x1, y1);
auto hi = static_cast<std::uchar_t>((color >> 8) & 0xff);
auto lo = static_cast<std::uchar_t>(color & 0xff);
auto row = std::array<std::uchar_t, vamp_pico::display_width * 2>{};
for (auto i = std::size_t{}; i < row.size(); i += 2U) {
row[i] = hi;
row[i + 1U] = lo;
if (state.active && state.title_image[0] == 0 && !title_.failed()) {
reject_title("");
}
gpio_put(pin_lcd_dc, 1);
select();
for (auto yy = y0; yy <= y1; ++yy) {
spi_write_blocking(spi1, row.data(), static_cast<std::size_t>(x1 - x0 + 1) * 2U);
if (active_ != state.active || page_ != state.page || pages_ != state.pages) {
dirty(188U, 216U);
}
deselect();
auto part = std::string_view{state.part};
auto part_sprite = part == "Acoustic" ? sprite::ACOUSTIC : part == "Electric" ? sprite::ELECTRIC
: part == "Vocal" ? sprite::VOCAL
: sprite::COUNT;
if (part_ != part_sprite) {
dirty(144U, 172U);
}
part_ = part_sprite;
active_ = state.active;
page_ = state.page;
pages_ = state.pages;
link_ = link;
busy_ = busy;
}
auto vamp_pico::display::draw_text(std::int32_t x, std::int32_t y, std::char_t const *text, std::uint16_t color, std::int32_t scale) -> void {
if (text == nullptr || scale <= 0) {
auto vamp_pico::display::accept_title(std::string_view identifier, std::string_view image) -> std::bool_t {
if (!title_.accept(identifier, image)) {
return false;
}
dirty(56U, 120U);
dirty(144U, 172U);
return true;
}
auto vamp_pico::display::title_ready() const -> std::bool_t {
return title_.ready();
}
auto vamp_pico::display::reject_title(std::string_view identifier) -> void {
if (title_.fail(identifier)) {
dirty(144U, 172U);
}
}
auto vamp_pico::display::poll(std::uint64_t now_ms) -> void {
if (now_ms - last_write_ms_ < 8U) {
return;
}
auto foreground = vamp_style::color{
.r = static_cast<std::uint8_t>(((color >> 11U) & 0x1fU) * 255U / 31U),
.g = static_cast<std::uint8_t>(((color >> 5U) & 0x3fU) * 255U / 63U),
.b = static_cast<std::uint8_t>((color & 0x1fU) * 255U / 31U),
.a = 255,
};
auto cursor = x;
auto previous = std::char_t{};
for (auto i = std::size_t{}; i < std::strlen(text); ++i) {
if (previous != std::char_t{}) {
cursor += vamp_style::ui_font_pair_adjust(previous, text[i]) * scale;
last_write_ms_ = now_ms;
auto written = std::size_t{};
for (auto scanned = std::size_t{}; scanned < dirty_.size() && written < 4U; ++scanned) {
auto y = next_row_;
next_row_ = (next_row_ + 1U) % dirty_.size();
if (!dirty_[y]) {
continue;
}
auto glyph = vamp_style::ui_font_glyph(text[i]);
for (auto row = std::int32_t{}; row < vamp_style::ui_font_height; ++row) {
auto col = std::int32_t{};
while (col < static_cast<std::int32_t>(glyph.width)) {
auto row_offset = static_cast<std::size_t>(row * vamp_style::ui_font_max_width);
while (col < static_cast<std::int32_t>(glyph.width) &&
glyph.alpha[row_offset + static_cast<std::size_t>(col)] == 0U) {
++col;
}
auto start = col;
auto alpha = start < static_cast<std::int32_t>(glyph.width)
? glyph.alpha[row_offset + static_cast<std::size_t>(start)]
: std::uint8_t{};
while (col < static_cast<std::int32_t>(glyph.width) &&
glyph.alpha[row_offset + static_cast<std::size_t>(col)] == alpha) {
++col;
}
if (col > start) {
auto run_color = vamp_style::rgb565(blend(foreground, vamp_style::background, alpha));
fill_rect(cursor + start * scale, y + row * scale, (col - start) * scale, scale, run_color);
}
static auto row = std::array<std::uint16_t, display_width>{};
row.fill(vamp_style::rgb565(y < 24U ? (busy_ ? vamp_style::danger : status_color(link_)) : vamp_style::background));
if (active_ && title_.ready()) {
overlay(row, y, title_.image(), 12U, 56U, vamp_style::ink);
} else if (!active_) {
centered(row, y, sprite::VAMP, 74U, vamp_style::ink);
}
auto label = busy_ ? sprite::BUSY : link_ == link_status::ERROR ? sprite::ERROR
: link_ == link_status::CONNECTING ? sprite::CONNECTING
: link_ == link_status::OFFLINE ? sprite::OFFLINE
: active_ && title_.failed() ? sprite::UNAVAILABLE
: active_ && !title_.ready() ? sprite::LOADING
: active_ && part_ != sprite::COUNT ? part_
: sprite::READY;
centered(row, y, label, 144U, vamp_style::muted);
if (active_ && pages_ != 0U && y >= 188U && y < 216U) {
auto digits = counter_text(page_, pages_);
auto size = std::strlen(digits.data());
auto label_image = sprite_image(sprite::PAGE);
auto width = label_image.width + 8U + static_cast<std::size_t>(size) * digit_width;
auto x = (row.size() - width) / 2U;
if (size != 0U) {
overlay(row, y, label_image, x, 188U, vamp_style::muted);
}
x += label_image.width + 8U;
for (auto index = std::size_t{}; index < size; ++index) {
auto value = digits[index] == '/' ? sprite::SLASH : static_cast<sprite>(static_cast<std::size_t>(sprite::DIGIT_0) + digits[index] - '0');
overlay(row, y, sprite_image(value), x, 188U, vamp_style::muted);
x += digit_width;
}
}
cursor += static_cast<std::int32_t>(glyph.advance) * scale;
if (cursor >= vamp_pico::display_width) {
return;
static auto bytes = std::array<std::uchar_t, display_width * 2>{};
for (auto x = std::size_t{}; x < row.size(); ++x) {
bytes[x * 2U] = static_cast<std::uchar_t>(row[x] >> 8U);
bytes[x * 2U + 1U] = static_cast<std::uchar_t>(row[x]);
}
previous = text[i];
set_window(0, static_cast<std::int32_t>(y), display_width - 1, static_cast<std::int32_t>(y));
data(bytes.data(), bytes.size());
dirty_[y] = false;
++written;
}
}
auto vamp_pico::display::show_boot(std::char_t const *line1, std::char_t const *line2, std::uint16_t accent) -> void {
remote_valid_ = false;
fill(vamp_style::rgb565(vamp_style::background));
fill_rect(0, 0, vamp_pico::display_width, 28, accent);
draw_text(14, 48, "VAMP", vamp_style::rgb565(vamp_style::ink), 2);
if (line1 != nullptr) {
draw_text(92, 54, line1, vamp_style::rgb565(vamp_style::ink), 1);
}
if (line2 != nullptr) {
draw_text(14, 96, line2, vamp_style::rgb565(vamp_style::muted), 1);
}
}
auto vamp_pico::display::show_remote(vamp_style::color status, std::char_t const *title, std::char_t const *part, std::uint32_t page, std::uint32_t pages, std::bool_t active, std::bool_t busy) -> void {
auto const *title_text = title == nullptr ? "" : title;
auto const *part_text = part == nullptr ? "" : part;
auto status_changed = !remote_valid_ || !same_color(last_status_, status) || last_busy_ != busy;
auto content_changed = !remote_valid_ || last_active_ != active || last_busy_ != busy || !same_text(last_title_, title_text) ||
!same_text(last_part_, part_text) || last_page_ != page || last_pages_ != pages;
if (!remote_valid_) {
fill(vamp_style::rgb565(vamp_style::background));
}
if (status_changed) {
fill_rect(0, 0, vamp_pico::display_width, 24, vamp_style::rgb565(busy ? vamp_style::danger : status));
}
if (content_changed) {
fill_rect(0, 24, vamp_pico::display_width, vamp_pico::display_height - 24, vamp_style::rgb565(vamp_style::background));
if (active && title_text[0] != '\0') {
draw_centered_text(*this, 66, title_text, vamp_style::rgb565(vamp_style::ink), 1, vamp_pico::display_width - 24);
draw_centered_text(*this, 118, busy ? "BUSY" : part_text, vamp_style::rgb565(busy ? vamp_style::ink : vamp_style::muted), 1, vamp_pico::display_width - 24);
if (pages != 0U) {
auto page_text = std::array<std::char_t, 32>{};
auto written = std::snprintf(page_text.data(), page_text.size(), "PAGE %" PRIu32 "/%" PRIu32, page, pages);
if (written > 0) {
draw_centered_text(*this, 168, page_text.data(), vamp_style::rgb565(vamp_style::muted), 1, vamp_pico::display_width - 24);
}
}
} else if (busy) {
draw_centered_text(*this, 118, "BUSY", vamp_style::rgb565(vamp_style::ink), 1, vamp_pico::display_width - 24);
}
}
last_status_ = status;
last_active_ = active;
last_busy_ = busy;
copy_text(last_title_, title_text);
copy_text(last_part_, part_text);
last_page_ = page;
last_pages_ = pages;
remote_valid_ = true;
}
src/vamp_pico/http_client.cpp
+86 -3
View File
@@ -1,5 +1,6 @@
#include <cstdio>
#include <cstdint>
#include <charconv>
#include <lwip/ip_addr.h>
#include <lwip/pbuf.h>
@@ -30,6 +31,65 @@ auto response_is_success(std::string_view response) -> std::bool_t {
response[12] == ' ' && response.find("\r\n\r\n") != std::string_view::npos;
}
auto header_equal(std::string_view name, std::string_view expected) -> std::bool_t {
if (name.size() != expected.size()) {
return false;
}
for (auto index = std::size_t{}; index < name.size(); ++index) {
auto character = name[index];
if (character >= 'A' && character <= 'Z') {
character += 'a' - 'A';
}
if (character != expected[index]) {
return false;
}
}
return true;
}
auto title_body_offset(std::string_view response) -> std::size_t {
if (response.substr(0U, 13U) != "HTTP/1.1 200 " && response.substr(0U, 13U) != "HTTP/1.0 200 ") {
return 0U;
}
auto marker = response.find("\r\n\r\n");
if (marker == std::string_view::npos || marker + 4U > 1024U) {
return 0U;
}
auto length_seen = false;
auto type_seen = false;
auto cursor = response.find("\r\n") + 2U;
while (cursor < marker) {
auto end = response.find("\r\n", cursor);
auto line = response.substr(cursor, end - cursor);
auto colon = line.find(':');
if (colon == std::string_view::npos) {
return 0U;
}
auto name = line.substr(0U, colon);
auto value = line.substr(colon + 1U);
while (!value.empty() && (value.front() == ' ' || value.front() == '\t')) {
value.remove_prefix(1U);
}
if (header_equal(name, "content-length")) {
auto length = std::size_t{};
auto parsed = std::from_chars(value.data(), value.data() + value.size(), length);
if (length_seen || parsed.ec != std::errc{} || parsed.ptr != value.data() + value.size() || length != vamp_protocol::title_image_size) {
return 0U;
}
length_seen = true;
} else if (header_equal(name, "content-type")) {
if (type_seen || value != "application/octet-stream") {
return 0U;
}
type_seen = true;
} else if (header_equal(name, "transfer-encoding") || header_equal(name, "content-encoding")) {
return 0U;
}
cursor = end + 2U;
}
return length_seen && type_seen && vamp_protocol::valid_title_image(response.substr(marker + 4U)) ? marker + 4U : 0U;
}
auto clear_callbacks(tcp_pcb *pcb) -> void {
tcp_arg(pcb, nullptr);
tcp_recv(pcb, nullptr);
@@ -65,7 +125,8 @@ struct http_client_callbacks {
return finish(client, error == ERR_OK && response_is_success(response));
}
auto size = buffer->tot_len;
auto remaining = client.response_buffer_.size() - client.response_size_;
auto capacity = client.request_.kind == request_kind::TITLE ? client.response_buffer_.size() : 1024U;
auto remaining = capacity - client.response_size_;
auto valid = error == ERR_OK && size <= remaining;
if (valid) {
auto copied = pbuf_copy_partial(buffer, client.response_buffer_.data() + client.response_size_, size, 0);
@@ -113,17 +174,27 @@ auto vamp_pico::http_client::start(remote_request request, std::uint64_t now_ms)
status_ = http_status::IN_FLIGHT;
success_ = false;
response_size_ = 0U;
title_offset_ = 0U;
started_ms_ = now_ms;
timeout_ms_ = request.kind == request_kind::DISPLAY ? 1000U : 700U;
timeout_ms_ = request.kind == request_kind::TITLE ? 1500U : request.kind == request_kind::DISPLAY ? 1000U
: 700U;
auto const *path = vamp_protocol::remote_display_path;
if (request.kind == request_kind::COMMAND) {
path = vamp_protocol::page_command_path(request.command);
} else if (request.kind == request_kind::HEARTBEAT) {
path = vamp_protocol::remote_heartbeat_path;
} else if (request.kind == request_kind::TITLE) {
path = vamp_protocol::remote_title_path;
if (request.title_id.back() != 0 || !vamp_protocol::valid_title_id(request.title_id.data())) {
http_client_callbacks::finish(*this, false);
return http_start_result::STARTED;
}
}
auto written = std::int32_t{};
if (request.kind == request_kind::DISPLAY) {
written = std::snprintf(request_buffer_.data(), request_buffer_.size(), "GET %s HTTP/1.1\r\nHost: %s:%d\r\nConnection: close\r\n\r\n", path, VAMP_ZERO_HOST, VAMP_ZERO_PORT);
} else if (request.kind == request_kind::TITLE) {
written = std::snprintf(request_buffer_.data(), request_buffer_.size(), "POST %s HTTP/1.1\r\nHost: %s:%d\r\nContent-Length: 73\r\nContent-Type: application/json\r\nConnection: close\r\n\r\n{\"id\":\"%s\"}", path, VAMP_ZERO_HOST, VAMP_ZERO_PORT, request.title_id.data());
} else {
written = std::snprintf(request_buffer_.data(), request_buffer_.size(), "POST %s HTTP/1.1\r\nHost: %s:%d\r\nContent-Length: 2\r\nContent-Type: application/json\r\nConnection: close\r\n\r\n{}", path, VAMP_ZERO_HOST, VAMP_ZERO_PORT);
}
@@ -162,7 +233,10 @@ auto vamp_pico::http_client::take_result() -> std::optional<http_result> {
auto result = http_result{};
result.request = request_;
result.success = success_;
if (result.success && result.request.kind != request_kind::HEARTBEAT) {
if (result.success && result.request.kind == request_kind::TITLE) {
title_offset_ = title_body_offset({response_buffer_.data(), response_size_});
result.success = title_offset_ != 0U;
} else if (result.success && result.request.kind != request_kind::HEARTBEAT) {
auto text = std::string_view{response_buffer_.data(), response_size_};
auto marker = text.find("\r\n\r\n");
result.success = marker != std::string_view::npos && parse_remote_display_body(text.substr(marker + 4U), result.display);
@@ -176,4 +250,13 @@ auto vamp_pico::http_client::cancel() -> void {
http_client_callbacks::finish(*this, false);
status_ = http_status::IDLE;
response_size_ = 0U;
title_offset_ = 0U;
}
auto vamp_pico::http_client::title_image() const -> std::string_view {
return title_offset_ == 0U ? std::string_view{} : std::string_view{response_buffer_.data() + title_offset_, vamp_protocol::title_image_size};
}
auto vamp_pico::http_client::current_kind() const -> request_kind {
return request_.kind;
}
src/vamp_pico/input.cpp
+26 -2
View File
@@ -1,4 +1,5 @@
#include <vamp_pico/input.hpp>
#include <algorithm>
auto vamp_pico::request_scheduler::enqueue(vamp_protocol::page_command command) -> enqueue_result {
@@ -7,6 +8,7 @@ auto vamp_pico::request_scheduler::enqueue(vamp_protocol::page_command command)
}
commands_[(head_ + size_) % commands_.size()] = command;
++size_;
title_pending_ = false;
return enqueue_result::QUEUED;
}
@@ -20,10 +22,15 @@ auto vamp_pico::request_scheduler::next(std::uint64_t now_ms) -> std::optional<r
request.command = commands_[head_];
head_ = (head_ + 1U) % commands_.size();
--size_;
} else if (title_pending_) {
request.kind = request_kind::TITLE;
request.title_id = title_id_;
title_pending_ = false;
} else if (completed_ && now_ms - completed_ms_ < 1000U) {
return std::nullopt;
}
in_flight_ = true;
active_kind_ = request.kind;
return request;
}
@@ -32,10 +39,27 @@ auto vamp_pico::request_scheduler::complete(std::uint64_t now_ms) -> void {
return;
}
in_flight_ = false;
completed_ = true;
completed_ms_ = now_ms;
if (active_kind_ != request_kind::TITLE) {
completed_ = true;
completed_ms_ = now_ms;
}
}
auto vamp_pico::request_scheduler::queued() const -> std::size_t {
return size_;
}
auto vamp_pico::request_scheduler::want_title(std::string_view identifier) -> void {
title_pending_ = size_ == 0U && vamp_protocol::valid_title_id(identifier);
title_id_.fill(0);
if (title_pending_) {
std::copy(identifier.begin(), identifier.end(), title_id_.begin());
}
}
auto vamp_pico::request_scheduler::cancel_title() -> void {
if (in_flight_ && active_kind_ == request_kind::TITLE) {
in_flight_ = false;
}
title_pending_ = false;
}
src/vamp_pico/main.cpp
+75 -71
View File
@@ -8,96 +8,100 @@
#include <vamp_protocol/page_control.hpp>
namespace {
enum class link_status {
CONNECTING,
RETRYING,
CONNECTED,
};
auto color_for(link_status status) -> vamp_style::color {
switch (status) {
case link_status::CONNECTING:
return vamp_style::connecting;
case link_status::RETRYING:
return vamp_style::retrying;
case link_status::CONNECTED:
return vamp_style::connected;
}
return vamp_style::retrying;
}
auto render(vamp_pico::display &display, link_status status, vamp_pico::remote_display_state const &remote, std::bool_t busy = false) -> void {
display.show_remote(color_for(status), remote.title, remote.part, remote.page, remote.pages, remote.active, busy);
}
auto connect_wifi(vamp_pico::display &display) -> std::bool_t {
auto empty = vamp_pico::remote_display_state{};
render(display, link_status::CONNECTING, empty);
if (cyw43_arch_init() != 0) {
display.show_boot("WIFI", "ERROR", vamp_style::rgb565(vamp_style::danger));
return false;
}
cyw43_arch_enable_sta_mode();
while (cyw43_arch_wifi_connect_timeout_ms(VAMP_PICO_WIFI_SSID, VAMP_PICO_WIFI_PASSPHRASE, CYW43_AUTH_WPA2_AES_PSK, 15000) != 0) {
render(display, link_status::RETRYING, empty);
sleep_ms(2000);
render(display, link_status::CONNECTING, empty);
}
render(display, link_status::RETRYING, empty);
return true;
}
}
auto main() -> int {
stdio_init_all();
auto display = vamp_pico::display{};
// Artwork and network buffers live outside the Pico's small main stack.
static auto display = vamp_pico::display{};
static auto client = vamp_pico::http_client{};
auto touch = vamp_pico::touch_controller{};
display.init();
touch.init();
if (!connect_wifi(display)) {
while (true) {
sleep_ms(1000);
}
}
auto remote = vamp_pico::remote_display_state{};
auto status = link_status::RETRYING;
auto status = vamp_pico::link_status::CONNECTING;
auto gesture = vamp_protocol::gesture_state{};
auto scheduler = vamp_pico::request_scheduler{};
// Keep the owned request buffers off the Pico's small main stack.
static auto client = vamp_pico::http_client{};
auto busy_until_ms = std::uint64_t{};
render(display, status, remote);
auto wifi_ready = cyw43_arch_init() == 0;
if (wifi_ready) {
cyw43_arch_enable_sta_mode();
}
auto wifi_up = false;
auto joining = false;
auto retry_at_ms = std::uint64_t{};
auto join_started_ms = std::uint64_t{};
while (true) {
auto point = vamp_pico::touch_point{};
auto touching = touch.read(point);
auto now_ms = time_us_64() / 1000U;
auto link = CYW43_LINK_DOWN;
if (wifi_ready) {
cyw43_arch_lwip_begin();
link = cyw43_tcpip_link_status(&cyw43_state, CYW43_ITF_STA);
cyw43_arch_lwip_end();
}
if (link != CYW43_LINK_UP) {
if (wifi_up) {
client.cancel();
scheduler = vamp_pico::request_scheduler{};
retry_at_ms = now_ms + 2000U;
}
wifi_up = false;
if (joining && (link < 0 || now_ms - join_started_ms >= 15000U)) {
joining = false;
retry_at_ms = now_ms + 2000U;
}
if (wifi_ready && !joining && now_ms >= retry_at_ms) {
joining = cyw43_arch_wifi_connect_async(VAMP_PICO_WIFI_SSID, VAMP_PICO_WIFI_PASSPHRASE, CYW43_AUTH_WPA2_AES_PSK) == 0;
join_started_ms = now_ms;
retry_at_ms = now_ms + 2000U;
}
status = !wifi_ready ? vamp_pico::link_status::ERROR : joining ? vamp_pico::link_status::CONNECTING
: vamp_pico::link_status::OFFLINE;
} else {
wifi_up = true;
joining = false;
}
if (auto command = gesture.update(touching, now_ms); command) {
if (scheduler.enqueue(*command) == vamp_pico::enqueue_result::BUSY) {
if (!wifi_up || scheduler.enqueue(*command) == vamp_pico::enqueue_result::BUSY) {
busy_until_ms = now_ms + 1000U;
} else if (client.current_kind() == vamp_pico::request_kind::TITLE) {
// An image request is expendable; a command must not wait for its timeout.
client.cancel();
scheduler.cancel_title();
}
}
auto network = client.poll(now_ms);
if (network == vamp_pico::http_status::READY) {
auto result = client.take_result();
scheduler.complete(now_ms);
network = vamp_pico::http_status::IDLE;
if (result && result->success) {
remote = result->display;
status = link_status::CONNECTED;
} else {
status = link_status::RETRYING;
remote = vamp_pico::remote_display_state{};
if (wifi_up) {
auto network = client.poll(now_ms);
if (network == vamp_pico::http_status::READY) {
auto result = client.take_result();
scheduler.complete(now_ms);
network = vamp_pico::http_status::IDLE;
if (result && result->request.kind == vamp_pico::request_kind::TITLE) {
if (result->success && scheduler.queued() == 0U) {
display.accept_title(result->request.title_id.data(), client.title_image());
} else if (!result->success) {
display.reject_title(result->request.title_id.data());
}
} else if (result && result->success) {
remote = result->display;
status = vamp_pico::link_status::CONNECTED;
display.show_remote(status, remote);
if (remote.active && !display.title_ready()) {
scheduler.want_title(remote.title_image.data());
}
} else {
status = vamp_pico::link_status::OFFLINE;
}
}
if (network == vamp_pico::http_status::IDLE) {
if (auto request = scheduler.next(now_ms); request) {
client.start(*request, now_ms);
}
}
}
if (network == vamp_pico::http_status::IDLE) {
if (auto request = scheduler.next(now_ms); request) {
client.start(*request, now_ms);
}
}
render(display, status, remote, now_ms < busy_until_ms);
auto busy = now_ms < busy_until_ms || scheduler.queued() != 0U;
display.show_remote(status, remote, busy);
display.poll(now_ms);
sleep_ms(8);
}
}
src/vamp_pico/remote_display.cpp
+5
View File
@@ -57,6 +57,11 @@ auto vamp_pico::parse_remote_display_body(std::string_view body, vamp_pico::remo
parsed.active = value == "1";
} else if (key == "title") {
copy_value(value, parsed.title, sizeof(parsed.title));
} else if (key == "title_image") {
if (!value.empty() && !vamp_protocol::valid_title_id(value)) {
return false;
}
copy_value(value, parsed.title_image.data(), parsed.title_image.size());
} else if (key == "part") {
copy_value(value, parsed.part, sizeof(parsed.part));
} else if (key == "mode") {
src/vamp_zero/app.cpp
+8
View File
@@ -205,6 +205,10 @@ auto validate_score_pages(boost::asio::any_io_executor executor, vamp_zero::conf
auto validate_catalog(boost::asio::any_io_executor executor, vamp_zero::config const &config, vamp_zero::catalog const &catalog) -> boost::asio::awaitable<std::expected<void, modulus::error>> {
for (auto const &part : catalog.performance.parts) {
auto title = vamp_zero::validated_title_path(config.runtime.assets_path, part.title_image);
if (!title) {
co_return std::unexpected(title.error());
}
auto reference = vamp_zero::score_reference{.category = vamp_zero::score_category::PERFORMANCE, .id = part.id};
auto valid = co_await validate_score_pages(executor, config, std::move(reference));
if (!valid) {
@@ -213,6 +217,10 @@ auto validate_catalog(boost::asio::any_io_executor executor, vamp_zero::config c
}
for (auto const &exercise : catalog.practice.exercises) {
for (auto const &practice_part : exercise.parts) {
auto title = vamp_zero::validated_title_path(config.runtime.assets_path, practice_part.title_image);
if (!title) {
co_return std::unexpected(title.error());
}
auto reference = vamp_zero::score_reference{.category = vamp_zero::score_category::PRACTICE, .id = practice_part.id};
auto valid = co_await validate_score_pages(executor, config, std::move(reference));
if (!valid) {
src/vamp_zero/database.cpp
+12 -5
View File
@@ -88,8 +88,13 @@ auto count_value(noto::sqlite_value const &value) -> std::size_t {
return count == nullptr || *count <= 0 ? 0U : static_cast<std::size_t>(*count);
}
auto valid_title_image(noto::sqlite_value const &value) -> std::bool_t {
auto text = text_value(value);
return text.size() == 64U && std::all_of(text.begin(), text.end(), [](std::char_t c) { return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f'); });
}
auto compatible_table(noto::connection &connection, std::string const &table) -> boost::asio::awaitable<std::bool_t> {
auto columns = std::vector<std::string_view>{"id", "title", "composer", "key_signature", "source_path", "source_json", "measure_count", "total_pages"};
auto columns = std::vector<std::string_view>{"id", "title", "composer", "key_signature", "source_path", "source_json", "measure_count", "total_pages", "title_image"};
if (table == "practice_exercises") {
columns.push_back("exercise_id");
}
@@ -159,7 +164,7 @@ auto vamp_zero::async_load_performance_library(boost::asio::any_io_executor exec
co_return std::unexpected(connected.error());
}
auto selected =
co_await connection.async_prepare("SELECT id, title, composer, part, source_path, measure_count, key_signature, total_pages FROM "
co_await connection.async_prepare("SELECT id, title, composer, part, source_path, measure_count, key_signature, total_pages, title_image FROM "
"songs ORDER BY title COLLATE NOCASE, part COLLATE NOCASE");
if (!selected) {
auto close = co_await connection.async_close();
@@ -180,7 +185,7 @@ auto vamp_zero::async_load_performance_library(boost::asio::any_io_executor exec
break;
}
auto const &row = **next;
if (row.size() != 8U || count_value(row[5]) == 0U || count_value(row[7]) == 0U) {
if (row.size() != 9U || count_value(row[5]) == 0U || count_value(row[7]) == 0U || !valid_title_image(row[8])) {
status = std::unexpected(vamp_zero::make_error(vamp_zero::error_code::PERSISTENCE, "invalid performance metadata"));
break;
}
@@ -198,6 +203,7 @@ auto vamp_zero::async_load_performance_library(boost::asio::any_io_executor exec
.source_path = std::move(*source),
.measure_count = count_value(row[5]),
.total_pages = count_value(row[7]),
.title_image = text_value(row[8]),
});
}
auto closed = co_await close_catalog_query(connection, *selected, std::move(status));
@@ -214,7 +220,7 @@ auto vamp_zero::async_load_practice_library(boost::asio::any_io_executor executo
if (!connected) {
co_return std::unexpected(connected.error());
}
auto selected = co_await connection.async_prepare("SELECT id, title, composer, key_signature, source_path, measure_count, total_pages, part, exercise_id "
auto selected = co_await connection.async_prepare("SELECT id, title, composer, key_signature, source_path, measure_count, total_pages, part, exercise_id, title_image "
"FROM practice_exercises ORDER BY exercise_id, id");
if (!selected) {
auto close = co_await connection.async_close();
@@ -235,7 +241,7 @@ auto vamp_zero::async_load_practice_library(boost::asio::any_io_executor executo
break;
}
auto const &row = **next;
if (row.size() != 9U || count_value(row[5]) == 0U || count_value(row[6]) == 0U || text_value(row[8]).empty() || text_value(row[3]).empty()) {
if (row.size() != 10U || count_value(row[5]) == 0U || count_value(row[6]) == 0U || text_value(row[8]).empty() || text_value(row[3]).empty() || !valid_title_image(row[9])) {
status = std::unexpected(vamp_zero::make_error(vamp_zero::error_code::PERSISTENCE, "invalid practice metadata"));
break;
}
@@ -253,6 +259,7 @@ auto vamp_zero::async_load_practice_library(boost::asio::any_io_executor executo
.source_path = std::move(*source),
.measure_count = count_value(row[5]),
.total_pages = count_value(row[6]),
.title_image = text_value(row[9]),
};
exercises.push_back({
.id = text_value(row[8]),
src/vamp_zero/page_control.cpp
+4
View File
@@ -435,6 +435,7 @@ auto vamp_zero::app_status_json(vamp_zero::app_state const &state) -> boost::jso
auto vamp_zero::remote_display_text(vamp_zero::app_state const &state) -> std::string {
auto title = std::string{};
auto title_image = std::string{};
auto part = std::string{};
auto page = std::size_t{};
auto pages = std::size_t{};
@@ -443,6 +444,7 @@ auto vamp_zero::remote_display_text(vamp_zero::app_state const &state) -> std::s
if (auto const *selected = active_part(state); selected != nullptr) {
active = true;
title = selected->title;
title_image = selected->title_image;
part = selected->part;
page = state.page_index + 1U;
pages = selected->total_pages;
@@ -453,6 +455,7 @@ auto vamp_zero::remote_display_text(vamp_zero::app_state const &state) -> std::s
active = true;
title = selected->title;
part = practice_part == nullptr ? std::string{} : practice_part->part;
title_image = practice_part == nullptr ? std::string{} : practice_part->title_image;
page = state.page_index + 1U;
pages = practice_part == nullptr ? 0U : practice_part->total_pages;
}
@@ -461,6 +464,7 @@ auto vamp_zero::remote_display_text(vamp_zero::app_state const &state) -> std::s
output << "mode=" << vamp_zero::to_string(state.mode) << "\n";
output << "active=" << (active ? "1" : "0") << "\n";
output << "title=" << sanitize_line_value(title) << "\n";
output << "title_image=" << title_image << "\n";
output << "part=" << sanitize_line_value(part) << "\n";
output << "page=" << page << "\n";
output << "pages=" << pages << "\n";
src/vamp_zero/portal.cpp
+116 -1
View File
@@ -1,24 +1,111 @@
#include <vamp_zero/portal.hpp>
#include <chrono>
#include <array>
#include <expected>
#include <filesystem>
#include <fstream>
#include <memory>
#include <mutex>
#include <optional>
#include <set>
#include <string>
#include <utility>
#include <boost/asio/awaitable.hpp>
#include <boost/json/object.hpp>
#include <boost/json/value.hpp>
#include <openssl/sha.h>
#include <vamp_protocol/page_control.hpp>
#include <vamp_zero/page_control.hpp>
#include <vamp_zero/errors.hpp>
namespace {
auto read_asset(std::filesystem::path const &root, std::filesystem::path const &relative, std::size_t limit, std::string *resolved = nullptr) -> std::optional<std::string> {
auto error = std::error_code{};
auto canonical_root = std::filesystem::canonical(root, error);
if (error) {
return std::nullopt;
}
auto path = std::filesystem::canonical(canonical_root / relative, error);
auto contained = path.lexically_relative(canonical_root);
if (error || contained.empty() || *contained.begin() == ".." || !std::filesystem::is_regular_file(path, error) || error) {
return std::nullopt;
}
auto size = std::filesystem::file_size(path, error);
if (error || size == 0U || size > limit) {
return std::nullopt;
}
auto bytes = std::string(static_cast<std::size_t>(size), '\0');
auto input = std::ifstream{path, std::ios::binary};
if (!input.read(bytes.data(), static_cast<std::streamsize>(bytes.size())) || input.peek() != std::ifstream::traits_type::eof()) {
return std::nullopt;
}
if (resolved != nullptr) {
*resolved = path.string();
}
return bytes;
}
struct title_file {
std::string path;
std::string bytes;
};
auto read_title(std::filesystem::path const &root, std::string const &identifier) -> std::optional<title_file> {
if (!vamp_protocol::valid_title_id(identifier)) {
return std::nullopt;
}
auto resolved = std::string{};
auto bytes = read_asset(root, std::filesystem::path{"pico/titles"} / (identifier + ".mask"), vamp_protocol::title_image_size, &resolved);
if (!bytes || !vamp_protocol::valid_title_image(*bytes)) {
return std::nullopt;
}
auto error = std::error_code{};
auto directory = std::filesystem::canonical(root / "pico/titles", error);
if (error || std::filesystem::path{resolved}.parent_path() != directory) {
return std::nullopt;
}
auto digest = std::array<std::uchar_t, SHA256_DIGEST_LENGTH>{};
SHA256(reinterpret_cast<std::uchar_t const *>(bytes->data()), bytes->size(), digest.data());
auto hash = std::string{};
constexpr auto digits = std::string_view{"0123456789abcdef"};
for (auto byte : digest) {
hash.push_back(digits[byte >> 4U]);
hash.push_back(digits[byte & 15U]);
}
if (hash != identifier) {
return std::nullopt;
}
return title_file{std::move(resolved), std::move(*bytes)};
}
class title_assets {
public:
title_assets(std::filesystem::path root, vamp_zero::catalog const &catalog)
: root_{std::move(root)} {
for (auto const &part : catalog.performance.parts) {
identifiers_.insert(part.title_image);
}
for (auto const &exercise : catalog.practice.exercises) {
for (auto const &part : exercise.parts) {
identifiers_.insert(part.title_image);
}
}
}
auto image(std::string const &identifier) const -> std::optional<title_file> {
return identifiers_.contains(identifier) ? read_title(root_, identifier) : std::nullopt;
}
private:
std::filesystem::path root_;
std::set<std::string> identifiers_{};
};
auto portal_html() -> std::string {
return R"html(<!doctype html><html><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Product Vamp</title></head><body><main><h1>Product Vamp</h1><pre id="status">Loading...</pre><p><button id="prev">Previous</button> <button id="select">Select</button> <button id="next">Next</button></p><script>
return R"html(<!doctype html><html><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Product Vamp</title><style>@font-face{font-family:PetalumaScript;src:url('/assets/PetalumaScript.otf') format('opentype');font-display:swap}body,button,pre{font-family:PetalumaScript;font-size:22px}pre{white-space:pre-wrap;overflow-wrap:anywhere}button{font:inherit}</style></head><body><main><h1>Product Vamp</h1><pre id="status">Loading...</pre><p><button id="prev">Previous</button> <button id="select">Select</button> <button id="next">Next</button></p><script>
async function refresh(){const r=await fetch('/api/status');const j=await r.json();document.getElementById('status').textContent=JSON.stringify(j.payload||j,null,2);}
document.getElementById('prev').onclick=async()=>{await fetch('/api/remote/previous',{method:'POST'});refresh();};
document.getElementById('select').onclick=async()=>{await fetch('/api/remote/select',{method:'POST'});refresh();};
@@ -41,6 +128,14 @@ auto remote_response(vamp_zero::app_state const &state) -> ligo::service_respons
}
auto vamp_zero::validated_title_path(std::string const &assets_path, std::string const &identifier) -> std::expected<std::string, modulus::error> {
auto title = read_title(assets_path, identifier);
if (!title) {
return std::unexpected(vamp_zero::make_error(vamp_zero::error_code::INVALID_ARGUMENT, "invalid or missing generated Pico title; regenerate assets"));
}
return std::move(title->path);
}
vamp_zero::control_state::control_state(vamp_zero::config config, vamp_zero::app_state state)
: config_{std::move(config)}, state_{std::move(state)} {}
@@ -163,6 +258,26 @@ auto vamp_zero::portal_status_json(vamp_zero::config const &config, vamp_zero::a
auto vamp_zero::make_router(vamp_zero::control_state &state) -> ligo::service_router {
auto router = ligo::service_router{};
auto assets = state.config().runtime.assets_path;
auto artwork = std::make_shared<title_assets const>(assets, *state.snapshot().catalog);
router.add_response_route("GET", "/assets/PetalumaScript.otf", [assets](ligo::request_context, boost::json::value) -> boost::asio::awaitable<std::expected<ligo::service_response, modulus::error>> {
auto font = read_asset(assets, "fonts/petaluma/PetalumaScript.otf", 1024U * 1024U);
if (!font) {
co_return std::unexpected(vamp_zero::make_error(vamp_zero::error_code::INVALID_ARGUMENT, "PetalumaScript font unavailable"));
}
co_return ligo::text_response("font/otf", std::move(*font));
});
router.add_response_route("POST", vamp_protocol::remote_title_path, [artwork](ligo::request_context, boost::json::value payload) -> boost::asio::awaitable<std::expected<ligo::service_response, modulus::error>> {
auto const *id = payload.is_object() && payload.as_object().size() == 1U ? payload.as_object().if_contains("id") : nullptr;
if (id == nullptr || !id->is_string() || !vamp_protocol::valid_title_id(id->as_string())) {
co_return std::unexpected(vamp_zero::make_error(vamp_zero::error_code::INVALID_ARGUMENT, "invalid title identifier"));
}
auto bytes = artwork->image(std::string{id->as_string()});
if (!bytes) {
co_return std::unexpected(vamp_zero::make_error(vamp_zero::error_code::INVALID_ARGUMENT, "generated title unavailable"));
}
co_return ligo::text_response("application/octet-stream", std::move(bytes->bytes));
});
router.add_route("GET", "/api/health", [](ligo::request_context, boost::json::value) -> boost::asio::awaitable<std::expected<boost::json::value, modulus::error>> {
co_return boost::json::object{{"product", "product-vamp"}, {"device", "zero"}, {"status", "ready"}};
});
src/vamp_zero/ui.cpp
-1
View File
@@ -15,7 +15,6 @@
#include <luma.hpp>
#include <vamp_style.hpp>
#include <vamp_style/font.hpp>
#include <vamp_zero/color.hpp>
#include <vamp_zero/database.hpp>
#include <vamp_zero/errors.hpp>
tests/CMakeLists.txt
+1
View File
@@ -6,6 +6,7 @@ add_executable(product-vamp-test
test_catalog_runtime.cpp
test_health.cpp
test_pico.cpp
test_portal_artwork.cpp
test_playback.cpp
test_score_loader.cpp
test_touch_input.cpp
tests/fixtures/engraving-python.png
Vendored
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 28 KiB

tests/fixtures/notation-python.png
Vendored
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 57 KiB

tests/musicxml_roundtrip.py
+7 -7
View File
@@ -1,9 +1,9 @@
from __future__ import annotations
from pathlib import Path
import pathlib
from generator import harmony
from generator.musicxml import load_musicxml
import generator.harmony
import generator.musicxml
def same(left, right, location: str) -> None:
if left != right:
@@ -16,9 +16,9 @@ def tuplet_members(staff: dict, identifier: str) -> list:
positions = event_positions(staff)
return [positions[event['id']] for measure in staff['measures'] for voice in measure['voices'] for event in voice['notes'] if identifier in event.get('tuplet_ids', [])]
def audit_roundtrip(source: Path, output: Path) -> None:
before = load_musicxml(source)['staves'][0]
after = load_musicxml(output)['staves'][0]
def audit_roundtrip(source: pathlib.Path, output: pathlib.Path) -> None:
before = generator.musicxml.load_musicxml(source)['staves'][0]
after = generator.musicxml.load_musicxml(output)['staves'][0]
before_positions, after_positions = event_positions(before), event_positions(after)
before_instruments = {i['id']: i for i in before.get('instruments', [])}
after_instruments = {i['id']: i for i in after.get('instruments', [])}
@@ -36,7 +36,7 @@ def audit_roundtrip(source: Path, output: Path) -> None:
for a, b in zip(original['harmonies'], rendered['harmonies']):
for field in ('root', 'bass', 'onset'):
same(a.get(field), b.get(field), f'{location} harmony {field}')
same(harmony.degrees(a), harmony.degrees(b), f'{location} harmony degrees')
same(generator.harmony.degrees(a), generator.harmony.degrees(b), f'{location} harmony degrees')
same(len(original['voices']), len(rendered['voices']), f'{location} voice count')
for first, second in zip(original['voices'], rendered['voices']):
same(len(first['notes']), len(second['notes']), f'{location} voice {first["id"]} event count')
tests/rhythm_fixtures.py
+20
View File
@@ -0,0 +1,20 @@
import fractions
import functools
import generator.rhythm
@functools.cache
def library() -> generator.rhythm.Library:
snippets = []
for beats in (2, 3, 4, 5):
bars = []
for mask in range(1 << beats):
events = tuple(generator.rhythm.Event(fractions.Fraction(i), fractions.Fraction(1), bool(mask & (1 << i)), 'quarter') for i in range(beats))
bar = generator.rhythm.Bar(events, fractions.Fraction(beats))
bars.append(bar)
snippets.append(generator.rhythm.Snippet(f'{beats}-4/quarters/{mask}', f'{beats}/4', (bar,)))
for phase in (0, 1):
events = tuple(generator.rhythm.Event(fractions.Fraction(i, 2), fractions.Fraction(1, 2), i % 2 == phase, 'eighth') for i in range(beats * 2))
snippets.append(generator.rhythm.Snippet(f'{beats}-4/eighths/{phase}', f'{beats}/4', (generator.rhythm.Bar(events, fractions.Fraction(beats)),)))
snippets.append(generator.rhythm.Snippet(f'{beats}-4/pair', f'{beats}/4', (bars[1], bars[-1])))
return generator.rhythm.Library(tuple(snippets))
tests/test.cpp
+1 -1
View File
@@ -97,7 +97,7 @@ auto score_asset_json(std::string title, std::string key = "C", std::string part
auto value = boost::json::parse(R"json({
"music":{"metadata":{"title":"Fixture","composer":"Composer","key_signature":"C","time_signature":"4/4"},
"staves":[{"id":"staff-lead","name":"Lead","measures":[{"id":"measure-1","voices":[]}]}]},
"presentation":{"width":976,"height":484,"pages":[{"path":"fixture.pages/0001.png",
"presentation":{"title_image":"0000000000000000000000000000000000000000000000000000000000000000","width":976,"height":484,"pages":[{"path":"fixture.pages/0001.png",
"sha256":"0000000000000000000000000000000000000000000000000000000000000000"}]}
})json");
auto &metadata = value.as_object()["music"].as_object()["metadata"].as_object();
tests/test_accompaniment.py
+378
View File
@@ -0,0 +1,378 @@
import copy
import dataclasses
import fractions
import pathlib
import tempfile
import unittest
import xml.etree.ElementTree
import generator.accompaniment
import generator.model
import generator.playback
import generator.practice.generate
import generator.rhythm
import tests.rhythm_fixtures
import tests.test_musicxml
import tests.test_parts
def xml_bars(contents, *, beats=4, divisions=1, implicit=False):
root = xml.etree.ElementTree.fromstring(tests.test_musicxml.xml_score(contents[0], f'<time><beats>{beats}</beats><beat-type>4</beat-type></time>', divisions=divisions, implicit=implicit))
for index, content in enumerate(contents[1:], 2):
root.find('part').append(xml.etree.ElementTree.fromstring(f'<measure number="{index}">{content}</measure>'))
return xml.etree.ElementTree.tostring(root, encoding='unicode')
def melody_measures(count, *, beats=4, dense=False):
notes = ''.join(tests.test_musicxml.xml_note(duration=1, kind='eighth' if dense else 'quarter') for _ in range(beats * (2 if dense else 1)))
content = tests.test_parts.xml_harmony() + notes
return tests.test_musicxml.parse(xml_bars([content] * count, beats=beats, divisions=2 if dense else 1))['staves'][0]['measures']
def quarter_bar(beats, attacks):
return generator.rhythm.Bar(tuple(generator.rhythm.Event(fractions.Fraction(i), fractions.Fraction(1), i in attacks, 'quarter') for i in range(beats)), fractions.Fraction(beats))
def timing(events):
return [(generator.model.fraction(n['onset']), generator.model.fraction(n['duration']), n['kind']) for n in events]
def tied_library(continuation_duration):
first = tests.test_musicxml.xml_note(extra='<tie type="start"/>')
second = tests.test_musicxml.xml_note(duration=continuation_duration, kind='whole' if continuation_duration == 4 else 'half', extra='<tie type="stop"/>')
if continuation_duration == 2:
second += '<note><rest/><duration>2</duration><type>half</type></note>'
with tempfile.TemporaryDirectory() as temporary:
root = pathlib.Path(temporary)
path = root / '4-4/2/tied.xml'
path.parent.mkdir(parents=True)
path.write_text(xml_bars([first, second]), encoding='utf-8')
return generator.rhythm.load_library(root)
def held_half_bar(sounding=True):
return generator.rhythm.Bar((
generator.rhythm.Event(fractions.Fraction(), fractions.Fraction(2), sounding, 'half'),
generator.rhythm.Event(fractions.Fraction(2), fractions.Fraction(1), False, 'quarter'),
generator.rhythm.Event(fractions.Fraction(3), fractions.Fraction(1), True, 'quarter'),
), fractions.Fraction(4))
class RhythmLibraryTests(unittest.TestCase):
def test_unbracketed_time_modifications_are_rejected(self):
extra = '<time-modification><actual-notes>3</actual-notes><normal-notes>2</normal-notes></time-modification>'
source = xml_bars([tests.test_musicxml.xml_note(duration=2, kind='quarter', extra=extra) * 6], divisions=3)
with self.assertRaisesRegex(ValueError, 'time modifications'):
self.load({'4-4/1/ratio.xml': source})
def load(self, sources):
with tempfile.TemporaryDirectory() as temporary:
root = pathlib.Path(temporary)
for relative, content in sources.items():
path = root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding='utf-8')
return generator.rhythm.load_library(root)
def test_snippet_retains_exact_timing_rests_dots_and_beams(self):
content = tests.test_musicxml.xml_note(duration=3, kind='eighth', extra='<dot/><beam number="1">begin</beam>')
content += tests.test_musicxml.xml_note(duration=1, kind='16th', extra='<beam number="1">end</beam><beam number="2">backward hook</beam>')
content += '<note><rest/><duration>4</duration><type>quarter</type></note>'
content += tests.test_musicxml.xml_note(duration=8, kind='half')
library = self.load({'4-4/1/dotted.xml': xml_bars([content], divisions=4)})
snippet = library.for_meter('4/4')[0]
self.assertEqual((snippet.id, snippet.meter), ('4-4/1/dotted.xml', '4/4'))
self.assertEqual(snippet.bars, (generator.rhythm.Bar((
generator.rhythm.Event(fractions.Fraction(0), fractions.Fraction(3, 4), True, 'eighth', 1, ((1, 'begin'),)),
generator.rhythm.Event(fractions.Fraction(3, 4), fractions.Fraction(1, 4), True, '16th', 0, ((1, 'end'), (2, 'backward hook'))),
generator.rhythm.Event(fractions.Fraction(1), fractions.Fraction(1), False, 'quarter'),
generator.rhythm.Event(fractions.Fraction(2), fractions.Fraction(2), True, 'half'),
), fractions.Fraction(4)),))
self.assertEqual(snippet.bars[0].attacks(), (0, fractions.Fraction(3, 4), 2))
self.assertTrue(all(type(e.onset) is fractions.Fraction and type(e.duration) is fractions.Fraction for e in snippet.bars[0].events))
def test_two_bar_ties_are_continuations_without_new_attacks(self):
content = xml_bars([tests.test_musicxml.xml_note(extra='<tie type="start"/>'), tests.test_musicxml.xml_note(extra='<tie type="stop"/>')])
snippet = self.load({'4-4/2/tied.xml': content}).for_meter('4/4')[0]
self.assertEqual(len(snippet.bars), 2)
self.assertEqual([bar.attacks() for bar in snippet.bars], [(0,), ()])
self.assertEqual([bar.events[0].continuation for bar in snippet.bars], [False, True])
self.assertEqual([bar.length for bar in snippet.bars], [4, 4])
def test_beat_comparison_counts_held_half_notes_and_quarter_reattacks(self):
half = held_half_bar()
silent_half = held_half_bar(False)
quarters = quarter_bar(4, {0, 1, 3})
silent_quarter = quarter_bar(4, {1, 3})
for left, right, changed in ((half, silent_half, [0, 1]), (half, quarters, [1]), (quarters, silent_quarter, [0])):
with self.subTest(left=left, right=right):
self.assertEqual([beat for beat in range(4) if left.beat(beat) != right.beat(beat)], changed)
def test_snippets_and_nested_events_are_immutable(self):
library = tests.rhythm_fixtures.library()
snippet = library.for_meter('4/4')[0]
for value, name, replacement in ((snippet, 'meter', '3/4'), (snippet, 'bars', ()), (snippet.bars[0], 'events', ()), (snippet.bars[0].events[0], 'duration', fractions.Fraction(9))):
with self.subTest(field=name), self.assertRaises(dataclasses.FrozenInstanceError):
setattr(value, name, replacement)
with self.assertRaises(TypeError):
library.snippets['4/4'] = ()
with self.assertRaises(TypeError):
snippet.bars[0].events[0] = snippet.bars[0].events[0]
def test_supported_meters_and_source_order(self):
sources = {}
for beats in (2, 3, 4, 5):
content = xml_bars([''.join(tests.test_musicxml.xml_note(duration=1, kind='quarter') for _ in range(beats))], beats=beats)
for name in ('z', 'a'):
sources[f'{beats}-4/1/{name}.xml'] = content
library = self.load(sources)
for beats in (2, 3, 4, 5):
self.assertEqual([s.id for s in library.for_meter(f'{beats}/4')], [f'{beats}-4/1/a.xml', f'{beats}-4/1/z.xml'])
self.assertTrue(all(s.bars[0].length == beats for s in library.for_meter(f'{beats}/4')))
def test_missing_empty_unsupported_and_misfiled_libraries_fail(self):
with tempfile.TemporaryDirectory() as temporary, self.assertRaisesRegex(ValueError, 'not found'):
generator.rhythm.load_library(pathlib.Path(temporary) / 'missing')
with self.assertRaisesRegex(ValueError, 'empty'):
self.load({})
source = xml_bars([tests.test_musicxml.xml_note()])
for path in ('snippet.xml', '4-4/snippet.xml', '4-4/3/snippet.xml', '4-4/1/nested/snippet.xml', '6-8/1/snippet.xml'):
with self.subTest(path=path), self.assertRaisesRegex(ValueError, 'meter/series'):
self.load({path: source})
with self.assertRaisesRegex(ValueError, 'meter disagrees'):
self.load({'3-4/1/snippet.xml': source})
library = self.load({'4-4/1/snippet.xml': source})
for meter in ('2/4', '6/8', '7/4', '4/8', 'invalid'):
with self.subTest(meter=meter), self.assertRaisesRegex(ValueError, 'rhythm snippets'):
library.for_meter(meter)
with self.assertRaisesRegex(ValueError, 'duplicate rhythm identity'):
generator.rhythm.Library(library.for_meter('4/4') * 2)
def test_invalid_source_formats_fail(self):
quarter = tests.test_musicxml.xml_note(duration=1, kind='quarter')
whole = tests.test_musicxml.xml_note()
triplet = ''.join(tests.test_musicxml.xml_note(duration=1, kind='eighth', extra='<time-modification><actual-notes>3</actual-notes><normal-notes>2</normal-notes><normal-type>eighth</normal-type></time-modification><notations><tuplet type="' + ('start' if i == 0 else 'stop') + '"/></notations>' if i in (0, 2) else '<time-modification><actual-notes>3</actual-notes><normal-notes>2</normal-notes><normal-type>eighth</normal-type></time-modification>') for i in range(3))
cases = {
'three bars': xml_bars([whole] * 3),
'pickup': xml_bars([quarter], implicit=True),
'multiple voices': xml_bars([whole + '<backup><duration>4</duration></backup>' + tests.test_musicxml.xml_note('E', extra='<voice>2</voice>')]),
'meter change': xml_bars([whole, '<attributes><time><beats>3</beats><beat-type>4</beat-type></time></attributes>' + quarter * 3]),
'mid-measure meter': xml_bars([quarter + '<attributes><time><beats>3</beats><beat-type>4</beat-type></time></attributes>' + quarter * 3]),
'grace': xml_bars(['<note><grace/><pitch><step>D</step><octave>4</octave></pitch><type>eighth</type></note>' + whole]),
'spacer': xml_bars(['<forward><duration>1</duration><voice>1</voice></forward>' + quarter * 3]),
'tremolo': xml_bars([tests.test_musicxml.xml_note(extra='<notations><ornaments><tremolo type="single">1</tremolo></ornaments></notations>')]),
'tuplet': xml_bars([triplet + tests.test_musicxml.xml_note(duration=9, kind='half', extra='<dot/>')], divisions=3),
'malformed XML': '<broken',
'wrong root': '<score-timewise version="4.0"/>',
}
for name, content in cases.items():
with self.subTest(source=name), self.assertRaises(ValueError):
self.load({'4-4/1/invalid.xml': content})
class AccompanimentPlanningTests(unittest.TestCase):
def test_pickup_coverage_excludes_notes_that_ended_before_the_excerpt(self):
measure = melody_measures(1)[0]
measure.update(implicit=True, duration=generator.model.rational(1))
item = generator.accompaniment.context(measure, None)
bar = quarter_bar(4, {0, 1})
self.assertEqual(generator.accompaniment.sounding_spans(bar, item), ())
self.assertEqual(generator.accompaniment.rank(bar, [(item, ())], False)[0], 1)
def test_melody_context_uses_audible_hidden_notes_but_not_cues(self):
measure = melody_measures(1)[0]
notes = measure['voices'][0]['notes']
notes[0]['hidden'] = True
notes[1]['cue'] = True
notes[2]['pitches'][0]['cue'] = True
item = generator.accompaniment.context(measure, None)
self.assertEqual(item.attacks, frozenset((0, 3)))
self.assertEqual(item.occupied, ((0, 1), (3, 4)))
def test_fourth_bar_cannot_replace_a_held_half_note_as_one_beat_variation(self):
sounding, silent = held_half_bar(), held_half_bar(False)
library = generator.rhythm.Library((generator.rhythm.Snippet('sounding-half', '4/4', (sounding,)), generator.rhythm.Snippet('silent-half', '4/4', (silent,))))
content = tests.test_parts.xml_harmony() + tests.test_musicxml.xml_note()
measures = tests.test_musicxml.parse(xml_bars([content] * 4))['staves'][0]['measures']
plan = generator.accompaniment.plan_section(measures, library)
self.assertEqual(plan.acoustic, (sounding,) * 4)
self.assertEqual(plan.electric, (silent,) * 4)
def test_section_repeats_four_bar_motif_with_at_most_one_beat_variation(self):
for beats in (2, 3, 4, 5):
measures = melody_measures(12, beats=beats, dense=True)
before = copy.deepcopy(measures)
library = tests.rhythm_fixtures.library()
plan = generator.accompaniment.plan_section(measures, library)
for part in (plan.acoustic, plan.electric):
with self.subTest(beats=beats, part=part):
self.assertEqual(len(part), 12)
self.assertEqual(part[:4], part[4:8])
self.assertEqual(part[:4], part[8:])
self.assertEqual(part[0], part[2])
self.assertLessEqual(sum(part[1].beat(i) != part[3].beat(i) for i in range(beats)), 1)
self.assertTrue(all(bar in [b for s in library.for_meter(f'{beats}/4') for b in s.bars] for bar in part))
reordered = generator.rhythm.Library(tuple(reversed([s for group in library.snippets.values() for s in group])))
self.assertEqual(generator.accompaniment.plan_section(measures, reordered), plan)
self.assertEqual(measures, before)
def test_acoustic_and_electric_coordinate_when_space_is_available(self):
content = tests.test_parts.xml_harmony() + tests.test_musicxml.xml_note()
measures = tests.test_musicxml.parse(xml_bars([content] * 4))['staves'][0]['measures']
plan = generator.accompaniment.plan_section(measures, tests.rhythm_fixtures.library())
for index in range(3):
acoustic, electric = set(plan.acoustic[index].attacks()), set(plan.electric[index].attacks())
self.assertTrue(acoustic)
self.assertTrue(electric)
self.assertFalse(acoustic & electric)
self.assertNotIn(0, acoustic | electric)
self.assertGreater(len(acoustic), len(electric))
def test_dense_melody_allows_collisions_and_keeps_accompaniment_audible(self):
measures = melody_measures(8, dense=True)
plan = generator.accompaniment.plan_section(measures, tests.rhythm_fixtures.library())
for index, measure in enumerate(measures):
melody = {generator.model.fraction(n['onset']) for n in measure['voices'][0]['notes']}
for bars in (plan.acoustic, plan.electric):
events = generator.accompaniment.realize(measure, bars[index], None)
attacks = {generator.model.fraction(n['onset']) for n in events if n['kind'] == 'note'}
self.assertTrue(attacks, (index, bars))
self.assertTrue(attacks & melody)
def test_section_and_local_meter_changes_restart_the_motif(self):
library = generator.rhythm.Library(tuple(generator.rhythm.Snippet(f'{beats}-4/pair', f'{beats}/4', (quarter_bar(beats, {0}), quarter_bar(beats, {1}))) for beats in (3, 4)))
segments = [melody_measures(3), melody_measures(2, beats=3), melody_measures(3), melody_measures(4)]
segments[-1][0]['section_label'] = 'Bridge'
measures = [m for segment in segments for m in segment]
plan = generator.accompaniment.plan_section(measures, library)
start = 0
for segment in segments:
expected = generator.accompaniment.plan_section(segment, library)
self.assertEqual(plan.acoustic[start:start + len(segment)], expected.acoustic)
self.assertEqual(plan.electric[start:start + len(segment)], expected.electric)
self.assertEqual(plan.acoustic[start].attacks(), (0,))
start += len(segment)
def test_mid_measure_meter_changes_and_unsupported_meters_fail(self):
content = tests.test_parts.xml_harmony() + tests.test_musicxml.xml_note(duration=1, kind='quarter') + '<attributes><time><beats>3</beats><beat-type>4</beat-type></time></attributes>' + tests.test_musicxml.xml_note(duration=3, kind='half', extra='<dot/>')
measures = tests.test_musicxml.parse(xml_bars([content]))['staves'][0]['measures']
with self.assertRaisesRegex(ValueError, 'measure 1.*mid-measure meter'):
generator.accompaniment.plan_section(measures, tests.rhythm_fixtures.library())
content = '<time><beats>6</beats><beat-type>8</beat-type></time>'
source = tests.test_musicxml.xml_score(tests.test_parts.xml_harmony() + tests.test_musicxml.xml_note(duration=3, kind='half', extra='<dot/>'), content)
with self.assertRaisesRegex(ValueError, '6/8'):
generator.accompaniment.plan_section(tests.test_musicxml.parse(source)['staves'][0]['measures'], tests.rhythm_fixtures.library())
def test_carried_harmony_and_no_chord_control_active_intervals(self):
content = tests.test_musicxml.xml_note()
measures = tests.test_musicxml.parse(xml_bars([content, tests.test_parts.xml_harmony(kind='none', offset=2) + content, content]))['staves'][0]['measures']
initial = melody_measures(1)[0]['harmonies'][0]
previous = generator.accompaniment.carried_chords(measures, initial)
self.assertEqual(previous, [initial, initial, measures[1]['harmonies'][0]])
contexts = [generator.accompaniment.context(m, chord) for m, chord in zip(measures, previous)]
self.assertEqual([item.intervals for item in contexts], [((0, 4),), ((0, 2),), ()])
plan = generator.accompaniment.plan_section(measures, tests.rhythm_fixtures.library(), initial=initial)
self.assertTrue(generator.accompaniment.sounding_attacks(plan.acoustic[0], contexts[0]))
self.assertEqual(generator.accompaniment.sounding_attacks(plan.acoustic[2], contexts[2]), ())
class AccompanimentRealizationTests(unittest.TestCase):
def test_harmonic_gating_removes_dangling_beams(self):
content = tests.test_parts.xml_harmony() + tests.test_parts.xml_harmony(kind='none', offset=1) + tests.test_musicxml.xml_note(duration=8)
measure = tests.test_musicxml.parse(xml_bars([content], divisions=2))['staves'][0]['measures'][0]
bar = generator.rhythm.Bar((generator.rhythm.Event(fractions.Fraction(), fractions.Fraction(1, 2), True, 'eighth', beams=((1, 'begin'),)), generator.rhythm.Event(fractions.Fraction(1, 2), fractions.Fraction(1, 2), True, 'eighth', beams=((1, 'end'),)), generator.rhythm.Event(fractions.Fraction(1), fractions.Fraction(3), False, 'half', 1)), fractions.Fraction(4))
events = generator.accompaniment.realize(measure, bar, None)
self.assertEqual([n['kind'] for n in events], ['note', 'rest', 'rest'])
self.assertTrue(all('beams' not in n for n in events))
def test_harmony_and_no_chord_cut_sustain_without_injecting_attacks(self):
bar = generator.rhythm.Bar((generator.rhythm.Event(fractions.Fraction(), fractions.Fraction(4), True, 'whole'),), fractions.Fraction(4))
for kind in ('major', 'none'):
content = tests.test_parts.xml_harmony() + tests.test_parts.xml_harmony('F', kind, offset=3) + tests.test_parts.xml_harmony('G', offset=5) + tests.test_musicxml.xml_note(duration=8)
measure = tests.test_musicxml.parse(xml_bars([content], divisions=2))['staves'][0]['measures'][0]
before = copy.deepcopy(measure)
events = generator.accompaniment.realize(measure, bar, None)
self.assertEqual(timing(events), [(0, fractions.Fraction(3, 2), 'note'), (fractions.Fraction(3, 2), fractions.Fraction(5, 2), 'rest')])
self.assertEqual(events[0]['_harmony']['root']['letter'], 'c')
self.assertEqual(measure, before)
def test_chord_changes_preserve_authored_rhythm_rests_until_next_attack(self):
bar = generator.rhythm.Bar((generator.rhythm.Event(fractions.Fraction(), fractions.Fraction(3), False, 'half', 1), generator.rhythm.Event(fractions.Fraction(3), fractions.Fraction(1), True, 'quarter')), fractions.Fraction(4))
content = tests.test_parts.xml_harmony() + tests.test_parts.xml_harmony('F', offset=1) + tests.test_parts.xml_harmony(kind='none', offset=2) + tests.test_parts.xml_harmony('G', offset=3) + tests.test_musicxml.xml_note()
measure = tests.test_musicxml.parse(xml_bars([content]))['staves'][0]['measures'][0]
events = generator.accompaniment.realize(measure, bar, None)
self.assertEqual(timing(events), [(0, 3, 'rest'), (3, 1, 'note')])
self.assertEqual(events[1]['_harmony']['root']['letter'], 'g')
self.assertEqual(events[0]['written'], {'type': 'half', 'dots': 1})
def test_identical_and_coincident_chords_do_not_retrigger_sustains(self):
bar = generator.rhythm.Bar((generator.rhythm.Event(fractions.Fraction(), fractions.Fraction(4), True, 'whole'),), fractions.Fraction(4))
content = tests.test_parts.xml_harmony() + tests.test_parts.xml_harmony('F') + tests.test_parts.xml_harmony('F', offset=2) + tests.test_musicxml.xml_note()
measure = tests.test_musicxml.parse(xml_bars([content]))['staves'][0]['measures'][0]
events = generator.accompaniment.realize(measure, bar, None)
self.assertEqual(timing(events), [(0, 4, 'note')])
self.assertEqual(events[0]['_harmony']['root']['letter'], 'f')
def test_pickup_uses_right_edge_of_nominal_bar_with_exact_clipping(self):
bar = generator.rhythm.Bar((generator.rhythm.Event(fractions.Fraction(), fractions.Fraction(3), False, 'half', 1), generator.rhythm.Event(fractions.Fraction(3), fractions.Fraction(1), True, 'quarter')), fractions.Fraction(4))
content = tests.test_parts.xml_harmony() + tests.test_musicxml.xml_note(duration=3, kind='quarter', extra='<dot/>')
measure = tests.test_musicxml.parse(xml_bars([content], divisions=2, implicit=True))['staves'][0]['measures'][0]
self.assertEqual(generator.accompaniment.context(measure, None).offset, fractions.Fraction(5, 2))
events = generator.accompaniment.realize(measure, bar, None)
self.assertEqual(timing(events), [(0, fractions.Fraction(1, 2), 'rest'), (fractions.Fraction(1, 2), 1, 'note')])
self.assertEqual(events[1]['written'], {'type': 'quarter', 'dots': 0})
library = generator.rhythm.Library((generator.rhythm.Snippet('pickup', '4/4', (bar,)),))
plan = generator.accompaniment.plan_section([measure], library)
self.assertEqual(plan.acoustic, (bar,))
self.assertEqual(plan.electric, (bar,))
class TiedAccompanimentTests(unittest.TestCase):
def source(self, root='C', kind='major', offset=0):
first = tests.test_parts.xml_harmony() + tests.test_musicxml.xml_note(extra='<tie type="start"/>')
second = tests.test_parts.xml_harmony(root, kind, offset) + tests.test_musicxml.xml_note(extra='<tie type="stop"/>')
return tests.test_musicxml.parse(xml_bars([first, second]))
def assert_tied_chord(self, first, continuation):
self.assertTrue(first['pitches'])
self.assertEqual(len(first['pitches']), len(continuation['pitches']))
for previous, current in zip(first['pitches'], continuation['pitches']):
self.assertEqual(current['tie_from'], previous['id'])
self.assertEqual(previous['tie_to'], current['id'])
self.assertEqual(generator.model.pitch_number(current), generator.model.pitch_number(previous))
self.assertEqual(current['tablature'], previous['tablature'])
for note in (first, continuation):
self.assertNotIn('_rhythm_continuation', note)
for pitch in note['pitches']:
self.assertEqual(pitch['duration'], note['duration'])
self.assertEqual(pitch['written'], note['written'])
def test_whole_note_tie_keeps_second_bar_audible_without_a_new_attack(self):
self.check_tied_rhythm(4)
def test_half_note_continuation_preserves_tone_duration_strings_and_following_rest(self):
self.check_tied_rhythm(2)
def check_tied_rhythm(self, continuation_duration):
source = self.source()
before = copy.deepcopy(source)
library = tied_library(continuation_duration)
bars = library.for_meter('4/4')[0].bars
self.assertEqual(bars[1].attacks(), ())
plan = generator.accompaniment.plan_section(source['staves'][0]['measures'], library)
self.assertEqual(plan.acoustic, bars)
self.assertEqual(plan.electric, bars)
result = generator.practice.generate.practice_document(source, 'acoustic', rhythms=library)
first, second = tests.test_musicxml.notes(result), tests.test_musicxml.notes(result, 1)
self.assertEqual(timing(first), [(0, 4, 'note')])
expected = [(0, continuation_duration, 'note')]
if continuation_duration == 2:
expected.append((2, 2, 'rest'))
self.assertEqual(second[1]['pitches'], [])
self.assertEqual(timing(second), expected)
self.assert_tied_chord(first[0], second[0])
self.assertEqual({generator.model.pitch_number(p) % 12 for p in first[0]['pitches']}, {0, 4, 7})
generator.model.validate_document(result)
timeline = generator.playback.compile_timeline(result)
self.assertEqual(generator.model.fraction(timeline['duration']), 8)
self.assertEqual(len(timeline['events']), len(first[0]['pitches']))
self.assertTrue(all(generator.model.fraction(e['onset']) == 0 and generator.model.fraction(e['duration']) == 4 + continuation_duration for e in timeline['events']))
self.assertEqual(source, before)
self.assertEqual(library.for_meter('4/4')[0].bars, bars)
def test_changed_harmony_breaks_rhythm_ties_without_inventing_attacks(self):
for root, kind in (('F', 'major'), ('C', 'minor'), ('C', 'none')):
with self.subTest(root=root, kind=kind):
source = self.source(root, kind)
before = copy.deepcopy(source)
result = generator.practice.generate.practice_document(source, 'acoustic', rhythms=tied_library(2))
first, second = tests.test_musicxml.notes(result), tests.test_musicxml.notes(result, 1)
self.assertEqual(timing(first), [(0, 4, 'note')])
self.assertEqual(timing(second), [(0, 2, 'rest'), (2, 2, 'rest')])
self.assertEqual({generator.model.pitch_number(p) % 12 for p in first[0]['pitches']}, {0, 4, 7})
self.assertEqual(second[0]['pitches'], [])
for note in first + second:
for pitch in note['pitches']:
self.assertNotIn('tie_from', pitch)
self.assertNotIn('tie_to', pitch)
self.assertEqual(pitch['duration'], note['duration'])
generator.model.validate_document(result)
events = generator.playback.compile_timeline(result)['events']
self.assertEqual({(generator.model.fraction(e['onset']), generator.model.fraction(e['duration'])) for e in events}, {(0, 4)})
self.assertEqual(source, before)
def test_harmony_change_inside_continuation_cuts_tied_sound_and_preserves_rest(self):
source = self.source('F', offset=1)
result = generator.practice.generate.practice_document(source, 'acoustic', rhythms=tied_library(2))
first, second = tests.test_musicxml.notes(result), tests.test_musicxml.notes(result, 1)
self.assertEqual(timing(first), [(0, 4, 'note')])
self.assertEqual(timing(second), [(0, 1, 'note'), (1, 1, 'rest'), (2, 2, 'rest')])
self.assert_tied_chord(first[0], second[0])
generator.model.validate_document(result)
events = generator.playback.compile_timeline(result)['events']
self.assertEqual(len(events), len(first[0]['pitches']))
self.assertTrue(all(generator.model.fraction(e['onset']) == 0 and generator.model.fraction(e['duration']) == 5 for e in events))
tests/test_build_bootstrap.py
+13 -13
View File
@@ -1,18 +1,18 @@
import json
import importlib.util
import json
import os
from pathlib import Path
import pathlib
import shutil
import subprocess
import sys
import tempfile
import types
import unittest
from unittest.mock import Mock, patch
import unittest.mock
class BuildBootstrapTests(unittest.TestCase):
def test_arm_profile_selects_versioned_compilers_available_in_ci_and_docker(self):
root = Path(__file__).resolve().parents[1]
root = pathlib.Path(__file__).resolve().parents[1]
profile = (root / 'conan/profiles/arm_64_gcc12_release').read_text()
self.assertIn('tools.build:compiler_executables={"c": "aarch64-linux-gnu-gcc-12", "cpp": "aarch64-linux-gnu-g++-12"}', profile)
for path in ('.gitea/workflows/ci.yml', 'package/Dockerfile.build'):
@@ -22,12 +22,12 @@ class BuildBootstrapTests(unittest.TestCase):
self.assertNotIn(f'{compiler}-aarch64-linux-gnu', source)
@unittest.skipUnless(shutil.which('conan'), 'Conan is required for package identity checks')
def test_arm_package_ids_exclude_unverified_internal_binaries_only(self):
root = Path(__file__).resolve().parents[1]
root = pathlib.Path(__file__).resolve().parents[1]
if not (root / 'build/build_tools/conan/profiles/arm_64_gcc12_release').is_file():
self.skipTest('Configure first to export the shared Conan profiles')
profile = root / 'conan/profiles/arm_64_gcc12_release'
with tempfile.TemporaryDirectory() as temporary:
directory = Path(temporary)
directory = pathlib.Path(temporary)
environment = dict(os.environ, CONAN_HOME=str(directory / 'conan'), CONAN_NON_INTERACTIVE='1')
command = ['conan', 'graph', 'info', str(directory), f'--profile:host={profile}', f'--profile:build={profile}', '--no-remote', '--format=json']
def package_id(*extra):
@@ -46,7 +46,7 @@ class BuildBootstrapTests(unittest.TestCase):
self.assertNotEqual(current, legacy)
self.assertNotEqual(current, different_compiler)
def test_build_recipe_is_declared_and_editor_commands_do_not_bootstrap(self):
root = Path(__file__).resolve().parents[1]
root = pathlib.Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('vamp_bootstrap', root / 'modulus.py')
bootstrap = importlib.util.module_from_spec(spec)
spec.loader.exec_module(bootstrap)
@@ -57,14 +57,14 @@ class BuildBootstrapTests(unittest.TestCase):
package = types.ModuleType('modulus')
cli = types.ModuleType('modulus.cli')
package.cli = cli
docker_run = Mock()
docker_run = unittest.mock.Mock()
cli.docker_run = docker_run
cli.main = Mock(return_value=0)
cli.main = unittest.mock.Mock(return_value=0)
for command in ('clangd', 'format'):
with patch.dict(sys.modules, {'modulus': package, 'modulus.cli': cli}), \
patch.object(bootstrap, 'ensure_runtime') as prepare, \
patch.object(bootstrap, 'runtime_ready', return_value=True), \
patch.object(sys, 'path', list(sys.path)):
with unittest.mock.patch.dict(sys.modules, {'modulus': package, 'modulus.cli': cli}), \
unittest.mock.patch.object(bootstrap, 'ensure_runtime') as prepare, \
unittest.mock.patch.object(bootstrap, 'runtime_ready', return_value=True), \
unittest.mock.patch.object(sys, 'path', list(sys.path)):
self.assertEqual(bootstrap.delegate(command, []), 0)
prepare.assert_not_called()
docker_run.assert_not_called()
tests/test_catalog.py
+28 -27
View File
@@ -1,67 +1,68 @@
import json
from pathlib import Path
import pathlib
import tempfile
import unittest
import unittest.mock
from generator.practice.generate import generate_practice_scores
from generator.arrange.generate import generate_performance_scores
from generator.score import validate_generated_tree
from generator.summary import validate_practice_parts
from generator.parts import PART_TYPES
from unittest.mock import patch
from tests.test_musicxml import xml_note, xml_score
import generator.arrange.generate
import generator.parts
import generator.practice.generate
import generator.score
import generator.summary
import tests.rhythm_fixtures
import tests.test_musicxml
class CatalogTests(unittest.TestCase):
def test_every_numbered_source_is_wired_to_three_part_paths(self):
source = Path('data/practice')
source = pathlib.Path('data/practice')
sources = sorted(source.glob('*.xml'))
self.assertEqual([p.name for p in sources], [f'{i:04}.xml' for i in range(1, 367)])
expected = {Path(path.stem) / (part + '.json') for path in sources for part in PART_TYPES}
expected = {pathlib.Path(path.stem) / (part + '.json') for path in sources for part in generator.parts.PART_TYPES}
self.assertEqual(len(expected), 1098)
with tempfile.TemporaryDirectory() as temporary, patch('generator.practice.generate._practice_document', side_effect=lambda document, part: document):
output = Path(temporary)
self.assertEqual(generate_practice_scores(source, output), len(expected))
with tempfile.TemporaryDirectory() as temporary, unittest.mock.patch('generator.practice.generate._practice_document', side_effect=lambda document, part, plan: document):
output = pathlib.Path(temporary)
self.assertEqual(generator.practice.generate.generate_practice_scores(source, output, rhythms=tests.rhythm_fixtures.library()), len(expected))
self.assertEqual({p.relative_to(output) for p in output.rglob('*.json')}, expected)
validate_practice_parts(output)
validate_generated_tree(output)
generator.summary.validate_practice_parts(output)
generator.score.validate_generated_tree(output)
def test_numbers_and_duplicate_titles_remain_distinct_and_songs_are_excluded(self):
with tempfile.TemporaryDirectory() as temporary:
source, output = Path(temporary) / 'source', Path(temporary) / 'output'
source, output = pathlib.Path(temporary) / 'source', pathlib.Path(temporary) / 'output'
source.mkdir(parents=True)
for name in ('1000.xml', '0000.xml', '0001.xml', '0999.xml', '0010.xml', '0366.xml'):
(source / name).write_text(xml_score(xml_note()), encoding='utf-8')
self.assertEqual(generate_practice_scores(source, output), 18)
(source / name).write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()), encoding='utf-8')
self.assertEqual(generator.practice.generate.generate_practice_scores(source, output, rhythms=tests.rhythm_fixtures.library()), 18)
self.assertEqual(sorted(p.name for p in output.iterdir()), ['0000', '0001', '0010', '0366', '0999', '1000'])
for directory in output.iterdir():
self.assertEqual({p.name for p in directory.glob('*.json')}, {(part + '.json') for part in PART_TYPES})
self.assertEqual({p.name for p in directory.glob('*.json')}, {(part + '.json') for part in generator.parts.PART_TYPES})
self.assertEqual(json.loads((directory / 'vocal.json').read_text())['metadata']['title'], 'Fixture')
def test_invalid_names_and_legacy_nesting_preserve_output(self):
for name in ('1.xml', '001.xml', '00001.xml', '\u0660\u0660\u0660\u0661.xml', '0001-copy.xml', '0001_copy.xml', 'chord/0001.xml', 'practice/melody/lesson/0001.xml'):
with self.subTest(name=name), tempfile.TemporaryDirectory() as temporary:
source, output = Path(temporary) / 'source', Path(temporary) / 'output'
source, output = pathlib.Path(temporary) / 'source', pathlib.Path(temporary) / 'output'
for relative in ('0002.xml', name):
path = source / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(xml_score(xml_note()), encoding='utf-8')
path.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()), encoding='utf-8')
output.mkdir()
sentinel = output / 'existing'
sentinel.write_text('unchanged')
with self.assertRaisesRegex(ValueError, 'practice sources'):
generate_practice_scores(source, output)
generator.practice.generate.generate_practice_scores(source, output, rhythms=tests.rhythm_fixtures.library())
self.assertEqual(list(output.iterdir()), [sentinel])
self.assertEqual(sentinel.read_text(), 'unchanged')
def test_empty_part_directory_fails(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
root = pathlib.Path(temporary)
(root / '0001').mkdir()
with self.assertRaisesRegex(ValueError, 'practice generator drift'):
validate_practice_parts(root)
generator.summary.validate_practice_parts(root)
def test_songs_generate_only_authored_key_parts(self):
with tempfile.TemporaryDirectory() as temporary:
source, output = Path(temporary) / 'songs', Path(temporary) / 'output'
source, output = pathlib.Path(temporary) / 'songs', pathlib.Path(temporary) / 'output'
source.mkdir()
(source / 'song.xml').write_text(xml_score(xml_note('E', alter=-1), '<key><fifths>-3</fifths></key>'))
self.assertEqual(generate_performance_scores(source, output), 3)
(source / 'song.xml').write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note('E', alter=-1), '<key><fifths>-3</fifths></key>'))
self.assertEqual(generator.arrange.generate.generate_performance_scores(source, output, rhythms=tests.rhythm_fixtures.library()), 3)
self.assertEqual({p.name for p in output.iterdir()}, {'song-acoustic.json', 'song-electric.json', 'song-vocal.json'})
for path in output.iterdir():
metadata = json.loads(path.read_text())['metadata']
tests/test_catalog_database.py
+52 -30
View File
@@ -1,17 +1,18 @@
from contextlib import closing
import contextlib
import json
import pathlib
import sqlite3
import subprocess
import tempfile
import unittest
from pathlib import Path
from generator.catalog import APPLICATION_ID, build_catalog
import generator.catalog
def bundle(title='Fixture', key='C', part='Lead'):
return {
'music': {'metadata': {'title': title, 'composer': 'Composer', 'key_signature': key, 'time_signature': '4/4'},
'staves': [{'name': part, 'arrangement_part_type': part.lower(), 'measures': [{'voices': []}]}]},
'presentation': {'width': 976, 'height': 484, 'pages': [{'path': 'fixture.pages/1.png', 'sha256': '0' * 64}]},
'presentation': {'width': 976, 'height': 484, 'title_image': '0' * 64, 'pages': [{'path': 'fixture.pages/1.png', 'sha256': '0' * 64}]},
}
def write_score(root, relative, document=None):
@@ -24,97 +25,118 @@ class CatalogTests(unittest.TestCase):
def setUp(self):
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name)
self.root = pathlib.Path(self.temporary.name)
(self.root / 'songs').mkdir()
def rows(self, table):
with closing(sqlite3.connect(self.root / 'catalog.db')) as connection:
with contextlib.closing(sqlite3.connect(self.root / 'catalog.db')) as connection:
connection.row_factory = sqlite3.Row
return [dict(row) for row in connection.execute(f'SELECT * FROM {table} ORDER BY id')]
def test_metadata_relative_paths_and_complete_documents(self):
document = bundle('Song', 'F')
document['presentation']['title_image'] = '0123456789abcdef' * 4
write_score(self.root, 'songs/tune.json', document)
for part in ('Acoustic', 'Electric', 'Vocal'):
write_score(self.root, f'practice/0001/{part.lower()}.json', bundle('Exercise', 'Eb', part))
database = build_catalog(self.root)
database = generator.catalog.build_catalog(self.root)
self.assertEqual(database, self.root / 'catalog.db')
song = self.rows('songs')[0]
self.assertEqual(json.loads(song['source_json']), document)
self.assertEqual((song['id'], song['part'], song['source_path']), ('tune', 'Lead', 'songs/tune.json'))
self.assertEqual((song['measure_count'], song['total_pages']), (1, 1))
self.assertEqual(song['title_image'], document['presentation']['title_image'])
practice = self.rows('practice_exercises')
self.assertEqual([row['id'] for row in practice], ['0001-acoustic', '0001-electric', '0001-vocal'])
self.assertEqual(practice[0]['exercise_id'], '0001')
self.assertNotIn('practice_type_id', practice[0])
self.assertNotIn('lesson_group_id', practice[0])
self.assertEqual(practice[0]['key_signature'], 'Eb')
self.assertTrue(all(row['title_image'] == '0' * 64 for row in practice))
self.assertTrue(all(json.loads(row['source_json'])['presentation']['title_image'] == row['title_image'] for row in practice))
self.assertNotIn('authored_key_signature', practice[0])
with closing(sqlite3.connect(database)) as connection:
self.assertEqual(connection.execute('PRAGMA application_id').fetchone()[0], APPLICATION_ID)
with contextlib.closing(sqlite3.connect(database)) as connection:
self.assertEqual(connection.execute('PRAGMA application_id').fetchone()[0], generator.catalog.APPLICATION_ID)
self.assertEqual(connection.execute('PRAGMA integrity_check').fetchall(), [('ok',)])
self.assertEqual(connection.execute("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name").fetchall(), [('practice_exercises',), ('songs',)])
self.assertFalse(Path(str(database) + '-wal').exists())
self.assertFalse(Path(str(database) + '-journal').exists())
for table in ('songs', 'practice_exercises'):
columns = {row[1]: row for row in connection.execute(f'PRAGMA table_info({table})')}
self.assertEqual((columns['title_image'][2], columns['title_image'][3]), ('TEXT', 1))
self.assertFalse(pathlib.Path(str(database) + '-wal').exists())
self.assertFalse(pathlib.Path(str(database) + '-journal').exists())
def test_rebuild_is_deterministic_relocatable_and_replaces_deleted_rows(self):
score = write_score(self.root, 'songs/tune.json')
database = build_catalog(self.root)
database = generator.catalog.build_catalog(self.root)
previous = database.read_bytes()
build_catalog(self.root)
generator.catalog.build_catalog(self.root)
self.assertEqual(database.read_bytes(), previous)
with tempfile.TemporaryDirectory() as other:
root = Path(other)
root = pathlib.Path(other)
write_score(root, 'songs/tune.json')
self.assertEqual(build_catalog(root).read_bytes(), previous)
self.assertEqual(generator.catalog.build_catalog(root).read_bytes(), previous)
score.rename(score.with_name('renamed.json'))
build_catalog(self.root)
generator.catalog.build_catalog(self.root)
self.assertEqual([row['id'] for row in self.rows('songs')], ['renamed'])
score.with_name('renamed.json').unlink()
build_catalog(self.root)
generator.catalog.build_catalog(self.root)
self.assertEqual(self.rows('songs'), [])
def test_invalid_input_preserves_existing_database(self):
score = write_score(self.root, 'songs/tune.json')
database = build_catalog(self.root)
database = generator.catalog.build_catalog(self.root)
previous = database.read_bytes()
for document in ({}, bundle(title=''), bundle(key=None)):
score.write_text(json.dumps(document))
with self.assertRaises(ValueError):
build_catalog(self.root)
generator.catalog.build_catalog(self.root)
self.assertEqual(database.read_bytes(), previous)
self.assertFalse(list(self.root.glob('.vamp-catalog-*')))
def test_missing_parts_and_metadata_disagreement_fail(self):
for part in ('Acoustic', 'Electric'):
write_score(self.root, f'practice/0001/{part.lower()}.json', bundle(part=part))
with self.assertRaisesRegex(ValueError, 'all three parts'):
build_catalog(self.root)
generator.catalog.build_catalog(self.root)
write_score(self.root, 'practice/0001/vocal.json', bundle(key='F', part='Vocal'))
with self.assertRaisesRegex(ValueError, 'disagree'):
build_catalog(self.root)
generator.catalog.build_catalog(self.root)
def test_missing_or_invalid_title_image_preserves_existing_database(self):
score = write_score(self.root, 'songs/tune.json')
database = generator.catalog.build_catalog(self.root)
previous = database.read_bytes()
invalid = [bundle() for _ in range(9)]
invalid[0]['presentation'].pop('title_image')
for document, value in zip(invalid[1:], (None, 64, '', '0' * 63, '0' * 65, 'A' * 64, 'g' * 64, '0' * 63 + '\n')):
document['presentation']['title_image'] = value
for document in invalid:
with self.subTest(presentation=document['presentation']):
score.write_text(json.dumps(document), encoding='utf-8')
with self.assertRaisesRegex(ValueError, 'title image'):
generator.catalog.build_catalog(self.root)
self.assertEqual(database.read_bytes(), previous)
self.assertFalse(list(self.root.glob('.vamp-catalog-*')))
def test_keyless_parts_preserve_their_original_key(self):
for part in ('Acoustic', 'Electric', 'Vocal'):
write_score(self.root, f'practice/0001/{part.lower()}.json', bundle(key='No key', part=part))
build_catalog(self.root)
generator.catalog.build_catalog(self.root)
self.assertTrue(all(row['key_signature'] == 'No key' for row in self.rows('practice_exercises')))
def test_invalid_layout_and_escaping_source_fail(self):
with self.assertRaisesRegex(ValueError, 'directory not found'):
build_catalog(self.root / 'missing')
generator.catalog.build_catalog(self.root / 'missing')
self.assertFalse((self.root / 'missing').exists())
score = write_score(self.root, 'songs/nested/tune.json')
with self.assertRaisesRegex(ValueError, 'unexpected songs'):
build_catalog(self.root)
generator.catalog.build_catalog(self.root)
score.unlink()
(self.root / 'songs/escape.json').symlink_to('/etc/passwd')
with self.assertRaises(ValueError):
build_catalog(self.root)
generator.catalog.build_catalog(self.root)
def test_runtime_asset_packaging_excludes_score_json(self):
import subprocess
write_score(self.root, 'songs/tune.json')
build_catalog(self.root)
generator.catalog.build_catalog(self.root)
(self.root / 'songs/tune.png').write_bytes(b'page')
with tempfile.TemporaryDirectory() as temporary:
destination = Path(temporary)
destination = pathlib.Path(temporary)
subprocess.run(['cmake', f'-DSOURCE={self.root}', f'-DDESTINATION={destination}', '-P', 'cmake/runtime_assets.cmake'], check=True, capture_output=True)
self.assertTrue((destination / 'catalog.db').exists())
for name in ('vamp.example.json', 'vamp.development.json'):
config = json.loads((Path('package/config') / name).read_text())
self.assertEqual(Path(config['database']['path']).name, 'catalog.db')
config = json.loads((pathlib.Path('package/config') / name).read_text())
self.assertEqual(pathlib.Path(config['database']['path']).name, 'catalog.db')
self.assertTrue((destination / 'songs/tune.png').exists())
self.assertFalse(list(destination.rglob('*.json')))
tests/test_catalog_runtime.cpp
+5 -5
View File
@@ -83,7 +83,7 @@ auto catalog_bundle(std::string_view title, std::string_view key = "C", std::str
auto value = boost::json::parse(R"json({
"music":{"metadata":{"title":"Fixture","composer":"Composer","key_signature":"C","time_signature":"4/4"},
"staves":[{"name":"Lead","measures":[{"voices":[]},{"voices":[]},{"voices":[]}]}]},
"presentation":{"width":976,"height":484,"pages":[
"presentation":{"width":976,"height":484,"title_image":"0000000000000000000000000000000000000000000000000000000000000000","pages":[
{"path":"fixture.pages/1.png","sha256":"0000000000000000000000000000000000000000000000000000000000000000"},
{"path":"fixture.pages/2.png","sha256":"1111111111111111111111111111111111111111111111111111111111111111"}]}
})json");
@@ -341,12 +341,12 @@ BOOST_AUTO_TEST_CASE(catalog_reload_uses_cached_metadata_without_parsing_score_j
BOOST_REQUIRE(db.build());
// Complete cached metadata must not require parsing or retaining the score body.
db.query("INSERT INTO songs(id, title, composer, source_json, part, source_path, "
"key_signature, measure_count, total_pages) "
"VALUES('tune', 'Metadata Song', 'Composer', '{}', 'Lead', 'songs/tune.json', 'F', 3, 2)");
"key_signature, measure_count, total_pages, title_image) "
"VALUES('tune', 'Metadata Song', 'Composer', '{}', 'Lead', 'songs/tune.json', 'F', 3, 2, '0000000000000000000000000000000000000000000000000000000000000000')");
for (auto part_name : vamp_zero::practice_part_names()) {
db.query("INSERT INTO practice_exercises(id, exercise_id, "
"title, composer, key_signature, source_json, source_path, measure_count, total_pages, part) "
"VALUES(?1, '0001', 'Metadata Exercise', 'Composer', 'Eb', '{}', ?3, 3, 2, ?2)",
"title, composer, key_signature, source_json, source_path, measure_count, total_pages, part, title_image) "
"VALUES(?1, '0001', 'Metadata Exercise', 'Composer', 'Eb', '{}', ?3, 3, 2, ?2, '0000000000000000000000000000000000000000000000000000000000000000')",
{noto::sqlite_value{std::string{"practice_part-"} + std::string{part_name}}, noto::sqlite_value{std::string{part_name}},
noto::sqlite_value{std::string{"practice/0001/"} + std::string{part_name} + ".json"}});
}
tests/test_engraving.py
+161 -94
View File
@@ -1,35 +1,106 @@
import collections
import copy
from dataclasses import replace
from fractions import Fraction
from pathlib import Path
import dataclasses
import fractions
import pathlib
import unittest
import unittest.mock
import xml.etree.ElementTree
from generator.engraving import EngravingError, Fonts, Style, layout_score, rasterize_page
from generator.engraving.audit import audit_layout
from generator.engraving.notation import ScoreIndex
from generator.model import NOTE_TYPES, TYPE_LENGTHS, fraction, rational
from generator.musicxml import load_musicxml
from generator.practice.generate import practice_document
from tests.test_musicxml import notes, parse, xml_note, xml_score
import PIL.Image
import PIL.ImageChops
import generator.engraving.audit
import generator.engraving.cache
import generator.engraving.fonts
import generator.engraving.layout
import generator.engraving.notation
import generator.engraving.raster
import generator.engraving.types
import generator.guitar
import generator.model
import generator.musicxml
import generator.playback
import generator.practice.generate
import tests.rhythm_fixtures
import tests.test_musicxml
class EngravingTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.fonts = Fonts(Path('package/assets/fonts/petaluma'))
cls.fonts = generator.engraving.fonts.Fonts(pathlib.Path('package/assets/fonts/petaluma'))
def layout(self, source, style=None):
before = copy.deepcopy(source)
layout = layout_score(source, style or Style(), self.fonts)
layout = generator.engraving.layout.layout_score(source, style or generator.engraving.types.Style(), self.fonts)
self.assertEqual(source, before)
return layout
def primitives(self, layout, role):
return [p for page in layout.pages for p in page.primitives if p.role == role]
def test_notation_fixture_visual_golden(self):
from PIL import Image, ImageChops
layout = self.layout(load_musicxml(Path('tests/fixtures/notation.xml')))
layout = self.layout(generator.musicxml.load_musicxml(pathlib.Path('tests/fixtures/notation.xml')))
self.assertEqual(len(layout.pages), 1)
actual = rasterize_page(layout.pages[0], self.fonts)
with Image.open('tests/fixtures/notation-python.png') as expected:
self.assertIsNone(ImageChops.difference(actual, expected.convert('RGB')).getbbox())
actual = generator.engraving.raster.rasterize_page(layout.pages[0], self.fonts)
with PIL.Image.open('tests/fixtures/notation-python.png') as expected:
self.assertIsNone(PIL.ImageChops.difference(actual, expected.convert('RGB')).getbbox())
def test_aligned_bands_match_full_pages_with_spans_tab_and_sampling_phases(self):
tied = tests.test_musicxml.xml_note('E', 2, 'half', extra='<tie type="start"/>') + tests.test_musicxml.xml_note('E', 2, 'half', extra='<tie type="stop"/>')
dense = ''.join(tests.test_musicxml.xml_note('G', 1, '64th') for _ in range(64))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(dense, divisions=16))
documents = [generator.musicxml.load_musicxml(pathlib.Path('tests/fixtures/notation.xml')),
tests.test_musicxml.parse(tests.test_musicxml.xml_score(tied)),
generator.practice.generate.practice_document(source, 'electric', rhythms=tests.rhythm_fixtures.library())]
checked = 0
for sampling in (1, 2, 3, 4):
for document in documents:
style = generator.engraving.types.Style(supersampling=sampling, margin_y=18.25)
for page in self.layout(document, style).pages:
with self.subTest(sampling=sampling, title=document['metadata']['title'], page=page.number):
bands = generator.engraving.raster.page_bands(page)
self.assertIsNotNone(bands)
with generator.engraving.raster.rasterize_band(page, bands[1].primitives, bands[1].top, bands[1].bottom, self.fonts) as body:
with generator.engraving.raster.compose_page(page, self.fonts, bands, body) as actual, generator.engraving.raster.rasterize_page(page, self.fonts) as reference:
self.assertIsNone(PIL.ImageChops.difference(actual, reference).getbbox())
checked += 1
self.assertGreater(checked, 12)
def test_band_key_tracks_positions_dimensions_and_complete_context(self):
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
systems = generator.engraving.layout.layout_systems(source, generator.engraving.types.Style(), self.fonts)
page = generator.engraving.layout.paginate(systems, source, self.fonts, generator.engraving.types.Style())[0]
key = generator.engraving.cache.band_fingerprint('body', page, generator.engraving.raster.page_bands(page)[1])
source['metadata']['title'] = 'g' * 100
changed = generator.engraving.layout.paginate(systems, source, self.fonts, generator.engraving.types.Style())[0]
self.assertNotEqual(page.systems[0].box, changed.systems[0].box)
self.assertNotEqual(key, generator.engraving.cache.band_fingerprint('body', changed, generator.engraving.raster.page_bands(changed)[1]))
self.assertNotEqual(key, generator.engraving.cache.band_fingerprint('another body', page, generator.engraving.raster.page_bands(page)[1]))
self.assertNotEqual(key, generator.engraving.cache.band_fingerprint('body', dataclasses.replace(page, supersampling=2), generator.engraving.raster.page_bands(page)[1]))
def test_nonseparable_page_falls_back_to_full_page(self):
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
page = self.layout(source).pages[0]
decoration = self.fonts.text('Overlay', 50, page.systems[0].box.top + 4, 12, 'subtitle', ())
primitive = self.fonts.text('Extra', 50, 400, 12, 'other', ())
for altered in (dataclasses.replace(page, primitives=page.primitives + (primitive,)),
dataclasses.replace(page, primitives=page.primitives[:1] + (decoration,) + page.primitives[1:])):
self.assertIsNone(generator.engraving.raster.page_bands(altered))
def test_musical_style_does_not_become_a_subtitle(self):
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
before = self.layout(source)
source['metadata']['style'] = 'Unrelated musical metadata'
self.assertEqual(before, self.layout(source))
self.assertNotEqual(before, self.layout(source, generator.engraving.types.Style(column_gap=16)))
def test_candidate_columns_are_reused_without_changing_clipped_sustains(self):
content = tests.test_musicxml.xml_note(extra='<voice>held</voice>') + '<backup><duration>4</duration></backup>'
content += ''.join(tests.test_musicxml.xml_note('G', 1, 'quarter', extra='<voice>moving</voice>') for _ in range(4))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))
index = generator.engraving.notation.ScoreIndex(source)
columns = collections.OrderedDict()
style = generator.engraving.types.Style()
with unittest.mock.patch('generator.engraving.layout.column', wraps=generator.engraving.layout.column) as draw:
for end in (2, 3, 4, 4):
actual = generator.engraving.layout.make_system(index, fractions.Fraction(), fractions.Fraction(end), self.fonts, style, columns=columns)
with unittest.mock.patch('generator.engraving.layout.column', wraps=draw._mock_wraps):
expected = generator.engraving.layout.make_system(index, fractions.Fraction(), fractions.Fraction(end), self.fonts, style)
self.assertEqual(actual, expected)
self.assertLess(draw.call_count, 2 + 3 + 4 + 4)
def test_accidental_work_is_bounded_by_local_event_count(self):
class Unscannable(list):
def __iter__(self):
@@ -41,10 +112,10 @@ class EngravingTests(unittest.TestCase):
Counted.visited += 1
yield value
for count in (100, 200, 400):
content = ''.join(xml_note(duration=1, kind='quarter', alter=i % 2) for i in range(4))
content = ''.join(tests.test_musicxml.xml_note(duration=1, kind='quarter', alter=i % 2) for i in range(4))
extra = ''.join(f'<measure number="{i + 1}">{content}</measure>' for i in range(1, count))
document = parse(xml_score(content).replace('</part></score-partwise>', extra + '</part></score-partwise>'))
index = ScoreIndex(document)
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content).replace('</part></score-partwise>', extra + '</part></score-partwise>'))
index = generator.engraving.notation.ScoreIndex(document)
expected = index.accidentals
index.events = Unscannable(index.events)
index.events_by_onset = {time: Counted(events) for time, events in index.events_by_onset.items()}
@@ -52,66 +123,65 @@ class EngravingTests(unittest.TestCase):
self.assertEqual(index.accidental_state(), expected)
self.assertEqual(Counted.visited, count * 4)
def test_indexed_columns_preserve_sustains_and_timed_annotations(self):
from generator.engraving.layout import column
class Unscannable(list):
def __iter__(self):
raise AssertionError('whole-score scan from column layout')
content = xml_note(extra='<voice>held</voice>') + '<backup><duration>4</duration></backup>'
content += ''.join(xml_note('G', 1, 'quarter', extra='<voice>moving</voice>') for _ in range(4))
content = tests.test_musicxml.xml_note(extra='<voice>held</voice>') + '<backup><duration>4</duration></backup>'
content += ''.join(tests.test_musicxml.xml_note('G', 1, 'quarter', extra='<voice>moving</voice>') for _ in range(4))
content += '<harmony><root><root-step>F</root-step></root><kind>major</kind><offset>-2</offset></harmony>'
content += '<direction><direction-type><metronome><beat-unit>quarter</beat-unit><per-minute>90</per-minute></metronome></direction-type><offset>-2</offset></direction>'
index = ScoreIndex(parse(xml_score(content)))
expected = column(index, Fraction(2), Fraction(2), Fraction(4), self.fonts, Style())
index = generator.engraving.notation.ScoreIndex(tests.test_musicxml.parse(tests.test_musicxml.xml_score(content)))
expected = generator.engraving.layout.column(index, fractions.Fraction(2), fractions.Fraction(2), fractions.Fraction(4), self.fonts, generator.engraving.types.Style())
index.events = Unscannable(index.events)
index.measures = Unscannable(index.measures)
actual = column(index, Fraction(2), Fraction(2), Fraction(4), self.fonts, Style())
actual = generator.engraving.layout.column(index, fractions.Fraction(2), fractions.Fraction(2), fractions.Fraction(4), self.fonts, generator.engraving.types.Style())
self.assertEqual(actual, expected)
self.assertEqual(len(index.events_at(Fraction(2), include_sustained=True)), 2)
self.assertEqual(len(index.events_at(fractions.Fraction(2), include_sustained=True)), 2)
self.assertTrue(any(p.role == 'harmony' for p in actual[0]))
self.assertEqual(len(index.events_at(Fraction(4), include_sustained=True)), 0)
self.assertEqual(len(index.events_at(fractions.Fraction(4), include_sustained=True)), 0)
def test_every_written_note_and_rest_value(self):
for kind in NOTE_TYPES:
for kind in generator.model.NOTE_TYPES:
for dots in (0, 1, 3):
duration = TYPE_LENGTHS[kind] * (2 - Fraction(1, 2 ** dots))
duration = generator.model.TYPE_LENGTHS[kind] * (2 - fractions.Fraction(1, 2 ** dots))
for rest in (False, True):
with self.subTest(kind=kind, dots=dots, rest=rest):
content = xml_note(duration=duration.numerator, kind=kind, extra='<dot/>' * dots)
content = tests.test_musicxml.xml_note(duration=duration.numerator, kind=kind, extra='<dot/>' * dots)
if rest:
content = f'<note><rest/><duration>{duration.numerator}</duration><type>{kind}</type>' + '<dot/>' * dots + '</note>'
source = parse(xml_score(content, divisions=duration.denominator, implicit=True))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content, divisions=duration.denominator, implicit=True))
layout = self.layout(source)
self.assertEqual(len(self.primitives(layout, 'dot')), dots)
self.assertEqual(len(self.primitives(layout, 'rest' if rest else 'head')), 1)
def test_sixteen_dots_are_not_truncated(self):
duration = TYPE_LENGTHS['quarter'] * (2 - Fraction(1, 2 ** 16))
source = parse(xml_score(xml_note(duration=duration.numerator, kind='quarter', extra='<dot/>' * 16), divisions=duration.denominator, implicit=True))
duration = generator.model.TYPE_LENGTHS['quarter'] * (2 - fractions.Fraction(1, 2 ** 16))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=duration.numerator, kind='quarter', extra='<dot/>' * 16), divisions=duration.denominator, implicit=True))
self.assertEqual(len(self.primitives(self.layout(source), 'dot')), 16)
def test_arbitrary_voice_ids_and_more_than_four_voices(self):
content = ''
for i in range(6):
if i:
content += '<backup><duration>4</duration></backup>'
content += xml_note(extra=f'<voice>part-{i}</voice>')
source = parse(xml_score(content))
content += tests.test_musicxml.xml_note(extra=f'<voice>part-{i}</voice>')
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))
self.assertEqual(len(self.primitives(self.layout(source), 'head')), 6)
def test_microtonal_numeric_fallback_and_explicit_missing_glyph(self):
source = parse(xml_score(xml_note(alter='0.1')))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(alter='0.1')))
self.assertEqual(self.primitives(self.layout(source), 'accidental')[0].value, '(+1/10 st)')
notes(source)[0]['pitches'][0]['accidental_display'] = {'name': 'other', 'smufl': 'missingVampGlyph'}
with self.assertRaisesRegex(EngravingError, 'measure 1, voice 1.*missingVampGlyph'):
tests.test_musicxml.notes(source)[0]['pitches'][0]['accidental_display'] = {'name': 'other', 'smufl': 'missingVampGlyph'}
with self.assertRaisesRegex(generator.engraving.types.EngravingError, 'measure 1, voice 1.*missingVampGlyph'):
self.layout(source)
def test_accidentals_are_shared_by_simultaneous_voices_without_losing_conflicts(self):
content = xml_note(duration=1, kind='quarter', alter=1, extra='<voice>a</voice>')
content += xml_note(duration=1, kind='quarter', alter=1, extra='<voice>a</voice>')
content += xml_note(duration=2, kind='half', extra='<voice>a</voice>')
source = parse(xml_score(content))
content = tests.test_musicxml.xml_note(duration=1, kind='quarter', alter=1, extra='<voice>a</voice>')
content += tests.test_musicxml.xml_note(duration=1, kind='quarter', alter=1, extra='<voice>a</voice>')
content += tests.test_musicxml.xml_note(duration=2, kind='half', extra='<voice>a</voice>')
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))
self.assertEqual([p.value for p in self.primitives(self.layout(source), 'accidental')], ['accidentalSharp', 'accidentalNatural'])
conflict = parse(xml_score(xml_note(alter=1, extra='<voice>a</voice>') + '<backup><duration>4</duration></backup>' + xml_note(extra='<voice>b</voice>')))
conflict = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(alter=1, extra='<voice>a</voice>') + '<backup><duration>4</duration></backup>' + tests.test_musicxml.xml_note(extra='<voice>b</voice>')))
self.assertEqual(len(self.primitives(self.layout(conflict), 'accidental')), 2)
def test_dense_measure_splits_exactly_without_modifying_music(self):
content = ''.join(xml_note('G', 1, '64th', extra='<voice>fast</voice>') for _ in range(64))
content += '<backup><duration>64</duration></backup>' + xml_note('C', 64, 'whole', extra='<voice>held</voice>')
source = parse(xml_score(content, divisions=16))
content = ''.join(tests.test_musicxml.xml_note('G', 1, '64th', extra='<voice>fast</voice>') for _ in range(64))
content += '<backup><duration>64</duration></backup>' + tests.test_musicxml.xml_note('C', 64, 'whole', extra='<voice>held</voice>')
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content, divisions=16))
layout = self.layout(source)
systems = [s for p in layout.pages for s in p.systems]
self.assertGreater(len(systems), 1)
@@ -121,40 +191,40 @@ class EngravingTests(unittest.TestCase):
self.assertGreater(len(pieces), 1)
self.assertTrue(self.primitives(layout, 'tie'))
for page in layout.pages:
self.assertEqual(rasterize_page(page, self.fonts).size, (976, 484))
self.assertEqual(generator.engraving.raster.rasterize_page(page, self.fonts).size, (976, 484))
def test_hidden_pitch_does_not_suppress_a_visible_accidental(self):
source = parse(xml_score(xml_note(duration=2, kind='half', alter=1) + xml_note(duration=2, kind='half', alter=1)))
notes(source)[0]['hidden'] = True
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=2, kind='half', alter=1) + tests.test_musicxml.xml_note(duration=2, kind='half', alter=1)))
tests.test_musicxml.notes(source)[0]['hidden'] = True
self.assertEqual([p.value for p in self.primitives(self.layout(source), 'accidental')], ['accidentalSharp'])
def test_octave_shift_changes_display_not_pitch(self):
source = parse(xml_score(xml_note(octave=5)))
original = ScoreIndex(source).pitch_position(notes(source)[0]['pitches'][0], Fraction())
source['staves'][0]['octave_shifts'] = [{'id': 'octave', 'start': rational(0), 'end': rational(4), 'octaves': -1}]
index = ScoreIndex(source)
self.assertEqual(index.pitch_position(notes(source)[0]['pitches'][0], Fraction()), original - 7)
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(octave=5)))
original = generator.engraving.notation.ScoreIndex(source).pitch_position(tests.test_musicxml.notes(source)[0]['pitches'][0], fractions.Fraction())
source['staves'][0]['octave_shifts'] = [{'id': 'octave', 'start': generator.model.rational(0), 'end': generator.model.rational(4), 'octaves': -1}]
index = generator.engraving.notation.ScoreIndex(source)
self.assertEqual(index.pitch_position(tests.test_musicxml.notes(source)[0]['pitches'][0], fractions.Fraction()), original - 7)
self.assertTrue(any(p.value == '8va' for p in self.primitives(self.layout(source), 'octave-shift')))
def test_wide_pitch_range_can_break_inside_a_measure_vertically(self):
source = parse(xml_score(xml_note(duration=2, kind='half', octave=10) + xml_note(duration=2, kind='half', octave=-1)))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=2, kind='half', octave=10) + tests.test_musicxml.xml_note(duration=2, kind='half', octave=-1)))
layout = self.layout(source)
self.assertEqual(len(layout.pages), 2)
self.assertEqual(len(self.primitives(layout, 'head')), 2)
def test_cue_grace_and_tremolo_are_drawn(self):
grace = '<note><grace slash="yes"/><pitch><step>D</step><octave>4</octave></pitch><type>16th</type></note>'
source = parse(xml_score(grace + xml_note(duration=4, kind='whole', prefix='<cue/>')))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(grace + tests.test_musicxml.xml_note(duration=4, kind='whole', prefix='<cue/>')))
layout = self.layout(source)
self.assertEqual(len(self.primitives(layout, 'head')), 2)
self.assertEqual(len(self.primitives(layout, 'grace-slash')), 1)
source = parse(xml_score(xml_note(duration=2, kind='half', extra='<notations><ornaments><tremolo type="single">8</tremolo></ornaments></notations>'), implicit=True))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=2, kind='half', extra='<notations><ornaments><tremolo type="single">8</tremolo></ornaments></notations>'), implicit=True))
self.assertEqual(len(self.primitives(self.layout(source), 'tremolo')), 8)
def test_percussion_and_tab_use_explicit_display_positions(self):
percussion = '<note><unpitched><display-step>C</display-step><display-octave>5</display-octave></unpitched><duration>4</duration><type>whole</type><notehead>x</notehead></note>'
source = parse(xml_score(percussion, attributes='<clef><sign>percussion</sign></clef>'))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(percussion, attributes='<clef><sign>percussion</sign></clef>'))
self.assertEqual(self.primitives(self.layout(source), 'head')[0].value, 'noteheadXWhole')
tab = practice_document(parse(xml_score(xml_note('C', octave=7))), 'electric')
tab = generator.practice.generate.practice_document(tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note('C', octave=7))), 'electric', rhythms=tests.rhythm_fixtures.library())
self.assertEqual(self.primitives(self.layout(tab), 'tab')[0].value, '20')
def test_generated_tab_shares_columns_and_systems_without_duplicating_music(self):
content = ''.join(xml_note('G', 1, '64th') for _ in range(64))
source = practice_document(parse(xml_score(content, divisions=16)), 'electric')
content = ''.join(tests.test_musicxml.xml_note('G', 1, '64th') for _ in range(64))
source = generator.practice.generate.practice_document(tests.test_musicxml.parse(tests.test_musicxml.xml_score(content, divisions=16)), 'electric', rhythms=tests.rhythm_fixtures.library())
layout = self.layout(source)
systems = [s for page in layout.pages for s in page.systems]
self.assertGreater(len(systems), 1)
@@ -167,11 +237,10 @@ class EngravingTests(unittest.TestCase):
self.assertLess(max(p.box.bottom for p in system.primitives if not p.role.startswith('tab')), min(p.box.top for p in system.primitives if p.role.startswith('tab')))
self.assertEqual({p.sources[-1] for p in heads}, {p.sources[-1] for p in tabs})
self.assertTrue(all(any(abs(head.points[0][0] - tab.points[0][0]) < 0.01 for head in heads if head.sources[-1] == tab.sources[-1]) for tab in tabs))
from generator.playback import compile_timeline
self.assertEqual(len(compile_timeline(source)['events']), 64)
self.assertEqual(len(generator.playback.compile_timeline(source)['events']), 64)
def test_generated_tab_ties_and_barline_styles(self):
content = xml_note('E', 2, 'half', extra='<tie type="start"/>') + xml_note('E', 2, 'half', extra='<tie type="stop"/>') + '<barline location="right"><bar-style>dashed</bar-style></barline>'
source = practice_document(parse(xml_score(content)), 'electric')
content = tests.test_musicxml.xml_note('E', 2, 'half', extra='<tie type="start"/>') + tests.test_musicxml.xml_note('E', 2, 'half', extra='<tie type="stop"/>') + '<barline location="right"><bar-style>dashed</bar-style></barline>'
source = generator.practice.generate.practice_document(tests.test_musicxml.parse(tests.test_musicxml.xml_score(content)), 'electric', rhythms=tests.rhythm_fixtures.library())
layout = self.layout(source)
self.assertEqual(len(self.primitives(layout, 'tab-tie')), 1)
self.assertEqual(len(self.primitives(layout, 'barline')), len(self.primitives(layout, 'tab-barline')))
@@ -184,10 +253,9 @@ class EngravingTests(unittest.TestCase):
self.assertAlmostEqual(x1, x2)
self.assertAlmostEqual((y1 - baselines['staff']) * 5 / 4, y2 - baselines['tab-staff'])
def test_adjacent_string_tab_numerals_do_not_overlap(self):
from generator.guitar import place_guitar
content = ''.join(xml_note(step, octave=octave, prefix='<chord/>' if index else '') for index, (step, octave) in enumerate((('G', 3), ('C', 4), ('E', 4), ('G', 4), ('C', 5), ('E', 5))))
source = parse(xml_score(content))
place_guitar(source)
content = ''.join(tests.test_musicxml.xml_note(step, octave=octave, prefix='<chord/>' if index else '') for index, (step, octave) in enumerate((('G', 3), ('C', 4), ('E', 4), ('G', 4), ('C', 5), ('E', 5))))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))
generator.guitar.place_guitar(source)
layout = self.layout(source)
tabs = self.primitives(layout, 'tab')
self.assertEqual(len(tabs), 6)
@@ -197,63 +265,62 @@ class EngravingTests(unittest.TestCase):
def test_lyrics_hyphens_and_extenders(self):
begin = '<lyric number="1"><syllabic>begin</syllabic><text>hel</text></lyric>'
end = '<lyric number="1"><syllabic>end</syllabic><text>lo</text><extend type="start"/></lyric>'
source = parse(xml_score(xml_note(duration=1, kind='quarter', extra=begin) + xml_note(duration=1, kind='quarter', extra=end) + xml_note(duration=2, kind='half')))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=1, kind='quarter', extra=begin) + tests.test_musicxml.xml_note(duration=1, kind='quarter', extra=end) + tests.test_musicxml.xml_note(duration=2, kind='half')))
layout = self.layout(source)
self.assertTrue(self.primitives(layout, 'lyric-hyphen'))
self.assertTrue(self.primitives(layout, 'lyric-extension'))
def test_nested_and_cross_bar_tuplets_keep_span_ownership(self):
from xml.etree import ElementTree
modulation = '<time-modification><actual-notes>3</actual-notes><normal-notes>2</normal-notes></time-modification>'
content = ''.join(xml_note(duration=2, kind='quarter', extra=modulation) for _ in range(6))
tree = ElementTree.fromstring(xml_score(content, divisions=3))
second = ElementTree.SubElement(tree.find('part'), 'measure', number='2')
for element in ElementTree.fromstring(xml_score(content, divisions=3)).findall('part/measure/note'):
content = ''.join(tests.test_musicxml.xml_note(duration=2, kind='quarter', extra=modulation) for _ in range(6))
tree = xml.etree.ElementTree.fromstring(tests.test_musicxml.xml_score(content, divisions=3))
second = xml.etree.ElementTree.SubElement(tree.find('part'), 'measure', number='2')
for element in xml.etree.ElementTree.fromstring(tests.test_musicxml.xml_score(content, divisions=3)).findall('part/measure/note'):
second.append(element)
source = parse(ElementTree.tostring(tree, encoding='unicode'))
events = notes(source) + notes(source, 1)
source = tests.test_musicxml.parse(xml.etree.ElementTree.tostring(tree, encoding='unicode'))
events = tests.test_musicxml.notes(source) + tests.test_musicxml.notes(source, 1)
spans = [{'id': 'outer', 'voice': '1', 'start': events[0]['id'], 'end': events[-1]['id'], 'note_count': 3, 'space_count': 2, 'depth': 0, 'bracket': 'yes', 'show_number': 'both', 'show_type': 'both', 'line_shape': 'curved'}, {'id': 'inner', 'voice': '1', 'start': events[1]['id'], 'end': events[3]['id'], 'note_count': 3, 'space_count': 2, 'depth': 1, 'bracket': 'yes', 'show_number': 'actual'}]
source['staves'][0]['tuplets'] = spans
for i, event in enumerate(events):
event['tuplet_ids'] = ['outer'] + (['inner'] if 1 <= i <= 3 else [])
layout = self.layout(source, Style(width=600))
layout = self.layout(source, generator.engraving.types.Style(width=600))
self.assertGreater(len([s for p in layout.pages for s in p.systems]), 1)
self.assertEqual({p.sources[0] for p in self.primitives(layout, 'tuplet-bracket')}, {'outer', 'inner'})
def test_paired_tremolo_and_double_stems(self):
modulation = '<time-modification><actual-notes>2</actual-notes><normal-notes>1</normal-notes></time-modification>'
content = ''.join(xml_note(step, 2, 'whole', extra=modulation + f'<notations><ornaments><tremolo type="{action}">3</tremolo></ornaments></notations>') for step, action in [('C', 'start'), ('G', 'stop')])
self.assertEqual(len(self.primitives(self.layout(parse(xml_score(content))), 'tremolo')), 3)
source = parse(xml_score(xml_note(duration=1, kind='eighth', extra='<stem>double</stem>'), divisions=2, implicit=True))
content = ''.join(tests.test_musicxml.xml_note(step, 2, 'whole', extra=modulation + f'<notations><ornaments><tremolo type="{action}">3</tremolo></ornaments></notations>') for step, action in [('C', 'start'), ('G', 'stop')])
self.assertEqual(len(self.primitives(self.layout(tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))), 'tremolo')), 3)
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=1, kind='eighth', extra='<stem>double</stem>'), divisions=2, implicit=True))
layout = self.layout(source)
self.assertEqual(len(self.primitives(layout, 'stem')), 2)
self.assertEqual(len(self.primitives(layout, 'flag')), 2)
def test_meter_symbols_and_timed_attributes(self):
for symbol in ('common', 'cut', 'single-number', 'note', 'dotted-note', 'normal'):
source = parse(xml_score(xml_note()).replace('<time>', f'<time symbol="{symbol}">'))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()).replace('<time>', f'<time symbol="{symbol}">'))
with self.subTest(symbol=symbol):
self.assertTrue(self.primitives(self.layout(source), 'meter'))
change = '<attributes><key><fifths>-3</fifths></key><clef><sign>C</sign><line>3</line></clef></attributes>'
source = parse(xml_score(xml_note(duration=2, kind='half') + change + xml_note(duration=2, kind='half')))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=2, kind='half') + change + tests.test_musicxml.xml_note(duration=2, kind='half')))
layout = self.layout(source)
self.assertEqual([p.value for p in self.primitives(layout, 'clef')], ['gClef', 'cClef'])
def test_lyric_extender_continues_on_systems_without_syllables(self):
lyrics = '<lyric number="1"><text>Ah</text><extend type="start"/></lyric>'
content = ''.join(xml_note('G', 1, '64th', extra=lyrics if i == 0 else '') for i in range(64))
source = parse(xml_score(content, divisions=16))
content = ''.join(tests.test_musicxml.xml_note('G', 1, '64th', extra=lyrics if i == 0 else '') for i in range(64))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content, divisions=16))
layout = self.layout(source)
systems = [s for p in layout.pages for s in p.systems]
self.assertGreater(len(systems), 1)
self.assertTrue(all(any(p.role == 'lyric-extension' for p in s.primitives) for s in systems))
def test_audit_catches_removed_pitch_and_accidental_primitives(self):
source = parse(xml_score(xml_note(alter=1)))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(alter=1)))
layout = self.layout(source)
for role in ('head', 'accidental'):
pages = tuple(replace(p, primitives=tuple(g for g in p.primitives if g.role != role)) for p in layout.pages)
with self.assertRaisesRegex(EngravingError, 'no drawing primitive|accidental was omitted'):
audit_layout(source, replace(layout, pages=pages), self.fonts)
pages = tuple(dataclasses.replace(p, primitives=tuple(g for g in p.primitives if g.role != role)) for p in layout.pages)
with self.assertRaisesRegex(generator.engraving.types.EngravingError, 'no drawing primitive|accidental was omitted'):
generator.engraving.audit.audit_layout(source, dataclasses.replace(layout, pages=pages), self.fonts)
def test_fingerprint_includes_style_and_fixed_overflow_errors(self):
source = parse(xml_score(xml_note()))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
a = self.layout(source)
b = self.layout(source, Style(column_gap=14))
b = self.layout(source, generator.engraving.types.Style(column_gap=14))
self.assertNotEqual(a.fingerprint, b.fingerprint)
with self.assertRaisesRegex(EngravingError, 'page width'):
self.layout(source, Style(width=100))
with self.assertRaisesRegex(generator.engraving.types.EngravingError, 'page width'):
self.layout(source, generator.engraving.types.Style(width=100))
tests/test_engraving_cache.py
+231 -72
View File
@@ -4,21 +4,32 @@ import copy
import hashlib
import io
import json
from pathlib import Path
import pathlib
import shutil
import tempfile
import unittest
from unittest.mock import patch
import unittest.mock
from generator.__main__ import generate_assets
from generator.engraving import EngravingError, Fonts, Style, engrave_tree, layout_score
from generator.engraving.cache import renderer_fingerprint, renderer_namespace, restore_pages, score_fingerprint
from generator.score import validate_generated_tree
from tests.test_musicxml import parse, xml_note, xml_score
import PIL.Image
import PIL.ImageChops
import generator.__main__
import generator.engraving.cache
import generator.engraving.fonts
import generator.engraving.layout
import generator.engraving.pipeline
import generator.engraving.raster
import generator.engraving.types
import generator.engraving.validation
import generator.pico_artwork
import generator.score
import tests.rhythm_fixtures
import tests.test_musicxml
class EngravingCacheTests(unittest.TestCase):
def test_ci_snapshot_tracks_all_generator_inputs_and_keeps_fallback(self):
workflow = Path('.gitea/workflows/ci.yml').read_text()
self.assertIn("hashFiles('generator/**/*.py', 'generator/*.py', 'data/practice/*.xml', 'data/songs/*.xml')", workflow)
workflow = pathlib.Path('.gitea/workflows/ci.yml').read_text()
self.assertIn("hashFiles('generator/**/*.py', 'generator/*.py', 'data/practice/*.xml', 'data/songs/*.xml', 'data/rhythms/**/*.xml')", workflow)
self.assertIn('python3 -m generator --renderer-fingerprint', workflow)
self.assertIn('restore-keys: product-vamp-pages-${{ steps.renderer.outputs.fingerprint }}-', workflow)
self.assertIn('path: build/build_common/engraving-cache/${{ steps.renderer.outputs.fingerprint }}', workflow)
@@ -27,10 +38,10 @@ class EngravingCacheTests(unittest.TestCase):
def setUp(self):
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name)
self.root = pathlib.Path(self.temporary.name)
self.cache = self.root / 'cache'
self.fonts = Path('package/assets/fonts/petaluma')
self.document = parse(xml_score(xml_note()))
self.fonts = pathlib.Path('package/assets/fonts/petaluma')
self.document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
def write_scores(self, root, documents):
root.mkdir(parents=True, exist_ok=True)
for index, document in enumerate(documents):
@@ -40,16 +51,150 @@ class EngravingCacheTests(unittest.TestCase):
def seed_cache(self):
root = self.root / 'seed'
self.write_scores(root, [self.document])
engrave_tree(root, self.fonts, cache_dir=self.cache)
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache)
return next(self.cache.rglob('pages.json')).parent
def titled(self, title):
document = copy.deepcopy(self.document)
document['metadata']['title'] = title
return document
def body_calls(self, render):
return [call for call in render.call_args_list if any(p.role == 'staff' for p in call.args[1])]
def test_cold_titles_group_before_dispatch_and_render_body_once(self):
root = self.root / 'cold'
self.write_scores(root, [self.titled(f'Fixture {index}') for index in range(8)])
progress = io.StringIO()
with unittest.mock.patch('generator.engraving.layout.layout_systems', wraps=generator.engraving.layout.layout_systems) as layout, unittest.mock.patch('generator.engraving.raster.rasterize_band', wraps=generator.engraving.raster.rasterize_band) as render, unittest.mock.patch('generator.engraving.pipeline.concurrent.futures.ProcessPoolExecutor', side_effect=AssertionError('one body must have one owner')), contextlib.redirect_stdout(progress):
generator.engraving.pipeline.engrave_tree(root, self.fonts, jobs=4, progress=True)
layout.assert_called_once()
self.assertEqual(len(self.body_calls(render)), 1)
self.assertIn('fullscore 0, bodyhits 7, bodies 1, renderedpages 8', progress.getvalue())
self.assertIn('Engraving timings:', progress.getvalue())
for phase in ('prepare', 'layout', 'pagination', 'raster', 'cache', 'validation', 'wall'):
self.assertIn(phase + ' ', progress.getvalue())
def test_title_change_reuses_persistent_notation_and_matches_full_page(self):
self.seed_cache()
document = self.titled('Fixture A')
root = self.root / 'title'
self.write_scores(root, [document])
with unittest.mock.patch('generator.engraving.raster.rasterize_band', wraps=generator.engraving.raster.rasterize_band) as render:
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache)
self.assertFalse(self.body_calls(render))
self.assertEqual(len(list(self.cache.rglob('band.json'))), 1)
family = generator.engraving.fonts.Fonts(self.fonts)
layout = generator.engraving.layout.layout_score(document, generator.engraving.types.Style(), family)
with PIL.Image.open(root / '000.pages/0001.png') as actual:
reference = generator.engraving.raster.rasterize_page(layout.pages[0], family)
self.assertIsNone(PIL.ImageChops.difference(actual, reference).getbbox())
def test_duplicate_fullscores_share_rendering_without_persistent_cache(self):
root = self.root / 'duplicate'
self.write_scores(root, [self.document] * 8)
with unittest.mock.patch('generator.engraving.raster.compose_page', wraps=generator.engraving.raster.compose_page) as render, unittest.mock.patch('generator.engraving.validation.check_content', wraps=generator.engraving.validation.check_content) as decode:
trusted = generator.engraving.pipeline.engrave_tree(root, self.fonts, jobs=4)
render.assert_called_once()
decode.assert_called_once()
self.assertEqual(len(trusted), 1)
def test_full_page_validation_is_once_per_content_but_every_file_is_read(self):
self.seed_cache()
root = self.root / 'validation'
self.write_scores(root, [self.document] * 4)
with unittest.mock.patch('generator.engraving.validation.check_content', wraps=generator.engraving.validation.check_content) as decode:
trusted = generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache)
decode.assert_called_once()
read_bytes = pathlib.Path.read_bytes
reads = []
def read(path):
reads.append(path)
return read_bytes(path)
pages = sorted(root.glob('*.pages/*.png'))
digest = next(iter(trusted))
with unittest.mock.patch.object(pathlib.Path, 'read_bytes', read):
for path in pages:
generator.engraving.validation.check_page(path, trusted, expected_sha256=digest)
self.assertCountEqual(reads, pages)
decode.assert_called_once()
pages[-1].write_bytes(pages[-1].read_bytes() + b'changed')
with self.assertRaisesRegex(ValueError, 'checksum'):
generator.engraving.validation.check_page(pages[-1], trusted, expected_sha256=digest)
pages[-1].unlink()
pages[-1].symlink_to(pages[0])
with self.assertRaisesRegex(ValueError, 'regular PNG'):
generator.engraving.validation.check_page(pages[-1], trusted, expected_sha256=digest)
def test_font_content_change_creates_new_shared_notation_namespace(self):
self.seed_cache()
fonts = self.root / 'fonts'
shutil.copytree(self.fonts, fonts)
metadata = fonts / 'petaluma_metadata.json'
metadata.write_bytes(metadata.read_bytes() + b'\n')
root = self.root / 'font-change'
self.write_scores(root, [self.document])
with unittest.mock.patch('generator.engraving.raster.rasterize_band', wraps=generator.engraving.raster.rasterize_band) as render:
generator.engraving.pipeline.engrave_tree(root, fonts, cache_dir=self.cache)
self.assertEqual(len(self.body_calls(render)), 1)
self.assertEqual(len(list(self.cache.iterdir())), 2)
self.assertEqual(len(list(self.cache.rglob('band.json'))), 2)
def test_corrupt_shared_bands_are_rejected_and_repaired(self):
self.seed_cache()
manifest = next(self.cache.rglob('band.json'))
asset = manifest.with_name('band.png')
original, metadata = asset.read_bytes(), manifest.read_bytes()
for index, corruption in enumerate(('checksum', 'undecodable', 'blank', 'wrong_size', 'manifest', 'context', 'symlink', 'directory')):
with self.subTest(corruption=corruption):
if corruption in ('checksum', 'undecodable'):
asset.write_bytes(original + b'changed' if corruption == 'checksum' else original[:24])
elif corruption in ('blank', 'wrong_size'):
dimensions = json.loads(metadata)
size = (dimensions['width'], dimensions['height']) if corruption == 'blank' else (10, 10)
PIL.Image.new('RGB', size, 'white').save(asset)
elif corruption == 'manifest':
manifest.write_text('invalid json')
elif corruption == 'context':
manifest.write_text(json.dumps({**json.loads(metadata), 'context': 'different'}))
elif corruption == 'symlink':
external = self.root / 'external.png'
external.write_bytes(original)
asset.unlink()
asset.symlink_to(external)
else:
asset.unlink()
asset.mkdir()
if corruption in ('undecodable', 'blank', 'wrong_size'):
manifest.write_text(json.dumps({**json.loads(metadata), 'sha256': hashlib.sha256(asset.read_bytes()).hexdigest()}))
root = self.root / f'corrupt-band-{index}'
self.write_scores(root, [self.titled(f'Fixture A {index}')])
with unittest.mock.patch('generator.engraving.raster.rasterize_band', wraps=generator.engraving.raster.rasterize_band) as render:
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache)
self.assertEqual(len(self.body_calls(render)), 1)
self.assertEqual(asset.read_bytes(), original)
self.assertEqual(json.loads(manifest.read_bytes()), json.loads(metadata))
def test_band_eviction_reloads_final_pixels_without_rerendering(self):
family = generator.engraving.fonts.Fonts(self.fonts)
style = generator.engraving.types.Style()
documents = [self.document, tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note('D')))]
pages = [generator.engraving.layout.layout_score(document, style, family).pages[0] for document in documents]
result = generator.engraving.pipeline.GroupResult()
bands = generator.engraving.pipeline.BodyBands(self.root / 'spool', None, result, limit=1)
self.addCleanup(bands.close)
original = None
for index in (0, 1, 0):
page = pages[index]
notation = generator.engraving.cache.notation_fingerprint(documents[index], 'renderer')
image = bands.get(notation, page, generator.engraving.raster.page_bands(page)[1], family)
if index == 0:
if original is None:
original = image.tobytes()
else:
self.assertEqual(image.tobytes(), original)
self.assertEqual(len(bands.images), 1)
self.assertEqual(result.bodies, 2)
self.assertEqual(result.bodyhits, 1)
def test_cache_hit_preserves_bundle_bytes_across_output_directories(self):
cold, warm = self.root / 'cold', self.root / 'warm'
self.write_scores(cold, [self.document])
engrave_tree(cold, self.fonts, cache_dir=self.cache)
generator.engraving.pipeline.engrave_tree(cold, self.fonts, cache_dir=self.cache)
self.write_scores(warm, [self.document])
with patch('generator.engraving.layout_score', side_effect=AssertionError('cache miss')):
with patch('generator.engraving.rasterize_page', side_effect=AssertionError('cache miss')):
engrave_tree(warm, self.fonts, cache_dir=self.cache)
with unittest.mock.patch('generator.engraving.layout.layout_systems', side_effect=AssertionError('cache miss')):
with unittest.mock.patch('generator.engraving.raster.rasterize_page', side_effect=AssertionError('cache miss')):
generator.engraving.pipeline.engrave_tree(warm, self.fonts, cache_dir=self.cache)
self.assertEqual(self.files(cold), self.files(warm))
def test_playback_is_not_restored_from_page_cache(self):
self.seed_cache()
@@ -57,34 +202,34 @@ class EngravingCacheTests(unittest.TestCase):
changed['playback'] = {'new-playback': 'independent of page rendering'}
root = self.root / 'warm'
self.write_scores(root, [changed])
with patch('generator.engraving.layout_score', side_effect=AssertionError('cache miss')):
engrave_tree(root, self.fonts, cache_dir=self.cache)
with unittest.mock.patch('generator.engraving.layout.layout_systems', side_effect=AssertionError('cache miss')):
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache)
self.assertEqual(json.loads((root / '000.json').read_text())['playback'], changed['playback'])
def test_changed_music_and_renderer_invalidate_pages(self):
self.seed_cache()
changed = parse(xml_score(xml_note('D')).replace('Fixture', 'Changed'))
self.assertNotEqual(score_fingerprint(self.document, 'renderer'), score_fingerprint(changed, 'renderer'))
self.assertNotEqual(score_fingerprint(self.document, 'renderer'), score_fingerprint(self.document, 'new-renderer'))
changed = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note('D')).replace('Fixture', 'Changed'))
self.assertNotEqual(generator.engraving.cache.score_fingerprint(self.document, 'renderer'), generator.engraving.cache.score_fingerprint(changed, 'renderer'))
self.assertNotEqual(generator.engraving.cache.score_fingerprint(self.document, 'renderer'), generator.engraving.cache.score_fingerprint(self.document, 'new-renderer'))
root = self.root / 'changed'
self.write_scores(root, [changed])
with patch('generator.engraving.layout_score', wraps=layout_score) as render:
engrave_tree(root, self.fonts, cache_dir=self.cache)
with unittest.mock.patch('generator.engraving.layout.layout_systems', wraps=generator.engraving.layout.layout_systems) as render:
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache)
render.assert_called_once()
self.assertEqual(len(list(self.cache.rglob('pages.json'))), 2)
def test_renderer_fingerprint_tracks_fonts_style_code_and_dependencies(self):
fonts, style = Fonts(self.fonts), Style()
expected = renderer_fingerprint(fonts, style)
self.assertEqual(expected, renderer_fingerprint(fonts, style))
self.assertNotEqual(expected, renderer_fingerprint(fonts, Style(margin_x=29)))
with patch.object(fonts, 'fingerprint', return_value='changed-fonts'):
self.assertNotEqual(expected, renderer_fingerprint(fonts, style))
with patch('generator.engraving.cache.importlib.metadata.version', return_value='new-version'):
self.assertNotEqual(expected, renderer_fingerprint(fonts, style))
with patch('generator.engraving.cache.platform.python_version', return_value='0.0.0'):
self.assertNotEqual(expected, renderer_fingerprint(fonts, style))
read_bytes = Path.read_bytes
with patch.object(Path, 'read_bytes', lambda path: read_bytes(path) + (b'\n' if path.name == 'raster.py' else b'')):
self.assertNotEqual(expected, renderer_fingerprint(fonts, style))
fonts, style = generator.engraving.fonts.Fonts(self.fonts), generator.engraving.types.Style()
expected = generator.engraving.cache.renderer_fingerprint(fonts, style)
self.assertEqual(expected, generator.engraving.cache.renderer_fingerprint(fonts, style))
self.assertNotEqual(expected, generator.engraving.cache.renderer_fingerprint(fonts, generator.engraving.types.Style(margin_x=29)))
with unittest.mock.patch.object(fonts, 'fingerprint', return_value='changed-fonts'):
self.assertNotEqual(expected, generator.engraving.cache.renderer_fingerprint(fonts, style))
with unittest.mock.patch('generator.engraving.cache.importlib.metadata.version', return_value='new-version'):
self.assertNotEqual(expected, generator.engraving.cache.renderer_fingerprint(fonts, style))
with unittest.mock.patch('generator.engraving.cache.platform.python_version', return_value='0.0.0'):
self.assertNotEqual(expected, generator.engraving.cache.renderer_fingerprint(fonts, style))
read_bytes = pathlib.Path.read_bytes
with unittest.mock.patch.object(pathlib.Path, 'read_bytes', lambda path: read_bytes(path) + (b'\n' if path.name == 'raster.py' else b'')):
self.assertNotEqual(expected, generator.engraving.cache.renderer_fingerprint(fonts, style))
def test_corrupt_or_missing_pages_are_regenerated_and_repaired(self):
entry = self.seed_cache()
page = entry / '0001.png'
@@ -106,98 +251,112 @@ class EngravingCacheTests(unittest.TestCase):
target.mkdir()
root = self.root / corruption
self.write_scores(root, [self.document])
with patch('generator.engraving.layout_score', wraps=layout_score) as render:
engrave_tree(root, self.fonts, cache_dir=self.cache)
with unittest.mock.patch('generator.engraving.layout.layout_systems', wraps=generator.engraving.layout.layout_systems) as render:
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache)
render.assert_called_once()
self.assertEqual(page.read_bytes(), original)
self.assertIsNotNone(restore_pages(entry, self.root / 'restored'))
self.assertIsNotNone(generator.engraving.cache.restore_pages(entry, self.root / 'restored'))
def test_cache_manifest_cannot_escape_entry_or_inject_bundle_fields(self):
entry = self.seed_cache()
original = json.loads((entry / 'pages.json').read_text())
for invalid in ([], {}, [{'path': '../outside.png', 'sha256': original[0]['sha256']}], [{**original[0], 'extra': True}]):
for invalid in ([], {}, [{'path': '../outside.png', 'sha256': original[0]['sha256']}], [{**original[0], 'extra': True}], [{**original[0], 'sha256': None}], [{**original[0], 'sha256': 1}]):
with self.subTest(manifest=invalid):
(entry / 'pages.json').write_text(json.dumps(invalid))
self.assertIsNone(restore_pages(entry, self.root / 'output'))
self.assertIsNone(generator.engraving.cache.restore_pages(entry, self.root / 'output'))
(entry / 'pages.json').write_text(json.dumps(original))
page = entry / '0001.png'
external = self.root / 'outside.png'
page.rename(external)
page.symlink_to(external)
self.assertIsNone(restore_pages(entry, self.root / 'output'))
self.assertIsNone(generator.engraving.cache.restore_pages(entry, self.root / 'output'))
alias = self.root / 'alias'
alias.symlink_to(entry, target_is_directory=True)
self.assertIsNone(restore_pages(alias, self.root / 'output'))
self.assertIsNone(generator.engraving.cache.restore_pages(alias, self.root / 'output'))
def test_parallel_output_and_shared_cache_are_deterministic(self):
documents = [self.document, parse(xml_score(xml_note('D')))] * 4
documents = []
for index in range(4):
for letter in ('C', 'D'):
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(letter)))
document['metadata']['title'] = f'Fixture {index}'
documents.append(document)
serial, parallel, warm = (self.root / name for name in ('serial', 'parallel', 'warm'))
self.write_scores(serial, documents)
engrave_tree(serial, self.fonts)
generator.engraving.pipeline.engrave_tree(serial, self.fonts)
for root in (parallel, warm):
self.write_scores(root, documents)
engrave_tree(root, self.fonts, cache_dir=self.cache, jobs=2)
progress = io.StringIO()
with contextlib.redirect_stdout(progress):
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache, jobs=2, progress=True)
self.assertEqual(self.files(serial), self.files(root))
self.assertEqual(len(list(self.cache.rglob('pages.json'))), 2)
if root == parallel:
self.assertIn('fullscore 0, bodyhits 6, bodies 2, renderedpages 8', progress.getvalue())
else:
self.assertIn('fullscore 8, bodyhits 0, bodies 0, renderedpages 0', progress.getvalue())
self.assertEqual(len(list(self.cache.rglob('pages.json'))), 8)
self.assertEqual(len(list(self.cache.rglob('band.json'))), 2)
self.assertFalse(list(self.cache.rglob('.engraving-*')))
def test_environment_change_gets_new_namespace_and_clear_progress(self):
old = self.seed_cache()
root = self.root / 'new-environment'
self.write_scores(root, [self.document])
progress = io.StringIO()
with patch('generator.engraving.cache.platform.python_version', return_value='0.0.0'), contextlib.redirect_stdout(progress):
engrave_tree(root, self.fonts, cache_dir=self.cache, progress=True)
with unittest.mock.patch('generator.engraving.cache.platform.python_version', return_value='0.0.0'), contextlib.redirect_stdout(progress):
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache, progress=True)
self.assertIn('cold namespace', progress.getvalue())
self.assertIn('0 reused, 1 missing, 0 rejected', progress.getvalue())
self.assertTrue((old / 'pages.json').is_file())
self.assertEqual(len(list(self.cache.iterdir())), 2)
self.write_scores(root, [self.document])
progress = io.StringIO()
with patch('generator.engraving.cache.platform.python_version', return_value='0.0.0'), contextlib.redirect_stdout(progress), patch('generator.engraving.layout_score', side_effect=AssertionError('cache miss')):
engrave_tree(root, self.fonts, cache_dir=self.cache, progress=True)
with unittest.mock.patch('generator.engraving.cache.platform.python_version', return_value='0.0.0'), contextlib.redirect_stdout(progress), unittest.mock.patch('generator.engraving.layout.layout_systems', side_effect=AssertionError('cache miss')):
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache, progress=True)
self.assertIn('1 reused, 0 missing, 0 rejected', progress.getvalue())
def test_namespace_symlink_is_rejected(self):
self.cache.mkdir()
(self.cache / 'renderer').symlink_to(self.root, target_is_directory=True)
with self.assertRaisesRegex(ValueError, 'must not be symlinks'):
renderer_namespace(self.cache, 'renderer')
generator.engraving.cache.renderer_namespace(self.cache, 'renderer')
def test_only_changed_score_is_rendered_and_validation_reads_json_once(self):
root = self.root / 'scores'
self.write_scores(root, [self.document, self.document])
engrave_tree(root, self.fonts, cache_dir=self.cache)
changed = parse(xml_score(xml_note('D')))
self.write_scores(root, [self.document, changed])
with patch('generator.engraving.layout_score', wraps=layout_score) as render:
engrave_tree(root, self.fonts, cache_dir=self.cache)
self.write_scores(root / 'practice', [self.document, self.document])
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache)
changed = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note('D')))
self.write_scores(root / 'practice', [self.document, changed])
with unittest.mock.patch('generator.engraving.layout.layout_systems', wraps=generator.engraving.layout.layout_systems) as render:
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=self.cache)
render.assert_called_once()
read_text = Path.read_text
generator.pico_artwork.generate_titles(root, self.fonts)
read_text = pathlib.Path.read_text
reads = []
def read(path, *args, **kwargs):
reads.append(path)
return read_text(path, *args, **kwargs)
with patch.object(Path, 'read_text', read):
validate_generated_tree(root)
self.assertCountEqual(reads, sorted(root.glob('*.json')))
with unittest.mock.patch.object(pathlib.Path, 'read_text', read):
generator.score.validate_generated_tree(root)
self.assertCountEqual(reads, sorted(root.rglob('*.json')))
def test_invalid_worker_count_and_overlapping_cache_are_rejected(self):
root = self.root / 'assets'
self.write_scores(root, [self.document])
for jobs in (0, -1, 17, True):
with self.assertRaisesRegex(ValueError, 'between 1 and 16'):
engrave_tree(root, self.fonts, jobs=jobs)
generator.engraving.pipeline.engrave_tree(root, self.fonts, jobs=jobs)
for cache in (root, root / 'cache', self.root):
with self.assertRaisesRegex(ValueError, 'must not overlap'):
engrave_tree(root, self.fonts, cache_dir=cache)
generator.engraving.pipeline.engrave_tree(root, self.fonts, cache_dir=cache)
def test_parallel_generation_preserves_assets_on_worker_failure(self):
self.enterContext(unittest.mock.patch('generator.rhythm.load_library', return_value=tests.rhythm_fixtures.library()))
data, output = self.root / 'data', self.root / 'output'
(data / 'songs').mkdir(parents=True)
(data / 'practice').mkdir()
source = data / 'practice/0001.xml'
source.write_text(xml_score(xml_note()))
args = argparse.Namespace(data_dir=data, output_dir=output, static_assets_dir=Path('package/assets'), check=False, engraving_cache=self.cache, jobs=2)
generate_assets(args)
source.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
args = argparse.Namespace(data_dir=data, output_dir=output, static_assets_dir=pathlib.Path('package/assets'), check=False, engraving_cache=self.cache, jobs=2)
generator.__main__.generate_assets(args)
original = self.files(output)
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertEqual(original, self.files(output))
source.write_text(xml_score(xml_note(extra='<accidental smufl="unavailableGlyph">other</accidental>')))
with self.assertRaisesRegex(EngravingError, 'unavailableGlyph'):
generate_assets(args)
source.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(extra='<accidental smufl="unavailableGlyph">other</accidental>')))
with self.assertRaisesRegex(generator.engraving.types.EngravingError, 'unavailableGlyph'):
generator.__main__.generate_assets(args)
self.assertEqual(original, self.files(output))
self.assertFalse(list(output.parent.glob('.vamp-assets-*')))
tests/test_guitar.py
+100 -98
View File
@@ -1,185 +1,187 @@
import argparse
import copy
from fractions import Fraction
from pathlib import Path
import fractions
import pathlib
import random
import tempfile
import unittest
import unittest.mock
from generator import harmony
from generator.__main__ import generate_assets
from generator.arrange.generate import arrange_document
from generator.guitar import Placement, TabPosition, chord_positions, chord_requirements, nearest_positions, place_guitar, position_center, positions_for_pitch, within_fret_span
from generator.model import fraction, pitch_number, rational, validate_document
from generator.parts import PART_TYPES
from generator.pitch import concert_document
from generator.playback import compile_timeline
from generator.practice.generate import generate_practice_scores, practice_document
from generator.summary import validate_practice_parts
from tests.test_musicxml import notes, parse, xml_note, xml_score
import generator.__main__
import generator.arrange.generate
import generator.guitar
import generator.harmony
import generator.model
import generator.parts
import generator.pitch
import generator.playback
import generator.practice.generate
import generator.summary
import tests.rhythm_fixtures
import tests.test_musicxml
class GuitarTests(unittest.TestCase):
def test_fret_boundaries_and_span(self):
self.assertIn(TabPosition(6, 0, 40), positions_for_pitch(Fraction(40)))
self.assertIn(TabPosition(1, 20, 84), positions_for_pitch(Fraction(84)))
self.assertFalse(positions_for_pitch(Fraction(85)))
self.assertFalse(positions_for_pitch(Fraction(129, 2)))
self.assertTrue(within_fret_span((TabPosition(4, 11, 61), TabPosition(3, 12, 67), TabPosition(2, 14, 73))))
self.assertTrue(within_fret_span((TabPosition(6, 0, 40), TabPosition(1, 20, 84))))
self.assertFalse(within_fret_span((TabPosition(4, 10, 60), TabPosition(1, 20, 84))))
self.assertIn(generator.guitar.TabPosition(6, 0, 40), generator.guitar.positions_for_pitch(fractions.Fraction(40)))
self.assertIn(generator.guitar.TabPosition(1, 20, 84), generator.guitar.positions_for_pitch(fractions.Fraction(84)))
self.assertFalse(generator.guitar.positions_for_pitch(fractions.Fraction(85)))
self.assertFalse(generator.guitar.positions_for_pitch(fractions.Fraction(129, 2)))
self.assertTrue(generator.guitar.within_fret_span((generator.guitar.TabPosition(4, 11, 61), generator.guitar.TabPosition(3, 12, 67), generator.guitar.TabPosition(2, 14, 73))))
self.assertTrue(generator.guitar.within_fret_span((generator.guitar.TabPosition(6, 0, 40), generator.guitar.TabPosition(1, 20, 84))))
self.assertFalse(generator.guitar.within_fret_span((generator.guitar.TabPosition(4, 10, 60), generator.guitar.TabPosition(1, 20, 84))))
def test_theory_controls_omissions_and_bass(self):
for symbol in ('CM', 'G7b9', 'FM7#11/A', 'CmM7', 'Cdim7', 'C13', 'Csus', 'Cmadd9'):
with self.subTest(symbol=symbol):
required, allowed, bass = chord_requirements(harmony.parse_symbol(symbol))
candidates = chord_positions(required, allowed, bass)
required, allowed, bass = generator.guitar.chord_requirements(generator.harmony.parse_symbol(symbol))
candidates = generator.guitar.chord_positions(required, allowed, bass)
self.assertTrue(candidates)
for candidate in candidates:
covered = {p.midi % 12 for p in candidate}
self.assertTrue(set(required) <= covered <= set(allowed))
self.assertEqual(len(candidate), len({p.string_number for p in candidate}))
self.assertTrue(within_fret_span(candidate))
self.assertTrue(generator.guitar.within_fret_span(candidate))
if bass is not None:
self.assertEqual(min(p.midi for p in candidate) % 12, bass)
self.assertIn(2, chord_requirements(harmony.parse_symbol('Cmadd9'))[0])
self.assertNotIn(7, chord_requirements(harmony.parse_symbol('C7'))[0])
self.assertIn(7, chord_requirements(harmony.parse_symbol('CM'))[0])
self.assertEqual(chord_requirements({**harmony.parse_symbol('CM'), 'inversion': 0})[2], 0)
self.assertEqual(chord_requirements({**harmony.parse_symbol('CM'), 'inversion': 1})[2], 4)
self.assertTrue(any(p.fret > 12 for c in chord_positions(*chord_requirements(harmony.parse_symbol('CM'))) for p in c))
self.assertIn(2, generator.guitar.chord_requirements(generator.harmony.parse_symbol('Cmadd9'))[0])
self.assertNotIn(7, generator.guitar.chord_requirements(generator.harmony.parse_symbol('C7'))[0])
self.assertIn(7, generator.guitar.chord_requirements(generator.harmony.parse_symbol('CM'))[0])
self.assertEqual(generator.guitar.chord_requirements({**generator.harmony.parse_symbol('CM'), 'inversion': 0})[2], 0)
self.assertEqual(generator.guitar.chord_requirements({**generator.harmony.parse_symbol('CM'), 'inversion': 1})[2], 4)
self.assertTrue(any(p.fret > 12 for c in generator.guitar.chord_positions(*generator.guitar.chord_requirements(generator.harmony.parse_symbol('CM'))) for p in c))
def test_melody_placement_preserves_notes_timing_and_sound(self):
source = parse(xml_score(xml_note('E', 2, 'half', octave=4) + xml_note('F', 2, 'half', octave=4)))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note('E', 2, 'half', octave=4) + tests.test_musicxml.xml_note('F', 2, 'half', octave=4)))
before = copy.deepcopy(source)
electric = practice_document(source, 'electric')
vocal = practice_document(source, 'vocal')
validate_document(electric)
electric = generator.practice.generate.practice_document(source, 'electric', rhythms=tests.rhythm_fixtures.library())
vocal = generator.practice.generate.practice_document(source, 'vocal', rhythms=tests.rhythm_fixtures.library())
generator.model.validate_document(electric)
self.assertEqual(source, before)
self.assertEqual(compile_timeline(electric)['events'], compile_timeline(vocal)['events'])
self.assertEqual([(e['pitch'], e['onset'], e['duration']) for e in compile_timeline(electric)['events']], [(rational(fraction(e['pitch']) - 12), e['onset'], e['duration']) for e in compile_timeline(source)['events']])
for original, generated in zip(notes(source), notes(electric)):
self.assertEqual(pitch_number(generated['pitches'][0]), pitch_number(original['pitches'][0]))
self.assertEqual(generator.playback.compile_timeline(electric)['events'], generator.playback.compile_timeline(vocal)['events'])
self.assertEqual([(e['pitch'], e['onset'], e['duration']) for e in generator.playback.compile_timeline(electric)['events']], [(generator.model.rational(generator.model.fraction(e['pitch']) - 12), e['onset'], e['duration']) for e in generator.playback.compile_timeline(source)['events']])
for original, generated in zip(tests.test_musicxml.notes(source), tests.test_musicxml.notes(electric)):
self.assertEqual(generator.model.pitch_number(generated['pitches'][0]), generator.model.pitch_number(original['pitches'][0]))
self.assertIn('tablature', generated['pitches'][0])
def test_instrument_transposition_and_octave_clef_applied_once(self):
for attributes, expected in (('', 48), ('<transpose><chromatic>-12</chromatic><diatonic>-7</diatonic></transpose>', 48), ('<transpose><chromatic>0</chromatic></transpose>', 60), ('<clef><sign>G</sign><line>2</line><clef-octave-change>-1</clef-octave-change></clef>', 60), ('<clef><sign>F</sign><line>4</line></clef>', 48)):
source = parse(xml_score(xml_note(), attributes))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(), attributes))
for part in ('electric', 'vocal'):
document = practice_document(source, part)
self.assertEqual(fraction(compile_timeline(document)['events'][0]['pitch']), expected)
document = generator.practice.generate.practice_document(source, part, rhythms=tests.rhythm_fixtures.library())
self.assertEqual(generator.model.fraction(generator.playback.compile_timeline(document)['events'][0]['pitch']), expected)
def test_written_middle_c_and_c_sharp_six_have_matching_tab_and_audio(self):
for octave, alter, expected, tab in ((4, 0, 48, {'string': 5, 'fret': 3}), (6, 1, 73, {'string': 1, 'fret': 9}), (7, 0, 84, {'string': 1, 'fret': 20})):
source = parse(xml_score(xml_note(octave=octave, alter=alter)))
document = practice_document(source, 'electric')
tone = notes(document)[0]['pitches'][0]
self.assertEqual(pitch_number(tone), 12 * (octave + 1) + alter)
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(octave=octave, alter=alter)))
document = generator.practice.generate.practice_document(source, 'electric', rhythms=tests.rhythm_fixtures.library())
tone = tests.test_musicxml.notes(document)[0]['pitches'][0]
self.assertEqual(generator.model.pitch_number(tone), 12 * (octave + 1) + alter)
self.assertEqual(tone['tablature'], tab)
self.assertEqual(fraction(compile_timeline(document)['events'][0]['pitch']), expected)
validate_document(document)
self.assertEqual(generator.model.fraction(generator.playback.compile_timeline(document)['events'][0]['pitch']), expected)
generator.model.validate_document(document)
def test_guitar_normalization_is_idempotent_and_respects_timed_transposition(self):
change = '<attributes><transpose><chromatic>0</chromatic></transpose></attributes>'
source = parse(xml_score(xml_note(duration=2, kind='half') + change + xml_note(duration=2, kind='half')))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=2, kind='half') + change + tests.test_musicxml.xml_note(duration=2, kind='half')))
before = copy.deepcopy(source)
concert = concert_document(source, octave_transposing=True)
self.assertEqual(concert_document(concert, octave_transposing=True), concert)
concert = generator.pitch.concert_document(source, octave_transposing=True)
self.assertEqual(generator.pitch.concert_document(concert, octave_transposing=True), concert)
self.assertEqual(source, before)
document = practice_document(source, 'electric')
self.assertEqual([fraction(e['pitch']) for e in compile_timeline(document)['events']], [48, 60])
document = generator.practice.generate.practice_document(source, 'electric', rhythms=tests.rhythm_fixtures.library())
self.assertEqual([generator.model.fraction(e['pitch']) for e in generator.playback.compile_timeline(document)['events']], [48, 60])
def test_all_parts_share_written_octave_convention_in_songs_and_practice(self):
chord = '<harmony><root><root-step>C</root-step></root><kind>major</kind></harmony>'
source = parse(xml_score(chord + xml_note()))
for generate in (practice_document, arrange_document):
for part in PART_TYPES:
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(chord + tests.test_musicxml.xml_note()))
for generate in (generator.practice.generate.practice_document, generator.arrange.generate.arrange_document):
for part in generator.parts.PART_TYPES:
with self.subTest(generator=generate.__name__, part=part):
document = generate(source, part)
validate_document(document)
written = {p['id']: pitch_number(p) for m in document['staves'][0]['measures'] for v in m['voices'] for n in v['notes'] for p in n['pitches']}
events = compile_timeline(document)['events']
document = generate(source, part, rhythms=tests.rhythm_fixtures.library())
generator.model.validate_document(document)
written = {p['id']: generator.model.pitch_number(p) for m in document['staves'][0]['measures'] for v in m['voices'] for n in v['notes'] for p in n['pitches']}
events = generator.playback.compile_timeline(document)['events']
self.assertTrue(events)
self.assertTrue(all(fraction(e['pitch']) == written[e['id']] - 12 for e in events))
self.assertTrue(all(generator.model.fraction(e['pitch']) == written[e['id']] - 12 for e in events))
if part == 'vocal':
self.assertEqual(set(written.values()), {60})
self.assertTrue(all(m['transpose']['octave_change'] == -1 for m in document['staves'][0]['measures']))
def test_ties_and_overlapping_voices_keep_strings(self):
content = xml_note('E', 1, 'quarter', octave=4, extra='<tie type="start"/>') + xml_note('E', 3, 'half', octave=4, extra='<dot/><tie type="stop"/>')
content += '<backup><duration>4</duration></backup><forward><duration>1</duration></forward>' + xml_note('F', 3, 'half', octave=4, extra='<dot/><voice>2</voice>')
document = parse(xml_score(content))
place_guitar(document)
content = tests.test_musicxml.xml_note('E', 1, 'quarter', octave=4, extra='<tie type="start"/>') + tests.test_musicxml.xml_note('E', 3, 'half', octave=4, extra='<dot/><tie type="stop"/>')
content += '<backup><duration>4</duration></backup><forward><duration>1</duration></forward>' + tests.test_musicxml.xml_note('F', 3, 'half', octave=4, extra='<dot/><voice>2</voice>')
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))
generator.guitar.place_guitar(document)
first, continuation, other = [p['tablature'] for m in document['staves'][0]['measures'] for v in m['voices'] for n in v['notes'] for p in n['pitches']]
self.assertEqual(first, continuation)
self.assertNotEqual(continuation['string'], other['string'])
self.assertEqual(len(compile_timeline(document)['events']), 2)
self.assertEqual(len(generator.playback.compile_timeline(document)['events']), 2)
def test_proximity_chooses_a_path_without_relocating_tied_notes(self):
source = parse(xml_score(xml_note('B', 2, 'half', octave=5) + xml_note('C', 2, 'half', octave=6)))
place_guitar(source)
tabs = [n['pitches'][0]['tablature'] for n in notes(source)]
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note('B', 2, 'half', octave=5) + tests.test_musicxml.xml_note('C', 2, 'half', octave=6)))
generator.guitar.place_guitar(source)
tabs = [n['pitches'][0]['tablature'] for n in tests.test_musicxml.notes(source)]
self.assertEqual(tabs, [{'string': 1, 'fret': 19}, {'string': 1, 'fret': 20}])
def test_grid_search_matches_exhaustive_costs_and_tie_breaks(self):
randomizer = random.Random(17)
states = {}
for index in range(40):
positions = tuple(TabPosition(s, randomizer.randrange(21), 0) for s in randomizer.sample(range(1, 7), 3))
states[index] = Placement(tuple(randomizer.randrange(8) for _ in range(3)), tuple((str(i), p) for i, p in enumerate(positions)), None, ())
grid = nearest_positions(states)
positions = tuple(generator.guitar.TabPosition(s, randomizer.randrange(21), 0) for s in randomizer.sample(range(1, 7), 3))
states[index] = generator.guitar.Placement(tuple(randomizer.randrange(8) for _ in range(3)), tuple((str(i), p) for i, p in enumerate(positions)), None, ())
grid = generator.guitar.nearest_positions(states)
for x, row in enumerate(grid):
for y, actual in enumerate(row):
candidates = []
for index, state in enumerate(states.values()):
a, b = position_center(tuple(p for _, p in state.active))
a, b = generator.guitar.position_center(tuple(p for _, p in state.active))
candidates.append(((state.cost[0] + abs(x - a) + abs(y - b), *state.cost[1:]), index))
self.assertEqual(actual[:2], min(candidates))
def test_generated_chord_retains_strings_when_a_melody_enters(self):
content = xml_note() + '<backup><duration>4</duration></backup><forward><duration>2</duration></forward>' + xml_note('C', 2, 'half', octave=7, extra='<voice>2</voice>')
document = parse(xml_score(content))
content = tests.test_musicxml.xml_note() + '<backup><duration>4</duration></backup><forward><duration>2</duration></forward>' + tests.test_musicxml.xml_note('C', 2, 'half', octave=7, extra='<voice>2</voice>')
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))
measure = document['staves'][0]['measures'][0]
measure['transpose']['octave_change'] = -1
chord, melody = [n for v in measure['voices'] for n in v['notes']]
chord['pitches'] = []
chord['_harmony'] = harmony.parse_symbol('CM')
place_guitar(document)
validate_document(document)
chord['_harmony'] = generator.harmony.parse_symbol('CM')
generator.guitar.place_guitar(document)
generator.model.validate_document(document)
self.assertEqual(melody['pitches'][0]['tablature'], {'string': 1, 'fret': 20})
self.assertNotIn(1, [p['tablature']['string'] for p in chord['pitches']])
self.assertEqual(len(compile_timeline(document)['events']), 4)
self.assertEqual(len(generator.playback.compile_timeline(document)['events']), 4)
def test_more_than_six_simultaneous_tones_fail_without_omission(self):
content = '<backup><duration>4</duration></backup>'.join(xml_note(extra=f'<voice>{i}</voice>') for i in range(7))
document = parse(xml_score(content))
content = '<backup><duration>4</duration></backup>'.join(tests.test_musicxml.xml_note(extra=f'<voice>{i}</voice>') for i in range(7))
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))
before = copy.deepcopy(document)
with self.assertRaisesRegex(ValueError, 'no guitar placement'):
place_guitar(document)
generator.guitar.place_guitar(document)
self.assertEqual(before, document)
def test_source_contract_and_unplayable_music_fail(self):
cases = [(xml_note(octave=7, alter=1), '', '0-20'), (xml_note(alter='0.5'), '', '0-20'),
(xml_note() + xml_note('E', prefix='<chord/>'), '', 'monophonic'),
(xml_note() + '<backup><duration>4</duration></backup>' + xml_note('E', extra='<voice>2</voice>'), '', 'one monophonic voice'),
(xml_note(), '<staff-details><capo>1</capo></staff-details>', 'capo')]
cases = [(tests.test_musicxml.xml_note(octave=7, alter=1), '', '0-20'), (tests.test_musicxml.xml_note(alter='0.5'), '', '0-20'),
(tests.test_musicxml.xml_note() + tests.test_musicxml.xml_note('E', prefix='<chord/>'), '', 'monophonic'),
(tests.test_musicxml.xml_note() + '<backup><duration>4</duration></backup>' + tests.test_musicxml.xml_note('E', extra='<voice>2</voice>'), '', 'one monophonic voice'),
(tests.test_musicxml.xml_note(), '<staff-details><capo>1</capo></staff-details>', 'capo')]
for content, attributes, error in cases:
with self.subTest(error=error), self.assertRaisesRegex(ValueError, error):
practice_document(parse(xml_score(content, attributes)), 'electric')
for content, attributes in [(xml_note(), '<clef><sign>TAB</sign></clef>'), (xml_note(extra='<notations><technical><fingering>1</fingering></technical></notations>'), '')]:
generator.practice.generate.practice_document(tests.test_musicxml.parse(tests.test_musicxml.xml_score(content, attributes)), 'electric', rhythms=tests.rhythm_fixtures.library())
for content, attributes in [(tests.test_musicxml.xml_note(), '<clef><sign>TAB</sign></clef>'), (tests.test_musicxml.xml_note(extra='<notations><technical><fingering>1</fingering></technical></notations>'), '')]:
with self.assertRaisesRegex(ValueError, 'authored TAB'):
parse(xml_score(content, attributes))
tests.test_musicxml.parse(tests.test_musicxml.xml_score(content, attributes))
def test_three_parts_are_deterministic_without_transposition(self):
with tempfile.TemporaryDirectory() as temporary:
source, output = Path(temporary) / 'source', Path(temporary) / 'output'
source, output = pathlib.Path(temporary) / 'source', pathlib.Path(temporary) / 'output'
source.mkdir()
(source / '0000.xml').write_text(xml_score(xml_note('E', alter=-1), '<key><fifths>-3</fifths></key>'))
self.assertEqual(generate_practice_scores(source, output), 3)
validate_practice_parts(output)
(source / '0000.xml').write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note('E', alter=-1), '<key><fifths>-3</fifths></key>'))
self.assertEqual(generator.practice.generate.generate_practice_scores(source, output, rhythms=tests.rhythm_fixtures.library()), 3)
generator.summary.validate_practice_parts(output)
first = {p.name: p.read_bytes() for p in output.rglob('*.json')}
generate_practice_scores(source, output)
generator.practice.generate.generate_practice_scores(source, output, rhythms=tests.rhythm_fixtures.library())
self.assertEqual(first, {p.name: p.read_bytes() for p in output.rglob('*.json')})
(output / '0000/vocal.json').unlink()
with self.assertRaisesRegex(ValueError, 'practice generator drift'):
validate_practice_parts(output)
generator.summary.validate_practice_parts(output)
def test_unplayable_generation_preserves_previous_assets(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
root = pathlib.Path(temporary)
(root / 'data/songs').mkdir(parents=True)
(root / 'data/practice').mkdir()
(root / 'data/practice/0001.xml').write_text(xml_score(xml_note(octave=7, alter=1)))
(root / 'data/practice/0001.xml').write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(octave=7, alter=1)))
output = root / 'output'
output.mkdir()
(output / 'last-good').write_text('preserved')
args = argparse.Namespace(data_dir=root / 'data', output_dir=output, static_assets_dir=Path('package/assets'), check=False)
with self.assertRaisesRegex(ValueError, '0001.xml, part electric:.*0-20'):
generate_assets(args)
args = argparse.Namespace(data_dir=root / 'data', output_dir=output, static_assets_dir=pathlib.Path('package/assets'), check=False)
with unittest.mock.patch('generator.rhythm.load_library', return_value=tests.rhythm_fixtures.library()), self.assertRaisesRegex(ValueError, '0001.xml, part electric:.*0-20'):
generator.__main__.generate_assets(args)
self.assertEqual([p.name for p in output.iterdir()], ['last-good'])
self.assertEqual((output / 'last-good').read_text(), 'preserved')
tests/test_harmony.py
+44 -41
View File
@@ -1,14 +1,18 @@
from pathlib import Path
import pathlib
import tempfile
import unittest
from generator import harmony
from generator.model import rational
from generator.musicxml import MusicXMLError, load_musicxml
from generator.musicxml_writer import write_musicxml
from generator.practice.generate import practice_document
from generator.pitch import concert_document
from tests.test_musicxml import parse, xml_note, xml_score
import generator.engraving.fonts
import generator.engraving.layout
import generator.engraving.types
import generator.harmony
import generator.model
import generator.musicxml
import generator.musicxml_writer
import generator.pitch
import generator.practice.generate
import tests.rhythm_fixtures
import tests.test_musicxml
def chord_xml(text, kind='other', root='C', extra=''):
return f'<harmony><root><root-step>{root}</root-step></root><kind text="{text}">{kind}</kind>{extra}</harmony>'
@@ -20,82 +24,81 @@ class HarmonyTests(unittest.TestCase):
'C-maj7': 'CmM7', 'CmM': 'CmM7', 'C6': 'CM6', 'C-6': 'Cm6', 'Fmaj7#11/A': 'FM7#11/A', 'G7b9': 'G7b9'}
for spelling, canonical in cases.items():
with self.subTest(spelling=spelling):
chord = harmony.parse_symbol(spelling)
self.assertEqual(harmony.symbol(chord), canonical)
self.assertEqual(harmony.degrees(harmony.parse_symbol(canonical)), harmony.degrees(chord))
chord = generator.harmony.parse_symbol(spelling)
self.assertEqual(generator.harmony.symbol(chord), canonical)
self.assertEqual(generator.harmony.degrees(generator.harmony.parse_symbol(canonical)), generator.harmony.degrees(chord))
def test_extended_chords_include_sevenths_but_added_ninths_do_not(self):
for prefix, seventh in (('', -1), ('m', -1), ('M', 0), ('mM', 0)):
for highest in (9, 11, 13):
with self.subTest(prefix=prefix, highest=highest):
chord = harmony.parse_symbol(f'C{prefix}{highest}')
values = harmony.degrees(chord)
chord = generator.harmony.parse_symbol(f'C{prefix}{highest}')
values = generator.harmony.degrees(chord)
self.assertEqual(values[7], seventh)
self.assertTrue(all(values[degree] == 0 for degree in range(9, highest + 1, 2)))
self.assertEqual(harmony.symbol(chord), f'C{prefix}{highest}')
self.assertEqual(harmony.degrees(harmony.parse_symbol('Cmadd9')), {1: 0, 3: -1, 5: 0, 9: 0})
self.assertEqual(harmony.degrees(harmony.parse_symbol('C7no5')), {1: 0, 3: 0, 7: -1})
self.assertEqual(harmony.degrees(harmony.parse_symbol('C7sus')), {1: 0, 4: 0, 5: 0, 7: -1})
self.assertEqual(generator.harmony.symbol(chord), f'C{prefix}{highest}')
self.assertEqual(generator.harmony.degrees(generator.harmony.parse_symbol('Cmadd9')), {1: 0, 3: -1, 5: 0, 9: 0})
self.assertEqual(generator.harmony.degrees(generator.harmony.parse_symbol('C7no5')), {1: 0, 3: 0, 7: -1})
self.assertEqual(generator.harmony.degrees(generator.harmony.parse_symbol('C7sus')), {1: 0, 4: 0, 5: 0, 7: -1})
def test_soundslice_text_only_chords_become_structured(self):
for text, kind, values in (('mM7', 'major-minor', {1: 0, 3: -1, 5: 0, 7: 0}), ('m7b5', 'half-diminished', {1: 0, 3: -1, 5: -1, 7: -1}), ('7b9', 'dominant', {1: 0, 3: 0, 5: 0, 7: -1, 9: -1})):
with self.subTest(text=text):
source = parse(xml_score(chord_xml(text) + xml_note()))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(chord_xml(text) + tests.test_musicxml.xml_note()))
chord = source['staves'][0]['measures'][0]['harmonies'][0]
self.assertEqual(chord['kind'], kind)
self.assertEqual(harmony.degrees(chord), values)
self.assertEqual(generator.harmony.degrees(chord), values)
self.assertEqual(chord['symbol'], 'C' + text)
self.assertNotIn('kind_text', chord)
source = parse(xml_score(chord_xml('m9', 'minor-ninth') + xml_note()))
self.assertEqual(harmony.degrees(source['staves'][0]['measures'][0]['harmonies'][0])[7], -1)
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(chord_xml('m9', 'minor-ninth') + tests.test_musicxml.xml_note()))
self.assertEqual(generator.harmony.degrees(source['staves'][0]['measures'][0]['harmonies'][0])[7], -1)
def test_parenthesized_full_symbols_transpose_and_roundtrip(self):
for label, transposed in (('(Fm7)', '(Gm7)'), ('(Bb7)', '(C7)'), ('(Fm7/Ab)', '(Gm7/Bb)')):
with self.subTest(label=label):
source = parse(xml_score(chord_xml(label) + xml_note()))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(chord_xml(label) + tests.test_musicxml.xml_note()))
chord = source['staves'][0]['measures'][0]['harmonies'][0]
self.assertEqual(chord['symbol'], label)
self.assertTrue(chord['chord_optional'])
self.assertTrue(chord['parenthesized'])
practice = practice_document(source, 'vocal')['staves'][0]['measures'][0]['harmonies'][0]
practice = generator.practice.generate.practice_document(source, 'vocal', rhythms=tests.rhythm_fixtures.library())['staves'][0]['measures'][0]['harmonies'][0]
self.assertNotIn('chord_optional', practice)
self.assertEqual(harmony.symbol(practice), label)
self.assertEqual(generator.harmony.symbol(practice), label)
source['staves'][0]['measures'][0]['transpose'] = {'diatonic': 1, 'chromatic': 2, 'octave_change': 0}
shifted = concert_document(source)
shifted = generator.pitch.concert_document(source)
result = shifted['staves'][0]['measures'][0]['harmonies'][0]
self.assertEqual(result['symbol'], transposed)
with tempfile.TemporaryDirectory() as temporary:
path = Path(temporary) / 'score.xml'
write_musicxml(path, shifted)
restored = load_musicxml(path)['staves'][0]['measures'][0]['harmonies'][0]
path = pathlib.Path(temporary) / 'score.xml'
generator.musicxml_writer.write_musicxml(path, shifted)
restored = generator.musicxml.load_musicxml(path)['staves'][0]['measures'][0]['harmonies'][0]
self.assertEqual(restored, result)
def test_requested_labels_are_engraved_without_overlaps(self):
from generator.engraving import Fonts, Style, layout_score
labels = ('CM', 'Cm7', 'CM13#11', 'CmM7', 'CaugM7', 'Cdim7', 'Csus', 'Csus2', 'C7sus4', '(Fm7/Ab)')
for label in labels:
with self.subTest(label=label):
source = parse(xml_score(chord_xml(label) + xml_note()))
layout = layout_score(source, Style(), Fonts(Path('package/assets/fonts/petaluma')))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(chord_xml(label) + tests.test_musicxml.xml_note()))
layout = generator.engraving.layout.layout_score(source, generator.engraving.types.Style(), generator.engraving.fonts.Fonts(pathlib.Path('package/assets/fonts/petaluma')))
rendered = [p.value for page in layout.pages for p in page.primitives if p.role == 'harmony']
self.assertEqual(rendered, [harmony.symbol(harmony.parse_symbol(label))])
self.assertEqual(rendered, [generator.harmony.symbol(generator.harmony.parse_symbol(label))])
def test_unknown_or_contradictory_symbols_are_rejected(self):
for label in ('Cunknown', 'Cm79', 'C7b9#9', 'C7add7', 'C7no3no3', 'C7no#5', '(Cm7', 'Cm7)', 'C7/G/Ab'):
with self.subTest(label=label), self.assertRaises(ValueError):
harmony.parse_symbol(label)
generator.harmony.parse_symbol(label)
for content in (chord_xml('m7', 'major-seventh'), chord_xml('(Fm7)', 'minor-seventh'), chord_xml('(Fm7)', root='D'), chord_xml('m7/G', extra='<bass><bass-step>A</bass-step></bass>')):
with self.subTest(content=content), self.assertRaisesRegex(MusicXMLError, 'contradictory'):
parse(xml_score(content + xml_note()))
with self.subTest(content=content), self.assertRaisesRegex(generator.musicxml.MusicXMLError, 'contradictory'):
tests.test_musicxml.parse(tests.test_musicxml.xml_score(content + tests.test_musicxml.xml_note()))
def test_kind_text_cannot_override_canonical_output(self):
chord = harmony.parse_symbol('Cm7') | {'kind_text': '-7'}
self.assertEqual(harmony.symbol(chord), 'Cm7')
source = parse(xml_score(chord_xml('7', 'dominant', extra='<degree><degree-value>9</degree-value><degree-alter>-1</degree-alter><degree-type>add</degree-type></degree>') + xml_note()))
chord = generator.harmony.parse_symbol('Cm7') | {'kind_text': '-7'}
self.assertEqual(generator.harmony.symbol(chord), 'Cm7')
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(chord_xml('7', 'dominant', extra='<degree><degree-value>9</degree-value><degree-alter>-1</degree-alter><degree-type>add</degree-type></degree>') + tests.test_musicxml.xml_note()))
self.assertEqual(source['staves'][0]['measures'][0]['harmonies'][0]['symbol'], 'C7b9')
def test_current_exercise_and_song_exports_import_without_edits(self):
exercise = load_musicxml(Path('data/practice/0001.xml'))
exercise = generator.musicxml.load_musicxml(pathlib.Path('data/practice/0001.xml'))
measures = exercise['staves'][0]['measures']
self.assertEqual(len(measures), 20)
self.assertEqual(sum(len(n['pitches']) for m in measures for v in m['voices'] for n in v['notes']), 70)
self.assertEqual(measures[1]['harmonies'][0]['symbol'], 'Bm9')
self.assertEqual(measures[3]['harmonies'][0]['symbol'], 'DmM7')
self.assertEqual(measures[0]['voices'][0]['notes'][0]['duration'], rational(1))
self.assertEqual(measures[0]['voices'][0]['notes'][0]['duration'], generator.model.rational(1))
for name, count, key in (('autumn-leaves', 33, 'F'), ('it-could-happen-to-you', 32, 'Eb')):
source = load_musicxml(Path('data/songs') / f'{name}.xml')
source = generator.musicxml.load_musicxml(pathlib.Path('data/songs') / f'{name}.xml')
self.assertEqual(len(source['staves'][0]['measures']), count)
self.assertEqual(source['metadata']['key_signature'], key)
tests/test_health.cpp
+33 -5
View File
@@ -165,11 +165,16 @@ auto checksum(std::string const &bytes) -> std::string {
return result;
}
auto write_score(std::filesystem::path const &path, std::string part = "Lead") -> void {
auto write_score(std::filesystem::path const &path, std::filesystem::path const &assets, std::string part = "Lead") -> void {
auto score = boost::json::parse(R"({"music":{"metadata":{"title":"Fixture","composer":"Composer",
"key_signature":"C","time_signature":"4/4"},"staves":[{"name":"Lead","measures":[{"voices":[]}]}]},
"presentation":{"width":976,"height":484,"pages":[]}})");
score.at("music").at("staves").as_array().front().as_object()["name"] = part;
auto title = std::string{"VPTI\x28\x01\x40\x00", 8U};
title.resize(vamp_protocol::title_image_size, std::char_t{0x11});
auto title_image = checksum(title);
write_file(assets / "pico" / "titles" / (title_image + ".mask"), title);
score.at("presentation").as_object()["title_image"] = title_image;
auto &pages = score.at("presentation").at("pages").as_array();
for (auto index = std::uint32_t{1}; index <= 2U; ++index) {
auto relative = path.stem().string() + ".pages/" + std::to_string(index) + ".png";
@@ -182,12 +187,12 @@ auto write_score(std::filesystem::path const &path, std::string part = "Lead") -
auto make_assets(vamp_zero::config const &config) -> void {
auto root = std::filesystem::path{config.runtime.assets_path};
write_score(root / "songs" / "first.json");
write_score(root / "songs" / "second.json");
write_score(root / "songs" / "first.json", root);
write_score(root / "songs" / "second.json", root);
for (auto part_name : vamp_zero::practice_part_names()) {
auto name = std::string{part_name};
name.front() = static_cast<std::char_t>(name.front() - 'A' + 'a');
write_score(root / "practice" / "0001" / (name + ".json"), std::string{part_name});
write_score(root / "practice" / "0001" / (name + ".json"), root, std::string{part_name});
}
BOOST_REQUIRE(vamp_test::build_catalog(root, config.database.path));
}
@@ -387,6 +392,29 @@ BOOST_AUTO_TEST_CASE(readiness_validates_later_pages_and_parts) {
}
}
BOOST_AUTO_TEST_CASE(readiness_rejects_missing_or_corrupted_pico_titles) {
for (auto corrupt : {false, true}) {
auto root = temporary_directory{};
auto config = fixture_config(root.path);
make_assets(config);
auto titles = std::filesystem::path{config.runtime.assets_path} / "pico" / "titles";
auto path = std::filesystem::directory_iterator {
titles
} -> path();
if (corrupt) {
auto bytes = read_file(path);
bytes.back() ^= 1;
write_file(path, bytes);
} else {
std::filesystem::remove(path);
}
auto ready = run_once(root.path, config);
BOOST_REQUIRE(!ready);
BOOST_TEST(std::string{ready.error().message()}.find("Pico title") != std::string::npos);
require_no_probe(config);
}
}
BOOST_AUTO_TEST_CASE(readiness_validates_generated_audio_using_configured_asset_root) {
auto root = temporary_directory{};
auto config = fixture_config(root.path);
@@ -395,7 +423,7 @@ BOOST_AUTO_TEST_CASE(readiness_validates_generated_audio_using_configured_asset_
auto fixture = std::filesystem::path{VAMP_TEST_PLAYBACK_ASSETS_DIR};
std::filesystem::copy(fixture / "audio", assets / "audio", std::filesystem::copy_options::recursive);
auto path = assets / "practice/0001/acoustic.json";
write_score(path, "Acoustic");
write_score(path, assets, "Acoustic");
auto score = boost::json::parse(read_file(path));
score.as_object()["playback"] = boost::json::parse(read_file(fixture / "score.json")).at("playback");
write_file(path, boost::json::serialize(score));
tests/test_install_zero.py
+5 -5
View File
@@ -1,15 +1,15 @@
import json
import hashlib
import io
import json
import os
from pathlib import Path
import pathlib
import shutil
import subprocess
import tempfile
import tarfile
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
ROOT = pathlib.Path(__file__).resolve().parents[1]
MOCK = '''#!/usr/bin/env python3
import json, os, sys
from pathlib import Path
@@ -50,7 +50,7 @@ class InstallZeroTests(unittest.TestCase):
def setUp(self):
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name)
self.root = pathlib.Path(self.temporary.name)
self.bundle = self.root / 'bundle'
self.zero = self.bundle / 'zero'
self.zero.mkdir(parents=True)
tests/test_key_metadata.py
+39 -40
View File
@@ -1,40 +1,39 @@
import copy
from fractions import Fraction
import json
from pathlib import Path
import tempfile
import fractions
import pathlib
import unittest
from xml.etree import ElementTree
import xml.etree.ElementTree
from generator.engraving import Fonts, Style, engrave_tree, layout_score
from generator.engraving.attributes import staff_attributes
from generator.engraving.notation import ScoreIndex
from generator.model import key_from_fifths, parse_pitch_name, rational, validate_document
from generator.score import load_json, write_json
from generator.pitch import concert_document, shift_key
from tests.test_musicxml import parse, xml_note, xml_score
import generator.engraving.attributes
import generator.engraving.fonts
import generator.engraving.layout
import generator.engraving.notation
import generator.engraving.types
import generator.model
import generator.pitch
import tests.test_musicxml
FONT_ROOT = Path('package/assets/fonts/petaluma')
FONT_ROOT = pathlib.Path('package/assets/fonts/petaluma')
def instrument_score(document, steps, semitones):
source = copy.deepcopy(document)
for measure in source['staves'][0]['measures']:
measure['transpose'] = {'diatonic': steps, 'chromatic': semitones, 'octave_change': 0}
return concert_document(source)
return generator.pitch.concert_document(source)
def key_change(old, new, local=False, attributes=''):
initial = f'<key><fifths>{old}</fifths></key>' + attributes
change = f'<attributes><key><cancel>{old}</cancel><fifths>{new}</fifths></key></attributes>'
if local:
return parse(xml_score(xml_note(duration=2, kind='half') + change + xml_note(duration=2, kind='half'), initial))
root = ElementTree.fromstring(xml_score(xml_note(), initial))
root.find('part').append(ElementTree.fromstring('<measure number="2">' + change + xml_note() + '</measure>'))
return parse(ElementTree.tostring(root, encoding='unicode'))
return tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=2, kind='half') + change + tests.test_musicxml.xml_note(duration=2, kind='half'), initial))
root = xml.etree.ElementTree.fromstring(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(), initial))
root.find('part').append(xml.etree.ElementTree.fromstring('<measure number="2">' + change + tests.test_musicxml.xml_note() + '</measure>'))
return tests.test_musicxml.parse(xml.etree.ElementTree.tostring(root, encoding='unicode'))
class KeyCancellationTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.fonts = Fonts(FONT_ROOT)
cls.fonts = generator.engraving.fonts.Fonts(FONT_ROOT)
def test_sharp_flat_and_cross_sign_changes_keep_their_placement(self):
cases = [(1, 0, 'A', 3, 2, -54), (-2, -1, 'Ab', -4, -3, -36),
(-1, -2, 'G', 1, 0, -48), (1, 2, 'F', -1, 0, -24)]
@@ -44,58 +43,58 @@ class KeyCancellationTests(unittest.TestCase):
source = key_change(old, new, local)
before = copy.deepcopy(source)
result = instrument_score(source, *({'A': (1, 2), 'Ab': (-1, -2), 'G': (1, 2), 'F': (-1, -2)}[target]))
validate_document(result)
generator.model.validate_document(result)
measures = result['staves'][0]['measures']
self.assertEqual(measures[0]['key'], key_from_fifths(expected_old))
self.assertEqual(measures[0]['key'], generator.model.key_from_fifths(expected_old))
change = measures[0]['attributes'][0] if local else measures[1]
self.assertEqual(change['key'], key_from_fifths(expected_new) | {'cancel': expected_old})
self.assertEqual(change['key'], generator.model.key_from_fifths(expected_new) | {'cancel': expected_old})
self.assertEqual(len(measures), 1 if local else 2)
if local:
self.assertEqual(len(measures[0]['attributes']), 1)
self.assertEqual(change['onset'], rational(2))
self.assertEqual(change['onset'], generator.model.rational(2))
else:
self.assertTrue(all(not m['attributes'] for m in measures))
self.assertEqual(change['number'], '2')
time = Fraction(2 if local else 4)
index = ScoreIndex(result)
glyphs = staff_attributes(index, index.state(time), time, self.fonts, 12, 'change', ('key',))
time = fractions.Fraction(2 if local else 4)
index = generator.engraving.notation.ScoreIndex(result)
glyphs = generator.engraving.attributes.staff_attributes(index, index.state(time), time, self.fonts, 12, 'change', ('key',))
naturals = [p for p in glyphs if p.value == 'accidentalNatural']
self.assertEqual(len(naturals), 1)
self.assertEqual(naturals[0].points[0][1], natural_y)
self.assertEqual([p.value for p in glyphs[1:]], ['accidentalSharp' if expected_new > 0 else 'accidentalFlat'] * abs(expected_new))
layout = layout_score(result, Style(), self.fonts)
layout = generator.engraving.layout.layout_score(result, generator.engraving.types.Style(), self.fonts)
reference = 'measure-1:attributes:0' if local else 'measure-2:initial'
drawn = [p for page in layout.pages for p in page.primitives if p.role == 'accidental' and p.sources == (reference,)]
self.assertEqual([p.value for p in drawn], [p.value for p in glyphs])
self.assertEqual([p.points[0][1] - drawn[0].points[0][1] for p in drawn], [p.points[0][1] - glyphs[0].points[0][1] for p in glyphs])
self.assertEqual(source, before)
def test_metadata_and_initial_signature_transpose_cancellation(self):
source = parse(xml_score(xml_note(), '<key><cancel>1</cancel><fifths>0</fifths></key>'))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(), '<key><cancel>1</cancel><fifths>0</fifths></key>'))
result = instrument_score(source, 1, 2)
expected = key_from_fifths(2) | {'cancel': 3}
expected = generator.model.key_from_fifths(2) | {'cancel': 3}
self.assertEqual(result['metadata']['key'], expected)
self.assertEqual(result['staves'][0]['measures'][0]['key'], expected)
def test_zero_and_modal_cancellations_reference_the_old_signature(self):
for mode in ('major', 'minor', 'dorian', 'none'):
with self.subTest(mode=mode):
key = key_from_fifths(-1, mode) | {'cancel': 0}
result = shift_key(key, 1, 2)
key = generator.model.key_from_fifths(-1, mode) | {'cancel': 0}
result = generator.pitch.shift_key(key, 1, 2)
self.assertEqual(result['cancel'], 2)
self.assertEqual(result['mode'], mode)
self.assertEqual(result['alterations'], key_from_fifths(1)['alterations'])
self.assertEqual(result['alterations'], generator.model.key_from_fifths(1)['alterations'])
def test_unchanged_accidentals_do_not_receive_naturals(self):
for old, new, target in [(1, 0, 'F'), (-1, 0, 'G'), (1, 1, 'A')]:
with self.subTest(old=old, new=new, target=target):
result = instrument_score(key_change(old, new), *({'F': (-1, -2), 'G': (1, 2), 'A': (1, 2)}[target]))
index = ScoreIndex(result)
glyphs = staff_attributes(index, index.state(Fraction(4)), Fraction(4), self.fonts, 12, 'change', ('key',))
index = generator.engraving.notation.ScoreIndex(result)
glyphs = generator.engraving.attributes.staff_attributes(index, index.state(fractions.Fraction(4)), fractions.Fraction(4), self.fonts, 12, 'change', ('key',))
self.assertNotIn('accidentalNatural', [p.value for p in glyphs])
def test_unrepresentable_cancellation_fails_without_mutating_source(self):
for fifths, steps, semitones in [(14, 1, 2), (-14, -1, -2), (0, 0, Fraction(1, 2))]:
key = key_from_fifths(0) | {'cancel': fifths}
for fifths, steps, semitones in [(14, 1, 2), (-14, -1, -2), (0, 0, fractions.Fraction(1, 2))]:
key = generator.model.key_from_fifths(0) | {'cancel': fifths}
before = copy.deepcopy(key)
with self.subTest(fifths=fifths, semitones=semitones), self.assertRaisesRegex(ValueError, 'key cancellation.*represented'):
shift_key(key, steps, semitones)
generator.pitch.shift_key(key, steps, semitones)
self.assertEqual(key, before)
for local in (False, True):
source = key_change(0, 0, local)
@@ -111,8 +110,8 @@ class KeyCancellationTests(unittest.TestCase):
for local in (False, True):
with self.subTest(local=local):
source = key_change(1, 0, local, attributes)
result = concert_document(source)
self.assertEqual(result, concert_document(result))
result = generator.pitch.concert_document(source)
self.assertEqual(result, generator.pitch.concert_document(result))
measures = result['staves'][0]['measures']
change = measures[0]['attributes'][0] if local else measures[1]
self.assertEqual(change['key'], key_from_fifths(2) | {'cancel': 3})
self.assertEqual(change['key'], generator.model.key_from_fifths(2) | {'cancel': 3})
tests/test_music_model.py
+80 -80
View File
@@ -1,82 +1,88 @@
import argparse
import copy
from fractions import Fraction
from pathlib import Path
import fractions
import pathlib
import tempfile
import unittest
from unittest.mock import patch
import unittest.mock
from generator import harmony
from generator.__main__ import generate_assets
from generator.arrange.generate import arrange_document
from generator.parts import active_chord, generated_voice, propagated_chords
from generator.guitar import chord_pitches, chord_positions, chord_requirements
from generator.model import beat_lengths, fraction, key_from_fifths, pitch_number, rational, validate_document
from generator.musicxml import load_musicxml
from generator.styles import BUILT_IN_STYLES, canonical_style
from generator.pitch import concert_document, shift_pitch
from tests.test_musicxml import parse, notes, xml_note, xml_score
import generator.__main__
import generator.accompaniment
import generator.arrange.generate
import generator.guitar
import generator.harmony
import generator.model
import generator.musicxml
import generator.parts
import generator.pitch
import tests.rhythm_fixtures
import tests.test_musicxml
class MusicModelTests(unittest.TestCase):
def setUp(self):
engraving = patch('generator.__main__.engrave_tree')
engraving = unittest.mock.patch('generator.engraving.pipeline.engrave_tree')
engraving.start()
self.addCleanup(engraving.stop)
catalog = patch('generator.__main__.build_catalog')
catalog = unittest.mock.patch('generator.catalog.build_catalog')
catalog.start()
self.addCleanup(catalog.stop)
rhythms = unittest.mock.patch('generator.rhythm.load_library', return_value=tests.rhythm_fixtures.library())
rhythms.start()
self.addCleanup(rhythms.stop)
artwork = unittest.mock.patch('generator.pico_artwork.generate_titles')
artwork.start()
self.addCleanup(artwork.stop)
def test_chord_semantics_are_not_voicing_heuristics(self):
cases = {'C': {0, 4, 7}, 'Cdim': {0, 3, 6}, 'Cdim7': {0, 3, 6, 9}, 'Csus2': {0, 2, 7},
'C7b5': {0, 4, 6, 10}, 'G7b9': {7, 11, 2, 5, 8}, 'Fmaj7#11': {5, 9, 0, 4, 11}}
for label, expected in cases.items():
self.assertEqual(set(harmony.pitch_classes(harmony.parse_symbol(label))), expected, label)
self.assertEqual(set(generator.harmony.pitch_classes(generator.harmony.parse_symbol(label))), expected, label)
for label, essentials in [('G7b9', {11, 5, 8}), ('C13', {4, 10, 9})]:
definition = harmony.parse_symbol(label)
pitches = chord_pitches(definition, chord_positions(*chord_requirements(definition))[0])
self.assertTrue(essentials.issubset({pitch_number(p) % 12 for p in pitches}), label)
definition = generator.harmony.parse_symbol(label)
pitches = generator.guitar.chord_pitches(definition, generator.guitar.chord_positions(*generator.guitar.chord_requirements(definition))[0])
self.assertTrue(essentials.issubset({generator.model.pitch_number(p) % 12 for p in pitches}), label)
with self.assertRaises(ValueError):
harmony.parse_symbol('Cunrecognized')
generator.harmony.parse_symbol('Cunrecognized')
def test_transposition_preserves_spelling_octaves_modes(self):
for p in ({'letter': 'b', 'alter': 1, 'octave': 4}, {'letter': 'c', 'alter': -1, 'octave': 4}, {'letter': 'f', 'alter': 2, 'octave': 4}):
p['alter'] = rational(p['alter'])
self.assertEqual(shift_pitch(p, 0, 0), p)
self.assertEqual(pitch_number(shift_pitch(p, 1, 2)), pitch_number(p) + 2)
source = parse(xml_score(xml_note(), attributes='<key><fifths>-3</fifths><mode>minor</mode></key><transpose><diatonic>1</diatonic><chromatic>2</chromatic></transpose>'))
result = concert_document(source)
p['alter'] = generator.model.rational(p['alter'])
self.assertEqual(generator.pitch.shift_pitch(p, 0, 0), p)
self.assertEqual(generator.model.pitch_number(generator.pitch.shift_pitch(p, 1, 2)), generator.model.pitch_number(p) + 2)
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(), attributes='<key><fifths>-3</fifths><mode>minor</mode></key><transpose><diatonic>1</diatonic><chromatic>2</chromatic></transpose>'))
result = generator.pitch.concert_document(source)
self.assertEqual(result['metadata']['key_signature'], 'D minor')
self.assertEqual(result['metadata']['key']['alterations'], key_from_fifths(-1, 'minor')['alterations'])
self.assertEqual(result['metadata']['key']['alterations'], generator.model.key_from_fifths(-1, 'minor')['alterations'])
def test_additive_and_compound_grouping(self):
self.assertEqual(beat_lengths({'groups': [{'beats': [3, 2], 'unit': 8}]}), [Fraction(3, 2), Fraction(1)])
self.assertEqual(beat_lengths({'groups': [{'beats': [6], 'unit': 8}]}), [Fraction(3, 2)] * 2)
self.assertEqual(beat_lengths({'groups': [{'beats': [3], 'unit': 4}]}), [Fraction(1)] * 3)
self.assertEqual(generator.model.beat_lengths({'groups': [{'beats': [3, 2], 'unit': 8}]}), [fractions.Fraction(3, 2), fractions.Fraction(1)])
self.assertEqual(generator.model.beat_lengths({'groups': [{'beats': [6], 'unit': 8}]}), [fractions.Fraction(3, 2)] * 2)
self.assertEqual(generator.model.beat_lengths({'groups': [{'beats': [3], 'unit': 4}]}), [fractions.Fraction(1)] * 3)
def test_generated_rhythm_follows_mid_measure_meter(self):
change = '<attributes><time><beats>6</beats><beat-type>8</beat-type></time></attributes>'
source = parse(xml_score(xml_note(duration=1, kind='quarter') + change + xml_note(duration=3, kind='half', extra='<dot/>')))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=1, kind='quarter') + change + tests.test_musicxml.xml_note(duration=3, kind='half', extra='<dot/>')))
measure = source['staves'][0]['measures'][0]
events = generated_voice(measure, 'slash', None)
self.assertEqual([fraction(n['onset']) for n in events], [0, 1, Fraction(5, 2)])
self.assertEqual([fraction(n['duration']) for n in events], [1, Fraction(3, 2), Fraction(3, 2)])
events = generator.parts.generated_voice(measure, 'slash', None)
self.assertEqual([generator.model.fraction(n['onset']) for n in events], [0, 1, fractions.Fraction(5, 2)])
self.assertEqual([generator.model.fraction(n['duration']) for n in events], [1, fractions.Fraction(3, 2), fractions.Fraction(3, 2)])
def test_harmony_carry_uses_last_event_and_never_future_event(self):
c = harmony.parse_symbol('C7') | {'onset': rational(0)}
f = harmony.parse_symbol('F7') | {'onset': rational(2)}
c = generator.harmony.parse_symbol('C7') | {'onset': generator.model.rational(0)}
f = generator.harmony.parse_symbol('F7') | {'onset': generator.model.rational(2)}
measures = [{'harmonies': [c, f]}, {'harmonies': []}]
self.assertEqual(propagated_chords(measures), [None, f])
self.assertIsNone(active_chord({'harmonies': [f]}, Fraction(0), None))
self.assertEqual(active_chord(measures[0], Fraction(1), None), c)
self.assertEqual(generator.accompaniment.carried_chords(measures), [None, f])
self.assertIsNone(generator.accompaniment.active_chord({'harmonies': [f]}, fractions.Fraction(0), None))
self.assertEqual(generator.accompaniment.active_chord(measures[0], fractions.Fraction(1), None), c)
def test_tuplet_ratios_and_tie_links_validated(self):
document = parse(xml_score(xml_note()))
notes(document)[0]['pitches'][0]['tie_to'] = 'missing'
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
tests.test_musicxml.notes(document)[0]['pitches'][0]['tie_to'] = 'missing'
with self.assertRaisesRegex(ValueError, 'dangling tie'):
validate_document(document)
document = parse(xml_score(xml_note()))
notes(document)[0]['duration'] = {'numerator': 8, 'denominator': 2}
generator.model.validate_document(document)
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
tests.test_musicxml.notes(document)[0]['duration'] = {'numerator': 8, 'denominator': 2}
with self.assertRaisesRegex(ValueError, 'reduced'):
validate_document(document)
generator.model.validate_document(document)
def test_optional_turnaround_and_form(self):
source = load_musicxml(Path('data/songs/it-could-happen-to-you.xml'))
source = generator.musicxml.load_musicxml(pathlib.Path('data/songs/it-could-happen-to-you.xml'))
for part in ('acoustic', 'electric', 'vocal'):
result = arrange_document(source, part)
validate_document(result)
result = generator.arrange.generate.arrange_document(source, part, rhythms=tests.rhythm_fixtures.library())
generator.model.validate_document(result)
measures = result['staves'][0]['measures']
self.assertEqual(sum(bool(m.get('section_label')) for m in measures), 7)
self.assertFalse(measures[-1]['harmonies'])
@@ -87,68 +93,62 @@ class MusicModelTests(unittest.TestCase):
self.assertIn('(Bb7)', labels)
def test_legacy_repeat_model_fields_are_rejected(self):
for field in ('repeats', 'endings', 'initial_attributes'):
source = parse(xml_score(xml_note()))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
source['staves'][0]['measures'][0][field] = {}
with self.assertRaisesRegex(ValueError, 'repeat notation'):
validate_document(source)
generator.model.validate_document(source)
for field in ('sound_ties', 'printed_ties'):
source = parse(xml_score(xml_note()))
notes(source)[0]['pitches'][0][field] = []
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
tests.test_musicxml.notes(source)[0]['pitches'][0][field] = []
with self.assertRaisesRegex(ValueError, 'pass-specific tie'):
validate_document(source)
def test_player_styles_preserved(self):
self.assertEqual(len(BUILT_IN_STYLES), 50)
for text in ('Med.', 'Med. Swing', 'Medium', 'Medium Swing'):
self.assertEqual(canonical_style(text), 'Medium Swing')
with self.assertRaises(ValueError):
canonical_style('Unrecognized style')
generator.model.validate_document(source)
def test_failure_preserves_generated_assets(self):
with tempfile.TemporaryDirectory() as temp:
root = Path(temp)
root = pathlib.Path(temp)
data = root / 'data'
(data / 'songs').mkdir(parents=True)
(data / 'practice').mkdir()
(data / 'songs' / 'bad.xml').write_text(xml_score(xml_note(duration=5)), encoding='utf-8')
(data / 'songs' / 'bad.xml').write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=5)), encoding='utf-8')
output = root / 'output'
output.mkdir()
(output / 'last-good').write_text('preserved')
args = argparse.Namespace(data_dir=data, output_dir=output, static_assets_dir=Path('package/assets'), check=False)
args = argparse.Namespace(data_dir=data, output_dir=output, static_assets_dir=pathlib.Path('package/assets'), check=False)
with self.assertRaises(ValueError):
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertEqual((output / 'last-good').read_text(), 'preserved')
(data / 'songs' / 'bad.xml').write_text(xml_score(xml_note()), encoding='utf-8')
(data / 'songs' / 'bad.xml').write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()), encoding='utf-8')
args.check = True
with self.assertRaisesRegex(ValueError, 'summary'):
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertEqual((output / 'last-good').read_text(), 'preserved')
def test_instrument_transposition_is_converted_once(self):
source = parse(xml_score(xml_note(), attributes='<transpose><diatonic>-1</diatonic><chromatic>-2</chromatic></transpose>'))
concert = concert_document(source)
p = notes(concert)[0]['pitches'][0]
self.assertEqual((p['letter'], fraction(p['alter']), p['octave']), ('b', -1, 3))
self.assertEqual(concert_document(concert), concert)
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(), attributes='<transpose><diatonic>-1</diatonic><chromatic>-2</chromatic></transpose>'))
concert = generator.pitch.concert_document(source)
p = tests.test_musicxml.notes(concert)[0]['pitches'][0]
self.assertEqual((p['letter'], generator.model.fraction(p['alter']), p['octave']), ('b', -1, 3))
self.assertEqual(generator.pitch.concert_document(concert), concert)
self.assertEqual(concert['metadata']['key_signature'], 'Bb')
def test_deterministic_generation_and_removal(self):
with tempfile.TemporaryDirectory() as temp:
data, output = Path(temp) / 'data', Path(temp) / 'output'
data, output = pathlib.Path(temp) / 'data', pathlib.Path(temp) / 'output'
(data / 'songs').mkdir(parents=True)
(data / 'practice').mkdir()
source = data / 'songs' / 'fixture.xml'
source.write_text(xml_score(xml_note()), encoding='utf-8')
args = argparse.Namespace(data_dir=data, output_dir=output, static_assets_dir=Path('package/assets'), check=False)
generate_assets(args)
source.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()), encoding='utf-8')
args = argparse.Namespace(data_dir=data, output_dir=output, static_assets_dir=pathlib.Path('package/assets'), check=False)
generator.__main__.generate_assets(args)
first = {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()}
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertEqual(first, {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()})
source.unlink()
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertFalse(list(output.rglob('*.json')))
self.assertFalse(list(output.rglob('*.png')))
def test_pickup_tie_survives_vocal_restoration(self):
source = xml_score(xml_note(duration=1, kind='quarter', extra='<tie type="start"/>'), implicit=True)
source = source.replace('</measure></part>', '</measure><measure number="2">' + xml_note(extra='<tie type="stop"/>') + '</measure></part>')
result = arrange_document(parse(source), 'vocal')
validate_document(result)
source = tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=1, kind='quarter', extra='<tie type="start"/>'), implicit=True)
source = source.replace('</measure></part>', '</measure><measure number="2">' + tests.test_musicxml.xml_note(extra='<tie type="stop"/>') + '</measure></part>')
result = generator.arrange.generate.arrange_document(tests.test_musicxml.parse(source), 'vocal', rhythms=tests.rhythm_fixtures.library())
generator.model.validate_document(result)
measures = result['staves'][0]['measures']
start = measures[0]['voices'][0]['notes'][-1]['pitches'][0]
end = measures[1]['voices'][0]['notes'][0]['pitches'][0]
tests/test_musicxml.py
+61 -46
View File
@@ -1,10 +1,13 @@
from fractions import Fraction
from pathlib import Path
import fractions
import pathlib
import tempfile
import unittest
import xml.etree.ElementTree
from generator.musicxml import MusicXMLError, load_musicxml
from generator.model import NOTE_TYPES, TYPE_LENGTHS, fraction, pitch_number
import generator.harmony
import generator.model
import generator.musicxml
import generator.musicxml_writer
def xml_score(content, attributes='', *, divisions=1, implicit=False):
return ('<?xml version="1.0"?>'
@@ -22,39 +25,54 @@ def xml_note(step='C', duration=4, kind='whole', extra='', prefix='', alter=0, o
def parse(text):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / 'fixture.xml'
path = pathlib.Path(directory) / 'fixture.xml'
path.write_text(text, encoding='utf-8')
return load_musicxml(path)
return generator.musicxml.load_musicxml(path)
def notes(document, measure=0):
return document['staves'][0]['measures'][measure]['voices'][0]['notes']
class MusicXMLTests(unittest.TestCase):
def test_obsolete_style_hints_are_rejected(self):
for value in ('Medium Swing', 'Med.', 'Bossa Nova', 'Unrecognized style', ''):
with self.subTest(value=value), self.assertRaisesRegex(generator.musicxml.MusicXMLError, "fixture.xml: unsupported Vamp property 'vamp:style'"):
field = f'<identification><miscellaneous><miscellaneous-field name="vamp:style">{value}</miscellaneous-field></miscellaneous></identification>'
parse(xml_score(xml_note()).replace('<part-list>', field + '<part-list>'))
def test_style_free_roundtrip_keeps_barline_notation_and_fonts(self):
content = '<barline location="left"><bar-style>heavy-light</bar-style></barline>' + xml_note() + '<barline location="right"><bar-style>light-heavy</bar-style></barline>'
source = parse(xml_score(content))
self.assertNotIn('style', source['metadata'])
root = generator.musicxml_writer.musicxml_tree(source).getroot()
self.assertEqual(root.findall('identification/miscellaneous/miscellaneous-field'), [])
self.assertEqual(root.find('defaults/music-font').get('font-family'), 'Petaluma')
restored = parse(xml.etree.ElementTree.tostring(root, encoding='unicode'))
self.assertNotIn('style', restored['metadata'])
self.assertEqual(restored['staves'][0]['measures'][0]['barlines'], {'left': 'heavy-light', 'right': 'light-heavy'})
def test_measure_sound_tempo_matches_direction_sound(self):
sound = '<sound tempo="120"/>'
direct = parse(xml_score(sound + xml_note()))
direction = parse(xml_score('<direction>' + sound + '</direction>' + xml_note()))
self.assertEqual(direct, direction)
with self.assertRaises(MusicXMLError):
with self.assertRaises(generator.musicxml.MusicXMLError):
parse(xml_score('<sound tempo="0"/>' + xml_note()))
with self.assertRaises(MusicXMLError):
with self.assertRaises(generator.musicxml.MusicXMLError):
parse(xml_score('<sound dacapo="yes"/>' + xml_note()))
def test_regression_fixture(self):
score = load_musicxml(Path('tests/fixtures/notation.xml'))
score = generator.musicxml.load_musicxml(pathlib.Path('tests/fixtures/notation.xml'))
self.assertEqual(len(score['staves'][0]['measures']), 4)
self.assertEqual(len(score['staves'][0]['octave_shifts']), 1)
def test_catalog_imports(self):
sources = sorted(path for category in ('practice', 'songs') for path in (Path('data') / category).glob('*.xml'))
sources = sorted(path for category in ('practice', 'songs') for path in (pathlib.Path('data') / category).glob('*.xml'))
self.assertTrue(sources)
for path in sources:
with self.subTest(path=path):
self.assertEqual(set(load_musicxml(path)), {'metadata', 'staves'})
self.assertEqual(set(generator.musicxml.load_musicxml(path)), {'metadata', 'staves'})
def test_all_written_values_and_dots(self):
for kind in NOTE_TYPES:
for kind in generator.model.NOTE_TYPES:
for dots in (0, 1, 2, 3):
value = TYPE_LENGTHS[kind] * (2 - Fraction(1, 2**dots))
value = generator.model.TYPE_LENGTHS[kind] * (2 - fractions.Fraction(1, 2**dots))
score = parse(xml_score(xml_note(duration=value.numerator, kind=kind, extra='<dot/>' * dots), divisions=value.denominator, implicit=True))
self.assertEqual(fraction(notes(score)[0]['duration']), value)
self.assertEqual(generator.model.fraction(notes(score)[0]['duration']), value)
self.assertEqual(notes(score)[0]['written'], {'type': kind, 'dots': dots})
def test_divisions_do_not_change_music(self):
a = parse(xml_score(xml_note(duration=4)))
@@ -67,59 +85,57 @@ class MusicXMLTests(unittest.TestCase):
content = '<forward><duration>1</duration><voice>1</voice></forward>' + xml_note(duration=3, kind='half', extra='<dot/>')
events = notes(parse(xml_score(content)))
self.assertEqual([(n['id'], n['kind']) for n in events], [('spacer-1', 'spacer'), ('note-2', 'note')])
self.assertEqual([fraction(n['duration']) for n in events], [1, 3])
self.assertEqual([generator.model.fraction(n['duration']) for n in events], [1, 3])
def test_doctype_allowed_but_entities_rejected(self):
source = xml_score(xml_note())
source = source.replace('<score-partwise', '<!DOCTYPE score-partwise SYSTEM "https://invalid.test/partwise.dtd"><score-partwise', 1)
self.assertEqual(parse(source), parse(xml_score(xml_note())))
malicious = xml_score(xml_note()).replace('<score-partwise', '<!DOCTYPE score-partwise [<!ENTITY secret SYSTEM "file:///etc/passwd">]><score-partwise', 1)
with self.assertRaises(MusicXMLError):
with self.assertRaises(generator.musicxml.MusicXMLError):
parse(malicious)
with self.assertRaises(MusicXMLError):
with self.assertRaises(generator.musicxml.MusicXMLError):
parse('<score-partwise')
def test_harmony_inside_sustained_note_remains_independent(self):
h = '<harmony><root><root-step>F</root-step></root><kind>major</kind><offset>-2</offset></harmony>'
score = parse(xml_score(xml_note() + h))
self.assertEqual(len(notes(score)), 1)
self.assertEqual(fraction(score['staves'][0]['measures'][0]['harmonies'][0]['onset']), 2)
self.assertEqual(generator.model.fraction(score['staves'][0]['measures'][0]['harmonies'][0]['onset']), 2)
def test_partial_chord_ties_and_unequal_tones(self):
source = xml_note(duration=2, kind='half', extra='<tie type="start"/>')
source += xml_note('E', 1, 'quarter', prefix='<chord/>')
source += xml_note(duration=2, kind='half', extra='<tie type="stop"/>')
source += xml_note('G', 2, 'half', prefix='<chord/>')
result = notes(parse(xml_score(source)))
self.assertEqual(fraction(result[0]['pitches'][1]['duration']), 1)
self.assertEqual(generator.model.fraction(result[0]['pitches'][1]['duration']), 1)
self.assertEqual(result[0]['pitches'][0]['tie_to'], result[1]['pitches'][0]['id'])
self.assertNotIn('tie_to', result[0]['pitches'][1])
def test_standard_minor_harmonies_normalize_display_and_preserve_pitches(self):
from generator.harmony import pitch_classes
for text, kind, label, expected in (('-7', 'minor-seventh', 'm7', (0, 3, 7, 10)), ('-7b5', 'half-diminished', 'm7b5', (0, 3, 6, 10)), ('-maj7', 'major-minor', 'mM7', (0, 3, 7, 11)), ('-6', 'minor-sixth', 'm6', (0, 3, 7, 9))):
with self.subTest(text=text):
content = f'<harmony><root><root-step>C</root-step></root><kind text="{text}">{kind}</kind></harmony>'
score = parse(xml_score(content + xml_note()))
chord = score['staves'][0]['measures'][0]['harmonies'][0]
self.assertEqual(chord['symbol'], 'C' + label)
self.assertEqual(pitch_classes(chord), expected)
self.assertEqual(generator.harmony.pitch_classes(chord), expected)
def test_other_harmony_requires_degrees_or_recognized_text(self):
for kind in ('<kind>other</kind>', '<kind text="">other</kind>', '<kind text="unknown">other</kind>'):
with self.subTest(kind=kind), self.assertRaisesRegex(MusicXMLError, r'fixture.xml: part P1, measure 1:'):
with self.subTest(kind=kind), self.assertRaisesRegex(generator.musicxml.MusicXMLError, r'fixture.xml: part P1, measure 1:'):
parse(xml_score('<harmony><root><root-step>C</root-step></root>' + kind + '</harmony>' + xml_note()))
def test_other_harmony_with_explicit_degrees_remains_supported(self):
from generator.harmony import pitch_classes
degrees = ''.join(f'<degree><degree-value>{value}</degree-value><degree-alter>{alter}</degree-alter><degree-type>add</degree-type></degree>' for value, alter in ((1, 0), (3, -1), (5, 0), (7, 0)))
content = '<harmony><root><root-step>C</root-step></root><kind text="-7">other</kind>' + degrees + '</harmony>'
score = parse(xml_score(content + xml_note()))
chord = score['staves'][0]['measures'][0]['harmonies'][0]
self.assertEqual(chord['kind'], 'other')
self.assertEqual(chord['symbol'], 'Cm7')
self.assertEqual(pitch_classes(chord), (0, 3, 7, 10))
with self.assertRaisesRegex(MusicXMLError, 'contradictory harmony text'):
self.assertEqual(generator.harmony.pitch_classes(chord), (0, 3, 7, 10))
with self.assertRaisesRegex(generator.musicxml.MusicXMLError, 'contradictory harmony text'):
parse(xml_score(content.replace('text="-7"', 'text="maj7"') + xml_note()))
def test_grace_cue_and_measured_tremolo(self):
grace = '<note><grace slash="yes"/><pitch><step>D</step><octave>4</octave></pitch><type>16th</type></note>'
regular = xml_note(extra='<notations><ornaments><tremolo type="single">3</tremolo></ornaments></notations>', prefix='<cue/>')
result = notes(parse(xml_score(grace + regular)))
self.assertEqual(fraction(result[0]['duration']), 0)
self.assertEqual(generator.model.fraction(result[0]['duration']), 0)
self.assertTrue(result[0]['grace']['slash'])
self.assertTrue(result[1]['cue'])
self.assertEqual(result[1]['tremolo']['marks'], 3)
@@ -131,14 +147,14 @@ class MusicXMLTests(unittest.TestCase):
extra += '<notations>' + tuplet + '</notations>'
content += xml_note(duration=4, kind='quarter', extra=extra)
result = parse(xml_score(content, divisions=5))
self.assertEqual(sum(fraction(n['duration']) for n in notes(result)), 4)
self.assertTrue(all(fraction(n['duration']) == Fraction(4, 5) for n in notes(result)))
self.assertEqual(sum(generator.model.fraction(n['duration']) for n in notes(result)), 4)
self.assertTrue(all(generator.model.fraction(n['duration']) == fractions.Fraction(4, 5) for n in notes(result)))
self.assertEqual(result['staves'][0]['tuplets'][0]['note_count'], 5)
def test_inconsistent_written_timing_is_rejected(self):
modulation = '<time-modification><actual-notes>3</actual-notes><normal-notes>2</normal-notes></time-modification>'
with self.assertRaisesRegex(MusicXMLError, 'disagrees with written duration'):
with self.assertRaisesRegex(generator.musicxml.MusicXMLError, 'disagrees with written duration'):
parse(xml_score(xml_note(extra=modulation)))
with self.assertRaisesRegex(MusicXMLError, 'disagrees with written duration'):
with self.assertRaisesRegex(generator.musicxml.MusicXMLError, 'disagrees with written duration'):
parse(xml_score(xml_note(kind='half')))
def test_paired_tremolo_timing(self):
modulation = '<time-modification><actual-notes>2</actual-notes><normal-notes>1</normal-notes></time-modification>'
@@ -146,12 +162,12 @@ class MusicXMLTests(unittest.TestCase):
for step, action in [('C', 'start'), ('G', 'stop')])
result = notes(parse(xml_score(content)))
self.assertEqual(result[0]['tremolo']['partner'], result[1]['id'])
self.assertEqual(fraction(result[0]['duration']), 2)
self.assertEqual(generator.model.fraction(result[0]['duration']), 2)
def test_mid_measure_clef_and_key_changes(self):
change = '<attributes><key><fifths>-3</fifths><mode>minor</mode></key><clef><sign>C</sign><line>3</line></clef></attributes>'
result = parse(xml_score(xml_note(duration=2, kind='half') + change + xml_note(duration=2, kind='half')))
attr = result['staves'][0]['measures'][0]['attributes'][0]
self.assertEqual(fraction(attr['onset']), 2)
self.assertEqual(generator.model.fraction(attr['onset']), 2)
self.assertEqual(attr['key']['mode'], 'minor')
self.assertEqual(attr['clef']['sign'], 'C')
def test_optional_marker_contract(self):
@@ -160,7 +176,7 @@ class MusicXMLTests(unittest.TestCase):
score = parse(xml_score(marker + h + xml_note()))
self.assertTrue(score['staves'][0]['measures'][0]['harmonies'][0]['chord_optional'])
for content in (marker + xml_note(), marker + marker + h + xml_note(), marker.replace('optional-chord', 'unknown') + xml_note()):
with self.assertRaisesRegex(MusicXMLError, 'measure 1'):
with self.assertRaisesRegex(generator.musicxml.MusicXMLError, 'measure 1'):
parse(xml_score(content))
def test_repeat_navigation_and_pass_specific_ties_are_rejected(self):
rejected = [
@@ -173,38 +189,37 @@ class MusicXMLTests(unittest.TestCase):
'<sound dacapo="yes"/>', '<sound dalsegno="segno"/>', '<sound tocoda="coda"/>',
]
for content in rejected:
with self.subTest(content=content), self.assertRaisesRegex(MusicXMLError, 'measure 1'):
with self.subTest(content=content), self.assertRaisesRegex(generator.musicxml.MusicXMLError, 'measure 1'):
parse(xml_score(content + xml_note()))
for passes in ('1', ''):
with self.assertRaisesRegex(MusicXMLError, 'pass-specific ties'):
with self.assertRaisesRegex(generator.musicxml.MusicXMLError, 'pass-specific ties'):
parse(xml_score(xml_note(extra=f'<tie type="start" time-only="{passes}"/>')))
def test_rejects_musical_content_outside_contract(self):
extras = ['<notations><slur type="start"/></notations>', '<notations><articulations><accent/></articulations></notations>',
'<voice>2</voice><staff>2</staff>', '<pitch><unpitched/></pitch>',
'<notations><ornaments><trill-mark/></ornaments></notations>']
for extra in extras:
with self.subTest(extra=extra), self.assertRaises(MusicXMLError):
with self.subTest(extra=extra), self.assertRaises(generator.musicxml.MusicXMLError):
parse(xml_score(xml_note(extra=extra)))
for extra in ('<barline location="middle"><repeat direction="backward"/></barline>', '<backup><duration>5</duration></backup>', '<direction><direction-type><dynamics><ff/></dynamics></direction-type></direction>'):
with self.subTest(extra=extra), self.assertRaises(MusicXMLError):
with self.subTest(extra=extra), self.assertRaises(generator.musicxml.MusicXMLError):
parse(xml_score(xml_note() + extra))
self.assertEqual(fraction(notes(parse(xml_score(xml_note(alter='0.5'))))[0]['pitches'][0]['alter']), Fraction(1, 2))
with self.assertRaises(MusicXMLError):
self.assertEqual(generator.model.fraction(notes(parse(xml_score(xml_note(alter='0.5'))))[0]['pitches'][0]['alter']), fractions.Fraction(1, 2))
with self.assertRaises(generator.musicxml.MusicXMLError):
parse(xml_score(xml_note(extra='<tie type="stop"/>')))
with self.assertRaises(MusicXMLError):
with self.assertRaises(generator.musicxml.MusicXMLError):
parse(xml_score(xml_note(duration=5)))
def test_b_sharp_c_flat_preserve_octave(self):
b = notes(parse(xml_score(xml_note('B', alter=1))))[0]['pitches'][0]
c = notes(parse(xml_score(xml_note('C', alter=-1))))[0]['pitches'][0]
self.assertEqual(pitch_number(b), 72)
self.assertEqual(pitch_number(c), 59)
self.assertEqual(generator.model.pitch_number(b), 72)
self.assertEqual(generator.model.pitch_number(c), 59)
def test_musicxml_degree_alterations_are_relative(self):
h = '<harmony><root><root-step>C</root-step></root><kind>minor</kind><degree><degree-value>3</degree-value><degree-alter>1</degree-alter><degree-type>alter</degree-type></degree></harmony>'
from generator.harmony import degrees
score = parse(xml_score(h + xml_note()))
self.assertEqual(degrees(score['staves'][0]['measures'][0]['harmonies'][0])[3], 0)
self.assertEqual(generator.harmony.degrees(score['staves'][0]['measures'][0]['harmonies'][0])[3], 0)
h = h.replace('<kind>minor</kind>', '<kind text="maj7">minor</kind>')
with self.assertRaisesRegex(MusicXMLError, 'contradictory'):
with self.assertRaisesRegex(generator.musicxml.MusicXMLError, 'contradictory'):
parse(xml_score(h + xml_note()))
def test_nested_and_cross_bar_tuplets(self):
modulation = '<time-modification><actual-notes>9</actual-notes><normal-notes>4</normal-notes></time-modification>'
@@ -218,5 +233,5 @@ class MusicXMLTests(unittest.TestCase):
result = parse(text)
spans = result['staves'][0]['tuplets']
self.assertEqual([t['depth'] for t in spans], [0, 1])
self.assertEqual(fraction(notes(result, 1)[0]['duration']), Fraction(4, 9))
self.assertEqual(generator.model.fraction(notes(result, 1)[0]['duration']), fractions.Fraction(4, 9))
self.assertTrue(all(t['end'] == notes(result, 1)[0]['id'] for t in spans))
tests/test_parts.py
+140 -58
View File
@@ -1,114 +1,196 @@
import argparse
import copy
from fractions import Fraction
import fractions
import json
from pathlib import Path
import pathlib
import tempfile
import unittest
from unittest.mock import patch
import unittest.mock
import xml.etree.ElementTree
from generator.__main__ import generate_assets
from generator.arrange.generate import _arrange_document, arrange_document, generate_performance_scores
from generator.model import fraction, rational, validate_document
from generator.musicxml import load_musicxml
from generator.parts import PART_TYPES, generated_voice, prepare_document
from generator.pitch import concert_document
from generator.playback import compile_timeline
from generator.practice.generate import _practice_document, practice_document
from tests.test_musicxml import notes, parse, xml_note, xml_score
import generator.__main__
import generator.accompaniment
import generator.arrange.generate
import generator.model
import generator.musicxml
import generator.parts
import generator.pitch
import generator.playback
import generator.practice.generate
import tests.rhythm_fixtures
import tests.test_musicxml
def xml_harmony(root='C', kind='major', offset=0):
return f'<harmony><root><root-step>{root}</root-step></root><kind>{kind}</kind><offset>{offset}</offset></harmony>'
class PartGenerationTests(unittest.TestCase):
def test_accompaniment_ignores_melody_attacks_and_sustains_to_changes(self):
source = parse(xml_score(xml_harmony() + xml_harmony('F', offset=3) + ''.join(xml_note(duration=1, kind='eighth') for _ in range(8)), divisions=2))
for result in (practice_document(source, 'acoustic'), arrange_document(source, 'electric')):
validate_document(result)
def assert_complete_measure(self, measure):
cursor = fractions.Fraction()
for note in measure['voices'][0]['notes']:
self.assertEqual(generator.model.fraction(note['onset']), cursor)
self.assertGreater(generator.model.fraction(note['duration']), 0)
cursor += generator.model.fraction(note['duration'])
self.assertEqual(cursor, generator.model.fraction(measure['duration']))
def test_dense_melody_keeps_rhythmic_accompaniment_without_sustaining_across_changes(self):
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(xml_harmony() + xml_harmony('F', offset=3) + ''.join(tests.test_musicxml.xml_note(duration=1, kind='eighth') for _ in range(8)), divisions=2))
for result in (generator.practice.generate.practice_document(source, 'acoustic', rhythms=tests.rhythm_fixtures.library()), generator.arrange.generate.arrange_document(source, 'electric', rhythms=tests.rhythm_fixtures.library())):
generator.model.validate_document(result)
for measure in result['staves'][0]['measures']:
events = measure['voices'][0]['notes']
if measure.get('section_label') in ('Rhythm Lead-in', 'Guitar Melody', 'Guitar Solo'):
continue
self.assertEqual([fraction(n['onset']) for n in events], [0, Fraction(3, 2)])
self.assertEqual([fraction(n['duration']) for n in events], [Fraction(3, 2), Fraction(5, 2)])
self.assertTrue(all(n['pitches'] for n in events))
self.assertTrue(compile_timeline(result)['events'])
self.assert_complete_measure(measure)
sounding = [n for n in events if n['pitches']]
self.assertTrue(sounding)
self.assertLess(len(sounding), 8)
for note in sounding:
onset = generator.model.fraction(note['onset'])
if onset < fractions.Fraction(3, 2):
self.assertLessEqual(onset + generator.model.fraction(note['duration']), fractions.Fraction(3, 2))
self.assertLessEqual({generator.model.pitch_number(p) % 12 for p in note['pitches']}, {0, 4, 7} if onset < fractions.Fraction(3, 2) else {0, 5, 9})
self.assertTrue(generator.playback.compile_timeline(result)['events'])
def test_harmony_carry_and_explicit_silence(self):
content = xml_harmony(offset=1) + xml_harmony(kind='none', offset=2) + xml_harmony('F', offset=3) + xml_note()
source = xml_score(content).replace('</measure></part>', '</measure><measure number="2">' + xml_note() + '</measure><measure number="3">' + xml_harmony(kind='none') + xml_note() + '</measure><measure number="4">' + xml_note() + '</measure></part>')
result = practice_document(parse(source), 'acoustic')
self.assertEqual([n['kind'] for n in notes(result)], ['rest', 'note', 'rest', 'note'])
self.assertEqual([fraction(n['onset']) for n in notes(result)], [0, 1, 2, 3])
self.assertEqual([n['kind'] for n in notes(result, 1)], ['note'])
self.assertEqual(fraction(notes(result, 1)[0]['duration']), 4)
self.assertEqual([n['kind'] for n in notes(result, 2)], ['rest'])
self.assertEqual([n['kind'] for n in notes(result, 3)], ['rest'])
content = xml_harmony(offset=1) + xml_harmony(kind='none', offset=2) + xml_harmony('F', offset=3) + tests.test_musicxml.xml_note()
source = tests.test_musicxml.xml_score(content).replace('</measure></part>', '</measure><measure number="2">' + tests.test_musicxml.xml_note() + '</measure><measure number="3">' + xml_harmony(kind='none') + tests.test_musicxml.xml_note() + '</measure><measure number="4">' + tests.test_musicxml.xml_note() + '</measure></part>')
result = generator.practice.generate.practice_document(tests.test_musicxml.parse(source), 'acoustic', rhythms=tests.rhythm_fixtures.library())
for measure in result['staves'][0]['measures']:
self.assert_complete_measure(measure)
sounding = [n for n in tests.test_musicxml.notes(result) if n['pitches']]
for start, end in ((1, 2), (3, 4)):
self.assertTrue(any(start <= generator.model.fraction(n['onset']) < end for n in sounding))
for note in sounding:
onset = generator.model.fraction(note['onset'])
self.assertTrue(1 <= onset < 2 or 3 <= onset < 4)
self.assertLessEqual(onset + generator.model.fraction(note['duration']), 2 if onset < 2 else 4)
carried = [n for n in tests.test_musicxml.notes(result, 1) if n['pitches']]
self.assertTrue(carried)
self.assertTrue(all(generator.model.pitch_number(p) % 12 in {0, 5, 9} for n in carried for p in n['pitches']))
for index in (2, 3):
self.assertTrue(all(n['kind'] == 'rest' and not n['pitches'] for n in tests.test_musicxml.notes(result, index)))
generator.model.validate_document(result)
def test_accompaniment_handles_pickups_meters_and_coincident_markers(self):
for duration, kind, attributes, implicit in ((1, 'quarter', '', True), (3, 'half', '<time><beats>3</beats><beat-type>4</beat-type></time>', False)):
source = parse(xml_score(xml_harmony() + xml_harmony('F') + xml_note(duration=duration, kind=kind, extra='<dot/>' if duration == 3 else ''), attributes, implicit=implicit))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(xml_harmony() + xml_harmony('F') + tests.test_musicxml.xml_note(duration=duration, kind=kind, extra='<dot/>' if duration == 3 else ''), attributes, implicit=implicit))
measure = source['staves'][0]['measures'][0]
events = generated_voice(measure, 'accompaniment', None)
self.assertEqual(len(events), 1)
self.assertEqual(fraction(events[0]['duration']), duration)
self.assertEqual(events[0]['_harmony']['root']['letter'], 'f')
source = parse(xml_score(xml_harmony() + xml_note(duration=1, kind='quarter') + '<attributes><time><beats>6</beats><beat-type>8</beat-type></time></attributes>' + xml_note(duration=3, kind='half', extra='<dot/>')))
self.assertEqual(len(generated_voice(source['staves'][0]['measures'][0], 'accompaniment', None)), 1)
plan = generator.accompaniment.plan_section([measure], tests.rhythm_fixtures.library(), electric=False)
events = generator.accompaniment.realize(measure, plan.acoustic[0], None)
self.assertEqual(sum(generator.model.fraction(n['duration']) for n in events), duration)
sounding = [n for n in events if n['kind'] == 'note']
self.assertTrue(sounding)
self.assertTrue(all(n['_harmony']['root']['letter'] == 'f' for n in sounding))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(xml_harmony() + tests.test_musicxml.xml_note(duration=1, kind='quarter') + '<attributes><time><beats>6</beats><beat-type>8</beat-type></time></attributes>' + tests.test_musicxml.xml_note(duration=3, kind='half', extra='<dot/>')))
with self.assertRaisesRegex(ValueError, 'mid-measure meter'):
generator.practice.generate.practice_document(source, 'acoustic', rhythms=tests.rhythm_fixtures.library())
def test_exercise_006_has_accompaniment_in_every_measure(self):
result = practice_document(load_musicxml(Path('data/practice/0006.xml')), 'acoustic')
result = generator.practice.generate.practice_document(generator.musicxml.load_musicxml(pathlib.Path('data/practice/0006.xml')), 'acoustic', rhythms=tests.rhythm_fixtures.library())
self.assertEqual(len(result['staves'][0]['measures']), 12)
for measure in result['staves'][0]['measures']:
self.assertTrue(any(n['pitches'] for v in measure['voices'] for n in v['notes']), measure['id'])
validate_document(result)
self.assertTrue(compile_timeline(result)['events'])
generator.model.validate_document(result)
timeline = generator.playback.compile_timeline(result)
start = fractions.Fraction()
for measure in result['staves'][0]['measures']:
end = start + generator.model.fraction(measure['duration'])
self.assertTrue(any(generator.model.fraction(e['onset']) < end and generator.model.fraction(e['onset']) + generator.model.fraction(e['duration']) > start for e in timeline['events']), measure['id'])
start = end
def test_prepared_document_and_parts_have_independent_ownership(self):
source = parse(xml_score(xml_harmony() + xml_note()))
source['staves'][0]['octave_shifts'] = [{'id': 'octave', 'start': rational(0), 'end': rational(4), 'octaves': -1}]
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(xml_harmony() + tests.test_musicxml.xml_note()))
source['staves'][0]['octave_shifts'] = [{'id': 'octave', 'start': generator.model.rational(0), 'end': generator.model.rational(4), 'octaves': -1}]
original = copy.deepcopy(source)
for builder, wrapper, practice in ((_practice_document, practice_document, True), (_arrange_document, arrange_document, False)):
prepared = prepare_document(source, practice=practice)
for builder, wrapper, practice in ((generator.practice.generate._practice_document, generator.practice.generate.practice_document, True), (generator.arrange.generate._arrange_document, generator.arrange.generate.arrange_document, False)):
prepared = generator.parts.prepare_document(source, practice=practice)
before = copy.deepcopy(prepared)
parts = [builder(prepared, part) for part in PART_TYPES]
plan = generator.accompaniment.plan_section(prepared['staves'][0]['measures'], tests.rhythm_fixtures.library(), electric=False) if practice else generator.arrange.generate.plan_arrangement(prepared, tests.rhythm_fixtures.library())
plan_before = copy.deepcopy(plan)
parts = [builder(prepared, part, plan) for part in generator.parts.PART_TYPES]
self.assertEqual(prepared, before)
self.assertEqual(parts, [wrapper(source, part) for part in PART_TYPES])
self.assertEqual(plan, plan_before)
self.assertEqual(parts, [wrapper(source, part, rhythms=tests.rhythm_fixtures.library()) for part in generator.parts.PART_TYPES])
vocal = copy.deepcopy(parts[2])
parts[1]['metadata']['title'] = 'changed'
notes(parts[1])[0]['pitches'].clear()
tests.test_musicxml.notes(parts[1])[0]['pitches'].clear()
if parts[1]['staves'][0]['octave_shifts']:
parts[1]['staves'][0]['octave_shifts'][0]['octaves'] = 9
self.assertEqual(parts[2], vocal)
self.assertEqual(prepared, before)
self.assertEqual(source, original)
def test_melody_written_pitch_rhythm_rests_and_cross_bar_ties_are_preserved(self):
content = xml_harmony() + xml_harmony('F', offset=3)
content += tests.test_musicxml.xml_note(duration=2, kind='quarter', alter=1, extra='<tie type="start"/>')
content += tests.test_musicxml.xml_note(duration=6, kind='half', alter=1, extra='<dot/><tie type="stop"/><tie type="start"/>')
root = xml.etree.ElementTree.fromstring(tests.test_musicxml.xml_score(content, divisions=2))
second = tests.test_musicxml.xml_note(duration=4, kind='half', alter=1, extra='<tie type="stop"/>')
second += '<note><rest/><duration>1</duration><type>eighth</type></note>'
second += tests.test_musicxml.xml_note('D', duration=3, kind='quarter', alter=1, extra='<dot/>')
root.find('part').append(xml.etree.ElementTree.fromstring('<measure number="2">' + second + '</measure>'))
source = tests.test_musicxml.parse(xml.etree.ElementTree.tostring(root, encoding='unicode'))
before = copy.deepcopy(source)
expected = [n for m in source['staves'][0]['measures'] for n in m['voices'][0]['notes']]
for builder in (generator.practice.generate.practice_document, generator.arrange.generate.arrange_document):
for part in ('electric', 'vocal'):
with self.subTest(builder=builder.__module__, part=part):
result = builder(source, part, rhythms=tests.rhythm_fixtures.library())
generator.model.validate_document(result)
measures = result['staves'][0]['measures']
starts = [0] if builder is generator.practice.generate.practice_document else [i for i, m in enumerate(measures) if m.get('section_label') == ('Guitar Melody' if part == 'electric' else 'Vocal Melody')]
self.assertTrue(starts)
for start in starts:
notes = [n for m in measures[start:start + 2] for n in m['voices'][0]['notes']]
self.assertEqual(len(notes), len(expected))
for original, generated in zip(expected, notes):
for key in ('onset', 'duration', 'kind', 'written'):
self.assertEqual(generated[key], original[key])
self.assertEqual(len(generated['pitches']), len(original['pitches']))
for pitch, tone in zip(original['pitches'], generated['pitches']):
self.assertEqual(generator.model.pitch_number(tone), generator.model.pitch_number(pitch))
self.assertEqual(tone['written'], pitch['written'])
for key in ('tie_from', 'tie_to'):
self.assertEqual(key in tone, key in pitch)
self.assertEqual(notes[0]['pitches'][0]['tie_to'], notes[1]['pitches'][0]['id'])
self.assertEqual(notes[1]['pitches'][0]['tie_to'], notes[2]['pitches'][0]['id'])
if builder is generator.practice.generate.practice_document:
events = generator.playback.compile_timeline(result)['events']
self.assertEqual([(generator.model.fraction(e['onset']), generator.model.fraction(e['duration']), generator.model.fraction(e['pitch'])) for e in events], [(0, 6, 49), (fractions.Fraction(13, 2), fractions.Fraction(3, 2), 51)])
self.assertEqual(source, before)
def test_song_slug_collisions_fail_before_writes(self):
for names in (('one-two', 'one_two'), ('Song', 'song'), ('!!!', '???')):
with self.subTest(names=names), tempfile.TemporaryDirectory() as directory:
root = Path(directory)
root = pathlib.Path(directory)
source, output = root / 'songs', root / 'output'
source.mkdir()
output.mkdir()
(output / 'sentinel').write_text('preserved')
for name in names:
(source / (name + '.xml')).write_text(xml_score(xml_note()))
(source / (name + '.xml')).write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
with self.assertRaisesRegex(ValueError, 'conflicting output path') as raised:
generate_performance_scores(source, output)
generator.arrange.generate.generate_performance_scores(source, output, rhythms=tests.rhythm_fixtures.library())
for name in names:
self.assertIn(str(source / (name + '.xml')), str(raised.exception))
self.assertEqual([p.name for p in output.iterdir()], ['sentinel'])
def test_pipeline_parses_and_normalizes_each_source_once(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
root = pathlib.Path(directory)
for category, name in (('songs', 'song'), ('practice', '0001')):
(root / 'data' / category).mkdir(parents=True)
(root / 'data' / category / (name + '.xml')).write_text(xml_score(xml_harmony() + xml_note()))
args = argparse.Namespace(data_dir=root / 'data', output_dir=root / 'output', static_assets_dir=Path('package/assets'), check=False)
with patch('generator.arrange.generate.load_musicxml', wraps=load_musicxml) as song_load, patch('generator.practice.generate.load_musicxml', wraps=load_musicxml) as practice_load, patch('generator.parts.concert_document', wraps=concert_document) as normalize, patch('generator.__main__.engrave_tree'), patch('generator.__main__.build_catalog'), patch('generator.__main__.compile_tree'):
generate_assets(args)
song_load.assert_called_once()
practice_load.assert_called_once()
(root / 'data' / category / (name + '.xml')).write_text(tests.test_musicxml.xml_score(xml_harmony() + tests.test_musicxml.xml_note()))
args = argparse.Namespace(data_dir=root / 'data', output_dir=root / 'output', static_assets_dir=pathlib.Path('package/assets'), check=False)
with (
unittest.mock.patch('generator.rhythm.load_library', return_value=tests.rhythm_fixtures.library()) as load_library,
unittest.mock.patch('generator.musicxml.load_musicxml', wraps=generator.musicxml.load_musicxml) as load,
unittest.mock.patch('generator.pitch.concert_document', wraps=generator.pitch.concert_document) as normalize,
unittest.mock.patch('generator.engraving.pipeline.engrave_tree'),
unittest.mock.patch('generator.pico_artwork.generate_titles'),
unittest.mock.patch('generator.catalog.build_catalog'),
unittest.mock.patch('generator.playback.compile_tree'),
):
generator.__main__.generate_assets(args)
load_library.assert_called_once_with(args.data_dir.resolve() / 'rhythms')
self.assertEqual(load.call_args_list, [unittest.mock.call(args.data_dir.resolve() / 'songs/song.xml'), unittest.mock.call(args.data_dir.resolve() / 'practice/0001.xml')])
self.assertEqual(normalize.call_count, 2)
original = {p.relative_to(args.output_dir): p.read_bytes() for p in args.output_dir.rglob('*') if p.is_file()}
(root / 'data/practice/0002.xml').write_text('<broken')
with self.assertRaises(ValueError):
generate_assets(args)
with unittest.mock.patch('generator.rhythm.load_library', return_value=tests.rhythm_fixtures.library()), self.assertRaisesRegex(ValueError, '0002.xml'):
generator.__main__.generate_assets(args)
self.assertEqual(original, {p.relative_to(args.output_dir): p.read_bytes() for p in args.output_dir.rglob('*') if p.is_file()})
self.assertEqual(len(list(args.output_dir.rglob('*.json'))), 6)
self.assertEqual(json.loads((args.output_dir / 'songs/song-vocal.json').read_text())['metadata']['title'], 'Fixture')
tests/test_pico.cpp
+208
View File
@@ -14,6 +14,7 @@
#include <vamp_pico/http_client.hpp>
#include <vamp_pico/input.hpp>
#include <vamp_pico/artwork.hpp>
namespace {
@@ -232,6 +233,123 @@ BOOST_AUTO_TEST_CASE(pico_http_client_has_stable_ownership) {
namespace {
auto title_bytes() -> std::string {
auto bytes = std::string(vamp_protocol::title_image_size, '\0');
bytes.replace(0U, 8U, "VPTI\x28\x01\x40\x00", 8U);
bytes[8] = static_cast<char>(0xa3);
return bytes;
}
auto title_request(char digit = 'a') -> vamp_pico::remote_request {
auto request = vamp_pico::remote_request{vamp_pico::request_kind::TITLE};
request.title_id.fill(digit);
request.title_id.back() = 0;
return request;
}
auto title_response() -> std::string {
return "HTTP/1.1 200 OK\r\nContent-Type: application/octet-stream\r\nContent-Length: " +
std::to_string(vamp_protocol::title_image_size) + "\r\nConnection: close\r\n\r\n" + title_bytes();
}
}
BOOST_AUTO_TEST_CASE(pico_title_identity_parser_is_strict_and_transactional) {
auto state = vamp_pico::remote_display_state{};
auto identifier = std::string(64U, 'a');
BOOST_REQUIRE(vamp_pico::parse_remote_display_body("active=1\ntitle_image=" + identifier + "\npage=2\npages=3\n", state));
BOOST_TEST(std::string{state.title_image.data()} == identifier);
for (auto invalid : {identifier + "a", std::string(64U, 'A'), std::string{"../../title"}, std::string(63U, 'a')}) {
BOOST_CHECK(!vamp_pico::parse_remote_display_body("title_image=" + invalid + "\n", state));
BOOST_TEST(std::string{state.title_image.data()} == identifier);
BOOST_TEST(state.page == 2U);
}
BOOST_REQUIRE(vamp_pico::parse_remote_display_body("active=0\ntitle_image=\n", state));
BOOST_TEST(state.title_image[0] == 0);
}
BOOST_AUTO_TEST_CASE(pico_title_cache_discards_old_selection_and_validates_before_copy) {
auto cache = vamp_pico::title_cache{};
auto first = std::string(64U, 'a');
auto second = std::string(64U, 'b');
auto image = title_bytes();
BOOST_CHECK(cache.select(first));
BOOST_CHECK(!cache.ready());
BOOST_REQUIRE(cache.accept(first, image));
image[8] = 0;
BOOST_TEST(cache.image().at(0U, 0U) == 10U);
BOOST_TEST(cache.image().at(1U, 0U) == 3U);
BOOST_TEST(cache.image().at(296U, 0U) == 0U);
BOOST_CHECK(!cache.select(first));
BOOST_CHECK(cache.ready());
BOOST_CHECK(cache.select(second));
BOOST_CHECK(!cache.ready());
BOOST_CHECK(!cache.accept(first, image));
BOOST_CHECK(!cache.accept(second, image.substr(1U)));
image[4] = 0;
BOOST_CHECK(!cache.accept(second, image));
BOOST_CHECK(cache.accept(second, title_bytes()));
BOOST_CHECK(cache.select(""));
BOOST_CHECK(!cache.ready());
BOOST_CHECK(!cache.accept(second, title_bytes()));
}
BOOST_AUTO_TEST_CASE(pico_title_failure_is_visible_until_success_or_selection_change) {
auto cache = vamp_pico::title_cache{};
auto first = std::string(64U, 'a');
auto second = std::string(64U, 'b');
cache.select(first);
BOOST_CHECK(cache.fail(first));
BOOST_CHECK(cache.failed());
BOOST_CHECK(!cache.ready());
BOOST_CHECK(!cache.select(first));
BOOST_CHECK(cache.failed());
BOOST_REQUIRE(cache.accept(first, title_bytes()));
BOOST_CHECK(!cache.failed());
BOOST_CHECK(cache.ready());
cache.select(second);
BOOST_CHECK(!cache.failed());
BOOST_CHECK(!cache.fail(first));
BOOST_CHECK(cache.fail(second));
cache.select("");
BOOST_CHECK(!cache.failed());
BOOST_CHECK(cache.fail(""));
BOOST_CHECK(cache.failed());
}
BOOST_AUTO_TEST_CASE(pico_page_counters_preserve_catalog_maximum_without_truncation) {
BOOST_TEST(std::string{vamp_pico::counter_text(10000U, 10000U).data()} == "10000/10000");
BOOST_TEST(std::string{vamp_pico::counter_text(1U, 10000U).data()} == "1/10000");
BOOST_TEST(std::string{vamp_pico::counter_text(0U, 10000U).data()}.empty());
BOOST_TEST(std::string{vamp_pico::counter_text(10001U, 10001U).data()}.empty());
}
BOOST_AUTO_TEST_CASE(pico_title_requests_follow_commands_and_do_not_postpone_status) {
auto scheduler = vamp_pico::request_scheduler{};
BOOST_REQUIRE(scheduler.next(0U));
scheduler.complete(10U);
scheduler.want_title(std::string(64U, 'a'));
auto title = scheduler.next(11U);
BOOST_REQUIRE(title);
BOOST_CHECK(title->kind == vamp_pico::request_kind::TITLE);
BOOST_CHECK(!scheduler.next(100U));
BOOST_CHECK(scheduler.enqueue(vamp_protocol::page_command::NEXT) == vamp_pico::enqueue_result::QUEUED);
scheduler.cancel_title();
take_command(scheduler, 100U, vamp_protocol::page_command::NEXT);
scheduler.complete(110U);
scheduler.want_title(std::string(64U, 'b'));
title = scheduler.next(111U);
BOOST_REQUIRE(title);
BOOST_CHECK(title->kind == vamp_pico::request_kind::TITLE);
BOOST_TEST(std::string{title->title_id.data()} == std::string(64U, 'b'));
scheduler.complete(1600U);
auto status = scheduler.next(1600U);
BOOST_REQUIRE(status);
BOOST_CHECK(status->kind == vamp_pico::request_kind::DISPLAY);
}
namespace {
struct fake_network {
std::array<tcp_pcb, 64> pcbs{};
std::size_t allocated{};
@@ -695,3 +813,93 @@ BOOST_FIXTURE_TEST_CASE(pico_http_oversized_response_is_rejected_without_truncat
BOOST_TEST(network.acknowledged == 1025U);
BOOST_TEST(network.aborts == 1U);
}
BOOST_FIXTURE_TEST_CASE(pico_http_title_binary_is_fragmented_bounded_and_owned, fake_lwip_fixture) {
auto client = vamp_pico::http_client{};
auto request = title_request();
BOOST_CHECK(client.start(request, 0U) == vamp_pico::http_start_result::STARTED);
auto &pcb = network.pcbs[0];
BOOST_CHECK(deliver_connected(pcb) == ERR_OK);
BOOST_CHECK(network.written.find("POST /api/remote/title HTTP/1.1\r\n") == 0U);
auto marker = network.written.find("\r\n\r\n");
BOOST_REQUIRE(marker != std::string::npos);
BOOST_TEST(network.written.substr(marker + 4U) == "{\"id\":\"" + std::string(64U, 'a') + "\"}");
BOOST_TEST(network.written.size() - marker - 4U == 73U);
auto response = title_response();
for (auto offset = std::size_t{}; offset < response.size(); offset += 137U) {
BOOST_CHECK(deliver_data(pcb, std::string_view{response}.substr(offset, 137U)) == ERR_OK);
}
BOOST_CHECK(deliver_eof(pcb) == ERR_OK);
auto result = client.take_result();
BOOST_REQUIRE(result);
BOOST_REQUIRE(result->success);
BOOST_CHECK(result->request.kind == vamp_pico::request_kind::TITLE);
BOOST_CHECK(client.title_image() == title_bytes());
BOOST_TEST(network.acknowledged == response.size());
client.start({}, 1U);
BOOST_CHECK(client.title_image().empty());
}
BOOST_FIXTURE_TEST_CASE(pico_http_title_rejects_incorrect_binary_framing, fake_lwip_fixture) {
auto client = vamp_pico::http_client{};
auto valid = title_response();
auto extra = valid;
extra.insert(extra.find("\r\n\r\n"), "\r\nContent-Length: 9480");
auto chunked = valid;
chunked.insert(chunked.find("\r\n\r\n"), "\r\nTransfer-Encoding: chunked");
auto compressed = valid;
compressed.insert(compressed.find("\r\n\r\n"), "\r\nContent-Encoding: gzip");
auto wrong_type = valid;
wrong_type.replace(wrong_type.find("application/octet-stream"), 24U, "text/plain");
auto wrong_length = valid;
wrong_length.replace(wrong_length.find("9480"), 4U, "9479");
auto wrong_image = valid;
wrong_image[wrong_image.find("VPTI") + 4U] = 0;
for (auto const &response : {valid.substr(0U, valid.size() - 1U), valid + "x", extra, chunked, compressed, wrong_type, wrong_length, wrong_image}) {
BOOST_CHECK(client.start(title_request(), 0U) == vamp_pico::http_start_result::STARTED);
auto &pcb = network.pcbs[network.allocated - 1U];
BOOST_CHECK(deliver_connected(pcb) == ERR_OK);
BOOST_CHECK(deliver_data(pcb, response) == ERR_OK);
deliver_eof(pcb);
require_failure(client, 1U);
BOOST_CHECK(client.title_image().empty());
}
}
BOOST_FIXTURE_TEST_CASE(pico_http_title_timeout_overflow_and_bad_identifier_are_failures, fake_lwip_fixture) {
auto client = vamp_pico::http_client{};
BOOST_CHECK(client.start(title_request('z'), 0U) == vamp_pico::http_start_result::STARTED);
require_failure(client, 0U);
BOOST_TEST(network.allocated == 0U);
client.start(title_request(), 100U);
BOOST_CHECK(client.poll(1599U) == vamp_pico::http_status::IN_FLIGHT);
require_failure(client, 1600U);
client.start(title_request(), 1601U);
auto &pcb = network.pcbs[1];
deliver_connected(pcb);
BOOST_CHECK(deliver_data(pcb, std::string(vamp_protocol::title_image_size + 1025U, 'x')) == ERR_ABRT);
require_failure(client, 1602U);
}
BOOST_FIXTURE_TEST_CASE(pico_http_title_preemption_detaches_late_callbacks, fake_lwip_fixture) {
auto client = vamp_pico::http_client{};
auto scheduler = vamp_pico::request_scheduler{};
scheduler.want_title(std::string(64U, 'a'));
auto request = scheduler.next(0U);
BOOST_REQUIRE(request);
client.start(*request, 0U);
auto &old = network.pcbs[0];
deliver_connected(old);
deliver_data(old, std::string_view{title_response()}.substr(0U, 200U));
scheduler.enqueue(vamp_protocol::page_command::SELECT);
client.cancel();
scheduler.cancel_title();
request = scheduler.next(1U);
BOOST_REQUIRE(request);
BOOST_CHECK(request->kind == vamp_pico::request_kind::COMMAND);
client.start(*request, 1U);
deliver_data(old, title_response());
deliver_eof(old);
BOOST_CHECK(client.poll(2U) == vamp_pico::http_status::IN_FLIGHT);
BOOST_CHECK(client.title_image().empty());
}
tests/test_playback.py
+54 -57
View File
@@ -1,99 +1,98 @@
from fractions import Fraction
import fractions
import json
from pathlib import Path
import pathlib
import tempfile
import unittest
import wave
from generator.model import fraction
from generator.playback import PlaybackUnavailable, compile_timeline, generate_tone, validate_assets, write_pcm
from tests.test_musicxml import notes, parse, xml_note, xml_score
import generator.model
import generator.playback
import generator.score
import tests.test_musicxml
class PlaybackTests(unittest.TestCase):
def test_grace_steals_accumulate_at_both_ends_of_a_donor(self):
grace = '<note><grace {}/><pitch><step>{}</step><octave>4</octave></pitch><type>eighth</type></note>'
content = grace.format('steal-time-following="25"', 'D') + xml_note(duration=2, kind='half') + grace.format('steal-time-previous="25"', 'E') + xml_note('F', 2, 'half')
result = compile_timeline(parse(xml_score(content)))
self.assertEqual([(fraction(e['pitch']), fraction(e['onset']), fraction(e['duration'])) for e in result['events']], [(62, 0, Fraction(1, 2)), (60, Fraction(1, 2), 1), (64, Fraction(3, 2), Fraction(1, 2)), (65, 2, 2)])
content = grace.format('steal-time-following="25"', 'D') + tests.test_musicxml.xml_note(duration=2, kind='half') + grace.format('steal-time-previous="25"', 'E') + tests.test_musicxml.xml_note('F', 2, 'half')
result = generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(content)))
self.assertEqual([(generator.model.fraction(e['pitch']), generator.model.fraction(e['onset']), generator.model.fraction(e['duration'])) for e in result['events']], [(62, 0, fractions.Fraction(1, 2)), (60, fractions.Fraction(1, 2), 1), (64, fractions.Fraction(3, 2), fractions.Fraction(1, 2)), (65, 2, 2)])
for percent in (50, 60):
with self.subTest(percent=percent), self.assertRaisesRegex(PlaybackUnavailable, 'grace timing consumes'):
compile_timeline(parse(xml_score(content.replace('"25"', f'"{percent}"'))))
chord = content.replace(xml_note(duration=2, kind='half'), xml_note(duration=2, kind='half') + xml_note('G', 1, 'quarter', prefix='<chord/>'))
with self.assertRaisesRegex(PlaybackUnavailable, 'chord tone'):
compile_timeline(parse(xml_score(chord)))
with self.subTest(percent=percent), self.assertRaisesRegex(generator.playback.PlaybackUnavailable, 'grace timing consumes'):
generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(content.replace('"25"', f'"{percent}"'))))
chord = content.replace(tests.test_musicxml.xml_note(duration=2, kind='half'), tests.test_musicxml.xml_note(duration=2, kind='half') + tests.test_musicxml.xml_note('G', 1, 'quarter', prefix='<chord/>'))
with self.assertRaisesRegex(generator.playback.PlaybackUnavailable, 'chord tone'):
generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(chord)))
def test_grace_previous_and_make_time_preserve_their_timing(self):
for mode, expected, total in [('steal-time-previous="25"', [(0, Fraction(3, 2)), (Fraction(3, 2), Fraction(1, 2)), (2, 2)], 4), ('make-time="1"', [(0, 2), (2, 1), (3, 2)], 5)]:
for mode, expected, total in [('steal-time-previous="25"', [(0, fractions.Fraction(3, 2)), (fractions.Fraction(3, 2), fractions.Fraction(1, 2)), (2, 2)], 4), ('make-time="1"', [(0, 2), (2, 1), (3, 2)], 5)]:
grace = f'<note><grace {mode}/><pitch><step>D</step><octave>4</octave></pitch><type>eighth</type></note>'
result = compile_timeline(parse(xml_score(xml_note(duration=2, kind='half') + grace + xml_note('E', 2, 'half'))))
self.assertEqual([(fraction(e['onset']), fraction(e['duration'])) for e in result['events']], expected)
self.assertEqual(fraction(result['duration']), total)
result = generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=2, kind='half') + grace + tests.test_musicxml.xml_note('E', 2, 'half'))))
self.assertEqual([(generator.model.fraction(e['onset']), generator.model.fraction(e['duration'])) for e in result['events']], expected)
self.assertEqual(generator.model.fraction(result['duration']), total)
def test_chords_and_trailing_rest(self):
document = parse(xml_score(xml_note(duration=2, kind='half') + xml_note('E', 1, 'quarter', prefix='<chord/>') + '<note><rest/><duration>2</duration><type>half</type></note>'))
result = compile_timeline(document)
self.assertEqual(fraction(result['duration']), 4)
self.assertEqual([fraction(e['duration']) for e in result['events']], [2, 1])
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=2, kind='half') + tests.test_musicxml.xml_note('E', 1, 'quarter', prefix='<chord/>') + '<note><rest/><duration>2</duration><type>half</type></note>'))
result = generator.playback.compile_timeline(document)
self.assertEqual(generator.model.fraction(result['duration']), 4)
self.assertEqual([generator.model.fraction(e['duration']) for e in result['events']], [2, 1])
def test_tie_is_one_attack(self):
result = compile_timeline(parse(xml_score(xml_note(duration=2, kind='half', extra='<tie type="start"/>') + xml_note(duration=2, kind='half', extra='<tie type="stop"/>'))))
result = generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=2, kind='half', extra='<tie type="start"/>') + tests.test_musicxml.xml_note(duration=2, kind='half', extra='<tie type="stop"/>'))))
self.assertEqual(len(result['events']), 1)
self.assertEqual(fraction(result['events'][0]['duration']), 4)
self.assertEqual(generator.model.fraction(result['events'][0]['duration']), 4)
def test_octave_and_microtones(self):
result = compile_timeline(parse(xml_score(xml_note(alter='0.5'), '<transpose><diatonic>0</diatonic><chromatic>0</chromatic><octave-change>-1</octave-change></transpose>')))
self.assertEqual(fraction(result['events'][0]['pitch']), Fraction(97, 2))
result = generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(alter='0.5'), '<transpose><diatonic>0</diatonic><chromatic>0</chromatic><octave-change>-1</octave-change></transpose>')))
self.assertEqual(generator.model.fraction(result['events'][0]['pitch']), fractions.Fraction(97, 2))
def test_grace_default_and_cues(self):
grace = '<note><grace/><pitch><step>D</step><octave>4</octave></pitch><type>eighth</type></note>'
result = compile_timeline(parse(xml_score(grace + xml_note())))
self.assertEqual([fraction(e['duration']) for e in result['events']], [1, 3])
self.assertEqual([fraction(e['onset']) for e in result['events']], [0, 1])
self.assertEqual(compile_timeline(parse(xml_score(xml_note(prefix='<cue/>'))))['events'], [])
result = generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(grace + tests.test_musicxml.xml_note())))
self.assertEqual([generator.model.fraction(e['duration']) for e in result['events']], [1, 3])
self.assertEqual([generator.model.fraction(e['onset']) for e in result['events']], [0, 1])
self.assertEqual(generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(prefix='<cue/>'))))['events'], [])
def test_tremolo(self):
result = compile_timeline(parse(xml_score(xml_note(extra='<notations><ornaments><tremolo>3</tremolo></ornaments></notations>'))))
result = generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(extra='<notations><ornaments><tremolo>3</tremolo></ornaments></notations>'))))
self.assertEqual(len(result['events']), 32)
extra = '<time-modification><actual-notes>3</actual-notes><normal-notes>2</normal-notes></time-modification><notations><ornaments><tremolo>1</tremolo></ornaments></notations>'
result = compile_timeline(parse(xml_score(xml_note(duration=1, kind='eighth', extra=extra), divisions=3, implicit=True)))
self.assertEqual([fraction(e['duration']) for e in result['events']], [Fraction(1, 6)] * 2)
result = generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=1, kind='eighth', extra=extra), divisions=3, implicit=True)))
self.assertEqual([generator.model.fraction(e['duration']) for e in result['events']], [fractions.Fraction(1, 6)] * 2)
def test_hidden_pitches_sound_and_unpitched_is_unavailable(self):
hidden = xml_note().replace('<note>', '<note print-object="no">')
self.assertEqual(len(compile_timeline(parse(xml_score(hidden)))['events']), 1)
hidden = tests.test_musicxml.xml_note().replace('<note>', '<note print-object="no">')
self.assertEqual(len(generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(hidden)))['events']), 1)
unpitched = '<note><unpitched><display-step>C</display-step><display-octave>4</display-octave></unpitched><duration>4</duration><type>whole</type></note>'
with self.assertRaises(PlaybackUnavailable):
compile_timeline(parse(xml_score(unpitched)))
with self.assertRaises(generator.playback.PlaybackUnavailable):
generator.playback.compile_timeline(tests.test_musicxml.parse(tests.test_musicxml.xml_score(unpitched)))
def test_sounding_pitch_depends_on_instrument_transposition_not_part_or_path(self):
from generator.playback import compile_tree
from generator.score import write_json
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
source = parse(xml_score(xml_note()))
root = pathlib.Path(directory)
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
for part in ('acoustic', 'electric', 'vocal'):
source['staves'][0]['arrangement_part_type'] = part
source['staves'][0]['measures'][0]['transpose']['octave_change'] = -1 if part != 'vocal' else 0
write_json(root / f'{part}.json', source)
compile_tree(root)
generator.score.write_json(root / f'{part}.json', source)
generator.playback.compile_tree(root)
for part, pitch in [('acoustic', 48), ('electric', 48), ('vocal', 60)]:
playback = json.loads((root / f'{part}.json').read_text())['playback']
self.assertEqual(playback['samples'][0]['id'], generate_tone(root, Fraction(pitch))['id'])
self.assertEqual(playback['samples'][0]['id'], generator.playback.generate_tone(root, fractions.Fraction(pitch))['id'])
def test_audio_assets_validate_and_reject_corruption(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
root = pathlib.Path(directory)
generate_fixture(root)
manifest = json.loads((root / 'score.json').read_text())['playback']
validate_assets(manifest, root)
generator.playback.validate_assets(manifest, root)
sample = manifest['samples'][0]
path = root / sample['path']
path.write_bytes(path.read_bytes()[:-1])
with self.assertRaises(ValueError):
validate_assets(manifest, root)
generator.playback.validate_assets(manifest, root)
generate_fixture(root)
manifest['release']['sha256'] = write_pcm(root / 'audio/release.wav', [0.5] * 2400)
manifest['release']['sha256'] = generator.playback.write_pcm(root / 'audio/release.wav', [0.5] * 2400)
with self.assertRaisesRegex(ValueError, 'envelope'):
validate_assets(manifest, root)
generator.playback.validate_assets(manifest, root)
sample['path'] = '../outside.wav'
with self.assertRaises(ValueError):
validate_assets(manifest, root)
generator.playback.validate_assets(manifest, root)
def test_generated_tone_is_deterministic_and_loopable(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
a = generate_tone(root, Fraction(60))
b = generate_tone(root, Fraction(60))
root = pathlib.Path(directory)
a = generator.playback.generate_tone(root, fractions.Fraction(60))
b = generator.playback.generate_tone(root, fractions.Fraction(60))
self.assertEqual(a, b)
with wave.open(str(root / a['path'])) as audio:
self.assertEqual(audio.getframerate(), 48000)
@@ -102,11 +101,9 @@ class PlaybackTests(unittest.TestCase):
self.assertGreater(a['loop_end'], a['loop_start'])
def generate_fixture(directory):
from generator.score import write_json
from generator.playback import compile_tree
root = Path(directory)
write_json(root / 'score.json', parse(xml_score(xml_note(duration=2, kind='half') + xml_note('E', 2, 'half', prefix='<chord/>') + '<note><rest/><duration>2</duration><type>half</type></note>')))
compile_tree(root)
root = pathlib.Path(directory)
generator.score.write_json(root / 'score.json', tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=2, kind='half') + tests.test_musicxml.xml_note('E', 2, 'half', prefix='<chord/>') + '<note><rest/><duration>2</duration><type>half</type></note>')))
generator.playback.compile_tree(root)
if __name__ == '__main__':
import argparse
tests/test_portal_artwork.cpp
+138
View File
@@ -0,0 +1,138 @@
#include <boost/test/unit_test.hpp>
#include <array>
#include <chrono>
#include <filesystem>
#include <fstream>
#include <future>
#include <memory>
#include <string>
#include <boost/asio/co_spawn.hpp>
#include <boost/asio/io_context.hpp>
#include <boost/asio/use_future.hpp>
#include <boost/json/object.hpp>
#include <openssl/sha.h>
#include <vamp_pico/http_client.hpp>
#include <vamp_zero/portal.hpp>
namespace {
struct artwork_fixture {
std::filesystem::path root{std::filesystem::temp_directory_path() / ("vamp-title-test-" + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count()))};
std::string bytes = std::string(vamp_protocol::title_image_size, static_cast<std::char_t>(0xa3));
std::string identifier{};
artwork_fixture() {
bytes.replace(0U, 8U, "VPTI\x28\x01\x40\x00", 8U);
auto digest = std::array<std::uchar_t, SHA256_DIGEST_LENGTH>{};
SHA256(reinterpret_cast<std::uchar_t const *>(bytes.data()), bytes.size(), digest.data());
for (auto byte : digest) {
identifier.push_back("0123456789abcdef"[byte >> 4U]);
identifier.push_back("0123456789abcdef"[byte & 15U]);
}
std::filesystem::create_directories(root / "pico/titles");
write(bytes);
}
~artwork_fixture() {
auto error = std::error_code{};
std::filesystem::remove_all(root, error);
}
auto path() const -> std::filesystem::path {
return root / "pico/titles" / (identifier + ".mask");
}
auto write(std::string const &value) -> void {
auto stream = std::ofstream{path(), std::ios::binary};
stream.write(value.data(), static_cast<std::streamsize>(value.size()));
}
auto config() const -> vamp_zero::config {
auto value = vamp_zero::config{};
value.runtime.assets_path = root.string();
return value;
}
auto state() const -> vamp_zero::app_state {
auto catalog = std::make_shared<vamp_zero::catalog>();
catalog->performance.parts.push_back({.id = "part", .title = "Title", .part = "Acoustic", .total_pages = 10000U, .title_image = identifier});
auto value = vamp_zero::app_state{};
value.catalog = catalog;
value.mode = vamp_zero::app_mode::PERFORMANCE_SCORE;
value.active_part_id = "part";
value.page_index = 9999U;
return value;
}
};
auto call(ligo::service_router const &router, std::string method, std::string route, boost::json::value payload = {})
-> std::expected<ligo::service_response, modulus::error> {
auto io = boost::asio::io_context{};
auto future = boost::asio::co_spawn(io, router.call_response({.method = std::move(method), .route = std::move(route)}, std::move(payload)), boost::asio::use_future);
io.run();
return future.get();
}
}
BOOST_FIXTURE_TEST_CASE(portal_title_route_serves_registered_binary_and_remote_identity, artwork_fixture) {
auto control = vamp_zero::control_state{config(), state()};
auto router = vamp_zero::make_router(control);
auto status = call(router, "GET", vamp_protocol::remote_display_path);
BOOST_REQUIRE(status);
auto remote = vamp_pico::remote_display_state{};
BOOST_REQUIRE(vamp_pico::parse_remote_display_body(status->body, remote));
BOOST_TEST(std::string{remote.title_image.data()} == identifier);
BOOST_TEST(remote.page == 10000U);
BOOST_TEST(remote.pages == 10000U);
auto response = call(router, "POST", vamp_protocol::remote_title_path, boost::json::object{{"id", identifier}});
BOOST_REQUIRE(response);
BOOST_TEST(response->content_type == "application/octet-stream");
BOOST_TEST(response->body == bytes);
auto validated = vamp_zero::validated_title_path(root.string(), identifier);
BOOST_REQUIRE(validated);
BOOST_TEST(*validated == path().string());
BOOST_CHECK(!call(router, "GET", vamp_protocol::remote_title_path, boost::json::object{{"id", identifier}}));
}
BOOST_FIXTURE_TEST_CASE(portal_title_route_rejects_arbitrary_unregistered_and_corrupt_assets, artwork_fixture) {
auto control = vamp_zero::control_state{config(), state()};
auto router = vamp_zero::make_router(control);
for (auto const &id : {std::string{"../title"}, std::string(64U, 'f'), std::string(64U, 'A'), identifier + "a"}) {
BOOST_CHECK(!call(router, "POST", vamp_protocol::remote_title_path, boost::json::object{{"id", id}}));
}
BOOST_CHECK(!call(router, "POST", vamp_protocol::remote_title_path, boost::json::object{{"id", identifier}, {"path", path().string()}}));
BOOST_CHECK(!call(router, "POST", vamp_protocol::remote_title_path, boost::json::object{{"id", 1}}));
BOOST_CHECK(!call(router, "POST", vamp_protocol::remote_title_path));
auto empty = vamp_zero::control_state{config(), {}};
BOOST_CHECK(!call(vamp_zero::make_router(empty), "POST", vamp_protocol::remote_title_path, boost::json::object{{"id", identifier}}));
for (auto const &corrupt : {bytes.substr(1U), bytes + "x", std::string(bytes.size(), 'x')}) {
write(corrupt);
BOOST_CHECK(!vamp_zero::validated_title_path(root.string(), identifier));
BOOST_CHECK(!call(router, "POST", vamp_protocol::remote_title_path, boost::json::object{{"id", identifier}}));
}
auto modified = bytes;
modified.back() ^= 1;
write(modified);
BOOST_CHECK(!vamp_zero::validated_title_path(root.string(), identifier));
BOOST_CHECK(!call(router, "POST", vamp_protocol::remote_title_path, boost::json::object{{"id", identifier}}));
}
BOOST_FIXTURE_TEST_CASE(portal_title_validation_rejects_symlink_outside_title_directory, artwork_fixture) {
auto other = root / "other.mask";
std::filesystem::rename(path(), other);
std::filesystem::create_symlink(other, path());
BOOST_CHECK(!vamp_zero::validated_title_path(root.string(), identifier));
}
BOOST_FIXTURE_TEST_CASE(portal_uses_and_serves_only_the_packaged_petaluma_font, artwork_fixture) {
std::filesystem::create_directories(root / "fonts/petaluma");
std::filesystem::copy_file(std::filesystem::path{VAMP_TEST_SOURCE_DIR} / "package/assets/fonts/petaluma/PetalumaScript.otf", root / "fonts/petaluma/PetalumaScript.otf");
auto control = vamp_zero::control_state{config(), {}};
auto router = vamp_zero::make_router(control);
auto html = call(router, "GET", "/");
BOOST_REQUIRE(html);
BOOST_CHECK(html->body.find("font-family:PetalumaScript") != std::string::npos);
auto font = call(router, "GET", "/assets/PetalumaScript.otf");
BOOST_REQUIRE(font);
BOOST_TEST(font->content_type == "font/otf");
BOOST_TEST(font->body.substr(0U, 4U) == "OTTO");
BOOST_CHECK(!call(router, "GET", "/assets/other.otf"));
}
tests/test_publish_package.py
+31 -31
View File
@@ -1,82 +1,82 @@
import hashlib
import pathlib
import subprocess
import sys
import tarfile
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
import unittest.mock
from package.scripts import publish_latest_generic_package
import package.scripts.publish_latest_generic_package
class PackagePublishingTests(unittest.TestCase):
def test_configured_packager_matches_publisher(self):
root = Path(__file__).resolve().parents[1]
root = pathlib.Path(__file__).resolve().parents[1]
runtime = root / 'build/build_common/modulus'
if not (runtime / 'modulus/cli.py').is_file():
self.skipTest('Configure first to check the exported Modulus packager')
with patch.object(sys, 'path', [str(runtime), *sys.path]):
with unittest.mock.patch.object(sys, 'path', [str(runtime), *sys.path]):
import modulus.cli
import modulus.config
with tempfile.TemporaryDirectory() as temporary:
directory = Path(temporary)
directory = pathlib.Path(temporary)
context = modulus.cli.build_context(directory, modulus.config.load_repo_config(root))
name, registry, targets = publish_latest_generic_package.read_config(root / 'modulus.json')
name, registry, targets = package.scripts.publish_latest_generic_package.read_config(root / 'modulus.json')
def fake_build(context, command, **kwargs):
if command[:2] == ['cmake', '--install']:
stage = Path(command[2]).parent / 'stage'
stage = pathlib.Path(command[2]).parent / 'stage'
(stage / 'manifest.json').write_text('{}\n')
return 0
with patch.object(modulus.cli, 'in_docker', return_value=True), \
patch.object(modulus.cli, 'run', side_effect=fake_build), \
patch.object(modulus.cli, 'run_capture', return_value=subprocess.CompletedProcess([], 0, stdout='fixture\n', stderr='')), \
patch.object(modulus.cli, 'upload_generic_artifact') as upload:
with unittest.mock.patch.object(modulus.cli, 'in_docker', return_value=True), \
unittest.mock.patch.object(modulus.cli, 'run', side_effect=fake_build), \
unittest.mock.patch.object(modulus.cli, 'run_capture', return_value=subprocess.CompletedProcess([], 0, stdout='fixture\n', stderr='')), \
unittest.mock.patch.object(modulus.cli, 'upload_generic_artifact') as upload:
self.assertEqual(modulus.cli.package_artifacts(context, ['--no-remote']), 0)
upload.assert_not_called()
try:
artifacts = publish_latest_generic_package.discover_artifacts(directory / 'build/packages', name, targets)
artifacts = package.scripts.publish_latest_generic_package.discover_artifacts(directory / 'build/packages', name, targets)
except SystemExit as error:
self.fail(str(error))
for archive, checksum in zip(artifacts[::2], artifacts[1::2]):
self.assertEqual(checksum.read_text(), f'{hashlib.sha256(archive.read_bytes()).hexdigest()} {archive.name}\n')
with tarfile.open(archive) as package:
self.assertIn('manifest.json', package.getnames())
with tarfile.open(archive) as contents:
self.assertIn('manifest.json', contents.getnames())
def test_only_configured_target_archives_and_checksums_are_uploaded(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
root = pathlib.Path(temporary)
archive = root / 'product-vamp-arm_64_gcc12_release.tar.gz'
checksum = archive.with_name(archive.name + '.sha256')
for path in (archive, checksum, root / 'product-vamp-obsolete-arm_64_gcc12_release.tar.gz'):
path.write_bytes(b'fixture')
self.assertEqual(publish_latest_generic_package.discover_artifacts(root, 'product-vamp', ['arm_64_gcc12_release']), [archive, checksum])
self.assertEqual(package.scripts.publish_latest_generic_package.discover_artifacts(root, 'product-vamp', ['arm_64_gcc12_release']), [archive, checksum])
checksum.unlink()
with self.assertRaisesRegex(SystemExit, 'missing artifact files'):
publish_latest_generic_package.discover_artifacts(root, 'product-vamp', ['arm_64_gcc12_release'])
package.scripts.publish_latest_generic_package.discover_artifacts(root, 'product-vamp', ['arm_64_gcc12_release'])
archive.unlink()
with self.assertRaisesRegex(SystemExit, 'missing artifact files'):
publish_latest_generic_package.discover_artifacts(root, 'product-vamp', ['arm_64_gcc12_release'])
package.scripts.publish_latest_generic_package.discover_artifacts(root, 'product-vamp', ['arm_64_gcc12_release'])
def test_mismatched_filenames_report_actual_artifacts_without_uploading(self):
with tempfile.TemporaryDirectory() as temporary, patch.object(publish_latest_generic_package, 'request') as request:
archive = Path(temporary) / 'product-vamp-abc123-arm_64_gcc12_release.tar.gz'
with tempfile.TemporaryDirectory() as temporary, unittest.mock.patch.object(package.scripts.publish_latest_generic_package, 'request') as request:
archive = pathlib.Path(temporary) / 'product-vamp-abc123-arm_64_gcc12_release.tar.gz'
archive.write_bytes(b'fixture')
with self.assertRaises(SystemExit) as error:
publish_latest_generic_package.main(['--dist-dir', temporary, '--user', 'user', '--token', 'test-token'])
package.scripts.publish_latest_generic_package.main(['--dist-dir', temporary, '--user', 'user', '--token', 'test-token'])
self.assertIn(f'Found: {archive.name}', str(error.exception))
self.assertIn('Publish the current Modulus tooling', str(error.exception))
request.assert_not_called()
def test_missing_artifacts_do_not_touch_the_registry(self):
with tempfile.TemporaryDirectory() as temporary, patch.object(publish_latest_generic_package, 'request') as request:
with tempfile.TemporaryDirectory() as temporary, unittest.mock.patch.object(package.scripts.publish_latest_generic_package, 'request') as request:
with self.assertRaisesRegex(SystemExit, 'missing artifact files'):
publish_latest_generic_package.main(['--dist-dir', temporary, '--user', 'user', '--token', 'test-token'])
package.scripts.publish_latest_generic_package.main(['--dist-dir', temporary, '--user', 'user', '--token', 'test-token'])
request.assert_not_called()
def test_publication_replaces_the_single_registry_slot(self):
archive = Path('product-vamp-arm_64_gcc12_release.tar.gz')
with patch.object(publish_latest_generic_package, 'discover_artifacts', return_value=[archive]), \
patch.object(publish_latest_generic_package, 'list_versions', return_value=['latest', 'obsolete']), \
patch.object(publish_latest_generic_package, 'delete_version') as delete, \
patch.object(publish_latest_generic_package, 'upload_file') as upload:
self.assertEqual(publish_latest_generic_package.main(['--user', 'user', '--token', 'test-token']), 0)
archive = pathlib.Path('product-vamp-arm_64_gcc12_release.tar.gz')
with unittest.mock.patch.object(package.scripts.publish_latest_generic_package, 'discover_artifacts', return_value=[archive]), \
unittest.mock.patch.object(package.scripts.publish_latest_generic_package, 'list_versions', return_value=['latest', 'obsolete']), \
unittest.mock.patch.object(package.scripts.publish_latest_generic_package, 'delete_version') as delete, \
unittest.mock.patch.object(package.scripts.publish_latest_generic_package, 'upload_file') as upload:
self.assertEqual(package.scripts.publish_latest_generic_package.main(['--user', 'user', '--token', 'test-token']), 0)
self.assertEqual([call.args[-2] for call in delete.call_args_list], ['latest', 'obsolete'])
self.assertEqual(upload.call_args.args[2], archive)
url = publish_latest_generic_package.file_upload_url('https://example.test/generic', 'product-vamp', archive)
url = package.scripts.publish_latest_generic_package.file_upload_url('https://example.test/generic', 'product-vamp', archive)
self.assertEqual(url, 'https://example.test/generic/product-vamp/latest/product-vamp-arm_64_gcc12_release.tar.gz')
tests/test_scat.py
+55 -54
View File
@@ -1,77 +1,78 @@
import copy
from fractions import Fraction
from pathlib import Path
import fractions
import pathlib
import unittest
from generator.engraving import Fonts, Style, layout_score
from generator.engraving.layout import make_system
from generator.engraving.notation import ScoreIndex
from generator.model import rational
from generator.playback import compile_timeline
from generator.scat import add_scat, pulse, syllable
from tests.test_musicxml import notes, parse, xml_note, xml_score
import generator.engraving.fonts
import generator.engraving.layout
import generator.engraving.notation
import generator.engraving.types
import generator.model
import generator.playback
import generator.scat
import tests.test_musicxml
class ScatTests(unittest.TestCase):
def test_syllables_use_duration_pulse_phrase_and_previous_consonant(self):
self.assertEqual(syllable(Fraction(2), Fraction(1), False, True, ''), 'doo')
self.assertEqual(syllable(Fraction(1, 2), Fraction(1), True, False, ''), 'ba')
self.assertEqual(syllable(Fraction(1, 2), Fraction(1), False, False, ''), 'dit')
self.assertEqual(syllable(Fraction(1, 2), Fraction(1), False, False, 'dit'), 'da')
self.assertEqual(syllable(Fraction(1, 2), Fraction(1), True, True, ''), 'dat')
self.assertEqual(syllable(Fraction(1, 2), None, False, False, ''), 'ba')
self.assertEqual(generator.scat.syllable(fractions.Fraction(2), fractions.Fraction(1), False, True, ''), 'doo')
self.assertEqual(generator.scat.syllable(fractions.Fraction(1, 2), fractions.Fraction(1), True, False, ''), 'ba')
self.assertEqual(generator.scat.syllable(fractions.Fraction(1, 2), fractions.Fraction(1), False, False, ''), 'dit')
self.assertEqual(generator.scat.syllable(fractions.Fraction(1, 2), fractions.Fraction(1), False, False, 'dit'), 'da')
self.assertEqual(generator.scat.syllable(fractions.Fraction(1, 2), fractions.Fraction(1), True, True, ''), 'dat')
self.assertEqual(generator.scat.syllable(fractions.Fraction(1, 2), None, False, False, ''), 'ba')
def test_compound_additive_meter_and_pickups(self):
measure = {'meter': {'groups': [{'beats': [6], 'unit': 8}]}, 'duration': rational(3)}
self.assertEqual(pulse(measure, Fraction(3, 2)), (Fraction(3, 2), True))
measure = {'meter': {'groups': [{'beats': [6], 'unit': 8}]}, 'duration': generator.model.rational(3)}
self.assertEqual(generator.scat.pulse(measure, fractions.Fraction(3, 2)), (fractions.Fraction(3, 2), True))
measure['meter']['groups'][0]['beats'] = [3, 2]
self.assertEqual(pulse(measure, Fraction(3, 2)), (Fraction(1), True))
measure.update(implicit=True, duration=rational(Fraction(1, 2)))
self.assertEqual(pulse(measure, Fraction()), (Fraction(1), False))
self.assertEqual(generator.scat.pulse(measure, fractions.Fraction(3, 2)), (fractions.Fraction(1), True))
measure.update(implicit=True, duration=generator.model.rational(fractions.Fraction(1, 2)))
self.assertEqual(generator.scat.pulse(measure, fractions.Fraction()), (fractions.Fraction(1), False))
def test_ties_sustain_one_syllable_and_stop_before_rests(self):
content = xml_note(duration=1, kind='quarter', extra='<tie type="start"/>') + xml_note(duration=1, kind='quarter', extra='<tie type="stop"/>')
content = tests.test_musicxml.xml_note(duration=1, kind='quarter', extra='<tie type="start"/>') + tests.test_musicxml.xml_note(duration=1, kind='quarter', extra='<tie type="stop"/>')
content += '<note><rest/><duration>2</duration><type>half</type></note>'
document = parse(xml_score(content))
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))
original = copy.deepcopy(document)
before = compile_timeline(document)
add_scat(document)
self.assertEqual(compile_timeline(document), before)
self.assertEqual(notes(document)[0]['lyric'], 'doo')
self.assertEqual(notes(document)[0]['lyrics'][0]['extend_to'], rational(2))
self.assertNotIn('lyric', notes(document)[1])
self.assertNotIn('lyrics', notes(document)[2])
before = generator.playback.compile_timeline(document)
generator.scat.add_scat(document)
self.assertEqual(generator.playback.compile_timeline(document), before)
self.assertEqual(tests.test_musicxml.notes(document)[0]['lyric'], 'doo')
self.assertEqual(tests.test_musicxml.notes(document)[0]['lyrics'][0]['extend_to'], generator.model.rational(2))
self.assertNotIn('lyric', tests.test_musicxml.notes(document)[1])
self.assertNotIn('lyrics', tests.test_musicxml.notes(document)[2])
again = copy.deepcopy(original)
add_scat(again)
generator.scat.add_scat(again)
self.assertEqual(document, again)
fonts = Fonts(Path('package/assets/fonts/petaluma'))
layout = layout_score(document, Style(), fonts)
fonts = generator.engraving.fonts.Fonts(pathlib.Path('package/assets/fonts/petaluma'))
layout = generator.engraving.layout.layout_score(document, generator.engraving.types.Style(), fonts)
system = layout.pages[0].systems[0]
rest = next(a.x for a in system.anchors if a.fragment.kind == 'rest')
self.assertTrue(all(p.box.right < rest for p in system.primitives if p.role == 'lyric-extension'))
def test_cross_bar_tie_has_one_lyric_and_page_continuations(self):
text = xml_score(xml_note(extra='<tie type="start"/>'))
text = text.replace('</measure></part>', '</measure><measure number="2">' + xml_note(extra='<tie type="stop"/>') + '</measure></part>')
document = parse(text)
add_scat(document)
self.assertEqual(sum(bool(n.get('lyric')) for n in notes(document)), 1)
fonts = Fonts(Path('package/assets/fonts/petaluma'))
text = tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(extra='<tie type="start"/>'))
text = text.replace('</measure></part>', '</measure><measure number="2">' + tests.test_musicxml.xml_note(extra='<tie type="stop"/>') + '</measure></part>')
document = tests.test_musicxml.parse(text)
generator.scat.add_scat(document)
self.assertEqual(sum(bool(n.get('lyric')) for n in tests.test_musicxml.notes(document)), 1)
fonts = generator.engraving.fonts.Fonts(pathlib.Path('package/assets/fonts/petaluma'))
for start, end in ((0, 4), (4, 8)):
system = make_system(ScoreIndex(document), start, end, fonts, Style())
system = generator.engraving.layout.make_system(generator.engraving.notation.ScoreIndex(document), start, end, fonts, generator.engraving.types.Style())
self.assertTrue(any(p.role == 'lyric-extension' for p in system.primitives))
def test_long_tie_chain_draws_one_extender_per_system(self):
text = xml_score(xml_note(extra='<tie type="start"/>'))
text = text.replace('</measure></part>', '</measure><measure number="2">' + xml_note(extra='<tie type="stop"/><tie type="start"/>') + '</measure><measure number="3">' + xml_note(extra='<tie type="stop"/>') + '</measure></part>')
document = parse(text)
add_scat(document)
fonts = Fonts(Path('package/assets/fonts/petaluma'))
index = ScoreIndex(document)
text = tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(extra='<tie type="start"/>'))
text = text.replace('</measure></part>', '</measure><measure number="2">' + tests.test_musicxml.xml_note(extra='<tie type="stop"/><tie type="start"/>') + '</measure><measure number="3">' + tests.test_musicxml.xml_note(extra='<tie type="stop"/>') + '</measure></part>')
document = tests.test_musicxml.parse(text)
generator.scat.add_scat(document)
fonts = generator.engraving.fonts.Fonts(pathlib.Path('package/assets/fonts/petaluma'))
index = generator.engraving.notation.ScoreIndex(document)
for start, end in ((0, 4), (4, 8), (8, 12), (0, 12)):
system = make_system(index, start, end, fonts, Style())
system = generator.engraving.layout.make_system(index, start, end, fonts, generator.engraving.types.Style())
self.assertEqual(sum(p.role == 'lyric-extension' for p in system.primitives), 1)
def test_straight_offbeat_attacks_and_cues(self):
document = parse(xml_score(''.join(xml_note(duration=1, kind='eighth') for _ in range(8)), divisions=2))
before = compile_timeline(document)
add_scat(document)
self.assertEqual([n['lyric'] for n in notes(document)], ['ba', 'dit', 'ba', 'dit', 'ba', 'dit', 'ba', 'dat'])
self.assertEqual(compile_timeline(document), before)
document = parse(xml_score(xml_note(prefix='<cue/>')))
add_scat(document)
self.assertNotIn('lyrics', notes(document)[0])
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(''.join(tests.test_musicxml.xml_note(duration=1, kind='eighth') for _ in range(8)), divisions=2))
before = generator.playback.compile_timeline(document)
generator.scat.add_scat(document)
self.assertEqual([n['lyric'] for n in tests.test_musicxml.notes(document)], ['ba', 'dit', 'ba', 'dit', 'ba', 'dit', 'ba', 'dat'])
self.assertEqual(generator.playback.compile_timeline(document), before)
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(prefix='<cue/>')))
generator.scat.add_scat(document)
self.assertNotIn('lyrics', tests.test_musicxml.notes(document)[0])
tests/test_score_bundle.py
+164 -132
View File
@@ -1,191 +1,200 @@
import copy
import argparse
from fractions import Fraction
import copy
import fractions
import hashlib
import json
from pathlib import Path
import pathlib
import tempfile
import unittest
from unittest.mock import patch
from xml.etree import ElementTree
import unittest.mock
import xml.etree.ElementTree
from generator.arrange.generate import arrange_document
from generator.__main__ import generate_assets
from generator.engraving import ENGINE_NAME, EngravingError, Fonts, Style, check_page, engrave_tree, layout_score
from tests.musicxml_roundtrip import audit_roundtrip
from generator.model import fraction, pitch_number, rational, validate_document
from generator.musicxml import load_musicxml
from generator.musicxml_writer import write_musicxml
from generator.practice.generate import practice_document
from generator.score import load_json
from generator.pitch import concert_document
from tests.test_musicxml import notes, parse, xml_note, xml_score
import PIL.Image
import PIL.ImageChops
import generator.__main__
import generator.arrange.generate
import generator.engraving.fonts
import generator.engraving.layout
import generator.engraving.notation
import generator.engraving.pipeline
import generator.engraving.types
import generator.engraving.validation
import generator.model
import generator.musicxml
import generator.musicxml_writer
import generator.pico_artwork
import generator.pitch
import generator.playback
import generator.practice.generate
import generator.score
import tests.musicxml_roundtrip
import tests.rhythm_fixtures
import tests.test_musicxml
class ScoreBundleTests(unittest.TestCase):
def test_real_asset_regeneration_removes_pages_and_rolls_back_font_failure(self):
self.enterContext(unittest.mock.patch('generator.rhythm.load_library', return_value=tests.rhythm_fixtures.library()))
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
root = pathlib.Path(temporary)
data, output = root / 'data', root / 'output'
(data / 'songs').mkdir(parents=True)
(data / 'practice').mkdir()
source = data / 'practice/0001.xml'
source.write_text(xml_score(xml_note()))
args = argparse.Namespace(data_dir=data, output_dir=output, static_assets_dir=Path('package/assets'), check=False)
generate_assets(args)
source.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
args = argparse.Namespace(data_dir=data, output_dir=output, static_assets_dir=pathlib.Path('package/assets'), check=False)
generator.__main__.generate_assets(args)
original = {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()}
self.assertTrue(list(output.rglob('*.png')))
legacy_source = source.with_name('001.xml')
source.rename(legacy_source)
with self.assertRaisesRegex(ValueError, 'four-digit'):
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertEqual(original, {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()})
legacy_source.rename(source)
(data / 'rhythms' / 'drafts').mkdir(parents=True)
for path in ('rhythms/0001.xml', 'rhythms/drafts/0002.xml'):
(data / path).write_text('<unfinished')
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertEqual(original, {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()})
misplaced = data / '0001.xml'
misplaced.write_text(xml_score(xml_note()))
misplaced.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
with self.assertRaisesRegex(ValueError, 'belong directly under'):
generate_assets(args)
generator.__main__.generate_assets(args)
misplaced.unlink()
with patch('generator.__main__.compile_tree', side_effect=ValueError('invalid generated audio')):
with unittest.mock.patch('generator.playback.compile_tree', side_effect=ValueError('invalid generated audio')):
with self.assertRaisesRegex(ValueError, 'audio'):
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertEqual(original, {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()})
source.write_text(xml_score(xml_note() + '<barline><repeat direction="backward"/></barline>'))
source.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note() + '<barline><repeat direction="backward"/></barline>'))
with self.assertRaisesRegex(ValueError, 'write the music out'):
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertEqual(original, {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()})
source.write_text(xml_score(xml_note(extra='<accidental smufl="unavailableGlyph">other</accidental>')))
with self.assertRaisesRegex(EngravingError, 'unavailableGlyph'):
generate_assets(args)
source.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(extra='<accidental smufl="unavailableGlyph">other</accidental>')))
with self.assertRaisesRegex(generator.engraving.types.EngravingError, 'unavailableGlyph'):
generator.__main__.generate_assets(args)
self.assertEqual(original, {p.relative_to(output): p.read_bytes() for p in output.rglob('*') if p.is_file()})
source.write_text(xml_score(xml_note()))
source.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
renamed = data / 'practice/0127.xml'
source.rename(renamed)
source = renamed
legacy_output = output / 'practice/001'
legacy_output.mkdir()
(legacy_output / 'electric.json').write_text('{}')
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertFalse(legacy_output.exists())
self.assertFalse((output / 'practice/0001').exists())
self.assertEqual(len(list((output / 'practice/0127').glob('*.json'))), 3)
source.unlink()
generate_assets(args)
generator.__main__.generate_assets(args)
self.assertFalse(list(output.rglob('*.png')))
self.assertFalse(list(output.rglob('*.json')))
def round_trip(self, document):
with tempfile.TemporaryDirectory() as temporary:
path = Path(temporary) / 'roundtrip.xml'
write_musicxml(path, document)
return load_musicxml(path)
path = pathlib.Path(temporary) / 'roundtrip.xml'
generator.musicxml_writer.write_musicxml(path, document)
return generator.musicxml.load_musicxml(path)
def test_score_structure_is_validated_without_schema_labels(self):
document = parse(xml_score(xml_note()))
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
self.assertEqual(set(document), {'metadata', 'staves'})
for malformed in (None, [], {}, {'metadata': {}, 'staves': []}, {**document, 'staves': None}, {**document, 'staves': [{}]}):
with self.subTest(document=malformed), self.assertRaises(ValueError):
validate_document(malformed)
generator.model.validate_document(malformed)
def test_multiple_voices_keep_independent_ties_and_transformations(self):
def half(action):
return xml_note(duration=2, kind='half', extra=f'<voice>1</voice><tie type="{action}"/>')
return tests.test_musicxml.xml_note(duration=2, kind='half', extra=f'<voice>1</voice><tie type="{action}"/>')
content = half('start') + half('stop') + '<backup><duration>4</duration></backup>'
content += xml_note('C', extra='<voice>2</voice>')
source = parse(xml_score(content))
content += tests.test_musicxml.xml_note('C', extra='<voice>2</voice>')
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(content))
measure = source['staves'][0]['measures'][0]
self.assertEqual([v['id'] for v in measure['voices']], ['1', '2'])
self.assertNotIn('tie_from', measure['voices'][1]['notes'][0]['pitches'][0])
for document in (source, concert_document(source), arrange_document(source, 'vocal')):
validate_document(document)
for document in (source, generator.pitch.concert_document(source), generator.arrange.generate.arrange_document(source, 'vocal', rhythms=tests.rhythm_fixtures.library())):
generator.model.validate_document(document)
restored = self.round_trip(document)
validate_document(restored)
generator.model.validate_document(restored)
self.assertTrue(any(len(m['voices']) == 2 for m in restored['staves'][0]['measures']))
def test_contiguous_ties_continue_across_systems_without_repeat_state(self):
from generator.engraving.layout import make_system
from generator.engraving.notation import ScoreIndex
from generator.playback import compile_timeline
text = xml_score(xml_note(extra='<tie type="start"/>'))
text = text.replace('</measure></part>', '</measure><measure number="2">' + xml_note(extra='<tie type="stop"/>') + '<barline><bar-style>light-heavy</bar-style></barline></measure></part>')
source = parse(text)
index = ScoreIndex(source)
fonts = Fonts(Path('package/assets/fonts/petaluma'))
text = tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(extra='<tie type="start"/>'))
text = text.replace('</measure></part>', '</measure><measure number="2">' + tests.test_musicxml.xml_note(extra='<tie type="stop"/>') + '<barline><bar-style>light-heavy</bar-style></barline></measure></part>')
source = tests.test_musicxml.parse(text)
index = generator.engraving.notation.ScoreIndex(source)
fonts = generator.engraving.fonts.Fonts(pathlib.Path('package/assets/fonts/petaluma'))
for start, end in ((0, 4), (4, 8)):
system = make_system(index, start, end, fonts, Style())
system = generator.engraving.layout.make_system(index, start, end, fonts, generator.engraving.types.Style())
self.assertTrue(any(p.role == 'tie' and len(p.sources) == 2 for p in system.primitives))
self.assertEqual([fraction(e['duration']) for e in compile_timeline(source)['events']], [8])
self.assertEqual([generator.model.fraction(e['duration']) for e in generator.playback.compile_timeline(source)['events']], [8])
restored = self.round_trip(source)
self.assertEqual(notes(restored)[0]['pitches'], notes(source)[0]['pitches'])
self.assertEqual(tests.test_musicxml.notes(restored)[0]['pitches'], tests.test_musicxml.notes(source)[0]['pitches'])
self.assertEqual(restored['staves'][0]['measures'][1]['barlines'], {'right': 'light-heavy'})
def test_microtones_remain_exact_after_transposition_and_export(self):
source = parse(xml_score(xml_note(alter='0.5', extra='<accidental>quarter-sharp</accidental>')))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(alter='0.5', extra='<accidental>quarter-sharp</accidental>')))
source['staves'][0]['measures'][0]['transpose'] = {'diatonic': 1, 'chromatic': 2, 'octave_change': 0}
transposed = concert_document(source)
self.assertEqual(pitch_number(notes(transposed)[0]['pitches'][0]), Fraction(125, 2))
transposed = generator.pitch.concert_document(source)
self.assertEqual(generator.model.pitch_number(tests.test_musicxml.notes(transposed)[0]['pitches'][0]), fractions.Fraction(125, 2))
restored = self.round_trip(transposed)
self.assertEqual(notes(restored)[0]['pitches'], notes(transposed)[0]['pitches'])
unqualified = parse(xml_score(xml_note(alter='0.1')))
self.assertEqual(fraction(notes(unqualified)[0]['pitches'][0]['alter']), Fraction(1, 10))
layout = layout_score(unqualified, Style(), Fonts(Path('package/assets/fonts/petaluma')))
self.assertEqual(tests.test_musicxml.notes(restored)[0]['pitches'], tests.test_musicxml.notes(transposed)[0]['pitches'])
unqualified = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(alter='0.1')))
self.assertEqual(generator.model.fraction(tests.test_musicxml.notes(unqualified)[0]['pitches'][0]['alter']), fractions.Fraction(1, 10))
layout = generator.engraving.layout.layout_score(unqualified, generator.engraving.types.Style(), generator.engraving.fonts.Fonts(pathlib.Path('package/assets/fonts/petaluma')))
self.assertTrue(any(p.role == 'accidental' and p.value == '(+1/10 st)' for page in layout.pages for p in page.primitives))
def test_octave_shift_stop_at_a_bar_boundary_is_emitted_once(self):
tree = ElementTree.fromstring(xml_score(xml_note()))
ElementTree.SubElement(tree.find('part'), 'measure', number='2').append(ElementTree.fromstring(xml_note()))
source = parse(ElementTree.tostring(tree, encoding='unicode'))
source['staves'][0]['octave_shifts'] = [{'id': 'octave-1', 'start': rational(0), 'end': rational(4), 'octaves': 1}]
tree = xml.etree.ElementTree.fromstring(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
xml.etree.ElementTree.SubElement(tree.find('part'), 'measure', number='2').append(xml.etree.ElementTree.fromstring(tests.test_musicxml.xml_note()))
source = tests.test_musicxml.parse(xml.etree.ElementTree.tostring(tree, encoding='unicode'))
source['staves'][0]['octave_shifts'] = [{'id': 'octave-1', 'start': generator.model.rational(0), 'end': generator.model.rational(4), 'octaves': 1}]
restored = self.round_trip(source)
self.assertEqual(source['staves'][0]['octave_shifts'], restored['staves'][0]['octave_shifts'])
def test_unequal_chord_tones_are_retained_but_never_silently_flattened(self):
source = parse(xml_score(xml_note() + xml_note('E', 2, 'half', prefix='<chord/>')))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note() + tests.test_musicxml.xml_note('E', 2, 'half', prefix='<chord/>')))
restored = self.round_trip(source)
self.assertEqual(notes(restored)[0]['pitches'], notes(source)[0]['pitches'])
layout = layout_score(source, Style(), Fonts(Path('package/assets/fonts/petaluma')))
self.assertEqual(tests.test_musicxml.notes(restored)[0]['pitches'], tests.test_musicxml.notes(source)[0]['pitches'])
layout = generator.engraving.layout.layout_score(source, generator.engraving.types.Style(), generator.engraving.fonts.Fonts(pathlib.Path('package/assets/fonts/petaluma')))
heads = [p.value for page in layout.pages for p in page.primitives if p.role == 'head']
self.assertEqual(heads, ['noteheadWhole', 'noteheadHalf'])
def test_overlapping_notes_within_one_voice_are_rejected(self):
with self.assertRaisesRegex(ValueError, 'overlapping'):
parse(xml_score(xml_note() + '<backup><duration>4</duration></backup>' + xml_note('E')))
tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note() + '<backup><duration>4</duration></backup>' + tests.test_musicxml.xml_note('E')))
def test_multiple_lyric_verses_and_elisions_round_trip(self):
lyrics = '<lyric number="1"><syllabic>begin</syllabic><text>one</text><elision> </elision><text>two</text><extend type="start"/></lyric>'
lyrics += '<lyric number="2"><syllabic>single</syllabic><text>three</text></lyric>'
source = parse(xml_score(xml_note(extra=lyrics)))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(extra=lyrics)))
restored = self.round_trip(source)
self.assertEqual(notes(restored)[0]['lyrics'], notes(source)[0]['lyrics'])
self.assertEqual(tests.test_musicxml.notes(restored)[0]['lyrics'], tests.test_musicxml.notes(source)[0]['lyrics'])
def test_percussion_and_tablature_are_semantic_not_pitch_guesses(self):
percussion = '<note><unpitched><display-step>C</display-step><display-octave>5</display-octave></unpitched><duration>4</duration><type>whole</type><notehead>x</notehead></note>'
source = parse(xml_score(percussion, attributes='<clef><sign>percussion</sign></clef>'))
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(percussion, attributes='<clef><sign>percussion</sign></clef>'))
restored = self.round_trip(source)
self.assertEqual(notes(restored)[0]['pitches'], notes(source)[0]['pitches'])
self.assertEqual(tests.test_musicxml.notes(restored)[0]['pitches'], tests.test_musicxml.notes(source)[0]['pitches'])
with self.assertRaisesRegex(ValueError, 'authored TAB'):
parse(xml_score(xml_note(), attributes='<clef><sign>TAB</sign></clef>'))
tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(), attributes='<clef><sign>TAB</sign></clef>'))
for part in ('vocal', 'acoustic', 'electric'):
with self.assertRaisesRegex(ValueError, 'pitched'):
practice_document(source, part)
generator.practice.generate.practice_document(source, part, rhythms=tests.rhythm_fixtures.library())
with self.assertRaisesRegex(ValueError, 'pitched'):
arrange_document(source, part)
generator.arrange.generate.arrange_document(source, part, rhythms=tests.rhythm_fixtures.library())
def test_percussion_instrument_definitions_round_trip_and_are_audited(self):
tree = ElementTree.fromstring(xml_score('<note><unpitched><display-step>C</display-step><display-octave>5</display-octave></unpitched><duration>4</duration><instrument id="drum"/><type>whole</type></note>', attributes='<clef><sign>percussion</sign></clef>'))
tree = xml.etree.ElementTree.fromstring(tests.test_musicxml.xml_score('<note><unpitched><display-step>C</display-step><display-octave>5</display-octave></unpitched><duration>4</duration><instrument id="drum"/><type>whole</type></note>', attributes='<clef><sign>percussion</sign></clef>'))
definition = tree.find('part-list/score-part')
definition.append(ElementTree.fromstring('<score-instrument id="drum"><instrument-name>Snare</instrument-name><instrument-sound>drum.snare-drum</instrument-sound><solo/><virtual-instrument><virtual-name>Kit</virtual-name></virtual-instrument></score-instrument>'))
definition.append(ElementTree.fromstring('<midi-instrument id="drum"><midi-channel>10</midi-channel><midi-unpitched>39</midi-unpitched></midi-instrument>'))
source = parse(ElementTree.tostring(tree, encoding='unicode'))
definition.append(xml.etree.ElementTree.fromstring('<score-instrument id="drum"><instrument-name>Snare</instrument-name><instrument-sound>drum.snare-drum</instrument-sound><solo/><virtual-instrument><virtual-name>Kit</virtual-name></virtual-instrument></score-instrument>'))
definition.append(xml.etree.ElementTree.fromstring('<midi-instrument id="drum"><midi-channel>10</midi-channel><midi-unpitched>39</midi-unpitched></midi-instrument>'))
source = tests.test_musicxml.parse(xml.etree.ElementTree.tostring(tree, encoding='unicode'))
self.assertEqual(self.round_trip(source)['staves'][0]['instruments'], source['staves'][0]['instruments'])
changed = copy.deepcopy(source)
changed['staves'][0]['instruments'][0]['midi']['midi-unpitched'] = '40'
with tempfile.TemporaryDirectory() as temporary:
before, after = Path(temporary) / 'source.xml', Path(temporary) / 'output.xml'
write_musicxml(before, source)
write_musicxml(after, changed)
before, after = pathlib.Path(temporary) / 'source.xml', pathlib.Path(temporary) / 'output.xml'
generator.musicxml_writer.write_musicxml(before, source)
generator.musicxml_writer.write_musicxml(after, changed)
with self.assertRaisesRegex(ValueError, 'percussion instrument'):
audit_roundtrip(before, after)
tests.musicxml_roundtrip.audit_roundtrip(before, after)
source['staves'][0]['instruments'] = []
with self.assertRaisesRegex(ValueError, 'unknown score instrument'):
validate_document(source)
generator.model.validate_document(source)
def test_catalog_roundtrip_preserves_written_pitch_and_rhythm(self):
for path in sorted(path for category in ('practice', 'songs') for path in (Path('data') / category).glob('*.xml')):
for path in sorted(path for category in ('practice', 'songs') for path in (pathlib.Path('data') / category).glob('*.xml')):
with self.subTest(path=path):
source = load_musicxml(path)
source = generator.musicxml.load_musicxml(path)
restored = self.round_trip(source)
a, b = source['staves'][0]['measures'], restored['staves'][0]['measures']
self.assertEqual(len(a), len(b))
@@ -195,95 +204,118 @@ class ScoreBundleTests(unittest.TestCase):
self.assertEqual(v['notes'], w['notes'])
def test_missing_font_does_not_produce_a_bundle(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
source = parse(xml_score(xml_note()))
root = pathlib.Path(temporary)
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
path = root / 'fixture.json'
path.write_text(json.dumps(source))
with self.assertRaisesRegex(EngravingError, 'cannot load bundled'):
engrave_tree(root, root / 'missing-fonts')
with self.assertRaisesRegex(generator.engraving.types.EngravingError, 'cannot load bundled'):
generator.engraving.pipeline.engrave_tree(root, root / 'missing-fonts')
self.assertEqual(json.loads(path.read_text()), source)
def test_blank_and_wrong_sized_pages_are_rejected(self):
from PIL import Image
with tempfile.TemporaryDirectory() as temporary:
path = Path(temporary) / 'page.png'
path = pathlib.Path(temporary) / 'page.png'
for size in ((976, 484), (100, 100)):
Image.new('RGB', size, 'white').save(path)
with self.assertRaises(EngravingError):
check_page(path)
PIL.Image.new('RGB', size, 'white').save(path)
with self.assertRaises(generator.engraving.types.EngravingError):
generator.engraving.validation.check_page(path)
def test_audit_rejects_an_engraver_pitch_change(self):
with tempfile.TemporaryDirectory() as temporary:
source, output = Path(temporary) / 'source.xml', Path(temporary) / 'output.xml'
source.write_text(xml_score(xml_note()))
output.write_text(xml_score(xml_note('D')))
source, output = pathlib.Path(temporary) / 'source.xml', pathlib.Path(temporary) / 'output.xml'
source.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
output.write_text(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note('D')))
with self.assertRaisesRegex(ValueError, 'pitch letter'):
audit_roundtrip(source, output)
tests.musicxml_roundtrip.audit_roundtrip(source, output)
def test_audit_rejects_staff_details_and_barline_changes(self):
with tempfile.TemporaryDirectory() as temporary:
source, output = Path(temporary) / 'source.xml', Path(temporary) / 'output.xml'
document = parse(xml_score(xml_note()))
source, output = pathlib.Path(temporary) / 'source.xml', pathlib.Path(temporary) / 'output.xml'
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
for field, value in (('staff_details', {'lines': 6, 'tunings': []}), ('barlines', {'right': 'light-light'})):
with self.subTest(field=field):
original = copy.deepcopy(document)
original['staves'][0]['measures'][0][field] = value
write_musicxml(source, original)
write_musicxml(output, document)
generator.musicxml_writer.write_musicxml(source, original)
generator.musicxml_writer.write_musicxml(output, document)
with self.assertRaisesRegex(ValueError, 'engraver changed musical content'):
audit_roundtrip(source, output)
tests.musicxml_roundtrip.audit_roundtrip(source, output)
def test_audit_rejects_relocated_tuplet_brackets(self):
modulation = '<time-modification><actual-notes>3</actual-notes><normal-notes>2</normal-notes></time-modification>'
source = parse(xml_score(''.join(xml_note(duration=2, kind='quarter', extra=modulation) for _ in range(6)), divisions=3))
span = {'id': 'tuplet-1', 'voice': '1', 'start': notes(source)[0]['id'], 'end': notes(source)[2]['id'], 'note_count': 3, 'space_count': 2, 'depth': 0, 'bracket': 'auto', 'show_number': 'actual'}
source = tests.test_musicxml.parse(tests.test_musicxml.xml_score(''.join(tests.test_musicxml.xml_note(duration=2, kind='quarter', extra=modulation) for _ in range(6)), divisions=3))
span = {'id': 'tuplet-1', 'voice': '1', 'start': tests.test_musicxml.notes(source)[0]['id'], 'end': tests.test_musicxml.notes(source)[2]['id'], 'note_count': 3, 'space_count': 2, 'depth': 0, 'bracket': 'auto', 'show_number': 'actual'}
source['staves'][0]['tuplets'] = [span]
for note in notes(source)[:3]:
for note in tests.test_musicxml.notes(source)[:3]:
note['tuplet_ids'] = ['tuplet-1']
changed = copy.deepcopy(source)
changed['staves'][0]['tuplets'][0].update(start=notes(changed)[3]['id'], end=notes(changed)[5]['id'])
for index, note in enumerate(notes(changed)):
changed['staves'][0]['tuplets'][0].update(start=tests.test_musicxml.notes(changed)[3]['id'], end=tests.test_musicxml.notes(changed)[5]['id'])
for index, note in enumerate(tests.test_musicxml.notes(changed)):
note['tuplet_ids'] = ['tuplet-1'] if index >= 3 else []
with tempfile.TemporaryDirectory() as temporary:
before, after = Path(temporary) / 'source.xml', Path(temporary) / 'output.xml'
write_musicxml(before, source)
write_musicxml(after, changed)
before, after = pathlib.Path(temporary) / 'source.xml', pathlib.Path(temporary) / 'output.xml'
generator.musicxml_writer.write_musicxml(before, source)
generator.musicxml_writer.write_musicxml(after, changed)
with self.assertRaisesRegex(ValueError, 'members'):
audit_roundtrip(before, after)
tests.musicxml_roundtrip.audit_roundtrip(before, after)
def test_bundle_validation_checks_pages_and_rejects_symlink_escape(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
root = pathlib.Path(temporary)
assets = root / 'assets'
assets.mkdir()
data = Path('tests/fixtures/engraving.png').read_bytes()
data = pathlib.Path('tests/fixtures/engraving.png').read_bytes()
page = assets / 'page.png'
page.write_bytes(data)
document = load_musicxml(Path('tests/fixtures/engraving.xml'))
bundle = {'music': document, 'presentation': {'engine': ENGINE_NAME, 'width': 976, 'height': 484, 'pages': [{'path': 'page.png', 'sha256': hashlib.sha256(data).hexdigest()}]}}
document = generator.musicxml.load_musicxml(pathlib.Path('tests/fixtures/engraving.xml'))
bundle = {'music': document, 'presentation': {'engine': generator.engraving.validation.ENGINE_NAME, 'width': 976, 'height': 484, 'pages': [{'path': 'page.png', 'sha256': hashlib.sha256(data).hexdigest()}]}}
path = assets / 'score.json'
path.write_text(json.dumps(bundle))
self.assertEqual(load_json(path), document)
self.assertEqual(generator.score.load_json(path), document)
page.write_bytes(data + b'corrupt')
with self.assertRaisesRegex(ValueError, 'checksum'):
load_json(path)
generator.score.load_json(path)
page.unlink()
outside = root / 'outside.png'
outside.write_bytes(data)
page.symlink_to(outside)
with self.assertRaisesRegex(ValueError, 'escapes'):
load_json(path)
def test_python_engraver_visual_golden_and_determinism(self):
from PIL import Image, ImageChops
from generator.score import write_json
document = load_musicxml(Path('tests/fixtures/engraving.xml'))
generator.score.load_json(path)
def test_final_validation_reuses_content_checks_and_rechecks_files(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
root = pathlib.Path(temporary)
document = tests.test_musicxml.parse(tests.test_musicxml.xml_score(tests.test_musicxml.xml_note()))
for index in range(3):
generator.score.write_json(root / 'practice' / f'{index}.json', document)
trusted = generator.engraving.pipeline.engrave_tree(root, pathlib.Path('package/assets/fonts/petaluma'))
generator.pico_artwork.generate_titles(root, pathlib.Path('package/assets/fonts/petaluma'))
with unittest.mock.patch('generator.engraving.validation.check_content', side_effect=AssertionError('already validated content')):
generator.score.validate_generated_tree(root, trusted)
page = root / 'practice/2.pages/0001.png'
original = page.read_bytes()
page.write_bytes(original + b'changed')
with self.assertRaisesRegex(ValueError, 'checksum'):
generator.score.validate_generated_tree(root, trusted)
page.write_bytes(original)
with unittest.mock.patch('generator.engraving.validation.check_content', wraps=generator.engraving.validation.check_content) as decode:
generator.score.validate_generated_tree(root)
decode.assert_called_once()
PIL.Image.new('RGB', (976, 484), 'white').save(page)
path = root / 'practice/2.json'
bundle = json.loads(path.read_text())
bundle['presentation']['pages'][0]['sha256'] = hashlib.sha256(page.read_bytes()).hexdigest()
path.write_text(json.dumps(bundle))
with self.assertRaisesRegex(ValueError, 'blank page'):
generator.score.validate_generated_tree(root, trusted)
def test_python_engraver_visual_golden_and_determinism(self):
document = generator.musicxml.load_musicxml(pathlib.Path('tests/fixtures/engraving.xml'))
with tempfile.TemporaryDirectory() as temporary:
root = pathlib.Path(temporary)
before = None
for _ in range(2):
write_json(root / 'fixture.json', document)
engrave_tree(root, Path('package/assets/fonts/petaluma'))
generator.score.write_json(root / 'fixture.json', document)
generator.engraving.pipeline.engrave_tree(root, pathlib.Path('package/assets/fonts/petaluma'))
bundle = json.loads((root / 'fixture.json').read_text())
self.assertEqual(bundle['music'], document)
self.assertEqual(len(bundle['presentation']['pages']), 1)
page = root / bundle['presentation']['pages'][0]['path']
with Image.open(page) as actual, Image.open('tests/fixtures/engraving-python.png') as expected:
self.assertIsNone(ImageChops.difference(actual.convert('RGB'), expected.convert('RGB')).getbbox())
with PIL.Image.open(page) as actual, PIL.Image.open('tests/fixtures/engraving-python.png') as expected:
self.assertIsNone(PIL.ImageChops.difference(actual.convert('RGB'), expected.convert('RGB')).getbbox())
if before is not None:
self.assertEqual(bundle, before)
before = bundle
tests/test_score_generator.py
+120 -25
View File
@@ -1,36 +1,131 @@
from __future__ import annotations
import unittest
import unittest.mock
import hashlib
import json
import pathlib
import struct
import subprocess
import sys
import tempfile
from pathlib import Path
import unittest
import unittest.mock
from generator.__main__ import copy_runtime_fonts
from generator import generate_pico_font
import generator.__main__
import generator.pico_artwork
class FontAssetsTests(unittest.TestCase):
def test_font_generator_requires_kerning_backend(self):
with unittest.mock.patch.object(generate_pico_font.features, "check_feature", return_value=False):
with self.assertRaisesRegex(RuntimeError, "Raqm"):
generate_pico_font.generate()
def test_runtime_fonts_exclude_unused_assets(self):
root = Path(__file__).resolve().parents[1]
root = pathlib.Path(__file__).resolve().parents[1]
with tempfile.TemporaryDirectory() as directory:
output = Path(directory)
copy_runtime_fonts(root / "package/assets", output)
files = {path.name for path in (output / "fonts/petaluma").iterdir()}
self.assertEqual(files, {"PetalumaScript.otf", "OFL.txt"})
def test_font_generator_uses_explicit_build_output(self):
root = Path(__file__).resolve().parents[1]
output = pathlib.Path(directory)
generator.__main__.copy_runtime_fonts(root / 'package/assets', output)
files = {path.name for path in (output / 'fonts/petaluma').iterdir()}
self.assertEqual(files, {'PetalumaScript.otf', 'OFL.txt'})
class PicoArtworkTests(unittest.TestCase):
def test_requires_shaping_and_rejects_unsupported_text(self):
with unittest.mock.patch.object(generator.pico_artwork.PIL.features, 'check_feature', return_value=False):
with self.assertRaisesRegex(RuntimeError, 'Raqm'):
generator.pico_artwork.Renderer(generator.pico_artwork.DEFAULT_FONT_ROOT / 'PetalumaScript.otf')
renderer = generator.pico_artwork.Renderer(generator.pico_artwork.DEFAULT_FONT_ROOT / 'PetalumaScript.otf')
for text in ('', 'x' * 201, 'line\nbreak', '\U0001f600'):
with self.subTest(text=text):
with self.assertRaises(ValueError):
renderer.title(text)
with self.assertRaisesRegex(ValueError, 'ellipsis'):
renderer.line('long title', 5, 32, 24)
def test_static_atlas_is_bounded_and_every_digit_is_visible_and_distinct(self):
with tempfile.TemporaryDirectory() as directory:
output = Path(directory) / "generated/font.cpp"
command = [sys.executable, str(root / "generator/generate_pico_font.py"), "--output", str(output)]
subprocess.run(command, check=True, capture_output=True, text=True, cwd=directory)
self.assertNotIn("std::array<pair_adjustment, 0>", output.read_text(encoding="utf-8"))
before = output.stat().st_mtime_ns
subprocess.run([*command, "--check"], check=True, capture_output=True, text=True)
self.assertEqual(before, output.stat().st_mtime_ns)
output.write_text("stale", encoding="utf-8")
self.assertNotEqual(subprocess.run([*command, "--check"], capture_output=True).returncode, 0)
output = pathlib.Path(directory) / 'generated' / 'static.p4'
subprocess.run([sys.executable, '-m', 'generator.pico_artwork', '--static-output', str(output)], check=True)
atlas = output.read_bytes()
self.assertLess(len(atlas), 24000)
magic, count, reserved = struct.unpack_from('<4sHH', atlas)
self.assertEqual((magic, count, reserved), (b'VPSA', 24, 0))
digits = []
for index in range(count):
width, height, offset = struct.unpack_from('<HHI', atlas, 8 + index * 8)
self.assertEqual(height, 28)
self.assertLessEqual(width, 176)
self.assertGreaterEqual(offset, 8 + 8 * count)
size = (width * height + 1) // 2
coverage = atlas[offset:offset + size]
self.assertEqual(len(coverage), size)
levels = [value for byte in coverage for value in (byte >> 4, byte & 15)][:width * height]
self.assertGreater(sum(value > 0 for value in levels), 25)
self.assertTrue(any(0 < value < 15 for value in levels))
if index >= 13:
self.assertEqual(width, 22)
digits.append(coverage)
ink_rows = {pixel // width for pixel, value in enumerate(levels) if value}
self.assertGreater(len(ink_rows), 12)
self.assertEqual(len(set(digits)), 11)
page_width = struct.unpack_from('<H', atlas, 8 + 8 * 8)[0]
self.assertLessEqual(page_width + 8 + len('10000/10000') * generator.pico_artwork.DIGIT_WIDTH, 320 - 16)
before = atlas
generator.pico_artwork.generate_static(output)
self.assertEqual(output.read_bytes(), before)
self.assertEqual({path.name for path in output.parent.iterdir()}, {'static.p4'})
def test_titles_update_both_categories_share_across_parts_and_change_without_firmware(self):
with tempfile.TemporaryDirectory() as directory:
root = pathlib.Path(directory)
paths = []
for category, part in (('songs', 'Acoustic'), ('songs', 'Vocal'), ('practice', 'Electric')):
path = root / category / f'{part}.json'
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps({'music': {'metadata': {'title': 'Evening Song'}, 'staves': [{'name': part}]},
'presentation': {'pages': [{'path': 'page.png'}]}}))
paths.append(path)
generator.pico_artwork.generate_titles(root)
documents = [json.loads(path.read_text()) for path in paths]
identifiers = {document['presentation']['title_image'] for document in documents}
self.assertEqual(len(identifiers), 1)
identifier = identifiers.pop()
title = root / 'pico' / 'titles' / f'{identifier}.mask'
self.assertEqual(len(title.read_bytes()), 9480)
self.assertEqual(title.read_bytes()[:8], b'VPTI\x28\x01\x40\x00')
self.assertEqual(hashlib.sha256(title.read_bytes()).hexdigest(), identifier)
self.assertGreater(sum(byte != 0 for byte in title.read_bytes()[8:]), 100)
generator.pico_artwork.validate_title(root, identifier)
self.assertEqual(documents[0]['presentation']['pages'], [{'path': 'page.png'}])
documents[0]['music']['metadata']['title'] = 'Morning Song'
paths[0].write_text(json.dumps(documents[0]))
generator.pico_artwork.generate_titles(root)
replacement = json.loads(paths[0].read_text())['presentation']['title_image']
self.assertNotEqual(replacement, identifier)
generator.pico_artwork.validate_title(root, replacement)
self.assertFalse((root / 'pico' / 'titles.json').exists())
def test_validator_rejects_bad_identifiers_corruption_size_and_symlink_escape(self):
renderer = generator.pico_artwork.Renderer(generator.pico_artwork.DEFAULT_FONT_ROOT / 'PetalumaScript.otf')
image = renderer.title('A complete title')
identifier = hashlib.sha256(image).hexdigest()
with tempfile.TemporaryDirectory() as directory:
root = pathlib.Path(directory) / 'assets'
titles = root / 'pico' / 'titles'
titles.mkdir(parents=True)
path = titles / f'{identifier}.mask'
path.write_bytes(image)
generator.pico_artwork.validate_title(root, identifier)
for invalid in ('', '../title', identifier.upper(), 'a' * 65, 'z' * 64):
with self.subTest(identifier=invalid):
with self.assertRaises(ValueError):
generator.pico_artwork.validate_title(root, invalid)
for corrupted in (image[:-1], image + b'x', b'BAD!' + image[4:], image[:8] + bytes(len(image) - 8)):
path.write_bytes(corrupted)
with self.assertRaises(ValueError):
generator.pico_artwork.validate_title(root, identifier)
path.unlink()
outside = pathlib.Path(directory) / 'outside.mask'
outside.write_bytes(image)
path.symlink_to(outside)
with self.assertRaisesRegex(ValueError, 'escapes'):
generator.pico_artwork.validate_title(root, identifier)
def test_long_titles_are_bounded_without_clipping_or_hanging(self):
renderer = generator.pico_artwork.Renderer(generator.pico_artwork.DEFAULT_FONT_ROOT / 'PetalumaScript.otf')
for title in ('A very long title ' * 11, 'W' * 200, 'A'):
image = renderer.title(title)
self.assertEqual(len(image), 9480)
levels = [value for byte in image[8:] for value in (byte >> 4, byte & 15)]
self.assertGreater(max(levels), 0)
self.assertFalse(any(levels[:296]))
self.assertFalse(any(levels[-296:]))