PDF Render
ci / checks (push) Successful in 52m51s

This commit is contained in:
khodowany
2026-10-08 00:07:07 -05:00
parent 841a3f974e
commit 300097a37e
12 changed files with 394 additions and 15 deletions
+2
View File
@@ -170,6 +170,8 @@ See [Score Generation](TECHNICAL_GUIDE.md#score-generation) for supported MusicX
Build-time engraving is implemented in Python in `generator/engraving`, using the bundled Petaluma OTF fonts and SMuFL metadata. MuseScore is an optional authoring tool, not a build dependency or rendering backend. Configure installs the optional Modulus asset dependencies. Exact musical timing is retained through layout, with line breaks between complete measures. Missing glyphs, unrepresentable page geometry, and failed notation audits stop generation before replacing existing assets. Build-time engraving is implemented in Python in `generator/engraving`, using the bundled Petaluma OTF fonts and SMuFL metadata. MuseScore is an optional authoring tool, not a build dependency or rendering backend. Configure installs the optional Modulus asset dependencies. Exact musical timing is retained through layout, with line breaks between complete measures. Missing glyphs, unrepresentable page geometry, and failed notation audits stop generation before replacing existing assets.
Every normal build also writes printable PDFs to `build/<target>/generated_assets/print/`: one complete acoustic, electric and vocal part per song, including every arranged iteration and the ending. Print at actual size on portrait US Letter paper. Pages have white backgrounds, black notation, at least half-inch printer margins, and multiple systems per sheet; guitar parts include TAB. These are fresh 300 DPI renders, without the duplicated lines used by playback transition pages. PDFs use the existing Modulus-managed Pillow dependency, are cached separately, and remain build-only artifacts rather than being installed on the device.
`generator/palette.json` preserves the sheet music’s existing dark palette across Zero, Pico, score pages, and previews. Menus use the same Petaluma Script lettering, quiet rules, tune/composer hierarchy and restrained accent colors for a jazz Real Book feel. Performance status shows Count-in, Playing or Sustain, BPM, page progress and a hold-to-exit hint. CMake generates the shared C++ colors from the same palette used by Python. Palette changes invalidate the engraving cache without changing music, spacing, or pagination. `generator/palette.json` preserves the sheet music’s existing dark palette across Zero, Pico, score pages, and previews. Menus use the same Petaluma Script lettering, quiet rules, tune/composer hierarchy and restrained accent colors for a jazz Real Book feel. Performance status shows Count-in, Playing or Sustain, BPM, page progress and a hold-to-exit hint. CMake generates the shared C++ colors from the same palette used by Python. Palette changes invalidate the engraving cache without changing music, spacing, or pagination.
Petaluma is from Steinberg under the SIL Open Font License. Petaluma supplies music glyphs, and Petaluma Script supplies all visible text on score pages and both device interfaces. The Zero uses packaged `PetalumaScript.otf` for dynamic UI text; the Pico displays prerendered Petaluma Script artwork. Runtime packages retain `OFL.txt`; the music font and engraver stay build-only. Petaluma is from Steinberg under the SIL Open Font License. Petaluma supplies music glyphs, and Petaluma Script supplies all visible text on score pages and both device interfaces. The Zero uses packaged `PetalumaScript.otf` for dynamic UI text; the Pico displays prerendered Petaluma Script artwork. Runtime packages retain `OFL.txt`; the music font and engraver stay build-only.
+12
View File
@@ -291,6 +291,18 @@ Bundles preserve the complete `music` object. Presentation identifies the `Vamp
Pages are 976x484 pixels with nominal 10-pixel guitar or 12-pixel vocal staff spacing, 28-pixel horizontal margins and 18-pixel vertical margins. The two acoustic rest lanes sit closer to the staff, with collision offsets preserving their separation. Systems and pages contain complete measures; tied notes and notation spans may continue across their boundaries. Presentation fragments preserve exact timing and never change the semantic JSON. A measure that cannot fit at readable spacing fails with source and measure context; the uniform part-wide scaling described above fits pairs of complete systems vertically. Blank pages, incorrect dimensions, and ink reaching the page edge are rejected. Runtime letterboxing preserves aspect ratio. Luma's `load_texture_uncached()` lets Product Vamp retain at most two textures without a hidden strong asset-cache reference. The UI reports missing/corrupt assets instead of falling back to the removed renderer. Runtime validates bundle metadata, paths, dimensions, and page checksums; the build validates musical semantics. Pages are 976x484 pixels with nominal 10-pixel guitar or 12-pixel vocal staff spacing, 28-pixel horizontal margins and 18-pixel vertical margins. The two acoustic rest lanes sit closer to the staff, with collision offsets preserving their separation. Systems and pages contain complete measures; tied notes and notation spans may continue across their boundaries. Presentation fragments preserve exact timing and never change the semantic JSON. A measure that cannot fit at readable spacing fails with source and measure context; the uniform part-wide scaling described above fits pairs of complete systems vertically. Blank pages, incorrect dimensions, and ink reaching the page edge are rejected. Runtime letterboxing preserves aspect ratio. Luma's `load_texture_uncached()` lets Product Vamp retain at most two textures without a hidden strong asset-cache reference. The UI reports missing/corrupt assets instead of falling back to the removed renderer. Runtime validates bundle metadata, paths, dimensions, and page checksums; the build validates musical semantics.
### Printable Parts
`generator.printing.generate_tree` runs after screen engraving inside the normal asset staging transaction. It reads each completed song/part's musical document, not its screen PNGs or overlapping playback views, and writes `print/<score-stem>.pdf`. Every arranged measure and the instrumental ending appear once in order. Acoustic/electric parts retain standard notation and TAB; vocal parts retain lyrics and rests. No music, runtime bundle fields or catalog rows are changed. Both CMake installation and runtime image copying exclude the `print` directory.
Print layout uses portrait US Letter (612 × 792 PDF points), a minimum half-inch clear margin, and fixed five-point guitar/six-point vocal staff spacing. It reserves the actual overhang of centered measure numbers inside the margin. Complete systems fill the available page height with consistent notation size; the final page keeps natural vertical spacing. Headers identify the tune, composer and instrument; printed page numbers are independent of device pagination. The existing musical/fragment/TAB audits remain active, with variable systems per page allowed only by the print caller. Screen audits still require two systems except the odd final system.
The shared primitive renderer accepts explicit colors and output scale while retaining unchanged screen defaults. Print draws black on white directly at 2550 × 3300 pixels (300 DPI), then writes grayscale image pages through Pillow's PDF writer at quality 95. This is a raster PDF, not searchable/vector notation. One page is rendered and appended at a time to bound image memory. The pinned Pillow parser checks page count, Letter media boxes and image resolution/color; tests decode the embedded JPEGs and verify clear margins. No new Python package is required: Modulus's existing `assets` group owns Pillow, fonttools and aggdraw. Timestamps and output-path metadata are omitted for byte-reproducible PDFs.
With `--engraving-cache`, print results use a separate `print/<print-renderer>/<score-key>/` namespace. Keys cover score content, print settings, fonts, renderer source and dependency versions. Cached PDFs have checked hashes and page metadata; missing or corrupt entries regenerate. Native and ARM builds share the cache locally. This print namespace is separate from the screen-only CI cache snapshot. Output PDFs are validated before the build publishes; failure preserves the previous complete asset directory. Stale PDFs disappear when source parts disappear. Temporary print files live only in build/cache staging.
Print rollout verification: `python3 modulus.py python -m unittest discover -s tests -p 'test_*.py'` passed 458 tests; the seven `tests.test_printing` tests were rerun after the final footer-margin adjustment. `python3 modulus.py build --target x86_64_gcc12_debug --target arm_64_gcc12_release` passed with byte-identical PDFs on both targets. All of Me has 10/8/5 acoustic/electric/vocal pages and Autumn Leaves has 9/7/4, totaling 43. All pages were checked for Letter size, 300 DPI grayscale images and pure-white half-inch margins; representative opening, solo, lyric and ending pages were visually inspected. All 394 preexisting score JSON/screen PNG files stayed byte-identical. PDF generation took 9.7 seconds uncached and 0.1 seconds cached on this development machine; this is the print stage only, not a clean whole-product build benchmark. Physical printing was not tested.
### Validation And Publication ### Validation And Publication
Generation validates sources and complete staged output before replacing the previous asset directory. Replacement is always complete; deleted sources cannot leave stale JSON, pages, audio, or Pico title assets. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalog. Runtime packages contain `catalog.db` with embedded semantic JSON and title-image identifiers, PNG pages, supplied WAV recordings, Pico title images, and the OTF UI text font, not MusicXML, MuseScore, or Python dependencies. Python test discovery runs model/import/export tests; the normal asset build additionally qualifies and audits every catalog score. Generation validates sources and complete staged output before replacing the previous asset directory. Replacement is always complete; deleted sources cannot leave stale JSON, pages, audio, or Pico title assets. CMake tracks the input list as well as file contents, so additions and removals regenerate the catalog. Runtime packages contain `catalog.db` with embedded semantic JSON and title-image identifiers, PNG pages, supplied WAV recordings, Pico title images, and the OTF UI text font, not MusicXML, MuseScore, or Python dependencies. Python test discovery runs model/import/export tests; the normal asset build additionally qualifies and audits every catalog score.
+1 -1
View File
@@ -1,4 +1,4 @@
if(NOT EXISTS "${SOURCE}/catalog.db") if(NOT EXISTS "${SOURCE}/catalog.db")
message(FATAL_ERROR "Missing generated catalog: ${SOURCE}/catalog.db") message(FATAL_ERROR "Missing generated catalog: ${SOURCE}/catalog.db")
endif() endif()
file(COPY "${SOURCE}/" DESTINATION "${DESTINATION}" PATTERN "*.json" EXCLUDE PATTERN "analysis" EXCLUDE) file(COPY "${SOURCE}/" DESTINATION "${DESTINATION}" PATTERN "*.json" EXCLUDE PATTERN "analysis" EXCLUDE PATTERN "print" EXCLUDE)
+1 -1
View File
@@ -195,7 +195,7 @@ install(TARGETS vamp_zero
) )
install(FILES package/config/vamp.example.json DESTINATION zero/config) install(FILES package/config/vamp.example.json DESTINATION zero/config)
install(DIRECTORY ${VAMP_GENERATED_ASSETS_DIR}/ DESTINATION zero/share/product_vamp install(DIRECTORY ${VAMP_GENERATED_ASSETS_DIR}/ DESTINATION zero/share/product_vamp
PATTERN "*.json" EXCLUDE PATTERN "analysis" EXCLUDE) PATTERN "*.json" EXCLUDE PATTERN "analysis" EXCLUDE PATTERN "print" EXCLUDE)
install(PROGRAMS install(PROGRAMS
package/zero/install_zero.sh package/zero/install_zero.sh
package/zero/start_ap.sh package/zero/start_ap.sh
+3
View File
@@ -18,6 +18,7 @@ import generator.engraving.types
import generator.parts import generator.parts
import generator.pico_artwork import generator.pico_artwork
import generator.performance import generator.performance
import generator.printing
import generator.score import generator.score
import generator.summary import generator.summary
import platform import platform
@@ -103,6 +104,8 @@ def generate_assets(args: argparse.Namespace) -> None:
generator.performance.validate_recording_lengths(staged, audio_assets) generator.performance.validate_recording_lengths(staged, audio_assets)
report('engraving pages') report('engraving pages')
validated_pages = generator.engraving.pipeline.engrave_tree(staged, static_assets_dir / 'fonts' / 'petaluma', cache_dir=cache_dir, jobs=getattr(args, 'jobs', 1), progress=progress) validated_pages = generator.engraving.pipeline.engrave_tree(staged, static_assets_dir / 'fonts' / 'petaluma', cache_dir=cache_dir, jobs=getattr(args, 'jobs', 1), progress=progress)
report('generating printable parts')
generator.printing.generate_tree(staged, static_assets_dir / 'fonts' / 'petaluma', cache_dir=cache_dir, progress=progress)
report('rendering Pico titles') report('rendering Pico titles')
generator.pico_artwork.generate_titles(staged, static_assets_dir / 'fonts' / 'petaluma') generator.pico_artwork.generate_titles(staged, static_assets_dir / 'fonts' / 'petaluma')
report('validating score, page, and audio assets') report('validating score, page, and audio assets')
+4 -3
View File
@@ -19,15 +19,16 @@ def audit_primitives(primitives, width, height, fonts):
if any(abs(a - b) > 0.001 for a, b in zip((actual.left, actual.top, actual.right, actual.bottom), (item.box.left, item.box.top, item.box.right, item.box.bottom))): if any(abs(a - b) > 0.001 for a, b in zip((actual.left, actual.top, actual.right, actual.bottom), (item.box.left, item.box.top, item.box.right, item.box.bottom))):
raise generator.engraving.types.EngravingError(f'{item.sources}: glyph bounds disagree with its actual outline') raise generator.engraving.types.EngravingError(f'{item.sources}: glyph bounds disagree with its actual outline')
def audit_layout(document: dict, layout: generator.engraving.types.LayoutDocument, fonts: generator.engraving.fonts.Fonts) -> None: def audit_layout(document: dict, layout: generator.engraving.types.LayoutDocument, fonts: generator.engraving.fonts.Fonts, *, systems_per_page: int | None = 2) -> None:
index = generator.engraving.notation.ScoreIndex(document) index = generator.engraving.notation.ScoreIndex(document)
fragments, primitives = {}, {} fragments, primitives = {}, {}
previous_end = fractions.Fraction() previous_end = fractions.Fraction()
scale = layout.pages[0].systems[0].scale scale = layout.pages[0].systems[0].scale
for page in layout.pages: for page in layout.pages:
if len(page.systems) != 2 and (page is not layout.pages[-1] or len(page.systems) != 1): if not page.systems or (systems_per_page is not None and len(page.systems) != systems_per_page and (page is not layout.pages[-1] or not 1 <= len(page.systems) < systems_per_page)):
raise generator.engraving.types.EngravingError('pages must contain two music systems, except an odd final system') raise generator.engraving.types.EngravingError('pages must contain two music systems, except an odd final system')
if len(page.systems) == 2 and page.systems[0].box.bottom + layout.style.system_gap > page.systems[1].box.top + 0.01: for first, second in zip(page.systems, page.systems[1:]):
if first.box.bottom + layout.style.system_gap > second.box.top + 0.01:
raise generator.engraving.types.EngravingError('music systems overlap their page rows') raise generator.engraving.types.EngravingError('music systems overlap their page rows')
audit_primitives(page.primitives, page.width, page.height, fonts) audit_primitives(page.primitives, page.width, page.height, fonts)
header = generator.engraving.types.bounds([p for p in page.primitives if p.role in ('title', 'subtitle')]) header = generator.engraving.types.bounds([p for p in page.primitives if p.role in ('title', 'subtitle')])
+13 -9
View File
@@ -30,12 +30,14 @@ class OutlinePen(fontTools.pens.basePen.BasePen):
def _endPath(self): def _endPath(self):
pass pass
def rasterize_page(page: generator.engraving.types.Page, fonts: generator.engraving.fonts.Fonts) -> PIL.Image.Image: def rasterize_page(page: generator.engraving.types.Page, fonts: generator.engraving.fonts.Fonts, *, ink=None, background=None, output_scale=1) -> PIL.Image.Image:
return rasterize_band(page, page.primitives, 0, page.height, fonts) return rasterize_band(page, page.primitives, 0, page.height, fonts, ink=ink, background=background, output_scale=output_scale)
def rasterize_band(page: generator.engraving.types.Page, primitives: tuple[generator.engraving.types.Primitive, ...], top: int, bottom: int, fonts: generator.engraving.fonts.Fonts) -> PIL.Image.Image: def rasterize_band(page: generator.engraving.types.Page, primitives: tuple[generator.engraving.types.Primitive, ...], top: int, bottom: int, fonts: generator.engraving.fonts.Fonts, *, ink=None, background=None, output_scale=1) -> PIL.Image.Image:
scale = page.supersampling ink = generator.theme.rgb('ink') if ink is None else ink
image = PIL.Image.new('RGB', (page.width * scale, (bottom - top) * scale), generator.theme.rgb('background')) background = generator.theme.rgb('background') if background is None else background
scale = page.supersampling * output_scale
image = PIL.Image.new('RGB', (round(page.width * scale), round((bottom - top) * scale)), background)
context = aggdraw.Draw(image) context = aggdraw.Draw(image)
for item in primitives: for item in primitives:
points = [(x * scale, y * scale - top * scale) for x, y in item.points] points = [(x * scale, y * scale - top * scale) for x, y in item.points]
@@ -56,16 +58,18 @@ def rasterize_band(page: generator.engraving.types.Page, primitives: tuple[gener
if item.kind == 'polygon': if item.kind == 'polygon':
path.close() path.close()
if item.kind in ('glyph', 'polygon'): if item.kind in ('glyph', 'polygon'):
context.path(path, None, aggdraw.Brush(generator.theme.rgb('ink'))) context.path(path, None, aggdraw.Brush(ink))
else: else:
context.path(path, aggdraw.Pen(generator.theme.rgb('ink'), item.thickness * scale)) context.path(path, aggdraw.Pen(ink, item.thickness * scale))
context.flush() context.flush()
drawing = PIL.ImageDraw.Draw(image) drawing = PIL.ImageDraw.Draw(image)
for item in primitives: for item in primitives:
if item.kind == 'text': if item.kind == 'text':
x, y = item.points[0] x, y = item.points[0]
drawing.text((x * scale, y * scale - top * scale), item.value, font=fonts.text_font(item.size * scale), anchor='ls', fill=generator.theme.rgb('ink')) drawing.text((x * scale, y * scale - top * scale), item.value, font=fonts.text_font(item.size * scale), anchor='ls', fill=ink)
return image.resize((page.width, bottom - top), PIL.Image.Resampling.LANCZOS) result = image.resize((round(page.width * output_scale), round((bottom - top) * output_scale)), PIL.Image.Resampling.LANCZOS)
image.close()
return result
@dataclasses.dataclass(frozen=True) @dataclasses.dataclass(frozen=True)
class Band: class Band:
+190
View File
@@ -0,0 +1,190 @@
"""Build-only Letter PDFs from the complete arranged parts, never screen PNGs."""
from __future__ import annotations
import dataclasses
import hashlib
import json
import math
import pathlib
import shutil
import tempfile
import time
import PIL.PdfParser
import generator.engraving.audit
import generator.engraving.cache
import generator.engraving.fonts
import generator.engraving.layout
import generator.engraving.notation
import generator.engraving.raster
import generator.engraving.types
import generator.model
# Two layout units per PDF point: guitar staff spacing is five points.
# Raster resolution is independent of layout and the device's display theme.
STYLE = generator.engraving.types.Style(width=1224, height=1584, margin_x=72, margin_y=72, page_clearance=28, system_gap=32, supersampling=1)
DPI = 300
PIXELS = (2550, 3300)
MEDIA_BOX = [0, 0, 612, 792]
VERSION = 'letter-parts-v1'
def layout_part(document, fonts):
generator.model.validate_document(document)
style = dataclasses.replace(generator.engraving.types.score_style(document, STYLE), system_gap=STYLE.system_gap)
# Measure numbers are centered over barlines and can extend left of the
# staff. Reserve their actual overhang inside the half-inch paper margin.
overhang = max(-fonts.text(str(m['number']), 0, 0, 12, 'measure_number', (), True).box.left for m in document['staves'][0]['measures'])
style = dataclasses.replace(style, margin_x=style.margin_x + math.ceil(max(0, overhang)) + 4)
index = generator.engraving.notation.ScoreIndex(document)
systems = generator.engraving.layout.layout_systems(document, style, fonts, index=index)
# The shared header wraps long credits; print it at twice its display size.
header_style = dataclasses.replace(style, width=style.width // 2, margin_x=style.margin_x / 2, margin_y=style.margin_y / 2)
header = tuple(p.scaled(2) for p in generator.engraving.layout.part_header(document, fonts, header_style))
top = generator.engraving.types.bounds(header).bottom + style.page_clearance
bottom = style.height - style.margin_y - 20 - style.page_clearance
pages, placed, cursor = [], [], top
def finish():
number = len(pages) + 1
# Include clearance for numeric descenders and JPEG block-edge ringing.
footer = fonts.text(str(number), style.width / 2, style.height - style.margin_y - 8, 20, 'page_number', (), True)
pages.append(generator.engraving.types.Page(number, style.width, style.height, style.supersampling, tuple(placed),
(*header, *(p for system in placed for p in system.primitives), footer)))
for system in systems:
system = generator.engraving.layout.make_system(index, system.start, system.end, fonts, style, justify=system.end < index.end, staff_width=style.width - 2 * style.margin_x - 2)
height = system.box.bottom - system.box.top
if height > bottom - top:
raise generator.engraving.types.EngravingError(f'at musical offset {system.start}: system cannot fit a Letter page at readable size')
if placed and cursor + height > bottom:
finish()
placed, cursor = [], top
dx, dy = style.margin_x + 1, cursor - system.box.top
placed.append(dataclasses.replace(system, primitives=tuple(p.moved(dx, dy) for p in system.primitives),
anchors=tuple(a.moved(dx, dy) for a in system.anchors), box=system.box.moved(dx, dy)))
cursor += height + style.system_gap
finish()
result = generator.engraving.types.LayoutDocument(tuple(pages), style, fonts.fingerprint(style))
generator.engraving.audit.audit_layout(document, result, fonts, systems_per_page=None)
if any(p.box.left < STYLE.margin_x or p.box.right > style.width - STYLE.margin_x or
p.box.top < STYLE.margin_y or p.box.bottom > style.height - STYLE.margin_y for page in pages for p in page.primitives):
raise generator.engraving.types.EngravingError('print notation exceeds the half-inch printer margins')
return result
def rasterize(page, fonts):
return generator.engraving.raster.rasterize_page(page, fonts, ink=(0, 0, 0), background=(255, 255, 255), output_scale=DPI / 144)
def validate_pdf(path, expected_pages=None):
"""Check the narrow PDF format written by the pinned Pillow asset dependency.
This adapter deliberately uses Pillow's parser, not a new general PDF stack.
Tests also decode the embedded page images to check print color/resolution.
"""
try:
if path.is_symlink() or not path.is_file():
raise ValueError('expected a regular print PDF')
pdf = PIL.PdfParser.PdfParser(filename=str(path))
try:
count = len(pdf.pages)
if not 0 < count <= 10000 or (expected_pages is not None and count != expected_pages):
raise ValueError('invalid print PDF page count')
for reference in pdf.pages:
page = pdf.read_indirect(reference)
if page[b'MediaBox'] != MEDIA_BOX:
raise ValueError('print PDF must use portrait US Letter')
image = pdf.read_indirect(page[b'Resources'][b'XObject'][b'image'])
if (image.dictionary[b'Width'], image.dictionary[b'Height']) != PIXELS or image.dictionary[b'ColorSpace'] != b'DeviceGray':
raise ValueError('print PDF must contain 300 DPI grayscale pages')
return count
finally:
pdf.close()
except (OSError, KeyError, TypeError, IndexError, PIL.PdfParser.PdfFormatError) as error:
raise ValueError(f'{path.name}: invalid print PDF: {error}') from error
def write_pdf(path, document, fonts):
layout = layout_part(document, fonts)
# Append one page at a time so a long arrangement never keeps a chorus of
# full-resolution raster images in memory. The enclosing build is staged.
for page in layout.pages:
with rasterize(page, fonts) as rgb, rgb.convert('L') as image:
if image.size != PIXELS or image.getextrema()[0] == 255:
raise ValueError('print renderer produced an empty or incorrectly sized page')
image.save(path, format='PDF', append=page.number > 1, resolution=DPI, quality=95,
title=document['metadata']['title'] + ' - ' + document['staves'][0].get('name', 'Part'),
creator='Product Vamp', creationDate=None, modDate=None)
validate_pdf(path, len(layout.pages))
return len(layout.pages)
def renderer_key(fonts):
identity = generator.engraving.cache.renderer_identity(fonts, STYLE)
identity.update(print_version=VERSION, print_code=hashlib.sha256(pathlib.Path(__file__).read_bytes()).hexdigest(), dpi=DPI, ink='black', background='white', quality=95)
return hashlib.sha256(json.dumps(identity, sort_keys=True).encode()).hexdigest()
def restore(entry, key, target):
try:
if generator.engraving.cache.has_symlink(entry):
return None
pdf, manifest = entry / 'part.pdf', entry / 'print.json'
if pdf.is_symlink() or manifest.is_symlink():
return None
info = json.loads(manifest.read_text())
if info['key'] != key or info['sha256'] != hashlib.sha256(pdf.read_bytes()).hexdigest():
return None
pages = validate_pdf(pdf, info['pages'])
shutil.copyfile(pdf, target)
return pages
except (OSError, ValueError, KeyError, TypeError):
return None
def store(entry, key, target, pages):
if generator.engraving.cache.has_symlink(entry.parent):
raise ValueError('print cache must not contain symlinks')
entry.parent.mkdir(parents=True, exist_ok=True)
if entry.is_symlink() or entry.is_file():
entry.unlink()
entry.mkdir(exist_ok=True)
with tempfile.TemporaryDirectory(prefix='.print_', dir=entry.parent) as temporary:
staged = pathlib.Path(temporary)
shutil.copyfile(target, staged / 'part.pdf')
(staged / 'print.json').write_text(json.dumps({'key': key, 'sha256': hashlib.sha256(target.read_bytes()).hexdigest(), 'pages': pages}))
for name in ('part.pdf', 'print.json'):
destination = entry / name
if destination.is_dir() and not destination.is_symlink():
shutil.rmtree(destination)
(staged / name).replace(destination)
def generate_tree(root, font_path, *, cache_dir=None, progress=False):
"""Called inside the normal build's atomic staging directory."""
if cache_dir and (cache_dir.resolve().is_relative_to(root.resolve()) or root.resolve().is_relative_to(cache_dir.resolve())):
raise ValueError('print cache and generated assets must not overlap')
destination = root / 'print'
destination.mkdir(exist_ok=True)
paths = sorted((root / 'songs').glob('*.json'))
fonts = generator.engraving.fonts.Fonts(font_path)
renderer = renderer_key(fonts)
expected, pages_total, reused = set(), 0, 0
started = time.monotonic()
for path in paths:
bundle = json.loads(path.read_text())
document = bundle.get('music', bundle)
target = destination / path.with_suffix('.pdf').name
expected.add(target)
key = generator.engraving.cache.score_fingerprint(document, renderer)
entry = cache_dir / 'print' / renderer / key if cache_dir else None
pages = restore(entry, key, target) if entry else None
if pages is None:
pages = write_pdf(target, document, fonts)
if entry:
store(entry, key, target, pages)
else:
reused += 1
pages_total += pages
if progress:
print(f'Print: {target.name}: {pages} pages', flush=True)
for stale in destination.glob('*.pdf'):
if stale not in expected:
stale.unlink()
if progress:
print(f'Print complete: {len(paths)} PDFs, {pages_total} pages, {reused} cached; {time.monotonic() - started:.1f}s', flush=True)
return len(paths)
+4
View File
@@ -111,6 +111,8 @@ class CatalogTests(unittest.TestCase):
write_score(self.root, 'songs/tune.json') write_score(self.root, 'songs/tune.json')
generator.catalog.build_catalog(self.root) generator.catalog.build_catalog(self.root)
(self.root / 'songs/tune.png').write_bytes(b'page') (self.root / 'songs/tune.png').write_bytes(b'page')
(self.root / 'print').mkdir()
(self.root / 'print/tune.pdf').write_bytes(b'build-only PDF')
with tempfile.TemporaryDirectory() as temporary: with tempfile.TemporaryDirectory() as temporary:
destination = pathlib.Path(temporary) destination = pathlib.Path(temporary)
subprocess.run(['cmake', f'-DSOURCE={self.root}', f'-DDESTINATION={destination}', '-P', 'cmake/runtime_assets.cmake'], check=True, capture_output=True) subprocess.run(['cmake', f'-DSOURCE={self.root}', f'-DDESTINATION={destination}', '-P', 'cmake/runtime_assets.cmake'], check=True, capture_output=True)
@@ -120,6 +122,8 @@ class CatalogTests(unittest.TestCase):
self.assertEqual(pathlib.Path(config['database']['path']).name, 'catalog.db') self.assertEqual(pathlib.Path(config['database']['path']).name, 'catalog.db')
self.assertTrue((destination / 'songs/tune.png').exists()) self.assertTrue((destination / 'songs/tune.png').exists())
self.assertFalse(list(destination.rglob('*.json'))) self.assertFalse(list(destination.rglob('*.json')))
self.assertFalse((destination / 'print').exists())
self.assertIn('PATTERN "print" EXCLUDE', pathlib.Path('cmake/zero.cmake').read_text())
def test_menu_query_uses_index_and_preserves_binary_identity_order(self): def test_menu_query_uses_index_and_preserves_binary_identity_order(self):
for name, title, composer, part in (('z', 'Same', 'Bea', 'Acoustic'), ('b', 'Same', 'Ada', 'Vocal'), ('a', 'Same', 'Ada', 'Acoustic'), ('c', 'same', 'Ada', 'Electric'), ('d', 'Same', 'Ada', 'Acoustic')): for name, title, composer, part in (('z', 'Same', 'Bea', 'Acoustic'), ('b', 'Same', 'Ada', 'Vocal'), ('a', 'Same', 'Ada', 'Acoustic'), ('c', 'same', 'Ada', 'Electric'), ('d', 'Same', 'Ada', 'Acoustic')):
document = bundle(title=title, part=part) document = bundle(title=title, part=part)
+6
View File
@@ -360,10 +360,16 @@ class EngravingCacheTests(unittest.TestCase):
(output / 'solo_xml').mkdir() (output / 'solo_xml').mkdir()
(output / 'solo_xml/stale.xml').write_text('obsolete solo export') (output / 'solo_xml/stale.xml').write_text('obsolete solo export')
(output / 'solo_xml/manifest.solo').write_text('{}') (output / 'solo_xml/manifest.solo').write_text('{}')
(output / 'print/stale.pdf').write_bytes(b'obsolete print part')
generator.__main__.generate_assets(args) generator.__main__.generate_assets(args)
self.assertEqual(original, self.files(output)) self.assertEqual(original, self.files(output))
self.assertFalse((output / 'solo_xml').exists()) self.assertFalse((output / 'solo_xml').exists())
self.assertFalse(list(output.rglob('*.xml'))) self.assertFalse(list(output.rglob('*.xml')))
self.assertEqual(len(list(output.glob('print/*.pdf'))), 3)
with unittest.mock.patch('generator.printing.generate_tree', side_effect=ValueError('print rendering failed')):
with self.assertRaisesRegex(ValueError, 'print rendering failed'):
generator.__main__.generate_assets(args)
self.assertEqual(original, self.files(output))
source.write_text(tests.test_musicxml.xml_score('<harmony><root><root-step>C</root-step></root><kind>major</kind></harmony>' + tests.test_musicxml.xml_note(extra='<accidental smufl="unavailableGlyph">other</accidental>'))) source.write_text(tests.test_musicxml.xml_score('<harmony><root><root-step>C</root-step></root><kind>major</kind></harmony>' + tests.test_musicxml.xml_note(extra='<accidental smufl="unavailableGlyph">other</accidental>')))
with self.assertRaisesRegex(generator.engraving.types.EngravingError, 'unavailableGlyph'): with self.assertRaisesRegex(generator.engraving.types.EngravingError, 'unavailableGlyph'):
generator.__main__.generate_assets(args) generator.__main__.generate_assets(args)
+1
View File
@@ -144,6 +144,7 @@ class MusicModelTests(unittest.TestCase):
generator.__main__.generate_assets(args) generator.__main__.generate_assets(args)
self.assertFalse(list(output.rglob('*.json'))) self.assertFalse(list(output.rglob('*.json')))
self.assertFalse(list(output.rglob('*.png'))) self.assertFalse(list(output.rglob('*.png')))
self.assertFalse(list(output.rglob('*.pdf')))
def test_pickup_tie_survives_vocal_restoration(self): def test_pickup_tie_survives_vocal_restoration(self):
source = tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=1, kind='quarter', extra='<tie type="start"/>'), implicit=True) source = tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=1, kind='quarter', extra='<tie type="start"/>'), implicit=True)
source = source.replace('</measure></part>', '</measure><measure number="2">' + tests.test_musicxml.xml_note(extra='<tie type="stop"/>') + '</measure></part>') source = source.replace('</measure></part>', '</measure><measure number="2">' + tests.test_musicxml.xml_note(extra='<tie type="stop"/>') + '</measure></part>')
+156
View File
@@ -0,0 +1,156 @@
"""Build-only print layout, PDF contents and reproducible cache behavior."""
import copy
import dataclasses
import io
import json
import pathlib
import tempfile
import unittest
import unittest.mock
import PIL.Image
import PIL.PdfParser
import generator.arrange.generate
import generator.engraving.audit
import generator.engraving.fonts
import generator.engraving.types
import generator.model
import generator.musicxml
import generator.printing
import generator.score
import tests.solo_fixtures
import tests.test_musicxml
class PrintableParts(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.font_path = pathlib.Path('package/assets/fonts/petaluma')
cls.fonts = generator.engraving.fonts.Fonts(cls.font_path)
def setUp(self):
temporary = tempfile.TemporaryDirectory()
self.addCleanup(temporary.cleanup)
self.root = pathlib.Path(temporary.name)
self.source = tests.solo_fixtures.song(count=2, chords=('C7',))
def tree(self, name, document=None):
root = self.root / name
(root / 'songs').mkdir(parents=True)
generator.score.write_json(root / 'songs/part.json', document or self.source)
return root
def test_letter_pagination_preserves_complete_order_without_transition_duplicates(self):
source = tests.solo_fixtures.song(count=32, chords=('C7',))
before = copy.deepcopy(source)
layout = generator.printing.layout_part(source, self.fonts)
self.assertGreater(len(layout.pages), 1)
self.assertGreater(len(layout.pages[0].systems), 2)
self.assertEqual(source, before)
systems = [s for p in layout.pages for s in p.systems]
self.assertEqual(systems[0].start, 0)
self.assertEqual(systems[-1].end, 128)
self.assertTrue(all(a.end == b.start for a, b in zip(systems, systems[1:])))
for page in layout.pages:
self.assertEqual((page.width, page.height), (1224, 1584))
self.assertTrue(all(p.box.left >= 72 and p.box.right <= 1152 for p in page.primitives))
self.assertTrue(all(p.box.top >= 72 and p.box.bottom <= 1512 for p in page.primitives))
# Sharing content audits must not relax the screen's two-system contract.
with self.assertRaisesRegex(ValueError, 'two music systems'):
generator.engraving.audit.audit_layout(source, layout, self.fonts)
def test_content_audits_remain_active_for_print(self):
source = generator.musicxml.load_musicxml(pathlib.Path('tests/fixtures/engraving.xml'))
layout = generator.printing.layout_part(source, self.fonts)
roles = {p.role for page in layout.pages for p in page.primitives}
self.assertTrue({'tie', 'lyric', 'harmony', 'accidental'} <= roles)
page = layout.pages[0]
broken = dataclasses.replace(page, primitives=tuple(p for p in page.primitives if p.role != 'tie'))
with self.assertRaisesRegex(ValueError, 'tie was omitted'):
generator.engraving.audit.audit_layout(source, dataclasses.replace(layout, pages=(broken,)), self.fonts, systems_per_page=None)
def test_guitar_tab_pickups_and_final_three_bar_hold_are_included(self):
source = tests.test_musicxml.xml_score(tests.test_musicxml.xml_note(duration=1, kind='quarter'), implicit=True)
source = source.replace('</measure></part>', '</measure><measure number="2"><harmony><root><root-step>C</root-step></root><kind>major</kind></harmony>' + tests.test_musicxml.xml_note() + '</measure></part>')
source = tests.test_musicxml.parse(source)
for part in ('acoustic', 'electric', 'vocal'):
document = generator.arrange.generate.arrange_document(source, part)
if part != 'vocal':
document['staves'][0]['tablature'] = True
before = copy.deepcopy(document)
layout = generator.printing.layout_part(document, self.fonts)
self.assertEqual(document, before)
self.assertEqual(layout.pages[-1].systems[-1].end, sum(generator.model.fraction(m['duration']) for m in document['staves'][0]['measures']))
self.assertTrue(all(m['ending_sustain'] for m in document['staves'][0]['measures'][-3:]))
roles = {p.role for page in layout.pages for p in page.primitives}
self.assertIn('rest', roles)
if part != 'vocal':
self.assertTrue({'tab', 'tab_staff', 'tie'} <= roles)
def test_pdf_pages_are_letter_300dpi_white_grayscale_and_reproducible(self):
source = tests.solo_fixtures.song(count=32, chords=('C7',))
first, second = self.root / 'first.pdf', self.root / 'second.pdf'
count = generator.printing.write_pdf(first, source, self.fonts)
self.assertGreater(count, 1)
generator.printing.write_pdf(second, source, self.fonts)
self.assertEqual(first.read_bytes(), second.read_bytes())
pdf = PIL.PdfParser.PdfParser(filename=str(first))
try:
self.assertEqual(len(pdf.pages), count)
self.assertNotIn(b'CreationDate', pdf.info)
self.assertNotIn(b'ModDate', pdf.info)
for reference in pdf.pages:
page = pdf.read_indirect(reference)
self.assertEqual(page[b'MediaBox'], [0, 0, 612, 792])
stream = pdf.read_indirect(page[b'Resources'][b'XObject'][b'image'])
with PIL.Image.open(io.BytesIO(stream.buf)) as image:
self.assertEqual(image.size, (2550, 3300))
self.assertEqual(image.mode, 'L')
self.assertEqual(image.getextrema(), (0, 255))
for box in ((0, 0, 150, 3300), (2400, 0, 2550, 3300), (0, 0, 2550, 150), (0, 3150, 2550, 3300)):
with image.crop(box) as margin:
self.assertEqual(margin.getextrema(), (255, 255))
finally:
pdf.close()
with self.assertRaisesRegex(ValueError, 'page count'):
generator.printing.validate_pdf(first, count + 1)
def test_single_page_cache_hit_corruption_and_stale_removal(self):
root, cache = self.tree('assets'), self.root / 'cache'
self.assertEqual(generator.printing.generate_tree(root, self.font_path, cache_dir=cache), 1)
target = root / 'print/part.pdf'
original = target.read_bytes()
self.assertEqual(generator.printing.validate_pdf(target), 1)
warm = self.tree('warm')
with unittest.mock.patch('generator.printing.write_pdf', side_effect=AssertionError('cache miss')):
generator.printing.generate_tree(warm, self.font_path, cache_dir=cache)
self.assertEqual(original, (warm / 'print/part.pdf').read_bytes())
asset = next(cache.rglob('part.pdf'))
asset.write_bytes(original + b'corrupt')
with unittest.mock.patch('generator.printing.write_pdf', wraps=generator.printing.write_pdf) as render:
generator.printing.generate_tree(root, self.font_path, cache_dir=cache)
render.assert_called_once()
self.assertEqual(asset.read_bytes(), original)
info = next(cache.rglob('print.json'))
metadata = json.loads(info.read_text())
info.write_text(json.dumps({**metadata, 'pages': metadata['pages'] + 1}))
generator.printing.generate_tree(root, self.font_path, cache_dir=cache)
self.assertEqual(json.loads(info.read_text()), metadata)
(root / 'songs/part.json').unlink()
generator.printing.generate_tree(root, self.font_path, cache_dir=cache)
self.assertFalse(list((root / 'print').iterdir()))
def test_print_cache_tracks_music_style_and_fonts(self):
root, cache = self.tree('assets'), self.root / 'cache'
generator.printing.generate_tree(root, self.font_path, cache_dir=cache)
changed = copy.deepcopy(self.source)
changed['metadata']['title'] = 'Another title'
generator.score.write_json(root / 'songs/part.json', changed)
with unittest.mock.patch('generator.printing.write_pdf', wraps=generator.printing.write_pdf) as render:
generator.printing.generate_tree(root, self.font_path, cache_dir=cache)
render.assert_called_once()
key = generator.printing.renderer_key(self.fonts)
with unittest.mock.patch('generator.printing.STYLE', dataclasses.replace(generator.printing.STYLE, system_gap=40)):
self.assertNotEqual(key, generator.printing.renderer_key(self.fonts))
with unittest.mock.patch.object(self.fonts, 'fingerprint', return_value='changed-fonts'):
self.assertNotEqual(key, generator.printing.renderer_key(self.fonts))
def test_print_cache_cannot_overlap_published_assets(self):
root = self.tree('assets')
for cache in (root, root / 'cache', root.parent):
with self.assertRaisesRegex(ValueError, 'must not overlap'):
generator.printing.generate_tree(root, self.font_path, cache_dir=cache)
if __name__ == '__main__':
unittest.main()