Publishing a prebuilt image can wait. Remove the GitHub Actions image
workflow and the IMAGE_PREFIX/IMAGE_TAG image name in docker-compose.yml,
and have the README's quick start build with docker compose up -d
--build, which also rebuilds after a git pull. The mirror's docs and
script no longer mention Actions or the package.
mirror/github-mirror.sh publishes main and the tags on it to a read-only
GitHub mirror, as one extra commit that adds a notice under the README
title, so every forge commit is unchanged there and each sync is a
fast-forward. A systemd user service and hourly timer run it from a host
that can push to GitHub; --settings turns off issues, pull requests and
the rest there.
A GitHub Actions workflow builds the image on the mirror and pushes it to
ghcr.io, freeing runner disk first since the CUDA build of torch is
several GB. docker-compose.yml names the image after IMAGE_PREFIX and
IMAGE_TAG, so setting them pulls the published image instead of
building, and the README says how.
The README described a CLI with a minimal web UI and told people to fix
charts in Yass. Rewrite it around how songtrace is used now: what it
does, its requirements up front (an NVIDIA GPU, tested on 12 GB, Docker
with the NVIDIA Container Toolkit, a Whisper server of your own, disk for
models and per-song files), a Docker quick start including writing songs
straight into a game's collection, then reviewing, editing and
rebuilding in the web UI. The command line moves near the end.
Regenerate the screenshot to show the note sheet: the screenshot script
now seeds the featured song with syllables, a chart, a pitch track and a
line break, and always scrolls to the review. The example config no
longer sets a real person as #CREATOR, and uses a reserved example host
for Whisper.
Since breaks could be moved, the sheet widened its window to show the
breaks either side of a line. Before an instrumental passage the break
sits mid-gap, sometimes 15s away, so the line's notes were crushed into
a corner of the sheet. The window now grows at most 3s past the line to
show a break; one further off is pinned to the edge, labelled with an
arrow toward where it lies, and still drags inward.
Line breaks were always placed by songtrace, midway between lines and
recomputed around edited ones, so a break could not be put where a singer
breathes. The sheet now draws the breaks either side of a line as markers
that drag along the beat grid, kept between the lines they separate, and
double-click back to automatic. Break moves share the line's undo history
and SAVE LINE, which saves the notes only if they changed, so moving a
break never locks either line's notes.
Moved breaks are stored in edits.json keyed by the line each comes before,
follow their lines when the lyrics change, and win over the computed break
on every rebuild, clamped so a break never lands inside a line's notes.
The review lists each break with that line and whether it was moved, and
PUT /api/jobs/{id}/breaks takes {line: seconds | null}. merge now passes
untouched notes through as they are rather than rebuilding them, which
added float noise to their durations.
The beat grid came from the detected tempo (or the BPM override) and a GAP
pinned to the first note, so it did not always line up with the music,
and snapping did not land on its beats. A GRID bar on the sheet now sets
both, redrawing the grid live as values are tried, with x2 and /2 for a
tempo detected at double or half speed and ALIGN TO NOTE to move GAP the
least that puts the selected note on a beat. Note editing pauses while a
trial grid is unapplied, so notes never snap to a grid that is then
cancelled.
The grid is stored with the song's edits, so every rebuild keeps it, and
grid charts with it in place of its own choice; it records that choice in
tempo.json for RESET to return to. A GAP after the first note moves back
by whole beats, as notes on negative beats are rejected by players. PUT
and DELETE /api/jobs/{id}/grid set and reset it, rewriting the charts in
place.
The sheet's controls were listed in one run-on line under it, which was
hard to scan and grew with every feature. Replace it with a panel grouped
the way an editor's reference card is (select, move, length, notes, kind,
syllable, line) with keys drawn as key caps, opened by a KEYS button or ?,
closed by Esc, and remembered open or closed across visits.
The list is data (SHORTCUTS), and a test checks every letter key it names
is one the sheet's key handler handles, so the two cannot drift apart.
A song's edits are kept through any rebuild, but nothing said which songs
had them. GET /api/jobs and GET /api/rebuild-all now report each song's
edited and set-aside lines, read from its edits file, on the job standing
for the song only, since the runs it replaced would claim the same edits.
Queue cards show them as a badge, and the rebuild dialog lists them per
song and says hand edits are kept.
Notes on the sheet can now be selected and edited: dragged in time and
pitch, resized from either end, nudged and stretched from the keyboard,
split, merged, added, deleted, given a syllable or a kind (golden,
freestyle, rap), with undo/redo. Everything snaps to the chart's beat grid
and to semitones, as the chart itself will. SAVE LINE stores the line
through the edits API, which rewrites the charts in place; REVERT goes back
to the generated notes; a line's edits set aside because its lyrics changed
are listed with a way to discard them. Unsaved changes are not lost to
paging, following playback, switching songs or closing the tab.
The operations live in static/edit-model.js as pure functions, unit-tested
with node --test. The sheet hit-tests by geometry rather than event
targets, since a render replaces the elements under the pointer.
The review drew each line's notes as thin bars in a 20px strip, with no
pitch axis or beat grid, so it was hard to tell where a note sat. Add a
sheet that shows one line at a time the way an UltraStar editor does: a
row per semitone, shaded like piano keys and named in a gutter, the
chart's own beat grid with bars every 4 and 16 beats, notes as blocks
styled by kind (golden, freestyle, rap) with low-confidence ones outlined,
neighbouring lines faded in for context, syllables and aligned words
beneath, and a playhead that follows the transport, paging with it.
The detected pitch curve is drawn over the notes, served by a new GET
/api/jobs/{id}/pitch in note semitones, so a note off the sung pitch
stands out. The sheet is its own ES module; its geometry is unit-tested
with node --test, run from pytest and skipped where Node is absent.
The groundwork for an in-app editor. Edits live in edits.json in the work
dir, which no stage produces, so no rebuild removes them. An edit is a
whole lyric line as the user left it, in seconds, so a rebuild with a new
BPM re-snaps it. grid lays the edits over the freshly generated notes
(neighbouring lines give way to an edited one), writes the result to
notes_final.json with warnings brought up to date, and charts that. When
the lyrics change, edits follow their lines by text and order; an edit
whose line's words changed is set aside as stale and reported, not applied.
Notes gain a kind, so golden, freestyle and rap notes can be charted. The
review endpoint returns the merged notes with their syllables, kinds and
edit state, plus the chart's beat grid. PUT/DELETE
/api/jobs/{id}/lines/{n} save and revert a line, DELETE
/api/jobs/{id}/stale/{n} discards a stale edit; each rewrites the work and
packaged charts in place with the tempo already chosen, so a save
re-encodes nothing.
A fixed 0.5 alignment-score cutoff flagged 73-99% of notes on sung
vocals, so the flag pointed nowhere. Scores still separate bad words from
good ones, but their scale differs from song to song: flag a word below a
quarter of the song's median score, or one shorter than 0.08s or longer
than 3s, which no singer produces and which a word stretched over
something else does. Flagged notes less than 2s apart share one warning.
The worker set status "done" before removing the job's upload and before
setting finished_at, so a client polling for done could see the upload
still there and finished_at null. Record the outcome and publish status,
error and finished_at together, under the lock, as the last step.
Debian bookworm's python3-venv seeds pip 23.0.1, which rejects the
PyTorch CPU index's typing_extensions wheel because its metadata spells
the name with an underscore. It then falls back to the sdist, whose build
needs flit_core, which that index does not carry.
The aligner had to explain every frame of a section with the lyrics, so
background speech or laughter between lines pulled the next line's
letters onto it, which then came out as a burst of tiny notes before the
real line was rushed to catch up. MMS_FA has a star token that matches
any audio; put one before, between and after the lines of each section,
as ctc-forced-aligner does by default, and drop the stars' spans from the
result.
A line with nothing in the transcript was always dropped as not sung. When
Whisper skips a passage (one song lost a whole verse, 50s of clear singing
with no segments), that drops lines that were sung. Find singing in the
vocal stem that no segment covers; a run of unmatched lines with such
singing between its heard neighbours is kept, and that singing joins the
alignment sections so those lines have somewhere to go. Only singing that
kept a line is added, so an untranscribed ad-lib draws no lines into it.
The dialog now lists the songs it would rebuild, all ticked, each with when
it was last built and whether that was a rebuild, so songs already rebuilt
by hand can be unticked. GET /api/rebuild-all returns that list, so the
page no longer repeats the server's rule for which job stands for a song,
and POST takes the ticked job ids. A selected job that has since been
rebuilt is reported as skipped rather than rebuilt twice.
When an update improves a stage, existing songs only pick it up when each
is rebuilt from that stage. POST /api/rebuild-all queues an ordinary
rebuild with unchanged inputs for the current job of every song: jobs that
were themselves rebuilt are skipped, as are songs still in progress and
songs whose upload is gone when the stage is acquire. The sidebar button
opens a dialog to pick the stage and confirm the count.
Runs on pull_request_target so the workflow always comes from main, on the
confined runner tier. Installs CPU-only torch before the package so pip
does not pull the CUDA build, then runs pytest. .dejima/allowed-domains
declares the hosts the job fetches from.
Separated stems keep faint bleed during instrumental passages, and crepe
finds a pitch in it with enough confidence to pass the voicing mask. A word
whose alignment ran into such a gap was then filled with notes for the
whole gap. Zero the confidence of frames more than 25 dB below the song's
90th-percentile A-weighted loudness, relative so it holds at any mastering
level.
The align, transcribe, lyrics and notes tests quoted lines from published
songs, and the UltraStar parser fixtures were complete community charts of
commercial songs. Replace them with counting words and two hand-written
charts that keep what the parser tests rely on: CRLF line endings,
Latin-1 bytes, a decimal-comma BPM, freestyle/golden/rap notes, negative
pitch and a two-number line break.
A missing span covers whole Whisper segments, so it often also holds lines
that were heard and matched. Skipping every aligned word starting inside one
dropped those lines' notes and left empty line breaks in the chart. Words
from lines never heard are already excluded via dropped_lines, so the span
now only drives the warning and the review marker.
YouTube breaks old yt-dlp releases faster than songtrace is released.
songtrace now checks PyPI daily, and again after a download fails outright,
installs a newer stable release into its cache dir and tries it before the
bundled copy, which is never modified. yt-dlp runs in a subprocess through
its CLI, all from one module, so a cached release never mixes with the
imported one. ytdlp_version pins a release or turns updating off.
songtrace serve is long-lived but only wants the GPU in bursts, and until now
it kept everything it ever loaded. align cached wav2vec2 in a module dict that
was never cleared, torchcrepe caches its model on its own infer() function,
and even once the last reference to a model goes, PyTorch keeps the freed
blocks in its allocator rather than returning them to the driver. On a shared
box that is the whole GPU held against one idle process.
Stages now hand it back as soon as they are done with it. A stage that caches
a model registers how to drop it with @on_release, so models.py needs to know
nothing about the pipeline, and run_pipeline calls release() in a finally
after each stage — not once per job, because transcribe sits between separate
and align waiting on the Whisper server, which can be minutes of a GPU nothing
is using.
release() answers the bytes it handed back and the stage logs a line when that
is not zero, which is the only part of this visible from outside. It will not
initialise CUDA just to clean up after a run that never touched it.
The cost is reloading each model once per job rather than once per process,
seconds against stages that run for minutes. The CUDA context itself stays
until the process exits; releasing that would mean a subprocess per stage.
Two problems found in review.
CREATE SONG emptied the form but left the previous song's review, player,
output path and queue selection in place, which reads as though the blank
form and that song belong together. It now clears the page down to a blank
sheet, and a `composing` flag stops the next poll selecting the newest song
back into it.
The reason it cleared so little is that clearDetail reached for an element id
that no longer exists, so it threw partway through and left everything after
the first two lines untouched. It now targets the review box and takes the
reason to show, since "no songs queued yet" is wrong when you have simply
deselected.
Separately, rebuild plans from the fields that changed, which cannot see a
change in songtrace itself: rebuilding a song packaged before the video's
audio was stripped found every stage cached and kept the echo. The rebuild
form gains "restart from", the CLI's --force-from, which combines with the
planner rather than replacing it — an explicit stage may widen what is rerun
but never narrow it below what the edit requires.
Also: a browser submits a textarea with CRLF, so every rebuild from the page
looked like a lyrics edit and redid the alignment. Lyrics are normalised to LF
on the way in. And [hidden] is only display:none, which .field's display:flex
was winning, so the new control showed in new-song mode despite the property
being set.
Checking whether the notes line up meant packaging the song, finding it in
the game and starting it there. The review panel now plays the song folder
itself: a transport above the lyrics, the line being sung highlighted, words
lighting up as they are reached, and a playhead sweeping each line against
its notes. Clicking a line or a word plays from there.
Playback is modelled on the game rather than on whatever is convenient: #MP3
is the audio and the clock, #VIDEO is muted background slaved to it, and the
times are the same seconds the notes were generated from. A word that lights
up late here is late in the game.
package now strips the audio track from the video it writes. It is the same
recording as the #MP3, so a player that does not mute the video plays both a
few frames apart, which is heard as an echo. The compatible-video branch
remuxes with -c copy -an rather than copying the file, so it still does not
re-encode. The browser player mutes the video as well, which covers song
folders built before this.
A new /api/jobs/{id}/media/{audio,video,cover} serves what the song's own
#MP3/#VIDEO/#COVER headers name, refusing anything resolving outside the song
folder. Range requests come from FileResponse, so seeking works.
Also, per review: CREATE SONG moves out of the form and into the top of the
queue, where it is always available, and a successful submission keeps the
fields it sent instead of emptying them.
Selecting a queue entry filled the review panel but left the form empty, so
the only way to adjust a song was the EDIT & REBUILD button, and the only way
to see what a song was built from was to click it. Selection now loads the
job's inputs, which is what the form is for: pick a song, change a line of
lyrics, hit REBUILD SONG.
That makes the EDIT & REBUILD action redundant, so it goes. CANCEL becomes
NEW SONG INSTEAD, which is what it does now that the form is rarely empty.
The landing page deliberately keeps its blank form: the job auto-selected on
load fills the review panel only, so opening the page is still the way to
start a new song.
Three things the old button-driven flow never had to handle:
- Advanced is opened when the job has a genre, year, language, bpm or a kept
cover. Those carry into a rebuild either way, so leaving them behind a
closed summary meant rebuilding with values that were never on screen.
- Submit is disabled, with the reason shown, while the selected job is queued
or running, since /rebuild answers 409 until it settles.
- Selecting a different entry asks before discarding edits, and deleting the
job the form is holding drops it rather than leaving the form pointed at an
id that would 404.
yt-dlp declares `dependencies = []` and puts everything in extras, so the
bare `yt-dlp` requirement installed it with no yt-dlp-ejs and no JavaScript
runtime. Without those it cannot solve YouTube's player challenges and falls
back to media URLs that YouTube rejects with HTTP 403, which is the failure
seen when adding a song. Require `yt-dlp[default,curl-cffi,deno]` instead;
the deno extra ships the JS runtime as a wheel, so the image needs no extra
apt packages.
Around that, make the stage survive the next time YouTube changes something:
the download is retried against each player client in `ytdlp_clients`
(default `["default", "tv"]`, the latter not needing a PO token) before
giving up, yt-dlp's retry counts are raised, and its own log lines are routed
into the job log so a PO-token demand can be told apart from a rate limit.
The final error names the running yt-dlp version, since a stale one is the
usual cause.
Cookies, TLS impersonation and a bgutil PO token provider URL are now config
fields, so an operator can work around a block without a code change.
`_classify_downloads` skips .part/.ytdl scratch files, which retries make far
more likely to be lying around.
- DELETE now checks under the shared lock whether another job with the
same slug is queued/running before removing the job entry, so
deleting a done parent can no longer rmtree a rebuild's active
work/output directories.
- _discover_jobs now wraps the iterdir/stat/read_warnings calls in the
same try/except as the JSON parse, so one corrupt or torn sidecar
file (e.g. from a killed mid-write) no longer crashes the whole app
at boot.
- Fixed test_delete_tolerates_a_work_dir_that_is_already_gone, which
had become vacuous since _write_sidecar now eagerly creates the work
dir; it now removes the dir before asserting DELETE tolerates its
absence.
- Added a regression test covering the DELETE slug-collision race.
.review-lane spans the full timeline box height (top:0; bottom:0) to
group each word with its note visually, but as a positioned element it
painted after the unpositioned .review-roll SVG, so the lane's
background covered the note bars for every other word. Give .review-roll
position: relative so DOM order (lanes, then notes) governs paint order
instead.
Replace arbitrary demo pitches with the actual melody transcribed from
Gilbert DeBenedetti's public-domain "Level 4, Flowing" arrangement (E
minor, 6/8), converted to UltraStar semitones, with one note per word
across both demo lines.
The review panel used to show a full line of packed text with one
undifferentiated strip of notes below it, so it was impossible to tell
which notes belonged to which word. The forced-alignment stage already
produces per-word timestamps in alignment.json, but they were never
surfaced past the pipeline.
/api/jobs/{id}/review now includes each line's per-word timing, and the
frontend spaces words out by their real sung timing (karaoke-style)
instead of normal text flow, with the note strip sharing that same time
axis so each note lands directly under its word. Alternating lane
backgrounds make the word/note pairing unambiguous. Dropped lines and
jobs without alignment data keep the old plain-text rendering.
Also fix a secondary ripple in build_notes(): a skipped row (inside a
missing span) no longer advances prev_line/prev_end, so it can't corrupt
line-initial spacing or break timing for its own line's later words.
renderLyricsReview was caching its job ID only in the done path, allowing
stale data to persist when switching between done/running/failed jobs.
Now clears the cache in the failed and not-done branches so a previously-
viewed done job always re-fetches when re-selected.
Regression test: test_lyrics_review_cache_clears_on_job_status_change
verifies both branches clear the cache key before rendering placeholders.
- Added classify_lyrics_match import to wire lyric classification into align stage
- Changed _assign_lines_to_sections signature to accept line_indices list instead of ref
- Modified _run to classify lyrics match, filter dropped lines, and write align_meta.json
- Sort segments by start time before processing for correct matching of non-chronological transcripts
- Added tests for dropped line detection and missing span flagging
- add test_rebuild_inherits_the_parents_cover: covers the cover_path
inheritance mechanism in the rebuild endpoint (Task 2's upload-retention
design relied on it, but nothing exercised it)
- tighten test_lineage_marker_survives_in_the_accessibility_tree to assert
the literal " rebuilt" screen-reader text, not just that rebuilt_from
is referenced somewhere in paintCard
- normalize submit's youtube_url/genre/language the same way rebuild does,
for symmetry, plus a pinning test
- pin acquire's stage outputs to [meta.json, audio.wav] in test_acquire.py,
since web.py's _require_source hardcodes those same filenames
Directly assert STAGE_LINE does not match the [rebuild] line, rather than
inferring it from job.stage, which ok_runner's later [acquire] running
line would overwrite either way.
branderson