Commit Graph
86 Commits
Author SHA1 Message Date
branderson 55e8297672 acquire: install yt-dlp's YouTube extras and retry across player clients
yt-dlp declares `dependencies = []` and puts everything in extras, so the
bare `yt-dlp` requirement installed it with no yt-dlp-ejs and no JavaScript
runtime. Without those it cannot solve YouTube's player challenges and falls
back to media URLs that YouTube rejects with HTTP 403, which is the failure
seen when adding a song. Require `yt-dlp[default,curl-cffi,deno]` instead;
the deno extra ships the JS runtime as a wheel, so the image needs no extra
apt packages.

Around that, make the stage survive the next time YouTube changes something:
the download is retried against each player client in `ytdlp_clients`
(default `["default", "tv"]`, the latter not needing a PO token) before
giving up, yt-dlp's retry counts are raised, and its own log lines are routed
into the job log so a PO-token demand can be told apart from a rate limit.
The final error names the running yt-dlp version, since a stale one is the
usual cause.

Cookies, TLS impersonation and a bgutil PO token provider URL are now config
fields, so an operator can work around a block without a code change.
`_classify_downloads` skips .part/.ytdl scratch files, which retries make far
more likely to be lying around.
2026-09-25 18:24:40 -07:00
branderson a7622c2d01 Merge pull request 'Add rebuild-from-scratch, delete, and cross-restart persistence for songs' (#5) from song-rebuild-persistence into main
Reviewed-on: #5
2026-07-28 03:25:48 -07:00
branderson c489dd5742 docs: regenerate webui.png 2026-07-28 03:26:08 -07:00
branderson f1b967b487 Merge remote-tracking branch 'origin/main' into song-rebuild-persistence
# Conflicts:
#	src/songtrace/web.py
#	tests/test_web.py
2026-07-28 03:24:04 -07:00
branderson e141cc5773 Fix DELETE slug-collision race and widen _discover_jobs error handling
- DELETE now checks under the shared lock whether another job with the
  same slug is queued/running before removing the job entry, so
  deleting a done parent can no longer rmtree a rebuild's active
  work/output directories.
- _discover_jobs now wraps the iterdir/stat/read_warnings calls in the
  same try/except as the JSON parse, so one corrupt or torn sidecar
  file (e.g. from a killed mid-write) no longer crashes the whole app
  at boot.
- Fixed test_delete_tolerates_a_work_dir_that_is_already_gone, which
  had become vacuous since _write_sidecar now eagerly creates the work
  dir; it now removes the dir before asserting DELETE tolerates its
  absence.
- Added a regression test covering the DELETE slug-collision race.
2026-07-28 03:06:09 -07:00
branderson 92948b2ea7 webui: add rebuild-from-scratch and delete actions 2026-07-28 02:52:43 -07:00
branderson c8c3d941ca web: persist song input as a sidecar and reload jobs on startup 2026-07-28 02:38:55 -07:00
branderson 7b8fce5b6e web: add delete endpoint 2026-07-28 02:34:09 -07:00
branderson cb10400a21 web: add force-rebuild-from-scratch endpoint 2026-07-28 02:31:23 -07:00
branderson 98e4f9532f pipeline: extract read_warnings for reuse by startup discovery 2026-07-28 02:28:27 -07:00
branderson 77bebcf9e1 Merge pull request 'Webui: open video source, restructure lyrics form, and add a visual per-line review' (#4) from worktree-webui-lyrics-review into main
Reviewed-on: #4
2026-07-28 01:42:56 -07:00
branderson 986f26b8ed Fix note bars hidden under alternating word-lane tint
.review-lane spans the full timeline box height (top:0; bottom:0) to
group each word with its note visually, but as a positioned element it
painted after the unpositioned .review-roll SVG, so the lane's
background covered the note bars for every other word. Give .review-roll
position: relative so DOM order (lanes, then notes) governs paint order
instead.
2026-07-28 01:43:04 -07:00
branderson d557ac6386 Use real Greensleeves melody in webui screenshot fixture
Replace arbitrary demo pitches with the actual melody transcribed from
Gilbert DeBenedetti's public-domain "Level 4, Flowing" arrangement (E
minor, 6/8), converted to UltraStar semitones, with one note per word
across both demo lines.
2026-07-28 01:35:44 -07:00
branderson 5023ea69d3 review: lay out lyrics words along a shared time axis with notes aligned beneath
The review panel used to show a full line of packed text with one
undifferentiated strip of notes below it, so it was impossible to tell
which notes belonged to which word. The forced-alignment stage already
produces per-word timestamps in alignment.json, but they were never
surfaced past the pipeline.

/api/jobs/{id}/review now includes each line's per-word timing, and the
frontend spaces words out by their real sung timing (karaoke-style)
instead of normal text flow, with the note strip sharing that same time
axis so each note lands directly under its word. Alternating lane
backgrounds make the word/note pairing unambiguous. Dropped lines and
jobs without alignment data keep the old plain-text rendering.
2026-07-28 01:22:48 -07:00
branderson 0599f6b665 docs: regenerate webui.png with the new lyrics review panel 2026-07-28 00:48:42 -07:00
branderson 2debfb171c transcribe: scope missing_spans to the unmatched audio itself, not the whole vocal section
Also fix a secondary ripple in build_notes(): a skipped row (inside a
missing span) no longer advances prev_line/prev_end, so it can't corrupt
line-initial spacing or break timing for its own line's later words.
2026-07-28 00:48:19 -07:00
branderson ac904cb09f screenshot: select the new Greensleeves fixture so it actually appears in the regenerated screenshot 2026-07-28 00:48:19 -07:00
branderson 8e89b08571 screenshot fixture: add songs with dropped lines, missing span, low-confidence notes 2026-07-28 00:48:08 -07:00
branderson 6a98ea0f21 fix: clear lyrics-review cache on job status change
renderLyricsReview was caching its job ID only in the done path, allowing
stale data to persist when switching between done/running/failed jobs.
Now clears the cache in the failed and not-done branches so a previously-
viewed done job always re-fetches when re-selected.

Regression test: test_lyrics_review_cache_clears_on_job_status_change
verifies both branches clear the cache key before rendering placeholders.
2026-07-27 23:36:56 -07:00
branderson 903e83dbcc webui: replace flat warnings list with per-line SVG piano-roll review 2026-07-27 23:24:39 -07:00
branderson a4e0b45953 web: add GET /api/jobs/{id}/review for per-line note data 2026-07-27 23:18:26 -07:00
branderson ffd662fb38 notes: skip notes in unmatched spans, warn on dropped lines and missing lyrics 2026-07-27 23:13:39 -07:00
branderson 43f0554796 align: remove unused RefLyrics import 2026-07-27 23:11:30 -07:00
branderson ebb7fc4433 align: exclude never-sung lines from proportional distribution, record align_meta.json
- Added classify_lyrics_match import to wire lyric classification into align stage
- Changed _assign_lines_to_sections signature to accept line_indices list instead of ref
- Modified _run to classify lyrics match, filter dropped lines, and write align_meta.json
- Sort segments by start time before processing for correct matching of non-chronological transcripts
- Added tests for dropped line detection and missing span flagging
2026-07-27 23:06:02 -07:00
branderson cf1c7410ab transcribe: classify reference lines as matched/dropped, flag unmatched singing 2026-07-27 22:58:58 -07:00
branderson 5bc4855264 webui: move video source out of Advanced, give lyrics a tall column 2026-07-27 22:51:36 -07:00
branderson a650c2f7ec Move webui image to top 2026-07-27 19:59:46 -07:00
branderson 283da7e1a9 Fix yass link 2026-07-27 19:54:36 -07:00
branderson 292fa25d5e Fix Melody Mania link 2026-07-27 19:51:53 -07:00
branderson 0eaab6aeed Remove karaoke maker reference 2026-07-27 19:49:30 -07:00
branderson 7a48942f1c Merge pull request 'Rename project from karaoke-maker to songtrace' (#3) from songtrace-rename into main
Reviewed-on: branderson/karaoke-maker#3
2026-07-27 18:42:57 -07:00
branderson 6c15c3d8f1 docs: fix migration guidance for old console script and orphaned container 2026-07-27 18:36:42 -07:00
branderson d2b7986cb9 docs: rename to songtrace and add tagline 2026-07-27 18:26:50 -07:00
branderson 3f90f4857f refactor: restyle web UI wordmark as SONG / TRACE 2026-07-27 18:24:21 -07:00
branderson 5553614fdf refactor: rename config env vars and default config path to songtrace 2026-07-27 18:21:30 -07:00
branderson 8a87534c5d refactor: rename CLI command and distribution to songtrace 2026-07-27 18:18:11 -07:00
branderson 322f8eb98b refactor: move package from karaoke_maker to songtrace 2026-07-27 18:14:23 -07:00
branderson a3078bb2fc Convert to AGPLv3 2026-07-27 17:37:56 -07:00
branderson bd1b4aeb6e Add GPLv3 license and webui screenshot 2026-07-27 17:28:07 -07:00
branderson f2a97f4f10 Fix torchcodec issue 2026-07-27 10:05:59 -07:00
branderson fca9e874f3 Support whisper asr-webservice 2026-07-27 09:53:03 -07:00
branderson bd5593c3ed Merge pull request 'Rerun failed tracks and edit-and-rebuild existing ones' (#2) from rerun-rebuild into main
Reviewed-on: branderson/karaoke-maker#2
2026-07-27 09:19:56 -07:00
branderson f7b59e7e14 test: close coverage gaps found in final branch review
- add test_rebuild_inherits_the_parents_cover: covers the cover_path
  inheritance mechanism in the rebuild endpoint (Task 2's upload-retention
  design relied on it, but nothing exercised it)
- tighten test_lineage_marker_survives_in_the_accessibility_tree to assert
  the literal " rebuilt" screen-reader text, not just that rebuilt_from
  is referenced somewhere in paintCard
- normalize submit's youtube_url/genre/language the same way rebuild does,
  for symmetry, plus a pinning test
- pin acquire's stage outputs to [meta.json, audio.wav] in test_acquire.py,
  since web.py's _require_source hardcodes those same filenames
2026-07-27 02:55:56 -07:00
branderson 9a5ef49ab6 feat: rerun and edit-and-rebuild controls in the web ui 2026-07-27 02:30:21 -07:00
branderson 5bf63c7c60 feat: remove the output folder a rebuild superseded 2026-07-27 02:16:39 -07:00
branderson b997e7a471 fix: assert the rebuild log line can't be mistaken for a stage banner
Directly assert STAGE_LINE does not match the [rebuild] line, rather than
inferring it from job.stage, which ok_runner's later [acquire] running
line would overwrite either way.
2026-07-27 02:07:00 -07:00
branderson c016e3c09d feat: rebuild a song, redoing only the stages the edit invalidated 2026-07-27 01:59:36 -07:00
branderson 99c62822b4 feat: expose a job's inputs for prefilling the rebuild form 2026-07-27 01:53:54 -07:00
branderson fd0ea79a28 feat: rerun a finished job as a linked new job 2026-07-27 01:49:29 -07:00
branderson a646685c61 fix: wrap unlink in try/except to never raise from cleanup 2026-07-27 01:44:57 -07:00