Webui: open video source, restructure lyrics form, and add a visual per-line review #4
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: branderson/songtrace#4
Reference in New Issue
Block a user
Summary generated by LLM agent
Summary
Three related webui improvements for songtrace's song review flow:
difflibopcode classification and excluded from the proportional time-distribution used for force-alignment (with a warning), instead of silently distorting timing for the lines that were sung. Sung-but-unreferenced audio (ad-libs, spoken asides) is similarly flagged and excluded from note generation.GET /api/jobs/{job_id}/reviewendpoint.CLI behavior is unchanged; this only touches the webui and the pipeline stages that feed it.
Notable fix from final review
A whole-branch review caught a real bug that no single task's narrower review could see:
classify_lyrics_match()was scoping "missing span" (unmatched audio) bounds to the entire containing vocal section rather than just the unmatched audio itself. Sincevocal_sections()merges any segments within 6s of each other, a matched lyric line sharing a section with even a small ad-lib had all of its notes silently dropped from the piano roll. Fixed by scoping missing spans to the padded/merged bounds of just the unmatched segments (commit2a7b53e), plus a related ordering bug inbuild_notes()that could corrupt line-initial spacing/break timing for a skipped row's own line. Both fixes have dedicated regression tests.Known residual risks (please verify locally)
docs/images/webui.pngwas not regenerated for the same reason. To update it:pip install -e '.[dev]' && playwright install chromium && python scripts/screenshot_webui.py(the script now includes a "Landslide" fixture demonstrating dropped lines, a missing span, and low-confidence notes).Test plan
PYTHONPATH=src .venv/bin/python -m pytest tests/ -q)docs/images/webui.pnglocallyBoth residual risks noted in the PR description are now resolved:
♩ unmatched singing, 2:10–2:18missing-span marker render correctly with real CSS.docs/images/webui.pnglocally: done, commit153c83b(already pushed) — shows the new review panel for the Landslide fixture.Full test suite still green (253 passed, 1 skipped).
153c83bd99toaa21d60d5eThe review panel used to show a full line of packed text with one undifferentiated strip of notes below it, so it was impossible to tell which notes belonged to which word. The forced-alignment stage already produces per-word timestamps in alignment.json, but they were never surfaced past the pipeline. /api/jobs/{id}/review now includes each line's per-word timing, and the frontend spaces words out by their real sung timing (karaoke-style) instead of normal text flow, with the note strip sharing that same time axis so each note lands directly under its word. Alternating lane backgrounds make the word/note pairing unambiguous. Dropped lines and jobs without alignment data keep the old plain-text rendering.