webui: a note sheet with a real pitch axis and beat grid #30

Merged
branderson merged 10 commits from editor into main 2026-10-15 16:52:45 -07:00
Collaborator

Summary generated by LLM agent

Phase 2, part 1 of the editor: a note sheet that makes a line's layout readable. Editing on this sheet is the next PR.

The review drew each line's notes as 2-unit bars in a 20-unit strip, with no pitch axis and no beat grid. In practice they read as underlines, and it was hard to tell where a note sat vertically.

The sheet

Modelled on Yass's page view: one line at a time, between the player and the lyric list.

  • Pitch axis: one row per semitone, shaded like piano keys (sharps darker). Note names in a gutter, with every C labelled with its octave. The range is fitted to the line with three semitones' room and never less than 15 rows, so rows stay a size you can aim at, about 20px.
  • Beat grid: taken from the chart's own BPM and GAP, so notes visibly sit where the game will play them. A line every 4 UltraStar beats, stronger every 16, plus single beats once they're at least 10px apart.
  • Notes: solid blocks styled by kind: normal purple, golden gold, freestyle dashed, rap aqua, golden rap gold with an aqua edge. Low-confidence notes get a dashed red outline, so they can't be confused with golden ones. Neighbouring lines' notes are faded in at the edges for context.
  • Pitch curve: the detected pitch is drawn over the notes. Where a note is right, the curve runs through it; a wrong pitch shows as the curve running in another row.
  • Text: syllables sit under their notes, as in Yass, with the aligned words beneath them.
  • Navigation: a playhead follows the transport, and the sheet pages to the line being sung. PREV/NEXT page by hand, and clicking the sheet seeks there. Badges mark a line as edited or as having low-confidence notes.

API

GET /api/jobs/{id}/pitch returns the pitch track every 20ms in note semitones (0 = C4), with null where nothing is voiced. It uses the notes stage's own voicing cut-off, so the curve agrees with what the notes were built from.

Code

The sheet is its own ES module, static/sheet.js. The page loads it with a dynamic import() and needs only a few hooks: load on review, page on line change, clear on reset. Its geometry (note names, pitch range, beat-grid weights including beats before GAP, line spans) is plain exported functions, unit-tested with node --test in tests/js/. tests/test_static_js.py runs those from pytest, and skips where Node is missing; CI's node:22 image has it.

Testing

  • I drove the real page in Chromium against a server seeded with a synthetic song: counting-word lyrics, a melody with one deliberately wrong note, a vibrato pitch curve, and one line saved as edited through the API, with golden, rap and low-confidence notes. No script errors.
  • I checked these from screenshots: the semitone rows, note names and beat grid; the wrong note standing out against the curve; the edited badge and note kinds; paging; the faded context notes. Two fixes came from them: single-beat lines are now hidden until there's room, and the curve is drawn over the notes rather than under, where it had been hidden exactly when the pitch was right.
  • New pytest tests for the pitch endpoint: semitone values, gaps where unvoiced, and 404 without a pitch track.
  • Full suite passes, except test_version (needs the package installed).

The README screenshot isn't regenerated here, because scripts/screenshot_webui.py needs Python Playwright. I'll update it with the editing PR.

**Summary generated by LLM agent** Phase 2, part 1 of the editor: a note sheet that makes a line's layout readable. Editing on this sheet is the next PR. The review drew each line's notes as 2-unit bars in a 20-unit strip, with no pitch axis and no beat grid. In practice they read as underlines, and it was hard to tell where a note sat vertically. ## The sheet Modelled on Yass's page view: one line at a time, between the player and the lyric list. - **Pitch axis:** one row per semitone, shaded like piano keys (sharps darker). Note names in a gutter, with every C labelled with its octave. The range is fitted to the line with three semitones' room and never less than 15 rows, so rows stay a size you can aim at, about 20px. - **Beat grid:** taken from the chart's own BPM and GAP, so notes visibly sit where the game will play them. A line every 4 UltraStar beats, stronger every 16, plus single beats once they're at least 10px apart. - **Notes:** solid blocks styled by kind: normal purple, golden gold, freestyle dashed, rap aqua, golden rap gold with an aqua edge. Low-confidence notes get a dashed red outline, so they can't be confused with golden ones. Neighbouring lines' notes are faded in at the edges for context. - **Pitch curve:** the detected pitch is drawn over the notes. Where a note is right, the curve runs through it; a wrong pitch shows as the curve running in another row. - **Text:** syllables sit under their notes, as in Yass, with the aligned words beneath them. - **Navigation:** a playhead follows the transport, and the sheet pages to the line being sung. PREV/NEXT page by hand, and clicking the sheet seeks there. Badges mark a line as edited or as having low-confidence notes. ## API `GET /api/jobs/{id}/pitch` returns the pitch track every 20ms in note semitones (0 = C4), with `null` where nothing is voiced. It uses the notes stage's own voicing cut-off, so the curve agrees with what the notes were built from. ## Code The sheet is its own ES module, `static/sheet.js`. The page loads it with a dynamic `import()` and needs only a few hooks: load on review, page on line change, clear on reset. Its geometry (note names, pitch range, beat-grid weights including beats before GAP, line spans) is plain exported functions, unit-tested with `node --test` in `tests/js/`. `tests/test_static_js.py` runs those from pytest, and skips where Node is missing; CI's `node:22` image has it. ## Testing - I drove the real page in Chromium against a server seeded with a synthetic song: counting-word lyrics, a melody with one deliberately wrong note, a vibrato pitch curve, and one line saved as edited through the API, with golden, rap and low-confidence notes. No script errors. - I checked these from screenshots: the semitone rows, note names and beat grid; the wrong note standing out against the curve; the edited badge and note kinds; paging; the faded context notes. Two fixes came from them: single-beat lines are now hidden until there's room, and the curve is drawn over the notes rather than under, where it had been hidden exactly when the pitch was right. - New pytest tests for the pitch endpoint: semitone values, gaps where unvoiced, and 404 without a pitch track. - Full suite passes, except `test_version` (needs the package installed). The README screenshot isn't regenerated here, because `scripts/screenshot_webui.py` needs Python Playwright. I'll update it with the editing PR.
branderson added 1 commit 2026-10-15 14:17:10 -07:00
webui: a note sheet with a real pitch axis and beat grid
tests / test (pull_request_target) Successful in 4m20s
7fa1f3912f
The review drew each line's notes as thin bars in a 20px strip, with no
pitch axis or beat grid, so it was hard to tell where a note sat. Add a
sheet that shows one line at a time the way an UltraStar editor does: a
row per semitone, shaded like piano keys and named in a gutter, the
chart's own beat grid with bars every 4 and 16 beats, notes as blocks
styled by kind (golden, freestyle, rap) with low-confidence ones outlined,
neighbouring lines faded in for context, syllables and aligned words
beneath, and a playhead that follows the transport, paging with it.

The detected pitch curve is drawn over the notes, served by a new GET
/api/jobs/{id}/pitch in note semitones, so a note off the sung pitch
stands out. The sheet is its own ES module; its geometry is unit-tested
with node --test, run from pytest and skipped where Node is absent.
branderson force-pushed editor from e30babe287 to 7fa1f3912f 2026-10-15 14:17:10 -07:00 Compare
branderson added 8 commits 2026-10-15 16:22:43 -07:00
Notes on the sheet can now be selected and edited: dragged in time and
pitch, resized from either end, nudged and stretched from the keyboard,
split, merged, added, deleted, given a syllable or a kind (golden,
freestyle, rap), with undo/redo. Everything snaps to the chart's beat grid
and to semitones, as the chart itself will. SAVE LINE stores the line
through the edits API, which rewrites the charts in place; REVERT goes back
to the generated notes; a line's edits set aside because its lyrics changed
are listed with a way to discard them. Unsaved changes are not lost to
paging, following playback, switching songs or closing the tab.

The operations live in static/edit-model.js as pure functions, unit-tested
with node --test. The sheet hit-tests by geometry rather than event
targets, since a render replaces the elements under the pointer.
The sheet's controls were listed in one run-on line under it, which was
hard to scan and grew with every feature. Replace it with a panel grouped
the way an editor's reference card is (select, move, length, notes, kind,
syllable, line) with keys drawn as key caps, opened by a KEYS button or ?,
closed by Esc, and remembered open or closed across visits.

The list is data (SHORTCUTS), and a test checks every letter key it names
is one the sheet's key handler handles, so the two cannot drift apart.
The beat grid came from the detected tempo (or the BPM override) and a GAP
pinned to the first note, so it did not always line up with the music,
and snapping did not land on its beats. A GRID bar on the sheet now sets
both, redrawing the grid live as values are tried, with x2 and /2 for a
tempo detected at double or half speed and ALIGN TO NOTE to move GAP the
least that puts the selected note on a beat. Note editing pauses while a
trial grid is unapplied, so notes never snap to a grid that is then
cancelled.

The grid is stored with the song's edits, so every rebuild keeps it, and
grid charts with it in place of its own choice; it records that choice in
tempo.json for RESET to return to. A GAP after the first note moves back
by whole beats, as notes on negative beats are rejected by players. PUT
and DELETE /api/jobs/{id}/grid set and reset it, rewriting the charts in
place.
Line breaks were always placed by songtrace, midway between lines and
recomputed around edited ones, so a break could not be put where a singer
breathes. The sheet now draws the breaks either side of a line as markers
that drag along the beat grid, kept between the lines they separate, and
double-click back to automatic. Break moves share the line's undo history
and SAVE LINE, which saves the notes only if they changed, so moving a
break never locks either line's notes.

Moved breaks are stored in edits.json keyed by the line each comes before,
follow their lines when the lyrics change, and win over the computed break
on every rebuild, clamped so a break never lands inside a line's notes.
The review lists each break with that line and whether it was moved, and
PUT /api/jobs/{id}/breaks takes {line: seconds | null}. merge now passes
untouched notes through as they are rather than rebuilding them, which
added float noise to their durations.
Reviewed-on: #35
Reviewed-by: branderson <brad@branderson.io>
Reviewed-on: #34
Reviewed-by: branderson <brad@branderson.io>
Reviewed-on: #33
Reviewed-by: branderson <brad@branderson.io>
Reviewed-on: #31
Reviewed-by: branderson <brad@branderson.io>
dejima-agent added 1 commit 2026-10-15 16:35:57 -07:00
Merge remote-tracking branch 'origin/main' into editor-merge
tests / test (pull_request_target) Successful in 4m21s
44241d0c18
# Conflicts:
#	tests/test_web.py
branderson approved these changes 2026-10-15 16:52:34 -07:00
branderson merged commit b03dc89d04 into main 2026-10-15 16:52:45 -07:00
Sign in to join this conversation.
No Reviewers
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: branderson/songtrace#30