Compare commits

...
29 Commits
Author SHA1 Message Date
branderson 82147bf295 Merge remote-tracking branch 'origin/main' into editor-merge
# Conflicts:
#	tests/test_web.py
2026-10-15 16:36:27 -07:00
branderson b5e6cd326b Merge pull request 'webui: edit a song's notes on the sheet' (#31) from sheet-editing into editor
Reviewed-on: #31
Reviewed-by: branderson <brad@branderson.io>
2026-10-15 16:22:40 -07:00
branderson 449f742d4b Merge pull request 'webui: mark songs with hand edits in the queue and REBUILD ALL FROM…' (#32) from edited-marks into main
Reviewed-on: #32
Reviewed-by: branderson <brad@branderson.io>
2026-10-14 23:28:27 -07:00
branderson b781c6cb0c Merge pull request 'webui: a grouped keyboard shortcut panel for the sheet' (#33) from sheet-shortcuts into sheet-editing
Reviewed-on: #33
Reviewed-by: branderson <brad@branderson.io>
2026-10-14 23:27:53 -07:00
branderson 3fbf39f9d6 Merge pull request 'webui: set a song's BPM and GAP from the sheet' (#34) from grid-edit into sheet-shortcuts
Reviewed-on: #34
Reviewed-by: branderson <brad@branderson.io>
2026-10-14 23:17:45 -07:00
branderson 82cafa96bf Merge pull request 'webui: move line breaks on the sheet' (#35) from break-edit into grid-edit
Reviewed-on: #35
Reviewed-by: branderson <brad@branderson.io>
2026-10-14 18:54:21 -07:00
branderson 30e4f8f202 webui: move line breaks on the sheet
Line breaks were always placed by songtrace, midway between lines and
recomputed around edited ones, so a break could not be put where a singer
breathes. The sheet now draws the breaks either side of a line as markers
that drag along the beat grid, kept between the lines they separate, and
double-click back to automatic. Break moves share the line's undo history
and SAVE LINE, which saves the notes only if they changed, so moving a
break never locks either line's notes.

Moved breaks are stored in edits.json keyed by the line each comes before,
follow their lines when the lyrics change, and win over the computed break
on every rebuild, clamped so a break never lands inside a line's notes.
The review lists each break with that line and whether it was moved, and
PUT /api/jobs/{id}/breaks takes {line: seconds | null}. merge now passes
untouched notes through as they are rather than rebuilding them, which
added float noise to their durations.
2026-10-13 16:24:17 -07:00
branderson bdfd7d8429 webui: set a song's BPM and GAP from the sheet
The beat grid came from the detected tempo (or the BPM override) and a GAP
pinned to the first note, so it did not always line up with the music,
and snapping did not land on its beats. A GRID bar on the sheet now sets
both, redrawing the grid live as values are tried, with x2 and /2 for a
tempo detected at double or half speed and ALIGN TO NOTE to move GAP the
least that puts the selected note on a beat. Note editing pauses while a
trial grid is unapplied, so notes never snap to a grid that is then
cancelled.

The grid is stored with the song's edits, so every rebuild keeps it, and
grid charts with it in place of its own choice; it records that choice in
tempo.json for RESET to return to. A GAP after the first note moves back
by whole beats, as notes on negative beats are rejected by players. PUT
and DELETE /api/jobs/{id}/grid set and reset it, rewriting the charts in
place.
2026-10-13 16:08:57 -07:00
branderson 2aeb16decb webui: a grouped keyboard shortcut panel for the sheet
The sheet's controls were listed in one run-on line under it, which was
hard to scan and grew with every feature. Replace it with a panel grouped
the way an editor's reference card is (select, move, length, notes, kind,
syllable, line) with keys drawn as key caps, opened by a KEYS button or ?,
closed by Esc, and remembered open or closed across visits.

The list is data (SHORTCUTS), and a test checks every letter key it names
is one the sheet's key handler handles, so the two cannot drift apart.
2026-10-12 23:13:45 -07:00
branderson 9a11db5794 webui: mark songs with hand edits in the queue and REBUILD ALL FROM…
A song's edits are kept through any rebuild, but nothing said which songs
had them. GET /api/jobs and GET /api/rebuild-all now report each song's
edited and set-aside lines, read from its edits file, on the job standing
for the song only, since the runs it replaced would claim the same edits.
Queue cards show them as a badge, and the rebuild dialog lists them per
song and says hand edits are kept.
2026-10-12 19:04:33 -07:00
branderson 381a499f2b webui: edit a song's notes on the sheet
Notes on the sheet can now be selected and edited: dragged in time and
pitch, resized from either end, nudged and stretched from the keyboard,
split, merged, added, deleted, given a syllable or a kind (golden,
freestyle, rap), with undo/redo. Everything snaps to the chart's beat grid
and to semitones, as the chart itself will. SAVE LINE stores the line
through the edits API, which rewrites the charts in place; REVERT goes back
to the generated notes; a line's edits set aside because its lyrics changed
are listed with a way to discard them. Unsaved changes are not lost to
paging, following playback, switching songs or closing the tab.

The operations live in static/edit-model.js as pure functions, unit-tested
with node --test. The sheet hit-tests by geometry rather than event
targets, since a render replaces the elements under the pointer.
2026-10-11 02:37:00 -07:00
branderson 793a78972d webui: a note sheet with a real pitch axis and beat grid
The review drew each line's notes as thin bars in a 20px strip, with no
pitch axis or beat grid, so it was hard to tell where a note sat. Add a
sheet that shows one line at a time the way an UltraStar editor does: a
row per semitone, shaded like piano keys and named in a gutter, the
chart's own beat grid with bars every 4 and 16 beats, notes as blocks
styled by kind (golden, freestyle, rap) with low-confidence ones outlined,
neighbouring lines faded in for context, syllables and aligned words
beneath, and a playhead that follows the transport, paging with it.

The detected pitch curve is drawn over the notes, served by a new GET
/api/jobs/{id}/pitch in note semitones, so a note off the sung pitch
stands out. The sheet is its own ES module; its geometry is unit-tested
with node --test, run from pytest and skipped where Node is absent.
2026-10-11 01:19:12 -07:00
branderson fa4a78f278 Merge pull request 'Keep manual edits to a song across rebuilds' (#29) from edit-layer into main
Reviewed-on: #29
Reviewed-by: branderson <brad@branderson.io>
2026-10-11 01:10:17 -07:00
branderson 5e2afc1254 Keep manual edits to a song across rebuilds
The groundwork for an in-app editor. Edits live in edits.json in the work
dir, which no stage produces, so no rebuild removes them. An edit is a
whole lyric line as the user left it, in seconds, so a rebuild with a new
BPM re-snaps it. grid lays the edits over the freshly generated notes
(neighbouring lines give way to an edited one), writes the result to
notes_final.json with warnings brought up to date, and charts that. When
the lyrics change, edits follow their lines by text and order; an edit
whose line's words changed is set aside as stale and reported, not applied.

Notes gain a kind, so golden, freestyle and rap notes can be charted. The
review endpoint returns the merged notes with their syllables, kinds and
edit state, plus the chart's beat grid. PUT/DELETE
/api/jobs/{id}/lines/{n} save and revert a line, DELETE
/api/jobs/{id}/stale/{n} discards a stale edit; each rewrites the work and
packaged charts in place with the tempo already chosen, so a save
re-encodes nothing.
2026-10-11 01:04:37 -07:00
branderson 7a86726c5c Merge pull request 'notes: judge low confidence against the song, and flag unsingable words' (#28) from relative-confidence into main
Reviewed-on: #28
Reviewed-by: branderson <brad@branderson.io>
2026-10-11 00:45:44 -07:00
branderson 211e75c448 Merge pull request 'web: report a job finished only once everything finishing it is done' (#27) from job-done-last into main
Reviewed-on: #27
Reviewed-by: branderson <brad@branderson.io>
2026-10-11 00:22:02 -07:00
branderson d4cdb4292d notes: judge low confidence against the song, and flag unsingable words
A fixed 0.5 alignment-score cutoff flagged 73-99% of notes on sung
vocals, so the flag pointed nowhere. Scores still separate bad words from
good ones, but their scale differs from song to song: flag a word below a
quarter of the song's median score, or one shorter than 0.08s or longer
than 3s, which no singer produces and which a word stretched over
something else does. Flagged notes less than 2s apart share one warning.
2026-10-10 23:33:53 -07:00
branderson 15124e512f web: report a job finished only once everything finishing it is done
The worker set status "done" before removing the job's upload and before
setting finished_at, so a client polling for done could see the upload
still there and finished_at null. Record the outcome and publish status,
error and finished_at together, under the lock, as the last step.
2026-10-10 23:29:37 -07:00
branderson 2fbff3553c Merge pull request 'align: keep lines Whisper missed when their singing went untranscribed' (#24) from unheard-lines into main
Reviewed-on: #24
Reviewed-by: branderson <brad@branderson.io>
2026-10-10 22:08:59 -07:00
branderson 8dd6ae5a17 Merge remote-tracking branch 'origin/main' into unheard-lines 2026-10-10 22:03:47 -07:00
branderson 4dff0dde4c Merge pull request 'ci: upgrade pip in the venv before installing torch' (#26) from ci-pip into main
Reviewed-on: #26
Reviewed-by: branderson <brad@branderson.io>
2026-10-10 21:50:58 -07:00
branderson c0343edb64 ci: upgrade pip in the venv before installing torch
Debian bookworm's python3-venv seeds pip 23.0.1, which rejects the
PyTorch CPU index's typing_extensions wheel because its metadata spells
the name with an underscore. It then falls back to the sdist, whose build
needs flit_core, which that index does not carry.
2026-10-10 21:38:30 -07:00
branderson 4749a7810f Merge pull request 'align: let a star token absorb non-lyric audio between lines' (#25) from line-stars into unheard-lines
Reviewed-on: #25
Reviewed-by: branderson <brad@branderson.io>
2026-10-10 20:56:15 -07:00
branderson 060669fda7 Merge pull request 'webui: add REBUILD ALL FROM… to rebuild every song from a stage' (#23) from rebuild-all into main
Reviewed-on: #23
Reviewed-by: branderson <brad@branderson.io>
2026-10-10 19:33:09 -07:00
branderson 15f01c50eb Merge pull request 'ci: run the test suite on pull requests' (#22) from ci-tests into main
Reviewed-on: #22
Reviewed-by: branderson <brad@branderson.io>
2026-10-10 19:32:43 -07:00
branderson f3c43e0410 align: let a star token absorb non-lyric audio between lines
The aligner had to explain every frame of a section with the lyrics, so
background speech or laughter between lines pulled the next line's
letters onto it, which then came out as a burst of tiny notes before the
real line was rushed to catch up. MMS_FA has a star token that matches
any audio; put one before, between and after the lines of each section,
as ctc-forced-aligner does by default, and drop the stars' spans from the
result.
2026-10-10 19:21:58 -07:00
branderson f5273b12e0 align: keep lines Whisper missed when their singing went untranscribed
A line with nothing in the transcript was always dropped as not sung. When
Whisper skips a passage (one song lost a whole verse, 50s of clear singing
with no segments), that drops lines that were sung. Find singing in the
vocal stem that no segment covers; a run of unmatched lines with such
singing between its heard neighbours is kept, and that singing joins the
alignment sections so those lines have somewhere to go. Only singing that
kept a line is added, so an untranscribed ad-lib draws no lines into it.
2026-10-10 18:58:54 -07:00
branderson 2eea510784 webui: let REBUILD ALL FROM… skip individual songs
The dialog now lists the songs it would rebuild, all ticked, each with when
it was last built and whether that was a rebuild, so songs already rebuilt
by hand can be unticked. GET /api/rebuild-all returns that list, so the
page no longer repeats the server's rule for which job stands for a song,
and POST takes the ticked job ids. A selected job that has since been
rebuilt is reported as skipped rather than rebuilt twice.
2026-10-10 18:16:02 -07:00
branderson 895c65bb4e webui: add REBUILD ALL FROM… to rebuild every song from a stage
When an update improves a stage, existing songs only pick it up when each
is rebuilt from that stage. POST /api/rebuild-all queues an ordinary
rebuild with unchanged inputs for the current job of every song: jobs that
were themselves rebuilt are skipped, as are songs still in progress and
songs whose upload is gone when the stage is acquire. The sidebar button
opens a dialog to pick the stage and confirm the count.
2026-10-10 18:06:13 -07:00
22 changed files with 3267 additions and 78 deletions
+3
View File
@@ -70,6 +70,9 @@ jobs:
apt-get update
apt-get install -y --no-install-recommends python3 python3-venv
python3 -m venv /opt/venv
# bookworm's pip 23.0 rejects the PyTorch index's typing_extensions
# wheel over its name's spelling, then cannot build it from source
/opt/venv/bin/pip install --retries 5 --upgrade pip
echo /opt/venv/bin >> "$GITHUB_PATH"
# The CPU build first: torch from PyPI is the CUDA build, several GB of
+39
View File
@@ -55,6 +55,39 @@ before this change still have it — check with `ffprobe <song>.mp4` for an
audio stream, and rebuild with **Restart from** set to `package` to clear it
without re-downloading.
## Editing a song
Above the lyrics, the note sheet shows one line at a time on a pitch-by-beat
grid, with the pitch songtrace detected drawn over the notes: where a note is
right the curve runs through it, so a wrong one stands out. Fix it there and
**SAVE LINE**; the song folder's chart is rewritten on the spot.
Drag a note to move it and its ends to resize it; everything snaps to the
chart's beat grid and to semitones. Double-click empty space to add a note,
or a note to change its syllable. With the sheet focused: ←/→ move, ↑/↓
change pitch (Shift for an octave), Shift+←/→ change length, Tab selects the
next note, S splits, M merges with the next note, G/F/R make a note golden,
freestyle or rap, Delete removes, Ctrl+Z undoes, Ctrl+S saves and P plays
the line.
The ↵ markers either side of a line are its line breaks: drag one to move it
(it snaps to beats and stays between the lines it separates) and double-click
one to put it back where songtrace places it. Moving a break does not mark
either line's notes as edited, so they still follow a rebuild.
The **GRID** bar above the notes sets the chart's BPM and GAP. Changes redraw
the grid as you type and are charted on **APPLY**; ×2 and ÷2 fix a tempo
detected at double or half speed, and **ALIGN TO NOTE** moves GAP the least
that puts the selected note on a beat. Notes keep their timing in seconds and
re-snap to the new grid. **RESET** returns to the BPM songtrace chose (your
BPM override, or the detected tempo) with GAP at the first note.
Edits survive every rebuild, REBUILD ALL FROM… included: they are kept apart
from what the pipeline generates and laid back over it each time. **REVERT**
returns a line to the generated notes. Changing a line's lyrics sets its edit
aside rather than applying it to different words; the sheet lists those so
you can discard them.
## Configuration
Config is TOML. Resolution order:
@@ -177,6 +210,12 @@ to redo a stage anyway — pick `package` to repackage without re-downloading.
It can only widen what is rerun, so an edit that invalidates an earlier stage
still wins.
**REBUILD ALL FROM…**, under CREATE SONG, does the same for every song at once:
pick the stage an update improved and each song is rebuilt from there with the
inputs it has now. Every song starts ticked; untick any already rebuilt by hand
(each shows when it was last built). Songs still in progress are skipped, as
are songs whose uploaded video is gone when the stage is `acquire`.
## YouTube download failures
`HTTP Error 403: Forbidden` from the acquire stage means YouTube served the
+233
View File
@@ -0,0 +1,233 @@
"""Manual edits to a song, kept apart from what the pipeline generates.
Edits live in work/<slug>/edits.json, which no stage produces, so no rebuild
removes them; grid lays them over the freshly generated notes every time. An
edit is a whole lyric line as the user left it, timed in seconds so that a
rebuild with a different BPM re-snaps it rather than breaking it.
"""
import difflib
import json
import math
import time
from pathlib import Path
from .notes import MIN_NOTE_SEC, TimedNote, low_confidence_warnings
EDITS_FILE = "edits.json"
FINAL_FILE = "notes_final.json"
KINDS = {":", "*", "F", "R", "G"}
def _empty(lyrics: list[list[str]]) -> dict:
# grid: the BPM and GAP set in the editor, or None for the pipeline's own;
# breaks: line breaks moved by hand, keyed by the line each comes before
return {"lyrics": lyrics, "lines": {}, "stale": [], "grid": None, "breaks": {}}
def load(work: Path, lyrics: list[list[str]]) -> dict:
"""The song's edits, keyed to `lyrics`.
Edits are stored with the lyrics they were made against. When those
differ, each edited line is matched to its line in the new lyrics by
text and order; an edit whose line's words changed is set aside as stale
rather than applied to words it was not made for.
"""
path = work / EDITS_FILE
if not path.exists():
return _empty(lyrics)
data = json.loads(path.read_text())
if data["lyrics"] != lyrics:
data = _remap(data, lyrics)
save(work, data)
return data
def save(work: Path, data: dict) -> None:
path = work / EDITS_FILE
if not (data["lines"] or data["stale"] or data.get("grid") or data.get("breaks")):
path.unlink(missing_ok=True)
return
tmp = path.with_suffix(".tmp")
tmp.write_text(json.dumps(data))
tmp.replace(path)
def _remap(data: dict, lyrics: list[list[str]]) -> dict:
old = [tuple(line) for line in data["lyrics"]]
new = [tuple(line) for line in lyrics]
moved = {}
for tag, i1, i2, j1, _ in difflib.SequenceMatcher(
a=old, b=new, autojunk=False).get_opcodes():
if tag == "equal":
moved.update({i1 + k: j1 + k for k in range(i2 - i1)})
lines, stale = {}, list(data["stale"])
for key, edit in data["lines"].items():
to = moved.get(int(key))
if to is None:
stale.append(edit)
else:
lines[str(to)] = {**edit, "line_index": to}
# a moved break goes with the line it comes before; with that line gone,
# there is nothing left for it to separate
breaks = {str(moved[int(k)]): t for k, t in data.get("breaks", {}).items()
if int(k) in moved}
return {"lyrics": lyrics, "lines": lines, "stale": stale, "grid": data.get("grid"),
"breaks": breaks}
def validate_notes(notes: list[dict]) -> list[dict]:
"""The notes of a line edit, checked and reduced to what is stored."""
clean = []
for n in notes:
try:
note = {"start": float(n["start"]), "duration": float(n["duration"]),
"pitch": int(n["pitch"]), "text": str(n["text"]),
"kind": str(n.get("kind", ":"))}
except (KeyError, TypeError, ValueError):
raise ValueError(f"malformed note {n!r}")
if note["start"] < 0 or note["duration"] <= 0:
raise ValueError(f"a note needs start >= 0 and duration > 0: {n!r}")
if note["kind"] not in KINDS:
raise ValueError(f"unknown note kind {note['kind']!r}")
if not note["text"].strip():
raise ValueError("a note needs text")
clean.append(note)
return sorted(clean, key=lambda n: n["start"])
def set_line(data: dict, index: int, notes: list[dict]) -> None:
data["lines"][str(index)] = {
"line_index": index, "words": data["lyrics"][index],
"notes": validate_notes(notes), "saved_at": time.time()}
def clear_line(data: dict, index: int) -> bool:
return data["lines"].pop(str(index), None) is not None
def set_breaks(data: dict, changes: dict) -> None:
"""Move line breaks, each keyed by the line it comes before, to a time in
seconds; None puts one back to where the notes alone would place it."""
breaks = dict(data.get("breaks", {}))
for key, t in changes.items():
try:
line = int(key)
except (TypeError, ValueError):
raise ValueError(f"a break is keyed by the line it comes before, not {key!r}")
if not 0 < line < len(data["lyrics"]):
raise ValueError(f"no line {line} for a break to come before")
if t is None:
breaks.pop(str(line), None)
continue
try:
t = float(t)
except (TypeError, ValueError):
raise ValueError(f"a break needs a time in seconds, not {t!r}")
if not (math.isfinite(t) and t >= 0):
raise ValueError(f"a break needs a time of 0 or more, not {t}")
breaks[str(line)] = t
data["breaks"] = breaks
def set_grid(data: dict, bpm, gap_ms) -> None:
"""Set the chart's BPM (UltraStar's, a quarter of a beat per beat) and GAP."""
try:
bpm, gap_ms = float(bpm), float(gap_ms)
except (TypeError, ValueError):
raise ValueError("bpm and gap_ms must be numbers")
if not (math.isfinite(bpm) and 0 < bpm <= 2000):
raise ValueError(f"bpm must be above 0 and at most 2000, not {bpm}")
if not (math.isfinite(gap_ms) and gap_ms >= 0):
raise ValueError(f"gap_ms must be 0 or more, not {gap_ms}")
data["grid"] = {"bpm": bpm, "gap_ms": gap_ms}
def clear_grid(data: dict) -> bool:
had = bool(data.get("grid"))
data["grid"] = None
return had
def merge(notes: list[dict], breaks: list[float], data: dict
) -> tuple[list[dict], list[float]]:
"""Generated notes and breaks with every edited line laid over them.
An edited line replaces the generated notes of that line, and the user's
timing wins: generated notes of earlier lines are cut back to end where it
starts, and those of later lines to start where it ends. Line breaks next
to an edited line move to the middle of the gap it leaves; a break moved
by hand stays where it was put, kept between the lines it separates.
"""
edited = {int(k): e["notes"] for k, e in data["lines"].items()}
moved = {int(k): t for k, t in data.get("breaks", {}).items()}
if not edited and not moved:
return notes, breaks
spans = {li: (ns[0]["start"], ns[-1]["start"] + ns[-1]["duration"])
for li, ns in edited.items() if ns}
kept = []
for n in notes:
li = n["line_index"]
if li in edited:
continue
start, end = n["start"], n["start"] + n["duration"]
for eli, (a, b) in spans.items():
if li < eli:
end = min(end, a)
elif li > eli:
start = max(start, b)
if (start, end) == (n["start"], n["start"] + n["duration"]):
kept.append(n)
elif end - start >= MIN_NOTE_SEC:
kept.append({**n, "start": start, "duration": end - start})
for li, ns in edited.items():
kept += [{**n, "line_index": li, "low_confidence": False} for n in ns]
kept.sort(key=lambda n: (n["start"], n["line_index"]))
def line_after(t: float) -> int | None:
return next((n["line_index"] for n in notes if n["start"] >= t), None)
first = {}
last = {}
for n in kept:
first.setdefault(n["line_index"], n)
last[n["line_index"]] = n
order = sorted(first, key=lambda li: first[li]["start"])
out_breaks = []
for prev, nxt in zip(order, order[1:]):
end = last[prev]["start"] + last[prev]["duration"]
if nxt in moved:
# where it was put, but never inside either line's notes
out_breaks.append(min(max(moved[nxt], end), max(first[nxt]["start"], end)))
continue
if prev not in edited and nxt not in edited:
# the generated break between these two lines, untouched
gen = [t for t in breaks if line_after(t) == nxt]
if gen:
out_breaks.append(gen[0])
continue
out_breaks.append((end + first[nxt]["start"]) / 2)
return kept, out_breaks
def finalize(work: Path, lyrics: list[list[str]]) -> dict:
"""The song as it is to be charted: generated notes with the edits laid
over them, warnings brought up to date, written to notes_final.json."""
generated = json.loads((work / "notes_sec.json").read_text())
data = load(work, lyrics)
notes, breaks = merge(generated["notes"], generated["breaks"], data)
warnings = generated.get("warnings", [])
if data["lines"]:
# low-confidence warnings must describe the merged notes, so a line
# the user fixed stops being reported; the others are about lyrics
warnings = (low_confidence_warnings([TimedNote(**n) for n in notes])
+ [w for w in warnings if "low-confidence" not in w])
if data["stale"]:
n = len(data["stale"])
warnings.append(f"{n} edited line{' no longer matches' if n == 1 else 's no longer match'}"
" the lyrics; set aside in the editor")
final = {"notes": notes, "breaks": breaks, "warnings": warnings,
"edited_lines": sorted(int(k) for k in data["lines"]),
"stale": data["stale"], "grid": data.get("grid"),
"moved_breaks": sorted(int(k) for k in data.get("breaks", {}))}
(work / FINAL_FILE).write_text(json.dumps(final))
return final
+35 -5
View File
@@ -3,7 +3,11 @@ from pathlib import Path
import numpy as np
from .ultrastar import LineBreak, Note, UltraStarSong, format_bpm
from .ultrastar import (LineBreak, Note, UltraStarSong, format_bpm, get_header,
parse_bpm, parse_ultrastar, serialize_ultrastar)
TEMPO_FILE = "tempo.json"
def detect_bpm(instrumental_path: Path) -> float:
@@ -25,11 +29,20 @@ def scale_bpm(bpm: float) -> float:
def to_ultrastar(notes: list[dict], breaks: list[float], file_bpm: float,
headers: list[tuple[str, str]]) -> UltraStarSong:
headers: list[tuple[str, str]],
gap_ms: float | None = None) -> UltraStarSong:
"""Snap notes to the beat grid. GAP defaults to the first note's start; a
GAP set later than that would put notes on negative beats, which players
reject, so it moves back by whole beats, keeping the grid where it was."""
if not notes:
raise ValueError("no notes to place on the beat grid")
gap_ms = round(notes[0]["start"] * 1000)
spb = 60 / (4 * file_bpm)
first_ms = notes[0]["start"] * 1000
if gap_ms is None:
gap_ms = first_ms
elif gap_ms > first_ms:
gap_ms -= math.ceil((gap_ms - first_ms) / (spb * 1000)) * spb * 1000
gap_ms = max(0, round(gap_ms))
def beat(t: float) -> int:
return round((t - gap_ms / 1000) / spb)
@@ -49,10 +62,27 @@ def to_ultrastar(notes: list[dict], breaks: list[float], file_bpm: float,
else:
b = max(beat(t), cursor)
length = max(1, beat(t + n["duration"]) - beat(t))
body.append(Note(kind=":", beat=b, length=length, pitch=n["pitch"],
text=n["text"]))
body.append(Note(kind=n.get("kind", ":"), beat=b, length=length,
pitch=n["pitch"], text=n["text"]))
cursor = b + length
out_headers = list(headers)
out_headers.append(("BPM", format_bpm(file_bpm)))
out_headers.append(("GAP", str(gap_ms)))
return UltraStarSong(headers=out_headers, body=body)
def rechart(txt_path: Path, notes: list[dict], breaks: list[float],
bpm: float | None = None, gap_ms: float | None = None) -> None:
"""Replace a chart's notes, keeping its headers and, unless given new
ones, its BPM.
For an edit saved from the editor: the packaged chart's headers may differ
from the work copy's (a re-encoded #VIDEO, a dropped #COVER), so each
keeps its own.
"""
song = parse_ultrastar(txt_path.read_text(encoding="utf-8"))
if bpm is None:
bpm = parse_bpm(get_header(song, "BPM"))
headers = [(k, v) for k, v in song.headers if k.upper() not in ("BPM", "GAP")]
txt_path.write_text(serialize_ultrastar(to_ultrastar(notes, breaks, bpm, headers, gap_ms)),
encoding="utf-8")
+32 -12
View File
@@ -7,7 +7,17 @@ import pyphen
from .languages import LANGUAGES
CONF_MASK = 0.5 # pitch frames below this are unvoiced
LOW_SCORE = 0.5 # word alignment score below this -> low_confidence
# A word is low-confidence when it scores below this share of the song's
# median alignment score: the scale differs a lot from song to song, so no
# fixed cutoff holds across them.
LOW_SCORE_OF_MEDIAN = 0.25
# ...or when no one sings a word that fast, or it was stretched over more than
# it can be holding.
MIN_WORD_SEC = 0.08
MAX_WORD_SEC = 3.0
# Flagged notes closer than this share one warning, so the list reads as
# places to look rather than one line per note.
WARN_GAP_SEC = 2.0
SPLIT_SEMITONES = 0.8
MIN_SEG_SEC = 0.05
MIN_NOTE_SEC = 0.05 # alignment may hand us zero-length words
@@ -24,6 +34,7 @@ class TimedNote:
text: str # display text incl. leading space / "~"
low_confidence: bool
line_index: int
kind: str = ":" # UltraStar note type; generated notes are all normal
def hz_to_midi_float(hz: float) -> float:
@@ -84,6 +95,7 @@ def build_notes(rows: list[dict], times, f0, conf, language: str | None,
) -> tuple[list[TimedNote], list[float], list[str]]:
notes: list[TimedNote] = []
breaks: list[float] = []
floor = LOW_SCORE_OF_MEDIAN * float(np.median([r["score"] for r in rows])) if rows else 0.0
prev_line, prev_end = None, 0.0
for row in rows:
if prev_line is not None and row["line_index"] > prev_line:
@@ -97,7 +109,8 @@ def build_notes(rows: list[dict], times, f0, conf, language: str | None,
_text(row["word"], 0, line_initial), True,
row["line_index"]))
continue
low = row["score"] < LOW_SCORE
low = (row["score"] < floor
or not MIN_WORD_SEC <= row["end"] - row["start"] <= MAX_WORD_SEC)
sylls = syllabify(row["word"], language)
if len(segs) < len(sylls):
segs = _subdivide(segs, len(sylls))
@@ -158,18 +171,25 @@ def _fmt(t: float) -> str:
return f"{int(t // 60)}:{int(t % 60):02d}"
def low_confidence_warnings(notes: list[TimedNote]) -> list[str]:
runs: list[list[TimedNote]] = []
for n in notes:
if not n.low_confidence:
continue
last = runs[-1][-1] if runs else None
if last is not None and n.start - (last.start + last.duration) < WARN_GAP_SEC:
runs[-1].append(n)
else:
runs.append([n])
return [
f"{len(run)} low-confidence note{'' if len(run) == 1 else 's'} "
f"near {_fmt(run[0].start)}–{_fmt(run[-1].start + run[-1].duration)}"
for run in runs]
def _warnings(notes: list[TimedNote], dropped_lines=(), missing_spans=(),
ref_lines=()) -> list[str]:
warnings, run = [], []
for n in list(notes) + [None]:
if n is not None and n.low_confidence:
run.append(n)
elif run:
first, last = run[0], run[-1]
warnings.append(
f"{len(run)} low-confidence note{'' if len(run) == 1 else 's'} "
f"near {_fmt(first.start)}–{_fmt(last.start + last.duration)}")
run = []
warnings = low_confidence_warnings(notes)
for group in _consecutive_runs(dropped_lines):
n = len(group)
text = " ".join(ref_lines[group[0]]) if group[0] < len(ref_lines) else ""
+5 -1
View File
@@ -47,7 +47,11 @@ def song_slug(inp: SongInput) -> str:
return name.replace("/", "").replace("\0", "")
def read_warnings(work: Path) -> list[str]:
notes_file = work / "notes_sec.json"
# grid's copy accounts for the user's edits; notes' is all a song built
# before edits existed has
notes_file = work / "notes_final.json"
if not notes_file.exists():
notes_file = work / "notes_sec.json"
if not notes_file.exists():
return []
return json.loads(notes_file.read_text()).get("warnings", [])
+72 -9
View File
@@ -1,12 +1,24 @@
import json
from pathlib import Path
import numpy as np
import soundfile as sf
from ..lyrics import alignment_token, normalize_lyrics
from ..models import on_release
from ..pipeline import Ctx, PipelineError, Stage
from .transcribe import classify_lyrics_match, load_transcript, lyrics_similarity, vocal_sections
from ..whisper_client import TranscriptSegment
from .transcribe import (classify_lyrics_match, load_transcript, lyrics_similarity,
recovered_spans, vocal_sections)
SIMILARITY_THRESHOLD = 0.4
# MMS_FA's wildcard token: any audio the transcript does not cover
STAR = "*"
# Singing is audio within this much of the stem's loud level; separation
# bleed sits far below it.
SINGING_BELOW_LOUD_DB = 25.0
# Shorter stretches are breaths, ad-libs or Whisper's loose segment edges.
MIN_UNHEARD_SEC = 3.0
# a stage aligns section by section, so the model is cached across the calls
# within one run and dropped at the end of it
@@ -40,7 +52,7 @@ def force_align_section(vocals_path: Path, start: float, end: float,
waveform[:, lo:hi], sr, bundle.sample_rate)
# MMS_FA only models a-z and apostrophe; anything else becomes the star token.
vocab = bundle.get_dict()
tokens = ["".join(c if c in vocab else "*" for c in t) for t in tokens]
tokens = ["".join(c if c in vocab else STAR for c in t) for t in tokens]
with torch.inference_mode():
emission, _ = _get_model(device)(waveform.to(device))
spans = bundle.get_aligner()(emission[0], bundle.get_tokenizer()(tokens))
@@ -50,6 +62,39 @@ def force_align_section(vocals_path: Path, start: float, end: float,
sum(t.score for t in s) / len(s)) for s in spans]
def untranscribed_singing(vocals_path: Path, segments,
frame: float = 0.05) -> list[tuple[float, float]]:
"""Stretches of the vocal stem with singing in them but no transcript."""
data, sr = sf.read(str(vocals_path), dtype="float32", always_2d=True)
hop = int(sr * frame)
n = len(data) // hop
if n == 0:
return []
rms = np.sqrt((data[:n * hop].mean(1).reshape(n, hop) ** 2).mean(1))
db = 20 * np.log10(rms + 1e-10)
sung = db >= np.quantile(db, 0.9) - SINGING_BELOW_LOUD_DB
times = np.arange(n) * frame
for s in segments:
sung[(times >= min(s.start, s.end)) & (times < max(s.start, s.end))] = False
spans, start = [], None
for t, on in zip(times, sung):
if on and start is None:
start = float(t)
elif not on and start is not None:
spans.append((start, float(t)))
start = None
if start is not None:
spans.append((start, float(times[-1] + frame)))
# a breath or a quiet syllable must not split one unsung passage in two
merged: list[list[float]] = []
for a, b in spans:
if merged and a - merged[-1][1] < 1.0:
merged[-1][1] = b
else:
merged.append([a, b])
return [(a, b) for a, b in merged if b - a >= MIN_UNHEARD_SEC]
def _assign_lines_to_sections(line_indices: list[int], sections) -> list[list[int]]:
"""Split matched lyric line indices across vocal sections proportionally to section duration."""
total = sum(b - a for a, b in sections) or 1.0
@@ -92,9 +137,20 @@ def _enforce_monotonic(rows: list[dict]) -> None:
def _align_section(vocals: Path, sec: tuple[float, float], words: list[tuple[str, int]],
language: str | None, device: str) -> list[dict]:
tokens = [alignment_token(w, language) for w, _ in words]
alignable = [t for t in tokens if t]
aligned = iter(force_align_section(vocals, sec[0], sec[1], alignable, device)
if alignable else [])
# A star before, between and after lines lets speech, laughter or an
# ad-lib there go unexplained rather than claim the next line's letters.
seq, prev_line = [], None
for (_, line_index), token in zip(words, tokens):
if not token:
continue
if line_index != prev_line:
seq.append(STAR)
prev_line = line_index
seq.append(token)
if seq:
seq.append(STAR)
spans = force_align_section(vocals, sec[0], sec[1], seq, device) if seq else []
aligned = iter(span for span, token in zip(spans, seq) if token != STAR)
rows, blank = [], (None, None, 0.0)
for (word, line_index), token in zip(words, tokens):
start, end, score = next(aligned, blank) if token else blank
@@ -112,11 +168,18 @@ def _run(ctx: Ctx) -> None:
raise PipelineError(
f"lyrics don't match audio (similarity {sim:.2f}) — "
"wrong lyrics or wrong recording?")
# normalised so an inverted span can never collapse a section onto one instant
sections = [(min(a, b), max(a, b)) for a, b in vocal_sections(segments)]
dropped_lines, missing_spans = classify_lyrics_match(segments, ref)
matched = [i for i in range(len(ref.lines)) if i not in set(dropped_lines)]
vocals = ctx.work / "vocals.wav"
unheard = untranscribed_singing(vocals, segments)
dropped_lines, missing_spans = classify_lyrics_match(segments, ref,
unheard=unheard)
# lines Whisper missed are aligned against the singing it had no text for;
# only that singing, so an untranscribed ad-lib cannot draw lines into it
recovered = [TranscriptSegment(a, b, "")
for a, b in recovered_spans(segments, ref, unheard)]
# normalised so an inverted span can never collapse a section onto one instant
sections = [(min(a, b), max(a, b))
for a, b in vocal_sections(segments + recovered)]
matched = [i for i in range(len(ref.lines)) if i not in set(dropped_lines)]
rows = []
for sec, line_idxs in zip(sections, _assign_lines_to_sections(matched, sections)):
words = [(w, li) for li in line_idxs for w in ref.lines[li]]
+14 -4
View File
@@ -1,13 +1,16 @@
import json
from pathlib import Path
from .. import grid
from .. import edits, grid
from ..lyrics import normalize_lyrics
from ..pipeline import Ctx, PipelineError, Stage, song_slug
from ..ultrastar import serialize_ultrastar
def _run(ctx: Ctx) -> None:
data = json.loads((ctx.work / "notes_sec.json").read_text())
data = edits.finalize(ctx.work, normalize_lyrics(ctx.inp.lyrics_text).lines)
if data["edited_lines"]:
ctx.log(f"[grid] kept {len(data['edited_lines'])} edited line(s)")
meta = json.loads((ctx.work / "meta.json").read_text())
if ctx.inp.bpm_override is not None:
bpm = ctx.inp.bpm_override
@@ -16,6 +19,12 @@ def _run(ctx: Ctx) -> None:
bpm = grid.scale_bpm(grid.detect_bpm(ctx.work / "instrumental.wav"))
except ValueError:
raise PipelineError("could not detect a usable tempo - is the audio silent?")
# the pipeline's own tempo, which RESET in the editor returns to
(ctx.work / grid.TEMPO_FILE).write_text(json.dumps({"bpm": bpm}))
gap_ms = None
if data["grid"]:
bpm, gap_ms = data["grid"]["bpm"], data["grid"]["gap_ms"]
ctx.log("[grid] using the BPM and GAP set in the editor")
slug = song_slug(ctx.inp)
video_ext = Path(meta["video_file"]).suffix
headers = [("TITLE", ctx.inp.title), ("ARTIST", ctx.inp.artist)]
@@ -28,7 +37,7 @@ def _run(ctx: Ctx) -> None:
headers += [("CREATOR", ctx.cfg.creator), ("MP3", f"{slug}.mp3"),
("COVER", f"{slug}.jpg"), ("VIDEO", f"{slug}{video_ext}")]
try:
song = grid.to_ultrastar(data["notes"], data["breaks"], bpm, headers)
song = grid.to_ultrastar(data["notes"], data["breaks"], bpm, headers, gap_ms)
except ValueError:
raise PipelineError("no notes were generated - the lyrics may not match the audio")
(ctx.work / "song.txt").write_text(serialize_ultrastar(song), encoding="utf-8")
@@ -36,5 +45,6 @@ def _run(ctx: Ctx) -> None:
def grid_stage() -> Stage:
return Stage(name="grid",
outputs=lambda ctx: [ctx.work / "song.txt"],
outputs=lambda ctx: [ctx.work / "song.txt",
ctx.work / edits.FINAL_FILE],
run=_run)
+62 -16
View File
@@ -39,28 +39,74 @@ def lyrics_similarity(segments, ref: RefLyrics) -> float:
return matcher.ratio()
def classify_lyrics_match(segments, ref: RefLyrics, pad: float = 1.0
def _line_times(segments, matcher, seg_of, line_of) -> dict[int, tuple[float, float]]:
"""For each reference line that was heard, the start of the first and the
end of the last transcript segment its words were heard in. Words in a
'replace' block are paired with heard words proportionally."""
times: dict[int, tuple[float, float]] = {}
for tag, ai1, ai2, bj1, bj2 in matcher.get_opcodes():
if tag not in ("equal", "replace"):
continue
for j in range(bj1, bj2):
seg = segments[seg_of[ai1 + (j - bj1) * (ai2 - ai1) // (bj2 - bj1)]]
a, b = times.get(line_of[j], (seg.start, seg.end))
times[line_of[j]] = (min(a, seg.start), max(b, seg.end))
return times
def _unheard_runs(segments, ref: RefLyrics, unheard
) -> tuple[list[list[int]], list[list[tuple[float, float]]]]:
"""Runs of reference lines with nothing heard for them, each with the
untranscribed singing between the heard lines either side of it."""
matcher, seg_of, line_of = match_lyrics(segments, ref)
times = _line_times(segments, matcher, seg_of, line_of)
alignable = set(line_of)
runs, spans, run, prev_end = [], [], [], 0.0
def close(next_start: float) -> None:
if run:
runs.append(list(run))
spans.append([(max(a, prev_end), min(b, next_start)) for a, b in unheard
if a < next_start and b > prev_end])
run.clear()
for li in range(len(ref.lines)):
if li in times:
close(times[li][0])
prev_end = times[li][1]
elif li in alignable:
run.append(li)
close(float("inf"))
return runs, spans
def recovered_spans(segments, ref: RefLyrics, unheard) -> list[tuple[float, float]]:
"""The untranscribed singing that stands in for lines Whisper missed, so
alignment can place them there."""
return _merge_spans([sp for spans in _unheard_runs(segments, ref, unheard)[1]
for sp in spans])
def classify_lyrics_match(segments, ref: RefLyrics, pad: float = 1.0, unheard=()
) -> tuple[list[int], list[tuple[float, float]]]:
"""Split reference lines into matched vs. dropped, and flag sung spans no
reference line explains.
dropped_lines: reference line indices covered only by 'insert' opcodes
(present in the lyrics, never heard). missing_spans: the padded, merged
bounds of the heard segments behind 'delete' opcodes (heard, but matching
no reference word). A segment can also hold matched lines, so a span
marks where to look, not words to discard.
"""
matcher, seg_of, line_of = match_lyrics(segments, ref)
covered = [False] * len(line_of)
for tag, _, _, bj1, bj2 in matcher.get_opcodes():
if tag in ("equal", "replace"):
for j in range(bj1, bj2):
covered[j] = True
by_line = {}
for j, li in enumerate(line_of):
by_line.setdefault(li, []).append(covered[j])
dropped_lines = sorted(li for li, cov in by_line.items() if not any(cov))
(present in the lyrics, never heard), unless `unheard` (sung spans the
transcript has no text for) has singing between the heard lines either
side of them: then Whisper missed them, not the singer.
missing_spans: the padded, merged bounds of the heard segments behind
'delete' opcodes (heard, but matching no reference word). A segment can
also hold matched lines, so a span marks where to look, not words to
discard.
"""
runs, run_spans = _unheard_runs(segments, ref, unheard)
dropped_lines = sorted(li for run, sp in zip(runs, run_spans) if not sp
for li in run)
matcher, seg_of, _ = match_lyrics(segments, ref)
spans = []
for tag, ai1, ai2, _, _ in matcher.get_opcodes():
if tag != "delete":
+158
View File
@@ -0,0 +1,158 @@
// Editing one line's notes. Every operation returns a new array and leaves
// its input alone, so undo is just keeping the old one. Times are seconds,
// snapped to the chart's beat grid as the chart itself will be. Notes carry
// a client-side `id` so a selection survives reordering; it is never saved.
const MIN_BEATS = 1;
const spbOf = (grid) => 60 / (4 * grid.bpm);
const gapOf = (grid) => grid.gap_ms / 1000;
const byStart = (a, b) => a.start - b.start;
let nextId = 1;
export const newId = () => nextId++;
export function beatOf(t, grid) {
return (t - gapOf(grid)) / spbOf(grid);
}
export function timeOf(beat, grid) {
return gapOf(grid) + beat * spbOf(grid);
}
export function snap(t, grid) {
return timeOf(Math.round(beatOf(t, grid)), grid);
}
// A note as the chart will have it: start on a beat, at least one beat long.
function onGrid(n, grid) {
const start = snap(n.start, grid);
const beats = Math.max(MIN_BEATS, Math.round((n.end - n.start) / spbOf(grid)));
return { ...n, start, end: start + beats * spbOf(grid) };
}
export function withIds(notes) {
return notes.map(n => ({ ...n, id: n.id ?? newId() })).sort(byStart);
}
export function move(notes, ids, beats, semitones, grid) {
const spb = spbOf(grid);
return notes.map(n => {
if (!ids.has(n.id)) return n;
const g = onGrid(n, grid);
const shift = Math.max(beats * spb, -g.start); // never before the song starts
return { ...g, start: g.start + shift, end: g.end + shift, pitch: n.pitch + semitones };
}).sort(byStart);
}
// Drag one edge of a note to time t; the other edge stays put.
export function resize(notes, id, edge, t, grid) {
const spb = spbOf(grid);
return notes.map(n => {
if (n.id !== id) return n;
const g = onGrid(n, grid);
if (edge === "end") return { ...g, end: Math.max(snap(t, grid), g.start + spb) };
return { ...g, start: Math.max(0, Math.min(snap(t, grid), g.end - spb)) };
}).sort(byStart);
}
// Longer or shorter by whole beats, keeping the start.
export function stretch(notes, ids, beats, grid) {
const spb = spbOf(grid);
return notes.map(n => {
if (!ids.has(n.id)) return n;
const g = onGrid(n, grid);
return { ...g, end: Math.max(g.end + beats * spb, g.start + spb) };
});
}
// In two at the beat nearest the middle; the second half holds the syllable.
export function split(notes, id, grid) {
const i = notes.findIndex(n => n.id === id);
if (i < 0) return { notes, ids: [] };
const g = onGrid(notes[i], grid);
const beats = Math.round((g.end - g.start) / spbOf(grid));
if (beats < 2) return { notes, ids: [] };
const mid = g.start + Math.floor(beats / 2) * spbOf(grid);
const a = { ...g, end: mid };
const b = { ...g, id: newId(), start: mid, text: "~", low_confidence: false };
return { notes: [...notes.slice(0, i), a, b, ...notes.slice(i + 1)], ids: [a.id, b.id] };
}
// Into one with the note after it; a held "~" adds no text.
export function merge(notes, id) {
const i = notes.findIndex(n => n.id === id);
if (i < 0 || i + 1 >= notes.length) return notes;
const a = notes[i];
const b = notes[i + 1];
const text = b.text.trim() === "~" ? a.text : a.text + b.text;
return [...notes.slice(0, i), { ...a, end: b.end, text }, ...notes.slice(i + 2)];
}
export function remove(notes, ids) {
return notes.filter(n => !ids.has(n.id));
}
export function insert(notes, t, pitch, grid) {
const start = Math.max(0, snap(t, grid));
const note = { id: newId(), start, end: start + 2 * spbOf(grid), pitch,
text: "~", kind: ":", low_confidence: false };
return { notes: [...notes, note].sort(byStart), id: note.id };
}
// Applying a kind every selected note already has turns it back to normal.
export function setKind(notes, ids, kind) {
const chosen = notes.filter(n => ids.has(n.id));
const to = chosen.length && chosen.every(n => n.kind === kind) ? ":" : kind;
return notes.map(n => (ids.has(n.id) ? { ...n, kind: to } : n));
}
// A syllable that starts a word keeps the space before it that marks it so.
export function setText(notes, id, text) {
const clean = text.trim();
if (!clean) return notes;
return notes.map(n => {
if (n.id !== id) return n;
const lead = n.text.startsWith(" ") && !text.startsWith(" ") ? " " : "";
return { ...n, text: lead + (text.startsWith(" ") ? " " + clean : clean) };
});
}
// What the server stores for a line.
export function toSaved(notes) {
return notes.map(n => ({ start: n.start, duration: n.end - n.start, pitch: n.pitch,
text: n.text, kind: n.kind ?? ":" }));
}
export function history(initial) {
const past = [];
const future = [];
let present = initial;
return {
get present() { return present; },
get canUndo() { return past.length > 0; },
get canRedo() { return future.length > 0; },
push(next) {
if (next === present) return;
past.push(present);
future.length = 0;
present = next;
},
undo() {
if (past.length) { future.push(present); present = past.pop(); }
return present;
},
redo() {
if (future.length) { past.push(present); present = future.pop(); }
return present;
},
};
}
// Where a dragged line break lands: on a beat, but kept between the end of
// the line before it and the start of the line it introduces.
export function placeBreak(t, lo, hi, grid) {
const snapped = snap(t, grid);
if (hi < lo) return lo;
return Math.min(Math.max(snapped, lo), hi);
}
+426 -3
View File
@@ -113,6 +113,61 @@ body {
#sidebar .eyebrow, #jobs .empty-note { padding: 0 18px; }
#create-song { display: block; width: calc(100% - 36px); margin: 0 18px 20px; padding: 9px 12px; }
#rebuild-all { display: block; margin: -8px 18px 20px; }
#rebuild-all-note { margin: -12px 18px 20px; }
#rebuild-all-dialog {
width: min(480px, calc(100vw - 32px));
padding: 26px 28px;
color: var(--text);
background: var(--panel);
border: 1px solid var(--border);
}
#rebuild-all-dialog::backdrop { background: rgb(10 10 15 / 0.75); }
#rebuild-all-dialog h2 {
margin: 0 0 14px;
font-family: var(--font-display);
font-weight: 700;
font-size: 0.8rem;
letter-spacing: 0.22em;
color: var(--text-muted);
}
#rebuild-all-form { display: grid; gap: 16px; }
#rebuild-all-form p { margin: 0; font-size: 0.88rem; line-height: 1.5; }
#rebuild-all-count { color: var(--warn); }
#rebuild-all-songs { margin: 0; padding: 0; border: 0; min-width: 0; }
#rebuild-all-songs legend {
margin-bottom: 6px;
padding: 0;
font-family: var(--font-display);
font-weight: 600;
font-size: 0.68rem;
letter-spacing: 0.14em;
text-transform: uppercase;
color: var(--text-muted);
}
#rebuild-all-list {
max-height: 40vh;
overflow-y: auto;
border-top: 1px solid var(--border);
border-bottom: 1px solid var(--border);
}
.rebuild-all-row {
display: grid;
grid-template-columns: auto 1fr;
gap: 1px 10px;
align-items: baseline;
padding: 7px 4px;
font-size: 0.88rem;
cursor: pointer;
}
.rebuild-all-row input { accent-color: var(--purple); }
.rebuild-all-row:has(input:disabled) { cursor: default; color: var(--text-muted); }
.rebuild-all-row .meta { grid-column: 2; font-size: 0.76rem; color: var(--text-muted); }
.rebuild-all-every { padding-top: 0; color: var(--text-muted); }
#rebuild-all-error { min-height: 1.2em; color: var(--fail); }
#rebuild-all-error:empty { display: none; }
#jobs { display: flex; flex-direction: column; gap: 1px; margin-top: 8px; }
.job {
@@ -337,6 +392,166 @@ button { cursor: pointer; }
/* the native transport renders light otherwise, against everything around it */
#player-media { display: block; width: 100%; margin-top: 8px; color-scheme: dark; }
#sheet { margin: 14px 0; padding: 12px 14px 10px; background: var(--panel); border: 1px solid var(--border); }
#sheet[hidden] { display: none; }
.sheet-nav { display: flex; align-items: center; gap: 12px; margin-bottom: 12px; }
.sheet-title {
min-width: 9em;
text-align: center;
font-family: var(--font-display);
font-weight: 700;
font-size: 0.72rem;
letter-spacing: 0.18em;
color: var(--text-muted);
}
.sheet-nav .action:disabled { opacity: 0.35; cursor: default; }
.sheet-badges { display: flex; gap: 8px; margin-left: auto; }
.sheet-badge {
padding: 2px 8px;
font-family: var(--font-display);
font-weight: 600;
font-size: 0.66rem;
letter-spacing: 0.1em;
text-transform: uppercase;
border: 1px solid currentColor;
border-radius: 2px;
}
.sheet-badge-edited { color: var(--aqua); }
.sheet-badge-low { color: var(--fail); }
.sheet-grid {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: 8px 10px;
margin-bottom: 12px;
padding: 8px 10px;
border: 1px solid var(--border);
}
.sheet-grid[hidden] { display: none; }
.sheet-grid-head {
font-family: var(--font-display);
font-weight: 700;
font-size: 0.66rem;
letter-spacing: 0.16em;
color: var(--aqua);
}
.sheet-grid-field {
display: flex;
align-items: center;
gap: 6px;
margin-left: 6px;
font-family: var(--font-display);
font-weight: 600;
font-size: 0.66rem;
letter-spacing: 0.1em;
color: var(--text-muted);
}
.sheet-grid-field input { width: 6.5em; padding: 3px 2px; font: 0.85rem var(--font-mono); }
.sheet-grid .action:disabled { opacity: 0.35; cursor: default; }
.sheet-grid .sheet-save:not(:disabled) { color: var(--bg); background: var(--purple); border-color: var(--purple); }
.sheet-grid-source { flex: 1; min-width: 7em; font-size: 0.76rem; text-align: right; color: var(--text-muted); }
.sheet-grid-pending { color: var(--warn); }
.sheet-tools { display: flex; flex-wrap: wrap; align-items: center; gap: 8px 14px; margin-bottom: 12px; }
.sheet-tool-group { display: flex; gap: 6px; }
.sheet-tools .action:disabled { opacity: 0.35; cursor: default; }
.sheet-tools .sheet-save:not(:disabled) { color: var(--bg); background: var(--purple); border-color: var(--purple); }
.sheet-status { flex: 1; min-width: 6em; font-size: 0.78rem; color: var(--text-muted); text-align: right; }
.sheet-status-error { color: var(--fail); }
.sheet-badge-dirty { color: var(--warn); }
.sheet-plot { position: relative; cursor: crosshair; touch-action: none; }
.sheet-plot:focus-visible { outline: 2px solid var(--aqua); outline-offset: 2px; }
.sheet-note-selected { stroke: var(--text); stroke-width: 2; stroke-dasharray: none; }
.sheet-syllable-selected { color: var(--aqua); }
.sheet-text-input {
position: absolute;
top: -2px;
width: 7em;
padding: 1px 4px;
font-size: 0.8rem;
background: var(--panel-raised);
border: 1px solid var(--aqua);
}
.sheet-stale { margin-bottom: 12px; padding: 8px 10px; font-size: 0.82rem; border: 1px solid var(--warn); }
.sheet-stale[hidden] { display: none; }
.sheet-stale-head { color: var(--warn); margin-bottom: 6px; }
.sheet-stale-row { display: flex; align-items: center; gap: 10px; margin-top: 4px; }
.sheet-stale-text { flex: 1; color: var(--text-muted); }
.sheet-nav > .action[aria-expanded] { margin-left: 4px; }
.sheet-keys {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(320px, 1fr));
gap: 14px 22px;
margin-bottom: 14px;
padding: 12px 14px;
background: var(--panel-raised);
border: 1px solid var(--border);
}
.sheet-keys[hidden] { display: none; }
.sheet-keys-head {
margin-bottom: 6px;
font-family: var(--font-display);
font-weight: 700;
font-size: 0.66rem;
letter-spacing: 0.16em;
text-transform: uppercase;
color: var(--aqua);
}
.sheet-keys dl { display: grid; grid-template-columns: auto 1fr; gap: 4px 10px; margin: 0; font-size: 0.78rem; }
.sheet-keys dt { white-space: nowrap; color: var(--text-muted); }
.sheet-keys dd { margin: 0; color: var(--text); }
.sheet-keys kbd {
display: inline-block;
min-width: 1.4em;
padding: 0 5px;
font: 0.72rem var(--font-mono);
text-align: center;
color: var(--text);
background: var(--panel);
border: 1px solid var(--control);
border-bottom-width: 2px;
border-radius: 3px;
}
.sheet-help[hidden] { display: none; }
.sheet-help { margin-top: 10px; font-size: 0.74rem; line-height: 1.5; color: var(--text-muted); }
.sheet-svg { display: block; }
.sheet-row { fill: var(--panel-raised); }
.sheet-row-black { fill: var(--bg); }
.sheet-octave { stroke: var(--control); stroke-width: 1; }
.sheet-key { fill: var(--text-muted); font: 10px var(--font-mono); text-anchor: end; dominant-baseline: middle; }
.sheet-key-c { fill: var(--text); font-weight: 600; }
.sheet-beat { stroke: var(--border); stroke-width: 1; opacity: 0.35; }
.sheet-beat-1 { stroke: var(--control); opacity: 0.55; }
.sheet-beat-2 { stroke: var(--text-muted); opacity: 0.8; }
.sheet-breaks { pointer-events: none; }
.sheet-break { stroke: var(--text-muted); stroke-width: 1.5; stroke-dasharray: 5 4; }
.sheet-break-moved { stroke: var(--aqua); stroke-dasharray: none; }
.sheet-break-label { fill: var(--text-muted); stroke: none; font: 12px var(--font-mono); }
.sheet-break-label.sheet-break-moved { fill: var(--aqua); stroke: none; }
.sheet-curve { pointer-events: none; }
.sheet-curve polyline { fill: none; stroke: var(--aqua); stroke-width: 1.5; opacity: 0.9; }
.sheet-note { fill: var(--purple); }
.sheet-note-golden { fill: #E8B931; }
.sheet-note-free { fill: none; stroke: var(--purple); stroke-width: 1.5; stroke-dasharray: 4 3; }
.sheet-note-rap { fill: var(--aqua); }
.sheet-note-golden-rap { fill: #E8B931; stroke: var(--aqua); stroke-width: 2; }
.sheet-note-low { stroke: var(--fail); stroke-width: 2; stroke-dasharray: 3 2; }
.sheet-note-context { opacity: 0.28; }
.sheet-playhead { position: absolute; top: 0; height: 300px; width: 2px; margin-left: -1px; background: var(--aqua); pointer-events: none; }
.sheet-playhead[hidden] { display: none; }
.sheet-syllables, .sheet-words { position: relative; overflow: hidden; }
.sheet-syllables { height: 22px; margin-top: 6px; overflow: visible; }
.sheet-syllable { position: absolute; top: 0; font-size: 0.8rem; white-space: pre; color: var(--text); cursor: text; }
.sheet-words { height: 24px; margin-top: 4px; }
.sheet-word {
position: absolute;
top: 0;
padding-top: 4px;
font-size: 0.78rem;
white-space: nowrap;
color: var(--text-muted);
border-top: 2px solid var(--border);
}
#lyrics-review { display: flex; flex-direction: column; gap: 10px; }
.review-line { padding: 8px 12px; background: var(--panel); border: 1px solid var(--border); }
.review-line-active { background: var(--panel-raised); border-color: var(--aqua); }
@@ -408,6 +623,11 @@ button { cursor: pointer; }
}
.lineage:hover { border-bottom-color: var(--aqua); }
.job-mark { grid-area: 1 / 2; justify-self: end; font-size: 0.8rem; color: var(--text-muted); }
.job-edits { grid-area: 2 / 2; justify-self: end; display: flex; gap: 6px; font-size: 0.72rem; }
.job-edits-kept { color: var(--aqua); }
.job-edits-stale { color: var(--warn); }
.rebuild-all-kept { color: var(--aqua); }
.rebuild-all-aside { color: var(--warn); }
#detail {
margin: 13px 0 0;
@@ -518,11 +738,40 @@ button { cursor: pointer; }
</div>
<div>
<button type="button" id="create-song" class="primary">CREATE SONG</button>
<button type="button" id="rebuild-all" class="action">REBUILD ALL FROM…</button>
<p id="rebuild-all-note" class="empty-note" role="status" hidden></p>
<h2 class="eyebrow">QUEUE</h2>
<div id="jobs"></div>
</div>
</aside>
<dialog id="rebuild-all-dialog" aria-labelledby="rebuild-all-heading">
<form id="rebuild-all-form" method="dialog">
<h2 id="rebuild-all-heading">REBUILD ALL SONGS</h2>
<p>Reruns the songs ticked below from the chosen stage onwards with the
inputs they have now, for when a songtrace update improves that stage.
Lines edited by hand are kept.</p>
<label class="field">
<span>Restart from</span>
<select name="force_from" required></select>
</label>
<fieldset id="rebuild-all-songs">
<legend>Songs</legend>
<label class="rebuild-all-row rebuild-all-every">
<input type="checkbox" id="rebuild-all-toggle" checked>
<span>All songs</span>
</label>
<div id="rebuild-all-list"></div>
</fieldset>
<p id="rebuild-all-count"></p>
<p id="rebuild-all-error" role="alert"></p>
<div class="form-actions">
<button type="submit" class="primary">REBUILD SELECTED</button>
<button type="button" class="action" id="rebuild-all-cancel">CANCEL</button>
</div>
</form>
</dialog>
<main id="main">
<section id="new-song">
<h2>NEW SONG</h2>
@@ -575,6 +824,7 @@ button { cursor: pointer; }
<video id="player-video" muted playsinline preload="metadata"></video>
<audio id="player-media" controls preload="metadata"></audio>
</div>
<div id="sheet" hidden></div>
<div id="lyrics-review"><p class="empty-note">no songs queued yet — nothing to review</p></div>
<div id="output-path"></div>
<div id="review-actions"></div>
@@ -692,6 +942,7 @@ form.addEventListener("input", () => { formDirty = true; });
// scrolled the panel to. Leaving another song's review under an empty form
// reads as though the two belong together.
createSong.addEventListener("click", () => {
if (sheet?.isDirty() && !confirm("Discard unsaved changes to this line?")) return;
if (formDirty && !confirm("Discard your unsaved edits to this form?")) return;
composing = true;
selectedId = null;
@@ -702,6 +953,107 @@ createSong.addEventListener("click", () => {
form.elements.title.focus();
});
const rebuildAllButton = document.getElementById("rebuild-all");
const rebuildAllNote = document.getElementById("rebuild-all-note");
const rebuildAllDialog = document.getElementById("rebuild-all-dialog");
const rebuildAllForm = document.getElementById("rebuild-all-form");
const rebuildAllCount = document.getElementById("rebuild-all-count");
const rebuildAllError = document.getElementById("rebuild-all-error");
const rebuildAllList = document.getElementById("rebuild-all-list");
const rebuildAllToggle = document.getElementById("rebuild-all-toggle");
const rebuildAllSubmit = rebuildAllForm.querySelector("button[type=submit]");
function rebuildAllBoxes() {
return [...rebuildAllList.querySelectorAll("input:not(:disabled)")];
}
function updateRebuildAllCount() {
const boxes = rebuildAllBoxes();
const chosen = boxes.filter(b => b.checked).length;
rebuildAllToggle.checked = chosen === boxes.length && chosen > 0;
rebuildAllToggle.indeterminate = chosen > 0 && chosen < boxes.length;
rebuildAllCount.textContent = chosen === 1 ? "1 song will be rebuilt."
: `${chosen} songs will be rebuilt.`;
// with nothing ticked no ids are sent, which the API reads as every song
rebuildAllSubmit.disabled = chosen === 0;
}
function rebuildAllRow(song) {
const busy = song.status === "queued" || song.status === "running";
const row = document.createElement("label");
row.className = "rebuild-all-row";
const box = document.createElement("input");
box.type = "checkbox";
box.name = "ids";
box.value = song.id;
box.checked = !busy;
box.disabled = busy;
const name = document.createElement("span");
name.textContent = `${song.title} · ${song.artist}`;
const meta = document.createElement("span");
meta.className = "meta";
const when = song.finished_at && new Date(song.finished_at * 1000).toLocaleString(
undefined, { dateStyle: "medium", timeStyle: "short" });
meta.textContent = busy ? "in progress, skipped"
: `${song.status === "failed" ? "failed" : song.rebuilt_from ? "rebuilt" : "built"} ${when || ""}`.trim();
if (song.edited_lines) {
const kept = document.createElement("span");
kept.className = "rebuild-all-kept";
kept.textContent = ` · ${song.edited_lines} edited line${song.edited_lines === 1 ? "" : "s"} kept`;
meta.append(kept);
}
if (song.stale_edits) {
const aside = document.createElement("span");
aside.className = "rebuild-all-aside";
aside.textContent = ` · ${song.stale_edits} set aside`;
meta.append(aside);
}
row.append(box, name, meta);
return row;
}
rebuildAllButton.addEventListener("click", async () => {
rebuildAllError.textContent = "";
rebuildAllForm.reset();
const resp = await fetch("/api/rebuild-all");
const songs = resp.ok ? await resp.json() : [];
rebuildAllList.replaceChildren(...songs.map(rebuildAllRow));
updateRebuildAllCount();
rebuildAllDialog.showModal();
});
rebuildAllList.addEventListener("change", updateRebuildAllCount);
rebuildAllToggle.addEventListener("change", () => {
for (const box of rebuildAllBoxes()) box.checked = rebuildAllToggle.checked;
updateRebuildAllCount();
});
document.getElementById("rebuild-all-cancel").addEventListener(
"click", () => rebuildAllDialog.close());
rebuildAllForm.addEventListener("submit", async (e) => {
e.preventDefault();
const stage = rebuildAllForm.elements.force_from.value;
const resp = await fetch("/api/rebuild-all", {
method: "POST", body: new FormData(rebuildAllForm) });
if (!resp.ok) {
const body = await resp.json().catch(() => ({}));
rebuildAllError.textContent = body.detail || `error: ${resp.status}`;
return;
}
const { ids, skipped } = await resp.json();
rebuildAllDialog.close();
let note = `Queued ${ids.length} rebuild${ids.length === 1 ? "" : "s"} from ${stage}.`;
if (skipped.length) {
const reasons = [...new Set(skipped.map(s => s.reason))].join("; ");
note += ` Skipped ${skipped.length}: ${reasons}.`;
}
rebuildAllNote.textContent = note;
rebuildAllNote.hidden = false;
refreshJobs();
});
function jobCard(job) {
const card = document.createElement("button");
card.type = "button";
@@ -732,7 +1084,10 @@ function jobCard(job) {
markLabel.className = "sr-only";
mark.append(markGlyph, markLabel);
card.append(title, artist, stateLine, time, mark);
const edits = document.createElement("span");
edits.className = "job-edits";
card.append(title, artist, stateLine, time, mark, edits);
card.addEventListener("click", () => selectJob(job.id));
paintCard(card, job);
return card;
@@ -740,7 +1095,7 @@ function jobCard(job) {
function paintCard(card, job) {
const state = STATE[job.status] ?? { glyph: "·", cls: "st-queued" };
const [title, artist, stateLine, time, mark] = card.children;
const [title, artist, stateLine, time, mark, edits] = card.children;
card.className = `job ${state.cls}`;
if (job.id === selectedId) card.setAttribute("aria-current", "true");
else card.removeAttribute("aria-current");
@@ -751,6 +1106,33 @@ function paintCard(card, job) {
time.textContent = elapsed(job);
mark.children[0].textContent = job.rebuilt_from ? "↻" : "";
mark.children[1].textContent = job.rebuilt_from ? " rebuilt" : "";
paintEdits(edits, job);
}
// hand edits a rebuild keeps, and any set aside because the lyrics changed
function paintEdits(node, job) {
const parts = [];
if (job.edited_lines) parts.push(["job-edits-kept", `✎ ${job.edited_lines}`]);
if (job.stale_edits) parts.push(["job-edits-stale", `⚠ ${job.stale_edits}`]);
const label = [
job.edited_lines ? `${job.edited_lines} edited line${job.edited_lines === 1 ? "" : "s"}` : "",
job.stale_edits ? `${job.stale_edits} set aside` : "",
].filter(Boolean).join(", ");
if (node.title === label) return;
node.title = label;
node.replaceChildren(...parts.map(([cls, text]) => {
const span = document.createElement("span");
span.className = cls;
span.setAttribute("aria-hidden", "true");
span.textContent = text;
return span;
}));
if (label) {
const sr = document.createElement("span");
sr.className = "sr-only";
sr.textContent = label;
node.append(sr);
}
}
// patched in place rather than replaced wholesale: detaching a card would blur
@@ -780,6 +1162,8 @@ async function loadStages() {
const select = form.elements.force_from;
select.replaceChildren(new Option("only what changed", ""),
...STAGES.map(s => new Option(s, s)));
rebuildAllForm.elements.force_from.replaceChildren(
new Option("choose a stage", ""), ...STAGES.map(s => new Option(s, s)));
}
function renderStageTrack(job) {
@@ -851,6 +1235,8 @@ async function refreshJobs() {
}
async function selectJob(id) {
if (id !== selectedId && sheet?.isDirty() &&
!confirm("Discard unsaved changes to this line?")) return;
// the form already holds this job; reloading it would drop edits in progress
const reload = id !== rebuildOf;
if (reload && formDirty && !confirm("Discard your unsaved edits to this form?")) return;
@@ -977,6 +1363,7 @@ function lineEntry(row, line) {
const labels = [...row.querySelectorAll(".review-word")];
return {
row,
index: line.index,
t0: laid[0].start,
t1: Math.max(...laid.map(x => x.end)),
words: line.words.map((w, i) => ({ el: labels[i], start: w.start, end: w.end })),
@@ -1023,6 +1410,32 @@ const media = document.getElementById("player-media");
const video = document.getElementById("player-video");
const DRIFT = 0.25; // seconds before the picture is worth re-seeking
// the note sheet lives in its own module, loaded before any review renders
let sheet = null;
let stopAt = null; // set by playing one line: where playback pauses again
const sheetReady = import("/static/sheet.js").then((m) => {
sheet = m.createSheet(document.getElementById("sheet"), {
seek: (t) => {
media.currentTime = t;
if (media.paused) syncPlayback();
},
playLine: (from, to) => {
if (player.hidden) return;
media.currentTime = from;
stopAt = to;
media.play().catch(() => {});
},
// a saved line changes the notes the lyric list draws
onSaved: (data) => {
const box = document.getElementById("lyrics-review");
const { nodes, entries } = buildLyricsReview(data);
box.replaceChildren(...nodes);
bindTimeline(player.hidden ? [] : entries);
if (!player.hidden) syncPlayback();
},
});
});
function slaveVideo(force) {
if (video.hidden || !video.getAttribute("src")) return;
if (force || Math.abs(video.currentTime - media.currentTime) > DRIFT) {
@@ -1036,7 +1449,10 @@ for (const event of ["seeked", "ratechange"]) {
slaveVideo(true);
});
}
media.addEventListener("pause", () => video.pause());
media.addEventListener("pause", () => {
video.pause();
stopAt = null;
});
let timeline = [];
let activeEntry = null;
let syncFrame = null;
@@ -1077,10 +1493,12 @@ function syncPlayback() {
activeEntry = entry;
if (entry) {
entry.row.classList.add("review-line-active");
sheet?.showLine(entry.index);
focusActive();
}
}
if (entry) paintEntry(entry, t);
sheet?.setTime(t);
}
// centred so there is context either side, like a scrolling lyric sheet; only
@@ -1092,6 +1510,7 @@ function focusActive() {
}
function pumpPlayback() {
if (stopAt !== null && media.currentTime >= stopAt) media.pause();
syncPlayback();
slaveVideo(false);
syncFrame = media.paused ? null : requestAnimationFrame(pumpPlayback);
@@ -1140,6 +1559,7 @@ function setPlayerSource(job, available) {
}
function resetPlayer() {
sheet?.clear();
// the 3s poll reaches here for every running job; reloading a cleared
// element each time would be pure churn
if (player.hidden && !timeline.length) return;
@@ -1187,6 +1607,9 @@ async function renderLyricsReview(job) {
}
const { nodes, entries } = buildLyricsReview(data);
box.replaceChildren(...nodes);
await sheetReady;
if (job.id !== selectedId) return;
sheet.load(job.id, data);
// with nothing to play, a highlighted line and a click-to-seek cursor would
// both be promising something the song folder cannot deliver
const playable = setPlayerSource(job, data.media);
+951
View File
@@ -0,0 +1,951 @@
// The note sheet: one lyric line at a time on a pitch-by-beat grid, the way
// an UltraStar editor lays a song out, with the detected pitch drawn over
// the notes so a wrong one stands out. The line on show can be edited.
import * as edit from "./edit-model.js";
const SVG = "http://www.w3.org/2000/svg";
const NAMES = ["C", "C♯", "D", "D♯", "E", "F", "F♯", "G", "G♯", "A", "A♯", "B"];
const BLACK = new Set([1, 3, 6, 8, 10]);
const GUTTER = 46; // px of note names left of the grid
const PLOT_HEIGHT = 300; // px; rows divide this between the pitches shown
const CONTEXT = 0.75; // seconds shown either side of the line
const EDGE = 6; // px from a note's end that grabs it for resizing
const KINDS = { "*": "golden", F: "free", R: "rap", G: "golden-rap" };
const KEYS_OPEN = "songtrace.sheet-keys";
// The sheet's controls, grouped as an editor's reference card lists them.
// "+" joins keys pressed together, " / " separates alternatives, and a
// lowercase word is something done with the mouse rather than a key.
export const SHORTCUTS = [
["Select", [
["click", "select a note"],
["Shift+click", "add to the selection"],
["Tab / Shift+Tab", "next, previous note"],
["Esc", "clear the selection"]]],
["Move", [
["drag", "move in time and pitch"],
["← / →", "a beat earlier, later"],
["↑ / ↓", "a semitone up, down"],
["Shift+↑ / Shift+↓", "an octave up, down"]]],
["Length", [
["drag an end", "resize"],
["Shift+← / Shift+→", "a beat shorter, longer"]]],
["Notes", [
["double-click", "add a note, on empty space"],
["S", "split in two"],
["M", "merge with the next"],
["Delete", "remove"]]],
["Kind", [
["G", "golden"],
["F", "freestyle"],
["R", "rap"],
["G / F / R", "again: back to normal"]]],
["Line break", [
["drag a ↵ marker", "move it, between the lines it separates"],
["double-click a ↵", "back to where songtrace puts it"]]],
["Syllable", [
["double-click", "edit, on a note"],
["Enter", "edit the selected note"]]],
["Line", [
["Ctrl+Z", "undo"],
["Ctrl+Shift+Z / Ctrl+Y", "redo"],
["Ctrl+S", "save"],
["P", "play"],
["Page Up / Page Down", "previous, next line"],
["?", "show or hide these keys"]]],
];
// "Shift+Tab / Esc" as key caps, with mouse actions left as words
export function comboParts(combo) {
return combo.split(" / ").map(alt => alt.split("+").map(part => ({
text: part, key: !/^[a-z]/.test(part),
})));
}
const wrap = (n, d) => ((n % d) + d) % d;
const mod12 = (p) => wrap(p, 12);
// UltraStar pitch 0 is C4 (MIDI 60)
export function noteName(pitch) {
return NAMES[mod12(pitch)] + (Math.floor((pitch + 60) / 12) - 1);
}
export function isBlackKey(pitch) {
return BLACK.has(mod12(pitch));
}
// The pitches to show: the line's notes with room around them, and never
// less than an octave and a bit, so rows stay a consistent size to aim at.
export function pitchRange(pitches, { pad = 3, min = 15 } = {}) {
if (!pitches.length) return { lo: -7, hi: 7 };
let lo = Math.min(...pitches) - pad;
let hi = Math.max(...pitches) + pad;
const short = min - (hi - lo + 1);
if (short > 0) {
lo -= Math.floor(short / 2);
hi += Math.ceil(short / 2);
}
return { lo, hi };
}
// UltraStar beats between t0 and t1 on the chart's grid. A beat is a quarter
// of the #BPM beat; weight 2 marks every 16th, 1 every 4th, as editors do.
export function beatLines(t0, t1, grid) {
const spb = 60 / (4 * grid.bpm);
const gap = grid.gap_ms / 1000;
const out = [];
for (let k = Math.ceil((t0 - gap) / spb); gap + k * spb <= t1; k++) {
const weight = wrap(k, 16) === 0 ? 2 : wrap(k, 4) === 0 ? 1 : 0;
out.push({ t: gap + k * spb, beat: k, weight });
}
return out;
}
// The seconds a line occupies, from its notes and the words it was aligned to.
export function lineSpan(line) {
const ts = [...line.notes.flatMap(n => [n.start, n.end]),
...line.words.flatMap(w => [w.start, w.end])];
return ts.length ? { start: Math.min(...ts), end: Math.max(...ts) } : null;
}
function el(name, attrs = {}, parent = null) {
const node = document.createElementNS(SVG, name);
for (const [k, v] of Object.entries(attrs)) node.setAttribute(k, v);
if (parent) parent.append(node);
return node;
}
function div(cls, text = null) {
const node = document.createElement("div");
node.className = cls;
if (text !== null) node.textContent = text;
return node;
}
export function createSheet(root, { seek, playLine, onSaved }) {
const button = (label, title, onClick, cls = "action") => {
const b = document.createElement("button");
b.type = "button";
b.className = cls;
b.textContent = label;
b.title = title;
b.addEventListener("click", onClick);
return b;
};
const title = div("sheet-title");
const badges = div("sheet-badges");
const prev = button("◀ PREV", "previous line (Page Up)", () => page(-1));
const next = button("NEXT ▶", "next line (Page Down)", () => page(1));
const nav = div("sheet-nav");
const keysButton = button("KEYS ?", "keyboard shortcuts (?)", () => toggleKeys());
keysButton.setAttribute("aria-expanded", "false");
nav.append(prev, title, next, badges, keysButton);
const tools = {
undo: button("UNDO", "Ctrl+Z", () => undo()),
redo: button("REDO", "Ctrl+Shift+Z", () => redo()),
split: button("SPLIT", "split the selected note (S)", () => splitSelected()),
merge: button("MERGE", "merge with the next note (M)", () => mergeSelected()),
remove: button("DELETE", "delete the selected notes (Delete)", () => removeSelected()),
golden: button("GOLDEN", "golden note (G)", () => kind("*")),
free: button("FREESTYLE", "freestyle note (F)", () => kind("F")),
rap: button("RAP", "rap note (R)", () => kind("R")),
play: button("▶ LINE", "play this line (P)", () => playCurrent()),
discard: button("DISCARD", "drop unsaved changes to this line", () => discard()),
revert: button("REVERT", "go back to the generated notes for this line", () => revert()),
save: button("SAVE LINE", "Ctrl+S", () => save(), "action sheet-save"),
};
const toolbar = div("sheet-tools");
const group = (...items) => { const g = div("sheet-tool-group"); g.append(...items); return g; };
const status = div("sheet-status");
status.setAttribute("role", "status");
toolbar.append(group(tools.undo, tools.redo),
group(tools.split, tools.merge, tools.remove),
group(tools.golden, tools.free, tools.rap),
group(tools.play), status,
group(tools.discard, tools.revert, tools.save));
const plot = div("sheet-plot");
plot.tabIndex = 0;
plot.setAttribute("aria-label", "note sheet: select notes and edit them with the mouse or keyboard");
const svg = el("svg", { class: "sheet-svg" });
const playhead = div("sheet-playhead");
playhead.hidden = true;
plot.append(svg, playhead);
const syllables = div("sheet-syllables");
const words = div("sheet-words");
const help = div("sheet-help", "Press ? for keyboard shortcuts.");
const keys = div("sheet-keys");
keys.id = "sheet-keys";
keysButton.setAttribute("aria-controls", keys.id);
for (const [group, entries] of SHORTCUTS) {
const section = div("sheet-keys-group");
const heading = div("sheet-keys-head", group);
const list = document.createElement("dl");
for (const [combo, action] of entries) {
const dt = document.createElement("dt");
comboParts(combo).forEach((alt, i) => {
if (i) dt.append(" / ");
alt.forEach((part, j) => {
if (j) dt.append("+");
const node = document.createElement(part.key ? "kbd" : "span");
node.textContent = part.text;
dt.append(node);
});
});
const dd = document.createElement("dd");
dd.textContent = action;
list.append(dt, dd);
}
section.append(heading, list);
keys.append(section);
}
const stale = div("sheet-stale");
const gridBar = div("sheet-grid");
const field = (label, step, unit) => {
const wrap = document.createElement("label");
wrap.className = "sheet-grid-field";
const input = document.createElement("input");
input.type = "number";
input.step = step;
input.min = "0";
wrap.append(label, input);
if (unit) wrap.append(unit);
return { wrap, input };
};
const bpmField = field("BPM", "0.01");
const gapField = field("GAP", "1", "ms");
const gridSource = div("sheet-grid-source");
const gridTools = {
double: button("×2", "double the BPM", () => tryGrid({ bpm: grid().bpm * 2 })),
half: button("÷2", "halve the BPM", () => tryGrid({ bpm: grid().bpm / 2 })),
earlier: button("−10", "GAP 10ms earlier", () => tryGrid({ gap_ms: grid().gap_ms - 10 })),
later: button("+10", "GAP 10ms later", () => tryGrid({ gap_ms: grid().gap_ms + 10 })),
align: button("ALIGN TO NOTE", "move GAP the least that puts the selected note on a beat",
() => alignToNote()),
cancel: button("CANCEL", "drop the BPM and GAP being tried", () => { gridDraft = null; render(); }),
reset: button("RESET", "back to the BPM songtrace chose, with GAP at the first note",
() => resetGrid()),
apply: button("APPLY", "chart the song on this BPM and GAP", () => applyGrid(),
"action sheet-save"),
};
const gridHead = div("sheet-grid-head", "GRID");
gridBar.append(gridHead, bpmField.wrap, gridTools.double, gridTools.half,
gapField.wrap, gridTools.earlier, gridTools.later, gridTools.align,
gridSource, gridTools.cancel, gridTools.reset, gridTools.apply);
for (const [f, key] of [[bpmField, "bpm"], [gapField, "gap_ms"]]) {
f.input.addEventListener("input", () => {
const v = Number(f.input.value);
if (f.input.value !== "" && Number.isFinite(v) && v >= 0 && (key === "gap_ms" || v > 0)) {
tryGrid({ [key]: v }, f.input);
}
});
}
root.replaceChildren(nav, stale, gridBar, toolbar, keys, plot, syllables, words, help);
// stays as the person left it, across lines, songs and visits
function showKeys(open) {
keys.hidden = !open;
help.hidden = open;
keysButton.setAttribute("aria-expanded", String(open));
try { localStorage.setItem(KEYS_OPEN, open ? "1" : ""); } catch {}
}
function toggleKeys() {
showKeys(keys.hidden);
}
let keysOpen = false;
try { keysOpen = !!localStorage.getItem(KEYS_OPEN); } catch {}
showKeys(keysOpen);
let review = null;
let curve = null;
let index = null;
let jobId = null;
let view = null; // what the last render laid out: window and scale
let layout = null; // window and pitch range, kept steady while editing
let hist = null; // the current line's notes, with undo
let baseline = null; // what the line holds on the server
let preview = null; // notes mid-drag, not yet in the history
let breakPreview = null; // a break mid-drag: { which, t }
let selected = new Set();
let drag = null;
let saving = false;
const visible = () => review.lines.filter(l => !l.dropped && lineSpan(l));
// the line being edited: its notes and the breaks either side of it, as
// one state so undo covers both. A break is its time, undefined where there
// is none to move, or null when it is to go back to automatic.
const notes = () => preview ?? hist?.present.notes ?? [];
const dirty = () => !!hist && hist.present !== baseline;
let gridDraft = null; // BPM and GAP being tried out, not yet applied
// the grid being drawn and snapped to: the one being tried out, if any
const grid = () => gridDraft ?? review?.grid;
// notes snap to the grid, so they wait while a new one is being tried
const canEdit = () => !!review?.grid && !gridDraft && !saving;
function say(text, error = false) {
status.textContent = text;
status.classList.toggle("sheet-status-error", error);
}
// the window and pitch range that hold the line, grown (never shrunk while
// editing) so rows do not jump under the pointer
function fit(reset) {
const line = review.lines[index];
const ts = [...notes().flatMap(n => [n.start, n.end]),
...line.words.flatMap(w => [w.start, w.end])];
const span = ts.length ? { start: Math.min(...ts), end: Math.max(...ts) } : lineSpan(line);
const range = pitchRange(notes().map(n => n.pitch));
const want = { t0: span.start - CONTEXT, t1: span.end + CONTEXT, ...range };
// the breaks either side are part of the line's page, so they stay on it
for (const slot of breakSlots()) {
const t = breakTime(slot);
want.t0 = Math.min(want.t0, t - 0.3);
want.t1 = Math.max(want.t1, t + 0.3);
}
layout = reset || !layout ? want : {
t0: Math.min(layout.t0, want.t0), t1: Math.max(layout.t1, want.t1),
lo: Math.min(layout.lo, want.lo), hi: Math.max(layout.hi, want.hi),
};
}
// edits whose line's lyrics have since changed: kept, not applied, until
// the person who made them decides
function paintStale() {
const items = review?.stale ?? [];
stale.hidden = !items.length;
stale.replaceChildren();
if (!items.length) return;
stale.append(div("sheet-stale-head",
`${items.length} edited line${items.length === 1 ? " was" : "s were"} set aside ` +
"because the lyrics changed:"));
items.forEach((item, n) => {
const row = div("sheet-stale-row");
row.append(div("sheet-stale-text", `"${item.text}"`),
button("DISCARD", "throw this edit away", () => discardStale(n)));
stale.append(row);
});
}
async function discardStale(n) {
const resp = await fetch(`/api/jobs/${jobId}/stale/${n}`, { method: "DELETE" });
if (!resp.ok) { say(`error ${resp.status}`, true); return; }
review = await resp.json();
paintStale();
onSaved(review);
}
function open(i) {
index = i;
const line = review.lines[i];
const lines = visible();
const at = lines.findIndex(l => l.index === i);
const breakBefore = (l) => review.breaks?.find(b => b.line === l?.index)?.time;
baseline = {
notes: edit.withIds(line.notes.map(n => ({ ...n, kind: n.kind ?? ":" }))),
before: at > 0 ? breakBefore(line) : undefined,
after: breakBefore(lines[at + 1]),
};
hist = edit.history(baseline);
breakPreview = null;
preview = null;
selected = new Set();
say("");
fit(true);
paintStale();
render();
}
// asked for by the person using the sheet, so it may cost them their edits
function goTo(i) {
if (!review || i === index) return;
if (dirty() && !confirm("Discard unsaved changes to this line?")) return;
open(i);
}
function page(step) {
const lines = visible();
const at = lines.findIndex(l => l.index === index);
const to = lines[Math.min(Math.max(at + step, 0), lines.length - 1)];
if (to) goTo(to.index);
}
function change(next, keep = selected) {
hist.push({ ...hist.present, notes: next });
selected = new Set([...keep].filter(id => next.some(n => n.id === id)));
fit(false);
render();
}
function changeBreak(which, t) {
if (hist.present[which] === t) return;
hist.push({ ...hist.present, [which]: t });
render();
}
// the breaks either side of the line, each with the gap it may move in:
// from the end of the line before it to the start of the line after
function breakSlots() {
const ns = notes();
if (!hist || !ns.length) return [];
const lines = visible();
const at = lines.findIndex(l => l.index === index);
const prev = lines[at - 1];
const next = lines[at + 1];
const slots = [];
if (prev?.notes.length && hist.present.before !== undefined) {
slots.push({ which: "before", line: index,
lo: Math.max(...prev.notes.map(n => n.end)), hi: ns[0].start });
}
if (next?.notes.length && hist.present.after !== undefined) {
slots.push({ which: "after", line: next.index,
lo: Math.max(...ns.map(n => n.end)), hi: Math.min(...next.notes.map(n => n.start)) });
}
return slots;
}
// where a break is drawn; one going back to automatic is shown mid-gap,
// roughly where it will land
function breakTime(slot) {
const v = breakPreview?.which === slot.which ? breakPreview.t : hist.present[slot.which];
const t = v === null ? (slot.lo + slot.hi) / 2 : v;
return Math.min(Math.max(t, slot.lo), Math.max(slot.hi, slot.lo));
}
const undo = () => { if (hist?.canUndo) { hist.undo(); fit(false); render(); } };
const redo = () => { if (hist?.canRedo) { hist.redo(); fit(false); render(); } };
const one = () => (selected.size === 1 ? [...selected][0] : null);
function splitSelected() {
if (one() === null || !grid()) return;
const out = edit.split(notes(), one(), grid());
if (out.notes !== notes()) change(out.notes, new Set([out.ids[0]]));
}
function mergeSelected() {
if (one() !== null) change(edit.merge(notes(), one()));
}
function removeSelected() {
if (selected.size) change(edit.remove(notes(), selected), new Set());
}
function kind(k) {
if (selected.size) change(edit.setKind(notes(), selected, k));
}
function playCurrent() {
if (!layout) return;
const ns = notes();
const start = ns.length ? ns[0].start : layout.t0 + CONTEXT;
const end = ns.length ? Math.max(...ns.map(n => n.end)) : layout.t1 - CONTEXT;
playLine(Math.max(0, start - 0.3), end + 0.2);
}
function discard() {
if (!dirty()) return;
open(index);
}
// a change saved to the server, which answers with the refreshed review
async function send(path, method, body, busy, done) {
saving = true;
render();
say(busy);
try {
const resp = await fetch(`/api/jobs/${jobId}/${path}`, {
method, headers: { "Content-Type": "application/json" },
body: body ? JSON.stringify(body) : undefined,
});
if (!resp.ok) {
const err = await resp.json().catch(() => ({}));
say(err.detail || `error ${resp.status}`, true);
return false;
}
const data = await resp.json();
const at = index;
review = data;
gridDraft = null;
open(at);
say(done);
onSaved(data);
return true;
} catch {
say("could not reach songtrace", true);
return false;
} finally {
saving = false;
render();
}
}
// only what changed: a moved break alone must not mark the line edited
async function save() {
if (!dirty() || saving) return;
const state = hist.present;
const lines = visible();
const next = lines[lines.findIndex(l => l.index === index) + 1];
const moved = {};
if (state.before !== baseline.before) moved[index] = state.before;
if (state.after !== baseline.after && next) moved[next.index] = state.after;
if (state.notes !== baseline.notes &&
!await send(`lines/${index}`, "PUT", { notes: edit.toSaved(state.notes) }, "saving…", "saved")) {
return;
}
if (Object.keys(moved).length) await send("breaks", "PUT", moved, "saving…", "saved");
}
function revert() {
if (!review.lines[index].edited || saving) return;
if (!confirm("Go back to the generated notes for this line? Your edit is lost.")) return;
send(`lines/${index}`, "DELETE", null, "reverting…", "reverted to the generated notes");
}
// redraws on the trial grid straight away; nothing is saved until APPLY
function tryGrid(change, typing = null) {
const next = { ...grid(), ...change };
next.gap_ms = Math.max(0, next.gap_ms);
const same = review.grid && next.bpm === review.grid.bpm && next.gap_ms === review.grid.gap_ms;
gridDraft = same ? null : next;
render(typing);
}
function alignToNote() {
const note = notes().find(n => selected.has(n.id));
if (!note) return;
const g = grid();
const spbMs = 60000 / (4 * g.bpm);
const atMs = note.start * 1000;
const offset = (atMs - g.gap_ms) - Math.round((atMs - g.gap_ms) / spbMs) * spbMs;
tryGrid({ gap_ms: Math.round((g.gap_ms + offset) * 100) / 100 });
}
async function applyGrid() {
if (!gridDraft || saving) return;
if (dirty()) { say("save or discard this line's changes first", true); return; }
await send("grid", "PUT", gridDraft, "charting on the new grid…", "grid applied");
}
async function resetGrid() {
if (!review.grid_edited || saving) return;
if (dirty()) { say("save or discard this line's changes first", true); return; }
await send("grid", "DELETE", null, "resetting the grid…",
"grid reset to the BPM songtrace chose");
}
function paintGrid(typing) {
const g = grid();
gridBar.hidden = !review.grid;
if (!g) return;
const round = (v, d) => String(Math.round(v * 10 ** d) / 10 ** d);
// the field being typed in keeps what was typed, half-finished or not
if (typing !== bpmField.input) bpmField.input.value = round(g.bpm, 2);
if (typing !== gapField.input) gapField.input.value = round(g.gap_ms, 2);
gridSource.textContent = gridDraft ? "not applied"
: review.grid_edited ? "set in the editor"
: review.auto_bpm ? "songtrace's choice" : "";
gridSource.classList.toggle("sheet-grid-pending", !!gridDraft);
gridTools.align.disabled = saving || selected.size !== 1;
gridTools.cancel.disabled = !gridDraft || saving;
gridTools.apply.disabled = !gridDraft || saving;
gridTools.reset.disabled = !review.grid_edited || !!gridDraft || saving;
for (const b of [gridTools.double, gridTools.half, gridTools.earlier, gridTools.later]) {
b.disabled = saving;
}
}
function render(typing = null) {
if (!review || index === null) return;
const line = review.lines[index];
const { t0, t1, lo, hi } = layout;
const width = Math.max(plot.clientWidth, 320);
const rowH = PLOT_HEIGHT / (hi - lo + 1);
const x = (t) => GUTTER + ((t - t0) / (t1 - t0)) * (width - GUTTER);
const y = (p) => (hi - p) * rowH; // top of the row for pitch p
view = { t0, t1, x, width, rowH, hi,
time: (px) => t0 + ((px - GUTTER) / (width - GUTTER)) * (t1 - t0),
pitch: (py) => hi - Math.floor(py / rowH) };
svg.setAttribute("viewBox", `0 0 ${width} ${PLOT_HEIGHT}`);
svg.setAttribute("width", width);
svg.setAttribute("height", PLOT_HEIGHT);
svg.replaceChildren();
const rows = el("g", { class: "sheet-rows" }, svg);
for (let p = lo; p <= hi; p++) {
el("rect", { x: GUTTER, y: y(p), width: width - GUTTER, height: rowH,
class: isBlackKey(p) ? "sheet-row sheet-row-black" : "sheet-row" }, rows);
if (mod12(p) === 0) {
el("line", { x1: GUTTER, x2: width, y1: y(p) + rowH, y2: y(p) + rowH,
class: "sheet-octave" }, rows);
}
// every C names its octave; the naturals between get a name when
// there is room for it
if (mod12(p) === 0 || (rowH >= 16 && !isBlackKey(p))) {
const label = el("text", { x: GUTTER - 6, y: y(p) + rowH / 2,
class: mod12(p) === 0 ? "sheet-key sheet-key-c" : "sheet-key" },
rows);
label.textContent = noteName(p);
}
}
if (grid()) {
const beats = el("g", { class: "sheet-beats" }, svg);
const spbPx = x(60 / (4 * grid().bpm)) - x(0);
for (const b of beatLines(t0, t1, grid())) {
if (b.weight === 0 && spbPx < 10) continue;
el("line", { x1: x(b.t), x2: x(b.t), y1: 0, y2: PLOT_HEIGHT,
class: `sheet-beat sheet-beat-${b.weight}` }, beats);
}
}
const layer = el("g", { class: "sheet-notes" }, svg);
const pad = Math.min(2, rowH * 0.12);
const drawNote = (n, current) => {
const cls = ["sheet-note", `sheet-note-${KINDS[n.kind] ?? "normal"}`];
if (!current) cls.push("sheet-note-context");
if (current && n.low_confidence) cls.push("sheet-note-low");
if (current && selected.has(n.id)) cls.push("sheet-note-selected");
const rect = el("rect", { x: x(n.start), y: y(n.pitch) + pad,
width: Math.max(x(n.end) - x(n.start), 2), height: rowH - 2 * pad,
rx: Math.min(4, rowH / 3), class: cls.join(" ") }, layer);
if (current) rect.dataset.id = n.id;
el("title", {}, rect).textContent = `${n.text.trim() || "~"} · ${noteName(n.pitch)}`;
};
for (const l of review.lines) {
if (l.dropped || l.index === index) continue;
for (const n of l.notes) if (n.end >= t0 && n.start <= t1) drawNote(n, false);
}
for (const n of notes()) drawNote(n, true);
const marks = el("g", { class: "sheet-breaks" }, svg);
for (const slot of breakSlots()) {
const t = breakTime(slot);
const value = hist.present[slot.which];
const byHand = value !== null &&
(value !== baseline[slot.which] || review.breaks?.find(b => b.line === slot.line)?.moved);
const cls = `sheet-break${byHand ? " sheet-break-moved" : ""}`;
el("line", { x1: x(t), x2: x(t), y1: 0, y2: PLOT_HEIGHT, class: cls }, marks);
el("text", { x: x(t) + 3, y: 11, class: `${cls} sheet-break-label` }, marks)
.textContent = "↵";
}
syllables.replaceChildren(...notes().map(n => {
const s = div("sheet-syllable" + (selected.has(n.id) ? " sheet-syllable-selected" : ""),
n.text.trim() || "~");
s.style.left = `${x(n.start)}px`;
s.dataset.id = n.id;
return s;
}));
// over the notes: the curve matters most where it agrees with one
if (curve) {
const g = el("g", { class: "sheet-curve" }, svg);
const from = Math.max(0, Math.floor((t0 - curve.start) / curve.step));
const to = Math.min(curve.pitch.length, Math.ceil((t1 - curve.start) / curve.step));
let pts = [];
const flush = () => {
if (pts.length > 1) el("polyline", { points: pts.join(" ") }, g);
pts = [];
};
for (let k = from; k < to; k++) {
const p = curve.pitch[k];
if (p === null || p < lo - 0.5 || p > hi + 0.5) { flush(); continue; }
pts.push(`${x(curve.start + k * curve.step).toFixed(1)},${(y(p) + rowH / 2).toFixed(1)}`);
}
flush();
}
words.replaceChildren(...line.words.map(w => {
const node = div("sheet-word", w.text);
node.style.left = `${x(w.start)}px`;
node.style.width = `${Math.max(x(w.end) - x(w.start), 2)}px`;
return node;
}));
const lines = visible();
const at = lines.findIndex(l => l.index === index);
title.textContent = `LINE ${at + 1} / ${lines.length}`;
prev.disabled = at <= 0;
next.disabled = at >= lines.length - 1;
const tags = [];
if (dirty()) tags.push(["unsaved", "sheet-badge-dirty"]);
if (line.edited) tags.push(["edited", "sheet-badge-edited"]);
if (notes().some(n => n.low_confidence)) tags.push(["low confidence", "sheet-badge-low"]);
badges.replaceChildren(...tags.map(([text, cls]) => div(`sheet-badge ${cls}`, text)));
paintTools();
paintGrid(typing);
}
function paintSelection() {
for (const node of root.querySelectorAll("[data-id]")) {
const on = selected.has(Number(node.dataset.id));
node.classList.toggle(node.tagName === "rect" ? "sheet-note-selected"
: "sheet-syllable-selected", on);
}
paintTools();
paintGrid();
}
function paintTools() {
const line = review.lines[index];
const editable = canEdit();
tools.undo.disabled = !editable || !hist.canUndo;
tools.redo.disabled = !editable || !hist.canRedo;
tools.split.disabled = tools.merge.disabled = !editable || one() === null;
for (const b of [tools.remove, tools.golden, tools.free, tools.rap]) {
b.disabled = !editable || !selected.size;
}
tools.discard.disabled = !dirty() || saving;
tools.save.disabled = !dirty() || saving;
tools.revert.disabled = !line.edited || saving;
}
// --- pointer -------------------------------------------------------------
function hit(e) {
const box = plot.getBoundingClientRect();
const px = e.clientX - box.left;
const py = e.clientY - box.top;
// by geometry, not the event's target: a render replaces the elements
// under the pointer, and the topmost note drawn wins
if (canEdit()) {
const brk = breakSlots().find(s => Math.abs(px - view.x(breakTime(s))) <= 5);
if (brk) return { px, py, note: null, brk, mode: "break" };
}
const pitch = view.pitch(py);
const note = [...notes()].reverse().find(n => n.pitch === pitch &&
px >= view.x(n.start) - 1 && px <= view.x(n.end) + 1) ?? null;
let mode = null;
if (note) {
const left = view.x(note.start);
const right = view.x(note.end);
mode = right - left > 3 * EDGE && px >= right - EDGE ? "end"
: right - left > 3 * EDGE && px <= left + EDGE ? "start" : "move";
}
return { px, py, note, brk: null, mode };
}
plot.addEventListener("pointermove", (e) => {
if (!view) return;
if (!drag) {
const { note, brk, mode } = hit(e);
plot.style.cursor = brk ? "col-resize"
: !note || !canEdit() ? "crosshair"
: mode === "move" ? "grab" : "ew-resize";
return;
}
const box = plot.getBoundingClientRect();
const px = e.clientX - box.left;
const py = e.clientY - box.top;
if (Math.abs(px - drag.px) + Math.abs(py - drag.py) > 3) drag.moved = true;
if (drag.moved && drag.brk) {
const { which, lo, hi } = drag.brk;
breakPreview = { which, t: edit.placeBreak(view.time(px), lo, hi, grid()) };
render();
return;
}
if (!drag.moved || !drag.note) return;
if (drag.mode === "move") {
const spbPx = view.x(60 / (4 * grid().bpm)) - view.x(0);
const beats = Math.round((px - drag.px) / spbPx);
const semis = Math.round((drag.py - py) / view.rowH);
preview = edit.move(drag.base, selected, beats, semis, grid());
} else {
preview = edit.resize(drag.base, drag.note.id, drag.mode, view.time(px), grid());
}
render();
});
plot.addEventListener("pointerdown", (e) => {
if (!view || e.button !== 0) return;
// the browser's own focus handling would otherwise take focus off the
// sheet, and with it the keyboard
e.preventDefault();
plot.focus();
const { px, py, note, brk, mode } = hit(e);
if (px < GUTTER) return;
if (brk) plot.setPointerCapture(e.pointerId);
if (note && canEdit()) {
if (e.shiftKey) {
selected = new Set(selected);
if (selected.has(note.id)) selected.delete(note.id); else selected.add(note.id);
} else if (!selected.has(note.id)) {
selected = new Set([note.id]);
}
plot.setPointerCapture(e.pointerId);
plot.style.cursor = mode === "move" ? "grabbing" : "ew-resize";
paintSelection();
}
drag = { px, py, note, brk, mode, base: notes(), moved: false };
});
const endDrag = (e, cancelled) => {
if (!drag) return;
const d = drag;
drag = null;
if (breakPreview) {
const { which, t } = breakPreview;
breakPreview = null;
if (cancelled) render(); else changeBreak(which, t);
} else if (preview) {
const next = preview;
preview = null;
if (cancelled) render(); else change(next);
} else if (!d.moved && !d.note && !d.brk && !cancelled) {
selected = new Set();
render();
seek(view.time(d.px));
}
if (plot.hasPointerCapture?.(e.pointerId)) plot.releasePointerCapture(e.pointerId);
};
plot.addEventListener("pointerup", (e) => endDrag(e, false));
plot.addEventListener("pointercancel", (e) => endDrag(e, true));
plot.addEventListener("dblclick", (e) => {
if (!view || !canEdit()) return;
const { px, py, note, brk } = hit(e);
if (px < GUTTER) return;
if (brk) { changeBreak(brk.which, null); return; } // back to automatic
if (note) { editText(note.id); return; }
const out = edit.insert(notes(), view.time(px), view.pitch(py), grid());
change(out.notes, new Set([out.id]));
editText(out.id);
});
// --- syllable text -------------------------------------------------------
function editText(id) {
const note = notes().find(n => n.id === id);
if (!note) return;
const input = document.createElement("input");
input.className = "sheet-text-input";
input.value = note.text.trim();
input.style.left = `${view.x(note.start)}px`;
input.setAttribute("aria-label", "syllable");
syllables.append(input);
input.focus();
input.select();
let done = false;
const finish = (commit) => {
if (done) return;
done = true;
const value = input.value;
input.remove();
if (commit) {
const next = edit.setText(notes(), id, value);
if (next !== notes()) change(next);
}
plot.focus();
};
input.addEventListener("keydown", (e) => {
e.stopPropagation();
if (e.key === "Enter") finish(true);
else if (e.key === "Escape") finish(false);
});
input.addEventListener("blur", () => finish(true));
}
syllables.addEventListener("dblclick", (e) => {
const id = e.target.dataset?.id;
if (id) editText(Number(id));
});
// --- keyboard ------------------------------------------------------------
plot.addEventListener("keydown", (e) => {
if (!view) return;
const ctrl = e.ctrlKey || e.metaKey;
const key = e.key;
const handled = () => { e.preventDefault(); e.stopPropagation(); };
if (ctrl && key.toLowerCase() === "s") { handled(); save(); return; }
if (ctrl && key.toLowerCase() === "z") { handled(); e.shiftKey ? redo() : undo(); return; }
if (ctrl && key.toLowerCase() === "y") { handled(); redo(); return; }
if (key === "PageUp") { handled(); page(-1); return; }
if (key === "PageDown") { handled(); page(1); return; }
if (ctrl || e.altKey || !canEdit()) return;
const ns = notes();
if (key === "Tab") {
handled();
if (!ns.length) return;
const at = ns.findIndex(n => selected.has(n.id));
const to = at < 0 ? (e.shiftKey ? ns.length - 1 : 0)
: (at + (e.shiftKey ? -1 : 1) + ns.length) % ns.length;
selected = new Set([ns[to].id]);
render();
return;
}
if (key === "?") { handled(); toggleKeys(); return; }
if (key === "Escape") {
handled();
if (!keys.hidden) showKeys(false);
else { selected = new Set(); render(); }
return;
}
if (key.toLowerCase() === "p") { handled(); playCurrent(); return; }
if (!selected.size) return;
const arrows = { ArrowLeft: -1, ArrowRight: 1 };
if (key in arrows) {
handled();
change(e.shiftKey ? edit.stretch(ns, selected, arrows[key], grid())
: edit.move(ns, selected, arrows[key], 0, grid()));
} else if (key === "ArrowUp" || key === "ArrowDown") {
handled();
change(edit.move(ns, selected, 0, (key === "ArrowUp" ? 1 : -1) * (e.shiftKey ? 12 : 1), grid()));
} else if (key === "Delete" || key === "Backspace") {
handled(); removeSelected();
} else if (key === "Enter" && one() !== null) {
handled(); editText(one());
} else if (key.toLowerCase() === "s") {
handled(); splitSelected();
} else if (key.toLowerCase() === "m") {
handled(); mergeSelected();
} else if ({ g: "*", f: "F", r: "R" }[key.toLowerCase()]) {
handled(); kind({ g: "*", f: "F", r: "R" }[key.toLowerCase()]);
}
});
window.addEventListener("beforeunload", (e) => {
if (dirty()) e.preventDefault();
});
new ResizeObserver(() => render()).observe(plot);
return {
// keeps the line on show when the song is the same one, refreshed
load(id, data) {
const same = id === jobId;
review = data;
root.hidden = false;
if (!same) {
jobId = id;
curve = null;
gridDraft = null;
fetch(`/api/jobs/${id}/pitch`)
.then(r => (r.ok ? r.json() : null))
.then(c => { if (jobId === id) { curve = c; render(); } })
.catch(() => {});
}
const lines = visible();
if (!lines.length) { root.hidden = true; return; }
open(same && lines.some(l => l.index === index) ? index : lines[0].index);
},
clear() {
review = null;
jobId = null;
curve = null;
index = null;
view = layout = hist = baseline = preview = gridDraft = null;
root.hidden = true;
},
// playback following the song: never at the cost of unsaved edits
showLine(i) {
if (review && i !== index && !dirty()) open(i);
},
isDirty: dirty,
setTime(t) {
if (!view) return;
const inside = t >= view.t0 && t <= view.t1;
playhead.hidden = !inside;
if (inside) playhead.style.left = `${view.x(t)}px`;
},
};
}
+231 -14
View File
@@ -9,14 +9,17 @@ import uuid
from dataclasses import asdict, dataclass, field
from pathlib import Path
from fastapi import FastAPI, File, Form, HTTPException, UploadFile
import numpy as np
from fastapi import Body, FastAPI, File, Form, HTTPException, UploadFile
from fastapi.concurrency import run_in_threadpool
from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles
from . import edits, grid
from . import rebuild as rebuild_policy
from .config import Config
from .lyrics import normalize_lyrics
from .notes import CONF_MASK
from .pipeline import (STAGE_ORDER, PipelineError, SongInput, read_warnings,
run_pipeline, song_slug)
from .ultrastar import get_header, parse_ultrastar
@@ -77,7 +80,7 @@ def _cleanup_uploads(cfg: Config, job_id: str, video_path: Path | None) -> None:
Covers are left behind: they are small, and a rebuild inherits cover_path, so
removing one would silently strip #COVER from the rebuilt song. A rebuild can
also inherit its parent's upload, so the directory is not always uploads/<job_id>.
Never raises — the caller has already marked the job done.
Never raises, so cleaning up cannot fail a job that otherwise succeeded.
"""
uploads = (cfg.work_dir / "uploads").resolve()
dirs = {uploads / job_id}
@@ -132,6 +135,7 @@ def create_app(cfg: Config, runner=run_pipeline) -> FastAPI:
jobs: dict[str, Job] = {}
order: list[str] = []
lock = threading.Lock()
edit_lock = threading.Lock()
todo: queue.Queue[str] = queue.Queue()
for job in _discover_jobs(cfg):
@@ -193,6 +197,7 @@ def create_app(cfg: Config, runner=run_pipeline) -> FastAPI:
if m := STAGE_LINE.match(msg):
job.stage = m[1]
status, error = "done", None
try:
force_from = _prepare_rebuild(job, log)
_write_sidecar(job)
@@ -201,18 +206,15 @@ def create_app(cfg: Config, runner=run_pipeline) -> FastAPI:
job.output_dir = str(result.output_dir)
job.warnings = list(result.warnings)
_drop_superseded_output(job, log)
with lock:
job.status = "done"
# uploads survive a failure so the job stays rerunnable; acquire
# has already copied the video into the work dir by now.
_cleanup_uploads(cfg, job_id, job.inp.video_path)
except BaseException as e:
with lock:
job.status = "failed"
job.error = str(e)
finally:
with lock:
job.finished_at = time.time()
status, error = "failed", str(e)
# last, and all at once: a client that sees the job finished must
# find everything finishing it does already done
with lock:
job.status, job.error, job.finished_at = status, error, time.time()
threading.Thread(target=worker, daemon=True).start()
@@ -329,17 +331,77 @@ def create_app(cfg: Config, runner=run_pipeline) -> FastAPI:
rebuild_policy.plan(parent.inp, inp), force_from))
return _enqueue(new_id, inp, rebuilt_from=job_id, force_stage=force_from)
def _current_songs() -> list[Job]:
"""The job that stands for each song now, newest first."""
with lock:
# a job that was rebuilt is history: its rebuild is the song now,
# possibly under another slug
rebuilt = {j.rebuilt_from for j in jobs.values()}
latest = {song_slug(j.inp): j for j in (jobs[i] for i in order)
if j.id not in rebuilt}
position = {job_id: n for n, job_id in enumerate(order)}
return sorted(latest.values(), key=lambda j: position[j.id], reverse=True)
def _edit_counts(job: Job) -> dict:
"""How much of the song was edited by hand, from its edits file."""
path = cfg.work_dir / song_slug(job.inp) / edits.EDITS_FILE
try:
data = json.loads(path.read_text())
except (OSError, ValueError):
return {"edited_lines": 0, "stale_edits": 0}
return {"edited_lines": len(data.get("lines", {})),
"stale_edits": len(data.get("stale", []))}
@app.get("/api/rebuild-all")
def rebuild_all_candidates():
return [{"id": j.id, "title": j.title, "artist": j.artist,
"status": j.status, "rebuilt_from": j.rebuilt_from,
"finished_at": j.finished_at, **_edit_counts(j)}
for j in _current_songs()]
@app.post("/api/rebuild-all", status_code=201)
def rebuild_all(force_from: str = Form(...), ids: list[str] | None = Form(None)):
"""Rebuild every song, or the ones in `ids`, from `force_from` on, for
when songtrace itself improved that stage. Each is an ordinary rebuild
with unchanged inputs."""
if force_from not in STAGE_ORDER:
raise HTTPException(400, f"unknown stage {force_from!r}")
songs = _current_songs()
queued, skipped = [], []
if ids is not None:
current = {j.id for j in songs}
skipped = [{"id": i, "reason": "no longer the current job for its song"}
for i in ids if i not in current]
songs = [j for j in songs if j.id in set(ids)]
for job in songs:
if job.status in ("queued", "running"):
skipped.append({"id": job.id, "reason": "still in progress"})
continue
try:
_require_source(job.inp, job.inp, force_from)
except HTTPException as e:
skipped.append({"id": job.id, "reason": e.detail})
continue
queued.append(_enqueue(uuid.uuid4().hex, job.inp, rebuilt_from=job.id,
force_stage=force_from)["id"])
return {"ids": queued, "skipped": skipped}
@app.get("/api/stages")
def list_stages():
return {"stages": list(STAGE_ORDER)}
@app.get("/api/jobs")
def list_jobs():
# edits belong to the song as it is now, so only the job standing for
# it carries them; the runs it replaced would claim them twice
counts = {j.id: _edit_counts(j) for j in _current_songs()}
none = {"edited_lines": 0, "stale_edits": 0}
with lock:
return [{"id": j.id, "title": j.title, "artist": j.artist,
"status": j.status, "stage": j.stage,
"created_at": j.created_at, "started_at": j.started_at,
"finished_at": j.finished_at, "rebuilt_from": j.rebuilt_from}
"finished_at": j.finished_at, "rebuilt_from": j.rebuilt_from,
**counts.get(j.id, none)}
for j in (jobs[i] for i in reversed(order))]
@app.get("/api/jobs/{job_id}")
@@ -423,7 +485,9 @@ def create_app(cfg: Config, runner=run_pipeline) -> FastAPI:
if job.status != "done":
raise HTTPException(404)
work = cfg.work_dir / song_slug(job.inp)
notes_file = work / "notes_sec.json"
notes_file = work / edits.FINAL_FILE
if not notes_file.exists():
notes_file = work / "notes_sec.json"
if not notes_file.exists():
raise HTTPException(404)
notes_data = json.loads(notes_file.read_text())
@@ -443,13 +507,166 @@ def create_app(cfg: Config, runner=run_pipeline) -> FastAPI:
for n in notes_data["notes"]:
by_line.setdefault(n["line_index"], []).append({
"start": n["start"], "end": n["start"] + n["duration"],
"pitch": n["pitch"], "low_confidence": n["low_confidence"]})
"pitch": n["pitch"], "low_confidence": n["low_confidence"],
"text": n["text"], "kind": n.get("kind", ":")})
edited = set(notes_data.get("edited_lines", []))
lines = [{"index": i, "text": " ".join(words), "dropped": i in dropped,
"edited": i in edited,
"notes": by_line.get(i, []), "words": words_by_line.get(i, [])}
for i, words in enumerate(ref.lines)]
stale = [{"text": " ".join(e["words"]), "notes": e["notes"]}
for e in notes_data.get("stale", [])]
media = {kind: _resolve_media(job, header) is not None
for kind, header in MEDIA_HEADERS.items()}
return {"lines": lines, "missing_spans": missing_spans, "media": media}
# each break with the line it comes before, the key a move is saved by
moved = set(notes_data.get("moved_breaks", []))
starts = sorted((n["start"], n["line_index"]) for n in notes_data["notes"]
if "line_index" in n)
breaks = []
for t in notes_data.get("breaks", []):
after = next((li for start, li in starts if start >= t), None)
if after is not None:
breaks.append({"line": after, "time": t, "moved": after in moved})
return {"lines": lines, "missing_spans": missing_spans, "media": media,
"stale": stale, "breaks": breaks, "grid": _chart_grid(work),
"grid_edited": bool(notes_data.get("grid")), "auto_bpm": _auto_bpm(work)}
@app.get("/api/jobs/{job_id}/pitch")
def get_pitch(job_id: str):
"""The detected vocal pitch, to draw behind the notes: UltraStar
semitones (0 = C4) every `step` seconds, null where nothing is sung."""
job = _finished_job(job_id)
path = cfg.work_dir / song_slug(job.inp) / "pitch.npz"
if not path.exists():
raise HTTPException(404)
data = np.load(path)
times, f0, conf = data["times"], data["f0"], data["conf"]
# every other 10ms frame is still finer than a note can be drawn
stride = 2
f0, conf = f0[::stride], conf[::stride]
voiced = (conf >= CONF_MASK) & (f0 > 0)
semis = 9 + 12 * np.log2(np.where(voiced, f0, 440.0) / 440.0)
hop = float(times[1] - times[0]) if len(times) > 1 else 0.01
return {"start": float(times[0]) if len(times) else 0.0, "step": hop * stride,
"pitch": [round(float(v), 2) if ok else None
for v, ok in zip(semis, voiced)]}
def _chart_grid(work: Path) -> dict | None:
"""The beat grid the chart was built on, for snapping edits to it."""
txt = work / "song.txt"
if not txt.exists():
return None
song = parse_ultrastar(txt.read_text(encoding="utf-8"))
bpm, gap = get_header(song, "BPM"), get_header(song, "GAP")
if bpm is None or gap is None:
return None
return {"bpm": float(bpm.replace(",", ".")), "gap_ms": float(gap.replace(",", "."))}
def _editable(job_id: str) -> Job:
job = _finished_job(job_id)
slug = song_slug(job.inp)
with lock:
# a build of this song would overwrite the chart being edited
if any(j.status in ("queued", "running") and song_slug(j.inp) == slug
for j in jobs.values()):
raise HTTPException(409, "this song is being rebuilt")
if job.status != "done":
raise HTTPException(409, "only a finished song can be edited")
return job
def _edit(job_id: str, change) -> dict:
"""Apply `change` to the song's edits, then rewrite its charts."""
job = _editable(job_id)
work = cfg.work_dir / song_slug(job.inp)
lyrics = normalize_lyrics(job.inp.lyrics_text).lines
with edit_lock:
data = edits.load(work, lyrics)
try:
change(data, lyrics)
except ValueError as e:
raise HTTPException(400, str(e))
edits.save(work, data)
final = edits.finalize(work, lyrics)
# the editor's grid, else the pipeline's own tempo with GAP at
# the first note, as the grid stage would chart it
chosen = final["grid"] or {"bpm": _auto_bpm(work), "gap_ms": None}
charts = [work / "song.txt"]
if job.output_dir:
charts.append(Path(job.output_dir) / f"{song_slug(job.inp)}.txt")
for txt in charts:
if txt.exists():
grid.rechart(txt, final["notes"], final["breaks"],
chosen["bpm"], chosen["gap_ms"])
with lock:
job.warnings = list(final["warnings"])
return get_review(job_id)
def _auto_bpm(work: Path) -> float | None:
"""The tempo the pipeline chose: recorded by grid, or for a song built
before that was, the one its chart already has."""
try:
return float(json.loads((work / grid.TEMPO_FILE).read_text())["bpm"])
except (OSError, ValueError, KeyError):
g = _chart_grid(work)
return g["bpm"] if g else None
@app.put("/api/jobs/{job_id}/breaks")
def save_breaks(job_id: str, body: dict = Body(...)):
"""Move line breaks: {line: seconds}, keyed by the line each comes
before; null puts one back where the notes alone would place it."""
return _edit(job_id, lambda data, lyrics: edits.set_breaks(data, body))
@app.put("/api/jobs/{job_id}/grid")
def save_grid(job_id: str, body: dict = Body(...)):
"""Set the chart's BPM and GAP, kept through rebuilds like any edit."""
job = _editable(job_id)
work = cfg.work_dir / song_slug(job.inp)
tempo = work / grid.TEMPO_FILE
# a song built before grid recorded its tempo: the chart's, taken now
# while it is still the pipeline's, is what RESET must return to
if not tempo.exists() and (bpm := _auto_bpm(work)) is not None:
tempo.write_text(json.dumps({"bpm": bpm}))
return _edit(job_id, lambda data, lyrics: edits.set_grid(
data, body.get("bpm"), body.get("gap_ms")))
@app.delete("/api/jobs/{job_id}/grid")
def reset_grid(job_id: str):
"""Go back to the BPM the pipeline chose and GAP at the first note."""
def change(data, lyrics):
if not edits.clear_grid(data):
raise HTTPException(404, "the grid was not set in the editor")
return _edit(job_id, change)
def _line(lyrics, index: int) -> int:
if not 0 <= index < len(lyrics):
raise HTTPException(404, f"no line {index}")
return index
@app.put("/api/jobs/{job_id}/lines/{index}")
def save_line(job_id: str, index: int, body: dict = Body(...)):
"""Save a line as edited: its full note list, in seconds."""
notes = body.get("notes")
if not isinstance(notes, list):
raise HTTPException(400, "body must be {\"notes\": [...]}")
return _edit(job_id, lambda data, lyrics: edits.set_line(
data, _line(lyrics, index), notes))
@app.delete("/api/jobs/{job_id}/lines/{index}")
def revert_line(job_id: str, index: int):
"""Drop a line's edit, so it follows what the pipeline generates."""
def change(data, lyrics):
if not edits.clear_line(data, _line(lyrics, index)):
raise HTTPException(404, f"line {index} has no edit")
return _edit(job_id, change)
@app.delete("/api/jobs/{job_id}/stale/{n}")
def discard_stale(job_id: str, n: int):
"""Throw away an edit whose line's lyrics have since changed."""
def change(data, lyrics):
if not 0 <= n < len(data["stale"]):
raise HTTPException(404, f"no stale edit {n}")
data["stale"].pop(n)
return _edit(job_id, change)
return app
+129
View File
@@ -0,0 +1,129 @@
import assert from "node:assert/strict";
import test from "node:test";
import * as m from "../../src/songtrace/static/edit-model.js";
// 300 BPM: one UltraStar beat is 0.05s, starting at GAP = 1s
const GRID = { bpm: 300, gap_ms: 1000 };
const near = (a, b) => Math.abs(a - b) < 1e-9;
function line() {
return m.withIds([
{ start: 1.52, end: 1.71, pitch: 0, text: "one", kind: ":", low_confidence: false },
{ start: 1.8, end: 2.0, pitch: 2, text: " two", kind: ":", low_confidence: true },
]);
}
test("snapping lands on the beat grid from GAP", () => {
assert.ok(near(m.snap(1.52, GRID), 1.5));
assert.ok(near(m.snap(1.53, GRID), 1.55));
assert.ok(near(m.snap(0.97, GRID), 0.95)); // beats before GAP too
});
test("moving snaps to the grid, then shifts by beats and semitones", () => {
const notes = line();
const out = m.move(notes, new Set([notes[0].id]), 2, -3, GRID);
assert.ok(near(out[0].start, 1.6) && near(out[0].end, 1.8));
assert.equal(out[0].pitch, -3);
assert.equal(out[1], notes[1]); // untouched note is the same object
assert.equal(notes[0].start, 1.52); // input left alone
});
test("moving never goes before the start of the song", () => {
const notes = m.withIds([{ start: 0.1, end: 0.2, pitch: 0, text: "x", kind: ":" }]);
const out = m.move(notes, new Set([notes[0].id]), -100, 0, GRID);
assert.ok(near(out[0].start, 0));
});
test("resizing moves one edge and keeps at least a beat", () => {
const notes = line();
const id = notes[0].id;
assert.ok(near(m.resize(notes, id, "end", 1.9, GRID)[0].end, 1.9));
assert.ok(near(m.resize(notes, id, "end", 1.0, GRID)[0].end, 1.55));
const startMoved = m.resize(notes, id, "start", 1.4, GRID)[0];
assert.ok(near(startMoved.start, 1.4) && near(startMoved.end, 1.7));
});
test("stretching changes length by whole beats", () => {
const notes = line();
const out = m.stretch(notes, new Set([notes[1].id]), 2, GRID);
assert.ok(near(out[1].start, 1.8) && near(out[1].end, 2.1));
assert.ok(near(m.stretch(notes, new Set([notes[1].id]), -10, GRID)[1].end, 1.85));
});
test("splitting halves a note on a beat and holds the syllable", () => {
const notes = line();
const { notes: out, ids } = m.split(notes, notes[1].id, GRID);
assert.equal(out.length, 3);
assert.ok(near(out[1].end, 1.9) && near(out[2].start, 1.9));
assert.deepEqual([out[1].text, out[2].text], [" two", "~"]);
assert.equal(ids.length, 2);
// one beat cannot be split
const short = m.withIds([{ start: 1.0, end: 1.05, pitch: 0, text: "x", kind: ":" }]);
assert.equal(m.split(short, short[0].id, GRID).notes, short);
});
test("merging joins a note with the next, dropping a held ~", () => {
const notes = m.withIds([
{ start: 1.0, end: 1.2, pitch: 0, text: "sev", kind: ":" },
{ start: 1.2, end: 1.4, pitch: 0, text: "en", kind: ":" },
{ start: 1.4, end: 1.6, pitch: 0, text: "~", kind: ":" },
]);
const once = m.merge(notes, notes[0].id);
assert.deepEqual(once.map(n => n.text), ["seven", "~"]);
const twice = m.merge(once, once[0].id);
assert.deepEqual(twice.map(n => [n.text, n.end]), [["seven", 1.6]]);
assert.equal(m.merge(twice, twice[0].id), twice); // nothing after it
});
test("inserting adds a two-beat held note on the grid", () => {
const { notes, id } = m.insert(line(), 1.76, 4, GRID);
const added = notes.find(n => n.id === id);
assert.ok(near(added.start, 1.75) && near(added.end, 1.85));
assert.deepEqual([added.pitch, added.text, added.kind], [4, "~", ":"]);
assert.deepEqual(notes.map(n => n.text), ["one", "~", " two"]);
});
test("a kind toggles back to normal when every selected note has it", () => {
const notes = line();
const both = new Set(notes.map(n => n.id));
const golden = m.setKind(notes, both, "*");
assert.deepEqual(golden.map(n => n.kind), ["*", "*"]);
assert.deepEqual(m.setKind(golden, both, "*").map(n => n.kind), [":", ":"]);
const mixed = m.setKind(golden, new Set([notes[0].id]), "F");
assert.deepEqual(m.setKind(mixed, both, "F").map(n => n.kind), ["F", "F"]);
});
test("editing text keeps a word-initial space and refuses blank", () => {
const notes = line();
assert.equal(m.setText(notes, notes[1].id, "too")[1].text, " too");
assert.equal(m.setText(notes, notes[0].id, "won")[0].text, "won");
assert.equal(m.setText(notes, notes[0].id, " won")[0].text, " won");
assert.equal(m.setText(notes, notes[0].id, " "), notes);
});
test("saved notes carry duration and kind, not ids", () => {
const saved = m.toSaved(line());
assert.deepEqual(Object.keys(saved[0]).sort(), ["duration", "kind", "pitch", "start", "text"]);
assert.ok(near(saved[0].duration, 0.19));
});
test("history undoes and redoes, and a new change drops the redo", () => {
const h = m.history("a");
h.push("b");
h.push("c");
assert.equal(h.undo(), "b");
assert.equal(h.undo(), "a");
assert.equal(h.canUndo, false);
assert.equal(h.redo(), "b");
h.push("d");
assert.equal(h.canRedo, false);
assert.equal(h.present, "d");
});
test("a dragged break snaps to a beat but stays between its lines", () => {
assert.ok(near(m.placeBreak(1.52, 1.4, 1.8, GRID), 1.5));
assert.equal(m.placeBreak(1.0, 1.42, 1.8, GRID), 1.42); // not into the line before
assert.equal(m.placeBreak(2.5, 1.4, 1.78, GRID), 1.78); // nor into the line after
assert.equal(m.placeBreak(1.6, 1.9, 1.8, GRID), 1.9); // lines overlap: at the end of the first
});
+72
View File
@@ -0,0 +1,72 @@
import assert from "node:assert/strict";
import test from "node:test";
import { beatLines, isBlackKey, lineSpan, noteName, pitchRange }
from "../../src/songtrace/static/sheet.js";
test("note names follow UltraStar's C4 = 0", () => {
assert.equal(noteName(0), "C4");
assert.equal(noteName(9), "A4");
assert.equal(noteName(-1), "B3");
assert.equal(noteName(13), "C♯5");
assert.equal(noteName(-13), "B2");
});
test("black keys are the sharps, in any octave", () => {
assert.deepEqual([0, 1, 2, 3, 4, 5, 6].map(isBlackKey),
[false, true, false, true, false, false, true]);
assert.equal(isBlackKey(-11), true); // C#3
assert.equal(isBlackKey(-12), false); // C3
});
test("the pitch range pads the line's notes and keeps a minimum height", () => {
assert.deepEqual(pitchRange([0, 5]), { lo: -4, hi: 10 }); // 15 rows
assert.deepEqual(pitchRange([0, 20]), { lo: -3, hi: 23 }); // padded only
assert.deepEqual(pitchRange([]), { lo: -7, hi: 7 });
});
test("beat lines sit on the chart's grid with bar weights", () => {
// 300 BPM: an UltraStar beat is 60 / (4 * 300) = 0.05s, from GAP 1.0s
const lines = beatLines(1.0, 1.85, { bpm: 300, gap_ms: 1000 });
assert.equal(lines.length, 18);
assert.deepEqual(lines.slice(0, 5).map(b => b.weight), [2, 0, 0, 0, 1]);
assert.equal(lines[16].weight, 2);
assert.ok(Math.abs(lines[4].t - 1.2) < 1e-9);
});
test("beats before GAP keep their weights", () => {
const lines = beatLines(0.75, 1.0, { bpm: 300, gap_ms: 1000 });
assert.deepEqual(lines.map(b => [b.beat, b.weight]),
[[-5, 0], [-4, 1], [-3, 0], [-2, 0], [-1, 0], [0, 2]]);
});
test("a line spans its notes and words", () => {
assert.deepEqual(lineSpan({ notes: [{ start: 2, end: 3 }], words: [{ start: 1.5, end: 2.5 }] }),
{ start: 1.5, end: 3 });
assert.equal(lineSpan({ notes: [], words: [] }), null);
});
import { readFileSync } from "node:fs";
import { SHORTCUTS, comboParts } from "../../src/songtrace/static/sheet.js";
test("shortcut combos split into keys and mouse actions", () => {
assert.deepEqual(comboParts("Shift+click / Tab"), [
[{ text: "Shift", key: true }, { text: "click", key: false }],
[{ text: "Tab", key: true }],
]);
assert.deepEqual(comboParts("Page Up"), [[{ text: "Page Up", key: true }]]);
});
test("every letter key the shortcut list names is one the sheet handles", () => {
const source = readFileSync(new URL("../../src/songtrace/static/sheet.js", import.meta.url), "utf8");
const handler = source.slice(source.indexOf('plot.addEventListener("keydown"'));
const letters = SHORTCUTS.flatMap(([, entries]) => entries)
.flatMap(([combo]) => comboParts(combo).flat())
.filter(p => p.key && /^[A-Z]$/.test(p.text))
.map(p => p.text.toLowerCase());
assert.ok(letters.length >= 6);
for (const letter of new Set(letters)) {
assert.ok(handler.includes(`"${letter}"`) || handler.includes(`${letter}:`),
`the list names ${letter.toUpperCase()} but the sheet does not handle it`);
}
});
+75
View File
@@ -1,11 +1,23 @@
import json
import numpy as np
import pytest
import soundfile as sf
from songtrace.config import Config
from songtrace.lyrics import normalize_lyrics
from songtrace.pipeline import Ctx, PipelineError, SongInput
from songtrace.stages import align
from songtrace.whisper_client import TranscriptSegment
REAL_UNTRANSCRIBED_SINGING = align.untranscribed_singing
@pytest.fixture(autouse=True)
def no_untranscribed_singing(monkeypatch):
"""The tests have no vocals.wav; those about missed singing set their own."""
monkeypatch.setattr(align, "untranscribed_singing", lambda vocals, segments: [])
LYRICS = "it's one two three\nfour five six seven eight"
@@ -160,3 +172,66 @@ def test_align_flags_missing_span_for_unmatched_singing(tmp_path, monkeypatch):
meta = json.loads((tmp_path / "w" / "align_meta.json").read_text())
assert meta["dropped_lines"] == []
assert meta["missing_spans"] == [[19.0, 24.0]]
def _tone(seconds, sr, level):
t = np.arange(int(seconds * sr)) / sr
return level * np.sin(2 * np.pi * 220 * t)
def test_untranscribed_singing_finds_sung_audio_the_transcript_skips(
tmp_path):
sr = 16000
rng = np.random.default_rng(0)
audio = np.concatenate([
_tone(2, sr, 0.3), # 0-2s: sung and transcribed
np.zeros(2 * sr), # 2-4s: silence
_tone(5, sr, 0.3), # 4-9s: sung, never transcribed
np.zeros(3 * sr), # 9-12s: silence
_tone(1, sr, 0.3), # 12-13s: too short to count
np.zeros(3 * sr),
]) + 0.001 * rng.standard_normal(16 * sr) # separation bleed
path = tmp_path / "vocals.wav"
sf.write(path, audio.astype(np.float32), sr)
segments = [TranscriptSegment(0.0, 2.0, "one two")]
spans = REAL_UNTRANSCRIBED_SINGING(path, segments)
assert len(spans) == 1
assert spans[0][0] == pytest.approx(4.0, abs=0.1)
assert spans[0][1] == pytest.approx(9.0, abs=0.1)
def test_lines_whisper_missed_are_aligned_in_the_untranscribed_singing(
tmp_path, monkeypatch):
lyrics = "one two three\nfour five six\nseven eight nine"
transcript = [{"start": 1.0, "end": 3.0, "text": "one two three"},
{"start": 40.0, "end": 42.0, "text": "seven eight nine"}]
monkeypatch.setattr(align, "untranscribed_singing",
lambda vocals, segments: [(20.0, 26.0)])
rows = run_align(tmp_path, monkeypatch, lyrics, transcript)
check_invariants(rows, lyrics)
middle = [r for r in rows if r["line_index"] == 1]
assert all(19.0 <= r["start"] and r["end"] <= 27.0 for r in middle)
meta = json.loads((tmp_path / "w" / "align_meta.json").read_text())
assert meta["dropped_lines"] == []
def test_stars_bracket_each_line_and_take_no_words(tmp_path, monkeypatch):
lyrics = "one two\n...\nthree four"
transcript = [{"start": 0.0, "end": 10.0, "text": "one two three four"}]
seen = []
def spy(vocals_path, start, end, tokens, device):
seen.append(tokens)
# each star claims a long stretch, so a word given a star's span shows
spans = [(start + i, start + i + (5.0 if t == "*" else 0.5), 0.9)
for i, t in enumerate(tokens)]
return spans
monkeypatch.setattr(align, "force_align_section", spy)
ctx = make_ctx(tmp_path, lyrics, transcript)
align.align_stage().run(ctx)
rows = json.loads((ctx.work / "alignment.json").read_text())
assert seen == [["*", "one", "two", "*", "three", "four", "*"]]
timed = [r for r in rows if r["word"] != "..."]
assert all(r["end"] - r["start"] == pytest.approx(0.5) for r in timed)
check_invariants(rows, lyrics)
+189
View File
@@ -0,0 +1,189 @@
import json
import pytest
from songtrace import edits
LYRICS = [["one", "two"], ["three", "four"], ["five", "six"]]
def gen(start, line, pitch=0, dur=0.4, low=False, text="la"):
return {"start": start, "duration": dur, "pitch": pitch, "text": text,
"low_confidence": low, "line_index": line}
def edited(*lines):
data = edits._empty(LYRICS)
for index, notes in lines:
edits.set_line(data, index, notes)
return data
def note(start, pitch=5, dur=0.4, text="la", kind=":"):
return {"start": start, "duration": dur, "pitch": pitch, "text": text, "kind": kind}
GENERATED = [gen(1.0, 0), gen(1.5, 0), gen(4.0, 1), gen(4.5, 1), gen(8.0, 2), gen(8.5, 2)]
BREAKS = [3.0, 6.0]
def test_no_edits_leaves_the_generated_song_alone():
assert edits.merge(GENERATED, BREAKS, edits._empty(LYRICS)) == (GENERATED, BREAKS)
def test_an_edited_line_replaces_its_generated_notes():
notes, _ = edits.merge(GENERATED, BREAKS, edited(
(1, [note(4.2, pitch=7, kind="*"), note(4.8, pitch=9)])))
line1 = [n for n in notes if n["line_index"] == 1]
assert [(n["start"], n["pitch"], n["kind"]) for n in line1] == [
(4.2, 7, "*"), (4.8, 9, ":")]
assert not any(n["low_confidence"] for n in line1)
assert [n["line_index"] for n in notes] == [0, 0, 1, 1, 2, 2]
def test_neighbours_give_way_to_an_edited_line():
# line 1 edited to run from 1.7s to 8.6s, over its neighbours' notes
notes, _ = edits.merge(GENERATED, BREAKS, edited(
(1, [note(1.7), note(8.2, dur=0.4)])))
by_line = {}
for n in notes:
by_line.setdefault(n["line_index"], []).append(
(round(n["start"], 2), round(n["start"] + n["duration"], 2)))
assert by_line[0] == [(1.0, 1.4), (1.5, 1.7)] # cut back to end at 1.7s
assert by_line[2] == [(8.6, 8.9)] # 8.0 note gone, 8.5 starts at 8.6
def test_breaks_beside_an_edited_line_move_and_the_rest_stay():
data = edited((2, [note(9.0), note(9.5)]))
_, breaks = edits.merge(GENERATED, BREAKS, data)
assert breaks[0] == 3.0 # lines 0-1: generated
assert breaks[1] == pytest.approx((4.9 + 9.0) / 2) # lines 1-2: re-centred
def test_an_edit_follows_its_line_when_lines_are_added_above(tmp_path):
data = edited((1, [note(4.0)]))
edits.save(tmp_path, data)
moved = edits.load(tmp_path, [["zero"]] + LYRICS)
assert list(moved["lines"]) == ["2"] and moved["lines"]["2"]["line_index"] == 2
assert moved["stale"] == []
assert json.loads((tmp_path / edits.EDITS_FILE).read_text())["lyrics"][0] == ["zero"]
def test_an_edit_whose_words_changed_is_set_aside(tmp_path):
edits.save(tmp_path, edited((1, [note(4.0)]), (2, [note(8.0)])))
changed = [LYRICS[0], ["three", "more"], LYRICS[2]]
data = edits.load(tmp_path, changed)
assert list(data["lines"]) == ["2"]
assert [s["words"] for s in data["stale"]] == [["three", "four"]]
def test_validate_notes_checks_and_orders():
assert [n["start"] for n in edits.validate_notes([note(2.0), note(1.0)])] == [1.0, 2.0]
for bad in (note(1.0, kind="X"), note(1.0, dur=0), note(-1.0), note(1.0, text=" "),
{"start": 1.0}):
with pytest.raises(ValueError):
edits.validate_notes([bad])
def test_saving_no_edits_removes_the_file(tmp_path):
edits.save(tmp_path, edited((0, [note(1.0)])))
data = edits.load(tmp_path, LYRICS)
edits.clear_line(data, 0)
edits.save(tmp_path, data)
assert not (tmp_path / edits.EDITS_FILE).exists()
def _write_generated(work, notes, warnings):
(work / "notes_sec.json").write_text(json.dumps(
{"notes": notes, "breaks": BREAKS, "warnings": warnings}))
def test_finalize_without_edits_passes_warnings_through(tmp_path):
_write_generated(tmp_path, GENERATED, ["anything at all"])
final = edits.finalize(tmp_path, LYRICS)
assert final["warnings"] == ["anything at all"]
assert final["edited_lines"] == [] and final["stale"] == []
assert json.loads((tmp_path / edits.FINAL_FILE).read_text()) == final
def test_finalize_stops_reporting_a_line_the_user_fixed(tmp_path):
notes = [gen(1.0, 0), gen(4.0, 1, low=True), gen(8.0, 2)]
_write_generated(tmp_path, notes, ["1 low-confidence note near 0:04–0:04",
"missing lyrics near 0:20–0:25"])
edits.save(tmp_path, edited((1, [note(4.0)])))
final = edits.finalize(tmp_path, LYRICS)
assert final["warnings"] == ["missing lyrics near 0:20–0:25"]
assert final["edited_lines"] == [1]
def test_finalize_reports_stale_edits(tmp_path):
_write_generated(tmp_path, GENERATED, [])
edits.save(tmp_path, edited((1, [note(4.0)])))
final = edits.finalize(tmp_path, [LYRICS[0], ["other"], LYRICS[2]])
assert final["warnings"] == ["1 edited line no longer matches the lyrics; "
"set aside in the editor"]
def test_a_grid_set_in_the_editor_is_checked():
data = edits._empty(LYRICS)
edits.set_grid(data, "240.5", 1500)
assert data["grid"] == {"bpm": 240.5, "gap_ms": 1500.0}
for bpm, gap in ((0, 0), (-1, 0), (float("nan"), 0), (3000, 0), (300, -1), ("x", 0)):
with pytest.raises(ValueError):
edits.set_grid(data, bpm, gap)
def test_a_grid_alone_keeps_the_edits_file_and_survives_new_lyrics(tmp_path):
data = edits._empty(LYRICS)
edits.set_grid(data, 300, 1000)
edits.save(tmp_path, data)
assert (tmp_path / edits.EDITS_FILE).exists()
loaded = edits.load(tmp_path, [["zero"]] + LYRICS)
assert loaded["grid"] == {"bpm": 300.0, "gap_ms": 1000.0}
assert edits.clear_grid(loaded) and not edits.clear_grid(loaded)
edits.save(tmp_path, loaded)
assert not (tmp_path / edits.EDITS_FILE).exists()
def test_finalize_reports_the_grid(tmp_path):
_write_generated(tmp_path, GENERATED, [])
assert edits.finalize(tmp_path, LYRICS)["grid"] is None
data = edits._empty(LYRICS)
edits.set_grid(data, 300, 1000)
edits.save(tmp_path, data)
assert edits.finalize(tmp_path, LYRICS)["grid"] == {"bpm": 300.0, "gap_ms": 1000.0}
def test_moved_breaks_are_checked():
data = edits._empty(LYRICS)
edits.set_breaks(data, {"1": 3.5, 2: "6.25"})
assert data["breaks"] == {"1": 3.5, "2": 6.25}
edits.set_breaks(data, {1: None})
assert data["breaks"] == {"2": 6.25}
for bad in ({0: 1.0}, {3: 1.0}, {"x": 1.0}, {1: -1}, {1: "soon"}, {1: float("inf")}):
with pytest.raises(ValueError):
edits.set_breaks(data, bad)
def test_a_moved_break_stays_put_between_its_lines():
data = edits._empty(LYRICS)
edits.set_breaks(data, {1: 2.5})
notes, breaks = edits.merge(GENERATED, BREAKS, data)
assert notes == GENERATED # moving a break locks no notes
assert breaks == [2.5, 6.0]
def test_a_moved_break_never_lands_inside_a_line():
data = edits._empty(LYRICS)
edits.set_breaks(data, {1: 1.0, 2: 9.0}) # into line 0's notes, past line 2's start
_, breaks = edits.merge(GENERATED, BREAKS, data)
assert breaks == [pytest.approx(1.9), 8.0]
def test_moved_breaks_follow_their_lines_and_keep_the_file(tmp_path):
data = edits._empty(LYRICS)
edits.set_breaks(data, {1: 3.5, 2: 6.5})
edits.save(tmp_path, data)
assert (tmp_path / edits.EDITS_FILE).exists()
moved = edits.load(tmp_path, [["zero"], LYRICS[0], LYRICS[1], ["other"]])
assert moved["breaks"] == {"2": 3.5} # line 2's words changed: its break goes
+85 -2
View File
@@ -169,7 +169,7 @@ def test_grid_stage_writes_song_txt(tmp_path, monkeypatch):
monkeypatch.setattr(grid_stage_mod.grid, "detect_bpm", lambda p: 120.0)
ctx = make_ctx(work, language="English", genre="Pop", year=1999)
stage = grid_stage_mod.grid_stage()
assert stage.outputs(ctx) == [work / "song.txt"]
assert stage.outputs(ctx) == [work / "song.txt", work / "notes_final.json"]
stage.run(ctx)
song = parse_ultrastar((work / "song.txt").read_text(encoding="utf-8"))
assert [k for k, _ in song.headers] == [
@@ -222,7 +222,7 @@ def test_grid_stage_honours_zero_bpm_override(tmp_path, monkeypatch):
monkeypatch.setattr(grid_stage_mod.grid, "detect_bpm", boom)
captured = {}
def fake_to_ultrastar(notes, breaks, file_bpm, headers):
def fake_to_ultrastar(notes, breaks, file_bpm, headers, gap_ms=None):
captured["bpm"] = file_bpm
return to_ultrastar(notes, breaks, 240.0, headers)
@@ -257,3 +257,86 @@ def test_grid_stage_reports_no_notes_as_pipeline_error(tmp_path, monkeypatch):
def test_grid_stage_registered():
names = [s.name for s in all_stages()]
assert names[-1] == "package" and names[-2] == "grid"
def _regenerate(work, pitch):
"""What a rebuild leaves: fresh notes for both lines, at `pitch`."""
(work / "notes_sec.json").write_text(json.dumps({
"notes": [{**note(1.0, 0.5, pitch, "one"), "line_index": 0},
{**note(3.0, 0.5, pitch, "three"), "line_index": 1}],
"breaks": [2.0], "warnings": []}))
def test_an_edited_line_survives_a_rebuild(tmp_path, monkeypatch):
from songtrace import edits
work = tmp_path / "w"
write_work(work)
monkeypatch.setattr(grid_stage_mod.grid, "detect_bpm", lambda p: 120.0)
ctx = make_ctx(work)
ctx.inp.lyrics_text = "one\nthree"
data = edits.load(work, [["one"], ["three"]])
edits.set_line(data, 1, [{"start": 3.2, "duration": 0.6, "pitch": 9,
"text": "three", "kind": "*"}])
edits.save(work, data)
for regenerated_pitch in (0, 4):
_regenerate(work, regenerated_pitch)
grid_stage_mod.grid_stage().run(ctx)
song = parse_ultrastar((work / "song.txt").read_text(encoding="utf-8"))
notes = [n for n in song.body if isinstance(n, Note)]
assert [(n.kind, n.pitch, n.text) for n in notes] == [
(":", regenerated_pitch, "one"), ("*", 9, "three")]
def test_rechart_replaces_the_notes_and_keeps_headers_and_tempo(tmp_path):
from songtrace.grid import rechart
from songtrace.ultrastar import serialize_ultrastar
txt = tmp_path / "song.txt"
txt.write_text(serialize_ultrastar(to_ultrastar(
[note(1.0, 0.5)], [], 300.0,
[("TITLE", "Song"), ("VIDEO", "Band - Song.mp4")])), encoding="utf-8")
rechart(txt, [{**note(2.0, 0.5, 7, "new"), "kind": "F"}], [])
song = parse_ultrastar(txt.read_text(encoding="utf-8"))
assert [k for k, _ in song.headers] == ["TITLE", "VIDEO", "BPM", "GAP"]
assert get_header(song, "BPM") == "300" and get_header(song, "GAP") == "2000"
assert [(n.kind, n.pitch, n.text) for n in song.body] == [("F", 7, "new")]
def test_an_explicit_gap_sets_the_grid():
# 300 BPM: 0.05s beats; notes at 1.02s and 1.27s against GAP 1.0s
song = to_ultrastar([note(1.02, 0.1), note(1.27, 0.1)], [], 300.0, [], gap_ms=1000)
assert get_header(song, "GAP") == "1000"
assert [n.beat for n in song.body] == [0, 5]
def test_a_gap_after_the_first_note_moves_back_by_whole_beats():
# GAP 1.13s is after the first note at 1.02s: back 3 beats to 0.98s, so
# the grid lines stay where they were set and the note is on beat 1
song = to_ultrastar([note(1.02, 0.1)], [], 300.0, [], gap_ms=1130)
assert get_header(song, "GAP") == "980"
assert song.body[0].beat == 1
def test_the_grid_stage_uses_the_editors_grid_and_records_its_own(tmp_path, monkeypatch):
from songtrace import edits
work = tmp_path / "w"
write_work(work)
monkeypatch.setattr(grid_stage_mod.grid, "detect_bpm", lambda p: 120.0)
ctx = make_ctx(work)
data = edits.load(work, [["x"]])
edits.set_grid(data, 300.0, 900)
edits.save(work, data)
grid_stage_mod.grid_stage().run(ctx)
song = parse_ultrastar((work / "song.txt").read_text(encoding="utf-8"))
assert (get_header(song, "BPM"), get_header(song, "GAP")) == ("300", "900")
assert json.loads((work / "tempo.json").read_text()) == {"bpm": 240.0}
def test_rechart_can_set_a_new_bpm_and_gap(tmp_path):
from songtrace.grid import rechart
from songtrace.ultrastar import serialize_ultrastar
txt = tmp_path / "song.txt"
txt.write_text(serialize_ultrastar(to_ultrastar([note(1.0, 0.5)], [], 240.0, [])),
encoding="utf-8")
rechart(txt, [note(1.02, 0.1)], [], 300.0, 1000)
song = parse_ultrastar(txt.read_text(encoding="utf-8"))
assert (get_header(song, "BPM"), get_header(song, "GAP")) == ("300", "1000")
+40 -5
View File
@@ -4,7 +4,7 @@ import numpy as np
import pytest
from songtrace.config import Config
from songtrace.notes import (
MIN_NOTE_SEC, TimedNote, build_notes, hz_to_ultrastar, segment_span,
MIN_NOTE_SEC, TimedNote, _warnings, build_notes, hz_to_ultrastar, segment_span,
syllabify,
)
from songtrace.pipeline import Ctx, SongInput
@@ -127,11 +127,36 @@ def test_build_notes_unvoiced_word_flagged():
assert warnings[0] == "1 low-confidence note near 0:01–0:01"
def test_build_notes_low_score_word_flagged():
def _scored_rows(scores, dur=0.4):
return [{"word": w, "start": 1.0 + i, "end": 1.0 + i + dur, "score": sc,
"line_index": 0}
for i, (w, sc) in enumerate(zip(["one", "two", "three", "four", "five"], scores))]
def test_build_notes_flags_a_word_scoring_far_below_the_rest():
times, f0, conf = flat_pitch()
rows = [{"word": "hey", "start": 1.0, "end": 1.4, "score": 0.1, "line_index": 0}]
notes, _, warnings = build_notes(rows, times, f0, conf, "English")
assert all(n.low_confidence for n in notes) and len(warnings) == 1
notes, _, warnings = build_notes(_scored_rows([0.4, 0.4, 0.05, 0.4, 0.4]),
times, f0, conf, "English")
assert [n.low_confidence for n in notes] == [False, False, True, False, False]
assert len(warnings) == 1
def test_build_notes_judges_scores_against_the_song_not_a_fixed_cutoff():
# sung vocals score low throughout; uniformly low is not a reason to flag
times, f0, conf = flat_pitch()
notes, _, warnings = build_notes(_scored_rows([0.05] * 5), times, f0, conf, "English")
assert not any(n.low_confidence for n in notes) and warnings == []
def test_build_notes_flags_words_too_short_or_too_long_to_be_sung():
times, f0, conf = flat_pitch()
for dur in (0.05, 3.5):
rows = _scored_rows([0.4] * 3)
rows[1]["end"] = rows[1]["start"] + dur
rows[2]["start"], rows[2]["end"] = rows[1]["end"] + 0.1, rows[1]["end"] + 0.5
notes, _, _ = build_notes(rows, times, f0, conf, "English")
# flat pitch: one note per word
assert [n.low_confidence for n in notes] == [False, True, False], dur
def test_build_notes_unvoiced_word_inherits_previous_pitch():
@@ -334,3 +359,13 @@ def test_notes_carry_line_index():
rows = [{"word": "hey", "start": 1.0, "end": 1.4, "score": 0.9, "line_index": 3}]
notes, _, _ = build_notes(rows, times, f0, conf, "English")
assert notes[0].line_index == 3
def test_warnings_group_flagged_notes_close_together():
def note(start, low):
return TimedNote(start, 0.3, 0, "la", low, 0)
notes = [note(10.0, True), note(10.5, False), note(11.0, True), # 0.7s apart
note(20.0, True)] # far from both
assert _warnings(notes) == [
"2 low-confidence notes near 0:10–0:11",
"1 low-confidence note near 0:20–0:20"]
+15
View File
@@ -0,0 +1,15 @@
import shutil
import subprocess
from pathlib import Path
import pytest
def test_static_js_unit_tests():
"""The page's own logic, run under Node's test runner."""
node = shutil.which("node")
if node is None:
pytest.skip("node is not installed")
tests = sorted(str(p) for p in (Path(__file__).parent / "js").glob("*.test.mjs"))
proc = subprocess.run([node, "--test", *tests], capture_output=True, text=True)
assert proc.returncode == 0, proc.stdout + proc.stderr
+30 -1
View File
@@ -1,6 +1,7 @@
from songtrace.lyrics import normalize_lyrics
from songtrace.whisper_client import TranscriptSegment
from songtrace.stages.transcribe import classify_lyrics_match, lyrics_similarity, vocal_sections
from songtrace.stages.transcribe import (classify_lyrics_match, lyrics_similarity,
recovered_spans, vocal_sections)
def seg(s, e, t=""):
return TranscriptSegment(start=s, end=e, text=t)
@@ -64,3 +65,31 @@ def test_classify_lyrics_match_scopes_missing_span_to_adlib_not_whole_section():
assert missing == [(5.0, 9.0)]
for a, b in missing:
assert not (a <= 1.0 < b) # matched line's start (1.0) untouched
THREE_LINES = "one two three\nfour five six\nseven eight nine"
def test_lines_whisper_missed_are_kept_when_singing_went_untranscribed():
ref = normalize_lyrics(THREE_LINES)
segs = [seg(0, 2, "one two three"), seg(20, 22, "seven eight nine")]
assert classify_lyrics_match(segs, ref)[0] == [1]
dropped, _ = classify_lyrics_match(segs, ref, unheard=[(8.0, 14.0)])
assert dropped == []
def test_untranscribed_singing_elsewhere_does_not_keep_a_line():
ref = normalize_lyrics(THREE_LINES)
segs = [seg(0, 2, "one two three"), seg(20, 22, "seven eight nine")]
dropped, _ = classify_lyrics_match(segs, ref, unheard=[(30.0, 35.0)])
assert dropped == [1]
def test_recovered_spans_are_the_singing_between_heard_neighbours():
ref = normalize_lyrics(THREE_LINES)
segs = [seg(10, 12, "four five six"), seg(20, 22, "seven eight nine")]
unheard = [(3.0, 9.0), (14.0, 18.0), (30.0, 35.0)]
# line 0 comes before anything heard, so only the singing before line 1
# can stand in for it; the other spans have no missing line around them
assert recovered_spans(segs, ref, unheard) == [(3.0, 9.0)]
assert classify_lyrics_match(segs, ref, unheard=unheard)[0] == []
+371 -6
View File
@@ -834,7 +834,7 @@ def test_narrow_stage_labels_stay_in_the_accessibility_tree():
def test_rebuild_mode_reuses_the_one_form():
html = _index_html()
assert html.count("<form") == 1
assert html.count('name="lyrics"') == 1 # one song form, not a rebuild copy
assert _rebuild_flaws(html) == []
@@ -1412,13 +1412,16 @@ def test_review_endpoint_groups_notes_by_line(tmp_path):
wait_status(client, job_id, "done")
body = client.get(f"/api/jobs/{job_id}/review").json()
assert body["lines"] == [
{"index": 0, "text": "hi", "dropped": False,
"notes": [{"start": 1.0, "end": 1.4, "pitch": 0, "low_confidence": False}],
{"index": 0, "text": "hi", "dropped": False, "edited": False,
"notes": [{"start": 1.0, "end": 1.4, "pitch": 0, "low_confidence": False,
"text": "hi", "kind": ":"}],
"words": []},
{"index": 1, "text": "there", "dropped": False,
"notes": [{"start": 5.0, "end": 5.3, "pitch": 2, "low_confidence": True}],
{"index": 1, "text": "there", "dropped": False, "edited": False,
"notes": [{"start": 5.0, "end": 5.3, "pitch": 2, "low_confidence": True,
"text": "there", "kind": ":"}],
"words": []},
{"index": 2, "text": "gone", "dropped": True, "edited": False, "notes": [],
"words": []},
{"index": 2, "text": "gone", "dropped": True, "notes": [], "words": []},
]
assert body["missing_spans"] == [{"start": 8.0, "end": 9.5}]
@@ -1746,3 +1749,365 @@ def test_submitting_crlf_lyrics_stores_them_with_lf(tmp_path):
client, _ = make_client(tmp_path, ok_runner)
job_id = make_done_job(client, lyrics="one\r\ntwo")
assert client.get(f"/api/jobs/{job_id}/input").json()["lyrics"] == "one\ntwo"
def test_rebuild_all_rebuilds_every_song_from_the_stage(tmp_path):
seen = []
client, _ = make_client(tmp_path, capture_runner_factory(seen))
first = make_done_job(client, title="One")
second = make_done_job(client, title="Two")
resp = client.post("/api/rebuild-all", data={"force_from": "pitch"})
assert resp.status_code == 201
body = resp.json()
assert len(body["ids"]) == 2 and body["skipped"] == []
jobs = [wait_status(client, i, "done") for i in body["ids"]]
assert {j["rebuilt_from"] for j in jobs} == {first, second}
assert [s["force_from"] for s in seen[2:]] == ["pitch", "pitch"]
by_title = lambda runs: sorted((r["inp"] for r in runs), key=lambda i: i.title)
assert by_title(seen[2:]) == by_title(seen[:2])
def test_rebuild_all_skips_jobs_that_were_already_rebuilt(tmp_path):
client, _ = make_client(tmp_path, capture_runner_factory([]))
parent = make_done_job(client)
renamed = client.post(f"/api/jobs/{parent}/rebuild",
data=rebuild_form(title="Renamed")).json()["id"]
wait_status(client, renamed, "done")
body = client.post("/api/rebuild-all", data={"force_from": "notes"}).json()
assert len(body["ids"]) == 1
job = wait_status(client, body["ids"][0], "done")
assert job["rebuilt_from"] == renamed
def test_rebuild_all_takes_the_latest_job_for_a_song(tmp_path):
client, _ = make_client(tmp_path, capture_runner_factory([]))
make_done_job(client)
latest = make_done_job(client)
body = client.post("/api/rebuild-all", data={"force_from": "grid"}).json()
assert len(body["ids"]) == 1
assert wait_status(client, body["ids"][0], "done")["rebuilt_from"] == latest
def test_rebuild_all_skips_songs_in_progress(tmp_path):
gate = threading.Event()
def blocking_runner(inp, cfg, log=print, force_from=None, stages=None):
gate.wait(timeout=5)
return PipelineResult(output_dir=cfg.output_dir / song_slug(inp))
client, _ = make_client(tmp_path, blocking_runner)
try:
busy = client.post("/api/jobs", data={
"title": "T", "artist": "A", "lyrics": "x", "youtube_url": "http://y"}).json()["id"]
body = client.post("/api/rebuild-all", data={"force_from": "pitch"}).json()
assert body == {"ids": [], "skipped": [{"id": busy, "reason": "still in progress"}]}
finally:
gate.set()
def test_rebuild_all_from_acquire_skips_a_song_whose_upload_is_gone(tmp_path):
client, _ = make_client(tmp_path, capture_runner_factory([]))
job_id = client.post("/api/jobs", data={"title": "T", "artist": "A", "lyrics": "x"},
files={"video": ("v.mp4", io.BytesIO(b"data"), "video/mp4")}
).json()["id"]
wait_status(client, job_id, "done")
shutil.rmtree(tmp_path / "work" / "uploads", ignore_errors=True)
body = client.post("/api/rebuild-all", data={"force_from": "acquire"}).json()
assert body["ids"] == []
assert body["skipped"][0]["id"] == job_id
assert "upload" in body["skipped"][0]["reason"]
def test_rebuild_all_rejects_an_unknown_stage(tmp_path):
client, _ = make_client(tmp_path, ok_runner)
make_done_job(client)
assert client.post("/api/rebuild-all", data={"force_from": "nope"}).status_code == 400
assert client.post("/api/rebuild-all", data={}).status_code == 422
def test_rebuild_all_lists_the_current_job_of_each_song(tmp_path):
client, _ = make_client(tmp_path, capture_runner_factory([]))
one = make_done_job(client, title="One")
two = make_done_job(client, title="Two")
rerun = client.post(f"/api/jobs/{one}/rerun").json()["id"]
wait_status(client, rerun, "done")
songs = client.get("/api/rebuild-all").json()
assert [(s["id"], s["title"], s["rebuilt_from"]) for s in songs] == [
(rerun, "One", one), (two, "Two", None)]
assert all(s["status"] == "done" and s["finished_at"] for s in songs)
def test_rebuild_all_rebuilds_only_the_songs_selected(tmp_path):
client, _ = make_client(tmp_path, capture_runner_factory([]))
make_done_job(client, title="One")
two = make_done_job(client, title="Two")
resp = client.post("/api/rebuild-all", data={"force_from": "pitch", "ids": [two]})
body = resp.json()
assert len(body["ids"]) == 1 and body["skipped"] == []
assert wait_status(client, body["ids"][0], "done")["rebuilt_from"] == two
def test_rebuild_all_reports_a_selected_job_that_was_since_rebuilt(tmp_path):
client, _ = make_client(tmp_path, capture_runner_factory([]))
old = make_done_job(client)
rerun = client.post(f"/api/jobs/{old}/rerun").json()["id"]
wait_status(client, rerun, "done")
body = client.post("/api/rebuild-all", data={"force_from": "pitch", "ids": [old]}).json()
assert body == {"ids": [], "skipped": [
{"id": old, "reason": "no longer the current job for its song"}]}
def test_a_job_is_only_reported_finished_once_it_has_finished(tmp_path, monkeypatch):
"""Done must mean done: a slow upload cleanup used to show a 'done' job
with its upload still there and no finished_at."""
from songtrace import web
real_cleanup = web._cleanup_uploads
def slow_cleanup(*args):
time.sleep(0.3)
real_cleanup(*args)
monkeypatch.setattr(web, "_cleanup_uploads", slow_cleanup)
client, cfg = make_client(tmp_path, ok_runner)
job_id = client.post("/api/jobs", data={"title": "T", "artist": "A", "lyrics": "x"},
files={"video": ("v.mp4", io.BytesIO(b"data"), "video/mp4")}
).json()["id"]
job = wait_status(client, job_id, "done")
assert job["finished_at"] is not None
assert not (cfg.work_dir / "uploads" / job_id / "v.mp4").exists()
def chart_runner(inp, cfg, log=print, force_from=None, stages=None):
"""Leaves what a real build does for the editor: generated notes and a
chart in the work dir, and the packaged chart."""
from songtrace.grid import to_ultrastar
from songtrace.ultrastar import serialize_ultrastar
work = cfg.work_dir / song_slug(inp)
work.mkdir(parents=True, exist_ok=True)
notes = [{"start": 1.0, "duration": 0.5, "pitch": 0, "text": "one",
"low_confidence": False, "line_index": 0},
{"start": 3.0, "duration": 0.5, "pitch": 2, "text": "three",
"low_confidence": True, "line_index": 1}]
warnings = ["1 low-confidence note near 0:03–0:03"]
(work / "notes_sec.json").write_text(json.dumps(
{"notes": notes, "breaks": [2.0], "warnings": warnings}))
chart = serialize_ultrastar(to_ultrastar(notes, [2.0], 300.0, [("TITLE", inp.title)]))
(work / "song.txt").write_text(chart, encoding="utf-8")
out = cfg.output_dir / song_slug(inp)
out.mkdir(parents=True, exist_ok=True)
(out / f"{song_slug(inp)}.txt").write_text(chart, encoding="utf-8")
return PipelineResult(output_dir=out, warnings=warnings)
def make_charted_job(client, lyrics="one\nthree"):
job_id = client.post("/api/jobs", data={
"title": "T", "artist": "A", "lyrics": lyrics, "youtube_url": "http://y"}).json()["id"]
wait_status(client, job_id, "done")
return job_id
EDIT = {"notes": [{"start": 3.1, "duration": 0.5, "pitch": 9, "text": "three",
"kind": "*"}]}
def test_saving_a_line_rewrites_both_charts(tmp_path):
from songtrace.ultrastar import Note, parse_ultrastar
client, cfg = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
resp = client.put(f"/api/jobs/{job_id}/lines/1", json=EDIT)
assert resp.status_code == 200
line = resp.json()["lines"][1]
assert line["edited"] and line["notes"][0]["kind"] == "*" and line["notes"][0]["pitch"] == 9
assert resp.json()["grid"] == {"bpm": 300.0, "gap_ms": 1000.0}
for txt in (cfg.work_dir / "A - T" / "song.txt", cfg.output_dir / "A - T" / "A - T.txt"):
notes = [n for n in parse_ultrastar(txt.read_text(encoding="utf-8")).body
if isinstance(n, Note)]
assert [(n.kind, n.pitch) for n in notes] == [(":", 0), ("*", 9)], txt
# the fixed line was the one being warned about
assert client.get(f"/api/jobs/{job_id}").json()["warnings"] == []
def test_reverting_a_line_returns_it_to_the_generated_notes(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
client.put(f"/api/jobs/{job_id}/lines/1", json=EDIT)
resp = client.delete(f"/api/jobs/{job_id}/lines/1")
assert resp.status_code == 200
line = resp.json()["lines"][1]
assert not line["edited"] and line["notes"][0]["pitch"] == 2
assert client.delete(f"/api/jobs/{job_id}/lines/1").status_code == 404
def test_line_edits_are_checked(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
assert client.put(f"/api/jobs/{job_id}/lines/5", json=EDIT).status_code == 404
bad = {"notes": [{**EDIT["notes"][0], "kind": "X"}]}
assert client.put(f"/api/jobs/{job_id}/lines/1", json=bad).status_code == 400
assert client.put(f"/api/jobs/{job_id}/lines/1", json={"x": 1}).status_code == 400
def test_a_song_cannot_be_edited_while_it_is_being_rebuilt(tmp_path):
gate = threading.Event()
def runner(inp, cfg, log=print, force_from=None, stages=None):
if force_from:
gate.wait(timeout=5)
return chart_runner(inp, cfg)
client, _ = make_client(tmp_path, runner)
try:
job_id = make_charted_job(client)
client.post(f"/api/jobs/{job_id}/rebuild-full")
assert client.put(f"/api/jobs/{job_id}/lines/1", json=EDIT).status_code == 409
finally:
gate.set()
def test_a_stale_edit_can_be_discarded(tmp_path):
from songtrace import edits
client, cfg = make_client(tmp_path, chart_runner)
first = make_charted_job(client)
client.put(f"/api/jobs/{first}/lines/1", json=EDIT)
# the same song again with that line's words changed
second = make_charted_job(client, lyrics="one\nthree more")
work = cfg.work_dir / "A - T"
assert len(edits.load(work, [["one"], ["three", "more"]])["stale"]) == 1
resp = client.delete(f"/api/jobs/{second}/stale/0")
assert resp.status_code == 200 and resp.json()["stale"] == []
assert not (work / edits.EDITS_FILE).exists()
assert client.delete(f"/api/jobs/{second}/stale/0").status_code == 404
def test_pitch_curve_is_in_note_semitones_with_gaps_where_unvoiced(tmp_path):
import numpy as np
def runner(inp, cfg, log=print, force_from=None, stages=None):
result = chart_runner(inp, cfg)
times = np.arange(0, 0.1, 0.01)
f0 = np.array([440.0] * 4 + [261.63] * 4 + [0.0] * 2) # A4, C4, nothing
conf = np.array([0.9] * 4 + [0.9, 0.9, 0.1, 0.1] + [0.9] * 2)
np.savez(cfg.work_dir / song_slug(inp) / "pitch.npz", times=times, f0=f0, conf=conf)
return result
client, _ = make_client(tmp_path, runner)
job_id = make_charted_job(client)
body = client.get(f"/api/jobs/{job_id}/pitch").json()
assert body["start"] == 0.0 and body["step"] == pytest.approx(0.02)
assert body["pitch"][:2] == [9.0, 9.0] # A4 is 9 above C4
assert body["pitch"][2] == pytest.approx(0.0, abs=0.01) # C4
assert body["pitch"][3:] == [None, None] # unsure, then silent
def test_pitch_curve_is_404_without_a_pitch_track(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
assert client.get(f"/api/jobs/{job_id}/pitch").status_code == 404
def _chart_header(path, key):
from songtrace.ultrastar import get_header, parse_ultrastar
return get_header(parse_ultrastar(path.read_text(encoding="utf-8")), key)
def test_setting_the_grid_recharts_and_survives_line_saves(tmp_path):
client, cfg = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
resp = client.put(f"/api/jobs/{job_id}/grid", json={"bpm": 240, "gap_ms": 900})
assert resp.status_code == 200
body = resp.json()
assert body["grid"] == {"bpm": 240.0, "gap_ms": 900.0} and body["grid_edited"]
packaged = cfg.output_dir / "A - T" / "A - T.txt"
assert (_chart_header(packaged, "BPM"), _chart_header(packaged, "GAP")) == ("240", "900")
client.put(f"/api/jobs/{job_id}/lines/1", json=EDIT)
assert (_chart_header(packaged, "BPM"), _chart_header(packaged, "GAP")) == ("240", "900")
def test_resetting_the_grid_returns_to_the_pipelines_tempo(tmp_path):
client, cfg = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
(cfg.work_dir / "A - T" / "tempo.json").write_text(json.dumps({"bpm": 300.0}))
client.put(f"/api/jobs/{job_id}/grid", json={"bpm": 240, "gap_ms": 900})
resp = client.delete(f"/api/jobs/{job_id}/grid")
assert resp.status_code == 200
body = resp.json()
assert body["grid"] == {"bpm": 300.0, "gap_ms": 1000.0} # GAP back at the first note
assert not body["grid_edited"] and body["auto_bpm"] == 300.0
assert client.delete(f"/api/jobs/{job_id}/grid").status_code == 404
def test_a_bad_grid_is_refused(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
for body in ({"bpm": 0, "gap_ms": 0}, {"bpm": 300, "gap_ms": -5}, {"bpm": "x"}, {}):
assert client.put(f"/api/jobs/{job_id}/grid", json=body).status_code == 400
def test_reset_on_a_song_built_before_tempo_was_recorded_restores_its_bpm(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client) # charted at 300 BPM, no tempo.json
client.put(f"/api/jobs/{job_id}/grid", json={"bpm": 240, "gap_ms": 900})
body = client.delete(f"/api/jobs/{job_id}/grid").json()
assert body["grid"]["bpm"] == 300.0 and body["auto_bpm"] == 300.0
def test_moving_a_break_recharts_and_is_reported(tmp_path):
from songtrace.ultrastar import LineBreak, parse_ultrastar
client, cfg = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
assert client.get(f"/api/jobs/{job_id}/review").json()["breaks"] == [
{"line": 1, "time": 2.0, "moved": False}]
resp = client.put(f"/api/jobs/{job_id}/breaks", json={"1": 2.8})
assert resp.status_code == 200
assert resp.json()["breaks"] == [{"line": 1, "time": 2.8, "moved": True}]
assert not any(line["edited"] for line in resp.json()["lines"])
packaged = cfg.output_dir / "A - T" / "A - T.txt"
body = parse_ultrastar(packaged.read_text(encoding="utf-8")).body
# 300 BPM: 0.05s beats from GAP 1.0s, so 2.8s is beat 36
assert [b.beat for b in body if isinstance(b, LineBreak)] == [36]
back = client.put(f"/api/jobs/{job_id}/breaks", json={"1": None}).json()
assert back["breaks"] == [{"line": 1, "time": 2.0, "moved": False}]
def test_a_bad_break_is_refused(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
for body in ({"0": 1.0}, {"9": 1.0}, {"1": -2}, {"1": "x"}):
assert client.put(f"/api/jobs/{job_id}/breaks", json=body).status_code == 400
def _listed(client, job_id):
return next(j for j in client.get("/api/jobs").json() if j["id"] == job_id)
def test_the_queue_reports_a_songs_edited_lines(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
assert _listed(client, job_id)["edited_lines"] == 0
client.put(f"/api/jobs/{job_id}/lines/1", json=EDIT)
assert _listed(client, job_id)["edited_lines"] == 1
assert _listed(client, job_id)["stale_edits"] == 0
def test_edit_counts_follow_the_song_to_its_latest_run(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
first = make_charted_job(client)
client.put(f"/api/jobs/{first}/lines/1", json=EDIT)
rerun = client.post(f"/api/jobs/{first}/rerun").json()["id"]
wait_status(client, rerun, "done")
assert _listed(client, rerun)["edited_lines"] == 1
assert _listed(client, first)["edited_lines"] == 0
candidates = client.get("/api/rebuild-all").json()
assert [(c["id"], c["edited_lines"]) for c in candidates] == [(rerun, 1)]
def test_the_queue_reports_edits_set_aside(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
first = make_charted_job(client)
client.put(f"/api/jobs/{first}/lines/1", json=EDIT)
second = make_charted_job(client, lyrics="one\nthree more")
# the next save looks at the edits against the new lyrics, as a rebuild does
client.put(f"/api/jobs/{second}/lines/0", json={"notes": [
{"start": 1.0, "duration": 0.5, "pitch": 0, "text": "one"}]})
listed = _listed(client, second)
assert (listed["edited_lines"], listed["stale_edits"]) == (1, 1)