Compare commits

...
Author SHA1 Message Date
branderson 33d66c2af8 README: lead with requirements and Docker, refresh for the editor
The README described a CLI with a minimal web UI and told people to fix
charts in Yass. Rewrite it around how songtrace is used now: what it
does, its requirements up front (an NVIDIA GPU, tested on 12 GB, Docker
with the NVIDIA Container Toolkit, a Whisper server of your own, disk for
models and per-song files), a Docker quick start including writing songs
straight into a game's collection, then reviewing, editing and
rebuilding in the web UI. The command line moves near the end.

Regenerate the screenshot to show the note sheet: the screenshot script
now seeds the featured song with syllables, a chart, a pitch track and a
line break, and always scrolls to the review. The example config no
longer sets a real person as #CREATOR, and uses a reserved example host
for Whisper.
2026-10-15 17:06:19 -07:00
branderson c3660d267a webui: keep a far line break from stretching the sheet
Since breaks could be moved, the sheet widened its window to show the
breaks either side of a line. Before an instrumental passage the break
sits mid-gap, sometimes 15s away, so the line's notes were crushed into
a corner of the sheet. The window now grows at most 3s past the line to
show a break; one further off is pinned to the edge, labelled with an
arrow toward where it lies, and still drags inward.
2026-10-15 17:06:19 -07:00
branderson 2d7bde3d74 Merge pull request 'webui: a note sheet with a real pitch axis and beat grid' (#30) from editor into main
Reviewed-on: #30
Reviewed-by: branderson <brad@branderson.io>
2026-10-15 16:52:44 -07:00
branderson 82147bf295 Merge remote-tracking branch 'origin/main' into editor-merge
# Conflicts:
#	tests/test_web.py
2026-10-15 16:36:27 -07:00
branderson b5e6cd326b Merge pull request 'webui: edit a song's notes on the sheet' (#31) from sheet-editing into editor
Reviewed-on: #31
Reviewed-by: branderson <brad@branderson.io>
2026-10-15 16:22:40 -07:00
branderson 449f742d4b Merge pull request 'webui: mark songs with hand edits in the queue and REBUILD ALL FROM…' (#32) from edited-marks into main
Reviewed-on: #32
Reviewed-by: branderson <brad@branderson.io>
2026-10-14 23:28:27 -07:00
branderson b781c6cb0c Merge pull request 'webui: a grouped keyboard shortcut panel for the sheet' (#33) from sheet-shortcuts into sheet-editing
Reviewed-on: #33
Reviewed-by: branderson <brad@branderson.io>
2026-10-14 23:27:53 -07:00
branderson 3fbf39f9d6 Merge pull request 'webui: set a song's BPM and GAP from the sheet' (#34) from grid-edit into sheet-shortcuts
Reviewed-on: #34
Reviewed-by: branderson <brad@branderson.io>
2026-10-14 23:17:45 -07:00
branderson 82cafa96bf Merge pull request 'webui: move line breaks on the sheet' (#35) from break-edit into grid-edit
Reviewed-on: #35
Reviewed-by: branderson <brad@branderson.io>
2026-10-14 18:54:21 -07:00
branderson 30e4f8f202 webui: move line breaks on the sheet
Line breaks were always placed by songtrace, midway between lines and
recomputed around edited ones, so a break could not be put where a singer
breathes. The sheet now draws the breaks either side of a line as markers
that drag along the beat grid, kept between the lines they separate, and
double-click back to automatic. Break moves share the line's undo history
and SAVE LINE, which saves the notes only if they changed, so moving a
break never locks either line's notes.

Moved breaks are stored in edits.json keyed by the line each comes before,
follow their lines when the lyrics change, and win over the computed break
on every rebuild, clamped so a break never lands inside a line's notes.
The review lists each break with that line and whether it was moved, and
PUT /api/jobs/{id}/breaks takes {line: seconds | null}. merge now passes
untouched notes through as they are rather than rebuilding them, which
added float noise to their durations.
2026-10-13 16:24:17 -07:00
branderson bdfd7d8429 webui: set a song's BPM and GAP from the sheet
The beat grid came from the detected tempo (or the BPM override) and a GAP
pinned to the first note, so it did not always line up with the music,
and snapping did not land on its beats. A GRID bar on the sheet now sets
both, redrawing the grid live as values are tried, with x2 and /2 for a
tempo detected at double or half speed and ALIGN TO NOTE to move GAP the
least that puts the selected note on a beat. Note editing pauses while a
trial grid is unapplied, so notes never snap to a grid that is then
cancelled.

The grid is stored with the song's edits, so every rebuild keeps it, and
grid charts with it in place of its own choice; it records that choice in
tempo.json for RESET to return to. A GAP after the first note moves back
by whole beats, as notes on negative beats are rejected by players. PUT
and DELETE /api/jobs/{id}/grid set and reset it, rewriting the charts in
place.
2026-10-13 16:08:57 -07:00
branderson 2aeb16decb webui: a grouped keyboard shortcut panel for the sheet
The sheet's controls were listed in one run-on line under it, which was
hard to scan and grew with every feature. Replace it with a panel grouped
the way an editor's reference card is (select, move, length, notes, kind,
syllable, line) with keys drawn as key caps, opened by a KEYS button or ?,
closed by Esc, and remembered open or closed across visits.

The list is data (SHORTCUTS), and a test checks every letter key it names
is one the sheet's key handler handles, so the two cannot drift apart.
2026-10-12 23:13:45 -07:00
branderson 9a11db5794 webui: mark songs with hand edits in the queue and REBUILD ALL FROM…
A song's edits are kept through any rebuild, but nothing said which songs
had them. GET /api/jobs and GET /api/rebuild-all now report each song's
edited and set-aside lines, read from its edits file, on the job standing
for the song only, since the runs it replaced would claim the same edits.
Queue cards show them as a badge, and the rebuild dialog lists them per
song and says hand edits are kept.
2026-10-12 19:04:33 -07:00
branderson 381a499f2b webui: edit a song's notes on the sheet
Notes on the sheet can now be selected and edited: dragged in time and
pitch, resized from either end, nudged and stretched from the keyboard,
split, merged, added, deleted, given a syllable or a kind (golden,
freestyle, rap), with undo/redo. Everything snaps to the chart's beat grid
and to semitones, as the chart itself will. SAVE LINE stores the line
through the edits API, which rewrites the charts in place; REVERT goes back
to the generated notes; a line's edits set aside because its lyrics changed
are listed with a way to discard them. Unsaved changes are not lost to
paging, following playback, switching songs or closing the tab.

The operations live in static/edit-model.js as pure functions, unit-tested
with node --test. The sheet hit-tests by geometry rather than event
targets, since a render replaces the elements under the pointer.
2026-10-11 02:37:00 -07:00
branderson 793a78972d webui: a note sheet with a real pitch axis and beat grid
The review drew each line's notes as thin bars in a 20px strip, with no
pitch axis or beat grid, so it was hard to tell where a note sat. Add a
sheet that shows one line at a time the way an UltraStar editor does: a
row per semitone, shaded like piano keys and named in a gutter, the
chart's own beat grid with bars every 4 and 16 beats, notes as blocks
styled by kind (golden, freestyle, rap) with low-confidence ones outlined,
neighbouring lines faded in for context, syllables and aligned words
beneath, and a playhead that follows the transport, paging with it.

The detected pitch curve is drawn over the notes, served by a new GET
/api/jobs/{id}/pitch in note semitones, so a note off the sung pitch
stands out. The sheet is its own ES module; its geometry is unit-tested
with node --test, run from pytest and skipped where Node is absent.
2026-10-11 01:19:12 -07:00
branderson fa4a78f278 Merge pull request 'Keep manual edits to a song across rebuilds' (#29) from edit-layer into main
Reviewed-on: #29
Reviewed-by: branderson <brad@branderson.io>
2026-10-11 01:10:17 -07:00
17 changed files with 2227 additions and 235 deletions
+171 -194
View File
@@ -1,142 +1,194 @@
# songtrace
Turn a song and its lyrics into a playable UltraStar chart.
A CLI and minimal web UI that turns a song — title, artist, lyrics you supply,
and a YouTube URL or local video file — into a playable
[UltraStar](https://usdx.eu/format/)-format song folder. Also compatible with
[Melody Mania](https://melodymania.org/).
Self-hosted karaoke chart builder for [UltraStar](https://usdx.eu/format/) and
compatible games such as [Melody Mania](https://melodymania.org/). Give it a
song's video and lyrics and it charts the notes, timing and pitch, ready for
your collection. Fix anything in the built-in editor; your edits survive when
you rebuild your whole collection after a songtrace update.
![Web UI](docs/images/webui.png)
The pipeline:
## What it does
1. **acquire** — download audio/video via `yt-dlp` (or use a local video file)
2. **separate** — split vocals/instrumental with Demucs
3. **transcribe** — send the vocal stem to your own intranet Whisper server
(OpenAI-compatible, or [onerahmet/openai-whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)),
as a sanity check that the supplied lyrics match the audio and to locate
vocal sections
4. **align** — forced-align the *supplied* reference lyrics against the vocal
stem with wav2vec2 (torchaudio's MMS_FA) to get word timings
5. **pitch** — track pitch on the vocal stem with torchcrepe
6. **notes** — turn word timings + pitch into UltraStar notes
7. **grid** — quantise notes to a beat grid
8. **package** — write the final song folder (`.txt` + `.mp3` + video + cover)
You give it a title, artist, the lyrics, and a YouTube URL or a video file.
songtrace writes a song folder (`.txt` chart, `.mp3`, video and cover) into
your collection:
The shipped `.mp3` is the full original mix (vocals included) — the
separated stems are used for analysis only and are not included in the
output.
1. **acquire**: downloads the video with `yt-dlp`, or takes your file
2. **separate**: splits vocals from the backing track (Demucs)
3. **transcribe**: has your Whisper server transcribe the vocals, to check
the lyrics match the recording and to find where the singing is
4. **align**: places each word of *your* lyrics on the vocals (wav2vec2)
5. **pitch**: tracks the sung pitch (torchcrepe)
6. **notes**: turns word timings and pitch into notes
7. **grid**: snaps the notes to a beat grid
8. **package**: writes the song folder
**The output is a best-effort draft**, not a finished song. Every run ends by
printing the low-confidence regions it found (e.g.
`3 low-confidence notes near 0:42–0:47`) — open the result in an UltraStar
editor such as [Yass](https://www.yass-along.com/) or
[USDX](https://usdx.eu/) and touch those spots up before playing.
The result is a strong draft, not a finished chart. The web UI flags the notes
it is least sure of, plays the song back against the chart, and lets you fix
what it got wrong before you sing it.
## Reviewing a song
## Requirements
The web UI plays the finished song folder back against the review, so you can
check the timing without loading it into a game first. Pick a song in the
queue and the review panel gets the packaged video with a transport above the
lyrics; as it plays, the line being sung is highlighted, words light up as
they are reached, and a playhead sweeps each line against its notes. Click any
line or word to play from there.
- **An NVIDIA GPU with CUDA.** songtrace is developed on a 12 GB card.
Separating the vocals (Demucs) alone takes about 3.7 GB, and alignment holds
a whole stretch of singing in memory at once, so a long song whose vocals
never pause needs the most. Cards under 12 GB are untested. Each stage gives
the GPU back when it finishes (see [Sharing the GPU](#sharing-the-gpu)).
`device = "cpu"` in the config runs without a GPU, but it is untested, far
slower, and needs plenty of system RAM.
- **Docker** with the
[NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/),
so the container can see the GPU.
- **A Whisper server of your own.** songtrace does not include one. Either an
OpenAI-compatible API or
[whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)
works. If it shares songtrace's GPU, it needs its own share of the memory.
- **Disk space:** a few GB of model weights, downloaded on first use, plus a
working folder per song holding its source video and separated tracks,
typically several hundred MB.
Playback is modelled on the game: the `#MP3` is the audio and the clock, and
the `#VIDEO` is muted background slaved to it. It also uses the same seconds
the notes were generated from, so a word that lights up late in the browser is
late in the game too.
The packaged video has its audio track stripped for the same reason. It is the
same recording as the `#MP3`, so a player that does not mute the video plays
both at once, a few frames apart, which is heard as an echo. Songs built
before this change still have it — check with `ffprobe <song>.mp4` for an
audio stream, and rebuild with **Restart from** set to `package` to clear it
without re-downloading.
## Configuration
Config is TOML. Resolution order:
1. `--config <path>` passed explicitly on the command line
2. the `SONGTRACE_CONFIG` environment variable
3. `~/.config/songtrace/config.toml`
Copy the example and fill in your Whisper server:
## Quick start (Docker)
```sh
cp config.example.toml config.toml
$EDITOR config.toml # set whisper_url at minimum
```
See `config.example.toml` for every available field (`whisper_url`,
`whisper_api_key`, `whisper_model`, `whisper_api`, `output_dir`, `work_dir`,
`creator`, `demucs_model`, `crepe_model`, `device`, `bind_host`, `port`,
`ytdlp_clients`, `ytdlp_cookies`, `ytdlp_impersonate`,
`ytdlp_pot_provider_url`, `ytdlp_version`).
`whisper_api` selects the wire format: `"openai"` (default) speaks the
OpenAI-compatible `POST {whisper_url}/audio/transcriptions` API; `"asr-webservice"`
speaks [onerahmet/openai-whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)'s
`POST {whisper_url}/asr` API instead (point `whisper_url` at that server's base
URL with no `/v1` suffix — `whisper_model`/`whisper_api_key` are unused for
this backend, since the model is fixed by how that server was started).
## Docker
```sh
cp config.example.toml config.toml
$EDITOR config.toml
$EDITOR config.toml # set whisper_url to your Whisper server
docker compose up -d
```
The web UI is then at `http://localhost:8000`. Downloaded model weights
(Demucs, wav2vec2, ...) are cached under `./data` on the host so they survive
container recreation.
Open `http://localhost:8000`, fill in the form and press **CREATE SONG**.
Run the CLI inside the container instead of the web UI:
Finished songs go to `output_dir` (`/data/output` in the container). To write
them straight into your game's song collection, mount that folder there in
`docker-compose.yml`:
```sh
docker compose run --rm songtrace songtrace make \
--title "Song Title" --artist "Artist Name" \
--lyrics /data/lyrics.txt --youtube https://youtu.be/...
```yaml
volumes:
- ./config.toml:/config/config.toml:ro
- ./data:/data
- /path/to/your/UltraStar/songs:/data/output
```
GPU passthrough requires the [NVIDIA Container
Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/)
installed on the host; `docker-compose.yml` requests one GPU via the
`nvidia` device driver.
Everything else lives under `./data` on the host (model weights, and each
song's working files), so it survives recreating the container.
## Reviewing and editing a song
Pick a song in the queue. The review panel plays the packaged song with its
video, highlights the line and words being sung, and shows the chart on a note
sheet above the lyrics. Playback works like the game: the `#MP3` is the clock
and the video follows it muted, so a word that lights up late here is late in
the game too.
The note sheet shows one line at a time on a pitch-by-beat grid, with the
pitch songtrace detected drawn over the notes. Where a note is right the curve
runs through it, so a wrong one stands out. Fix it there and **SAVE LINE**; the
song's chart is rewritten on the spot.
- **Notes:** drag a note to move it, drag its ends to resize it. Everything
snaps to the beat grid and to semitones. Double-click empty space to add a
note, or a note to change its syllable. With the sheet focused, the arrow
keys move notes, S splits, M merges, G/F/R make a note golden, freestyle or
rap, Ctrl+Z undoes and Ctrl+S saves. Press **?** for every shortcut.
- **Line breaks:** the ↵ markers either side of a line. Drag one to move it,
and double-click it to hand it back to songtrace.
- **Tempo:** the **GRID** bar sets the chart's BPM and GAP. It redraws as you
type and charts on **APPLY**. ×2 and ÷2 fix a tempo detected at double or
half speed, and **ALIGN TO NOTE** puts the selected note on a beat.
**RESET** goes back to songtrace's choice.
- **Undoing an edit:** **REVERT** returns a line to the generated notes.
Edits are kept apart from what the pipeline generates and laid back over it on
every rebuild, so they survive all of them, collection-wide ones included. If
you change a line's lyrics, its edit is set aside rather than applied to
different words, and the sheet lists those for you to discard. Songs with edits
are marked ✎ in the queue.
## Rebuilding
Select a song and its inputs come back up in the form; change them and press
**REBUILD SONG**. Only the stages your change affects are rerun: fixing a line
of lyrics realigns without downloading or separating again.
A songtrace update can improve a stage without any of your inputs changing.
**Restart from** on the rebuild form redoes a stage anyway, and
**REBUILD ALL FROM…**, under CREATE SONG, does it for your whole collection:
pick the stage the update improved, untick any songs to leave alone, and each
song is rebuilt from there. Your edits are kept.
Stages in order: `acquire`, `separate`, `transcribe`, `align`, `pitch`,
`notes`, `grid`, `package`. Rebuilding from `notes` onward needs no GPU.
## Configuration
`config.toml` is TOML; `config.example.toml` lists every setting. songtrace
reads, in order: `--config <path>`, the `SONGTRACE_CONFIG` environment variable
(set to `/config/config.toml` in the container), then
`~/.config/songtrace/config.toml`.
`whisper_api` chooses how to talk to Whisper: `"openai"` (the default) uses
`POST {whisper_url}/audio/transcriptions`, and `"asr-webservice"` uses
whisper-asr-webservice's `POST {whisper_url}/asr`. For the latter, give
`whisper_url` without a `/v1` suffix; `whisper_model` and `whisper_api_key`
are ignored, since that server's model is fixed when it starts.
`creator` sets the `#CREATOR` header in every chart; it defaults to
`songtrace`.
## Sharing the GPU
`songtrace serve` is long-lived but only wants the GPU in bursts, so it hands
it back after every stage rather than holding it for the life of the process.
Three stages load a model — `separate` (Demucs), `align` (wav2vec2) and
`pitch` (torchcrepe) — and each one's weights are dropped as soon as that
stage finishes, followed by `torch.cuda.empty_cache()` to return the
allocator's freed blocks to the driver instead of keeping them for reuse. This
matters most between `separate` and `align`: `transcribe` sits between them
waiting on your Whisper server, which can be minutes of a GPU nothing is
using.
Stages that gave memory back say so in the job log:
Three stages load a model: `separate` (Demucs), `align` (wav2vec2) and `pitch`
(torchcrepe). Each releases its model and returns the memory to the driver as
soon as it finishes, rather than holding it for the life of the server. This
matters most while `transcribe` waits on your Whisper server, which can take
minutes. The job log says how much each stage gave back:
```
[separate] released 3704 MiB of GPU memory
```
The cost is reloading each model once per job instead of once per process,
which is seconds against stages that run for minutes. What this does not
release is the CUDA context itself, a few hundred MB that belongs to the
process until it exits; freeing that would mean running each stage in a
subprocess. If you need every last MB back, stop the container between runs.
Reloading the models costs seconds per song. The CUDA context itself, a few
hundred MB, stays with the process; stop the container to free it.
## Bare-metal
## YouTube download failures
YouTube changes its defences every few weeks, so songtrace keeps `yt-dlp`
current by itself. Once a day, and again when a download fails, it checks PyPI
for a newer stable release, installs it under `/data/models/cache`, and tries
it before the copy it shipped with. That copy is never modified and is always
the fallback, so a broken release costs a retry, not the download. The job log
names the `yt-dlp` version behind each attempt. `ytdlp_version` pins a release,
or `"bundled"` turns the updates off.
If downloads still fail with `HTTP Error 403: Forbidden`:
1. **The network is rate-limited or flagged.** Datacenter and VPN ranges are
throttled hardest. Try another network before changing config.
2. **A PO token is required.** If the log says so, run
[bgutil-ytdlp-pot-provider](https://github.com/Brainicism/bgutil-ytdlp-pot-provider)
alongside songtrace, install its plugin into the image, and point
`ytdlp_pot_provider_url` at it.
3. **Other levers:** `ytdlp_clients` tries each listed YouTube player client
in turn. `ytdlp_cookies` takes a Netscape cookie jar for age-gated videos;
don't use one from an account you care about, as YouTube bans automated
access. `ytdlp_impersonate` presents a browser's TLS fingerprint (for
example `"chrome"`).
To check the JavaScript runtime `yt-dlp` needs for YouTube is present, run
`docker compose exec songtrace yt-dlp -v --simulate <url>`; the `JS runtimes:`
line must not say `none`. A failed download keeps its working folder, so
running the job again resumes it.
## Command line
The web UI is the main way to use songtrace, but every song can also be made
from the command line, in the container or on a host with the requirements
above and `ffmpeg`:
```sh
pip install -e .
songtrace make --help
docker compose run --rm songtrace songtrace make \
--title "Song Title" --artist "Artist Name" \
--lyrics /data/lyrics.txt --youtube https://youtu.be/...
```
```
@@ -146,99 +198,24 @@ songtrace make --title TITLE --artist ARTIST --lyrics LYRICS_FILE \
[--bpm BPM] [--force-from STAGE] [--config CONFIG]
```
Run the web UI instead with `songtrace serve [--config CONFIG]`.
`--force-from <stage>` redoes that stage and everything after it, reusing the
cached output of the stages before. On a bare-metal host, install with
`pip install -e .` and run the web UI with `songtrace serve`.
## Re-running a stage
Each stage caches its output under `work_dir`. If a later stage needs
another pass (e.g. you want to retry alignment with different lyrics, or
pitch tracking looks off), re-run with `--force-from <stage>` to redo that
stage and everything after it, reusing the cached output of everything
before it:
```sh
songtrace make --title "..." --artist "..." --lyrics lyrics.txt \
--youtube https://youtu.be/... --force-from align
```
Valid stages, in pipeline order: `acquire`, `separate`, `transcribe`,
`align`, `pitch`, `notes`, `grid`, `package`.
In the web UI this is what selecting a queue entry does: its title, artist,
lyrics and the rest come back up in the form, so you can adjust them and hit
REBUILD SONG. Only the stages your edit actually invalidates are rerun —
fixing a line of lyrics realigns without re-downloading or re-separating.
CREATE SONG, at the top of the queue, clears the page to start a fresh one.
That planning works off the fields you changed, which cannot see a change in
songtrace itself: after an upgrade, a rebuild of an older song finds every
stage cached and does nothing. **Restart from** on the rebuild form is the way
to redo a stage anyway — pick `package` to repackage without re-downloading.
It can only widen what is rerun, so an edit that invalidates an earlier stage
still wins.
**REBUILD ALL FROM…**, under CREATE SONG, does the same for every song at once:
pick the stage an update improved and each song is rebuilt from there with the
inputs it has now. Every song starts ticked; untick any already rebuilt by hand
(each shows when it was last built). Songs still in progress are skipped, as
are songs whose uploaded video is gone when the stage is `acquire`.
## YouTube download failures
`HTTP Error 403: Forbidden` from the acquire stage means YouTube served the
page but refused the media URL. In rough order of likelihood:
1. **yt-dlp is stale.** YouTube changes its defences every few weeks, so
songtrace keeps yt-dlp current itself. Once a day, and again whenever a
download fails outright, it checks PyPI for a newer stable release,
installs it under `$XDG_CACHE_HOME/songtrace/` (`/data/models/cache` in the
container), and tries it before the copy installed with songtrace. That
copy is never modified and is always the fallback, so a release that breaks
compatibility costs a retry rather than the download. The job log names
the yt-dlp version behind each attempt.
`ytdlp_version` overrides this: `"bundled"` uses only the installed copy,
and a release number such as `"2026.08.19"` pins that release instead of
the newest.
2. **Missing JS runtime.** yt-dlp needs `yt-dlp-ejs` plus a JavaScript engine
to solve YouTube's player challenges; without them it silently falls back to
formats that 403. The `[default,curl-cffi,deno]` extras in `pyproject.toml`
cover this, for both the installed copy and the ones fetched at runtime.
Check with `docker compose exec songtrace yt-dlp -v --simulate <url>` — the debug
header must show a `JS runtimes:` line that is not `none`.
3. **The IP is rate-limited or flagged.** Datacenter and VPN ranges get
throttled hardest. Try from another network before changing config.
4. **A PO token is required.** If the log says so explicitly, run
[bgutil-ytdlp-pot-provider](https://github.com/Brainicism/bgutil-ytdlp-pot-provider)
as a sidecar, install its plugin into the image, and point
`ytdlp_pot_provider_url` at it.
`ytdlp_clients` retries the download against each listed player client in
turn, so a client that YouTube has broken is skipped automatically.
`ytdlp_cookies` (a Netscape cookie jar, for age-gated videos) and
`ytdlp_impersonate` (a browser TLS fingerprint, e.g. `"chrome"`) are the other
two levers. Do not use a cookie jar from an account you care about — YouTube
bans for automated access.
Failed downloads leave the work dir intact, so re-running the job resumes
rather than starting over.
## Regenerating the web UI screenshot
`scripts/screenshot_webui.py` seeds the real app with a fixed set of jobs
covering every status and screenshots it to `docs/images/webui.png`. It uses
`ffmpeg` to give the selected song a playable folder so the shot includes the
player; without `ffmpeg` on `PATH` it still runs, just without it.
## Development
```sh
pip install -e '.[dev]'
playwright install chromium
python scripts/screenshot_webui.py
pytest
```
`pytest` also runs the web UI's JavaScript tests in `tests/js` when Node is
installed. Pull requests run the full suite in CI.
`scripts/screenshot_webui.py` regenerates `docs/images/webui.png` from a
seeded copy of the app (`playwright install chromium` first). With `ffmpeg` on
`PATH`, the shot includes the player.
## License
[AGPLv3](LICENSE)
+3 -3
View File
@@ -1,11 +1,11 @@
whisper_url = "http://whisper.intranet:9000/v1"
whisper_url = "http://whisper.example:9000/v1"
# whisper_api_key = ""
# whisper_model = "whisper-1"
# whisper_api = "openai" # or "asr-webservice" for onerahmet/openai-whisper-asr-webservice
# (set whisper_url without the /v1 suffix, e.g. "http://whisper.intranet:9000")
# (set whisper_url without the /v1 suffix, e.g. "http://whisper.example:9000")
output_dir = "/data/output"
work_dir = "/data/work"
creator = "brad"
# creator = "Your Name" # the #CREATOR header in every chart
# demucs_model = "htdemucs_ft"
# crepe_model = "full"
# device = "cuda"
Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

After

Width:  |  Height:  |  Size: 99 KiB

+30 -11
View File
@@ -15,11 +15,14 @@ import time
from pathlib import Path
import httpx
import numpy as np
import uvicorn
from playwright.sync_api import sync_playwright
from songtrace.config import Config
from songtrace.grid import to_ultrastar
from songtrace.pipeline import PipelineError, PipelineResult, song_slug
from songtrace.ultrastar import serialize_ultrastar
from songtrace.web import create_app
REPO_ROOT = Path(__file__).resolve().parent.parent
@@ -128,13 +131,29 @@ def make_runner():
if "review" in extra:
work = cfg.work_dir / song_slug(inp)
work.mkdir(parents=True, exist_ok=True)
# one note per word in the seeded lines, so each sings its word
notes = [
{**n, "duration": n["end"] - n["start"], "text": "~",
"line_index": line["index"]}
for line in extra["review"]["lines"] for n in line["notes"]
{**n, "duration": n["end"] - n["start"],
"text": (" " if k else "") + w["text"], "line_index": line["index"]}
for line in extra["review"]["lines"]
for k, (n, w) in enumerate(zip(line["notes"], line["words"]))
]
sung = [line for line in extra["review"]["lines"] if line["notes"]]
breaks = [(a["notes"][-1]["end"] + b["notes"][0]["start"]) / 2
for a, b in zip(sung, sung[1:])]
(work / "notes_sec.json").write_text(json.dumps({
"notes": notes, "breaks": [], "warnings": extra["warnings"]}))
"notes": notes, "breaks": breaks, "warnings": extra["warnings"]}))
# a chart for the beat grid, and a sung pitch with a little
# vibrato for the curve over the notes
(work / "song.txt").write_text(serialize_ultrastar(to_ultrastar(
notes, breaks, 240.0, [("TITLE", inp.title)])), encoding="utf-8")
times = np.arange(0, 60, 0.01)
f0, conf = np.zeros_like(times), np.zeros_like(times)
for n in notes:
m = (times >= n["start"] + 0.02) & (times < n["start"] + n["duration"] - 0.02)
semis = n["pitch"] + 0.2 * np.sin(2 * np.pi * 5.5 * times[m])
f0[m], conf[m] = 440 * 2 ** ((semis - 9) / 12), 0.9
np.savez(work / "pitch.npz", times=times, f0=f0, conf=conf)
(work / "align_meta.json").write_text(json.dumps({
"dropped_lines": [line["index"] for line in extra["review"]["lines"]
if line["dropped"]],
@@ -230,13 +249,13 @@ def capture(base_url: str, with_player: bool) -> None:
# current word colouring rather than an untouched transport
page.evaluate("document.getElementById('player-media').currentTime = 11.6")
page.wait_for_timeout(400)
# scrolled to the review: the player pins to the top of #main and
# the tracked lines sit under it, which is the panel in use
page.evaluate("""() => {
const main = document.getElementById('main');
main.scrollTo(0, document.getElementById('review').offsetTop - 12);
}""")
page.wait_for_timeout(400)
page.wait_for_selector(".sheet-curve polyline")
# scrolled to the review, the panel in use, with the note sheet whole
page.evaluate("""() => {
const main = document.getElementById('main');
main.scrollTo(0, document.getElementById('review').offsetTop - 12);
}""")
page.wait_for_timeout(400)
OUT_IMAGE.parent.mkdir(parents=True, exist_ok=True)
page.screenshot(path=str(OUT_IMAGE))
browser.close()
+68 -8
View File
@@ -7,6 +7,7 @@ rebuild with a different BPM re-snaps it rather than breaking it.
"""
import difflib
import json
import math
import time
from pathlib import Path
@@ -18,7 +19,9 @@ KINDS = {":", "*", "F", "R", "G"}
def _empty(lyrics: list[list[str]]) -> dict:
return {"lyrics": lyrics, "lines": {}, "stale": []}
# grid: the BPM and GAP set in the editor, or None for the pipeline's own;
# breaks: line breaks moved by hand, keyed by the line each comes before
return {"lyrics": lyrics, "lines": {}, "stale": [], "grid": None, "breaks": {}}
def load(work: Path, lyrics: list[list[str]]) -> dict:
@@ -41,7 +44,7 @@ def load(work: Path, lyrics: list[list[str]]) -> dict:
def save(work: Path, data: dict) -> None:
path = work / EDITS_FILE
if not data["lines"] and not data["stale"]:
if not (data["lines"] or data["stale"] or data.get("grid") or data.get("breaks")):
path.unlink(missing_ok=True)
return
tmp = path.with_suffix(".tmp")
@@ -64,7 +67,12 @@ def _remap(data: dict, lyrics: list[list[str]]) -> dict:
stale.append(edit)
else:
lines[str(to)] = {**edit, "line_index": to}
return {"lyrics": lyrics, "lines": lines, "stale": stale}
# a moved break goes with the line it comes before; with that line gone,
# there is nothing left for it to separate
breaks = {str(moved[int(k)]): t for k, t in data.get("breaks", {}).items()
if int(k) in moved}
return {"lyrics": lyrics, "lines": lines, "stale": stale, "grid": data.get("grid"),
"breaks": breaks}
def validate_notes(notes: list[dict]) -> list[dict]:
@@ -97,6 +105,49 @@ def clear_line(data: dict, index: int) -> bool:
return data["lines"].pop(str(index), None) is not None
def set_breaks(data: dict, changes: dict) -> None:
"""Move line breaks, each keyed by the line it comes before, to a time in
seconds; None puts one back to where the notes alone would place it."""
breaks = dict(data.get("breaks", {}))
for key, t in changes.items():
try:
line = int(key)
except (TypeError, ValueError):
raise ValueError(f"a break is keyed by the line it comes before, not {key!r}")
if not 0 < line < len(data["lyrics"]):
raise ValueError(f"no line {line} for a break to come before")
if t is None:
breaks.pop(str(line), None)
continue
try:
t = float(t)
except (TypeError, ValueError):
raise ValueError(f"a break needs a time in seconds, not {t!r}")
if not (math.isfinite(t) and t >= 0):
raise ValueError(f"a break needs a time of 0 or more, not {t}")
breaks[str(line)] = t
data["breaks"] = breaks
def set_grid(data: dict, bpm, gap_ms) -> None:
"""Set the chart's BPM (UltraStar's, a quarter of a beat per beat) and GAP."""
try:
bpm, gap_ms = float(bpm), float(gap_ms)
except (TypeError, ValueError):
raise ValueError("bpm and gap_ms must be numbers")
if not (math.isfinite(bpm) and 0 < bpm <= 2000):
raise ValueError(f"bpm must be above 0 and at most 2000, not {bpm}")
if not (math.isfinite(gap_ms) and gap_ms >= 0):
raise ValueError(f"gap_ms must be 0 or more, not {gap_ms}")
data["grid"] = {"bpm": bpm, "gap_ms": gap_ms}
def clear_grid(data: dict) -> bool:
had = bool(data.get("grid"))
data["grid"] = None
return had
def merge(notes: list[dict], breaks: list[float], data: dict
) -> tuple[list[dict], list[float]]:
"""Generated notes and breaks with every edited line laid over them.
@@ -104,10 +155,12 @@ def merge(notes: list[dict], breaks: list[float], data: dict
An edited line replaces the generated notes of that line, and the user's
timing wins: generated notes of earlier lines are cut back to end where it
starts, and those of later lines to start where it ends. Line breaks next
to an edited line move to the middle of the gap it leaves.
to an edited line move to the middle of the gap it leaves; a break moved
by hand stays where it was put, kept between the lines it separates.
"""
edited = {int(k): e["notes"] for k, e in data["lines"].items()}
if not edited:
moved = {int(k): t for k, t in data.get("breaks", {}).items()}
if not edited and not moved:
return notes, breaks
spans = {li: (ns[0]["start"], ns[-1]["start"] + ns[-1]["duration"])
for li, ns in edited.items() if ns}
@@ -122,7 +175,9 @@ def merge(notes: list[dict], breaks: list[float], data: dict
end = min(end, a)
elif li > eli:
start = max(start, b)
if end - start >= MIN_NOTE_SEC:
if (start, end) == (n["start"], n["start"] + n["duration"]):
kept.append(n)
elif end - start >= MIN_NOTE_SEC:
kept.append({**n, "start": start, "duration": end - start})
for li, ns in edited.items():
kept += [{**n, "line_index": li, "low_confidence": False} for n in ns]
@@ -139,13 +194,17 @@ def merge(notes: list[dict], breaks: list[float], data: dict
order = sorted(first, key=lambda li: first[li]["start"])
out_breaks = []
for prev, nxt in zip(order, order[1:]):
end = last[prev]["start"] + last[prev]["duration"]
if nxt in moved:
# where it was put, but never inside either line's notes
out_breaks.append(min(max(moved[nxt], end), max(first[nxt]["start"], end)))
continue
if prev not in edited and nxt not in edited:
# the generated break between these two lines, untouched
gen = [t for t in breaks if line_after(t) == nxt]
if gen:
out_breaks.append(gen[0])
continue
end = last[prev]["start"] + last[prev]["duration"]
out_breaks.append((end + first[nxt]["start"]) / 2)
return kept, out_breaks
@@ -168,6 +227,7 @@ def finalize(work: Path, lyrics: list[list[str]]) -> dict:
" the lyrics; set aside in the editor")
final = {"notes": notes, "breaks": breaks, "warnings": warnings,
"edited_lines": sorted(int(k) for k in data["lines"]),
"stale": data["stale"]}
"stale": data["stale"], "grid": data.get("grid"),
"moved_breaks": sorted(int(k) for k in data.get("breaks", {}))}
(work / FINAL_FILE).write_text(json.dumps(final))
return final
+24 -9
View File
@@ -7,6 +7,9 @@ from .ultrastar import (LineBreak, Note, UltraStarSong, format_bpm, get_header,
parse_bpm, parse_ultrastar, serialize_ultrastar)
TEMPO_FILE = "tempo.json"
def detect_bpm(instrumental_path: Path) -> float:
import librosa
y, sr = librosa.load(str(instrumental_path), sr=22050, mono=True)
@@ -26,11 +29,20 @@ def scale_bpm(bpm: float) -> float:
def to_ultrastar(notes: list[dict], breaks: list[float], file_bpm: float,
headers: list[tuple[str, str]]) -> UltraStarSong:
headers: list[tuple[str, str]],
gap_ms: float | None = None) -> UltraStarSong:
"""Snap notes to the beat grid. GAP defaults to the first note's start; a
GAP set later than that would put notes on negative beats, which players
reject, so it moves back by whole beats, keeping the grid where it was."""
if not notes:
raise ValueError("no notes to place on the beat grid")
gap_ms = round(notes[0]["start"] * 1000)
spb = 60 / (4 * file_bpm)
first_ms = notes[0]["start"] * 1000
if gap_ms is None:
gap_ms = first_ms
elif gap_ms > first_ms:
gap_ms -= math.ceil((gap_ms - first_ms) / (spb * 1000)) * spb * 1000
gap_ms = max(0, round(gap_ms))
def beat(t: float) -> int:
return round((t - gap_ms / 1000) / spb)
@@ -59,15 +71,18 @@ def to_ultrastar(notes: list[dict], breaks: list[float], file_bpm: float,
return UltraStarSong(headers=out_headers, body=body)
def rechart(txt_path: Path, notes: list[dict], breaks: list[float]) -> None:
"""Replace a chart's notes, keeping its headers and its BPM.
def rechart(txt_path: Path, notes: list[dict], breaks: list[float],
bpm: float | None = None, gap_ms: float | None = None) -> None:
"""Replace a chart's notes, keeping its headers and, unless given new
ones, its BPM.
For an edit saved from the editor: the tempo was settled when the song
was built, and the packaged chart's headers may differ from the work
copy's (a re-encoded #VIDEO, a dropped #COVER), so each keeps its own.
For an edit saved from the editor: the packaged chart's headers may differ
from the work copy's (a re-encoded #VIDEO, a dropped #COVER), so each
keeps its own.
"""
song = parse_ultrastar(txt_path.read_text(encoding="utf-8"))
bpm = parse_bpm(get_header(song, "BPM"))
if bpm is None:
bpm = parse_bpm(get_header(song, "BPM"))
headers = [(k, v) for k, v in song.headers if k.upper() not in ("BPM", "GAP")]
txt_path.write_text(serialize_ultrastar(to_ultrastar(notes, breaks, bpm, headers)),
txt_path.write_text(serialize_ultrastar(to_ultrastar(notes, breaks, bpm, headers, gap_ms)),
encoding="utf-8")
+7 -1
View File
@@ -19,6 +19,12 @@ def _run(ctx: Ctx) -> None:
bpm = grid.scale_bpm(grid.detect_bpm(ctx.work / "instrumental.wav"))
except ValueError:
raise PipelineError("could not detect a usable tempo - is the audio silent?")
# the pipeline's own tempo, which RESET in the editor returns to
(ctx.work / grid.TEMPO_FILE).write_text(json.dumps({"bpm": bpm}))
gap_ms = None
if data["grid"]:
bpm, gap_ms = data["grid"]["bpm"], data["grid"]["gap_ms"]
ctx.log("[grid] using the BPM and GAP set in the editor")
slug = song_slug(ctx.inp)
video_ext = Path(meta["video_file"]).suffix
headers = [("TITLE", ctx.inp.title), ("ARTIST", ctx.inp.artist)]
@@ -31,7 +37,7 @@ def _run(ctx: Ctx) -> None:
headers += [("CREATOR", ctx.cfg.creator), ("MP3", f"{slug}.mp3"),
("COVER", f"{slug}.jpg"), ("VIDEO", f"{slug}{video_ext}")]
try:
song = grid.to_ultrastar(data["notes"], data["breaks"], bpm, headers)
song = grid.to_ultrastar(data["notes"], data["breaks"], bpm, headers, gap_ms)
except ValueError:
raise PipelineError("no notes were generated - the lyrics may not match the audio")
(ctx.work / "song.txt").write_text(serialize_ultrastar(song), encoding="utf-8")
+158
View File
@@ -0,0 +1,158 @@
// Editing one line's notes. Every operation returns a new array and leaves
// its input alone, so undo is just keeping the old one. Times are seconds,
// snapped to the chart's beat grid as the chart itself will be. Notes carry
// a client-side `id` so a selection survives reordering; it is never saved.
const MIN_BEATS = 1;
const spbOf = (grid) => 60 / (4 * grid.bpm);
const gapOf = (grid) => grid.gap_ms / 1000;
const byStart = (a, b) => a.start - b.start;
let nextId = 1;
export const newId = () => nextId++;
export function beatOf(t, grid) {
return (t - gapOf(grid)) / spbOf(grid);
}
export function timeOf(beat, grid) {
return gapOf(grid) + beat * spbOf(grid);
}
export function snap(t, grid) {
return timeOf(Math.round(beatOf(t, grid)), grid);
}
// A note as the chart will have it: start on a beat, at least one beat long.
function onGrid(n, grid) {
const start = snap(n.start, grid);
const beats = Math.max(MIN_BEATS, Math.round((n.end - n.start) / spbOf(grid)));
return { ...n, start, end: start + beats * spbOf(grid) };
}
export function withIds(notes) {
return notes.map(n => ({ ...n, id: n.id ?? newId() })).sort(byStart);
}
export function move(notes, ids, beats, semitones, grid) {
const spb = spbOf(grid);
return notes.map(n => {
if (!ids.has(n.id)) return n;
const g = onGrid(n, grid);
const shift = Math.max(beats * spb, -g.start); // never before the song starts
return { ...g, start: g.start + shift, end: g.end + shift, pitch: n.pitch + semitones };
}).sort(byStart);
}
// Drag one edge of a note to time t; the other edge stays put.
export function resize(notes, id, edge, t, grid) {
const spb = spbOf(grid);
return notes.map(n => {
if (n.id !== id) return n;
const g = onGrid(n, grid);
if (edge === "end") return { ...g, end: Math.max(snap(t, grid), g.start + spb) };
return { ...g, start: Math.max(0, Math.min(snap(t, grid), g.end - spb)) };
}).sort(byStart);
}
// Longer or shorter by whole beats, keeping the start.
export function stretch(notes, ids, beats, grid) {
const spb = spbOf(grid);
return notes.map(n => {
if (!ids.has(n.id)) return n;
const g = onGrid(n, grid);
return { ...g, end: Math.max(g.end + beats * spb, g.start + spb) };
});
}
// In two at the beat nearest the middle; the second half holds the syllable.
export function split(notes, id, grid) {
const i = notes.findIndex(n => n.id === id);
if (i < 0) return { notes, ids: [] };
const g = onGrid(notes[i], grid);
const beats = Math.round((g.end - g.start) / spbOf(grid));
if (beats < 2) return { notes, ids: [] };
const mid = g.start + Math.floor(beats / 2) * spbOf(grid);
const a = { ...g, end: mid };
const b = { ...g, id: newId(), start: mid, text: "~", low_confidence: false };
return { notes: [...notes.slice(0, i), a, b, ...notes.slice(i + 1)], ids: [a.id, b.id] };
}
// Into one with the note after it; a held "~" adds no text.
export function merge(notes, id) {
const i = notes.findIndex(n => n.id === id);
if (i < 0 || i + 1 >= notes.length) return notes;
const a = notes[i];
const b = notes[i + 1];
const text = b.text.trim() === "~" ? a.text : a.text + b.text;
return [...notes.slice(0, i), { ...a, end: b.end, text }, ...notes.slice(i + 2)];
}
export function remove(notes, ids) {
return notes.filter(n => !ids.has(n.id));
}
export function insert(notes, t, pitch, grid) {
const start = Math.max(0, snap(t, grid));
const note = { id: newId(), start, end: start + 2 * spbOf(grid), pitch,
text: "~", kind: ":", low_confidence: false };
return { notes: [...notes, note].sort(byStart), id: note.id };
}
// Applying a kind every selected note already has turns it back to normal.
export function setKind(notes, ids, kind) {
const chosen = notes.filter(n => ids.has(n.id));
const to = chosen.length && chosen.every(n => n.kind === kind) ? ":" : kind;
return notes.map(n => (ids.has(n.id) ? { ...n, kind: to } : n));
}
// A syllable that starts a word keeps the space before it that marks it so.
export function setText(notes, id, text) {
const clean = text.trim();
if (!clean) return notes;
return notes.map(n => {
if (n.id !== id) return n;
const lead = n.text.startsWith(" ") && !text.startsWith(" ") ? " " : "";
return { ...n, text: lead + (text.startsWith(" ") ? " " + clean : clean) };
});
}
// What the server stores for a line.
export function toSaved(notes) {
return notes.map(n => ({ start: n.start, duration: n.end - n.start, pitch: n.pitch,
text: n.text, kind: n.kind ?? ":" }));
}
export function history(initial) {
const past = [];
const future = [];
let present = initial;
return {
get present() { return present; },
get canUndo() { return past.length > 0; },
get canRedo() { return future.length > 0; },
push(next) {
if (next === present) return;
past.push(present);
future.length = 0;
present = next;
},
undo() {
if (past.length) { future.push(present); present = past.pop(); }
return present;
},
redo() {
if (future.length) { past.push(present); present = future.pop(); }
return present;
},
};
}
// Where a dragged line break lands: on a beat, but kept between the end of
// the line before it and the start of the line it introduces.
export function placeBreak(t, lo, hi, grid) {
const snapped = snap(t, grid);
if (hi < lo) return lo;
return Math.min(Math.max(snapped, lo), hi);
}
+253 -4
View File
@@ -392,6 +392,166 @@ button { cursor: pointer; }
/* the native transport renders light otherwise, against everything around it */
#player-media { display: block; width: 100%; margin-top: 8px; color-scheme: dark; }
#sheet { margin: 14px 0; padding: 12px 14px 10px; background: var(--panel); border: 1px solid var(--border); }
#sheet[hidden] { display: none; }
.sheet-nav { display: flex; align-items: center; gap: 12px; margin-bottom: 12px; }
.sheet-title {
min-width: 9em;
text-align: center;
font-family: var(--font-display);
font-weight: 700;
font-size: 0.72rem;
letter-spacing: 0.18em;
color: var(--text-muted);
}
.sheet-nav .action:disabled { opacity: 0.35; cursor: default; }
.sheet-badges { display: flex; gap: 8px; margin-left: auto; }
.sheet-badge {
padding: 2px 8px;
font-family: var(--font-display);
font-weight: 600;
font-size: 0.66rem;
letter-spacing: 0.1em;
text-transform: uppercase;
border: 1px solid currentColor;
border-radius: 2px;
}
.sheet-badge-edited { color: var(--aqua); }
.sheet-badge-low { color: var(--fail); }
.sheet-grid {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: 8px 10px;
margin-bottom: 12px;
padding: 8px 10px;
border: 1px solid var(--border);
}
.sheet-grid[hidden] { display: none; }
.sheet-grid-head {
font-family: var(--font-display);
font-weight: 700;
font-size: 0.66rem;
letter-spacing: 0.16em;
color: var(--aqua);
}
.sheet-grid-field {
display: flex;
align-items: center;
gap: 6px;
margin-left: 6px;
font-family: var(--font-display);
font-weight: 600;
font-size: 0.66rem;
letter-spacing: 0.1em;
color: var(--text-muted);
}
.sheet-grid-field input { width: 6.5em; padding: 3px 2px; font: 0.85rem var(--font-mono); }
.sheet-grid .action:disabled { opacity: 0.35; cursor: default; }
.sheet-grid .sheet-save:not(:disabled) { color: var(--bg); background: var(--purple); border-color: var(--purple); }
.sheet-grid-source { flex: 1; min-width: 7em; font-size: 0.76rem; text-align: right; color: var(--text-muted); }
.sheet-grid-pending { color: var(--warn); }
.sheet-tools { display: flex; flex-wrap: wrap; align-items: center; gap: 8px 14px; margin-bottom: 12px; }
.sheet-tool-group { display: flex; gap: 6px; }
.sheet-tools .action:disabled { opacity: 0.35; cursor: default; }
.sheet-tools .sheet-save:not(:disabled) { color: var(--bg); background: var(--purple); border-color: var(--purple); }
.sheet-status { flex: 1; min-width: 6em; font-size: 0.78rem; color: var(--text-muted); text-align: right; }
.sheet-status-error { color: var(--fail); }
.sheet-badge-dirty { color: var(--warn); }
.sheet-plot { position: relative; cursor: crosshair; touch-action: none; }
.sheet-plot:focus-visible { outline: 2px solid var(--aqua); outline-offset: 2px; }
.sheet-note-selected { stroke: var(--text); stroke-width: 2; stroke-dasharray: none; }
.sheet-syllable-selected { color: var(--aqua); }
.sheet-text-input {
position: absolute;
top: -2px;
width: 7em;
padding: 1px 4px;
font-size: 0.8rem;
background: var(--panel-raised);
border: 1px solid var(--aqua);
}
.sheet-stale { margin-bottom: 12px; padding: 8px 10px; font-size: 0.82rem; border: 1px solid var(--warn); }
.sheet-stale[hidden] { display: none; }
.sheet-stale-head { color: var(--warn); margin-bottom: 6px; }
.sheet-stale-row { display: flex; align-items: center; gap: 10px; margin-top: 4px; }
.sheet-stale-text { flex: 1; color: var(--text-muted); }
.sheet-nav > .action[aria-expanded] { margin-left: 4px; }
.sheet-keys {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(320px, 1fr));
gap: 14px 22px;
margin-bottom: 14px;
padding: 12px 14px;
background: var(--panel-raised);
border: 1px solid var(--border);
}
.sheet-keys[hidden] { display: none; }
.sheet-keys-head {
margin-bottom: 6px;
font-family: var(--font-display);
font-weight: 700;
font-size: 0.66rem;
letter-spacing: 0.16em;
text-transform: uppercase;
color: var(--aqua);
}
.sheet-keys dl { display: grid; grid-template-columns: auto 1fr; gap: 4px 10px; margin: 0; font-size: 0.78rem; }
.sheet-keys dt { white-space: nowrap; color: var(--text-muted); }
.sheet-keys dd { margin: 0; color: var(--text); }
.sheet-keys kbd {
display: inline-block;
min-width: 1.4em;
padding: 0 5px;
font: 0.72rem var(--font-mono);
text-align: center;
color: var(--text);
background: var(--panel);
border: 1px solid var(--control);
border-bottom-width: 2px;
border-radius: 3px;
}
.sheet-help[hidden] { display: none; }
.sheet-help { margin-top: 10px; font-size: 0.74rem; line-height: 1.5; color: var(--text-muted); }
.sheet-svg { display: block; }
.sheet-row { fill: var(--panel-raised); }
.sheet-row-black { fill: var(--bg); }
.sheet-octave { stroke: var(--control); stroke-width: 1; }
.sheet-key { fill: var(--text-muted); font: 10px var(--font-mono); text-anchor: end; dominant-baseline: middle; }
.sheet-key-c { fill: var(--text); font-weight: 600; }
.sheet-beat { stroke: var(--border); stroke-width: 1; opacity: 0.35; }
.sheet-beat-1 { stroke: var(--control); opacity: 0.55; }
.sheet-beat-2 { stroke: var(--text-muted); opacity: 0.8; }
.sheet-breaks { pointer-events: none; }
.sheet-break { stroke: var(--text-muted); stroke-width: 1.5; stroke-dasharray: 5 4; }
.sheet-break-moved { stroke: var(--aqua); stroke-dasharray: none; }
.sheet-break-label { fill: var(--text-muted); stroke: none; font: 12px var(--font-mono); }
.sheet-break-label.sheet-break-moved { fill: var(--aqua); stroke: none; }
.sheet-curve { pointer-events: none; }
.sheet-curve polyline { fill: none; stroke: var(--aqua); stroke-width: 1.5; opacity: 0.9; }
.sheet-note { fill: var(--purple); }
.sheet-note-golden { fill: #E8B931; }
.sheet-note-free { fill: none; stroke: var(--purple); stroke-width: 1.5; stroke-dasharray: 4 3; }
.sheet-note-rap { fill: var(--aqua); }
.sheet-note-golden-rap { fill: #E8B931; stroke: var(--aqua); stroke-width: 2; }
.sheet-note-low { stroke: var(--fail); stroke-width: 2; stroke-dasharray: 3 2; }
.sheet-note-context { opacity: 0.28; }
.sheet-playhead { position: absolute; top: 0; height: 300px; width: 2px; margin-left: -1px; background: var(--aqua); pointer-events: none; }
.sheet-playhead[hidden] { display: none; }
.sheet-syllables, .sheet-words { position: relative; overflow: hidden; }
.sheet-syllables { height: 22px; margin-top: 6px; overflow: visible; }
.sheet-syllable { position: absolute; top: 0; font-size: 0.8rem; white-space: pre; color: var(--text); cursor: text; }
.sheet-words { height: 24px; margin-top: 4px; }
.sheet-word {
position: absolute;
top: 0;
padding-top: 4px;
font-size: 0.78rem;
white-space: nowrap;
color: var(--text-muted);
border-top: 2px solid var(--border);
}
#lyrics-review { display: flex; flex-direction: column; gap: 10px; }
.review-line { padding: 8px 12px; background: var(--panel); border: 1px solid var(--border); }
.review-line-active { background: var(--panel-raised); border-color: var(--aqua); }
@@ -463,6 +623,11 @@ button { cursor: pointer; }
}
.lineage:hover { border-bottom-color: var(--aqua); }
.job-mark { grid-area: 1 / 2; justify-self: end; font-size: 0.8rem; color: var(--text-muted); }
.job-edits { grid-area: 2 / 2; justify-self: end; display: flex; gap: 6px; font-size: 0.72rem; }
.job-edits-kept { color: var(--aqua); }
.job-edits-stale { color: var(--warn); }
.rebuild-all-kept { color: var(--aqua); }
.rebuild-all-aside { color: var(--warn); }
#detail {
margin: 13px 0 0;
@@ -584,7 +749,8 @@ button { cursor: pointer; }
<form id="rebuild-all-form" method="dialog">
<h2 id="rebuild-all-heading">REBUILD ALL SONGS</h2>
<p>Reruns the songs ticked below from the chosen stage onwards with the
inputs they have now, for when a songtrace update improves that stage.</p>
inputs they have now, for when a songtrace update improves that stage.
Lines edited by hand are kept.</p>
<label class="field">
<span>Restart from</span>
<select name="force_from" required></select>
@@ -658,6 +824,7 @@ button { cursor: pointer; }
<video id="player-video" muted playsinline preload="metadata"></video>
<audio id="player-media" controls preload="metadata"></audio>
</div>
<div id="sheet" hidden></div>
<div id="lyrics-review"><p class="empty-note">no songs queued yet — nothing to review</p></div>
<div id="output-path"></div>
<div id="review-actions"></div>
@@ -775,6 +942,7 @@ form.addEventListener("input", () => { formDirty = true; });
// scrolled the panel to. Leaving another song's review under an empty form
// reads as though the two belong together.
createSong.addEventListener("click", () => {
if (sheet?.isDirty() && !confirm("Discard unsaved changes to this line?")) return;
if (formDirty && !confirm("Discard your unsaved edits to this form?")) return;
composing = true;
selectedId = null;
@@ -829,6 +997,18 @@ function rebuildAllRow(song) {
undefined, { dateStyle: "medium", timeStyle: "short" });
meta.textContent = busy ? "in progress, skipped"
: `${song.status === "failed" ? "failed" : song.rebuilt_from ? "rebuilt" : "built"} ${when || ""}`.trim();
if (song.edited_lines) {
const kept = document.createElement("span");
kept.className = "rebuild-all-kept";
kept.textContent = ` · ${song.edited_lines} edited line${song.edited_lines === 1 ? "" : "s"} kept`;
meta.append(kept);
}
if (song.stale_edits) {
const aside = document.createElement("span");
aside.className = "rebuild-all-aside";
aside.textContent = ` · ${song.stale_edits} set aside`;
meta.append(aside);
}
row.append(box, name, meta);
return row;
}
@@ -904,7 +1084,10 @@ function jobCard(job) {
markLabel.className = "sr-only";
mark.append(markGlyph, markLabel);
card.append(title, artist, stateLine, time, mark);
const edits = document.createElement("span");
edits.className = "job-edits";
card.append(title, artist, stateLine, time, mark, edits);
card.addEventListener("click", () => selectJob(job.id));
paintCard(card, job);
return card;
@@ -912,7 +1095,7 @@ function jobCard(job) {
function paintCard(card, job) {
const state = STATE[job.status] ?? { glyph: "·", cls: "st-queued" };
const [title, artist, stateLine, time, mark] = card.children;
const [title, artist, stateLine, time, mark, edits] = card.children;
card.className = `job ${state.cls}`;
if (job.id === selectedId) card.setAttribute("aria-current", "true");
else card.removeAttribute("aria-current");
@@ -923,6 +1106,33 @@ function paintCard(card, job) {
time.textContent = elapsed(job);
mark.children[0].textContent = job.rebuilt_from ? "↻" : "";
mark.children[1].textContent = job.rebuilt_from ? " rebuilt" : "";
paintEdits(edits, job);
}
// hand edits a rebuild keeps, and any set aside because the lyrics changed
function paintEdits(node, job) {
const parts = [];
if (job.edited_lines) parts.push(["job-edits-kept", `✎ ${job.edited_lines}`]);
if (job.stale_edits) parts.push(["job-edits-stale", `⚠ ${job.stale_edits}`]);
const label = [
job.edited_lines ? `${job.edited_lines} edited line${job.edited_lines === 1 ? "" : "s"}` : "",
job.stale_edits ? `${job.stale_edits} set aside` : "",
].filter(Boolean).join(", ");
if (node.title === label) return;
node.title = label;
node.replaceChildren(...parts.map(([cls, text]) => {
const span = document.createElement("span");
span.className = cls;
span.setAttribute("aria-hidden", "true");
span.textContent = text;
return span;
}));
if (label) {
const sr = document.createElement("span");
sr.className = "sr-only";
sr.textContent = label;
node.append(sr);
}
}
// patched in place rather than replaced wholesale: detaching a card would blur
@@ -1025,6 +1235,8 @@ async function refreshJobs() {
}
async function selectJob(id) {
if (id !== selectedId && sheet?.isDirty() &&
!confirm("Discard unsaved changes to this line?")) return;
// the form already holds this job; reloading it would drop edits in progress
const reload = id !== rebuildOf;
if (reload && formDirty && !confirm("Discard your unsaved edits to this form?")) return;
@@ -1151,6 +1363,7 @@ function lineEntry(row, line) {
const labels = [...row.querySelectorAll(".review-word")];
return {
row,
index: line.index,
t0: laid[0].start,
t1: Math.max(...laid.map(x => x.end)),
words: line.words.map((w, i) => ({ el: labels[i], start: w.start, end: w.end })),
@@ -1197,6 +1410,32 @@ const media = document.getElementById("player-media");
const video = document.getElementById("player-video");
const DRIFT = 0.25; // seconds before the picture is worth re-seeking
// the note sheet lives in its own module, loaded before any review renders
let sheet = null;
let stopAt = null; // set by playing one line: where playback pauses again
const sheetReady = import("/static/sheet.js").then((m) => {
sheet = m.createSheet(document.getElementById("sheet"), {
seek: (t) => {
media.currentTime = t;
if (media.paused) syncPlayback();
},
playLine: (from, to) => {
if (player.hidden) return;
media.currentTime = from;
stopAt = to;
media.play().catch(() => {});
},
// a saved line changes the notes the lyric list draws
onSaved: (data) => {
const box = document.getElementById("lyrics-review");
const { nodes, entries } = buildLyricsReview(data);
box.replaceChildren(...nodes);
bindTimeline(player.hidden ? [] : entries);
if (!player.hidden) syncPlayback();
},
});
});
function slaveVideo(force) {
if (video.hidden || !video.getAttribute("src")) return;
if (force || Math.abs(video.currentTime - media.currentTime) > DRIFT) {
@@ -1210,7 +1449,10 @@ for (const event of ["seeked", "ratechange"]) {
slaveVideo(true);
});
}
media.addEventListener("pause", () => video.pause());
media.addEventListener("pause", () => {
video.pause();
stopAt = null;
});
let timeline = [];
let activeEntry = null;
let syncFrame = null;
@@ -1251,10 +1493,12 @@ function syncPlayback() {
activeEntry = entry;
if (entry) {
entry.row.classList.add("review-line-active");
sheet?.showLine(entry.index);
focusActive();
}
}
if (entry) paintEntry(entry, t);
sheet?.setTime(t);
}
// centred so there is context either side, like a scrolling lyric sheet; only
@@ -1266,6 +1510,7 @@ function focusActive() {
}
function pumpPlayback() {
if (stopAt !== null && media.currentTime >= stopAt) media.pause();
syncPlayback();
slaveVideo(false);
syncFrame = media.paused ? null : requestAnimationFrame(pumpPlayback);
@@ -1314,6 +1559,7 @@ function setPlayerSource(job, available) {
}
function resetPlayer() {
sheet?.clear();
// the 3s poll reaches here for every running job; reloading a cleared
// element each time would be pure churn
if (player.hidden && !timeline.length) return;
@@ -1361,6 +1607,9 @@ async function renderLyricsReview(job) {
}
const { nodes, entries } = buildLyricsReview(data);
box.replaceChildren(...nodes);
await sheetReady;
if (job.id !== selectedId) return;
sheet.load(job.id, data);
// with nothing to play, a highlighted line and a click-to-seek cursor would
// both be promising something the song folder cannot deliver
const playable = setPlayerSource(job, data.media);
+965
View File
@@ -0,0 +1,965 @@
// The note sheet: one lyric line at a time on a pitch-by-beat grid, the way
// an UltraStar editor lays a song out, with the detected pitch drawn over
// the notes so a wrong one stands out. The line on show can be edited.
import * as edit from "./edit-model.js";
const SVG = "http://www.w3.org/2000/svg";
const NAMES = ["C", "C♯", "D", "D♯", "E", "F", "F♯", "G", "G♯", "A", "A♯", "B"];
const BLACK = new Set([1, 3, 6, 8, 10]);
const GUTTER = 46; // px of note names left of the grid
const PLOT_HEIGHT = 300; // px; rows divide this between the pitches shown
const CONTEXT = 0.75; // seconds shown either side of the line
const BREAK_REACH = 3; // seconds past the line the page grows to show a break
const EDGE = 6; // px from a note's end that grabs it for resizing
const KINDS = { "*": "golden", F: "free", R: "rap", G: "golden-rap" };
const KEYS_OPEN = "songtrace.sheet-keys";
// The sheet's controls, grouped as an editor's reference card lists them.
// "+" joins keys pressed together, " / " separates alternatives, and a
// lowercase word is something done with the mouse rather than a key.
export const SHORTCUTS = [
["Select", [
["click", "select a note"],
["Shift+click", "add to the selection"],
["Tab / Shift+Tab", "next, previous note"],
["Esc", "clear the selection"]]],
["Move", [
["drag", "move in time and pitch"],
["← / →", "a beat earlier, later"],
["↑ / ↓", "a semitone up, down"],
["Shift+↑ / Shift+↓", "an octave up, down"]]],
["Length", [
["drag an end", "resize"],
["Shift+← / Shift+→", "a beat shorter, longer"]]],
["Notes", [
["double-click", "add a note, on empty space"],
["S", "split in two"],
["M", "merge with the next"],
["Delete", "remove"]]],
["Kind", [
["G", "golden"],
["F", "freestyle"],
["R", "rap"],
["G / F / R", "again: back to normal"]]],
["Line break", [
["drag a ↵ marker", "move it, between the lines it separates"],
["double-click a ↵", "back to where songtrace puts it"]]],
["Syllable", [
["double-click", "edit, on a note"],
["Enter", "edit the selected note"]]],
["Line", [
["Ctrl+Z", "undo"],
["Ctrl+Shift+Z / Ctrl+Y", "redo"],
["Ctrl+S", "save"],
["P", "play"],
["Page Up / Page Down", "previous, next line"],
["?", "show or hide these keys"]]],
];
// "Shift+Tab / Esc" as key caps, with mouse actions left as words
export function comboParts(combo) {
return combo.split(" / ").map(alt => alt.split("+").map(part => ({
text: part, key: !/^[a-z]/.test(part),
})));
}
const wrap = (n, d) => ((n % d) + d) % d;
const mod12 = (p) => wrap(p, 12);
// UltraStar pitch 0 is C4 (MIDI 60)
export function noteName(pitch) {
return NAMES[mod12(pitch)] + (Math.floor((pitch + 60) / 12) - 1);
}
export function isBlackKey(pitch) {
return BLACK.has(mod12(pitch));
}
// The pitches to show: the line's notes with room around them, and never
// less than an octave and a bit, so rows stay a consistent size to aim at.
export function pitchRange(pitches, { pad = 3, min = 15 } = {}) {
if (!pitches.length) return { lo: -7, hi: 7 };
let lo = Math.min(...pitches) - pad;
let hi = Math.max(...pitches) + pad;
const short = min - (hi - lo + 1);
if (short > 0) {
lo -= Math.floor(short / 2);
hi += Math.ceil(short / 2);
}
return { lo, hi };
}
// UltraStar beats between t0 and t1 on the chart's grid. A beat is a quarter
// of the #BPM beat; weight 2 marks every 16th, 1 every 4th, as editors do.
export function beatLines(t0, t1, grid) {
const spb = 60 / (4 * grid.bpm);
const gap = grid.gap_ms / 1000;
const out = [];
for (let k = Math.ceil((t0 - gap) / spb); gap + k * spb <= t1; k++) {
const weight = wrap(k, 16) === 0 ? 2 : wrap(k, 4) === 0 ? 1 : 0;
out.push({ t: gap + k * spb, beat: k, weight });
}
return out;
}
// The seconds a line occupies, from its notes and the words it was aligned to.
export function lineSpan(line) {
const ts = [...line.notes.flatMap(n => [n.start, n.end]),
...line.words.flatMap(w => [w.start, w.end])];
return ts.length ? { start: Math.min(...ts), end: Math.max(...ts) } : null;
}
function el(name, attrs = {}, parent = null) {
const node = document.createElementNS(SVG, name);
for (const [k, v] of Object.entries(attrs)) node.setAttribute(k, v);
if (parent) parent.append(node);
return node;
}
function div(cls, text = null) {
const node = document.createElement("div");
node.className = cls;
if (text !== null) node.textContent = text;
return node;
}
export function createSheet(root, { seek, playLine, onSaved }) {
const button = (label, title, onClick, cls = "action") => {
const b = document.createElement("button");
b.type = "button";
b.className = cls;
b.textContent = label;
b.title = title;
b.addEventListener("click", onClick);
return b;
};
const title = div("sheet-title");
const badges = div("sheet-badges");
const prev = button("◀ PREV", "previous line (Page Up)", () => page(-1));
const next = button("NEXT ▶", "next line (Page Down)", () => page(1));
const nav = div("sheet-nav");
const keysButton = button("KEYS ?", "keyboard shortcuts (?)", () => toggleKeys());
keysButton.setAttribute("aria-expanded", "false");
nav.append(prev, title, next, badges, keysButton);
const tools = {
undo: button("UNDO", "Ctrl+Z", () => undo()),
redo: button("REDO", "Ctrl+Shift+Z", () => redo()),
split: button("SPLIT", "split the selected note (S)", () => splitSelected()),
merge: button("MERGE", "merge with the next note (M)", () => mergeSelected()),
remove: button("DELETE", "delete the selected notes (Delete)", () => removeSelected()),
golden: button("GOLDEN", "golden note (G)", () => kind("*")),
free: button("FREESTYLE", "freestyle note (F)", () => kind("F")),
rap: button("RAP", "rap note (R)", () => kind("R")),
play: button("▶ LINE", "play this line (P)", () => playCurrent()),
discard: button("DISCARD", "drop unsaved changes to this line", () => discard()),
revert: button("REVERT", "go back to the generated notes for this line", () => revert()),
save: button("SAVE LINE", "Ctrl+S", () => save(), "action sheet-save"),
};
const toolbar = div("sheet-tools");
const group = (...items) => { const g = div("sheet-tool-group"); g.append(...items); return g; };
const status = div("sheet-status");
status.setAttribute("role", "status");
toolbar.append(group(tools.undo, tools.redo),
group(tools.split, tools.merge, tools.remove),
group(tools.golden, tools.free, tools.rap),
group(tools.play), status,
group(tools.discard, tools.revert, tools.save));
const plot = div("sheet-plot");
plot.tabIndex = 0;
plot.setAttribute("aria-label", "note sheet: select notes and edit them with the mouse or keyboard");
const svg = el("svg", { class: "sheet-svg" });
const playhead = div("sheet-playhead");
playhead.hidden = true;
plot.append(svg, playhead);
const syllables = div("sheet-syllables");
const words = div("sheet-words");
const help = div("sheet-help", "Press ? for keyboard shortcuts.");
const keys = div("sheet-keys");
keys.id = "sheet-keys";
keysButton.setAttribute("aria-controls", keys.id);
for (const [group, entries] of SHORTCUTS) {
const section = div("sheet-keys-group");
const heading = div("sheet-keys-head", group);
const list = document.createElement("dl");
for (const [combo, action] of entries) {
const dt = document.createElement("dt");
comboParts(combo).forEach((alt, i) => {
if (i) dt.append(" / ");
alt.forEach((part, j) => {
if (j) dt.append("+");
const node = document.createElement(part.key ? "kbd" : "span");
node.textContent = part.text;
dt.append(node);
});
});
const dd = document.createElement("dd");
dd.textContent = action;
list.append(dt, dd);
}
section.append(heading, list);
keys.append(section);
}
const stale = div("sheet-stale");
const gridBar = div("sheet-grid");
const field = (label, step, unit) => {
const wrap = document.createElement("label");
wrap.className = "sheet-grid-field";
const input = document.createElement("input");
input.type = "number";
input.step = step;
input.min = "0";
wrap.append(label, input);
if (unit) wrap.append(unit);
return { wrap, input };
};
const bpmField = field("BPM", "0.01");
const gapField = field("GAP", "1", "ms");
const gridSource = div("sheet-grid-source");
const gridTools = {
double: button("×2", "double the BPM", () => tryGrid({ bpm: grid().bpm * 2 })),
half: button("÷2", "halve the BPM", () => tryGrid({ bpm: grid().bpm / 2 })),
earlier: button("−10", "GAP 10ms earlier", () => tryGrid({ gap_ms: grid().gap_ms - 10 })),
later: button("+10", "GAP 10ms later", () => tryGrid({ gap_ms: grid().gap_ms + 10 })),
align: button("ALIGN TO NOTE", "move GAP the least that puts the selected note on a beat",
() => alignToNote()),
cancel: button("CANCEL", "drop the BPM and GAP being tried", () => { gridDraft = null; render(); }),
reset: button("RESET", "back to the BPM songtrace chose, with GAP at the first note",
() => resetGrid()),
apply: button("APPLY", "chart the song on this BPM and GAP", () => applyGrid(),
"action sheet-save"),
};
const gridHead = div("sheet-grid-head", "GRID");
gridBar.append(gridHead, bpmField.wrap, gridTools.double, gridTools.half,
gapField.wrap, gridTools.earlier, gridTools.later, gridTools.align,
gridSource, gridTools.cancel, gridTools.reset, gridTools.apply);
for (const [f, key] of [[bpmField, "bpm"], [gapField, "gap_ms"]]) {
f.input.addEventListener("input", () => {
const v = Number(f.input.value);
if (f.input.value !== "" && Number.isFinite(v) && v >= 0 && (key === "gap_ms" || v > 0)) {
tryGrid({ [key]: v }, f.input);
}
});
}
root.replaceChildren(nav, stale, gridBar, toolbar, keys, plot, syllables, words, help);
// stays as the person left it, across lines, songs and visits
function showKeys(open) {
keys.hidden = !open;
help.hidden = open;
keysButton.setAttribute("aria-expanded", String(open));
try { localStorage.setItem(KEYS_OPEN, open ? "1" : ""); } catch {}
}
function toggleKeys() {
showKeys(keys.hidden);
}
let keysOpen = false;
try { keysOpen = !!localStorage.getItem(KEYS_OPEN); } catch {}
showKeys(keysOpen);
let review = null;
let curve = null;
let index = null;
let jobId = null;
let view = null; // what the last render laid out: window and scale
let layout = null; // window and pitch range, kept steady while editing
let hist = null; // the current line's notes, with undo
let baseline = null; // what the line holds on the server
let preview = null; // notes mid-drag, not yet in the history
let breakPreview = null; // a break mid-drag: { which, t }
let selected = new Set();
let drag = null;
let saving = false;
const visible = () => review.lines.filter(l => !l.dropped && lineSpan(l));
// the line being edited: its notes and the breaks either side of it, as
// one state so undo covers both. A break is its time, undefined where there
// is none to move, or null when it is to go back to automatic.
const notes = () => preview ?? hist?.present.notes ?? [];
const dirty = () => !!hist && hist.present !== baseline;
let gridDraft = null; // BPM and GAP being tried out, not yet applied
// the grid being drawn and snapped to: the one being tried out, if any
const grid = () => gridDraft ?? review?.grid;
// notes snap to the grid, so they wait while a new one is being tried
const canEdit = () => !!review?.grid && !gridDraft && !saving;
function say(text, error = false) {
status.textContent = text;
status.classList.toggle("sheet-status-error", error);
}
// the window and pitch range that hold the line, grown (never shrunk while
// editing) so rows do not jump under the pointer
function fit(reset) {
const line = review.lines[index];
const ts = [...notes().flatMap(n => [n.start, n.end]),
...line.words.flatMap(w => [w.start, w.end])];
const span = ts.length ? { start: Math.min(...ts), end: Math.max(...ts) } : lineSpan(line);
const range = pitchRange(notes().map(n => n.pitch));
const want = { t0: span.start - CONTEXT, t1: span.end + CONTEXT, ...range };
// the breaks either side are part of the line's page, so it grows to
// show them, but not across a long instrumental: one further away is
// pinned to the edge instead (see shownBreakTime)
for (const slot of breakSlots()) {
const t = breakTime(slot);
want.t0 = Math.min(want.t0, Math.max(t - 0.3, span.start - BREAK_REACH));
want.t1 = Math.max(want.t1, Math.min(t + 0.3, span.end + BREAK_REACH));
}
layout = reset || !layout ? want : {
t0: Math.min(layout.t0, want.t0), t1: Math.max(layout.t1, want.t1),
lo: Math.min(layout.lo, want.lo), hi: Math.max(layout.hi, want.hi),
};
}
// edits whose line's lyrics have since changed: kept, not applied, until
// the person who made them decides
function paintStale() {
const items = review?.stale ?? [];
stale.hidden = !items.length;
stale.replaceChildren();
if (!items.length) return;
stale.append(div("sheet-stale-head",
`${items.length} edited line${items.length === 1 ? " was" : "s were"} set aside ` +
"because the lyrics changed:"));
items.forEach((item, n) => {
const row = div("sheet-stale-row");
row.append(div("sheet-stale-text", `"${item.text}"`),
button("DISCARD", "throw this edit away", () => discardStale(n)));
stale.append(row);
});
}
async function discardStale(n) {
const resp = await fetch(`/api/jobs/${jobId}/stale/${n}`, { method: "DELETE" });
if (!resp.ok) { say(`error ${resp.status}`, true); return; }
review = await resp.json();
paintStale();
onSaved(review);
}
function open(i) {
index = i;
const line = review.lines[i];
const lines = visible();
const at = lines.findIndex(l => l.index === i);
const breakBefore = (l) => review.breaks?.find(b => b.line === l?.index)?.time;
baseline = {
notes: edit.withIds(line.notes.map(n => ({ ...n, kind: n.kind ?? ":" }))),
before: at > 0 ? breakBefore(line) : undefined,
after: breakBefore(lines[at + 1]),
};
hist = edit.history(baseline);
breakPreview = null;
preview = null;
selected = new Set();
say("");
fit(true);
paintStale();
render();
}
// asked for by the person using the sheet, so it may cost them their edits
function goTo(i) {
if (!review || i === index) return;
if (dirty() && !confirm("Discard unsaved changes to this line?")) return;
open(i);
}
function page(step) {
const lines = visible();
const at = lines.findIndex(l => l.index === index);
const to = lines[Math.min(Math.max(at + step, 0), lines.length - 1)];
if (to) goTo(to.index);
}
function change(next, keep = selected) {
hist.push({ ...hist.present, notes: next });
selected = new Set([...keep].filter(id => next.some(n => n.id === id)));
fit(false);
render();
}
function changeBreak(which, t) {
if (hist.present[which] === t) return;
hist.push({ ...hist.present, [which]: t });
render();
}
// the breaks either side of the line, each with the gap it may move in:
// from the end of the line before it to the start of the line after
function breakSlots() {
const ns = notes();
if (!hist || !ns.length) return [];
const lines = visible();
const at = lines.findIndex(l => l.index === index);
const prev = lines[at - 1];
const next = lines[at + 1];
const slots = [];
if (prev?.notes.length && hist.present.before !== undefined) {
slots.push({ which: "before", line: index,
lo: Math.max(...prev.notes.map(n => n.end)), hi: ns[0].start });
}
if (next?.notes.length && hist.present.after !== undefined) {
slots.push({ which: "after", line: next.index,
lo: Math.max(...ns.map(n => n.end)), hi: Math.min(...next.notes.map(n => n.start)) });
}
return slots;
}
// where a break is drawn; one going back to automatic is shown mid-gap,
// roughly where it will land
function breakTime(slot) {
const v = breakPreview?.which === slot.which ? breakPreview.t : hist.present[slot.which];
const t = v === null ? (slot.lo + slot.hi) / 2 : v;
return Math.min(Math.max(t, slot.lo), Math.max(slot.hi, slot.lo));
}
// where a break is drawn and grabbed: on the page, pinned to its edge
// when it lies beyond it
function shownBreakTime(slot) {
const edge = (layout.t1 - layout.t0) * 0.003; // a few pixels in
const t = breakTime(slot);
const shown = Math.min(Math.max(t, layout.t0 + edge), layout.t1 - edge);
return { t: shown, pinned: shown !== t ? (t < shown ? "left" : "right") : null };
}
const undo = () => { if (hist?.canUndo) { hist.undo(); fit(false); render(); } };
const redo = () => { if (hist?.canRedo) { hist.redo(); fit(false); render(); } };
const one = () => (selected.size === 1 ? [...selected][0] : null);
function splitSelected() {
if (one() === null || !grid()) return;
const out = edit.split(notes(), one(), grid());
if (out.notes !== notes()) change(out.notes, new Set([out.ids[0]]));
}
function mergeSelected() {
if (one() !== null) change(edit.merge(notes(), one()));
}
function removeSelected() {
if (selected.size) change(edit.remove(notes(), selected), new Set());
}
function kind(k) {
if (selected.size) change(edit.setKind(notes(), selected, k));
}
function playCurrent() {
if (!layout) return;
const ns = notes();
const start = ns.length ? ns[0].start : layout.t0 + CONTEXT;
const end = ns.length ? Math.max(...ns.map(n => n.end)) : layout.t1 - CONTEXT;
playLine(Math.max(0, start - 0.3), end + 0.2);
}
function discard() {
if (!dirty()) return;
open(index);
}
// a change saved to the server, which answers with the refreshed review
async function send(path, method, body, busy, done) {
saving = true;
render();
say(busy);
try {
const resp = await fetch(`/api/jobs/${jobId}/${path}`, {
method, headers: { "Content-Type": "application/json" },
body: body ? JSON.stringify(body) : undefined,
});
if (!resp.ok) {
const err = await resp.json().catch(() => ({}));
say(err.detail || `error ${resp.status}`, true);
return false;
}
const data = await resp.json();
const at = index;
review = data;
gridDraft = null;
open(at);
say(done);
onSaved(data);
return true;
} catch {
say("could not reach songtrace", true);
return false;
} finally {
saving = false;
render();
}
}
// only what changed: a moved break alone must not mark the line edited
async function save() {
if (!dirty() || saving) return;
const state = hist.present;
const lines = visible();
const next = lines[lines.findIndex(l => l.index === index) + 1];
const moved = {};
if (state.before !== baseline.before) moved[index] = state.before;
if (state.after !== baseline.after && next) moved[next.index] = state.after;
if (state.notes !== baseline.notes &&
!await send(`lines/${index}`, "PUT", { notes: edit.toSaved(state.notes) }, "saving…", "saved")) {
return;
}
if (Object.keys(moved).length) await send("breaks", "PUT", moved, "saving…", "saved");
}
function revert() {
if (!review.lines[index].edited || saving) return;
if (!confirm("Go back to the generated notes for this line? Your edit is lost.")) return;
send(`lines/${index}`, "DELETE", null, "reverting…", "reverted to the generated notes");
}
// redraws on the trial grid straight away; nothing is saved until APPLY
function tryGrid(change, typing = null) {
const next = { ...grid(), ...change };
next.gap_ms = Math.max(0, next.gap_ms);
const same = review.grid && next.bpm === review.grid.bpm && next.gap_ms === review.grid.gap_ms;
gridDraft = same ? null : next;
render(typing);
}
function alignToNote() {
const note = notes().find(n => selected.has(n.id));
if (!note) return;
const g = grid();
const spbMs = 60000 / (4 * g.bpm);
const atMs = note.start * 1000;
const offset = (atMs - g.gap_ms) - Math.round((atMs - g.gap_ms) / spbMs) * spbMs;
tryGrid({ gap_ms: Math.round((g.gap_ms + offset) * 100) / 100 });
}
async function applyGrid() {
if (!gridDraft || saving) return;
if (dirty()) { say("save or discard this line's changes first", true); return; }
await send("grid", "PUT", gridDraft, "charting on the new grid…", "grid applied");
}
async function resetGrid() {
if (!review.grid_edited || saving) return;
if (dirty()) { say("save or discard this line's changes first", true); return; }
await send("grid", "DELETE", null, "resetting the grid…",
"grid reset to the BPM songtrace chose");
}
function paintGrid(typing) {
const g = grid();
gridBar.hidden = !review.grid;
if (!g) return;
const round = (v, d) => String(Math.round(v * 10 ** d) / 10 ** d);
// the field being typed in keeps what was typed, half-finished or not
if (typing !== bpmField.input) bpmField.input.value = round(g.bpm, 2);
if (typing !== gapField.input) gapField.input.value = round(g.gap_ms, 2);
gridSource.textContent = gridDraft ? "not applied"
: review.grid_edited ? "set in the editor"
: review.auto_bpm ? "songtrace's choice" : "";
gridSource.classList.toggle("sheet-grid-pending", !!gridDraft);
gridTools.align.disabled = saving || selected.size !== 1;
gridTools.cancel.disabled = !gridDraft || saving;
gridTools.apply.disabled = !gridDraft || saving;
gridTools.reset.disabled = !review.grid_edited || !!gridDraft || saving;
for (const b of [gridTools.double, gridTools.half, gridTools.earlier, gridTools.later]) {
b.disabled = saving;
}
}
function render(typing = null) {
if (!review || index === null) return;
const line = review.lines[index];
const { t0, t1, lo, hi } = layout;
const width = Math.max(plot.clientWidth, 320);
const rowH = PLOT_HEIGHT / (hi - lo + 1);
const x = (t) => GUTTER + ((t - t0) / (t1 - t0)) * (width - GUTTER);
const y = (p) => (hi - p) * rowH; // top of the row for pitch p
view = { t0, t1, x, width, rowH, hi,
time: (px) => t0 + ((px - GUTTER) / (width - GUTTER)) * (t1 - t0),
pitch: (py) => hi - Math.floor(py / rowH) };
svg.setAttribute("viewBox", `0 0 ${width} ${PLOT_HEIGHT}`);
svg.setAttribute("width", width);
svg.setAttribute("height", PLOT_HEIGHT);
svg.replaceChildren();
const rows = el("g", { class: "sheet-rows" }, svg);
for (let p = lo; p <= hi; p++) {
el("rect", { x: GUTTER, y: y(p), width: width - GUTTER, height: rowH,
class: isBlackKey(p) ? "sheet-row sheet-row-black" : "sheet-row" }, rows);
if (mod12(p) === 0) {
el("line", { x1: GUTTER, x2: width, y1: y(p) + rowH, y2: y(p) + rowH,
class: "sheet-octave" }, rows);
}
// every C names its octave; the naturals between get a name when
// there is room for it
if (mod12(p) === 0 || (rowH >= 16 && !isBlackKey(p))) {
const label = el("text", { x: GUTTER - 6, y: y(p) + rowH / 2,
class: mod12(p) === 0 ? "sheet-key sheet-key-c" : "sheet-key" },
rows);
label.textContent = noteName(p);
}
}
if (grid()) {
const beats = el("g", { class: "sheet-beats" }, svg);
const spbPx = x(60 / (4 * grid().bpm)) - x(0);
for (const b of beatLines(t0, t1, grid())) {
if (b.weight === 0 && spbPx < 10) continue;
el("line", { x1: x(b.t), x2: x(b.t), y1: 0, y2: PLOT_HEIGHT,
class: `sheet-beat sheet-beat-${b.weight}` }, beats);
}
}
const layer = el("g", { class: "sheet-notes" }, svg);
const pad = Math.min(2, rowH * 0.12);
const drawNote = (n, current) => {
const cls = ["sheet-note", `sheet-note-${KINDS[n.kind] ?? "normal"}`];
if (!current) cls.push("sheet-note-context");
if (current && n.low_confidence) cls.push("sheet-note-low");
if (current && selected.has(n.id)) cls.push("sheet-note-selected");
const rect = el("rect", { x: x(n.start), y: y(n.pitch) + pad,
width: Math.max(x(n.end) - x(n.start), 2), height: rowH - 2 * pad,
rx: Math.min(4, rowH / 3), class: cls.join(" ") }, layer);
if (current) rect.dataset.id = n.id;
el("title", {}, rect).textContent = `${n.text.trim() || "~"} · ${noteName(n.pitch)}`;
};
for (const l of review.lines) {
if (l.dropped || l.index === index) continue;
for (const n of l.notes) if (n.end >= t0 && n.start <= t1) drawNote(n, false);
}
for (const n of notes()) drawNote(n, true);
const marks = el("g", { class: "sheet-breaks" }, svg);
for (const slot of breakSlots()) {
const { t, pinned } = shownBreakTime(slot);
const value = hist.present[slot.which];
const byHand = value !== null &&
(value !== baseline[slot.which] || review.breaks?.find(b => b.line === slot.line)?.moved);
const cls = `sheet-break${byHand ? " sheet-break-moved" : ""}`;
el("line", { x1: x(t), x2: x(t), y1: 0, y2: PLOT_HEIGHT, class: cls }, marks);
const label = el("text", { x: pinned === "right" ? x(t) - 3 : x(t) + 3, y: 11,
"text-anchor": pinned === "right" ? "end" : "start",
class: `${cls} sheet-break-label` }, marks);
label.textContent = pinned === "right" ? "↵ →" : pinned === "left" ? "← ↵" : "↵";
}
syllables.replaceChildren(...notes().map(n => {
const s = div("sheet-syllable" + (selected.has(n.id) ? " sheet-syllable-selected" : ""),
n.text.trim() || "~");
s.style.left = `${x(n.start)}px`;
s.dataset.id = n.id;
return s;
}));
// over the notes: the curve matters most where it agrees with one
if (curve) {
const g = el("g", { class: "sheet-curve" }, svg);
const from = Math.max(0, Math.floor((t0 - curve.start) / curve.step));
const to = Math.min(curve.pitch.length, Math.ceil((t1 - curve.start) / curve.step));
let pts = [];
const flush = () => {
if (pts.length > 1) el("polyline", { points: pts.join(" ") }, g);
pts = [];
};
for (let k = from; k < to; k++) {
const p = curve.pitch[k];
if (p === null || p < lo - 0.5 || p > hi + 0.5) { flush(); continue; }
pts.push(`${x(curve.start + k * curve.step).toFixed(1)},${(y(p) + rowH / 2).toFixed(1)}`);
}
flush();
}
words.replaceChildren(...line.words.map(w => {
const node = div("sheet-word", w.text);
node.style.left = `${x(w.start)}px`;
node.style.width = `${Math.max(x(w.end) - x(w.start), 2)}px`;
return node;
}));
const lines = visible();
const at = lines.findIndex(l => l.index === index);
title.textContent = `LINE ${at + 1} / ${lines.length}`;
prev.disabled = at <= 0;
next.disabled = at >= lines.length - 1;
const tags = [];
if (dirty()) tags.push(["unsaved", "sheet-badge-dirty"]);
if (line.edited) tags.push(["edited", "sheet-badge-edited"]);
if (notes().some(n => n.low_confidence)) tags.push(["low confidence", "sheet-badge-low"]);
badges.replaceChildren(...tags.map(([text, cls]) => div(`sheet-badge ${cls}`, text)));
paintTools();
paintGrid(typing);
}
function paintSelection() {
for (const node of root.querySelectorAll("[data-id]")) {
const on = selected.has(Number(node.dataset.id));
node.classList.toggle(node.tagName === "rect" ? "sheet-note-selected"
: "sheet-syllable-selected", on);
}
paintTools();
paintGrid();
}
function paintTools() {
const line = review.lines[index];
const editable = canEdit();
tools.undo.disabled = !editable || !hist.canUndo;
tools.redo.disabled = !editable || !hist.canRedo;
tools.split.disabled = tools.merge.disabled = !editable || one() === null;
for (const b of [tools.remove, tools.golden, tools.free, tools.rap]) {
b.disabled = !editable || !selected.size;
}
tools.discard.disabled = !dirty() || saving;
tools.save.disabled = !dirty() || saving;
tools.revert.disabled = !line.edited || saving;
}
// --- pointer -------------------------------------------------------------
function hit(e) {
const box = plot.getBoundingClientRect();
const px = e.clientX - box.left;
const py = e.clientY - box.top;
// by geometry, not the event's target: a render replaces the elements
// under the pointer, and the topmost note drawn wins
if (canEdit()) {
const brk = breakSlots().find(s => Math.abs(px - view.x(shownBreakTime(s).t)) <= 5);
if (brk) return { px, py, note: null, brk, mode: "break" };
}
const pitch = view.pitch(py);
const note = [...notes()].reverse().find(n => n.pitch === pitch &&
px >= view.x(n.start) - 1 && px <= view.x(n.end) + 1) ?? null;
let mode = null;
if (note) {
const left = view.x(note.start);
const right = view.x(note.end);
mode = right - left > 3 * EDGE && px >= right - EDGE ? "end"
: right - left > 3 * EDGE && px <= left + EDGE ? "start" : "move";
}
return { px, py, note, brk: null, mode };
}
plot.addEventListener("pointermove", (e) => {
if (!view) return;
if (!drag) {
const { note, brk, mode } = hit(e);
plot.style.cursor = brk ? "col-resize"
: !note || !canEdit() ? "crosshair"
: mode === "move" ? "grab" : "ew-resize";
return;
}
const box = plot.getBoundingClientRect();
const px = e.clientX - box.left;
const py = e.clientY - box.top;
if (Math.abs(px - drag.px) + Math.abs(py - drag.py) > 3) drag.moved = true;
if (drag.moved && drag.brk) {
const { which, lo, hi } = drag.brk;
breakPreview = { which, t: edit.placeBreak(view.time(px), lo, hi, grid()) };
render();
return;
}
if (!drag.moved || !drag.note) return;
if (drag.mode === "move") {
const spbPx = view.x(60 / (4 * grid().bpm)) - view.x(0);
const beats = Math.round((px - drag.px) / spbPx);
const semis = Math.round((drag.py - py) / view.rowH);
preview = edit.move(drag.base, selected, beats, semis, grid());
} else {
preview = edit.resize(drag.base, drag.note.id, drag.mode, view.time(px), grid());
}
render();
});
plot.addEventListener("pointerdown", (e) => {
if (!view || e.button !== 0) return;
// the browser's own focus handling would otherwise take focus off the
// sheet, and with it the keyboard
e.preventDefault();
plot.focus();
const { px, py, note, brk, mode } = hit(e);
if (px < GUTTER) return;
if (brk) plot.setPointerCapture(e.pointerId);
if (note && canEdit()) {
if (e.shiftKey) {
selected = new Set(selected);
if (selected.has(note.id)) selected.delete(note.id); else selected.add(note.id);
} else if (!selected.has(note.id)) {
selected = new Set([note.id]);
}
plot.setPointerCapture(e.pointerId);
plot.style.cursor = mode === "move" ? "grabbing" : "ew-resize";
paintSelection();
}
drag = { px, py, note, brk, mode, base: notes(), moved: false };
});
const endDrag = (e, cancelled) => {
if (!drag) return;
const d = drag;
drag = null;
if (breakPreview) {
const { which, t } = breakPreview;
breakPreview = null;
if (cancelled) render(); else changeBreak(which, t);
} else if (preview) {
const next = preview;
preview = null;
if (cancelled) render(); else change(next);
} else if (!d.moved && !d.note && !d.brk && !cancelled) {
selected = new Set();
render();
seek(view.time(d.px));
}
if (plot.hasPointerCapture?.(e.pointerId)) plot.releasePointerCapture(e.pointerId);
};
plot.addEventListener("pointerup", (e) => endDrag(e, false));
plot.addEventListener("pointercancel", (e) => endDrag(e, true));
plot.addEventListener("dblclick", (e) => {
if (!view || !canEdit()) return;
const { px, py, note, brk } = hit(e);
if (px < GUTTER) return;
if (brk) { changeBreak(brk.which, null); return; } // back to automatic
if (note) { editText(note.id); return; }
const out = edit.insert(notes(), view.time(px), view.pitch(py), grid());
change(out.notes, new Set([out.id]));
editText(out.id);
});
// --- syllable text -------------------------------------------------------
function editText(id) {
const note = notes().find(n => n.id === id);
if (!note) return;
const input = document.createElement("input");
input.className = "sheet-text-input";
input.value = note.text.trim();
input.style.left = `${view.x(note.start)}px`;
input.setAttribute("aria-label", "syllable");
syllables.append(input);
input.focus();
input.select();
let done = false;
const finish = (commit) => {
if (done) return;
done = true;
const value = input.value;
input.remove();
if (commit) {
const next = edit.setText(notes(), id, value);
if (next !== notes()) change(next);
}
plot.focus();
};
input.addEventListener("keydown", (e) => {
e.stopPropagation();
if (e.key === "Enter") finish(true);
else if (e.key === "Escape") finish(false);
});
input.addEventListener("blur", () => finish(true));
}
syllables.addEventListener("dblclick", (e) => {
const id = e.target.dataset?.id;
if (id) editText(Number(id));
});
// --- keyboard ------------------------------------------------------------
plot.addEventListener("keydown", (e) => {
if (!view) return;
const ctrl = e.ctrlKey || e.metaKey;
const key = e.key;
const handled = () => { e.preventDefault(); e.stopPropagation(); };
if (ctrl && key.toLowerCase() === "s") { handled(); save(); return; }
if (ctrl && key.toLowerCase() === "z") { handled(); e.shiftKey ? redo() : undo(); return; }
if (ctrl && key.toLowerCase() === "y") { handled(); redo(); return; }
if (key === "PageUp") { handled(); page(-1); return; }
if (key === "PageDown") { handled(); page(1); return; }
if (ctrl || e.altKey || !canEdit()) return;
const ns = notes();
if (key === "Tab") {
handled();
if (!ns.length) return;
const at = ns.findIndex(n => selected.has(n.id));
const to = at < 0 ? (e.shiftKey ? ns.length - 1 : 0)
: (at + (e.shiftKey ? -1 : 1) + ns.length) % ns.length;
selected = new Set([ns[to].id]);
render();
return;
}
if (key === "?") { handled(); toggleKeys(); return; }
if (key === "Escape") {
handled();
if (!keys.hidden) showKeys(false);
else { selected = new Set(); render(); }
return;
}
if (key.toLowerCase() === "p") { handled(); playCurrent(); return; }
if (!selected.size) return;
const arrows = { ArrowLeft: -1, ArrowRight: 1 };
if (key in arrows) {
handled();
change(e.shiftKey ? edit.stretch(ns, selected, arrows[key], grid())
: edit.move(ns, selected, arrows[key], 0, grid()));
} else if (key === "ArrowUp" || key === "ArrowDown") {
handled();
change(edit.move(ns, selected, 0, (key === "ArrowUp" ? 1 : -1) * (e.shiftKey ? 12 : 1), grid()));
} else if (key === "Delete" || key === "Backspace") {
handled(); removeSelected();
} else if (key === "Enter" && one() !== null) {
handled(); editText(one());
} else if (key.toLowerCase() === "s") {
handled(); splitSelected();
} else if (key.toLowerCase() === "m") {
handled(); mergeSelected();
} else if ({ g: "*", f: "F", r: "R" }[key.toLowerCase()]) {
handled(); kind({ g: "*", f: "F", r: "R" }[key.toLowerCase()]);
}
});
window.addEventListener("beforeunload", (e) => {
if (dirty()) e.preventDefault();
});
new ResizeObserver(() => render()).observe(plot);
return {
// keeps the line on show when the song is the same one, refreshed
load(id, data) {
const same = id === jobId;
review = data;
root.hidden = false;
if (!same) {
jobId = id;
curve = null;
gridDraft = null;
fetch(`/api/jobs/${id}/pitch`)
.then(r => (r.ok ? r.json() : null))
.then(c => { if (jobId === id) { curve = c; render(); } })
.catch(() => {});
}
const lines = visible();
if (!lines.length) { root.hidden = true; return; }
open(same && lines.some(l => l.index === index) ? index : lines[0].index);
},
clear() {
review = null;
jobId = null;
curve = null;
index = null;
view = layout = hist = baseline = preview = gridDraft = null;
root.hidden = true;
},
// playback following the song: never at the cost of unsaved edits
showLine(i) {
if (review && i !== index && !dirty()) open(i);
},
isDirty: dirty,
setTime(t) {
if (!view) return;
const inside = t >= view.t0 && t <= view.t1;
playhead.hidden = !inside;
if (inside) playhead.style.left = `${view.x(t)}px`;
},
};
}
+91 -4
View File
@@ -9,6 +9,7 @@ import uuid
from dataclasses import asdict, dataclass, field
from pathlib import Path
import numpy as np
from fastapi import Body, FastAPI, File, Form, HTTPException, UploadFile
from fastapi.concurrency import run_in_threadpool
from fastapi.responses import FileResponse
@@ -18,6 +19,7 @@ from . import edits, grid
from . import rebuild as rebuild_policy
from .config import Config
from .lyrics import normalize_lyrics
from .notes import CONF_MASK
from .pipeline import (STAGE_ORDER, PipelineError, SongInput, read_warnings,
run_pipeline, song_slug)
from .ultrastar import get_header, parse_ultrastar
@@ -340,11 +342,21 @@ def create_app(cfg: Config, runner=run_pipeline) -> FastAPI:
position = {job_id: n for n, job_id in enumerate(order)}
return sorted(latest.values(), key=lambda j: position[j.id], reverse=True)
def _edit_counts(job: Job) -> dict:
"""How much of the song was edited by hand, from its edits file."""
path = cfg.work_dir / song_slug(job.inp) / edits.EDITS_FILE
try:
data = json.loads(path.read_text())
except (OSError, ValueError):
return {"edited_lines": 0, "stale_edits": 0}
return {"edited_lines": len(data.get("lines", {})),
"stale_edits": len(data.get("stale", []))}
@app.get("/api/rebuild-all")
def rebuild_all_candidates():
return [{"id": j.id, "title": j.title, "artist": j.artist,
"status": j.status, "rebuilt_from": j.rebuilt_from,
"finished_at": j.finished_at}
"finished_at": j.finished_at, **_edit_counts(j)}
for j in _current_songs()]
@app.post("/api/rebuild-all", status_code=201)
@@ -380,11 +392,16 @@ def create_app(cfg: Config, runner=run_pipeline) -> FastAPI:
@app.get("/api/jobs")
def list_jobs():
# edits belong to the song as it is now, so only the job standing for
# it carries them; the runs it replaced would claim them twice
counts = {j.id: _edit_counts(j) for j in _current_songs()}
none = {"edited_lines": 0, "stale_edits": 0}
with lock:
return [{"id": j.id, "title": j.title, "artist": j.artist,
"status": j.status, "stage": j.stage,
"created_at": j.created_at, "started_at": j.started_at,
"finished_at": j.finished_at, "rebuilt_from": j.rebuilt_from}
"finished_at": j.finished_at, "rebuilt_from": j.rebuilt_from,
**counts.get(j.id, none)}
for j in (jobs[i] for i in reversed(order))]
@app.get("/api/jobs/{job_id}")
@@ -501,8 +518,38 @@ def create_app(cfg: Config, runner=run_pipeline) -> FastAPI:
for e in notes_data.get("stale", [])]
media = {kind: _resolve_media(job, header) is not None
for kind, header in MEDIA_HEADERS.items()}
# each break with the line it comes before, the key a move is saved by
moved = set(notes_data.get("moved_breaks", []))
starts = sorted((n["start"], n["line_index"]) for n in notes_data["notes"]
if "line_index" in n)
breaks = []
for t in notes_data.get("breaks", []):
after = next((li for start, li in starts if start >= t), None)
if after is not None:
breaks.append({"line": after, "time": t, "moved": after in moved})
return {"lines": lines, "missing_spans": missing_spans, "media": media,
"stale": stale, "grid": _chart_grid(work)}
"stale": stale, "breaks": breaks, "grid": _chart_grid(work),
"grid_edited": bool(notes_data.get("grid")), "auto_bpm": _auto_bpm(work)}
@app.get("/api/jobs/{job_id}/pitch")
def get_pitch(job_id: str):
"""The detected vocal pitch, to draw behind the notes: UltraStar
semitones (0 = C4) every `step` seconds, null where nothing is sung."""
job = _finished_job(job_id)
path = cfg.work_dir / song_slug(job.inp) / "pitch.npz"
if not path.exists():
raise HTTPException(404)
data = np.load(path)
times, f0, conf = data["times"], data["f0"], data["conf"]
# every other 10ms frame is still finer than a note can be drawn
stride = 2
f0, conf = f0[::stride], conf[::stride]
voiced = (conf >= CONF_MASK) & (f0 > 0)
semis = 9 + 12 * np.log2(np.where(voiced, f0, 440.0) / 440.0)
hop = float(times[1] - times[0]) if len(times) > 1 else 0.01
return {"start": float(times[0]) if len(times) else 0.0, "step": hop * stride,
"pitch": [round(float(v), 2) if ok else None
for v, ok in zip(semis, voiced)]}
def _chart_grid(work: Path) -> dict | None:
"""The beat grid the chart was built on, for snapping edits to it."""
@@ -540,16 +587,56 @@ def create_app(cfg: Config, runner=run_pipeline) -> FastAPI:
raise HTTPException(400, str(e))
edits.save(work, data)
final = edits.finalize(work, lyrics)
# the editor's grid, else the pipeline's own tempo with GAP at
# the first note, as the grid stage would chart it
chosen = final["grid"] or {"bpm": _auto_bpm(work), "gap_ms": None}
charts = [work / "song.txt"]
if job.output_dir:
charts.append(Path(job.output_dir) / f"{song_slug(job.inp)}.txt")
for txt in charts:
if txt.exists():
grid.rechart(txt, final["notes"], final["breaks"])
grid.rechart(txt, final["notes"], final["breaks"],
chosen["bpm"], chosen["gap_ms"])
with lock:
job.warnings = list(final["warnings"])
return get_review(job_id)
def _auto_bpm(work: Path) -> float | None:
"""The tempo the pipeline chose: recorded by grid, or for a song built
before that was, the one its chart already has."""
try:
return float(json.loads((work / grid.TEMPO_FILE).read_text())["bpm"])
except (OSError, ValueError, KeyError):
g = _chart_grid(work)
return g["bpm"] if g else None
@app.put("/api/jobs/{job_id}/breaks")
def save_breaks(job_id: str, body: dict = Body(...)):
"""Move line breaks: {line: seconds}, keyed by the line each comes
before; null puts one back where the notes alone would place it."""
return _edit(job_id, lambda data, lyrics: edits.set_breaks(data, body))
@app.put("/api/jobs/{job_id}/grid")
def save_grid(job_id: str, body: dict = Body(...)):
"""Set the chart's BPM and GAP, kept through rebuilds like any edit."""
job = _editable(job_id)
work = cfg.work_dir / song_slug(job.inp)
tempo = work / grid.TEMPO_FILE
# a song built before grid recorded its tempo: the chart's, taken now
# while it is still the pipeline's, is what RESET must return to
if not tempo.exists() and (bpm := _auto_bpm(work)) is not None:
tempo.write_text(json.dumps({"bpm": bpm}))
return _edit(job_id, lambda data, lyrics: edits.set_grid(
data, body.get("bpm"), body.get("gap_ms")))
@app.delete("/api/jobs/{job_id}/grid")
def reset_grid(job_id: str):
"""Go back to the BPM the pipeline chose and GAP at the first note."""
def change(data, lyrics):
if not edits.clear_grid(data):
raise HTTPException(404, "the grid was not set in the editor")
return _edit(job_id, change)
def _line(lyrics, index: int) -> int:
if not 0 <= index < len(lyrics):
raise HTTPException(404, f"no line {index}")
+129
View File
@@ -0,0 +1,129 @@
import assert from "node:assert/strict";
import test from "node:test";
import * as m from "../../src/songtrace/static/edit-model.js";
// 300 BPM: one UltraStar beat is 0.05s, starting at GAP = 1s
const GRID = { bpm: 300, gap_ms: 1000 };
const near = (a, b) => Math.abs(a - b) < 1e-9;
function line() {
return m.withIds([
{ start: 1.52, end: 1.71, pitch: 0, text: "one", kind: ":", low_confidence: false },
{ start: 1.8, end: 2.0, pitch: 2, text: " two", kind: ":", low_confidence: true },
]);
}
test("snapping lands on the beat grid from GAP", () => {
assert.ok(near(m.snap(1.52, GRID), 1.5));
assert.ok(near(m.snap(1.53, GRID), 1.55));
assert.ok(near(m.snap(0.97, GRID), 0.95)); // beats before GAP too
});
test("moving snaps to the grid, then shifts by beats and semitones", () => {
const notes = line();
const out = m.move(notes, new Set([notes[0].id]), 2, -3, GRID);
assert.ok(near(out[0].start, 1.6) && near(out[0].end, 1.8));
assert.equal(out[0].pitch, -3);
assert.equal(out[1], notes[1]); // untouched note is the same object
assert.equal(notes[0].start, 1.52); // input left alone
});
test("moving never goes before the start of the song", () => {
const notes = m.withIds([{ start: 0.1, end: 0.2, pitch: 0, text: "x", kind: ":" }]);
const out = m.move(notes, new Set([notes[0].id]), -100, 0, GRID);
assert.ok(near(out[0].start, 0));
});
test("resizing moves one edge and keeps at least a beat", () => {
const notes = line();
const id = notes[0].id;
assert.ok(near(m.resize(notes, id, "end", 1.9, GRID)[0].end, 1.9));
assert.ok(near(m.resize(notes, id, "end", 1.0, GRID)[0].end, 1.55));
const startMoved = m.resize(notes, id, "start", 1.4, GRID)[0];
assert.ok(near(startMoved.start, 1.4) && near(startMoved.end, 1.7));
});
test("stretching changes length by whole beats", () => {
const notes = line();
const out = m.stretch(notes, new Set([notes[1].id]), 2, GRID);
assert.ok(near(out[1].start, 1.8) && near(out[1].end, 2.1));
assert.ok(near(m.stretch(notes, new Set([notes[1].id]), -10, GRID)[1].end, 1.85));
});
test("splitting halves a note on a beat and holds the syllable", () => {
const notes = line();
const { notes: out, ids } = m.split(notes, notes[1].id, GRID);
assert.equal(out.length, 3);
assert.ok(near(out[1].end, 1.9) && near(out[2].start, 1.9));
assert.deepEqual([out[1].text, out[2].text], [" two", "~"]);
assert.equal(ids.length, 2);
// one beat cannot be split
const short = m.withIds([{ start: 1.0, end: 1.05, pitch: 0, text: "x", kind: ":" }]);
assert.equal(m.split(short, short[0].id, GRID).notes, short);
});
test("merging joins a note with the next, dropping a held ~", () => {
const notes = m.withIds([
{ start: 1.0, end: 1.2, pitch: 0, text: "sev", kind: ":" },
{ start: 1.2, end: 1.4, pitch: 0, text: "en", kind: ":" },
{ start: 1.4, end: 1.6, pitch: 0, text: "~", kind: ":" },
]);
const once = m.merge(notes, notes[0].id);
assert.deepEqual(once.map(n => n.text), ["seven", "~"]);
const twice = m.merge(once, once[0].id);
assert.deepEqual(twice.map(n => [n.text, n.end]), [["seven", 1.6]]);
assert.equal(m.merge(twice, twice[0].id), twice); // nothing after it
});
test("inserting adds a two-beat held note on the grid", () => {
const { notes, id } = m.insert(line(), 1.76, 4, GRID);
const added = notes.find(n => n.id === id);
assert.ok(near(added.start, 1.75) && near(added.end, 1.85));
assert.deepEqual([added.pitch, added.text, added.kind], [4, "~", ":"]);
assert.deepEqual(notes.map(n => n.text), ["one", "~", " two"]);
});
test("a kind toggles back to normal when every selected note has it", () => {
const notes = line();
const both = new Set(notes.map(n => n.id));
const golden = m.setKind(notes, both, "*");
assert.deepEqual(golden.map(n => n.kind), ["*", "*"]);
assert.deepEqual(m.setKind(golden, both, "*").map(n => n.kind), [":", ":"]);
const mixed = m.setKind(golden, new Set([notes[0].id]), "F");
assert.deepEqual(m.setKind(mixed, both, "F").map(n => n.kind), ["F", "F"]);
});
test("editing text keeps a word-initial space and refuses blank", () => {
const notes = line();
assert.equal(m.setText(notes, notes[1].id, "too")[1].text, " too");
assert.equal(m.setText(notes, notes[0].id, "won")[0].text, "won");
assert.equal(m.setText(notes, notes[0].id, " won")[0].text, " won");
assert.equal(m.setText(notes, notes[0].id, " "), notes);
});
test("saved notes carry duration and kind, not ids", () => {
const saved = m.toSaved(line());
assert.deepEqual(Object.keys(saved[0]).sort(), ["duration", "kind", "pitch", "start", "text"]);
assert.ok(near(saved[0].duration, 0.19));
});
test("history undoes and redoes, and a new change drops the redo", () => {
const h = m.history("a");
h.push("b");
h.push("c");
assert.equal(h.undo(), "b");
assert.equal(h.undo(), "a");
assert.equal(h.canUndo, false);
assert.equal(h.redo(), "b");
h.push("d");
assert.equal(h.canRedo, false);
assert.equal(h.present, "d");
});
test("a dragged break snaps to a beat but stays between its lines", () => {
assert.ok(near(m.placeBreak(1.52, 1.4, 1.8, GRID), 1.5));
assert.equal(m.placeBreak(1.0, 1.42, 1.8, GRID), 1.42); // not into the line before
assert.equal(m.placeBreak(2.5, 1.4, 1.78, GRID), 1.78); // nor into the line after
assert.equal(m.placeBreak(1.6, 1.9, 1.8, GRID), 1.9); // lines overlap: at the end of the first
});
+72
View File
@@ -0,0 +1,72 @@
import assert from "node:assert/strict";
import test from "node:test";
import { beatLines, isBlackKey, lineSpan, noteName, pitchRange }
from "../../src/songtrace/static/sheet.js";
test("note names follow UltraStar's C4 = 0", () => {
assert.equal(noteName(0), "C4");
assert.equal(noteName(9), "A4");
assert.equal(noteName(-1), "B3");
assert.equal(noteName(13), "C♯5");
assert.equal(noteName(-13), "B2");
});
test("black keys are the sharps, in any octave", () => {
assert.deepEqual([0, 1, 2, 3, 4, 5, 6].map(isBlackKey),
[false, true, false, true, false, false, true]);
assert.equal(isBlackKey(-11), true); // C#3
assert.equal(isBlackKey(-12), false); // C3
});
test("the pitch range pads the line's notes and keeps a minimum height", () => {
assert.deepEqual(pitchRange([0, 5]), { lo: -4, hi: 10 }); // 15 rows
assert.deepEqual(pitchRange([0, 20]), { lo: -3, hi: 23 }); // padded only
assert.deepEqual(pitchRange([]), { lo: -7, hi: 7 });
});
test("beat lines sit on the chart's grid with bar weights", () => {
// 300 BPM: an UltraStar beat is 60 / (4 * 300) = 0.05s, from GAP 1.0s
const lines = beatLines(1.0, 1.85, { bpm: 300, gap_ms: 1000 });
assert.equal(lines.length, 18);
assert.deepEqual(lines.slice(0, 5).map(b => b.weight), [2, 0, 0, 0, 1]);
assert.equal(lines[16].weight, 2);
assert.ok(Math.abs(lines[4].t - 1.2) < 1e-9);
});
test("beats before GAP keep their weights", () => {
const lines = beatLines(0.75, 1.0, { bpm: 300, gap_ms: 1000 });
assert.deepEqual(lines.map(b => [b.beat, b.weight]),
[[-5, 0], [-4, 1], [-3, 0], [-2, 0], [-1, 0], [0, 2]]);
});
test("a line spans its notes and words", () => {
assert.deepEqual(lineSpan({ notes: [{ start: 2, end: 3 }], words: [{ start: 1.5, end: 2.5 }] }),
{ start: 1.5, end: 3 });
assert.equal(lineSpan({ notes: [], words: [] }), null);
});
import { readFileSync } from "node:fs";
import { SHORTCUTS, comboParts } from "../../src/songtrace/static/sheet.js";
test("shortcut combos split into keys and mouse actions", () => {
assert.deepEqual(comboParts("Shift+click / Tab"), [
[{ text: "Shift", key: true }, { text: "click", key: false }],
[{ text: "Tab", key: true }],
]);
assert.deepEqual(comboParts("Page Up"), [[{ text: "Page Up", key: true }]]);
});
test("every letter key the shortcut list names is one the sheet handles", () => {
const source = readFileSync(new URL("../../src/songtrace/static/sheet.js", import.meta.url), "utf8");
const handler = source.slice(source.indexOf('plot.addEventListener("keydown"'));
const letters = SHORTCUTS.flatMap(([, entries]) => entries)
.flatMap(([combo]) => comboParts(combo).flat())
.filter(p => p.key && /^[A-Z]$/.test(p.text))
.map(p => p.text.toLowerCase());
assert.ok(letters.length >= 6);
for (const letter of new Set(letters)) {
assert.ok(handler.includes(`"${letter}"`) || handler.includes(`${letter}:`),
`the list names ${letter.toUpperCase()} but the sheet does not handle it`);
}
});
+65
View File
@@ -122,3 +122,68 @@ def test_finalize_reports_stale_edits(tmp_path):
final = edits.finalize(tmp_path, [LYRICS[0], ["other"], LYRICS[2]])
assert final["warnings"] == ["1 edited line no longer matches the lyrics; "
"set aside in the editor"]
def test_a_grid_set_in_the_editor_is_checked():
data = edits._empty(LYRICS)
edits.set_grid(data, "240.5", 1500)
assert data["grid"] == {"bpm": 240.5, "gap_ms": 1500.0}
for bpm, gap in ((0, 0), (-1, 0), (float("nan"), 0), (3000, 0), (300, -1), ("x", 0)):
with pytest.raises(ValueError):
edits.set_grid(data, bpm, gap)
def test_a_grid_alone_keeps_the_edits_file_and_survives_new_lyrics(tmp_path):
data = edits._empty(LYRICS)
edits.set_grid(data, 300, 1000)
edits.save(tmp_path, data)
assert (tmp_path / edits.EDITS_FILE).exists()
loaded = edits.load(tmp_path, [["zero"]] + LYRICS)
assert loaded["grid"] == {"bpm": 300.0, "gap_ms": 1000.0}
assert edits.clear_grid(loaded) and not edits.clear_grid(loaded)
edits.save(tmp_path, loaded)
assert not (tmp_path / edits.EDITS_FILE).exists()
def test_finalize_reports_the_grid(tmp_path):
_write_generated(tmp_path, GENERATED, [])
assert edits.finalize(tmp_path, LYRICS)["grid"] is None
data = edits._empty(LYRICS)
edits.set_grid(data, 300, 1000)
edits.save(tmp_path, data)
assert edits.finalize(tmp_path, LYRICS)["grid"] == {"bpm": 300.0, "gap_ms": 1000.0}
def test_moved_breaks_are_checked():
data = edits._empty(LYRICS)
edits.set_breaks(data, {"1": 3.5, 2: "6.25"})
assert data["breaks"] == {"1": 3.5, "2": 6.25}
edits.set_breaks(data, {1: None})
assert data["breaks"] == {"2": 6.25}
for bad in ({0: 1.0}, {3: 1.0}, {"x": 1.0}, {1: -1}, {1: "soon"}, {1: float("inf")}):
with pytest.raises(ValueError):
edits.set_breaks(data, bad)
def test_a_moved_break_stays_put_between_its_lines():
data = edits._empty(LYRICS)
edits.set_breaks(data, {1: 2.5})
notes, breaks = edits.merge(GENERATED, BREAKS, data)
assert notes == GENERATED # moving a break locks no notes
assert breaks == [2.5, 6.0]
def test_a_moved_break_never_lands_inside_a_line():
data = edits._empty(LYRICS)
edits.set_breaks(data, {1: 1.0, 2: 9.0}) # into line 0's notes, past line 2's start
_, breaks = edits.merge(GENERATED, BREAKS, data)
assert breaks == [pytest.approx(1.9), 8.0]
def test_moved_breaks_follow_their_lines_and_keep_the_file(tmp_path):
data = edits._empty(LYRICS)
edits.set_breaks(data, {1: 3.5, 2: 6.5})
edits.save(tmp_path, data)
assert (tmp_path / edits.EDITS_FILE).exists()
moved = edits.load(tmp_path, [["zero"], LYRICS[0], LYRICS[1], ["other"]])
assert moved["breaks"] == {"2": 3.5} # line 2's words changed: its break goes
+42 -1
View File
@@ -222,7 +222,7 @@ def test_grid_stage_honours_zero_bpm_override(tmp_path, monkeypatch):
monkeypatch.setattr(grid_stage_mod.grid, "detect_bpm", boom)
captured = {}
def fake_to_ultrastar(notes, breaks, file_bpm, headers):
def fake_to_ultrastar(notes, breaks, file_bpm, headers, gap_ms=None):
captured["bpm"] = file_bpm
return to_ultrastar(notes, breaks, 240.0, headers)
@@ -299,3 +299,44 @@ def test_rechart_replaces_the_notes_and_keeps_headers_and_tempo(tmp_path):
assert [k for k, _ in song.headers] == ["TITLE", "VIDEO", "BPM", "GAP"]
assert get_header(song, "BPM") == "300" and get_header(song, "GAP") == "2000"
assert [(n.kind, n.pitch, n.text) for n in song.body] == [("F", 7, "new")]
def test_an_explicit_gap_sets_the_grid():
# 300 BPM: 0.05s beats; notes at 1.02s and 1.27s against GAP 1.0s
song = to_ultrastar([note(1.02, 0.1), note(1.27, 0.1)], [], 300.0, [], gap_ms=1000)
assert get_header(song, "GAP") == "1000"
assert [n.beat for n in song.body] == [0, 5]
def test_a_gap_after_the_first_note_moves_back_by_whole_beats():
# GAP 1.13s is after the first note at 1.02s: back 3 beats to 0.98s, so
# the grid lines stay where they were set and the note is on beat 1
song = to_ultrastar([note(1.02, 0.1)], [], 300.0, [], gap_ms=1130)
assert get_header(song, "GAP") == "980"
assert song.body[0].beat == 1
def test_the_grid_stage_uses_the_editors_grid_and_records_its_own(tmp_path, monkeypatch):
from songtrace import edits
work = tmp_path / "w"
write_work(work)
monkeypatch.setattr(grid_stage_mod.grid, "detect_bpm", lambda p: 120.0)
ctx = make_ctx(work)
data = edits.load(work, [["x"]])
edits.set_grid(data, 300.0, 900)
edits.save(work, data)
grid_stage_mod.grid_stage().run(ctx)
song = parse_ultrastar((work / "song.txt").read_text(encoding="utf-8"))
assert (get_header(song, "BPM"), get_header(song, "GAP")) == ("300", "900")
assert json.loads((work / "tempo.json").read_text()) == {"bpm": 240.0}
def test_rechart_can_set_a_new_bpm_and_gap(tmp_path):
from songtrace.grid import rechart
from songtrace.ultrastar import serialize_ultrastar
txt = tmp_path / "song.txt"
txt.write_text(serialize_ultrastar(to_ultrastar([note(1.0, 0.5)], [], 240.0, [])),
encoding="utf-8")
rechart(txt, [note(1.02, 0.1)], [], 300.0, 1000)
song = parse_ultrastar(txt.read_text(encoding="utf-8"))
assert (get_header(song, "BPM"), get_header(song, "GAP")) == ("300", "1000")
+15
View File
@@ -0,0 +1,15 @@
import shutil
import subprocess
from pathlib import Path
import pytest
def test_static_js_unit_tests():
"""The page's own logic, run under Node's test runner."""
node = shutil.which("node")
if node is None:
pytest.skip("node is not installed")
tests = sorted(str(p) for p in (Path(__file__).parent / "js").glob("*.test.mjs"))
proc = subprocess.run([node, "--test", *tests], capture_output=True, text=True)
assert proc.returncode == 0, proc.stdout + proc.stderr
+134
View File
@@ -1977,3 +1977,137 @@ def test_a_stale_edit_can_be_discarded(tmp_path):
assert resp.status_code == 200 and resp.json()["stale"] == []
assert not (work / edits.EDITS_FILE).exists()
assert client.delete(f"/api/jobs/{second}/stale/0").status_code == 404
def test_pitch_curve_is_in_note_semitones_with_gaps_where_unvoiced(tmp_path):
import numpy as np
def runner(inp, cfg, log=print, force_from=None, stages=None):
result = chart_runner(inp, cfg)
times = np.arange(0, 0.1, 0.01)
f0 = np.array([440.0] * 4 + [261.63] * 4 + [0.0] * 2) # A4, C4, nothing
conf = np.array([0.9] * 4 + [0.9, 0.9, 0.1, 0.1] + [0.9] * 2)
np.savez(cfg.work_dir / song_slug(inp) / "pitch.npz", times=times, f0=f0, conf=conf)
return result
client, _ = make_client(tmp_path, runner)
job_id = make_charted_job(client)
body = client.get(f"/api/jobs/{job_id}/pitch").json()
assert body["start"] == 0.0 and body["step"] == pytest.approx(0.02)
assert body["pitch"][:2] == [9.0, 9.0] # A4 is 9 above C4
assert body["pitch"][2] == pytest.approx(0.0, abs=0.01) # C4
assert body["pitch"][3:] == [None, None] # unsure, then silent
def test_pitch_curve_is_404_without_a_pitch_track(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
assert client.get(f"/api/jobs/{job_id}/pitch").status_code == 404
def _chart_header(path, key):
from songtrace.ultrastar import get_header, parse_ultrastar
return get_header(parse_ultrastar(path.read_text(encoding="utf-8")), key)
def test_setting_the_grid_recharts_and_survives_line_saves(tmp_path):
client, cfg = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
resp = client.put(f"/api/jobs/{job_id}/grid", json={"bpm": 240, "gap_ms": 900})
assert resp.status_code == 200
body = resp.json()
assert body["grid"] == {"bpm": 240.0, "gap_ms": 900.0} and body["grid_edited"]
packaged = cfg.output_dir / "A - T" / "A - T.txt"
assert (_chart_header(packaged, "BPM"), _chart_header(packaged, "GAP")) == ("240", "900")
client.put(f"/api/jobs/{job_id}/lines/1", json=EDIT)
assert (_chart_header(packaged, "BPM"), _chart_header(packaged, "GAP")) == ("240", "900")
def test_resetting_the_grid_returns_to_the_pipelines_tempo(tmp_path):
client, cfg = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
(cfg.work_dir / "A - T" / "tempo.json").write_text(json.dumps({"bpm": 300.0}))
client.put(f"/api/jobs/{job_id}/grid", json={"bpm": 240, "gap_ms": 900})
resp = client.delete(f"/api/jobs/{job_id}/grid")
assert resp.status_code == 200
body = resp.json()
assert body["grid"] == {"bpm": 300.0, "gap_ms": 1000.0} # GAP back at the first note
assert not body["grid_edited"] and body["auto_bpm"] == 300.0
assert client.delete(f"/api/jobs/{job_id}/grid").status_code == 404
def test_a_bad_grid_is_refused(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
for body in ({"bpm": 0, "gap_ms": 0}, {"bpm": 300, "gap_ms": -5}, {"bpm": "x"}, {}):
assert client.put(f"/api/jobs/{job_id}/grid", json=body).status_code == 400
def test_reset_on_a_song_built_before_tempo_was_recorded_restores_its_bpm(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client) # charted at 300 BPM, no tempo.json
client.put(f"/api/jobs/{job_id}/grid", json={"bpm": 240, "gap_ms": 900})
body = client.delete(f"/api/jobs/{job_id}/grid").json()
assert body["grid"]["bpm"] == 300.0 and body["auto_bpm"] == 300.0
def test_moving_a_break_recharts_and_is_reported(tmp_path):
from songtrace.ultrastar import LineBreak, parse_ultrastar
client, cfg = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
assert client.get(f"/api/jobs/{job_id}/review").json()["breaks"] == [
{"line": 1, "time": 2.0, "moved": False}]
resp = client.put(f"/api/jobs/{job_id}/breaks", json={"1": 2.8})
assert resp.status_code == 200
assert resp.json()["breaks"] == [{"line": 1, "time": 2.8, "moved": True}]
assert not any(line["edited"] for line in resp.json()["lines"])
packaged = cfg.output_dir / "A - T" / "A - T.txt"
body = parse_ultrastar(packaged.read_text(encoding="utf-8")).body
# 300 BPM: 0.05s beats from GAP 1.0s, so 2.8s is beat 36
assert [b.beat for b in body if isinstance(b, LineBreak)] == [36]
back = client.put(f"/api/jobs/{job_id}/breaks", json={"1": None}).json()
assert back["breaks"] == [{"line": 1, "time": 2.0, "moved": False}]
def test_a_bad_break_is_refused(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
for body in ({"0": 1.0}, {"9": 1.0}, {"1": -2}, {"1": "x"}):
assert client.put(f"/api/jobs/{job_id}/breaks", json=body).status_code == 400
def _listed(client, job_id):
return next(j for j in client.get("/api/jobs").json() if j["id"] == job_id)
def test_the_queue_reports_a_songs_edited_lines(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
job_id = make_charted_job(client)
assert _listed(client, job_id)["edited_lines"] == 0
client.put(f"/api/jobs/{job_id}/lines/1", json=EDIT)
assert _listed(client, job_id)["edited_lines"] == 1
assert _listed(client, job_id)["stale_edits"] == 0
def test_edit_counts_follow_the_song_to_its_latest_run(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
first = make_charted_job(client)
client.put(f"/api/jobs/{first}/lines/1", json=EDIT)
rerun = client.post(f"/api/jobs/{first}/rerun").json()["id"]
wait_status(client, rerun, "done")
assert _listed(client, rerun)["edited_lines"] == 1
assert _listed(client, first)["edited_lines"] == 0
candidates = client.get("/api/rebuild-all").json()
assert [(c["id"], c["edited_lines"]) for c in candidates] == [(rerun, 1)]
def test_the_queue_reports_edits_set_aside(tmp_path):
client, _ = make_client(tmp_path, chart_runner)
first = make_charted_job(client)
client.put(f"/api/jobs/{first}/lines/1", json=EDIT)
second = make_charted_job(client, lyrics="one\nthree more")
# the next save looks at the edits against the new lyrics, as a rebuild does
client.put(f"/api/jobs/{second}/lines/0", json={"notes": [
{"start": 1.0, "duration": 0.5, "pitch": 0, "text": "one"}]})
listed = _listed(client, second)
assert (listed["edited_lines"], listed["stale_edits"]) == (1, 1)