branderson/songtrace
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases Wiki Activity

README: lead with requirements and Docker, refresh for the editor #36

Merged
branderson merged 2 commits from readme-refresh into main 2026-10-15 17:21:45 -07:00
Conversation 1 Commits 2 Files Changed 5
+225 -248
5 changed files with 225 additions and 248 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+171 -227
View File
@@ -1,175 +1,194 @@
# songtrace # songtrace
Turn a song and its lyrics into a playable UltraStar chart. Self-hosted karaoke chart builder for [UltraStar](https://usdx.eu/format/) and
compatible games such as [Melody Mania](https://melodymania.org/). Give it a
A CLI and minimal web UI that turns a song — title, artist, lyrics you supply, song's video and lyrics and it charts the notes, timing and pitch, ready for
and a YouTube URL or local video file — into a playable your collection. Fix anything in the built-in editor; your edits survive when
[UltraStar](https://usdx.eu/format/)-format song folder. Also compatible with you rebuild your whole collection after a songtrace update.
[Melody Mania](https://melodymania.org/).
![Web UI](docs/images/webui.png) ![Web UI](docs/images/webui.png)
The pipeline: ## What it does
1. **acquire** — download audio/video via `yt-dlp` (or use a local video file) You give it a title, artist, the lyrics, and a YouTube URL or a video file.
2. **separate** — split vocals/instrumental with Demucs songtrace writes a song folder (`.txt` chart, `.mp3`, video and cover) into
3. **transcribe** — send the vocal stem to your own intranet Whisper server your collection:
(OpenAI-compatible, or [onerahmet/openai-whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)),
as a sanity check that the supplied lyrics match the audio and to locate
vocal sections
4. **align** — forced-align the *supplied* reference lyrics against the vocal
stem with wav2vec2 (torchaudio's MMS_FA) to get word timings
5. **pitch** — track pitch on the vocal stem with torchcrepe
6. **notes** — turn word timings + pitch into UltraStar notes
7. **grid** — quantise notes to a beat grid
8. **package** — write the final song folder (`.txt` + `.mp3` + video + cover)
The shipped `.mp3` is the full original mix (vocals included) — the 1. **acquire**: downloads the video with `yt-dlp`, or takes your file
separated stems are used for analysis only and are not included in the 2. **separate**: splits vocals from the backing track (Demucs)
output. 3. **transcribe**: has your Whisper server transcribe the vocals, to check
the lyrics match the recording and to find where the singing is
4. **align**: places each word of *your* lyrics on the vocals (wav2vec2)
5. **pitch**: tracks the sung pitch (torchcrepe)
6. **notes**: turns word timings and pitch into notes
7. **grid**: snaps the notes to a beat grid
8. **package**: writes the song folder
**The output is a best-effort draft**, not a finished song. Every run ends by The result is a strong draft, not a finished chart. The web UI flags the notes
printing the low-confidence regions it found (e.g. it is least sure of, plays the song back against the chart, and lets you fix
`3 low-confidence notes near 0:42–0:47`) — open the result in an UltraStar what it got wrong before you sing it.
editor such as [Yass](https://www.yass-along.com/) or
[USDX](https://usdx.eu/) and touch those spots up before playing.
## Reviewing a song ## Requirements
The web UI plays the finished song folder back against the review, so you can - **An NVIDIA GPU with CUDA.** songtrace is developed on a 12 GB card.
check the timing without loading it into a game first. Pick a song in the Separating the vocals (Demucs) alone takes about 3.7 GB, and alignment holds
queue and the review panel gets the packaged video with a transport above the a whole stretch of singing in memory at once, so a long song whose vocals
lyrics; as it plays, the line being sung is highlighted, words light up as never pause needs the most. Cards under 12 GB are untested. Each stage gives
they are reached, and a playhead sweeps each line against its notes. Click any the GPU back when it finishes (see [Sharing the GPU](#sharing-the-gpu)).
line or word to play from there. `device = "cpu"` in the config runs without a GPU, but it is untested, far
slower, and needs plenty of system RAM.
- **Docker** with the
[NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/),
so the container can see the GPU.
- **A Whisper server of your own.** songtrace does not include one. Either an
OpenAI-compatible API or
[whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)
works. If it shares songtrace's GPU, it needs its own share of the memory.
- **Disk space:** a few GB of model weights, downloaded on first use, plus a
working folder per song holding its source video and separated tracks,
typically several hundred MB.
Playback is modelled on the game: the `#MP3` is the audio and the clock, and ## Quick start (Docker)
the `#VIDEO` is muted background slaved to it. It also uses the same seconds
the notes were generated from, so a word that lights up late in the browser is
late in the game too.
The packaged video has its audio track stripped for the same reason. It is the
same recording as the `#MP3`, so a player that does not mute the video plays
both at once, a few frames apart, which is heard as an echo. Songs built
before this change still have it — check with `ffprobe <song>.mp4` for an
audio stream, and rebuild with **Restart from** set to `package` to clear it
without re-downloading.
## Editing a song
Above the lyrics, the note sheet shows one line at a time on a pitch-by-beat
grid, with the pitch songtrace detected drawn over the notes: where a note is
right the curve runs through it, so a wrong one stands out. Fix it there and
**SAVE LINE**; the song folder's chart is rewritten on the spot.
Drag a note to move it and its ends to resize it; everything snaps to the
chart's beat grid and to semitones. Double-click empty space to add a note,
or a note to change its syllable. With the sheet focused: ←/→ move, ↑/↓
change pitch (Shift for an octave), Shift+←/→ change length, Tab selects the
next note, S splits, M merges with the next note, G/F/R make a note golden,
freestyle or rap, Delete removes, Ctrl+Z undoes, Ctrl+S saves and P plays
the line.
The ↵ markers either side of a line are its line breaks: drag one to move it
(it snaps to beats and stays between the lines it separates) and double-click
one to put it back where songtrace places it. Moving a break does not mark
either line's notes as edited, so they still follow a rebuild.
The **GRID** bar above the notes sets the chart's BPM and GAP. Changes redraw
the grid as you type and are charted on **APPLY**; ×2 and ÷2 fix a tempo
detected at double or half speed, and **ALIGN TO NOTE** moves GAP the least
that puts the selected note on a beat. Notes keep their timing in seconds and
re-snap to the new grid. **RESET** returns to the BPM songtrace chose (your
BPM override, or the detected tempo) with GAP at the first note.
Edits survive every rebuild, REBUILD ALL FROM… included: they are kept apart
from what the pipeline generates and laid back over it each time. **REVERT**
returns a line to the generated notes. Changing a line's lyrics sets its edit
aside rather than applying it to different words; the sheet lists those so
you can discard them.
## Configuration
Config is TOML. Resolution order:
1. `--config <path>` passed explicitly on the command line
2. the `SONGTRACE_CONFIG` environment variable
3. `~/.config/songtrace/config.toml`
Copy the example and fill in your Whisper server:
```sh ```sh
cp config.example.toml config.toml cp config.example.toml config.toml
$EDITOR config.toml # set whisper_url at minimum $EDITOR config.toml # set whisper_url to your Whisper server
```
See `config.example.toml` for every available field (`whisper_url`,
`whisper_api_key`, `whisper_model`, `whisper_api`, `output_dir`, `work_dir`,
`creator`, `demucs_model`, `crepe_model`, `device`, `bind_host`, `port`,
`ytdlp_clients`, `ytdlp_cookies`, `ytdlp_impersonate`,
`ytdlp_pot_provider_url`, `ytdlp_version`).
`whisper_api` selects the wire format: `"openai"` (default) speaks the
OpenAI-compatible `POST {whisper_url}/audio/transcriptions` API; `"asr-webservice"`
speaks [onerahmet/openai-whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)'s
`POST {whisper_url}/asr` API instead (point `whisper_url` at that server's base
URL with no `/v1` suffix — `whisper_model`/`whisper_api_key` are unused for
this backend, since the model is fixed by how that server was started).
## Docker
```sh
cp config.example.toml config.toml
$EDITOR config.toml
docker compose up -d docker compose up -d
``` ```
The web UI is then at `http://localhost:8000`. Downloaded model weights Open `http://localhost:8000`, fill in the form and press **CREATE SONG**.
(Demucs, wav2vec2, ...) are cached under `./data` on the host so they survive
container recreation.
Run the CLI inside the container instead of the web UI: Finished songs go to `output_dir` (`/data/output` in the container). To write
them straight into your game's song collection, mount that folder there in
`docker-compose.yml`:
```sh ```yaml
docker compose run --rm songtrace songtrace make \ volumes:
--title "Song Title" --artist "Artist Name" \ - ./config.toml:/config/config.toml:ro
--lyrics /data/lyrics.txt --youtube https://youtu.be/... - ./data:/data
- /path/to/your/UltraStar/songs:/data/output
``` ```
GPU passthrough requires the [NVIDIA Container Everything else lives under `./data` on the host (model weights, and each
Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/) song's working files), so it survives recreating the container.
installed on the host; `docker-compose.yml` requests one GPU via the
`nvidia` device driver. ## Reviewing and editing a song
Pick a song in the queue. The review panel plays the packaged song with its
video, highlights the line and words being sung, and shows the chart on a note
sheet above the lyrics. Playback works like the game: the `#MP3` is the clock
and the video follows it muted, so a word that lights up late here is late in
the game too.
The note sheet shows one line at a time on a pitch-by-beat grid, with the
pitch songtrace detected drawn over the notes. Where a note is right the curve
runs through it, so a wrong one stands out. Fix it there and **SAVE LINE**; the
song's chart is rewritten on the spot.
- **Notes:** drag a note to move it, drag its ends to resize it. Everything
snaps to the beat grid and to semitones. Double-click empty space to add a
note, or a note to change its syllable. With the sheet focused, the arrow
keys move notes, S splits, M merges, G/F/R make a note golden, freestyle or
rap, Ctrl+Z undoes and Ctrl+S saves. Press **?** for every shortcut.
- **Line breaks:** the ↵ markers either side of a line. Drag one to move it,
and double-click it to hand it back to songtrace.
- **Tempo:** the **GRID** bar sets the chart's BPM and GAP. It redraws as you
type and charts on **APPLY**. ×2 and ÷2 fix a tempo detected at double or
half speed, and **ALIGN TO NOTE** puts the selected note on a beat.
**RESET** goes back to songtrace's choice.
- **Undoing an edit:** **REVERT** returns a line to the generated notes.
Edits are kept apart from what the pipeline generates and laid back over it on
every rebuild, so they survive all of them, collection-wide ones included. If
you change a line's lyrics, its edit is set aside rather than applied to
different words, and the sheet lists those for you to discard. Songs with edits
are marked ✎ in the queue.
## Rebuilding
Select a song and its inputs come back up in the form; change them and press
**REBUILD SONG**. Only the stages your change affects are rerun: fixing a line
of lyrics realigns without downloading or separating again.
A songtrace update can improve a stage without any of your inputs changing.
**Restart from** on the rebuild form redoes a stage anyway, and
**REBUILD ALL FROM…**, under CREATE SONG, does it for your whole collection:
pick the stage the update improved, untick any songs to leave alone, and each
song is rebuilt from there. Your edits are kept.
Stages in order: `acquire`, `separate`, `transcribe`, `align`, `pitch`,
`notes`, `grid`, `package`. Rebuilding from `notes` onward needs no GPU.
## Configuration
`config.toml` is TOML; `config.example.toml` lists every setting. songtrace
reads, in order: `--config <path>`, the `SONGTRACE_CONFIG` environment variable
(set to `/config/config.toml` in the container), then
`~/.config/songtrace/config.toml`.
`whisper_api` chooses how to talk to Whisper: `"openai"` (the default) uses
`POST {whisper_url}/audio/transcriptions`, and `"asr-webservice"` uses
whisper-asr-webservice's `POST {whisper_url}/asr`. For the latter, give
`whisper_url` without a `/v1` suffix; `whisper_model` and `whisper_api_key`
are ignored, since that server's model is fixed when it starts.
`creator` sets the `#CREATOR` header in every chart; it defaults to
`songtrace`.
## Sharing the GPU ## Sharing the GPU
`songtrace serve` is long-lived but only wants the GPU in bursts, so it hands Three stages load a model: `separate` (Demucs), `align` (wav2vec2) and `pitch`
it back after every stage rather than holding it for the life of the process. (torchcrepe). Each releases its model and returns the memory to the driver as
Three stages load a model — `separate` (Demucs), `align` (wav2vec2) and soon as it finishes, rather than holding it for the life of the server. This
`pitch` (torchcrepe) — and each one's weights are dropped as soon as that matters most while `transcribe` waits on your Whisper server, which can take
stage finishes, followed by `torch.cuda.empty_cache()` to return the minutes. The job log says how much each stage gave back:
allocator's freed blocks to the driver instead of keeping them for reuse. This
matters most between `separate` and `align`: `transcribe` sits between them
waiting on your Whisper server, which can be minutes of a GPU nothing is
using.
Stages that gave memory back say so in the job log:
``` ```
[separate] released 3704 MiB of GPU memory [separate] released 3704 MiB of GPU memory
``` ```
The cost is reloading each model once per job instead of once per process, Reloading the models costs seconds per song. The CUDA context itself, a few
which is seconds against stages that run for minutes. What this does not hundred MB, stays with the process; stop the container to free it.
release is the CUDA context itself, a few hundred MB that belongs to the
process until it exits; freeing that would mean running each stage in a
subprocess. If you need every last MB back, stop the container between runs.
## Bare-metal ## YouTube download failures
YouTube changes its defences every few weeks, so songtrace keeps `yt-dlp`
current by itself. Once a day, and again when a download fails, it checks PyPI
for a newer stable release, installs it under `/data/models/cache`, and tries
it before the copy it shipped with. That copy is never modified and is always
the fallback, so a broken release costs a retry, not the download. The job log
names the `yt-dlp` version behind each attempt. `ytdlp_version` pins a release,
or `"bundled"` turns the updates off.
If downloads still fail with `HTTP Error 403: Forbidden`:
1. **The network is rate-limited or flagged.** Datacenter and VPN ranges are
throttled hardest. Try another network before changing config.
2. **A PO token is required.** If the log says so, run
[bgutil-ytdlp-pot-provider](https://github.com/Brainicism/bgutil-ytdlp-pot-provider)
alongside songtrace, install its plugin into the image, and point
`ytdlp_pot_provider_url` at it.
3. **Other levers:** `ytdlp_clients` tries each listed YouTube player client
in turn. `ytdlp_cookies` takes a Netscape cookie jar for age-gated videos;
don't use one from an account you care about, as YouTube bans automated
access. `ytdlp_impersonate` presents a browser's TLS fingerprint (for
example `"chrome"`).
To check the JavaScript runtime `yt-dlp` needs for YouTube is present, run
`docker compose exec songtrace yt-dlp -v --simulate <url>`; the `JS runtimes:`
line must not say `none`. A failed download keeps its working folder, so
running the job again resumes it.
## Command line
The web UI is the main way to use songtrace, but every song can also be made
from the command line, in the container or on a host with the requirements
above and `ffmpeg`:
```sh ```sh
pip install -e . docker compose run --rm songtrace songtrace make \
songtrace make --help --title "Song Title" --artist "Artist Name" \
--lyrics /data/lyrics.txt --youtube https://youtu.be/...
``` ```
``` ```
@@ -179,99 +198,24 @@ songtrace make --title TITLE --artist ARTIST --lyrics LYRICS_FILE \
[--bpm BPM] [--force-from STAGE] [--config CONFIG] [--bpm BPM] [--force-from STAGE] [--config CONFIG]
``` ```
Run the web UI instead with `songtrace serve [--config CONFIG]`. `--force-from <stage>` redoes that stage and everything after it, reusing the
cached output of the stages before. On a bare-metal host, install with
`pip install -e .` and run the web UI with `songtrace serve`.
## Re-running a stage ## Development
Each stage caches its output under `work_dir`. If a later stage needs
another pass (e.g. you want to retry alignment with different lyrics, or
pitch tracking looks off), re-run with `--force-from <stage>` to redo that
stage and everything after it, reusing the cached output of everything
before it:
```sh
songtrace make --title "..." --artist "..." --lyrics lyrics.txt \
--youtube https://youtu.be/... --force-from align
```
Valid stages, in pipeline order: `acquire`, `separate`, `transcribe`,
`align`, `pitch`, `notes`, `grid`, `package`.
In the web UI this is what selecting a queue entry does: its title, artist,
lyrics and the rest come back up in the form, so you can adjust them and hit
REBUILD SONG. Only the stages your edit actually invalidates are rerun —
fixing a line of lyrics realigns without re-downloading or re-separating.
CREATE SONG, at the top of the queue, clears the page to start a fresh one.
That planning works off the fields you changed, which cannot see a change in
songtrace itself: after an upgrade, a rebuild of an older song finds every
stage cached and does nothing. **Restart from** on the rebuild form is the way
to redo a stage anyway — pick `package` to repackage without re-downloading.
It can only widen what is rerun, so an edit that invalidates an earlier stage
still wins.
**REBUILD ALL FROM…**, under CREATE SONG, does the same for every song at once:
pick the stage an update improved and each song is rebuilt from there with the
inputs it has now. Every song starts ticked; untick any already rebuilt by hand
(each shows when it was last built). Songs still in progress are skipped, as
are songs whose uploaded video is gone when the stage is `acquire`.
## YouTube download failures
`HTTP Error 403: Forbidden` from the acquire stage means YouTube served the
page but refused the media URL. In rough order of likelihood:
1. **yt-dlp is stale.** YouTube changes its defences every few weeks, so
songtrace keeps yt-dlp current itself. Once a day, and again whenever a
download fails outright, it checks PyPI for a newer stable release,
installs it under `$XDG_CACHE_HOME/songtrace/` (`/data/models/cache` in the
container), and tries it before the copy installed with songtrace. That
copy is never modified and is always the fallback, so a release that breaks
compatibility costs a retry rather than the download. The job log names
the yt-dlp version behind each attempt.
`ytdlp_version` overrides this: `"bundled"` uses only the installed copy,
and a release number such as `"2026.08.19"` pins that release instead of
the newest.
2. **Missing JS runtime.** yt-dlp needs `yt-dlp-ejs` plus a JavaScript engine
to solve YouTube's player challenges; without them it silently falls back to
formats that 403. The `[default,curl-cffi,deno]` extras in `pyproject.toml`
cover this, for both the installed copy and the ones fetched at runtime.
Check with `docker compose exec songtrace yt-dlp -v --simulate <url>` — the debug
header must show a `JS runtimes:` line that is not `none`.
3. **The IP is rate-limited or flagged.** Datacenter and VPN ranges get
throttled hardest. Try from another network before changing config.
4. **A PO token is required.** If the log says so explicitly, run
[bgutil-ytdlp-pot-provider](https://github.com/Brainicism/bgutil-ytdlp-pot-provider)
as a sidecar, install its plugin into the image, and point
`ytdlp_pot_provider_url` at it.
`ytdlp_clients` retries the download against each listed player client in
turn, so a client that YouTube has broken is skipped automatically.
`ytdlp_cookies` (a Netscape cookie jar, for age-gated videos) and
`ytdlp_impersonate` (a browser TLS fingerprint, e.g. `"chrome"`) are the other
two levers. Do not use a cookie jar from an account you care about — YouTube
bans for automated access.
Failed downloads leave the work dir intact, so re-running the job resumes
rather than starting over.
## Regenerating the web UI screenshot
`scripts/screenshot_webui.py` seeds the real app with a fixed set of jobs
covering every status and screenshots it to `docs/images/webui.png`. It uses
`ffmpeg` to give the selected song a playable folder so the shot includes the
player; without `ffmpeg` on `PATH` it still runs, just without it.
```sh ```sh
pip install -e '.[dev]' pip install -e '.[dev]'
playwright install chromium pytest
python scripts/screenshot_webui.py
``` ```
`pytest` also runs the web UI's JavaScript tests in `tests/js` when Node is
installed. Pull requests run the full suite in CI.
`scripts/screenshot_webui.py` regenerates `docs/images/webui.png` from a
seeded copy of the app (`playwright install chromium` first). With `ffmpeg` on
`PATH`, the shot includes the player.
## License ## License
[AGPLv3](LICENSE) [AGPLv3](LICENSE)
config.example.toml
+3 -3
View File
@@ -1,11 +1,11 @@
whisper_url = "http://whisper.intranet:9000/v1" whisper_url = "http://whisper.example:9000/v1"
# whisper_api_key = "" # whisper_api_key = ""
# whisper_model = "whisper-1" # whisper_model = "whisper-1"
# whisper_api = "openai" # or "asr-webservice" for onerahmet/openai-whisper-asr-webservice # whisper_api = "openai" # or "asr-webservice" for onerahmet/openai-whisper-asr-webservice
# (set whisper_url without the /v1 suffix, e.g. "http://whisper.intranet:9000") # (set whisper_url without the /v1 suffix, e.g. "http://whisper.example:9000")
output_dir = "/data/output" output_dir = "/data/output"
work_dir = "/data/work" work_dir = "/data/work"
creator = "brad" # creator = "Your Name" # the #CREATOR header in every chart
# demucs_model = "htdemucs_ft" # demucs_model = "htdemucs_ft"
# crepe_model = "full" # crepe_model = "full"
# device = "cuda" # device = "cuda"
docs/images/webui.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 84 KiB

After

Width:  |  Height:  |  Size: 99 KiB

scripts/screenshot_webui.py
+30 -11
View File
@@ -15,11 +15,14 @@ import time
from pathlib import Path from pathlib import Path
import httpx import httpx
import numpy as np
import uvicorn import uvicorn
from playwright.sync_api import sync_playwright from playwright.sync_api import sync_playwright
from songtrace.config import Config from songtrace.config import Config
from songtrace.grid import to_ultrastar
from songtrace.pipeline import PipelineError, PipelineResult, song_slug from songtrace.pipeline import PipelineError, PipelineResult, song_slug
from songtrace.ultrastar import serialize_ultrastar
from songtrace.web import create_app from songtrace.web import create_app
REPO_ROOT = Path(__file__).resolve().parent.parent REPO_ROOT = Path(__file__).resolve().parent.parent
@@ -128,13 +131,29 @@ def make_runner():
if "review" in extra: if "review" in extra:
work = cfg.work_dir / song_slug(inp) work = cfg.work_dir / song_slug(inp)
work.mkdir(parents=True, exist_ok=True) work.mkdir(parents=True, exist_ok=True)
# one note per word in the seeded lines, so each sings its word
notes = [ notes = [
{**n, "duration": n["end"] - n["start"], "text": "~", {**n, "duration": n["end"] - n["start"],
"line_index": line["index"]} "text": (" " if k else "") + w["text"], "line_index": line["index"]}
for line in extra["review"]["lines"] for n in line["notes"] for line in extra["review"]["lines"]
for k, (n, w) in enumerate(zip(line["notes"], line["words"]))
] ]
sung = [line for line in extra["review"]["lines"] if line["notes"]]
breaks = [(a["notes"][-1]["end"] + b["notes"][0]["start"]) / 2
for a, b in zip(sung, sung[1:])]
(work / "notes_sec.json").write_text(json.dumps({ (work / "notes_sec.json").write_text(json.dumps({
"notes": notes, "breaks": [], "warnings": extra["warnings"]})) "notes": notes, "breaks": breaks, "warnings": extra["warnings"]}))
# a chart for the beat grid, and a sung pitch with a little
# vibrato for the curve over the notes
(work / "song.txt").write_text(serialize_ultrastar(to_ultrastar(
notes, breaks, 240.0, [("TITLE", inp.title)])), encoding="utf-8")
times = np.arange(0, 60, 0.01)
f0, conf = np.zeros_like(times), np.zeros_like(times)
for n in notes:
m = (times >= n["start"] + 0.02) & (times < n["start"] + n["duration"] - 0.02)
semis = n["pitch"] + 0.2 * np.sin(2 * np.pi * 5.5 * times[m])
f0[m], conf[m] = 440 * 2 ** ((semis - 9) / 12), 0.9
np.savez(work / "pitch.npz", times=times, f0=f0, conf=conf)
(work / "align_meta.json").write_text(json.dumps({ (work / "align_meta.json").write_text(json.dumps({
"dropped_lines": [line["index"] for line in extra["review"]["lines"] "dropped_lines": [line["index"] for line in extra["review"]["lines"]
if line["dropped"]], if line["dropped"]],
@@ -230,13 +249,13 @@ def capture(base_url: str, with_player: bool) -> None:
# current word colouring rather than an untouched transport # current word colouring rather than an untouched transport
page.evaluate("document.getElementById('player-media').currentTime = 11.6") page.evaluate("document.getElementById('player-media').currentTime = 11.6")
page.wait_for_timeout(400) page.wait_for_timeout(400)
# scrolled to the review: the player pins to the top of #main and page.wait_for_selector(".sheet-curve polyline")
# the tracked lines sit under it, which is the panel in use # scrolled to the review, the panel in use, with the note sheet whole
page.evaluate("""() => { page.evaluate("""() => {
const main = document.getElementById('main'); const main = document.getElementById('main');
main.scrollTo(0, document.getElementById('review').offsetTop - 12); main.scrollTo(0, document.getElementById('review').offsetTop - 12);
}""") }""")
page.wait_for_timeout(400) page.wait_for_timeout(400)
OUT_IMAGE.parent.mkdir(parents=True, exist_ok=True) OUT_IMAGE.parent.mkdir(parents=True, exist_ok=True)
page.screenshot(path=str(OUT_IMAGE)) page.screenshot(path=str(OUT_IMAGE))
browser.close() browser.close()
src/songtrace/static/sheet.js
+21 -7
View File
@@ -10,6 +10,7 @@ const BLACK = new Set([1, 3, 6, 8, 10]);
const GUTTER = 46; // px of note names left of the grid const GUTTER = 46; // px of note names left of the grid
const PLOT_HEIGHT = 300; // px; rows divide this between the pitches shown const PLOT_HEIGHT = 300; // px; rows divide this between the pitches shown
const CONTEXT = 0.75; // seconds shown either side of the line const CONTEXT = 0.75; // seconds shown either side of the line
const BREAK_REACH = 3; // seconds past the line the page grows to show a break
const EDGE = 6; // px from a note's end that grabs it for resizing const EDGE = 6; // px from a note's end that grabs it for resizing
const KINDS = { "*": "golden", F: "free", R: "rap", G: "golden-rap" }; const KINDS = { "*": "golden", F: "free", R: "rap", G: "golden-rap" };
const KEYS_OPEN = "songtrace.sheet-keys"; const KEYS_OPEN = "songtrace.sheet-keys";
@@ -298,11 +299,13 @@ export function createSheet(root, { seek, playLine, onSaved }) {
const span = ts.length ? { start: Math.min(...ts), end: Math.max(...ts) } : lineSpan(line); const span = ts.length ? { start: Math.min(...ts), end: Math.max(...ts) } : lineSpan(line);
const range = pitchRange(notes().map(n => n.pitch)); const range = pitchRange(notes().map(n => n.pitch));
const want = { t0: span.start - CONTEXT, t1: span.end + CONTEXT, ...range }; const want = { t0: span.start - CONTEXT, t1: span.end + CONTEXT, ...range };
// the breaks either side are part of the line's page, so they stay on it // the breaks either side are part of the line's page, so it grows to
// show them, but not across a long instrumental: one further away is
// pinned to the edge instead (see shownBreakTime)
for (const slot of breakSlots()) { for (const slot of breakSlots()) {
const t = breakTime(slot); const t = breakTime(slot);
want.t0 = Math.min(want.t0, t - 0.3); want.t0 = Math.min(want.t0, Math.max(t - 0.3, span.start - BREAK_REACH));
want.t1 = Math.max(want.t1, t + 0.3); want.t1 = Math.max(want.t1, Math.min(t + 0.3, span.end + BREAK_REACH));
} }
layout = reset || !layout ? want : { layout = reset || !layout ? want : {
t0: Math.min(layout.t0, want.t0), t1: Math.max(layout.t1, want.t1), t0: Math.min(layout.t0, want.t0), t1: Math.max(layout.t1, want.t1),
@@ -413,6 +416,15 @@ export function createSheet(root, { seek, playLine, onSaved }) {
return Math.min(Math.max(t, slot.lo), Math.max(slot.hi, slot.lo)); return Math.min(Math.max(t, slot.lo), Math.max(slot.hi, slot.lo));
} }
// where a break is drawn and grabbed: on the page, pinned to its edge
// when it lies beyond it
function shownBreakTime(slot) {
const edge = (layout.t1 - layout.t0) * 0.003; // a few pixels in
const t = breakTime(slot);
const shown = Math.min(Math.max(t, layout.t0 + edge), layout.t1 - edge);
return { t: shown, pinned: shown !== t ? (t < shown ? "left" : "right") : null };
}
const undo = () => { if (hist?.canUndo) { hist.undo(); fit(false); render(); } }; const undo = () => { if (hist?.canUndo) { hist.undo(); fit(false); render(); } };
const redo = () => { if (hist?.canRedo) { hist.redo(); fit(false); render(); } }; const redo = () => { if (hist?.canRedo) { hist.redo(); fit(false); render(); } };
const one = () => (selected.size === 1 ? [...selected][0] : null); const one = () => (selected.size === 1 ? [...selected][0] : null);
@@ -617,14 +629,16 @@ export function createSheet(root, { seek, playLine, onSaved }) {
const marks = el("g", { class: "sheet-breaks" }, svg); const marks = el("g", { class: "sheet-breaks" }, svg);
for (const slot of breakSlots()) { for (const slot of breakSlots()) {
const t = breakTime(slot); const { t, pinned } = shownBreakTime(slot);
const value = hist.present[slot.which]; const value = hist.present[slot.which];
const byHand = value !== null && const byHand = value !== null &&
(value !== baseline[slot.which] || review.breaks?.find(b => b.line === slot.line)?.moved); (value !== baseline[slot.which] || review.breaks?.find(b => b.line === slot.line)?.moved);
const cls = `sheet-break${byHand ? " sheet-break-moved" : ""}`; const cls = `sheet-break${byHand ? " sheet-break-moved" : ""}`;
el("line", { x1: x(t), x2: x(t), y1: 0, y2: PLOT_HEIGHT, class: cls }, marks); el("line", { x1: x(t), x2: x(t), y1: 0, y2: PLOT_HEIGHT, class: cls }, marks);
el("text", { x: x(t) + 3, y: 11, class: `${cls} sheet-break-label` }, marks) const label = el("text", { x: pinned === "right" ? x(t) - 3 : x(t) + 3, y: 11,
.textContent = "↵"; "text-anchor": pinned === "right" ? "end" : "start",
class: `${cls} sheet-break-label` }, marks);
label.textContent = pinned === "right" ? "↵ →" : pinned === "left" ? "← ↵" : "↵";
} }
syllables.replaceChildren(...notes().map(n => { syllables.replaceChildren(...notes().map(n => {
@@ -708,7 +722,7 @@ export function createSheet(root, { seek, playLine, onSaved }) {
// by geometry, not the event's target: a render replaces the elements // by geometry, not the event's target: a render replaces the elements
// under the pointer, and the topmost note drawn wins // under the pointer, and the topmost note drawn wins
if (canEdit()) { if (canEdit()) {
const brk = breakSlots().find(s => Math.abs(px - view.x(breakTime(s))) <= 5); const brk = breakSlots().find(s => Math.abs(px - view.x(shownBreakTime(s).t)) <= 5);
if (brk) return { px, py, note: null, brk, mode: "break" }; if (brk) return { px, py, note: null, brk, mode: "break" };
} }
const pitch = view.pitch(py); const pitch = view.pitch(py);