diff --git a/README.md b/README.md index 824b585..9e3e5b1 100644 --- a/README.md +++ b/README.md @@ -1,175 +1,194 @@ # songtrace -Turn a song and its lyrics into a playable UltraStar chart. - -A CLI and minimal web UI that turns a song — title, artist, lyrics you supply, -and a YouTube URL or local video file — into a playable -[UltraStar](https://usdx.eu/format/)-format song folder. Also compatible with -[Melody Mania](https://melodymania.org/). +Self-hosted karaoke chart builder for [UltraStar](https://usdx.eu/format/) and +compatible games such as [Melody Mania](https://melodymania.org/). Give it a +song's video and lyrics and it charts the notes, timing and pitch, ready for +your collection. Fix anything in the built-in editor; your edits survive when +you rebuild your whole collection after a songtrace update. ![Web UI](docs/images/webui.png) -The pipeline: +## What it does -1. **acquire** — download audio/video via `yt-dlp` (or use a local video file) -2. **separate** — split vocals/instrumental with Demucs -3. **transcribe** — send the vocal stem to your own intranet Whisper server - (OpenAI-compatible, or [onerahmet/openai-whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)), - as a sanity check that the supplied lyrics match the audio and to locate - vocal sections -4. **align** — forced-align the *supplied* reference lyrics against the vocal - stem with wav2vec2 (torchaudio's MMS_FA) to get word timings -5. **pitch** — track pitch on the vocal stem with torchcrepe -6. **notes** — turn word timings + pitch into UltraStar notes -7. **grid** — quantise notes to a beat grid -8. **package** — write the final song folder (`.txt` + `.mp3` + video + cover) +You give it a title, artist, the lyrics, and a YouTube URL or a video file. +songtrace writes a song folder (`.txt` chart, `.mp3`, video and cover) into +your collection: -The shipped `.mp3` is the full original mix (vocals included) — the -separated stems are used for analysis only and are not included in the -output. +1. **acquire**: downloads the video with `yt-dlp`, or takes your file +2. **separate**: splits vocals from the backing track (Demucs) +3. **transcribe**: has your Whisper server transcribe the vocals, to check + the lyrics match the recording and to find where the singing is +4. **align**: places each word of *your* lyrics on the vocals (wav2vec2) +5. **pitch**: tracks the sung pitch (torchcrepe) +6. **notes**: turns word timings and pitch into notes +7. **grid**: snaps the notes to a beat grid +8. **package**: writes the song folder -**The output is a best-effort draft**, not a finished song. Every run ends by -printing the low-confidence regions it found (e.g. -`3 low-confidence notes near 0:42–0:47`) — open the result in an UltraStar -editor such as [Yass](https://www.yass-along.com/) or -[USDX](https://usdx.eu/) and touch those spots up before playing. +The result is a strong draft, not a finished chart. The web UI flags the notes +it is least sure of, plays the song back against the chart, and lets you fix +what it got wrong before you sing it. -## Reviewing a song +## Requirements -The web UI plays the finished song folder back against the review, so you can -check the timing without loading it into a game first. Pick a song in the -queue and the review panel gets the packaged video with a transport above the -lyrics; as it plays, the line being sung is highlighted, words light up as -they are reached, and a playhead sweeps each line against its notes. Click any -line or word to play from there. +- **An NVIDIA GPU with CUDA.** songtrace is developed on a 12 GB card. + Separating the vocals (Demucs) alone takes about 3.7 GB, and alignment holds + a whole stretch of singing in memory at once, so a long song whose vocals + never pause needs the most. Cards under 12 GB are untested. Each stage gives + the GPU back when it finishes (see [Sharing the GPU](#sharing-the-gpu)). + `device = "cpu"` in the config runs without a GPU, but it is untested, far + slower, and needs plenty of system RAM. +- **Docker** with the + [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/), + so the container can see the GPU. +- **A Whisper server of your own.** songtrace does not include one. Either an + OpenAI-compatible API or + [whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice) + works. If it shares songtrace's GPU, it needs its own share of the memory. +- **Disk space:** a few GB of model weights, downloaded on first use, plus a + working folder per song holding its source video and separated tracks, + typically several hundred MB. -Playback is modelled on the game: the `#MP3` is the audio and the clock, and -the `#VIDEO` is muted background slaved to it. It also uses the same seconds -the notes were generated from, so a word that lights up late in the browser is -late in the game too. - -The packaged video has its audio track stripped for the same reason. It is the -same recording as the `#MP3`, so a player that does not mute the video plays -both at once, a few frames apart, which is heard as an echo. Songs built -before this change still have it — check with `ffprobe .mp4` for an -audio stream, and rebuild with **Restart from** set to `package` to clear it -without re-downloading. - -## Editing a song - -Above the lyrics, the note sheet shows one line at a time on a pitch-by-beat -grid, with the pitch songtrace detected drawn over the notes: where a note is -right the curve runs through it, so a wrong one stands out. Fix it there and -**SAVE LINE**; the song folder's chart is rewritten on the spot. - -Drag a note to move it and its ends to resize it; everything snaps to the -chart's beat grid and to semitones. Double-click empty space to add a note, -or a note to change its syllable. With the sheet focused: ←/→ move, ↑/↓ -change pitch (Shift for an octave), Shift+←/→ change length, Tab selects the -next note, S splits, M merges with the next note, G/F/R make a note golden, -freestyle or rap, Delete removes, Ctrl+Z undoes, Ctrl+S saves and P plays -the line. - -The ↵ markers either side of a line are its line breaks: drag one to move it -(it snaps to beats and stays between the lines it separates) and double-click -one to put it back where songtrace places it. Moving a break does not mark -either line's notes as edited, so they still follow a rebuild. - -The **GRID** bar above the notes sets the chart's BPM and GAP. Changes redraw -the grid as you type and are charted on **APPLY**; ×2 and ÷2 fix a tempo -detected at double or half speed, and **ALIGN TO NOTE** moves GAP the least -that puts the selected note on a beat. Notes keep their timing in seconds and -re-snap to the new grid. **RESET** returns to the BPM songtrace chose (your -BPM override, or the detected tempo) with GAP at the first note. - -Edits survive every rebuild, REBUILD ALL FROM… included: they are kept apart -from what the pipeline generates and laid back over it each time. **REVERT** -returns a line to the generated notes. Changing a line's lyrics sets its edit -aside rather than applying it to different words; the sheet lists those so -you can discard them. - -## Configuration - -Config is TOML. Resolution order: - -1. `--config ` passed explicitly on the command line -2. the `SONGTRACE_CONFIG` environment variable -3. `~/.config/songtrace/config.toml` - -Copy the example and fill in your Whisper server: +## Quick start (Docker) ```sh cp config.example.toml config.toml -$EDITOR config.toml # set whisper_url at minimum -``` - -See `config.example.toml` for every available field (`whisper_url`, -`whisper_api_key`, `whisper_model`, `whisper_api`, `output_dir`, `work_dir`, -`creator`, `demucs_model`, `crepe_model`, `device`, `bind_host`, `port`, -`ytdlp_clients`, `ytdlp_cookies`, `ytdlp_impersonate`, -`ytdlp_pot_provider_url`, `ytdlp_version`). - -`whisper_api` selects the wire format: `"openai"` (default) speaks the -OpenAI-compatible `POST {whisper_url}/audio/transcriptions` API; `"asr-webservice"` -speaks [onerahmet/openai-whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)'s -`POST {whisper_url}/asr` API instead (point `whisper_url` at that server's base -URL with no `/v1` suffix — `whisper_model`/`whisper_api_key` are unused for -this backend, since the model is fixed by how that server was started). - -## Docker - -```sh -cp config.example.toml config.toml -$EDITOR config.toml +$EDITOR config.toml # set whisper_url to your Whisper server docker compose up -d ``` -The web UI is then at `http://localhost:8000`. Downloaded model weights -(Demucs, wav2vec2, ...) are cached under `./data` on the host so they survive -container recreation. +Open `http://localhost:8000`, fill in the form and press **CREATE SONG**. -Run the CLI inside the container instead of the web UI: +Finished songs go to `output_dir` (`/data/output` in the container). To write +them straight into your game's song collection, mount that folder there in +`docker-compose.yml`: -```sh -docker compose run --rm songtrace songtrace make \ - --title "Song Title" --artist "Artist Name" \ - --lyrics /data/lyrics.txt --youtube https://youtu.be/... +```yaml + volumes: + - ./config.toml:/config/config.toml:ro + - ./data:/data + - /path/to/your/UltraStar/songs:/data/output ``` -GPU passthrough requires the [NVIDIA Container -Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/) -installed on the host; `docker-compose.yml` requests one GPU via the -`nvidia` device driver. +Everything else lives under `./data` on the host (model weights, and each +song's working files), so it survives recreating the container. + +## Reviewing and editing a song + +Pick a song in the queue. The review panel plays the packaged song with its +video, highlights the line and words being sung, and shows the chart on a note +sheet above the lyrics. Playback works like the game: the `#MP3` is the clock +and the video follows it muted, so a word that lights up late here is late in +the game too. + +The note sheet shows one line at a time on a pitch-by-beat grid, with the +pitch songtrace detected drawn over the notes. Where a note is right the curve +runs through it, so a wrong one stands out. Fix it there and **SAVE LINE**; the +song's chart is rewritten on the spot. + +- **Notes:** drag a note to move it, drag its ends to resize it. Everything + snaps to the beat grid and to semitones. Double-click empty space to add a + note, or a note to change its syllable. With the sheet focused, the arrow + keys move notes, S splits, M merges, G/F/R make a note golden, freestyle or + rap, Ctrl+Z undoes and Ctrl+S saves. Press **?** for every shortcut. +- **Line breaks:** the ↵ markers either side of a line. Drag one to move it, + and double-click it to hand it back to songtrace. +- **Tempo:** the **GRID** bar sets the chart's BPM and GAP. It redraws as you + type and charts on **APPLY**. ×2 and ÷2 fix a tempo detected at double or + half speed, and **ALIGN TO NOTE** puts the selected note on a beat. + **RESET** goes back to songtrace's choice. +- **Undoing an edit:** **REVERT** returns a line to the generated notes. + +Edits are kept apart from what the pipeline generates and laid back over it on +every rebuild, so they survive all of them, collection-wide ones included. If +you change a line's lyrics, its edit is set aside rather than applied to +different words, and the sheet lists those for you to discard. Songs with edits +are marked ✎ in the queue. + +## Rebuilding + +Select a song and its inputs come back up in the form; change them and press +**REBUILD SONG**. Only the stages your change affects are rerun: fixing a line +of lyrics realigns without downloading or separating again. + +A songtrace update can improve a stage without any of your inputs changing. +**Restart from** on the rebuild form redoes a stage anyway, and +**REBUILD ALL FROM…**, under CREATE SONG, does it for your whole collection: +pick the stage the update improved, untick any songs to leave alone, and each +song is rebuilt from there. Your edits are kept. + +Stages in order: `acquire`, `separate`, `transcribe`, `align`, `pitch`, +`notes`, `grid`, `package`. Rebuilding from `notes` onward needs no GPU. + +## Configuration + +`config.toml` is TOML; `config.example.toml` lists every setting. songtrace +reads, in order: `--config `, the `SONGTRACE_CONFIG` environment variable +(set to `/config/config.toml` in the container), then +`~/.config/songtrace/config.toml`. + +`whisper_api` chooses how to talk to Whisper: `"openai"` (the default) uses +`POST {whisper_url}/audio/transcriptions`, and `"asr-webservice"` uses +whisper-asr-webservice's `POST {whisper_url}/asr`. For the latter, give +`whisper_url` without a `/v1` suffix; `whisper_model` and `whisper_api_key` +are ignored, since that server's model is fixed when it starts. + +`creator` sets the `#CREATOR` header in every chart; it defaults to +`songtrace`. ## Sharing the GPU -`songtrace serve` is long-lived but only wants the GPU in bursts, so it hands -it back after every stage rather than holding it for the life of the process. -Three stages load a model — `separate` (Demucs), `align` (wav2vec2) and -`pitch` (torchcrepe) — and each one's weights are dropped as soon as that -stage finishes, followed by `torch.cuda.empty_cache()` to return the -allocator's freed blocks to the driver instead of keeping them for reuse. This -matters most between `separate` and `align`: `transcribe` sits between them -waiting on your Whisper server, which can be minutes of a GPU nothing is -using. - -Stages that gave memory back say so in the job log: +Three stages load a model: `separate` (Demucs), `align` (wav2vec2) and `pitch` +(torchcrepe). Each releases its model and returns the memory to the driver as +soon as it finishes, rather than holding it for the life of the server. This +matters most while `transcribe` waits on your Whisper server, which can take +minutes. The job log says how much each stage gave back: ``` [separate] released 3704 MiB of GPU memory ``` -The cost is reloading each model once per job instead of once per process, -which is seconds against stages that run for minutes. What this does not -release is the CUDA context itself, a few hundred MB that belongs to the -process until it exits; freeing that would mean running each stage in a -subprocess. If you need every last MB back, stop the container between runs. +Reloading the models costs seconds per song. The CUDA context itself, a few +hundred MB, stays with the process; stop the container to free it. -## Bare-metal +## YouTube download failures + +YouTube changes its defences every few weeks, so songtrace keeps `yt-dlp` +current by itself. Once a day, and again when a download fails, it checks PyPI +for a newer stable release, installs it under `/data/models/cache`, and tries +it before the copy it shipped with. That copy is never modified and is always +the fallback, so a broken release costs a retry, not the download. The job log +names the `yt-dlp` version behind each attempt. `ytdlp_version` pins a release, +or `"bundled"` turns the updates off. + +If downloads still fail with `HTTP Error 403: Forbidden`: + +1. **The network is rate-limited or flagged.** Datacenter and VPN ranges are + throttled hardest. Try another network before changing config. +2. **A PO token is required.** If the log says so, run + [bgutil-ytdlp-pot-provider](https://github.com/Brainicism/bgutil-ytdlp-pot-provider) + alongside songtrace, install its plugin into the image, and point + `ytdlp_pot_provider_url` at it. +3. **Other levers:** `ytdlp_clients` tries each listed YouTube player client + in turn. `ytdlp_cookies` takes a Netscape cookie jar for age-gated videos; + don't use one from an account you care about, as YouTube bans automated + access. `ytdlp_impersonate` presents a browser's TLS fingerprint (for + example `"chrome"`). + +To check the JavaScript runtime `yt-dlp` needs for YouTube is present, run +`docker compose exec songtrace yt-dlp -v --simulate `; the `JS runtimes:` +line must not say `none`. A failed download keeps its working folder, so +running the job again resumes it. + +## Command line + +The web UI is the main way to use songtrace, but every song can also be made +from the command line, in the container or on a host with the requirements +above and `ffmpeg`: ```sh -pip install -e . -songtrace make --help +docker compose run --rm songtrace songtrace make \ + --title "Song Title" --artist "Artist Name" \ + --lyrics /data/lyrics.txt --youtube https://youtu.be/... ``` ``` @@ -179,99 +198,24 @@ songtrace make --title TITLE --artist ARTIST --lyrics LYRICS_FILE \ [--bpm BPM] [--force-from STAGE] [--config CONFIG] ``` -Run the web UI instead with `songtrace serve [--config CONFIG]`. +`--force-from ` redoes that stage and everything after it, reusing the +cached output of the stages before. On a bare-metal host, install with +`pip install -e .` and run the web UI with `songtrace serve`. -## Re-running a stage - -Each stage caches its output under `work_dir`. If a later stage needs -another pass (e.g. you want to retry alignment with different lyrics, or -pitch tracking looks off), re-run with `--force-from ` to redo that -stage and everything after it, reusing the cached output of everything -before it: - -```sh -songtrace make --title "..." --artist "..." --lyrics lyrics.txt \ - --youtube https://youtu.be/... --force-from align -``` - -Valid stages, in pipeline order: `acquire`, `separate`, `transcribe`, -`align`, `pitch`, `notes`, `grid`, `package`. - -In the web UI this is what selecting a queue entry does: its title, artist, -lyrics and the rest come back up in the form, so you can adjust them and hit -REBUILD SONG. Only the stages your edit actually invalidates are rerun — -fixing a line of lyrics realigns without re-downloading or re-separating. -CREATE SONG, at the top of the queue, clears the page to start a fresh one. - -That planning works off the fields you changed, which cannot see a change in -songtrace itself: after an upgrade, a rebuild of an older song finds every -stage cached and does nothing. **Restart from** on the rebuild form is the way -to redo a stage anyway — pick `package` to repackage without re-downloading. -It can only widen what is rerun, so an edit that invalidates an earlier stage -still wins. - -**REBUILD ALL FROM…**, under CREATE SONG, does the same for every song at once: -pick the stage an update improved and each song is rebuilt from there with the -inputs it has now. Every song starts ticked; untick any already rebuilt by hand -(each shows when it was last built). Songs still in progress are skipped, as -are songs whose uploaded video is gone when the stage is `acquire`. - -## YouTube download failures - -`HTTP Error 403: Forbidden` from the acquire stage means YouTube served the -page but refused the media URL. In rough order of likelihood: - -1. **yt-dlp is stale.** YouTube changes its defences every few weeks, so - songtrace keeps yt-dlp current itself. Once a day, and again whenever a - download fails outright, it checks PyPI for a newer stable release, - installs it under `$XDG_CACHE_HOME/songtrace/` (`/data/models/cache` in the - container), and tries it before the copy installed with songtrace. That - copy is never modified and is always the fallback, so a release that breaks - compatibility costs a retry rather than the download. The job log names - the yt-dlp version behind each attempt. - - `ytdlp_version` overrides this: `"bundled"` uses only the installed copy, - and a release number such as `"2026.08.19"` pins that release instead of - the newest. - -2. **Missing JS runtime.** yt-dlp needs `yt-dlp-ejs` plus a JavaScript engine - to solve YouTube's player challenges; without them it silently falls back to - formats that 403. The `[default,curl-cffi,deno]` extras in `pyproject.toml` - cover this, for both the installed copy and the ones fetched at runtime. - Check with `docker compose exec songtrace yt-dlp -v --simulate ` — the debug - header must show a `JS runtimes:` line that is not `none`. - -3. **The IP is rate-limited or flagged.** Datacenter and VPN ranges get - throttled hardest. Try from another network before changing config. - -4. **A PO token is required.** If the log says so explicitly, run - [bgutil-ytdlp-pot-provider](https://github.com/Brainicism/bgutil-ytdlp-pot-provider) - as a sidecar, install its plugin into the image, and point - `ytdlp_pot_provider_url` at it. - -`ytdlp_clients` retries the download against each listed player client in -turn, so a client that YouTube has broken is skipped automatically. -`ytdlp_cookies` (a Netscape cookie jar, for age-gated videos) and -`ytdlp_impersonate` (a browser TLS fingerprint, e.g. `"chrome"`) are the other -two levers. Do not use a cookie jar from an account you care about — YouTube -bans for automated access. - -Failed downloads leave the work dir intact, so re-running the job resumes -rather than starting over. - -## Regenerating the web UI screenshot - -`scripts/screenshot_webui.py` seeds the real app with a fixed set of jobs -covering every status and screenshots it to `docs/images/webui.png`. It uses -`ffmpeg` to give the selected song a playable folder so the shot includes the -player; without `ffmpeg` on `PATH` it still runs, just without it. +## Development ```sh pip install -e '.[dev]' -playwright install chromium -python scripts/screenshot_webui.py +pytest ``` +`pytest` also runs the web UI's JavaScript tests in `tests/js` when Node is +installed. Pull requests run the full suite in CI. + +`scripts/screenshot_webui.py` regenerates `docs/images/webui.png` from a +seeded copy of the app (`playwright install chromium` first). With `ffmpeg` on +`PATH`, the shot includes the player. + ## License [AGPLv3](LICENSE) diff --git a/config.example.toml b/config.example.toml index 0bf83ea..15b3426 100644 --- a/config.example.toml +++ b/config.example.toml @@ -1,11 +1,11 @@ -whisper_url = "http://whisper.intranet:9000/v1" +whisper_url = "http://whisper.example:9000/v1" # whisper_api_key = "" # whisper_model = "whisper-1" # whisper_api = "openai" # or "asr-webservice" for onerahmet/openai-whisper-asr-webservice - # (set whisper_url without the /v1 suffix, e.g. "http://whisper.intranet:9000") + # (set whisper_url without the /v1 suffix, e.g. "http://whisper.example:9000") output_dir = "/data/output" work_dir = "/data/work" -creator = "brad" +# creator = "Your Name" # the #CREATOR header in every chart # demucs_model = "htdemucs_ft" # crepe_model = "full" # device = "cuda" diff --git a/docs/images/webui.png b/docs/images/webui.png index 7283fd6..f23ed87 100644 Binary files a/docs/images/webui.png and b/docs/images/webui.png differ diff --git a/scripts/screenshot_webui.py b/scripts/screenshot_webui.py index c364fa2..0eb2caf 100644 --- a/scripts/screenshot_webui.py +++ b/scripts/screenshot_webui.py @@ -15,11 +15,14 @@ import time from pathlib import Path import httpx +import numpy as np import uvicorn from playwright.sync_api import sync_playwright from songtrace.config import Config +from songtrace.grid import to_ultrastar from songtrace.pipeline import PipelineError, PipelineResult, song_slug +from songtrace.ultrastar import serialize_ultrastar from songtrace.web import create_app REPO_ROOT = Path(__file__).resolve().parent.parent @@ -128,13 +131,29 @@ def make_runner(): if "review" in extra: work = cfg.work_dir / song_slug(inp) work.mkdir(parents=True, exist_ok=True) + # one note per word in the seeded lines, so each sings its word notes = [ - {**n, "duration": n["end"] - n["start"], "text": "~", - "line_index": line["index"]} - for line in extra["review"]["lines"] for n in line["notes"] + {**n, "duration": n["end"] - n["start"], + "text": (" " if k else "") + w["text"], "line_index": line["index"]} + for line in extra["review"]["lines"] + for k, (n, w) in enumerate(zip(line["notes"], line["words"])) ] + sung = [line for line in extra["review"]["lines"] if line["notes"]] + breaks = [(a["notes"][-1]["end"] + b["notes"][0]["start"]) / 2 + for a, b in zip(sung, sung[1:])] (work / "notes_sec.json").write_text(json.dumps({ - "notes": notes, "breaks": [], "warnings": extra["warnings"]})) + "notes": notes, "breaks": breaks, "warnings": extra["warnings"]})) + # a chart for the beat grid, and a sung pitch with a little + # vibrato for the curve over the notes + (work / "song.txt").write_text(serialize_ultrastar(to_ultrastar( + notes, breaks, 240.0, [("TITLE", inp.title)])), encoding="utf-8") + times = np.arange(0, 60, 0.01) + f0, conf = np.zeros_like(times), np.zeros_like(times) + for n in notes: + m = (times >= n["start"] + 0.02) & (times < n["start"] + n["duration"] - 0.02) + semis = n["pitch"] + 0.2 * np.sin(2 * np.pi * 5.5 * times[m]) + f0[m], conf[m] = 440 * 2 ** ((semis - 9) / 12), 0.9 + np.savez(work / "pitch.npz", times=times, f0=f0, conf=conf) (work / "align_meta.json").write_text(json.dumps({ "dropped_lines": [line["index"] for line in extra["review"]["lines"] if line["dropped"]], @@ -230,13 +249,13 @@ def capture(base_url: str, with_player: bool) -> None: # current word colouring rather than an untouched transport page.evaluate("document.getElementById('player-media').currentTime = 11.6") page.wait_for_timeout(400) - # scrolled to the review: the player pins to the top of #main and - # the tracked lines sit under it, which is the panel in use - page.evaluate("""() => { - const main = document.getElementById('main'); - main.scrollTo(0, document.getElementById('review').offsetTop - 12); - }""") - page.wait_for_timeout(400) + page.wait_for_selector(".sheet-curve polyline") + # scrolled to the review, the panel in use, with the note sheet whole + page.evaluate("""() => { + const main = document.getElementById('main'); + main.scrollTo(0, document.getElementById('review').offsetTop - 12); + }""") + page.wait_for_timeout(400) OUT_IMAGE.parent.mkdir(parents=True, exist_ok=True) page.screenshot(path=str(OUT_IMAGE)) browser.close() diff --git a/src/songtrace/static/sheet.js b/src/songtrace/static/sheet.js index e31b192..39b28d7 100644 --- a/src/songtrace/static/sheet.js +++ b/src/songtrace/static/sheet.js @@ -10,6 +10,7 @@ const BLACK = new Set([1, 3, 6, 8, 10]); const GUTTER = 46; // px of note names left of the grid const PLOT_HEIGHT = 300; // px; rows divide this between the pitches shown const CONTEXT = 0.75; // seconds shown either side of the line +const BREAK_REACH = 3; // seconds past the line the page grows to show a break const EDGE = 6; // px from a note's end that grabs it for resizing const KINDS = { "*": "golden", F: "free", R: "rap", G: "golden-rap" }; const KEYS_OPEN = "songtrace.sheet-keys"; @@ -298,11 +299,13 @@ export function createSheet(root, { seek, playLine, onSaved }) { const span = ts.length ? { start: Math.min(...ts), end: Math.max(...ts) } : lineSpan(line); const range = pitchRange(notes().map(n => n.pitch)); const want = { t0: span.start - CONTEXT, t1: span.end + CONTEXT, ...range }; - // the breaks either side are part of the line's page, so they stay on it + // the breaks either side are part of the line's page, so it grows to + // show them, but not across a long instrumental: one further away is + // pinned to the edge instead (see shownBreakTime) for (const slot of breakSlots()) { const t = breakTime(slot); - want.t0 = Math.min(want.t0, t - 0.3); - want.t1 = Math.max(want.t1, t + 0.3); + want.t0 = Math.min(want.t0, Math.max(t - 0.3, span.start - BREAK_REACH)); + want.t1 = Math.max(want.t1, Math.min(t + 0.3, span.end + BREAK_REACH)); } layout = reset || !layout ? want : { t0: Math.min(layout.t0, want.t0), t1: Math.max(layout.t1, want.t1), @@ -413,6 +416,15 @@ export function createSheet(root, { seek, playLine, onSaved }) { return Math.min(Math.max(t, slot.lo), Math.max(slot.hi, slot.lo)); } + // where a break is drawn and grabbed: on the page, pinned to its edge + // when it lies beyond it + function shownBreakTime(slot) { + const edge = (layout.t1 - layout.t0) * 0.003; // a few pixels in + const t = breakTime(slot); + const shown = Math.min(Math.max(t, layout.t0 + edge), layout.t1 - edge); + return { t: shown, pinned: shown !== t ? (t < shown ? "left" : "right") : null }; + } + const undo = () => { if (hist?.canUndo) { hist.undo(); fit(false); render(); } }; const redo = () => { if (hist?.canRedo) { hist.redo(); fit(false); render(); } }; const one = () => (selected.size === 1 ? [...selected][0] : null); @@ -617,14 +629,16 @@ export function createSheet(root, { seek, playLine, onSaved }) { const marks = el("g", { class: "sheet-breaks" }, svg); for (const slot of breakSlots()) { - const t = breakTime(slot); + const { t, pinned } = shownBreakTime(slot); const value = hist.present[slot.which]; const byHand = value !== null && (value !== baseline[slot.which] || review.breaks?.find(b => b.line === slot.line)?.moved); const cls = `sheet-break${byHand ? " sheet-break-moved" : ""}`; el("line", { x1: x(t), x2: x(t), y1: 0, y2: PLOT_HEIGHT, class: cls }, marks); - el("text", { x: x(t) + 3, y: 11, class: `${cls} sheet-break-label` }, marks) - .textContent = "↵"; + const label = el("text", { x: pinned === "right" ? x(t) - 3 : x(t) + 3, y: 11, + "text-anchor": pinned === "right" ? "end" : "start", + class: `${cls} sheet-break-label` }, marks); + label.textContent = pinned === "right" ? "↵ →" : pinned === "left" ? "← ↵" : "↵"; } syllables.replaceChildren(...notes().map(n => { @@ -708,7 +722,7 @@ export function createSheet(root, { seek, playLine, onSaved }) { // by geometry, not the event's target: a render replaces the elements // under the pointer, and the topmost note drawn wins if (canEdit()) { - const brk = breakSlots().find(s => Math.abs(px - view.x(breakTime(s))) <= 5); + const brk = breakSlots().find(s => Math.abs(px - view.x(shownBreakTime(s).t)) <= 5); if (brk) return { px, py, note: null, brk, mode: "break" }; } const pitch = view.pitch(py);