README: lead with requirements and Docker, refresh for the editor
The README described a CLI with a minimal web UI and told people to fix charts in Yass. Rewrite it around how songtrace is used now: what it does, its requirements up front (an NVIDIA GPU, tested on 12 GB, Docker with the NVIDIA Container Toolkit, a Whisper server of your own, disk for models and per-song files), a Docker quick start including writing songs straight into a game's collection, then reviewing, editing and rebuilding in the web UI. The command line moves near the end. Regenerate the screenshot to show the note sheet: the screenshot script now seeds the featured song with syllables, a chart, a pitch track and a line break, and always scrolls to the review. The example config no longer sets a real person as #CREATOR, and uses a reserved example host for Whisper.
This commit is contained in:
@@ -1,175 +1,194 @@
|
||||
# songtrace
|
||||
|
||||
Turn a song and its lyrics into a playable UltraStar chart.
|
||||
|
||||
A CLI and minimal web UI that turns a song — title, artist, lyrics you supply,
|
||||
and a YouTube URL or local video file — into a playable
|
||||
[UltraStar](https://usdx.eu/format/)-format song folder. Also compatible with
|
||||
[Melody Mania](https://melodymania.org/).
|
||||
Self-hosted karaoke chart builder for [UltraStar](https://usdx.eu/format/) and
|
||||
compatible games such as [Melody Mania](https://melodymania.org/). Give it a
|
||||
song's video and lyrics and it charts the notes, timing and pitch, ready for
|
||||
your collection. Fix anything in the built-in editor; your edits survive when
|
||||
you rebuild your whole collection after a songtrace update.
|
||||
|
||||

|
||||
|
||||
The pipeline:
|
||||
## What it does
|
||||
|
||||
1. **acquire** — download audio/video via `yt-dlp` (or use a local video file)
|
||||
2. **separate** — split vocals/instrumental with Demucs
|
||||
3. **transcribe** — send the vocal stem to your own intranet Whisper server
|
||||
(OpenAI-compatible, or [onerahmet/openai-whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)),
|
||||
as a sanity check that the supplied lyrics match the audio and to locate
|
||||
vocal sections
|
||||
4. **align** — forced-align the *supplied* reference lyrics against the vocal
|
||||
stem with wav2vec2 (torchaudio's MMS_FA) to get word timings
|
||||
5. **pitch** — track pitch on the vocal stem with torchcrepe
|
||||
6. **notes** — turn word timings + pitch into UltraStar notes
|
||||
7. **grid** — quantise notes to a beat grid
|
||||
8. **package** — write the final song folder (`.txt` + `.mp3` + video + cover)
|
||||
You give it a title, artist, the lyrics, and a YouTube URL or a video file.
|
||||
songtrace writes a song folder (`.txt` chart, `.mp3`, video and cover) into
|
||||
your collection:
|
||||
|
||||
The shipped `.mp3` is the full original mix (vocals included) — the
|
||||
separated stems are used for analysis only and are not included in the
|
||||
output.
|
||||
1. **acquire**: downloads the video with `yt-dlp`, or takes your file
|
||||
2. **separate**: splits vocals from the backing track (Demucs)
|
||||
3. **transcribe**: has your Whisper server transcribe the vocals, to check
|
||||
the lyrics match the recording and to find where the singing is
|
||||
4. **align**: places each word of *your* lyrics on the vocals (wav2vec2)
|
||||
5. **pitch**: tracks the sung pitch (torchcrepe)
|
||||
6. **notes**: turns word timings and pitch into notes
|
||||
7. **grid**: snaps the notes to a beat grid
|
||||
8. **package**: writes the song folder
|
||||
|
||||
**The output is a best-effort draft**, not a finished song. Every run ends by
|
||||
printing the low-confidence regions it found (e.g.
|
||||
`3 low-confidence notes near 0:42–0:47`) — open the result in an UltraStar
|
||||
editor such as [Yass](https://www.yass-along.com/) or
|
||||
[USDX](https://usdx.eu/) and touch those spots up before playing.
|
||||
The result is a strong draft, not a finished chart. The web UI flags the notes
|
||||
it is least sure of, plays the song back against the chart, and lets you fix
|
||||
what it got wrong before you sing it.
|
||||
|
||||
## Reviewing a song
|
||||
## Requirements
|
||||
|
||||
The web UI plays the finished song folder back against the review, so you can
|
||||
check the timing without loading it into a game first. Pick a song in the
|
||||
queue and the review panel gets the packaged video with a transport above the
|
||||
lyrics; as it plays, the line being sung is highlighted, words light up as
|
||||
they are reached, and a playhead sweeps each line against its notes. Click any
|
||||
line or word to play from there.
|
||||
- **An NVIDIA GPU with CUDA.** songtrace is developed on a 12 GB card.
|
||||
Separating the vocals (Demucs) alone takes about 3.7 GB, and alignment holds
|
||||
a whole stretch of singing in memory at once, so a long song whose vocals
|
||||
never pause needs the most. Cards under 12 GB are untested. Each stage gives
|
||||
the GPU back when it finishes (see [Sharing the GPU](#sharing-the-gpu)).
|
||||
`device = "cpu"` in the config runs without a GPU, but it is untested, far
|
||||
slower, and needs plenty of system RAM.
|
||||
- **Docker** with the
|
||||
[NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/),
|
||||
so the container can see the GPU.
|
||||
- **A Whisper server of your own.** songtrace does not include one. Either an
|
||||
OpenAI-compatible API or
|
||||
[whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)
|
||||
works. If it shares songtrace's GPU, it needs its own share of the memory.
|
||||
- **Disk space:** a few GB of model weights, downloaded on first use, plus a
|
||||
working folder per song holding its source video and separated tracks,
|
||||
typically several hundred MB.
|
||||
|
||||
Playback is modelled on the game: the `#MP3` is the audio and the clock, and
|
||||
the `#VIDEO` is muted background slaved to it. It also uses the same seconds
|
||||
the notes were generated from, so a word that lights up late in the browser is
|
||||
late in the game too.
|
||||
|
||||
The packaged video has its audio track stripped for the same reason. It is the
|
||||
same recording as the `#MP3`, so a player that does not mute the video plays
|
||||
both at once, a few frames apart, which is heard as an echo. Songs built
|
||||
before this change still have it — check with `ffprobe <song>.mp4` for an
|
||||
audio stream, and rebuild with **Restart from** set to `package` to clear it
|
||||
without re-downloading.
|
||||
|
||||
## Editing a song
|
||||
|
||||
Above the lyrics, the note sheet shows one line at a time on a pitch-by-beat
|
||||
grid, with the pitch songtrace detected drawn over the notes: where a note is
|
||||
right the curve runs through it, so a wrong one stands out. Fix it there and
|
||||
**SAVE LINE**; the song folder's chart is rewritten on the spot.
|
||||
|
||||
Drag a note to move it and its ends to resize it; everything snaps to the
|
||||
chart's beat grid and to semitones. Double-click empty space to add a note,
|
||||
or a note to change its syllable. With the sheet focused: ←/→ move, ↑/↓
|
||||
change pitch (Shift for an octave), Shift+←/→ change length, Tab selects the
|
||||
next note, S splits, M merges with the next note, G/F/R make a note golden,
|
||||
freestyle or rap, Delete removes, Ctrl+Z undoes, Ctrl+S saves and P plays
|
||||
the line.
|
||||
|
||||
The ↵ markers either side of a line are its line breaks: drag one to move it
|
||||
(it snaps to beats and stays between the lines it separates) and double-click
|
||||
one to put it back where songtrace places it. Moving a break does not mark
|
||||
either line's notes as edited, so they still follow a rebuild.
|
||||
|
||||
The **GRID** bar above the notes sets the chart's BPM and GAP. Changes redraw
|
||||
the grid as you type and are charted on **APPLY**; ×2 and ÷2 fix a tempo
|
||||
detected at double or half speed, and **ALIGN TO NOTE** moves GAP the least
|
||||
that puts the selected note on a beat. Notes keep their timing in seconds and
|
||||
re-snap to the new grid. **RESET** returns to the BPM songtrace chose (your
|
||||
BPM override, or the detected tempo) with GAP at the first note.
|
||||
|
||||
Edits survive every rebuild, REBUILD ALL FROM… included: they are kept apart
|
||||
from what the pipeline generates and laid back over it each time. **REVERT**
|
||||
returns a line to the generated notes. Changing a line's lyrics sets its edit
|
||||
aside rather than applying it to different words; the sheet lists those so
|
||||
you can discard them.
|
||||
|
||||
## Configuration
|
||||
|
||||
Config is TOML. Resolution order:
|
||||
|
||||
1. `--config <path>` passed explicitly on the command line
|
||||
2. the `SONGTRACE_CONFIG` environment variable
|
||||
3. `~/.config/songtrace/config.toml`
|
||||
|
||||
Copy the example and fill in your Whisper server:
|
||||
## Quick start (Docker)
|
||||
|
||||
```sh
|
||||
cp config.example.toml config.toml
|
||||
$EDITOR config.toml # set whisper_url at minimum
|
||||
```
|
||||
|
||||
See `config.example.toml` for every available field (`whisper_url`,
|
||||
`whisper_api_key`, `whisper_model`, `whisper_api`, `output_dir`, `work_dir`,
|
||||
`creator`, `demucs_model`, `crepe_model`, `device`, `bind_host`, `port`,
|
||||
`ytdlp_clients`, `ytdlp_cookies`, `ytdlp_impersonate`,
|
||||
`ytdlp_pot_provider_url`, `ytdlp_version`).
|
||||
|
||||
`whisper_api` selects the wire format: `"openai"` (default) speaks the
|
||||
OpenAI-compatible `POST {whisper_url}/audio/transcriptions` API; `"asr-webservice"`
|
||||
speaks [onerahmet/openai-whisper-asr-webservice](https://github.com/ahmetoner/whisper-asr-webservice)'s
|
||||
`POST {whisper_url}/asr` API instead (point `whisper_url` at that server's base
|
||||
URL with no `/v1` suffix — `whisper_model`/`whisper_api_key` are unused for
|
||||
this backend, since the model is fixed by how that server was started).
|
||||
|
||||
## Docker
|
||||
|
||||
```sh
|
||||
cp config.example.toml config.toml
|
||||
$EDITOR config.toml
|
||||
$EDITOR config.toml # set whisper_url to your Whisper server
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
The web UI is then at `http://localhost:8000`. Downloaded model weights
|
||||
(Demucs, wav2vec2, ...) are cached under `./data` on the host so they survive
|
||||
container recreation.
|
||||
Open `http://localhost:8000`, fill in the form and press **CREATE SONG**.
|
||||
|
||||
Run the CLI inside the container instead of the web UI:
|
||||
Finished songs go to `output_dir` (`/data/output` in the container). To write
|
||||
them straight into your game's song collection, mount that folder there in
|
||||
`docker-compose.yml`:
|
||||
|
||||
```sh
|
||||
docker compose run --rm songtrace songtrace make \
|
||||
--title "Song Title" --artist "Artist Name" \
|
||||
--lyrics /data/lyrics.txt --youtube https://youtu.be/...
|
||||
```yaml
|
||||
volumes:
|
||||
- ./config.toml:/config/config.toml:ro
|
||||
- ./data:/data
|
||||
- /path/to/your/UltraStar/songs:/data/output
|
||||
```
|
||||
|
||||
GPU passthrough requires the [NVIDIA Container
|
||||
Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/)
|
||||
installed on the host; `docker-compose.yml` requests one GPU via the
|
||||
`nvidia` device driver.
|
||||
Everything else lives under `./data` on the host (model weights, and each
|
||||
song's working files), so it survives recreating the container.
|
||||
|
||||
## Reviewing and editing a song
|
||||
|
||||
Pick a song in the queue. The review panel plays the packaged song with its
|
||||
video, highlights the line and words being sung, and shows the chart on a note
|
||||
sheet above the lyrics. Playback works like the game: the `#MP3` is the clock
|
||||
and the video follows it muted, so a word that lights up late here is late in
|
||||
the game too.
|
||||
|
||||
The note sheet shows one line at a time on a pitch-by-beat grid, with the
|
||||
pitch songtrace detected drawn over the notes. Where a note is right the curve
|
||||
runs through it, so a wrong one stands out. Fix it there and **SAVE LINE**; the
|
||||
song's chart is rewritten on the spot.
|
||||
|
||||
- **Notes:** drag a note to move it, drag its ends to resize it. Everything
|
||||
snaps to the beat grid and to semitones. Double-click empty space to add a
|
||||
note, or a note to change its syllable. With the sheet focused, the arrow
|
||||
keys move notes, S splits, M merges, G/F/R make a note golden, freestyle or
|
||||
rap, Ctrl+Z undoes and Ctrl+S saves. Press **?** for every shortcut.
|
||||
- **Line breaks:** the ↵ markers either side of a line. Drag one to move it,
|
||||
and double-click it to hand it back to songtrace.
|
||||
- **Tempo:** the **GRID** bar sets the chart's BPM and GAP. It redraws as you
|
||||
type and charts on **APPLY**. ×2 and ÷2 fix a tempo detected at double or
|
||||
half speed, and **ALIGN TO NOTE** puts the selected note on a beat.
|
||||
**RESET** goes back to songtrace's choice.
|
||||
- **Undoing an edit:** **REVERT** returns a line to the generated notes.
|
||||
|
||||
Edits are kept apart from what the pipeline generates and laid back over it on
|
||||
every rebuild, so they survive all of them, collection-wide ones included. If
|
||||
you change a line's lyrics, its edit is set aside rather than applied to
|
||||
different words, and the sheet lists those for you to discard. Songs with edits
|
||||
are marked ✎ in the queue.
|
||||
|
||||
## Rebuilding
|
||||
|
||||
Select a song and its inputs come back up in the form; change them and press
|
||||
**REBUILD SONG**. Only the stages your change affects are rerun: fixing a line
|
||||
of lyrics realigns without downloading or separating again.
|
||||
|
||||
A songtrace update can improve a stage without any of your inputs changing.
|
||||
**Restart from** on the rebuild form redoes a stage anyway, and
|
||||
**REBUILD ALL FROM…**, under CREATE SONG, does it for your whole collection:
|
||||
pick the stage the update improved, untick any songs to leave alone, and each
|
||||
song is rebuilt from there. Your edits are kept.
|
||||
|
||||
Stages in order: `acquire`, `separate`, `transcribe`, `align`, `pitch`,
|
||||
`notes`, `grid`, `package`. Rebuilding from `notes` onward needs no GPU.
|
||||
|
||||
## Configuration
|
||||
|
||||
`config.toml` is TOML; `config.example.toml` lists every setting. songtrace
|
||||
reads, in order: `--config <path>`, the `SONGTRACE_CONFIG` environment variable
|
||||
(set to `/config/config.toml` in the container), then
|
||||
`~/.config/songtrace/config.toml`.
|
||||
|
||||
`whisper_api` chooses how to talk to Whisper: `"openai"` (the default) uses
|
||||
`POST {whisper_url}/audio/transcriptions`, and `"asr-webservice"` uses
|
||||
whisper-asr-webservice's `POST {whisper_url}/asr`. For the latter, give
|
||||
`whisper_url` without a `/v1` suffix; `whisper_model` and `whisper_api_key`
|
||||
are ignored, since that server's model is fixed when it starts.
|
||||
|
||||
`creator` sets the `#CREATOR` header in every chart; it defaults to
|
||||
`songtrace`.
|
||||
|
||||
## Sharing the GPU
|
||||
|
||||
`songtrace serve` is long-lived but only wants the GPU in bursts, so it hands
|
||||
it back after every stage rather than holding it for the life of the process.
|
||||
Three stages load a model — `separate` (Demucs), `align` (wav2vec2) and
|
||||
`pitch` (torchcrepe) — and each one's weights are dropped as soon as that
|
||||
stage finishes, followed by `torch.cuda.empty_cache()` to return the
|
||||
allocator's freed blocks to the driver instead of keeping them for reuse. This
|
||||
matters most between `separate` and `align`: `transcribe` sits between them
|
||||
waiting on your Whisper server, which can be minutes of a GPU nothing is
|
||||
using.
|
||||
|
||||
Stages that gave memory back say so in the job log:
|
||||
Three stages load a model: `separate` (Demucs), `align` (wav2vec2) and `pitch`
|
||||
(torchcrepe). Each releases its model and returns the memory to the driver as
|
||||
soon as it finishes, rather than holding it for the life of the server. This
|
||||
matters most while `transcribe` waits on your Whisper server, which can take
|
||||
minutes. The job log says how much each stage gave back:
|
||||
|
||||
```
|
||||
[separate] released 3704 MiB of GPU memory
|
||||
```
|
||||
|
||||
The cost is reloading each model once per job instead of once per process,
|
||||
which is seconds against stages that run for minutes. What this does not
|
||||
release is the CUDA context itself, a few hundred MB that belongs to the
|
||||
process until it exits; freeing that would mean running each stage in a
|
||||
subprocess. If you need every last MB back, stop the container between runs.
|
||||
Reloading the models costs seconds per song. The CUDA context itself, a few
|
||||
hundred MB, stays with the process; stop the container to free it.
|
||||
|
||||
## Bare-metal
|
||||
## YouTube download failures
|
||||
|
||||
YouTube changes its defences every few weeks, so songtrace keeps `yt-dlp`
|
||||
current by itself. Once a day, and again when a download fails, it checks PyPI
|
||||
for a newer stable release, installs it under `/data/models/cache`, and tries
|
||||
it before the copy it shipped with. That copy is never modified and is always
|
||||
the fallback, so a broken release costs a retry, not the download. The job log
|
||||
names the `yt-dlp` version behind each attempt. `ytdlp_version` pins a release,
|
||||
or `"bundled"` turns the updates off.
|
||||
|
||||
If downloads still fail with `HTTP Error 403: Forbidden`:
|
||||
|
||||
1. **The network is rate-limited or flagged.** Datacenter and VPN ranges are
|
||||
throttled hardest. Try another network before changing config.
|
||||
2. **A PO token is required.** If the log says so, run
|
||||
[bgutil-ytdlp-pot-provider](https://github.com/Brainicism/bgutil-ytdlp-pot-provider)
|
||||
alongside songtrace, install its plugin into the image, and point
|
||||
`ytdlp_pot_provider_url` at it.
|
||||
3. **Other levers:** `ytdlp_clients` tries each listed YouTube player client
|
||||
in turn. `ytdlp_cookies` takes a Netscape cookie jar for age-gated videos;
|
||||
don't use one from an account you care about, as YouTube bans automated
|
||||
access. `ytdlp_impersonate` presents a browser's TLS fingerprint (for
|
||||
example `"chrome"`).
|
||||
|
||||
To check the JavaScript runtime `yt-dlp` needs for YouTube is present, run
|
||||
`docker compose exec songtrace yt-dlp -v --simulate <url>`; the `JS runtimes:`
|
||||
line must not say `none`. A failed download keeps its working folder, so
|
||||
running the job again resumes it.
|
||||
|
||||
## Command line
|
||||
|
||||
The web UI is the main way to use songtrace, but every song can also be made
|
||||
from the command line, in the container or on a host with the requirements
|
||||
above and `ffmpeg`:
|
||||
|
||||
```sh
|
||||
pip install -e .
|
||||
songtrace make --help
|
||||
docker compose run --rm songtrace songtrace make \
|
||||
--title "Song Title" --artist "Artist Name" \
|
||||
--lyrics /data/lyrics.txt --youtube https://youtu.be/...
|
||||
```
|
||||
|
||||
```
|
||||
@@ -179,99 +198,24 @@ songtrace make --title TITLE --artist ARTIST --lyrics LYRICS_FILE \
|
||||
[--bpm BPM] [--force-from STAGE] [--config CONFIG]
|
||||
```
|
||||
|
||||
Run the web UI instead with `songtrace serve [--config CONFIG]`.
|
||||
`--force-from <stage>` redoes that stage and everything after it, reusing the
|
||||
cached output of the stages before. On a bare-metal host, install with
|
||||
`pip install -e .` and run the web UI with `songtrace serve`.
|
||||
|
||||
## Re-running a stage
|
||||
|
||||
Each stage caches its output under `work_dir`. If a later stage needs
|
||||
another pass (e.g. you want to retry alignment with different lyrics, or
|
||||
pitch tracking looks off), re-run with `--force-from <stage>` to redo that
|
||||
stage and everything after it, reusing the cached output of everything
|
||||
before it:
|
||||
|
||||
```sh
|
||||
songtrace make --title "..." --artist "..." --lyrics lyrics.txt \
|
||||
--youtube https://youtu.be/... --force-from align
|
||||
```
|
||||
|
||||
Valid stages, in pipeline order: `acquire`, `separate`, `transcribe`,
|
||||
`align`, `pitch`, `notes`, `grid`, `package`.
|
||||
|
||||
In the web UI this is what selecting a queue entry does: its title, artist,
|
||||
lyrics and the rest come back up in the form, so you can adjust them and hit
|
||||
REBUILD SONG. Only the stages your edit actually invalidates are rerun —
|
||||
fixing a line of lyrics realigns without re-downloading or re-separating.
|
||||
CREATE SONG, at the top of the queue, clears the page to start a fresh one.
|
||||
|
||||
That planning works off the fields you changed, which cannot see a change in
|
||||
songtrace itself: after an upgrade, a rebuild of an older song finds every
|
||||
stage cached and does nothing. **Restart from** on the rebuild form is the way
|
||||
to redo a stage anyway — pick `package` to repackage without re-downloading.
|
||||
It can only widen what is rerun, so an edit that invalidates an earlier stage
|
||||
still wins.
|
||||
|
||||
**REBUILD ALL FROM…**, under CREATE SONG, does the same for every song at once:
|
||||
pick the stage an update improved and each song is rebuilt from there with the
|
||||
inputs it has now. Every song starts ticked; untick any already rebuilt by hand
|
||||
(each shows when it was last built). Songs still in progress are skipped, as
|
||||
are songs whose uploaded video is gone when the stage is `acquire`.
|
||||
|
||||
## YouTube download failures
|
||||
|
||||
`HTTP Error 403: Forbidden` from the acquire stage means YouTube served the
|
||||
page but refused the media URL. In rough order of likelihood:
|
||||
|
||||
1. **yt-dlp is stale.** YouTube changes its defences every few weeks, so
|
||||
songtrace keeps yt-dlp current itself. Once a day, and again whenever a
|
||||
download fails outright, it checks PyPI for a newer stable release,
|
||||
installs it under `$XDG_CACHE_HOME/songtrace/` (`/data/models/cache` in the
|
||||
container), and tries it before the copy installed with songtrace. That
|
||||
copy is never modified and is always the fallback, so a release that breaks
|
||||
compatibility costs a retry rather than the download. The job log names
|
||||
the yt-dlp version behind each attempt.
|
||||
|
||||
`ytdlp_version` overrides this: `"bundled"` uses only the installed copy,
|
||||
and a release number such as `"2026.08.19"` pins that release instead of
|
||||
the newest.
|
||||
|
||||
2. **Missing JS runtime.** yt-dlp needs `yt-dlp-ejs` plus a JavaScript engine
|
||||
to solve YouTube's player challenges; without them it silently falls back to
|
||||
formats that 403. The `[default,curl-cffi,deno]` extras in `pyproject.toml`
|
||||
cover this, for both the installed copy and the ones fetched at runtime.
|
||||
Check with `docker compose exec songtrace yt-dlp -v --simulate <url>` — the debug
|
||||
header must show a `JS runtimes:` line that is not `none`.
|
||||
|
||||
3. **The IP is rate-limited or flagged.** Datacenter and VPN ranges get
|
||||
throttled hardest. Try from another network before changing config.
|
||||
|
||||
4. **A PO token is required.** If the log says so explicitly, run
|
||||
[bgutil-ytdlp-pot-provider](https://github.com/Brainicism/bgutil-ytdlp-pot-provider)
|
||||
as a sidecar, install its plugin into the image, and point
|
||||
`ytdlp_pot_provider_url` at it.
|
||||
|
||||
`ytdlp_clients` retries the download against each listed player client in
|
||||
turn, so a client that YouTube has broken is skipped automatically.
|
||||
`ytdlp_cookies` (a Netscape cookie jar, for age-gated videos) and
|
||||
`ytdlp_impersonate` (a browser TLS fingerprint, e.g. `"chrome"`) are the other
|
||||
two levers. Do not use a cookie jar from an account you care about — YouTube
|
||||
bans for automated access.
|
||||
|
||||
Failed downloads leave the work dir intact, so re-running the job resumes
|
||||
rather than starting over.
|
||||
|
||||
## Regenerating the web UI screenshot
|
||||
|
||||
`scripts/screenshot_webui.py` seeds the real app with a fixed set of jobs
|
||||
covering every status and screenshots it to `docs/images/webui.png`. It uses
|
||||
`ffmpeg` to give the selected song a playable folder so the shot includes the
|
||||
player; without `ffmpeg` on `PATH` it still runs, just without it.
|
||||
## Development
|
||||
|
||||
```sh
|
||||
pip install -e '.[dev]'
|
||||
playwright install chromium
|
||||
python scripts/screenshot_webui.py
|
||||
pytest
|
||||
```
|
||||
|
||||
`pytest` also runs the web UI's JavaScript tests in `tests/js` when Node is
|
||||
installed. Pull requests run the full suite in CI.
|
||||
|
||||
`scripts/screenshot_webui.py` regenerates `docs/images/webui.png` from a
|
||||
seeded copy of the app (`playwright install chromium` first). With `ffmpeg` on
|
||||
`PATH`, the shot includes the player.
|
||||
|
||||
## License
|
||||
|
||||
[AGPLv3](LICENSE)
|
||||
|
||||
+3
-3
@@ -1,11 +1,11 @@
|
||||
whisper_url = "http://whisper.intranet:9000/v1"
|
||||
whisper_url = "http://whisper.example:9000/v1"
|
||||
# whisper_api_key = ""
|
||||
# whisper_model = "whisper-1"
|
||||
# whisper_api = "openai" # or "asr-webservice" for onerahmet/openai-whisper-asr-webservice
|
||||
# (set whisper_url without the /v1 suffix, e.g. "http://whisper.intranet:9000")
|
||||
# (set whisper_url without the /v1 suffix, e.g. "http://whisper.example:9000")
|
||||
output_dir = "/data/output"
|
||||
work_dir = "/data/work"
|
||||
creator = "brad"
|
||||
# creator = "Your Name" # the #CREATOR header in every chart
|
||||
# demucs_model = "htdemucs_ft"
|
||||
# crepe_model = "full"
|
||||
# device = "cuda"
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 84 KiB After Width: | Height: | Size: 99 KiB |
+30
-11
@@ -15,11 +15,14 @@ import time
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import numpy as np
|
||||
import uvicorn
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
from songtrace.config import Config
|
||||
from songtrace.grid import to_ultrastar
|
||||
from songtrace.pipeline import PipelineError, PipelineResult, song_slug
|
||||
from songtrace.ultrastar import serialize_ultrastar
|
||||
from songtrace.web import create_app
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
@@ -128,13 +131,29 @@ def make_runner():
|
||||
if "review" in extra:
|
||||
work = cfg.work_dir / song_slug(inp)
|
||||
work.mkdir(parents=True, exist_ok=True)
|
||||
# one note per word in the seeded lines, so each sings its word
|
||||
notes = [
|
||||
{**n, "duration": n["end"] - n["start"], "text": "~",
|
||||
"line_index": line["index"]}
|
||||
for line in extra["review"]["lines"] for n in line["notes"]
|
||||
{**n, "duration": n["end"] - n["start"],
|
||||
"text": (" " if k else "") + w["text"], "line_index": line["index"]}
|
||||
for line in extra["review"]["lines"]
|
||||
for k, (n, w) in enumerate(zip(line["notes"], line["words"]))
|
||||
]
|
||||
sung = [line for line in extra["review"]["lines"] if line["notes"]]
|
||||
breaks = [(a["notes"][-1]["end"] + b["notes"][0]["start"]) / 2
|
||||
for a, b in zip(sung, sung[1:])]
|
||||
(work / "notes_sec.json").write_text(json.dumps({
|
||||
"notes": notes, "breaks": [], "warnings": extra["warnings"]}))
|
||||
"notes": notes, "breaks": breaks, "warnings": extra["warnings"]}))
|
||||
# a chart for the beat grid, and a sung pitch with a little
|
||||
# vibrato for the curve over the notes
|
||||
(work / "song.txt").write_text(serialize_ultrastar(to_ultrastar(
|
||||
notes, breaks, 240.0, [("TITLE", inp.title)])), encoding="utf-8")
|
||||
times = np.arange(0, 60, 0.01)
|
||||
f0, conf = np.zeros_like(times), np.zeros_like(times)
|
||||
for n in notes:
|
||||
m = (times >= n["start"] + 0.02) & (times < n["start"] + n["duration"] - 0.02)
|
||||
semis = n["pitch"] + 0.2 * np.sin(2 * np.pi * 5.5 * times[m])
|
||||
f0[m], conf[m] = 440 * 2 ** ((semis - 9) / 12), 0.9
|
||||
np.savez(work / "pitch.npz", times=times, f0=f0, conf=conf)
|
||||
(work / "align_meta.json").write_text(json.dumps({
|
||||
"dropped_lines": [line["index"] for line in extra["review"]["lines"]
|
||||
if line["dropped"]],
|
||||
@@ -230,13 +249,13 @@ def capture(base_url: str, with_player: bool) -> None:
|
||||
# current word colouring rather than an untouched transport
|
||||
page.evaluate("document.getElementById('player-media').currentTime = 11.6")
|
||||
page.wait_for_timeout(400)
|
||||
# scrolled to the review: the player pins to the top of #main and
|
||||
# the tracked lines sit under it, which is the panel in use
|
||||
page.evaluate("""() => {
|
||||
const main = document.getElementById('main');
|
||||
main.scrollTo(0, document.getElementById('review').offsetTop - 12);
|
||||
}""")
|
||||
page.wait_for_timeout(400)
|
||||
page.wait_for_selector(".sheet-curve polyline")
|
||||
# scrolled to the review, the panel in use, with the note sheet whole
|
||||
page.evaluate("""() => {
|
||||
const main = document.getElementById('main');
|
||||
main.scrollTo(0, document.getElementById('review').offsetTop - 12);
|
||||
}""")
|
||||
page.wait_for_timeout(400)
|
||||
OUT_IMAGE.parent.mkdir(parents=True, exist_ok=True)
|
||||
page.screenshot(path=str(OUT_IMAGE))
|
||||
browser.close()
|
||||
|
||||
Reference in New Issue
Block a user