A CLI and minimal web UI that turns a song — title, artist, lyrics you supply, and a YouTube URL or local video file — into a playable UltraStar-format song folder. Also compatible with Melody Mania.

The pipeline:

acquire — download audio/video via yt-dlp (or use a local video file)
separate — split vocals/instrumental with Demucs
transcribe — send the vocal stem to your own intranet Whisper server (OpenAI-compatible, or onerahmet/openai-whisper-asr-webservice), as a sanity check that the supplied lyrics match the audio and to locate vocal sections
align — forced-align the supplied reference lyrics against the vocal stem with wav2vec2 (torchaudio's MMS_FA) to get word timings
pitch — track pitch on the vocal stem with torchcrepe
notes — turn word timings + pitch into UltraStar notes
grid — quantise notes to a beat grid
package — write the final song folder (.txt + .mp3 + video + cover)

The shipped .mp3 is the full original mix (vocals included) — the separated stems are used for analysis only and are not included in the output.

The output is a best-effort draft, not a finished song. Every run ends by printing the low-confidence regions it found (e.g. 3 low-confidence notes near 0:42–0:47) — open the result in an UltraStar editor such as Yass or USDX and touch those spots up before playing.

Configuration

Config is TOML. Resolution order:

--config <path> passed explicitly on the command line
the SONGTRACE_CONFIG environment variable
~/.config/songtrace/config.toml

Copy the example and fill in your Whisper server:

cp config.example.toml config.toml
$EDITOR config.toml   # set whisper_url at minimum

See config.example.toml for every available field (whisper_url, whisper_api_key, whisper_model, whisper_api, output_dir, work_dir, creator, demucs_model, crepe_model, device, bind_host, port).

whisper_api selects the wire format: "openai" (default) speaks the OpenAI-compatible POST {whisper_url}/audio/transcriptions API; "asr-webservice" speaks onerahmet/openai-whisper-asr-webservice's POST {whisper_url}/asr API instead (point whisper_url at that server's base URL with no /v1 suffix — whisper_model/whisper_api_key are unused for this backend, since the model is fixed by how that server was started).

Docker

cp config.example.toml config.toml
$EDITOR config.toml
docker compose up -d

The web UI is then at http://localhost:8000. Downloaded model weights (Demucs, wav2vec2, ...) are cached under ./data on the host so they survive container recreation.

Run the CLI inside the container instead of the web UI:

docker compose run --rm songtrace songtrace make \
  --title "Song Title" --artist "Artist Name" \
  --lyrics /data/lyrics.txt --youtube https://youtu.be/...

GPU passthrough requires the NVIDIA Container Toolkit installed on the host; docker-compose.yml requests one GPU via the nvidia device driver.

Bare-metal

pip install -e .
songtrace make --help

songtrace make --title TITLE --artist ARTIST --lyrics LYRICS_FILE \
  (--youtube URL | --video PATH) \
  [--genre GENRE] [--year YEAR] [--language LANGUAGE] [--cover COVER_IMAGE] \
  [--bpm BPM] [--force-from STAGE] [--config CONFIG]

Run the web UI instead with songtrace serve [--config CONFIG].

Re-running a stage

Each stage caches its output under work_dir. If a later stage needs another pass (e.g. you want to retry alignment with different lyrics, or pitch tracking looks off), re-run with --force-from <stage> to redo that stage and everything after it, reusing the cached output of everything before it:

songtrace make --title "..." --artist "..." --lyrics lyrics.txt \
  --youtube https://youtu.be/... --force-from align

Valid stages, in pipeline order: acquire, separate, transcribe, align, pitch, notes, grid, package.

Regenerating the web UI screenshot

scripts/screenshot_webui.py seeds the real app with a fixed set of jobs covering every status and screenshots it to docs/images/webui.png:

pip install -e '.[dev]'
playwright install chromium
python scripts/screenshot_webui.py

License

AGPLv3

README.md Unescape Escape

songtrace