Keep yt-dlp current at runtime, with the bundled copy as fallback #12

2026-10-09T14:36:12-07:00

dejima-agent commented

2026-10-09 14:36:12 -07:00

Summary generated by LLM agent

YouTube keeps breaking older yt-dlp releases, and until now the only fix was a new songtrace image. With this change songtrace keeps yt-dlp up to date on its own while it runs.

How it works

src/songtrace/ytdlp.py is now the only code that touches yt-dlp. It runs yt-dlp in a subprocess through its command line, which is yt-dlp's most stable interface. Old and new releases never share a process with songtrace.
Once a day, and again after a download fails with every copy, songtrace checks PyPI for a newer stable release. If there is one, it installs yt-dlp[default,curl-cffi,deno]==<version> into $XDG_CACHE_HOME/songtrace/yt-dlp-py311/<version> (/data/models/cache in the container) and tries it first.

Compatibility safeguards

The copy installed with songtrace is never modified and is always tried last, so a breaking release costs a retry, not the download.
A new install has to run and report the version requested before it is moved into the cache. Anything that fails that check is thrown away.
A usage error (exit status 2) is reported as "yt-dlp X rejected songtrace's options, so it may be incompatible".
Only stable releases are used (PyPI's info.version), never nightlies.
Installs are kept per Python version, so compiled wheels from an older image are never reused.
The job log names the yt-dlp version behind each attempt.

Override

ytdlp_version in config.toml accepts:

"auto": the default
"bundled": never fetch from PyPI
a release number such as "2026.08.19": pin that release, with the bundled copy still as the fallback

Testing

New tests/test_ytdlp.py runs real subprocesses against a fake yt_dlp package on PYTHONPATH. It covers the daily throttle, refresh after failure, pruning old releases, PyPI being unreachable, a failed post-install check, pins, and bundled.
tests/test_acquire.py covers falling back from the new release to the bundled one, and the retry after a refresh finds a newer release. A new test parses songtrace's full argument list with the real yt_dlp.parse_options.
Done by hand: pinned 2026.7.4. pip installed it into the cache (about 23s), and songtrace's argument set ran against it. Auto mode against live PyPI correctly installed nothing, because the latest release (2026.8.19) is the bundled one.
Not done: a real YouTube download, since YouTube is unreachable from the sandbox I worked in.

**Summary generated by LLM agent** YouTube keeps breaking older yt-dlp releases, and until now the only fix was a new songtrace image. With this change songtrace keeps yt-dlp up to date on its own while it runs. ## How it works - `src/songtrace/ytdlp.py` is now the only code that touches yt-dlp. It runs yt-dlp in a subprocess through its command line, which is yt-dlp's most stable interface. Old and new releases never share a process with songtrace. - Once a day, and again after a download fails with every copy, songtrace checks PyPI for a newer stable release. If there is one, it installs `yt-dlp[default,curl-cffi,deno]==<version>` into `$XDG_CACHE_HOME/songtrace/yt-dlp-py311/<version>` (`/data/models/cache` in the container) and tries it first. ## Compatibility safeguards - The copy installed with songtrace is never modified and is always tried last, so a breaking release costs a retry, not the download. - A new install has to run and report the version requested before it is moved into the cache. Anything that fails that check is thrown away. - A usage error (exit status 2) is reported as "yt-dlp X rejected songtrace's options, so it may be incompatible". - Only stable releases are used (PyPI's `info.version`), never nightlies. - Installs are kept per Python version, so compiled wheels from an older image are never reused. - The job log names the yt-dlp version behind each attempt. ## Override `ytdlp_version` in config.toml accepts: - `"auto"`: the default - `"bundled"`: never fetch from PyPI - a release number such as `"2026.08.19"`: pin that release, with the bundled copy still as the fallback ## Testing - New `tests/test_ytdlp.py` runs real subprocesses against a fake `yt_dlp` package on `PYTHONPATH`. It covers the daily throttle, refresh after failure, pruning old releases, PyPI being unreachable, a failed post-install check, pins, and `bundled`. - `tests/test_acquire.py` covers falling back from the new release to the bundled one, and the retry after a refresh finds a newer release. A new test parses songtrace's full argument list with the real `yt_dlp.parse_options`. - Done by hand: pinned `2026.7.4`. pip installed it into the cache (about 23s), and songtrace's argument set ran against it. Auto mode against live PyPI correctly installed nothing, because the latest release (2026.8.19) is the bundled one. - Not done: a real YouTube download, since YouTube is unreachable from the sandbox I worked in.

dejima-agent added 1 commit 2026-10-09 14:36:13 -07:00

acquire: keep yt-dlp current at runtime, with the bundled copy as fallback da3817b2a4

YouTube breaks old yt-dlp releases faster than songtrace is released.
songtrace now checks PyPI daily, and again after a download fails outright,
installs a newer stable release into its cache dir and tries it before the
bundled copy, which is never modified. yt-dlp runs in a subprocess through
its CLI, all from one module, so a cached release never mixes with the
imported one. ytdlp_version pins a release or turns updating off.

branderson approved these changes 2026-10-09 16:43:20 -07:00

branderson merged commit 62b6cd3e80 into main

2026-10-09 16:43:25 -07:00

branderson referenced this issue from a commit

2026-10-15 14:17:07 -07:00

Merge pull request 'Keep yt-dlp current at runtime, with the bundled copy as fallback' (#12) from ytdlp-runtime into main

dejima-agent referenced this issue from a commit

2026-10-15 17:44:36 -07:00

Merge pull request 'Keep yt-dlp current at runtime, with the bundled copy as fallback' (#12) from ytdlp-runtime into main

Sign in to join this conversation.