Milestone 4: yt-dlp provider and weighted source scheduler

Add a second playback source and a weighted scheduler mixing it with
Navidrome:

- Scheduler picks a provider by SOURCE_WEIGHTS, falling through to the
  others when one has nothing ready, so no source can stall playback.
- YtdlpProvider reads a hand-maintained config/urls.txt; container URLs
  (playlist/album/label/artist) are flat-extracted and one entry is
  drawn at random, honouring the anti-repeat window. Adds Track.source_url.
- YtdlpFetcher downloads bestaudio via the yt-dlp library, reusing the
  atomic hidden-temp-then-rename pattern; Liquidsoap decodes the result.
- Queue now dispatches to a fetcher registry keyed by backend.
- Sweep orphaned download temp files on daemon startup (leftovers from a
  killed container otherwise pile up and trip the stream fallback).

Verified end-to-end: yt-dlp opus decoded and served as 192 kbps MP3, and
the 3:1 default mix observed in play history.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
nemunaire 2026-07-02 17:58:24 +08:00
commit d1db6a11d8
13 changed files with 418 additions and 46 deletions

View file

@ -58,16 +58,27 @@ cp .env.example .env
If the Navidrome variables are left empty, the source is simply disabled and
the stream plays whatever is already in `cache/` (the milestone-1/2 behaviour).
For the yt-dlp source, list the URLs to draw from in `config/urls.txt` (copy
`config/urls.txt.example`). Each line is either a direct track URL or a
container URL (playlist, album, label, artist page) from which one track is
picked at random. The relative mix between sources is set by
`RADIEO_WEIGHT_NAVIDROME` / `RADIEO_WEIGHT_YTDLP` (a weight of 0 disables a
source); the file being absent also disables yt-dlp.
## Current status
**Milestone 3 — Navidrome provider: done.**
**Milestone 4 — yt-dlp provider: done.**
- `ingest` pulls tracks from an OpenSubsonic playlist (Navidrome), downloading
them into the shared cache ahead of playback (prefetch buffer).
- Two playback sources feed a weighted scheduler: a Navidrome/OpenSubsonic
playlist and a hand-maintained list of yt-dlp URLs (`config/urls.txt`).
Container URLs (playlist/album/label/artist) are expanded and one track is
drawn at random, honouring the anti-repeat window.
- Each source has its own fetcher (Subsonic stream / yt-dlp download); files are
cached ahead of playback (prefetch buffer) and decoded by Liquidsoap.
- Play history and LRU retention are tracked in a SQLite database under
`state/`: only the N most recently played files are kept on disk
(`RADIEO_RETENTION_KEEP`, default 20); anti-repeat avoids replaying a track
seen among the last plays.
(`RADIEO_RETENTION_KEEP`, default 20). Orphaned download temp files are swept
on startup.
- `GET /next` returns the next track as an annotated Liquidsoap URI with real
title/artist metadata (or an empty body when nothing is ready).
- `stream` (Liquidsoap v2.4.5) pulls via `request.dynamic` and falls back to the
@ -75,7 +86,7 @@ the stream plays whatever is already in `cache/` (the milestone-1/2 behaviour).
- HTTP stream served at `http://localhost:8000/radio.mp3` (MP3, 192 kbps),
multiple simultaneous listeners supported.
The yt-dlp and ListenBrainz sources come next.
The ListenBrainz suggestion feed comes next.
## Roadmap
@ -84,8 +95,8 @@ The yt-dlp and ListenBrainz sources come next.
switches to a `request.dynamic` source with the cache as fallback.
3. ✅ **Navidrome provider** — play from an OpenSubsonic playlist, with caching,
LRU retention and play history.
4. **yt-dlp provider** — fetch tracks from a maintained URL/artist list; weighted
mixing between sources.
4. **yt-dlp provider** — fetch tracks from a maintained URL/artist list;
weighted mixing between sources.
5. **Canonicalizer** — ListenBrainz MBID lookup for source-agnostic
de-duplication.
6. **ListenBrainz provider** — parse the RSS suggestions feed and resolve each