metadata: nightly full beet import re-autotags every album; a transient MB miss silently drops it from album_art_sync #270

New issue

Closed

opened 2026-09-18 18:04:21 -04:00 by mysticalsoap · 1 comment

mysticalsoap commented

2026-09-18 18:04:21 -04:00

Owner

What happens

album_art_sync.py starts every run with a whole-library beet import -q /music (beets/config.yaml, no incremental). That is not a no-op for albums already in the library: beets re-runs MusicBrainz autotagging on every album and replaces the existing items (-v shows Replacing item NNN: … per track; added and flexible attrs survive, the tagged fields come from the new match). Any run where a given album's MusicBrainz lookup comes back short (rate limit, timeout, fewer candidates) falls to quiet_fallback: asis, and the album's mb_releasegroupid / mb_albumid are wiped.

album_art_sync.py then treats the album as having no source, and because it already has a cover.jpg it silently pops the state entry (state.pop(album) in the no-source loop). No ntfy, no log line. The album stops being tracked against the archive until some later night re-matches it.

Evidence (2026-09-18)

Two sync runs ~40 min apart (forced recheck after #269, then the LONG SEASON override fix), each with its own full import:

run 1: 76 album(s) with a usable source, 75 covers rewritten
run 2: 74 album(s) with a usable source; state went 76 → 74 entries
dropped: A Tribe Called Quest/The Low End Theory, Daft Punk/Discovery, Jeff Buckley/Grace — beets showed rg= and mb_albumid= empty, data_source=MusicBrainz still set, added unchanged, track files untouched (only cover.jpg modified in 24h)
single-dir beet -v import -q on each afterwards matched at distance 0.00–0.01 and restored both ids (done, all three are back). So the misses were transient, and the nightly job re-rolls the dice for all ~106 albums every night.

Options (design call)

import: incremental: yes in beets/config.yaml — existing dirs are skipped, new albums still import. Retagged albums are already refreshed explicitly by music_retag.py via beet update path:…, so the nightly full re-autotag isn't what keeps them current. Cheapest, removes the flapping entirely.
Keep the full import but stop the silent drop: when an album that was matched in state loses its beets source, keep the entry (last known release-group) and report it through ntfy like newly_no_source.
Both.

Either way the state.pop path deserves at least a log line.

## What happens `album_art_sync.py` starts every run with a whole-library `beet import -q /music` (beets/config.yaml, no `incremental`). That is not a no-op for albums already in the library: beets re-runs MusicBrainz autotagging on every album and **replaces** the existing items (`-v` shows `Replacing item NNN: …` per track; `added` and flexible attrs survive, the tagged fields come from the new match). Any run where a given album's MusicBrainz lookup comes back short (rate limit, timeout, fewer candidates) falls to `quiet_fallback: asis`, and the album's `mb_releasegroupid` / `mb_albumid` are wiped. `album_art_sync.py` then treats the album as having no source, and because it already has a `cover.jpg` it silently pops the state entry (`state.pop(album)` in the no-source loop). No ntfy, no log line. The album stops being tracked against the archive until some later night re-matches it. ## Evidence (2026-09-18) Two sync runs ~40 min apart (forced recheck after #269, then the LONG SEASON override fix), each with its own full import: - run 1: `76 album(s) with a usable source`, 75 covers rewritten - run 2: `74 album(s) with a usable source`; state went 76 → 74 entries - dropped: `A Tribe Called Quest/The Low End Theory`, `Daft Punk/Discovery`, `Jeff Buckley/Grace` — beets showed `rg=` and `mb_albumid=` empty, `data_source=MusicBrainz` still set, `added` unchanged, track files untouched (only cover.jpg modified in 24h) - single-dir `beet -v import -q` on each afterwards matched at distance 0.00–0.01 and restored both ids (done, all three are back). So the misses were transient, and the nightly job re-rolls the dice for all ~106 albums every night. ## Options (design call) 1. `import: incremental: yes` in beets/config.yaml — existing dirs are skipped, new albums still import. Retagged albums are already refreshed explicitly by `music_retag.py` via `beet update path:…`, so the nightly full re-autotag isn't what keeps them current. Cheapest, removes the flapping entirely. 2. Keep the full import but stop the silent drop: when an album that was `matched` in state loses its beets source, keep the entry (last known release-group) and report it through ntfy like `newly_no_source`. 3. Both. Either way the `state.pop` path deserves at least a log line.

mysticalsoap added the

bug

label

2026-09-18 18:04:21 -04:00

mysticalsoap commented

2026-09-19 00:14:12 -04:00

Author

Owner

Proposed shape (discussed 2026-09-19): album art is static, so drop the interval recheck entirely and make re-fetching event-driven — source changed (already immediate) or cover.jpg missing (new nightly check, covers Lidarr rebuilding an album folder on a quality upgrade). Pair with option 1 (incremental: yes) so existing dirs aren't re-autotagged nightly. An edition swap stays under the same release group, and a real release-group change shows up as a beets re-match. Movie posters keep their schedule since Letterboxd art does change.

Proposed shape (discussed 2026-09-19): album art is static, so drop the interval recheck entirely and make re-fetching event-driven — source changed (already immediate) or cover.jpg missing (new nightly check, covers Lidarr rebuilding an album folder on a quality upgrade). Pair with option 1 (`incremental: yes`) so existing dirs aren't re-autotagged nightly. An edition swap stays under the same release group, and a real release-group change shows up as a beets re-match. Movie posters keep their schedule since Letterboxd art does change.

mysticalsoap referenced this issue from a pull request that will close it,

2026-09-19 01:46:34 -04:00

metadata: incremental beets import, event-driven album art re-fetch (#270) #272

mysticalsoap referenced this issue from a commit

2026-09-19 01:46:36 -04:00

metadata: beets import is incremental

mysticalsoap closed this issue

2026-09-19 12:54:18 -04:00

mysticalsoap referenced this issue from a commit

2026-09-19 12:54:19 -04:00

Merge pull request 'metadata: incremental beets import, event-driven album art re-fetch (#270)' (#272) from metadata/art-sync-event-driven into trunk