Commit Graph
95 Commits
Author SHA1 Message Date
imSp4rky d239964f0b fix(serve): apply staged service reloads when sessions end, with a max staging age
A staged tag was only retried on the refresh tick or a job completion, so a service with steady remote sessions never swapped in. Session end, expiry and eviction now trigger a single-flight apply, the tick applies after the pull, and serve.services_staged_max_age swaps an overdue tag in even while busy.
2026-09-22 10:40:49 -06:00
imSp4rky 6e681af513 feat(manifests): extend merge_segments rolling merge to HLS and ISM
RollingMerge takes the segment path and an append hook. ISM synthesises the init from segment 0 and converts PIFF per fragment; HLS qualifies only with one init, one key and no discontinuity. Both parsers check the merge reached every segment, and resume is decided per track.
2026-09-21 20:53:37 -06:00
imSp4rky 79d109d548 feat(dash): add merge_segments to append segments during the download
Opt-in rolling merge for DASH video and audio: each contiguous segment is appended to the output as it lands, so the post-download "Merging" stage goes away. Decrypts the init segment eagerly when decrypt_segments is on, disables resume for the track, and skips subtitle tracks.
2026-09-21 19:13:39 -06:00
imSp4rky 9c9e059e2d fix(remote): forward nested service cache files to and from the server
Cache keys are now the path relative to the service cache dir without .json, so Cacher keys like session_web/<sha1> reach the remote session. The credential filter runs on every path segment, the suffix is appended rather than substituted, and both sides cap the map at 64 keys.
2026-09-19 21:42:01 -06:00
imSp4rky af866e0c95 fix(vaults): harden the KID window check and clear a stale flag on licence 2026-09-18 17:57:00 -06:00
imSp4rky b30e6d7cbe fix(vaults): verify vault content keys per KID from the fragment map
Map each KID to its fragments from the ciphertext moov/moof (tenc, seig, ISM duration clock) and decode up to three windows per KID that never cross into another KID, so a clear lead cannot hide a wrong key and only the failing KID is flagged. Decode every frame (wrong-key HEVC keyframes decode clean), re-judge mid-GOP starts on keyframes, time out every FFmpeg run, and refuse to store a flagged pair again in SQLite.
2026-09-18 01:10:14 -06:00
imSp4rky cf2906f120 fix(cli): make the click param types the one source of codec spellings
--acodec now lists its codecs in --help, --vcodec drops the duplicate entries, and the REST API derives its allowlists from the same types, so AC-4, DDP, VORB and sub_format "original" stop being rejected and H264 stops being silently dropped. Adds shell completion for --acodec and repairs MultipleChoice completion. Raises the click floor to 8.2.
2026-09-17 19:22:19 -06:00
imSp4rky aec45e30cc feat(service): add GEOBLOCK negative geofence
Regions in GEOBLOCK refuse the service and everything else is allowed. A hit stops the run with a region error instead of auto-proxying; the check also covers proxy exits and the remote-dl client region.
2026-09-16 15:06:43 -06:00
imSp4rky aa36d4c225 feat(vaults): report bad keys to the vault that served them 2026-09-13 20:46:38 -06:00
imSp4rky a7d6791136 feat(remote): verify server-CDM vault keys on the client and report bad pairs
The licence response lists KIDs a server vault supplied, the client proves them with the decode check, and a bad pair goes to the new session keys/bad route so the server flags its own row. Client vault keys go first over --remote, and the dashboard health warns when no SQLite vault can store flags.
2026-09-12 15:13:08 -06:00
imSp4rky 7c82282813 feat(vaults): flag and skip wrong content keys from poisoned vaults
Verify a vault-sourced key with an FFmpeg decode after decryption. On failure flag the KID:KEY locally with its source, restore the ciphertext, and retry through the remaining vaults and then the CDM.
2026-09-12 13:11:44 -06:00
imSp4rky 9b49299dd4 fix(config): keep no and yes as language tags 2026-09-11 20:05:46 -06:00
imSp4rky c8a674d85d fix(naming): correct language tag rule matching and add state conditions
Rules took only a scalar, so the documented list syntax raised LanguageTagError after the mux. Normalise every condition, treat an invalid tag as no match, guard the surrounding post-mux config reads, and let a rule match dual, multi and dubbed so a release can carry its own tag.
2026-09-11 20:00:08 -06:00
imSp4rky d860eb9e01 fix(proxy): check the proxy exit IP before any service request
A dead proxy, or one that exits from this machine's IP, now fails with INVALID_PROXY instead of retrying every request; remote session create also builds the service off the event loop so a stuck check cannot block the server.
2026-09-10 17:08:04 -06:00
imSp4rky 2971a3a7da fix(manifests): flag DASH audio using the standard description role 2026-09-08 21:31:46 -06:00
imSp4rky 76d82df25b fix(api): keep error URLs readable and mask host paths in dashboard logs 2026-09-08 17:37:32 -06:00
imSp4rky b8c78c1a82 fix(api): tell the service that the client sent a cookie jar
A service that picks its auth path in init could only check the server's cookies directory, so a client-sent jar was invisible and the service fell back to the wrong path. The ctx now carries cookies_supplied, and a malformed cookie blob answers 400 instead of 500.
2026-09-07 13:20:55 -06:00
imSp4rky 7065cbf911 fix(drm): license every PlayReady header until all KIDs are covered
Only the first WRM header was read and licensed, so a manifest carrying several headers silently lost content keys. Collect KIDs from every header and license each distinct one until coverage is complete.
2026-09-07 10:05:09 -06:00
imSp4rky 85503fd87c test(downloader): cover probe judging and the submission window
Assert that a probe is judged on the trailing tick, that a backoff is temporary and climbs back to the cap, and that a worker always finds its next segment staged. Documents adaptive-workers behaviour in the downloading guide.
2026-09-06 20:03:46 -06:00
imSp4rky 22cc739e83 fix(downloader): correct truncation, ranged sizing, and session carry
Check a sequential download against the complete length whenever one is known, and settle the unknown-total case on the 416. An empty body of unknown length, a 206 with no complete length, and a 206 starting past the resume offset all passed the size checks before and were renamed into place. Four restart paths spent no attempt, so a host that always answered the same way cycled forever.
2026-09-06 20:03:08 -06:00
imSp4rky dffb8b27e3 fix(serve): lock shared session state and throttle progress writes
Handlers share one service instance, so a per-session lock keeps two requests out of it at once. Track threads merge and write job state under one lock, and the progress sink coalesces ticks that say the same thing.
2026-09-06 20:01:38 -06:00
imSp4rky adea7583f0 test(services): use placeholder tags instead of real service names 2026-09-05 22:16:22 -06:00
imSp4rky 0f9997be4c refactor(drm): drop the hardcoded service branch in the license flow
One service needed session_id in its license call, which a near-duplicate of get_content_keys supplied behind a service-tag test. service_licence now passes each service only the arguments its own license function declares, so the duplicate and the branch are gone. The REST API license paths and the base PlayReady delegate get the same filtering, replacing their fixed argument set.

Two behaviour changes ride along: an empty license challenge now raises EmptyLicense for every service, not just the one, and a service that declares only challenge now receives a working call instead of a TypeError.
2026-09-05 22:14:47 -06:00
imSp4rky e845e9ba28 fix(services): name the build a compiled service is missing
A service shipped as .so/.pyd failed with a bare ModuleNotFoundError. Report the build this interpreter needs, with its version and platform.
2026-09-05 22:13:25 -06:00
imSp4rky 2f28aa8e64 fix(dl): end a cancelled track by raising instead of returning short 2026-09-05 12:52:02 -06:00
imSp4rky 235839abc8 feat(audio): support DTS:X Profile 2 with MP4 muxing
Route titles with a dtsx/dtsy sample entry to MP4 via MP4Box, since Matroska has no CodecID for DTS-UHD. Add per-rendition HLS audio codec selection, fix merge_segments progress and its cleanup path.
2026-09-04 14:19:29 -06:00
imSp4rky e15c2f067e perf(kv): push whole services to SQL vaults in one call 2026-09-03 13:30:39 -06:00
imSp4rky 8354235d56 feat(serve): add operator dashboard endpoints and per-key rate limits 2026-09-02 23:52:55 -06:00
imSp4rky e00f6cca86 fix(dash): number SegmentTimeline segments from startNumber
Timeline manifests with startNumber between 2 and the segment count capped the range at the count, truncating live replays.
2026-09-01 14:05:06 -06:00
imSp4rky 7310273594 fix(ism): signal Dolby Vision and Atmos from Smooth manifests
Init segments now carry dvcC/dvvC with a profile-keyed sample entry and the dby1 brand; track ids cover the manifest URL so per-range ladders stop colliding, and the remote path carries the Atmos flag without inventing a JOC count.
2026-09-01 12:50:59 -06:00
imSp4rky 5128fef473 feat(serve): hot-reload service repos on a timer without a restart
Add serve.services_refresh_interval; refresh pulls the repos and re-imports only the changed services, staging any with active jobs until they finish. Report skipped/broken services in the log and on /api/services.
2026-08-26 23:14:23 -06:00
imSp4rky 4964869258 feat(services): add services_repo_force and compact refresh output
Force-reset repo clones on the automatic TTL refresh when enabled. Report refresh changes as one +/~/-/!TAG line per repo.
2026-08-26 22:15:39 -06:00
imSp4rky 0a68cc2452 fix(services): match service aliases without case
Uppercase entries in ALIASES never resolved, because get_tag lowercased the input but not the alias. Match real tags first so a service's own tag cannot be taken by another service's alias.
2026-08-24 11:56:46 -06:00
imSp4rky a4be2cd23a fix(music): remux songs into the container of their codec
A song skips the muxer, so the decrypted fragmented MP4 was the delivered file: its header states a length of zero and FLAC in an MP4 is not a FLAC stream, so playback stopped early and tagging failed.
2026-08-21 14:00:04 -06:00
imSp4rky aa42c6b805 test(core): cover session pool growth, DRM locks, playlist encoding, and hybrid L6 2026-08-20 13:59:00 -06:00
imSp4rky 537a1dc973 perf(dl): parallelise mux, licensing, and vault lookups; size the shared pool
Grow the service session pool to downloads * workers before track downloads, run mkvmerge jobs concurrently (muxing.concurrency, default 4), lock DRM preparation per key set instead of globally, and query local vaults first then remote vaults together.
2026-08-20 11:24:12 -06:00
imSp4rky 29d0828c33 style: clean up prose 2026-08-19 13:45:58 -06:00
imSp4rky 54adb82746 fix(vaults): look tables up by the name the vault stores
get_services() normalised table names through Services.get_tag() and then handed the result back as an identifier, so kv copy and sync skipped any table whose name differed in case or matched an alias. Yield stored names, and resolve tables case-insensitively.
2026-08-19 12:11:44 -06:00
imSp4rky 68c7d95173 refactor(music)!: make Song an ordinary Title on the generic path
Delete the dead grouped-download block, the planner, and the music service hooks; Song now flows through the one download loop, gets tagged via tag_file, and answers -w with {disc}x{track} keys.

BREAKING CHANGE: GROUP_AUDIO_DOWNLOADS, get_music_track_options and get_music_collection_label are gone. A music service returns one Audio track per quality and lets the framework choose.
2026-08-18 15:05:30 -06:00
imSp4rky 511f96d8db docs: strip non-essential comments, fold key facts into docstrings 2026-08-16 23:32:02 -06:00
imSp4rky 66bcbe9377 test(core): skip download integration tests when ffprobe is absent 2026-08-16 22:32:33 -06:00
imSp4rky 42718cbd99 fix(titles): drop folder {year?} when it repeats the dated season label
Dated episodes turn the season folder into the air year, so a folder template's {year?} rendered the same year twice (Title.2025.2025).
2026-08-16 22:01:12 -06:00
imSp4rky 619c86cd10 feat(downloader): resume failed downloads from completed segments across runs
Fingerprint-gated reuse of atomically-published segment files (DASH/ISM/HLS) behind a continue_downloads config option and one-off dl --continue-downloads flag.
2026-08-16 19:38:06 -06:00
imSp4rky dd9dafef91 fix(downloader): harden ranged reads, teardown and multiprocess sessions
Validates the 206 range end, bounds the DASH collapse, and preserves cookie domains and pool size in spawned children.
2026-08-16 17:07:28 -06:00
imSp4rky e3c1ce3511 refactor(tests): drop leading underscores from test helper functions
Completes the core underscore sweep for the test files this branch added; make_ctx/make_track/seg_urls avoid shadowing locals.
2026-08-16 12:52:36 -06:00
imSp4rky ee9d3edfe3 Merge branch 'dev' into feat/downloader-speed
# Conflicts:
#	unshackle/core/api/download_manager.py
#	unshackle/core/downloaders/requests.py
#	unshackle/core/manifests/dash.py
#	unshackle/core/session.py
#	unshackle/core/utils/webvtt.py
2026-08-16 12:43:01 -06:00
imSp4rky 21d5f1c5e1 refactor(core)!: drop leading underscores from internal definitions
Renames 314 defs and 41 module/class-level constants. Keeps 11 names underscored where the stripped form collides in scope, plus 5.5.0 shims for the three symbols external service repos call.
2026-08-15 22:07:41 -06:00
imSp4rky 4a21dbebb5 feat(dl): add daily date-based episodes with tvdb air-date fill
--daily flag plus Service.DAILY/Title.daily, -w selection by ISO air
date or colon date range, and a pre-1970 air-date guard.
2026-08-11 17:50:22 -06:00
imSp4rky e4f6ddec85 feat(dl): add absolute episode numbers filled from tvdb absolute order
Episode gains an additive absolute field services can set, an {absolute}
naming token (3-wide), and an --enrich fill from TVDB's absolute order
that never rewrites season or episode numbers.
2026-08-11 16:42:07 -06:00
imSp4rky 72581ba5d2 feat(metadata)!: replace animeapi with a first-class anilist provider
--anilist supersedes --animeapi (REST: anilist_id); the keyless AniList
GraphQL provider joins the registry with title search, mal: ID lookup,
an ANILIST mkv tag, and the anilist_title_language config option.
Services mark anime via Service.ANIME / Title.anime for anilist-first
metadata resolution with fall-through.
2026-08-11 16:42:05 -06:00