Commit Graph

4 Commits

Author SHA1 Message Date
imSp4rky
2f35a4d468 feat(api): aggregate REST download progress with weighting, track labels and mux stage
Replace the class-level Track.download monkeypatch with a per-job progress sink threaded through dl.result(). The API now reports a single aggregate signal instead of each track's bouncing 0-100%:

- bitrate-weighted completion so video/audio dominate subtitles
- completed_tracks/total_tracks counts and active_tracks labels (e.g. "video 2160p DV", "audio en-US 5.1")
- downloads fill 0-90%; repackaging (when needed) and a "muxing" stage carry it to 100% so post-download work is no longer frozen at 100%
- monotonic throughout (handles the download->decrypt callable reuse)

Also:
- accept "HDR10P" as the canonical API range value ("HDR10+" still works)
- declare AUTH_METHODS opt-in on the Service base
- raise typed APIError (WORKER_ERROR/DOWNLOAD_ERROR) from the worker path
- move the progress helpers to unshackle/core/api/progress.py
2026-06-08 15:37:40 -06:00
AviDev
1a3cd09fc8 fix(api): repair REST API downloads, add /services flags & live progress (#113)
* feat(api): live download phase, granular progress, swallowed-failure detection, per-request CDM

- Tee Track.download progress so the job reports real percentage (not just 5/100%) and a
  human-readable phase ('downloading video 1080p') via the new job.phase field.
- Detect a swallowed download-worker failure (dl.result() prints 'Download Failed' but
  exits 0) and raise, so the job is marked failed instead of completed-with-no-output.
- Per-request CDM override (dl_instance.cdm_override; get_cdm prefers it) so a job can use
  a specific CDM device without mutating shared config.

* feat(api): expose service capability flags + auth methods in /services

needs_auth / has_search / has_drm derived from which Service hooks are overridden, and
auth_methods inferred from what the service's authenticate() body references (cookies /
credentials), so clients can show only the relevant auth options per service.

* feat(api): per-request credential injection for downloads

Accept a 'credential' ('user:pass') job parameter and feed it into the credentials map that
dl.get_credentials() reads, so a client can authenticate a download without persisting
anything to disk. (Kept on the deployment branch; the PR branch uses the client-sent path.)

* feat(api): gate per-request CDM override behind serve.cdm_overrides

A per-request `cdm` selects a server-side device, so honour it only when allow-listed.
`serve.cdm_overrides` opts in: a list permits those device names, or `true` permits any
(single trusted client). Unset/false rejects every override with 403, so an arbitrary device
can't be selected by default.

* fix(api): redact credentials and proxy userinfo in serialized job parameters

Job parameters can carry a raw user:pass credential and a proxy URL with embedded userinfo;
mask them wherever parameters are serialized for an API response so secrets don't leak via
the job-detail endpoint. Also read skipped-subtitle / download-failure state from the dl
instance instead of scraping stdout, and drop the dead n_m3u8dl percent branch.

* feat(api): prefer explicit AUTH_METHODS class var over source inference in /services

Inferring auth methods from authenticate() source mostly returns both options because services
call super().authenticate(cookies, credential). Prefer an explicit AUTH_METHODS class var when a
service declares one, falling back to inference.

* style: use plain hyphens instead of em-dashes in comments

* feat(api): gate per-job credentials, isolate caches, scrub error fields

Address review feedback on #113:
- Gate per-request credential/credentials behind serve.allow_job_credentials (default off,
  403 when not opted in), mirroring the existing serve.cdm_overrides CDM gate.
- Isolate the token cache per credential: when a per-job credential is set, namespace
  config.directories.cache by its hash in the worker, so two clients on the same service
  with different credentials can't share a cached session.
- Scrub the credential, its password half, and proxy userinfo out of error_message,
  error_details, error_traceback and worker_stderr before they leave via the job-detail API.
- Remove the unused _execute_download_sync in-process path (would have leaked one job's
  credential into the shared global config).
- Document serve.cdm_overrides and serve.allow_job_credentials in the example config.
- Add tests for both gates (403 default, allowlist pass) and the parameter/error redaction.

* fix(dl): flag download_failed when result() swallows a worker failure

dl.result() catches a download-worker exception, reports it, and returns
without re-raising so the CLI still exits cleanly. An embedding caller (the
API worker) had no way to tell the title actually failed and would report
the job completed with no output. Expose a download_failed flag, set in the
swallow path, that the worker reads after result() returns.

* feat(api): surface skipped subtitles and pass skip_subtitle_errors

Thread skip_subtitle_errors from the job into dl() so the API can opt into
non-fatal subtitle handling, and report which subtitles were skipped: store
them on the job (dl.SkippedSubtitle dicts) and include them in job details
so a client can tell the user which weren't available.

---------

Co-authored-by: Avi Cohen <avraham.coh770@gmail.com>
2026-06-08 11:38:08 -06:00
imSp4rky
7654e91ebc feat(dl): gate s_lang/a_lang miss behind --best-available
Missing requested subtitle and audio languages now warn and continue when --best-available is set instead of hard-exiting. Without the flag, missing languages still produce an error and exit, matching the prior strict behavior. Audio missing-lang detection is now symmetric with subtitles.

- add find_missing_langs helper in core/utilities for reuse between s_lang and a_lang paths (skips all/best/orig sentinels)
- refactor dl.py s_lang/a_lang checks to share the helper
- add tests/lang_selection covering match primitives, helper output, and tricky langcodes corners (zh-Hans/zh-Hant/zh-CN/zh-TW/zh-HK, cmn/yue, fil/tl/tgl)
- clean up unused-var ruff F841 in tests/remote/unit/
2026-05-22 13:52:35 -06:00
imSp4rky
746b573711 test(remote): add unit + e2e suite for remote-services subsystem
Covers RemoteClient/RemoteService, REST routes, handlers, SessionStore, InputBridge, DownloadQueueManager, errors, compression, and serve CLI. E2e tier opts in via --live and can auto-spawn its own serve.
2026-05-21 10:45:25 -06:00