mirror of
https://github.com/unshackle-dl/unshackle.git
synced 2026-11-14 21:15:06 +00:00
feat(downloader): resume failed downloads from completed segments across runs
Fingerprint-gated reuse of atomically-published segment files (DASH/ISM/HLS) behind a continue_downloads config option and one-off dl --continue-downloads flag.
This commit is contained in:
@@ -202,6 +202,7 @@ order the service used, then renumbers the episodes into the order you asked for
|
||||
| `--workers` | downloader default | Per-track download threads. |
|
||||
| `--adaptive-workers` | off | Scale per-track segment workers dynamically (up to the `--workers` cap) based on measured CDN throughput and errors. |
|
||||
| `--download-processes` | `1` | Split large segment batches (24+) across this many download processes to exceed the single-process throughput cap on multi-gigabit connections. Ignored while a speed limit is set: the cap is one shared budget, which extra processes cannot share, so the download stays in a single process. |
|
||||
| `--continue-downloads` | off | Keep completed segment files across runs and resume a previously failed download. One-off enable of the [`continue_downloads`](../reference/configuration/download.md#continue_downloads) config option. |
|
||||
| `--downloads` | `1` | Number of tracks downloaded concurrently. |
|
||||
| `--speed-limit` | unlimited | Cap total download speed across all threads and tracks, e.g. `500k`, `5M`, `1.5G` or plain bytes/sec. Values are bytes, not bits (`5M` = 5.0 MB/s). `off` disables a configured limit. |
|
||||
| `--no-cache` | off | Bypass the title cache. |
|
||||
|
||||
@@ -613,6 +613,7 @@ Two related flags:
|
||||
| `--workers N` | Threads used per track for segment downloads. Default depends on the downloader. |
|
||||
| `--adaptive-workers` | Opt-in: start with a moderate per-track worker count and ramp it up or back off based on measured CDN throughput and errors, capped at `--workers`. Off by default (fixed worker count). |
|
||||
| `--download-processes N` | Opt-in: split a large segment batch across `N` download processes, each with its own worker pool. A single process tops out around 1.3 Gb/s (Python interpreter limit); `2` reaches 2.5GbE line rate on fast CDNs. Only engages for batches of 24+ segments. Ignored while `--speed-limit` (or serve's `global_speed_limit`) is set: the cap is one shared budget, which extra processes cannot share, so the download stays in a single process. Default `1`. |
|
||||
| `--continue-downloads` | Opt-in: keep completed segment files when a download fails so the next run resumes instead of restarting from zero. One-off enable of the [`continue_downloads`](../reference/configuration/download.md#continue_downloads) config option, which documents what can and cannot resume. |
|
||||
| `--downloads N` | Number of tracks downloaded concurrently. Default `1`. |
|
||||
| `--slow [MIN-MAX]` | Add a delay between titles to look more like a real device. `--slow` alone means 60-120s; `--slow 20-40` sets a custom range. Minimum 20s. |
|
||||
| `--no-cache` | Bypass the title cache for this download. |
|
||||
|
||||
@@ -92,6 +92,34 @@ dl:
|
||||
to set server-wide download defaults for every request it handles. Keys the API does not know
|
||||
are ignored there (`speed_limit`, for example; `serve` has its own `global_speed_limit`).
|
||||
|
||||
## `continue_downloads`
|
||||
|
||||
- **Type:** `bool` · **Default:** `false`
|
||||
|
||||
Keep a failed download's completed segment files so the next attempt resumes from where it
|
||||
stopped instead of restarting from byte zero.
|
||||
|
||||
Only whole, fully downloaded segments are reused; partially written segments are always
|
||||
discarded, and no partial-byte (`Range`) resume across runs takes place. Reuse also
|
||||
requires proof that the remote content is unchanged: the manifest URL, segment count, and
|
||||
byte ranges must all match the previous run, otherwise the download silently restarts
|
||||
clean.
|
||||
|
||||
!!! note "What can and cannot resume"
|
||||
- Segmented DASH, ISM, and HLS tracks resume.
|
||||
- Single-file (non-segmented) downloads and DASH tracks collapsed to a single ranged
|
||||
download never produce segment files, so this option does not apply to them (the
|
||||
downloader has its own in-flight partial-file handling for those).
|
||||
- HLS resumes Widevine/PlayReady and unencrypted content only; AES-128 content always
|
||||
restarts. An HLS failure during the post-download merge step also restarts.
|
||||
|
||||
For a one-off resume without changing config, pass `dl --continue-downloads` on the
|
||||
command line (the flag can only enable resume, never disable it).
|
||||
|
||||
```yaml title="Resume failed downloads"
|
||||
continue_downloads: true
|
||||
```
|
||||
|
||||
## `subtitle`
|
||||
|
||||
- **Type:** `dict` · **Default:** `{}`
|
||||
|
||||
@@ -31,7 +31,7 @@ Where the file lives, how unshackle discovers it, and how to edit it:
|
||||
|------|------|
|
||||
| [Directories](directories.md) | `directories`, `filenames` |
|
||||
| [Services & authentication](services.md) | `services`, `credentials`, `firefox_cookies`, `remote_services`, `serve` |
|
||||
| [Download & processing](download.md) | `dl`, `subtitle`, `audio`, `muxing`, `language_tags`, `dual_multi_mode` |
|
||||
| [Download & processing](download.md) | `dl`, `continue_downloads`, `subtitle`, `audio`, `muxing`, `language_tags`, `dual_multi_mode` |
|
||||
| [Output & naming](output.md) | `output_template`, `tag`, `tag_rules`, `tag_group_name`, `tag_imdb_tmdb`, `chapter_fallback_name`, `unicode_filenames` |
|
||||
| [DRM & CDM](drm.md) | `cdm`, `remote_cdm`, `decryption` |
|
||||
| [Network & proxy](network.md) | `network`, `headers`, `proxy_providers` |
|
||||
|
||||
Reference in New Issue
Block a user