mirror of
https://github.com/unshackle-dl/unshackle.git
synced 2026-11-12 20:15:10 +00:00
feat(dl)!: treat a missing -sl language as a warning
A missing -sl language now warns and continues, with or without --best-available. --require-subs counts a forced track only when selection keeps it, and a title with no track left is skipped instead of failing at mux. BREAKING CHANGE: a missing -sl subtitle language no longer fails the run. Use --require-subs to fail on it.
This commit is contained in:
imSp4rky
@@ -450,7 +450,7 @@ Make a download job. It requires `service` and `title_id`. Every other field is
|
||||
| `wanted` | string / string[] | `[]` | Episode/season selectors, as a list or as one comma-separated string. Accepts the part form, `"S01E01.2"`, and the air-date form, `"2026-08-11"` or `"2026-08-01:2026-08-31"`. For a music release, a selector is a track number, `"1-5"` or `"1,3,7"`, or `"{disc}x{track}"` such as `"2x3"`. |
|
||||
| `latest_episode` | boolean | `false` | Only the newest episode. |
|
||||
| `latest_episodes` | integer | `null` | Only the N newest episodes. A positive integer. When set, it overrides `latest_episode`. |
|
||||
| `lang` / `v_lang` / `a_lang` / `s_lang` | string[] | `["orig"]` / `[]` / `[]` / `["all"]` | Language filters. |
|
||||
| `lang` / `v_lang` / `a_lang` / `s_lang` | string[] | `["orig"]` / `[]` / `[]` / `["all"]` | Language filters. A missing `s_lang` language gives a warning and the job continues. |
|
||||
| `require_audio` | string[] | `[]` | Audio languages that must exist. The job fails if one is missing, even with `best_available`. |
|
||||
| `require_video` | string[] | `[]` | Video languages that must exist. The job fails if one is missing, even with `best_available`. |
|
||||
| `require_subs` | string[] | `[]` | Subtitle languages that must exist. The job fails if one is missing. `s_lang` still decides what to keep. |
|
||||
|
||||
@@ -84,7 +84,7 @@ unshackle dl [OPTIONS] SERVICE [SERVICE ARGS...]
|
||||
| `-c`, `--channels` | - | Audio channels; matches sub-layouts (5.1 ≈ 6.0). |
|
||||
| `-naa`, `--noatmos` | off | Exclude Dolby Atmos audio tracks. |
|
||||
| `--worst` | off | Pick the lowest bitrate within the requested quality. **Requires `-q`.** |
|
||||
| `--best-available`, `--warn-only` | off | Warn and continue when a requested resolution, range, or language is absent, instead of failing. With `--all-drm`, also when one DRM system does not license. |
|
||||
| `--best-available`, `--warn-only` | off | Warn and continue when a requested resolution, range, video language, or audio language is absent, instead of failing. With `--all-drm`, also when one DRM system does not license. |
|
||||
| `-rvb`, `--real-video-bitrate` | off | Probe real media size for true video bitrates, overriding the manifest. |
|
||||
| `-rab`, `--real-audio-bitrate` | off | Same for audio (slower). |
|
||||
|
||||
|
||||
@@ -97,7 +97,7 @@ By default a missing requested resolution is an error. Two flags change that:
|
||||
|
||||
| Flag | Behaviour |
|
||||
| --- | --- |
|
||||
| `--best-available` (`--warn-only`) | If the requested resolution(s) are not present, continue with the best that *is* available instead of failing. Also softens missing video/audio/subtitle languages and hybrid fallbacks, and with `--all-drm` a DRM system that does not license. |
|
||||
| `--best-available` (`--warn-only`) | If the requested resolution(s) are not present, continue with the best that *is* available instead of failing. Also softens missing video and audio languages and hybrid fallbacks, and with `--all-drm` a DRM system that does not license. |
|
||||
| `--worst` | Within the specified quality, pick the **lowest** bitrate rendition. **Requires `-q/--quality`.** |
|
||||
|
||||
```shell title="Never fail on a missing resolution"
|
||||
@@ -421,7 +421,8 @@ selects `pt-BR`. If the title has no such variant, the tag takes every regional
|
||||
|
||||
A tag with a region asks for that variant only: `-l es-419` selects `es-419`, never `es-ES`
|
||||
or `es-MX`. If the service has no `es-419` track, unshackle reports `es-419 not found` and
|
||||
stops. With `--best-available`, it continues with the other languages.
|
||||
stops. With `--best-available`, it continues with the other languages. A missing `-sl`
|
||||
language always gives a warning and the download continues; `--require-subs` makes it fail.
|
||||
|
||||
A track without a region matches only the region its language implies. `en` is US English
|
||||
and `pt` is Brazilian Portuguese, so `-l en-US` accepts a track tagged `en`, but `-l en-GB`
|
||||
|
||||
+7
-11
@@ -75,7 +75,7 @@ Subtitle selection happens through `dl` flags. The most important is language se
|
||||
| Flag | Purpose |
|
||||
| --- | --- |
|
||||
| `-sl`, `--s-lang` | Language(s) wanted for subtitles. Defaults to `all`. A `-` prefix excludes, e.g. `all,-es`. |
|
||||
| `--require-subs` | Require these languages to exist; if present, download **all** subtitles. Cannot be combined with `--s-lang`. |
|
||||
| `--require-subs` | Require these languages to exist. The title fails if one is missing. `--s-lang` still decides what to keep. |
|
||||
| `-fs`, `--forced-subs` | Include forced subtitle tracks (excluded by default). |
|
||||
| `-fso`, `--forced-subs-only` | Keep only forced subtitle tracks and drop the other subtitles; implies `-fs`. |
|
||||
| `-fsl`, `--forced-s-lang` | Language(s) wanted for forced subtitles; implies `-fs`. A `-` prefix excludes. |
|
||||
@@ -102,25 +102,21 @@ that variant:
|
||||
unshackle dl --s-lang es-419 EXAMPLE 'https://www.example.com/...'
|
||||
```
|
||||
|
||||
If a requested language is missing, unshackle stops with an error. With `--best-available`,
|
||||
it continues with the languages it found and tells you which ones it dropped. See
|
||||
If a requested language is missing, unshackle continues with the languages it found and
|
||||
tells you which ones it dropped. To stop the title instead, use `--require-subs`. See
|
||||
[exact vs fuzzy matching](downloading.md#exact-vs-fuzzy-matching) for the full rules.
|
||||
|
||||
### Requiring languages before downloading
|
||||
|
||||
`--require-subs` is a gate rather than a filter. It downloads **all** available
|
||||
subtitles, but only if every language you name is present. If any is missing, the title
|
||||
errors out. This is useful for batch jobs where you want a title without your must-have
|
||||
language to fail loudly, and not to silently give a partial result.
|
||||
`--require-subs` is a gate rather than a filter. If any language you name is missing, the
|
||||
title fails, even under `--best-available`. It does not select tracks: `--s-lang` still
|
||||
decides what to keep. This is useful for batch jobs where you want a title without your
|
||||
must-have language to fail loudly, and not to silently give a partial result.
|
||||
|
||||
```shell title="Fail unless both English and French subs exist, then grab everything"
|
||||
unshackle dl --require-subs en,fr EXAMPLE B0ABCDEFGH
|
||||
```
|
||||
|
||||
!!! warning "`--require-subs` and `--s-lang` are mutually exclusive"
|
||||
Because `--require-subs` forces downloading of all subtitle languages, it cannot be
|
||||
combined with `--s-lang`. Choose one or the other.
|
||||
|
||||
### Forced subtitles
|
||||
|
||||
Forced tracks carry only the important on-screen signs and foreign-dialogue lines meant
|
||||
|
||||
@@ -51,7 +51,7 @@ Common config keys. This is a useful subset, and every `dl` flag works:
|
||||
| `workers` | int | *(downloader default)* | Threads per track. |
|
||||
| `speed_limit` | str | *(unlimited)* | Total download speed cap across all downloads combined, e.g. `500k`, `5M`, `1.5G` or plain bytes/sec. Values are bytes, not bits (`5M` = 5.0 MB/s). |
|
||||
| `slow` | str | *(unset)* | Inter-title delay, e.g. `"20-40"`. |
|
||||
| `best_available` | bool | `false` | Warn instead of failing when a requested resolution, range, or language is missing, or when one DRM system does not license under `--all-drm`. The CLI flag is `--best-available`, also spelled `--warn-only`. |
|
||||
| `best_available` | bool | `false` | Warn instead of failing when a requested resolution, range, video language, or audio language is missing, or when one DRM system does not license under `--all-drm`. The CLI flag is `--best-available`, also spelled `--warn-only`. |
|
||||
| `generate_dv` | bool | `false` | With `-r HYBRID`, make Dolby Vision from the HDR10+ metadata when a title has no DV track. Needs `hdr10plus_tool`. See [Dolby Vision from HDR10+](../../guide/downloading.md#dolby-vision-from-hdr10). |
|
||||
| `proxy` | str | *(unset)* | Default proxy URI or 2-letter country. |
|
||||
| `no_folder` | bool | `false` | Do not create a per-title folder. |
|
||||
|
||||
@@ -141,6 +141,14 @@ def test_a_single_title_with_no_such_track_exits(
|
||||
assert "Skipping" not in caplog.text
|
||||
|
||||
|
||||
def test_subs_only_with_no_matching_subtitle_exits_a_single_title_and_skips_in_a_season(
|
||||
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
assert run(monkeypatch, caplog, {1: {}}, subs_only=True, s_lang=["fr"]) == 1
|
||||
assert run(monkeypatch, caplog, {1: {}, 2: {}}, subs_only=True, s_lang=["fr"]) is None
|
||||
assert caplog.text.count("There's no track left for the selected options. Skipping Show S01E") == 2
|
||||
|
||||
|
||||
def test_a_multi_title_run_skips_the_title_and_continues(
|
||||
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
|
||||
@@ -147,8 +147,41 @@ def test_a_forced_only_language_without_fs_is_reported_missing(
|
||||
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
service = RegionalService("en", ["ja"], [("en", ""), ("es-419", ""), ("es-ES", "forced")])
|
||||
assert run(monkeypatch, caplog, service, lang=["ja"], s_lang=["en", "es-ES"]) == 1
|
||||
assert "es-ES not found in subtitle tracks" in caplog.text
|
||||
ids = run(monkeypatch, caplog, service, lang=["ja"], s_lang=["en", "es-ES"])
|
||||
assert "es-ES not found in subtitle tracks, continuing with: en" in caplog.text
|
||||
assert ids["subs"] == ["en"]
|
||||
|
||||
|
||||
def test_a_missing_subtitle_language_continues_without_subtitles(
|
||||
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
service = RegionalService("en", ["ja"], [("en", "")])
|
||||
ids = run(monkeypatch, caplog, service, lang=["ja"], s_lang=["fr"])
|
||||
assert "fr not found in subtitle tracks, continuing without subtitles" in caplog.text
|
||||
assert ids["subs"] == []
|
||||
|
||||
|
||||
def test_a_missing_orig_subtitle_is_reported_as_orig(
|
||||
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
service = RegionalService("ja", ["ja"], [("en", "")])
|
||||
ids = run(monkeypatch, caplog, service, s_lang=["orig", "en"])
|
||||
assert "orig not found in subtitle tracks, continuing with: en" in caplog.text
|
||||
assert ids["subs"] == ["en"]
|
||||
|
||||
|
||||
def test_require_subs_fails_on_a_forced_track_that_is_dropped(
|
||||
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
service = RegionalService("ja", ["ja"], [("en", "forced")])
|
||||
assert run(monkeypatch, caplog, service, require_subs=["en"]) == 1
|
||||
|
||||
|
||||
def test_require_subs_counts_a_forced_track_that_fs_keeps(
|
||||
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
service = RegionalService("ja", ["ja"], [("en", "forced")])
|
||||
assert run(monkeypatch, caplog, service, require_subs=["en"], forced_subs=True)["subs"] == ["en-forced"]
|
||||
|
||||
|
||||
def test_an_exclusion_removes_every_regional_form(
|
||||
|
||||
+28
-43
@@ -119,7 +119,7 @@ from unshackle.core.utilities import (
|
||||
matched_language_tags,
|
||||
matching_languages,
|
||||
missing_required_langs,
|
||||
original_tag,
|
||||
orig_token,
|
||||
partition_exclusions,
|
||||
plain_subtitle_tags,
|
||||
resolve_sort_langs,
|
||||
@@ -1225,7 +1225,7 @@ class dl:
|
||||
"best_available",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help="Warn instead of failing when a requested resolution, range, or language is missing, and continue with what is available.",
|
||||
help="Warn instead of failing when a requested resolution, range, video language, or audio language is missing.",
|
||||
)
|
||||
@click.option(
|
||||
"--generate-dv",
|
||||
@@ -2538,7 +2538,11 @@ class dl:
|
||||
(
|
||||
"subtitle",
|
||||
require_subs if keep_subtitles else [],
|
||||
[t.language for t in title.tracks.subtitles if t.forced or not forced_subs_only],
|
||||
[
|
||||
t.language
|
||||
for t in title.tracks.subtitles
|
||||
if (t.forced if forced_subs_only else forced_subs or not t.forced)
|
||||
],
|
||||
),
|
||||
):
|
||||
missing_required = missing_required_langs(required, available, title.language, exact=exact_lang)
|
||||
@@ -2631,16 +2635,11 @@ class dl:
|
||||
)
|
||||
if keep_videos and video_languages and "all" not in video_languages:
|
||||
processed_video_lang = resolve_sort_langs(video_languages, title.language, exact=exact_lang)
|
||||
v_orig_token: Optional[str] = None
|
||||
if "orig" in video_languages:
|
||||
if title.language:
|
||||
v_orig_token = original_tag(title.language, exact_lang)
|
||||
else:
|
||||
self.log.warning(
|
||||
"Original language not available for title, skipping 'orig' selection for video"
|
||||
)
|
||||
if v_orig_token in video_languages:
|
||||
v_orig_token = None
|
||||
v_orig_token = orig_token(video_languages, title.language, exact_lang)
|
||||
if "orig" in video_languages and not title.language:
|
||||
self.log.warning(
|
||||
"Original language not available for title, skipping 'orig' selection for video"
|
||||
)
|
||||
missing_v_langs = find_missing_langs(
|
||||
processed_video_lang,
|
||||
[v.language for v in title.tracks.videos],
|
||||
@@ -2860,6 +2859,7 @@ class dl:
|
||||
)
|
||||
title.tracks.select_subtitles(lambda x: str(x.language) not in drop)
|
||||
if keep_subtitles and s_lang and "all" not in s_lang:
|
||||
s_orig_token = orig_token(s_lang, title.language, exact_lang)
|
||||
s_lang = resolve_sort_langs(s_lang, title.language, exact=exact_lang)
|
||||
# the pool selection draws from: forced tracks count only when -fs keeps them
|
||||
missing_langs = find_missing_langs(
|
||||
@@ -2868,25 +2868,15 @@ class dl:
|
||||
exact=exact_lang,
|
||||
)
|
||||
if missing_langs:
|
||||
missing_str = ", ".join(missing_langs)
|
||||
if best_available:
|
||||
remaining = [tok for tok in s_lang if tok not in missing_langs]
|
||||
if remaining:
|
||||
self.log.warning(
|
||||
f"{missing_str} not found in subtitle tracks, continuing with: {', '.join(remaining)}"
|
||||
)
|
||||
s_lang = remaining
|
||||
else:
|
||||
self.log.warning(
|
||||
f"{missing_str} not found in subtitle tracks, continuing without subtitles"
|
||||
)
|
||||
fsl_tags = matched_language_tags(
|
||||
fsl, [x.language for x in title.tracks.subtitles if x.forced], exact_lang
|
||||
)
|
||||
title.tracks.select_subtitles(lambda x: x.forced and str(x.language) in fsl_tags)
|
||||
else:
|
||||
self.log.error(missing_str + " not found in subtitle tracks")
|
||||
sys.exit(1)
|
||||
remaining = [tok for tok in s_lang if tok not in missing_langs]
|
||||
tail = (
|
||||
f"continuing with: {as_requested(remaining, s_orig_token)}"
|
||||
if remaining
|
||||
else "continuing without subtitles"
|
||||
)
|
||||
self.log.warning(
|
||||
f"{as_requested(missing_langs, s_orig_token)} not found in subtitle tracks, {tail}"
|
||||
)
|
||||
|
||||
if s_lang and title.tracks.subtitles:
|
||||
plain_tags = plain_subtitle_tags(s_lang, title.tracks.subtitles, exact_lang)
|
||||
@@ -2896,9 +2886,6 @@ class dl:
|
||||
title.tracks.select_subtitles(
|
||||
lambda x: str(x.language) in (forced_tags if x.forced else plain_tags)
|
||||
)
|
||||
if not title.tracks.subtitles and not best_available:
|
||||
self.log.error(f"There's no {s_lang} Subtitle Track...")
|
||||
sys.exit(1)
|
||||
|
||||
if keep_subtitles:
|
||||
# fsl, not forced_s_lang: unresolvable orig must keep all forced, not drop them
|
||||
@@ -2982,14 +2969,9 @@ class dl:
|
||||
audio_languages = a_lang or lang
|
||||
if audio_languages:
|
||||
processed_lang = resolve_sort_langs(audio_languages, title.language, exact=exact_lang)
|
||||
a_orig_token: Optional[str] = None
|
||||
if "orig" in audio_languages:
|
||||
if title.language:
|
||||
a_orig_token = original_tag(title.language, exact_lang)
|
||||
else:
|
||||
self.log.warning("Original language not available for title, skipping 'orig' selection")
|
||||
if a_orig_token in audio_languages:
|
||||
a_orig_token = None
|
||||
a_orig_token = orig_token(audio_languages, title.language, exact_lang)
|
||||
if "orig" in audio_languages and not title.language:
|
||||
self.log.warning("Original language not available for title, skipping 'orig' selection")
|
||||
|
||||
embedded_langs = embedded_audio_langs(
|
||||
title.tracks.videos, keep_videos and not audio_description_only
|
||||
@@ -3046,6 +3028,9 @@ class dl:
|
||||
kept_tracks.extend(title.tracks.attachments)
|
||||
|
||||
title.tracks = Tracks(kept_tracks, manifest_url=title.tracks.manifest_url)
|
||||
if not (title.tracks.videos or title.tracks.audio or title.tracks.subtitles or title.tracks.chapters):
|
||||
skip_title(title, "There's no track left for the selected options.")
|
||||
continue
|
||||
|
||||
selected_tracks, tracks_progress_callables = title.tracks.tree(add_progress=True)
|
||||
|
||||
|
||||
@@ -1003,7 +1003,7 @@ async def download(request: web.Request) -> web.Response:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: Language for subtitle tracks (a '-' prefix excludes, e.g. ["all", "-es"]) (default - ["all"])
|
||||
description: Language for subtitle tracks (a '-' prefix excludes, e.g. ["all", "-es"]); a missing one is skipped with a warning (default - ["all"])
|
||||
require_audio:
|
||||
type: array
|
||||
items:
|
||||
@@ -1018,7 +1018,7 @@ async def download(request: web.Request) -> web.Response:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: Required subtitle languages (default - [])
|
||||
description: Subtitle languages that must exist, else the job fails (default - [])
|
||||
forced_subs:
|
||||
type: boolean
|
||||
description: Include forced subtitle tracks (default - false)
|
||||
|
||||
@@ -473,6 +473,16 @@ def original_tag(language: Union[str, Language], exact: bool = False) -> str:
|
||||
return str(language) if exact else ServiceTag(str(language))
|
||||
|
||||
|
||||
def orig_token(
|
||||
requested: Sequence[str], language: Optional[Union[str, Language]], exact: bool = False
|
||||
) -> Optional[str]:
|
||||
"""The tag 'orig' resolved to, for as_requested. None when 'orig' did not resolve or the user also named that tag."""
|
||||
if "orig" not in requested or not language:
|
||||
return None
|
||||
tag = original_tag(language, exact)
|
||||
return None if tag in requested else tag
|
||||
|
||||
|
||||
def resolve_sort_langs(
|
||||
tokens: Sequence[str], original: Optional[Union[str, Language]] = None, *, exact: bool = False
|
||||
) -> list[str]:
|
||||
|
||||
Reference in New Issue
Block a user