diff --git a/docs/dev/rest-api/endpoints.md b/docs/dev/rest-api/endpoints.md index ecdb2e9a..a2b5c3b1 100644 --- a/docs/dev/rest-api/endpoints.md +++ b/docs/dev/rest-api/endpoints.md @@ -450,7 +450,7 @@ Make a download job. It requires `service` and `title_id`. Every other field is | `wanted` | string / string[] | `[]` | Episode/season selectors, as a list or as one comma-separated string. Accepts the part form, `"S01E01.2"`, and the air-date form, `"2026-08-11"` or `"2026-08-01:2026-08-31"`. For a music release, a selector is a track number, `"1-5"` or `"1,3,7"`, or `"{disc}x{track}"` such as `"2x3"`. | | `latest_episode` | boolean | `false` | Only the newest episode. | | `latest_episodes` | integer | `null` | Only the N newest episodes. A positive integer. When set, it overrides `latest_episode`. | -| `lang` / `v_lang` / `a_lang` / `s_lang` | string[] | `["orig"]` / `[]` / `[]` / `["all"]` | Language filters. | +| `lang` / `v_lang` / `a_lang` / `s_lang` | string[] | `["orig"]` / `[]` / `[]` / `["all"]` | Language filters. A missing `s_lang` language gives a warning and the job continues. | | `require_audio` | string[] | `[]` | Audio languages that must exist. The job fails if one is missing, even with `best_available`. | | `require_video` | string[] | `[]` | Video languages that must exist. The job fails if one is missing, even with `best_available`. | | `require_subs` | string[] | `[]` | Subtitle languages that must exist. The job fails if one is missing. `s_lang` still decides what to keep. | diff --git a/docs/guide/cli-reference.md b/docs/guide/cli-reference.md index 34a7efb0..4106ce8b 100644 --- a/docs/guide/cli-reference.md +++ b/docs/guide/cli-reference.md @@ -84,7 +84,7 @@ unshackle dl [OPTIONS] SERVICE [SERVICE ARGS...] | `-c`, `--channels` | - | Audio channels; matches sub-layouts (5.1 ≈ 6.0). | | `-naa`, `--noatmos` | off | Exclude Dolby Atmos audio tracks. | | `--worst` | off | Pick the lowest bitrate within the requested quality. **Requires `-q`.** | -| `--best-available`, `--warn-only` | off | Warn and continue when a requested resolution, range, or language is absent, instead of failing. With `--all-drm`, also when one DRM system does not license. | +| `--best-available`, `--warn-only` | off | Warn and continue when a requested resolution, range, video language, or audio language is absent, instead of failing. With `--all-drm`, also when one DRM system does not license. | | `-rvb`, `--real-video-bitrate` | off | Probe real media size for true video bitrates, overriding the manifest. | | `-rab`, `--real-audio-bitrate` | off | Same for audio (slower). | diff --git a/docs/guide/downloading.md b/docs/guide/downloading.md index f54340bd..4a94c6ee 100644 --- a/docs/guide/downloading.md +++ b/docs/guide/downloading.md @@ -97,7 +97,7 @@ By default a missing requested resolution is an error. Two flags change that: | Flag | Behaviour | | --- | --- | -| `--best-available` (`--warn-only`) | If the requested resolution(s) are not present, continue with the best that *is* available instead of failing. Also softens missing video/audio/subtitle languages and hybrid fallbacks, and with `--all-drm` a DRM system that does not license. | +| `--best-available` (`--warn-only`) | If the requested resolution(s) are not present, continue with the best that *is* available instead of failing. Also softens missing video and audio languages and hybrid fallbacks, and with `--all-drm` a DRM system that does not license. | | `--worst` | Within the specified quality, pick the **lowest** bitrate rendition. **Requires `-q/--quality`.** | ```shell title="Never fail on a missing resolution" @@ -421,7 +421,8 @@ selects `pt-BR`. If the title has no such variant, the tag takes every regional A tag with a region asks for that variant only: `-l es-419` selects `es-419`, never `es-ES` or `es-MX`. If the service has no `es-419` track, unshackle reports `es-419 not found` and -stops. With `--best-available`, it continues with the other languages. +stops. With `--best-available`, it continues with the other languages. A missing `-sl` +language always gives a warning and the download continues; `--require-subs` makes it fail. A track without a region matches only the region its language implies. `en` is US English and `pt` is Brazilian Portuguese, so `-l en-US` accepts a track tagged `en`, but `-l en-GB` diff --git a/docs/guide/subtitles.md b/docs/guide/subtitles.md index b46c33d9..f56d1899 100644 --- a/docs/guide/subtitles.md +++ b/docs/guide/subtitles.md @@ -75,7 +75,7 @@ Subtitle selection happens through `dl` flags. The most important is language se | Flag | Purpose | | --- | --- | | `-sl`, `--s-lang` | Language(s) wanted for subtitles. Defaults to `all`. A `-` prefix excludes, e.g. `all,-es`. | -| `--require-subs` | Require these languages to exist; if present, download **all** subtitles. Cannot be combined with `--s-lang`. | +| `--require-subs` | Require these languages to exist. The title fails if one is missing. `--s-lang` still decides what to keep. | | `-fs`, `--forced-subs` | Include forced subtitle tracks (excluded by default). | | `-fso`, `--forced-subs-only` | Keep only forced subtitle tracks and drop the other subtitles; implies `-fs`. | | `-fsl`, `--forced-s-lang` | Language(s) wanted for forced subtitles; implies `-fs`. A `-` prefix excludes. | @@ -102,25 +102,21 @@ that variant: unshackle dl --s-lang es-419 EXAMPLE 'https://www.example.com/...' ``` -If a requested language is missing, unshackle stops with an error. With `--best-available`, -it continues with the languages it found and tells you which ones it dropped. See +If a requested language is missing, unshackle continues with the languages it found and +tells you which ones it dropped. To stop the title instead, use `--require-subs`. See [exact vs fuzzy matching](downloading.md#exact-vs-fuzzy-matching) for the full rules. ### Requiring languages before downloading -`--require-subs` is a gate rather than a filter. It downloads **all** available -subtitles, but only if every language you name is present. If any is missing, the title -errors out. This is useful for batch jobs where you want a title without your must-have -language to fail loudly, and not to silently give a partial result. +`--require-subs` is a gate rather than a filter. If any language you name is missing, the +title fails, even under `--best-available`. It does not select tracks: `--s-lang` still +decides what to keep. This is useful for batch jobs where you want a title without your +must-have language to fail loudly, and not to silently give a partial result. ```shell title="Fail unless both English and French subs exist, then grab everything" unshackle dl --require-subs en,fr EXAMPLE B0ABCDEFGH ``` -!!! warning "`--require-subs` and `--s-lang` are mutually exclusive" - Because `--require-subs` forces downloading of all subtitle languages, it cannot be - combined with `--s-lang`. Choose one or the other. - ### Forced subtitles Forced tracks carry only the important on-screen signs and foreign-dialogue lines meant diff --git a/docs/reference/configuration/download.md b/docs/reference/configuration/download.md index 35052e95..80d0e856 100644 --- a/docs/reference/configuration/download.md +++ b/docs/reference/configuration/download.md @@ -51,7 +51,7 @@ Common config keys. This is a useful subset, and every `dl` flag works: | `workers` | int | *(downloader default)* | Threads per track. | | `speed_limit` | str | *(unlimited)* | Total download speed cap across all downloads combined, e.g. `500k`, `5M`, `1.5G` or plain bytes/sec. Values are bytes, not bits (`5M` = 5.0 MB/s). | | `slow` | str | *(unset)* | Inter-title delay, e.g. `"20-40"`. | -| `best_available` | bool | `false` | Warn instead of failing when a requested resolution, range, or language is missing, or when one DRM system does not license under `--all-drm`. The CLI flag is `--best-available`, also spelled `--warn-only`. | +| `best_available` | bool | `false` | Warn instead of failing when a requested resolution, range, video language, or audio language is missing, or when one DRM system does not license under `--all-drm`. The CLI flag is `--best-available`, also spelled `--warn-only`. | | `generate_dv` | bool | `false` | With `-r HYBRID`, make Dolby Vision from the HDR10+ metadata when a title has no DV track. Needs `hdr10plus_tool`. See [Dolby Vision from HDR10+](../../guide/downloading.md#dolby-vision-from-hdr10). | | `proxy` | str | *(unset)* | Default proxy URI or 2-letter country. | | `no_folder` | bool | `false` | Do not create a per-title folder. | diff --git a/tests/lang_selection/test_kind_only_selection.py b/tests/lang_selection/test_kind_only_selection.py index 81e0b4ca..ff64e1d6 100644 --- a/tests/lang_selection/test_kind_only_selection.py +++ b/tests/lang_selection/test_kind_only_selection.py @@ -141,6 +141,14 @@ def test_a_single_title_with_no_such_track_exits( assert "Skipping" not in caplog.text +def test_subs_only_with_no_matching_subtitle_exits_a_single_title_and_skips_in_a_season( + monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture +) -> None: + assert run(monkeypatch, caplog, {1: {}}, subs_only=True, s_lang=["fr"]) == 1 + assert run(monkeypatch, caplog, {1: {}, 2: {}}, subs_only=True, s_lang=["fr"]) is None + assert caplog.text.count("There's no track left for the selected options. Skipping Show S01E") == 2 + + def test_a_multi_title_run_skips_the_title_and_continues( monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture ) -> None: diff --git a/tests/lang_selection/test_regional_selection_loop.py b/tests/lang_selection/test_regional_selection_loop.py index faed3cf6..7812459f 100644 --- a/tests/lang_selection/test_regional_selection_loop.py +++ b/tests/lang_selection/test_regional_selection_loop.py @@ -147,8 +147,41 @@ def test_a_forced_only_language_without_fs_is_reported_missing( monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture ) -> None: service = RegionalService("en", ["ja"], [("en", ""), ("es-419", ""), ("es-ES", "forced")]) - assert run(monkeypatch, caplog, service, lang=["ja"], s_lang=["en", "es-ES"]) == 1 - assert "es-ES not found in subtitle tracks" in caplog.text + ids = run(monkeypatch, caplog, service, lang=["ja"], s_lang=["en", "es-ES"]) + assert "es-ES not found in subtitle tracks, continuing with: en" in caplog.text + assert ids["subs"] == ["en"] + + +def test_a_missing_subtitle_language_continues_without_subtitles( + monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture +) -> None: + service = RegionalService("en", ["ja"], [("en", "")]) + ids = run(monkeypatch, caplog, service, lang=["ja"], s_lang=["fr"]) + assert "fr not found in subtitle tracks, continuing without subtitles" in caplog.text + assert ids["subs"] == [] + + +def test_a_missing_orig_subtitle_is_reported_as_orig( + monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture +) -> None: + service = RegionalService("ja", ["ja"], [("en", "")]) + ids = run(monkeypatch, caplog, service, s_lang=["orig", "en"]) + assert "orig not found in subtitle tracks, continuing with: en" in caplog.text + assert ids["subs"] == ["en"] + + +def test_require_subs_fails_on_a_forced_track_that_is_dropped( + monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture +) -> None: + service = RegionalService("ja", ["ja"], [("en", "forced")]) + assert run(monkeypatch, caplog, service, require_subs=["en"]) == 1 + + +def test_require_subs_counts_a_forced_track_that_fs_keeps( + monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture +) -> None: + service = RegionalService("ja", ["ja"], [("en", "forced")]) + assert run(monkeypatch, caplog, service, require_subs=["en"], forced_subs=True)["subs"] == ["en-forced"] def test_an_exclusion_removes_every_regional_form( diff --git a/unshackle/commands/dl.py b/unshackle/commands/dl.py index 0fcbd091..a141c4bb 100644 --- a/unshackle/commands/dl.py +++ b/unshackle/commands/dl.py @@ -119,7 +119,7 @@ from unshackle.core.utilities import ( matched_language_tags, matching_languages, missing_required_langs, - original_tag, + orig_token, partition_exclusions, plain_subtitle_tags, resolve_sort_langs, @@ -1225,7 +1225,7 @@ class dl: "best_available", is_flag=True, default=False, - help="Warn instead of failing when a requested resolution, range, or language is missing, and continue with what is available.", + help="Warn instead of failing when a requested resolution, range, video language, or audio language is missing.", ) @click.option( "--generate-dv", @@ -2538,7 +2538,11 @@ class dl: ( "subtitle", require_subs if keep_subtitles else [], - [t.language for t in title.tracks.subtitles if t.forced or not forced_subs_only], + [ + t.language + for t in title.tracks.subtitles + if (t.forced if forced_subs_only else forced_subs or not t.forced) + ], ), ): missing_required = missing_required_langs(required, available, title.language, exact=exact_lang) @@ -2631,16 +2635,11 @@ class dl: ) if keep_videos and video_languages and "all" not in video_languages: processed_video_lang = resolve_sort_langs(video_languages, title.language, exact=exact_lang) - v_orig_token: Optional[str] = None - if "orig" in video_languages: - if title.language: - v_orig_token = original_tag(title.language, exact_lang) - else: - self.log.warning( - "Original language not available for title, skipping 'orig' selection for video" - ) - if v_orig_token in video_languages: - v_orig_token = None + v_orig_token = orig_token(video_languages, title.language, exact_lang) + if "orig" in video_languages and not title.language: + self.log.warning( + "Original language not available for title, skipping 'orig' selection for video" + ) missing_v_langs = find_missing_langs( processed_video_lang, [v.language for v in title.tracks.videos], @@ -2860,6 +2859,7 @@ class dl: ) title.tracks.select_subtitles(lambda x: str(x.language) not in drop) if keep_subtitles and s_lang and "all" not in s_lang: + s_orig_token = orig_token(s_lang, title.language, exact_lang) s_lang = resolve_sort_langs(s_lang, title.language, exact=exact_lang) # the pool selection draws from: forced tracks count only when -fs keeps them missing_langs = find_missing_langs( @@ -2868,25 +2868,15 @@ class dl: exact=exact_lang, ) if missing_langs: - missing_str = ", ".join(missing_langs) - if best_available: - remaining = [tok for tok in s_lang if tok not in missing_langs] - if remaining: - self.log.warning( - f"{missing_str} not found in subtitle tracks, continuing with: {', '.join(remaining)}" - ) - s_lang = remaining - else: - self.log.warning( - f"{missing_str} not found in subtitle tracks, continuing without subtitles" - ) - fsl_tags = matched_language_tags( - fsl, [x.language for x in title.tracks.subtitles if x.forced], exact_lang - ) - title.tracks.select_subtitles(lambda x: x.forced and str(x.language) in fsl_tags) - else: - self.log.error(missing_str + " not found in subtitle tracks") - sys.exit(1) + remaining = [tok for tok in s_lang if tok not in missing_langs] + tail = ( + f"continuing with: {as_requested(remaining, s_orig_token)}" + if remaining + else "continuing without subtitles" + ) + self.log.warning( + f"{as_requested(missing_langs, s_orig_token)} not found in subtitle tracks, {tail}" + ) if s_lang and title.tracks.subtitles: plain_tags = plain_subtitle_tags(s_lang, title.tracks.subtitles, exact_lang) @@ -2896,9 +2886,6 @@ class dl: title.tracks.select_subtitles( lambda x: str(x.language) in (forced_tags if x.forced else plain_tags) ) - if not title.tracks.subtitles and not best_available: - self.log.error(f"There's no {s_lang} Subtitle Track...") - sys.exit(1) if keep_subtitles: # fsl, not forced_s_lang: unresolvable orig must keep all forced, not drop them @@ -2982,14 +2969,9 @@ class dl: audio_languages = a_lang or lang if audio_languages: processed_lang = resolve_sort_langs(audio_languages, title.language, exact=exact_lang) - a_orig_token: Optional[str] = None - if "orig" in audio_languages: - if title.language: - a_orig_token = original_tag(title.language, exact_lang) - else: - self.log.warning("Original language not available for title, skipping 'orig' selection") - if a_orig_token in audio_languages: - a_orig_token = None + a_orig_token = orig_token(audio_languages, title.language, exact_lang) + if "orig" in audio_languages and not title.language: + self.log.warning("Original language not available for title, skipping 'orig' selection") embedded_langs = embedded_audio_langs( title.tracks.videos, keep_videos and not audio_description_only @@ -3046,6 +3028,9 @@ class dl: kept_tracks.extend(title.tracks.attachments) title.tracks = Tracks(kept_tracks, manifest_url=title.tracks.manifest_url) + if not (title.tracks.videos or title.tracks.audio or title.tracks.subtitles or title.tracks.chapters): + skip_title(title, "There's no track left for the selected options.") + continue selected_tracks, tracks_progress_callables = title.tracks.tree(add_progress=True) diff --git a/unshackle/core/api/routes.py b/unshackle/core/api/routes.py index c78bf60a..52de1a3c 100644 --- a/unshackle/core/api/routes.py +++ b/unshackle/core/api/routes.py @@ -1003,7 +1003,7 @@ async def download(request: web.Request) -> web.Response: type: array items: type: string - description: Language for subtitle tracks (a '-' prefix excludes, e.g. ["all", "-es"]) (default - ["all"]) + description: Language for subtitle tracks (a '-' prefix excludes, e.g. ["all", "-es"]); a missing one is skipped with a warning (default - ["all"]) require_audio: type: array items: @@ -1018,7 +1018,7 @@ async def download(request: web.Request) -> web.Response: type: array items: type: string - description: Required subtitle languages (default - []) + description: Subtitle languages that must exist, else the job fails (default - []) forced_subs: type: boolean description: Include forced subtitle tracks (default - false) diff --git a/unshackle/core/utilities.py b/unshackle/core/utilities.py index 51952659..3ca01819 100644 --- a/unshackle/core/utilities.py +++ b/unshackle/core/utilities.py @@ -473,6 +473,16 @@ def original_tag(language: Union[str, Language], exact: bool = False) -> str: return str(language) if exact else ServiceTag(str(language)) +def orig_token( + requested: Sequence[str], language: Optional[Union[str, Language]], exact: bool = False +) -> Optional[str]: + """The tag 'orig' resolved to, for as_requested. None when 'orig' did not resolve or the user also named that tag.""" + if "orig" not in requested or not language: + return None + tag = original_tag(language, exact) + return None if tag in requested else tag + + def resolve_sort_langs( tokens: Sequence[str], original: Optional[Union[str, Language]] = None, *, exact: bool = False ) -> list[str]: