mirror of
https://github.com/unshackle-dl/unshackle.git
synced 2026-11-12 20:15:10 +00:00
feat(dl): exclude languages with a - prefix on -l, -vl, -al, -sl and -fsl
This commit is contained in:
@@ -10,6 +10,7 @@ This changelog is automatically generated using [git-cliff](https://git-cliff.or
|
|||||||
|
|
||||||
### Features
|
### Features
|
||||||
|
|
||||||
|
- *dl*: Exclude a language with a `-` prefix on -l, -vl, -al, -sl and -fsl
|
||||||
- *api*: Report per-episode and per-track progress on download jobs
|
- *api*: Report per-episode and per-track progress on download jobs
|
||||||
- *api*: Add job retry/priority/clear, history, config and maintenance endpoints
|
- *api*: Add job retry/priority/clear, history, config and maintenance endpoints
|
||||||
- *api*: Expose choice values and multiple flag in service cli_params
|
- *api*: Expose choice values and multiple flag in service cli_params
|
||||||
|
|||||||
@@ -90,13 +90,13 @@ unshackle dl [OPTIONS] SERVICE [SERVICE ARGS...]
|
|||||||
|
|
||||||
| Flag | Default | Description |
|
| Flag | Default | Description |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `-l`, `--lang` | `orig` | Language(s) for **both** video and audio. `orig` = the title's original language; e.g. `orig,en`. |
|
| `-l`, `--lang` | `orig` | Language(s) for **both** video and audio. `orig` = the title's original language; e.g. `orig,en`. A `-` prefix excludes, e.g. `all,-es`. |
|
||||||
| `-vl`, `--v-lang` | - | Video-only language (overrides `-l` for video). |
|
| `-vl`, `--v-lang` | - | Video-only language (overrides `-l` for video). A `-` prefix excludes. |
|
||||||
| `-al`, `--a-lang` | - | Audio-only language (overrides `-l` for audio). |
|
| `-al`, `--a-lang` | - | Audio-only language (overrides `-l` for audio). A `-` prefix excludes. |
|
||||||
| `-sl`, `--s-lang` | `all` | Subtitle language(s). |
|
| `-sl`, `--s-lang` | `all` | Subtitle language(s). A `-` prefix excludes, e.g. `all,-es`. |
|
||||||
| `--require-subs` | - | Required subtitle langs; keeps **all** subs only if these exist. **Cannot combine with `--s-lang`.** |
|
| `--require-subs` | - | Required subtitle langs; keeps **all** subs only if these exist. **Cannot combine with `--s-lang`.** |
|
||||||
| `-fs`, `--forced-subs` | off | Include forced subtitle tracks. |
|
| `-fs`, `--forced-subs` | off | Include forced subtitle tracks. |
|
||||||
| `-fsl`, `--forced-s-lang` | none | Language(s) wanted for forced subtitles; implies `-fs`. |
|
| `-fsl`, `--forced-s-lang` | none | Language(s) wanted for forced subtitles; implies `-fs`. A `-` prefix excludes. |
|
||||||
| `--exact-lang` | off | Exact matching only: `-l es-419` matches `es-419`, not `es-ES`. Applies to selection and to sort order. |
|
| `--exact-lang` | off | Exact matching only: `-l es-419` matches `es-419`, not `es-ES`. Applies to selection and to sort order. |
|
||||||
| `--sub-format` | - | Output subtitle format (`SRT`/`srt`, `VTT`/`webvtt`, `ASS`/`ssa`, `TTML`, `SMI`, ...), or `original` to keep the source format. |
|
| `--sub-format` | - | Output subtitle format (`SRT`/`srt`, `VTT`/`webvtt`, `ASS`/`ssa`, `TTML`, `SMI`, ...), or `original` to keep the source format. |
|
||||||
|
|
||||||
|
|||||||
@@ -254,6 +254,47 @@ it is used. You can override each stream type independently:
|
|||||||
unshackle dl -al en EXAMPLE 81234567
|
unshackle dl -al en EXAMPLE 81234567
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Excluding languages
|
||||||
|
|
||||||
|
Put a `-` in front of a language to remove it from the selection. This works on `-l`,
|
||||||
|
`-vl`, `-al`, `-sl`, and `-fsl`, in the same way as it works on `-w` / `--wanted`.
|
||||||
|
|
||||||
|
```shell title="Every subtitle language except Spanish"
|
||||||
|
unshackle dl -sl all,-es EXAMPLE 81234567
|
||||||
|
```
|
||||||
|
|
||||||
|
The exclusions are subtracted from the languages the flag would otherwise select, and the
|
||||||
|
order of the values does not matter. If you give only exclusions, the flag keeps its own
|
||||||
|
default: `-sl -es` is the same as `-sl all,-es`, and `-l -es` is the same as `-l orig,-es`.
|
||||||
|
|
||||||
|
```shell title="Only exclusions"
|
||||||
|
# every subtitle language except Spanish and French
|
||||||
|
unshackle dl -sl -es,-fr EXAMPLE 81234567
|
||||||
|
|
||||||
|
# the original audio language, unless it is Spanish
|
||||||
|
unshackle dl -l -es EXAMPLE 81234567
|
||||||
|
```
|
||||||
|
|
||||||
|
`-vl` and `-al` have no default of their own: they cascade to `-l`. An override that names
|
||||||
|
a language replaces `-l` for that stream type, exclusions included. An override that gives
|
||||||
|
only exclusions adds them to the ones from `-l`.
|
||||||
|
|
||||||
|
```shell title="Cascade"
|
||||||
|
# video keeps the Japanese track; only the audio drops Spanish
|
||||||
|
unshackle dl -l en,-es -vl ja EXAMPLE 81234567
|
||||||
|
```
|
||||||
|
|
||||||
|
!!! note "Rules for exclusion tokens"
|
||||||
|
- `orig` is resolved per title, so `-sl -orig` drops the subtitles in the title's
|
||||||
|
original language.
|
||||||
|
- `--exact-lang` applies to exclusions with the same matching rules as selection:
|
||||||
|
`-es` removes exactly the tracks that `es` would select. Without it, `-es` also
|
||||||
|
removes `es-419` and `es-ES`.
|
||||||
|
- Tracks with no language tag are never excluded.
|
||||||
|
- `-all` is rejected. Name the languages you do not want instead.
|
||||||
|
- The comma form is the documented style. `-sl -es` also works, but a shell can read
|
||||||
|
a lone `-es` as another option.
|
||||||
|
|
||||||
### Sort order
|
### Sort order
|
||||||
|
|
||||||
`-l` and `-sl` **select** languages: naming some removes the rest. To keep every language
|
`-l` and `-sl` **select** languages: naming some removes the rest. To keep every language
|
||||||
@@ -296,6 +337,10 @@ every available subtitle language is downloaded.
|
|||||||
unshackle dl -sl en,es EXAMPLE 81234567
|
unshackle dl -sl en,es EXAMPLE 81234567
|
||||||
```
|
```
|
||||||
|
|
||||||
|
To keep the default and drop one language, use the `-` prefix described in
|
||||||
|
[Excluding languages](#excluding-languages). `-sl all,-es` keeps every subtitle language
|
||||||
|
except Spanish, forced Spanish subtitles included.
|
||||||
|
|
||||||
### Requiring subtitles
|
### Requiring subtitles
|
||||||
|
|
||||||
`--require-subs` takes a list of languages that **must** exist. If they all exist, *all*
|
`--require-subs` takes a list of languages that **must** exist. If they all exist, *all*
|
||||||
@@ -311,7 +356,8 @@ download on the presence of a specific subtitle track.
|
|||||||
Without this flag, forced subs are dropped from selection.
|
Without this flag, forced subs are dropped from selection.
|
||||||
- `-fsl` / `--forced-s-lang`: keep forced subtitles only in these languages (implies
|
- `-fsl` / `--forced-s-lang`: keep forced subtitles only in these languages (implies
|
||||||
`-fs`). Works independently of `--s-lang`, so `-sl all -fsl en` grabs every full
|
`-fs`). Works independently of `--s-lang`, so `-sl all -fsl en` grabs every full
|
||||||
subtitle but only the English forced track.
|
subtitle but only the English forced track. It accepts exclusions too: `-fsl all,-es`
|
||||||
|
keeps every forced subtitle except the Spanish one, and `-fsl -es` means the same.
|
||||||
- `--sub-format`: set the output subtitle format, converting only when necessary.
|
- `--sub-format`: set the output subtitle format, converting only when necessary.
|
||||||
Accepts codec names/values and common aliases (`srt`, `vtt`, `ass`, `ssa`, `ttml`,
|
Accepts codec names/values and common aliases (`srt`, `vtt`, `ass`, `ssa`, `ttml`,
|
||||||
etc.), or the literal `original` to keep the source format.
|
etc.), or the literal `original` to keep the source format.
|
||||||
@@ -665,12 +711,12 @@ authoritative list.
|
|||||||
| `--range` | `-r` | Color range(s); default `SDR`. |
|
| `--range` | `-r` | Color range(s); default `SDR`. |
|
||||||
| `--channels` | `-c` | Audio channel layout. |
|
| `--channels` | `-c` | Audio channel layout. |
|
||||||
| `--noatmos` | `-naa` | Exclude Atmos audio. |
|
| `--noatmos` | `-naa` | Exclude Atmos audio. |
|
||||||
| `--lang` | `-l` | Video + audio language(s); default `orig`. |
|
| `--lang` | `-l` | Video + audio language(s); default `orig`. `-` excludes, e.g. `all,-es`. |
|
||||||
| `--a-lang` | `-al` | Audio-only language override. |
|
| `--a-lang` | `-al` | Audio-only language override. `-` excludes. |
|
||||||
| `--v-lang` | `-vl` | Video-only language override. |
|
| `--v-lang` | `-vl` | Video-only language override. `-` excludes. |
|
||||||
| `--s-lang` | `-sl` | Subtitle language(s); default `all`. |
|
| `--s-lang` | `-sl` | Subtitle language(s); default `all`. `-` excludes, e.g. `all,-es`. |
|
||||||
| `--forced-subs` | `-fs` | Include forced subtitles. |
|
| `--forced-subs` | `-fs` | Include forced subtitles. |
|
||||||
| `--forced-s-lang` | `-fsl` | Forced subtitle language(s); implies `-fs`. |
|
| `--forced-s-lang` | `-fsl` | Forced subtitle language(s); implies `-fs`. `-` excludes. |
|
||||||
| `--sub-format` | | Output subtitle format. |
|
| `--sub-format` | | Output subtitle format. |
|
||||||
| `--wanted` | `-w` | Episode/season range. |
|
| `--wanted` | `-w` | Episode/season range. |
|
||||||
| `--select-titles` | | Interactively pick episodes or films. |
|
| `--select-titles` | | Interactively pick episodes or films. |
|
||||||
|
|||||||
@@ -67,10 +67,10 @@ Subtitle selection happens through `dl` flags. The most important is language se
|
|||||||
|
|
||||||
| Flag | Purpose |
|
| Flag | Purpose |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `-sl`, `--s-lang` | Language(s) wanted for subtitles. Defaults to `all`. |
|
| `-sl`, `--s-lang` | Language(s) wanted for subtitles. Defaults to `all`. A `-` prefix excludes, e.g. `all,-es`. |
|
||||||
| `--require-subs` | Require these languages to exist; if present, download **all** subtitles. Cannot be combined with `--s-lang`. |
|
| `--require-subs` | Require these languages to exist; if present, download **all** subtitles. Cannot be combined with `--s-lang`. |
|
||||||
| `-fs`, `--forced-subs` | Include forced subtitle tracks (excluded by default). |
|
| `-fs`, `--forced-subs` | Include forced subtitle tracks (excluded by default). |
|
||||||
| `-fsl`, `--forced-s-lang` | Language(s) wanted for forced subtitles; implies `-fs`. |
|
| `-fsl`, `--forced-s-lang` | Language(s) wanted for forced subtitles; implies `-fs`. A `-` prefix excludes. |
|
||||||
| `--exact-lang` | Exact language matching, with no regional variants. |
|
| `--exact-lang` | Exact language matching, with no regional variants. |
|
||||||
| `-S`, `--subs-only` | Download only subtitle tracks. |
|
| `-S`, `--subs-only` | Download only subtitle tracks. |
|
||||||
| `-ns`, `--no-subs` | Do not download subtitle tracks at all. |
|
| `-ns`, `--no-subs` | Do not download subtitle tracks at all. |
|
||||||
|
|||||||
@@ -31,9 +31,9 @@ Common keys (a useful subset; every `dl` flag works):
|
|||||||
|
|
||||||
| Key | Type | Default | Sets |
|
| Key | Type | Default | Sets |
|
||||||
|-----|------|---------|------|
|
|-----|------|---------|------|
|
||||||
| `lang` | list | `["orig"]` | Video/audio language(s); `orig` = original language. |
|
| `lang` | list | `["orig"]` | Video/audio language(s); `orig` = original language. A `-` prefix excludes, e.g. `[all, -es]`. |
|
||||||
| `a_lang` / `v_lang` | list | `[]` | Audio- / video-only language override. |
|
| `a_lang` / `v_lang` | list | `[]` | Audio- / video-only language override. A `-` prefix excludes. |
|
||||||
| `s_lang` | list | `["all"]` | Subtitle language(s). |
|
| `s_lang` | list | `["all"]` | Subtitle language(s). A `-` prefix excludes, e.g. `[all, -es]`. |
|
||||||
| `quality` | list | `[]` (best) | Resolution(s), e.g. `[1080]`. |
|
| `quality` | list | `[]` (best) | Resolution(s), e.g. `[1080]`. |
|
||||||
| `vcodec` | list | `[]` (any) | Video codec(s), e.g. `[H265]`. |
|
| `vcodec` | list | `[]` (any) | Video codec(s), e.g. `[H265]`. |
|
||||||
| `acodec` | list | `[]` (any) | Audio codec(s), e.g. `[EC3]`. |
|
| `acodec` | list | `[]` (any) | Audio codec(s), e.g. `[EC3]`. |
|
||||||
@@ -41,7 +41,7 @@ Common keys (a useful subset; every `dl` flag works):
|
|||||||
| `channels` | float | *(unset)* | Audio channels, e.g. `6` for 5.1. |
|
| `channels` | float | *(unset)* | Audio channels, e.g. `6` for 5.1. |
|
||||||
| `sub_format` | str | *(unset)* | Convert subtitles to this format (`srt`, `vtt`, `original`, ...). |
|
| `sub_format` | str | *(unset)* | Convert subtitles to this format (`srt`, `vtt`, `original`, ...). |
|
||||||
| `forced_subs` | bool | `false` | Include forced subtitle tracks. |
|
| `forced_subs` | bool | `false` | Include forced subtitle tracks. |
|
||||||
| `forced_s_lang` | list | `[]` | Forced subtitle language(s); implies `forced_subs`. |
|
| `forced_s_lang` | list | `[]` | Forced subtitle language(s); implies `forced_subs`. A `-` prefix excludes. |
|
||||||
| `no_subs` / `no_audio` / `no_chapters` | bool | `false` | Skip that track type. |
|
| `no_subs` / `no_audio` / `no_chapters` | bool | `false` | Skip that track type. |
|
||||||
| `downloads` | int | `1` | Tracks downloaded concurrently. |
|
| `downloads` | int | `1` | Tracks downloaded concurrently. |
|
||||||
| `workers` | int | *(downloader default)* | Threads per track. |
|
| `workers` | int | *(downloader default)* | Threads per track. |
|
||||||
@@ -61,6 +61,18 @@ dl:
|
|||||||
downloads: 2
|
downloads: 2
|
||||||
```
|
```
|
||||||
|
|
||||||
|
!!! note "Language exclusions in YAML"
|
||||||
|
The language keys accept the same `-` exclusion tokens as the command line. Give them
|
||||||
|
as a list or as one comma-separated string.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
dl:
|
||||||
|
s_lang: [all, -es]
|
||||||
|
a_lang: "orig,-es"
|
||||||
|
```
|
||||||
|
|
||||||
|
See [Excluding languages](../../guide/downloading.md#excluding-languages) for the rules.
|
||||||
|
|
||||||
!!! note "Per-service overrides"
|
!!! note "Per-service overrides"
|
||||||
A `dl` block nested under a service in [`services`](services.md) takes the same keys and
|
A `dl` block nested under a service in [`services`](services.md) takes the same keys and
|
||||||
applies to that service only.
|
applies to that service only.
|
||||||
|
|||||||
@@ -7,7 +7,13 @@ import click
|
|||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from unshackle.core.tracks.subtitle import Subtitle
|
from unshackle.core.tracks.subtitle import Subtitle
|
||||||
from unshackle.core.utils.click_types import QUALITY_LIST, SLOW_DELAY_RANGE, SeasonRange, SubtitleCodecChoice
|
from unshackle.core.utils.click_types import (
|
||||||
|
LANGUAGE_RANGE,
|
||||||
|
QUALITY_LIST,
|
||||||
|
SLOW_DELAY_RANGE,
|
||||||
|
SeasonRange,
|
||||||
|
SubtitleCodecChoice,
|
||||||
|
)
|
||||||
|
|
||||||
choice = SubtitleCodecChoice(Subtitle.Codec)
|
choice = SubtitleCodecChoice(Subtitle.Codec)
|
||||||
|
|
||||||
@@ -60,6 +66,23 @@ def test_slow_delay_range_accepts_bool(value, expected):
|
|||||||
assert SLOW_DELAY_RANGE.convert(value, None, None) == expected
|
assert SLOW_DELAY_RANGE.convert(value, None, None) == expected
|
||||||
|
|
||||||
|
|
||||||
|
# --- LanguageRange ----------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"value,expected",
|
||||||
|
[
|
||||||
|
("all,-es", ["all", "-es"]),
|
||||||
|
("-es", ["-es"]),
|
||||||
|
("all, -es ; -fr", ["all", "-es", "-fr"]),
|
||||||
|
(["all", "-es"], ["all", "-es"]),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_language_range_passes_exclusion_tokens_through(value, expected):
|
||||||
|
"""The '-' prefix is resolved at filter time, so the type must keep the token intact."""
|
||||||
|
assert LANGUAGE_RANGE.convert(value) == expected
|
||||||
|
|
||||||
|
|
||||||
# --- SeasonRange ------------------------------------------------------------
|
# --- SeasonRange ------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,80 @@
|
|||||||
|
"""Tests for the `-` exclusion prefix on the language flags (-l, -vl, -al, -sl, -fsl)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from langcodes import Language
|
||||||
|
|
||||||
|
from unshackle.core.utilities import excluded_language_tags, partition_exclusions
|
||||||
|
from unshackle.core.utils.click_types import LANGUAGE_RANGE
|
||||||
|
|
||||||
|
# --- partition_exclusions ---------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"tokens,expected",
|
||||||
|
[
|
||||||
|
(["all", "-es"], (["all"], ["es"])),
|
||||||
|
(["-es"], ([], ["es"])),
|
||||||
|
(["en", "ja"], (["en", "ja"], [])),
|
||||||
|
(["-es", "-fr", "en"], (["en"], ["es", "fr"])),
|
||||||
|
([], ([], [])),
|
||||||
|
(None, ([], [])),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_partition_splits_on_the_leading_dash(tokens, expected):
|
||||||
|
assert partition_exclusions(tokens) == expected
|
||||||
|
|
||||||
|
|
||||||
|
def test_partition_keeps_the_written_order_and_drops_repeats():
|
||||||
|
assert partition_exclusions(["ja", "en", "ja", "-es", "-es"]) == (["ja", "en"], ["es"])
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("token", ["-", " - ", "", " "])
|
||||||
|
def test_partition_skips_tokens_with_no_value(token):
|
||||||
|
assert partition_exclusions(["en", token]) == (["en"], [])
|
||||||
|
|
||||||
|
|
||||||
|
def test_partition_trims_whitespace_around_a_token():
|
||||||
|
assert partition_exclusions([" en ", "- es "]) == (["en"], ["es"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_partition_reads_the_tokens_language_range_produces():
|
||||||
|
assert partition_exclusions(LANGUAGE_RANGE.convert("all,-es;-fr")) == (["all"], ["es", "fr"])
|
||||||
|
|
||||||
|
|
||||||
|
# --- excluded_language_tags ---------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_close_match_excludes_the_regional_variants():
|
||||||
|
assert excluded_language_tags(["es"], ["es-419", "es-ES", "en-US"]) == {"es-419", "es-ES"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_exact_match_prefers_the_string_equal_tag():
|
||||||
|
# same string preference as selection: an exact 'es' only takes 'es' when it exists
|
||||||
|
assert excluded_language_tags(["es"], ["es", "es-ES", "es-419"], exact=True) == {"es"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_exact_match_falls_back_to_the_paradigm_variant():
|
||||||
|
# no literal 'es' available, so 'es' resolves the way an exact '-sl es' selection would
|
||||||
|
assert excluded_language_tags(["es"], ["es-ES", "es-419"], exact=True) == {"es-ES"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_exact_match_keeps_an_unnamed_regional_variant():
|
||||||
|
assert excluded_language_tags(["es-ES"], ["es-ES", "es-419"], exact=True) == {"es-ES"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_language_objects_are_accepted():
|
||||||
|
assert excluded_language_tags(["es"], [Language.get("es")]) == {"es"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_untagged_track_is_never_excluded():
|
||||||
|
assert excluded_language_tags(["es"], [None, "en-US"]) == set()
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_exclusions_excludes_nothing():
|
||||||
|
assert excluded_language_tags([], ["es-419", "en-US"]) == set()
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_unmatched_language_is_kept():
|
||||||
|
assert excluded_language_tags(["es", "fr"], ["en-US", "ja-JP"]) == set()
|
||||||
@@ -150,3 +150,24 @@ def test_language_range_yaml_shapes(value, expected):
|
|||||||
ctx = make_ctx()
|
ctx = make_ctx()
|
||||||
apply_service_dl_overrides(ctx, {"v_lang": value}, log)
|
apply_service_dl_overrides(ctx, {"v_lang": value}, log)
|
||||||
assert ctx.params["v_lang"] == expected
|
assert ctx.params["v_lang"] == expected
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"value,expected",
|
||||||
|
[
|
||||||
|
("all,-es", ["all", "-es"]),
|
||||||
|
("-es", ["-es"]),
|
||||||
|
(["all", "-es"], ["all", "-es"]),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_language_exclusion_tokens_survive_the_override_convert(value, expected):
|
||||||
|
"""A service dl override is re-converted, so its exclusion tokens must reach the filter stage intact."""
|
||||||
|
ctx = make_ctx()
|
||||||
|
apply_service_dl_overrides(ctx, {"v_lang": value}, log)
|
||||||
|
assert ctx.params["v_lang"] == expected
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_exclusion_beats_service_config():
|
||||||
|
ctx = make_ctx(["-vl", "-es"])
|
||||||
|
apply_service_dl_overrides(ctx, {"v_lang": ["en"]}, log)
|
||||||
|
assert ctx.params["v_lang"] == ["-es"]
|
||||||
|
|||||||
+111
-9
@@ -74,6 +74,7 @@ from unshackle.core.tracks.hybrid import Hybrid
|
|||||||
from unshackle.core.tracks.track import assert_fragments_decrypted, has_encrypted_sample_entry
|
from unshackle.core.tracks.track import assert_fragments_decrypted, has_encrypted_sample_entry
|
||||||
from unshackle.core.utilities import (
|
from unshackle.core.utilities import (
|
||||||
as_requested,
|
as_requested,
|
||||||
|
excluded_language_tags,
|
||||||
find_font_with_fallbacks,
|
find_font_with_fallbacks,
|
||||||
find_missing_langs,
|
find_missing_langs,
|
||||||
get_debug_logger,
|
get_debug_logger,
|
||||||
@@ -83,6 +84,7 @@ from unshackle.core.utilities import (
|
|||||||
is_exact_match,
|
is_exact_match,
|
||||||
keep_forced_subtitle,
|
keep_forced_subtitle,
|
||||||
log_event,
|
log_event,
|
||||||
|
partition_exclusions,
|
||||||
resolve_sort_langs,
|
resolve_sort_langs,
|
||||||
suggest_font_packages,
|
suggest_font_packages,
|
||||||
time_elapsed_since,
|
time_elapsed_since,
|
||||||
@@ -532,7 +534,7 @@ class dl:
|
|||||||
"--lang",
|
"--lang",
|
||||||
type=LANGUAGE_RANGE,
|
type=LANGUAGE_RANGE,
|
||||||
default="orig",
|
default="orig",
|
||||||
help="Language(s) wanted for Video and Audio (comma-separated). Use 'orig' to select the original language, e.g. 'orig,en' for both original and English.",
|
help="Language(s) wanted for Video and Audio (comma-separated). Use 'orig' to select the original language, e.g. 'orig,en' for both original and English. Prefix a value with '-' to exclude it, e.g. 'all,-es'.",
|
||||||
)
|
)
|
||||||
@click.option(
|
@click.option(
|
||||||
"--latest-episode",
|
"--latest-episode",
|
||||||
@@ -545,16 +547,22 @@ class dl:
|
|||||||
"--v-lang",
|
"--v-lang",
|
||||||
type=LANGUAGE_RANGE,
|
type=LANGUAGE_RANGE,
|
||||||
default=[],
|
default=[],
|
||||||
help="Language wanted for Video. You would use this if the video language doesn't match the audio.",
|
help="Language wanted for Video. You would use this if the video language doesn't match the audio. Prefix a value with '-' to exclude it, e.g. 'all,-es'.",
|
||||||
)
|
)
|
||||||
@click.option(
|
@click.option(
|
||||||
"-al",
|
"-al",
|
||||||
"--a-lang",
|
"--a-lang",
|
||||||
type=LANGUAGE_RANGE,
|
type=LANGUAGE_RANGE,
|
||||||
default=[],
|
default=[],
|
||||||
help="Language wanted for Audio, overrides -l/--lang for audio tracks.",
|
help="Language wanted for Audio, overrides -l/--lang for audio tracks. Prefix a value with '-' to exclude it, e.g. 'all,-es'.",
|
||||||
|
)
|
||||||
|
@click.option(
|
||||||
|
"-sl",
|
||||||
|
"--s-lang",
|
||||||
|
type=LANGUAGE_RANGE,
|
||||||
|
default=["all"],
|
||||||
|
help="Language wanted for Subtitles. Prefix a value with '-' to exclude it, e.g. 'all,-es'.",
|
||||||
)
|
)
|
||||||
@click.option("-sl", "--s-lang", type=LANGUAGE_RANGE, default=["all"], help="Language wanted for Subtitles.")
|
|
||||||
@click.option(
|
@click.option(
|
||||||
"--require-subs",
|
"--require-subs",
|
||||||
type=LANGUAGE_RANGE,
|
type=LANGUAGE_RANGE,
|
||||||
@@ -567,7 +575,7 @@ class dl:
|
|||||||
"--forced-s-lang",
|
"--forced-s-lang",
|
||||||
type=LANGUAGE_RANGE,
|
type=LANGUAGE_RANGE,
|
||||||
default=[],
|
default=[],
|
||||||
help="Languages wanted for forced subtitles, implies -fs. Keeps forced subs only in these languages.",
|
help="Languages wanted for forced subtitles, implies -fs. Keeps forced subs only in these languages. Prefix a value with '-' to exclude it, e.g. 'all,-es'.",
|
||||||
)
|
)
|
||||||
@click.option(
|
@click.option(
|
||||||
"--exact-lang",
|
"--exact-lang",
|
||||||
@@ -1340,14 +1348,51 @@ class dl:
|
|||||||
self._remote_service = service if self.server_cdm else None
|
self._remote_service = service if self.server_cdm else None
|
||||||
start_time = time.time()
|
start_time = time.time()
|
||||||
|
|
||||||
if forced_s_lang:
|
lang, lang_excl = partition_exclusions(lang)
|
||||||
|
v_lang, v_lang_excl = partition_exclusions(v_lang)
|
||||||
|
a_lang, a_lang_excl = partition_exclusions(a_lang)
|
||||||
|
s_lang, s_lang_excl = partition_exclusions(s_lang)
|
||||||
|
forced_s_lang, fsl_excl = partition_exclusions(forced_s_lang)
|
||||||
|
for flag, excludes in (
|
||||||
|
("-l/--lang", lang_excl),
|
||||||
|
("-vl/--v-lang", v_lang_excl),
|
||||||
|
("-al/--a-lang", a_lang_excl),
|
||||||
|
("-sl/--s-lang", s_lang_excl),
|
||||||
|
("-fsl/--forced-s-lang", fsl_excl),
|
||||||
|
):
|
||||||
|
if any(token.lower() == "all" for token in excludes):
|
||||||
|
self.log.error(f"{flag}: 'all' cannot be excluded. Name the languages to exclude instead.")
|
||||||
|
sys.exit(1)
|
||||||
|
# exclusion-only use falls back to the flag's own default; -vl/-al have none, they cascade to -l
|
||||||
|
if lang_excl and not lang:
|
||||||
|
lang = ["orig"]
|
||||||
|
if s_lang_excl and not s_lang:
|
||||||
|
s_lang = ["all"]
|
||||||
|
# an override that names languages replaces -l entirely, so it must not inherit its exclusions
|
||||||
|
video_excl = list(dict.fromkeys(v_lang_excl + ([] if v_lang else lang_excl)))
|
||||||
|
audio_excl = list(dict.fromkeys(a_lang_excl + ([] if a_lang else lang_excl)))
|
||||||
|
|
||||||
|
ctx = getattr(service, "ctx", None)
|
||||||
|
parent_params = ctx.parent.params if ctx and ctx.parent else None
|
||||||
|
if parent_params is not None:
|
||||||
|
# services read the raw ctx params, they must never see a '-xx' token
|
||||||
|
for name, includes, excludes in (
|
||||||
|
("lang", lang, lang_excl),
|
||||||
|
("v_lang", v_lang, v_lang_excl),
|
||||||
|
("a_lang", a_lang, a_lang_excl),
|
||||||
|
("s_lang", s_lang, s_lang_excl),
|
||||||
|
("forced_s_lang", forced_s_lang, fsl_excl),
|
||||||
|
):
|
||||||
|
if excludes and name in parent_params:
|
||||||
|
parent_params[name] = includes
|
||||||
|
|
||||||
|
if forced_s_lang or fsl_excl:
|
||||||
# services read forced_subs from raw ctx params, so the implication must be pushed there too
|
# services read forced_subs from raw ctx params, so the implication must be pushed there too
|
||||||
forced_subs = True
|
forced_subs = True
|
||||||
if "all" in forced_s_lang:
|
if "all" in forced_s_lang:
|
||||||
forced_s_lang = []
|
forced_s_lang = []
|
||||||
ctx = getattr(service, "ctx", None)
|
if parent_params is not None:
|
||||||
if ctx and ctx.parent:
|
parent_params["forced_subs"] = True
|
||||||
ctx.parent.params["forced_subs"] = True
|
|
||||||
|
|
||||||
if skip_dl:
|
if skip_dl:
|
||||||
DOWNLOAD_LICENCE_ONLY.set()
|
DOWNLOAD_LICENCE_ONLY.set()
|
||||||
@@ -1849,6 +1894,18 @@ class dl:
|
|||||||
)
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
|
song_excl = [
|
||||||
|
str(song.language) if t == "orig" else t for t in audio_excl if t != "orig" or song.language
|
||||||
|
]
|
||||||
|
if song_excl:
|
||||||
|
drop = excluded_language_tags(song_excl, [t.language for t in song.tracks.audio], exact_lang)
|
||||||
|
song.tracks.select_audio(lambda x: str(x.language) not in drop)
|
||||||
|
if not song.tracks.audio:
|
||||||
|
self.log.error(
|
||||||
|
f"Every Audio Track for {song.name} was excluded by -{', -'.join(song_excl)}..."
|
||||||
|
)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
audio_languages = a_lang or lang
|
audio_languages = a_lang or lang
|
||||||
if audio_languages:
|
if audio_languages:
|
||||||
processed_lang = []
|
processed_lang = []
|
||||||
@@ -2500,6 +2557,23 @@ class dl:
|
|||||||
title.tracks.chapters = []
|
title.tracks.chapters = []
|
||||||
|
|
||||||
with console.status("Selecting tracks...", spinner="dots"):
|
with console.status("Selecting tracks...", spinner="dots"):
|
||||||
|
|
||||||
|
def resolve_excludes(excludes: list[str], _title: Title_T = title) -> list[str]:
|
||||||
|
"""Resolve 'orig' against this title. An unresolvable 'orig' excludes nothing."""
|
||||||
|
resolved: list[str] = []
|
||||||
|
for token in excludes:
|
||||||
|
value = str(_title.language) if token == "orig" else token
|
||||||
|
if token == "orig" and not _title.language:
|
||||||
|
continue
|
||||||
|
if value not in resolved:
|
||||||
|
resolved.append(value)
|
||||||
|
return resolved
|
||||||
|
|
||||||
|
video_excl_r = resolve_excludes(video_excl)
|
||||||
|
audio_excl_r = resolve_excludes(audio_excl)
|
||||||
|
s_excl_r = resolve_excludes(s_lang_excl)
|
||||||
|
fsl_excl_r = resolve_excludes(fsl_excl)
|
||||||
|
|
||||||
if isinstance(title, (Movie, Episode)):
|
if isinstance(title, (Movie, Episode)):
|
||||||
# filter video tracks
|
# filter video tracks
|
||||||
if keep_videos and vcodec:
|
if keep_videos and vcodec:
|
||||||
@@ -2546,6 +2620,15 @@ class dl:
|
|||||||
self.log.error(f"No Video Track in {vbitrate_min}-{vbitrate_max}kbps range...")
|
self.log.error(f"No Video Track in {vbitrate_min}-{vbitrate_max}kbps range...")
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
|
if keep_videos and video_excl_r:
|
||||||
|
drop = excluded_language_tags(
|
||||||
|
video_excl_r, [t.language for t in title.tracks.videos], exact_lang
|
||||||
|
)
|
||||||
|
title.tracks.select_video(lambda x: str(x.language) not in drop)
|
||||||
|
if not title.tracks.videos:
|
||||||
|
self.log.error(f"Every Video Track was excluded by -{', -'.join(video_excl_r)}...")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
effective_video_lang = v_lang or lang
|
effective_video_lang = v_lang or lang
|
||||||
video_languages = [lang for lang in effective_video_lang if lang != "best"]
|
video_languages = [lang for lang in effective_video_lang if lang != "best"]
|
||||||
video_multi_lang = (
|
video_multi_lang = (
|
||||||
@@ -2826,6 +2909,12 @@ class dl:
|
|||||||
fsl = [t for t in forced_s_lang if t != "orig"]
|
fsl = [t for t in forced_s_lang if t != "orig"]
|
||||||
if "orig" in forced_s_lang and title.language:
|
if "orig" in forced_s_lang and title.language:
|
||||||
fsl.append(str(title.language))
|
fsl.append(str(title.language))
|
||||||
|
if keep_subtitles and s_excl_r:
|
||||||
|
# applies to forced subs too: an excluded language is unwanted in any form
|
||||||
|
drop = excluded_language_tags(
|
||||||
|
s_excl_r, [t.language for t in title.tracks.subtitles], exact_lang
|
||||||
|
)
|
||||||
|
title.tracks.select_subtitles(lambda x: str(x.language) not in drop)
|
||||||
if keep_subtitles and require_subs:
|
if keep_subtitles and require_subs:
|
||||||
missing_langs = [
|
missing_langs = [
|
||||||
lang
|
lang
|
||||||
@@ -2893,6 +2982,11 @@ class dl:
|
|||||||
)
|
)
|
||||||
elif not forced_subs:
|
elif not forced_subs:
|
||||||
title.tracks.select_subtitles(lambda x: not x.forced)
|
title.tracks.select_subtitles(lambda x: not x.forced)
|
||||||
|
if fsl_excl_r:
|
||||||
|
drop = excluded_language_tags(
|
||||||
|
fsl_excl_r, [t.language for t in title.tracks.subtitles if t.forced], exact_lang
|
||||||
|
)
|
||||||
|
title.tracks.select_subtitles(lambda x: not (x.forced and str(x.language) in drop))
|
||||||
|
|
||||||
# filter audio tracks
|
# filter audio tracks
|
||||||
# might have no audio tracks if part of the video, e.g. transport stream hls
|
# might have no audio tracks if part of the video, e.g. transport stream hls
|
||||||
@@ -2929,6 +3023,14 @@ class dl:
|
|||||||
if not title.tracks.audio:
|
if not title.tracks.audio:
|
||||||
self.log.error(f"No Audio Track in {abitrate_min}-{abitrate_max}kbps range...")
|
self.log.error(f"No Audio Track in {abitrate_min}-{abitrate_max}kbps range...")
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
if audio_excl_r:
|
||||||
|
drop = excluded_language_tags(
|
||||||
|
audio_excl_r, [t.language for t in title.tracks.audio], exact_lang
|
||||||
|
)
|
||||||
|
title.tracks.select_audio(lambda x: str(x.language) not in drop)
|
||||||
|
if not title.tracks.audio:
|
||||||
|
self.log.error(f"Every Audio Track was excluded by -{', -'.join(audio_excl_r)}...")
|
||||||
|
sys.exit(1)
|
||||||
audio_languages = a_lang or lang
|
audio_languages = a_lang or lang
|
||||||
if audio_languages:
|
if audio_languages:
|
||||||
processed_lang = []
|
processed_lang = []
|
||||||
|
|||||||
@@ -164,6 +164,49 @@ def is_exact_match(language: Union[str, Language], languages: Sequence[Union[str
|
|||||||
return closest_match(language, list(map(str, languages)))[1] <= LANGUAGE_EXACT_DISTANCE
|
return closest_match(language, list(map(str, languages)))[1] <= LANGUAGE_EXACT_DISTANCE
|
||||||
|
|
||||||
|
|
||||||
|
def partition_exclusions(tokens: Optional[Sequence[str]]) -> tuple[list[str], list[str]]:
|
||||||
|
"""
|
||||||
|
Split selection tokens into wanted values and values excluded with a leading '-'.
|
||||||
|
|
||||||
|
Both sides keep the order they were written in and drop later repeats. A bare '-'
|
||||||
|
carries no value and is skipped.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
>>> partition_exclusions(["all", "-es"])
|
||||||
|
(['all'], ['es'])
|
||||||
|
"""
|
||||||
|
includes: list[str] = []
|
||||||
|
excludes: list[str] = []
|
||||||
|
for raw in tokens or []:
|
||||||
|
token = str(raw).strip()
|
||||||
|
if not token or token == "-":
|
||||||
|
continue
|
||||||
|
target = includes
|
||||||
|
if token.startswith("-"):
|
||||||
|
token = token[1:].strip()
|
||||||
|
if not token:
|
||||||
|
continue
|
||||||
|
target = excludes
|
||||||
|
if token not in target:
|
||||||
|
target.append(token)
|
||||||
|
return includes, excludes
|
||||||
|
|
||||||
|
|
||||||
|
def excluded_language_tags(
|
||||||
|
excludes: Sequence[str],
|
||||||
|
available: Sequence[Union[str, Language, None]],
|
||||||
|
exact: bool = False,
|
||||||
|
) -> set[str]:
|
||||||
|
"""
|
||||||
|
The language tags out of `available` that the exclusion tokens remove.
|
||||||
|
|
||||||
|
Matches through matching_languages so an exclusion drops exactly the tracks the same
|
||||||
|
token would select, including its exact-mode string preference. Untagged tracks are
|
||||||
|
never excluded.
|
||||||
|
"""
|
||||||
|
return {tag for token in excludes for tag in matching_languages(token, available, exact)}
|
||||||
|
|
||||||
|
|
||||||
def keep_forced_subtitle(
|
def keep_forced_subtitle(
|
||||||
forced: bool,
|
forced: bool,
|
||||||
language: Union[str, Language],
|
language: Union[str, Language],
|
||||||
|
|||||||
Reference in New Issue
Block a user