mirror of
https://github.com/unshackle-dl/unshackle.git
synced 2026-11-17 06:25:07 +00:00
Compare commits
5
Commits
65e807bf5f
...
96c3840b82
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
96c3840b82 | ||
|
|
86f26c31d5 | ||
|
|
d5a8ed2a39 | ||
|
|
15e46d8bcc | ||
|
|
f9c1c3d390 |
@@ -135,6 +135,9 @@ Services that require a login read either **cookies** or **credentials**.
|
||||
rows a text editor broke. unshackle refuses a JSON export. Export the cookies
|
||||
again in Netscape format.
|
||||
|
||||
unshackle sends each cookie only to the host and path that its row names, as a
|
||||
browser does. A domain that starts with a dot also covers its subdomains.
|
||||
|
||||
- **Credentials**: store a username and password per service in your config:
|
||||
|
||||
```yaml title="unshackle.yaml"
|
||||
|
||||
@@ -105,8 +105,8 @@ Every variable below is valid in both output and folder templates. unshackle tak
|
||||
| `title_type` | Media kind of the title | `movie`, `series`, `music` |
|
||||
| `year` | Release year | `2024` |
|
||||
| `source` | Service tag / class name (empty with `--no-source`) | `EXAMPLE` |
|
||||
| `quality` | Resolution with scan suffix | `1080p`, `2160p`, `576i` |
|
||||
| `resolution` | Resolution number only | `1080` |
|
||||
| `quality` | Standard resolution with scan suffix | `1080p`, `2160p`, `576i` |
|
||||
| `resolution` | Standard resolution number only | `1080` |
|
||||
| `video` | Video codec | `H.264`, `H.265` |
|
||||
| `hdr` | Dynamic-range label | `HDR`, `HDR10P`, `DV`, `DV.HDR`, `DV.HDR10P`, `HLG` |
|
||||
| `hfr` | High frame rate marker (frame rate > 30) | `HFR` |
|
||||
@@ -115,6 +115,15 @@ Every variable below is valid in both output and folder templates. unshackle tak
|
||||
| `repack` | `REPACK` when the `--repack` flag is used | `REPACK` |
|
||||
| `lang_tag` | Result of your `language_tags` rules, if configured | `MULTi` |
|
||||
|
||||
!!! note "`quality` and `resolution` use the standard height"
|
||||
unshackle names a video by the nearest standard height (4320, 2160, 1440, 1080, 720, 576, 540,
|
||||
480, 360 or 240) when it is within 10%. It measures the short side and the height that a 16:9
|
||||
picture of the same width has, and uses the larger one. So a cropped 1920x800, a padded
|
||||
1920x1088 and a 1080x1920 portrait video are all `1080p`. A short side within 16 lines of
|
||||
2160, 1440 or 1080 gives that height, so an ultrawide 2560x1080 is `1080p`. A size with no
|
||||
standard height near it, such as 1600x900, keeps its own height. `dl -q` and the
|
||||
`dl --export` file name use the same rule.
|
||||
|
||||
!!! note "How `language_tags` rules match"
|
||||
unshackle tries the rules in order, and they compare languages with the same [matching rules](downloading.md#exact-vs-fuzzy-matching) as `-l`: a rule condition of `en` matches `en-US`, `en-GB`, and any other `en-*` form, but `es-419` matches only `es-419`. Within a single rule, **all** conditions must match (AND logic) for it to apply. The first rule that fully matches wins. If none match, unshackle removes `{lang_tag?}` from the filename cleanly.
|
||||
|
||||
|
||||
@@ -441,6 +441,7 @@ subtitle:
|
||||
| `sidecar_format` | `srt` | a format name (e.g. `srt`, `vtt`) or `original` | Format used for sidecar files. |
|
||||
| `group_by` | `type` | `type`, `language` | Whether all tracks of one type stay together, or each language stays next to its own variants. |
|
||||
| `language_priority` | *(unset)* | a list of languages | Languages to sort to the top. The rest keep the default order. Nothing is removed. |
|
||||
| `dedupe_codecs` | *(unset)* | a list of codecs, e.g. `[VTT, STPP, WVTT]` | Keep one copy of a subtitle that the service sends in several codecs: the earliest listed codec wins. |
|
||||
|
||||
### Which settings to change
|
||||
|
||||
@@ -454,6 +455,7 @@ only when you want something the defaults do not do:
|
||||
| Music and lyric lines stripped along with the sound effects | `sdh_method: filter-subs` |
|
||||
| Lyric lines kept | `sdh_method: subby`, or install SubtitleEdit |
|
||||
| No stripped copies made at all | `strip_sdh: false` |
|
||||
| One copy of each subtitle when the service sends the same one in several codecs | `dedupe_codecs: [VTT, STPP, WVTT]` |
|
||||
| One backend for every conversion, whatever you install | `conversion_method: <backend>` |
|
||||
|
||||
Two keys read as if they control markup, and do not:
|
||||
|
||||
@@ -194,6 +194,7 @@ in or writes them as sidecar files. See [Subtitles](../../guide/subtitles.md) fo
|
||||
| `type_priority` | list | *(unset)* | Ordered ranking of subtitle types (`forced`, `normal`, `sdh`; CC counts as SDH). Types you leave out fall to the end. |
|
||||
| `group_by` | str | `"type"` | Which key groups the subtitle tracks. `type` puts all forced tracks together, then all normal, then all SDH. `language` keeps each language next to its own variants. |
|
||||
| `language_priority` | list | *(unset)* | Languages to sort to the top, in the order you give. The rest keep the default order: original language first, then alphabetical. Nothing is removed. |
|
||||
| `dedupe_codecs` | list | *(unset)* | Subtitle codecs in order of preference, by value or name, for example `[VTT, STPP, WVTT]`. When tracks of the same language, forced/SDH/CC type and name come in several listed codecs, unshackle drops the ones in a later codec. It always keeps codecs you leave out. Unset keeps every track. |
|
||||
|
||||
```yaml
|
||||
subtitle:
|
||||
|
||||
@@ -134,8 +134,9 @@ Put the more specific rules first, because unshackle stops at the first match.
|
||||
Matching is exact, and `hdr` is a composite label for Dolby Vision titles: a DV title with
|
||||
an HDR10 base layer gives `DV.HDR`, and with an HDR10+ base layer `DV.HDR10P`. Plain `DV`
|
||||
does **not** match those. Name every variant you want, as the example above does. Note
|
||||
also that HDR10 gives `HDR`, not `HDR10`. The full set of values is `HDR`, `HDR10P`, `DV`,
|
||||
`DV.HDR`, `DV.HDR10P`, and `HLG`, as listed in the
|
||||
also that HDR10 gives `HDR`, not `HDR10`, and so does a PQ video track with no HDR10
|
||||
metadata. The full set of values is `HDR`, `HDR10P`, `DV`, `DV.HDR`, `DV.HDR10P`, and
|
||||
`HLG`, as listed in the
|
||||
[variable table](../../guide/output-and-naming.md#template-variables).
|
||||
|
||||
### Operators
|
||||
|
||||
@@ -76,7 +76,6 @@ def test_codec_names_renames_video_and_each_hdr_part(monkeypatch: pytest.MonkeyP
|
||||
height=2160,
|
||||
format="HEVC",
|
||||
scan_type="Progressive",
|
||||
other_display_aspect_ratio=["16:9"],
|
||||
hdr_format_commercial="HDR10+",
|
||||
hdr_format="Dolby Vision, Version 1.0, Profile 8.1",
|
||||
transfer_characteristics=None,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Tests for the HDR tag in the filename context when MediaInfo misses HDR10+."""
|
||||
"""Tests for the HDR tag in the filename context when MediaInfo misses HDR10+ or the HDR format."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -22,7 +22,9 @@ def clean_config(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(config, "codec_names", {})
|
||||
|
||||
|
||||
def hdr_tag(commercial: str, range_: Video.Range) -> str:
|
||||
def hdr_tag(
|
||||
commercial: str, range_: Video.Range, transfer: str = "PQ", hdr_format: str = "SMPTE ST 2086", profile: str = ""
|
||||
) -> str:
|
||||
movie = Movie(id_="movie-0001", service=DummyService, name="The Film", year=2024, language="en")
|
||||
movie.tracks = Tracks(Video(url="https://example.test/v.mpd", language="en", codec=Video.Codec.AV1, range_=range_))
|
||||
video = types.SimpleNamespace(
|
||||
@@ -30,10 +32,10 @@ def hdr_tag(commercial: str, range_: Video.Range) -> str:
|
||||
height=1080,
|
||||
format="AV1",
|
||||
scan_type="Progressive",
|
||||
other_display_aspect_ratio=["16:9"],
|
||||
hdr_format_commercial=commercial,
|
||||
hdr_format="SMPTE ST 2086",
|
||||
transfer_characteristics="PQ",
|
||||
hdr_format=hdr_format,
|
||||
hdr_format_profile=profile,
|
||||
transfer_characteristics=transfer,
|
||||
transfer_characteristics_original=None,
|
||||
frame_rate="23.976",
|
||||
)
|
||||
@@ -51,3 +53,31 @@ def test_an_hdr10_track_keeps_the_hdr_tag() -> None:
|
||||
|
||||
def test_mediainfo_hdr10p_is_unchanged() -> None:
|
||||
assert hdr_tag("HDR10 / HDR10+", Video.Range.HDR10) == "HDR10P"
|
||||
|
||||
|
||||
def test_pq_without_an_hdr_format_gets_the_hdr_tag() -> None:
|
||||
assert hdr_tag("", Video.Range.HDR10) == "HDR"
|
||||
|
||||
|
||||
def test_hlg_without_an_hdr_format_keeps_the_hlg_tag() -> None:
|
||||
assert hdr_tag("", Video.Range.HLG, transfer="HLG") == "HLG"
|
||||
|
||||
|
||||
def test_sdr_gets_no_hdr_tag() -> None:
|
||||
assert hdr_tag("", Video.Range.SDR, transfer="BT.709") == ""
|
||||
|
||||
|
||||
def test_a_profile_8_dv_track_with_an_unnamed_pq_base_gets_dv_hdr() -> None:
|
||||
assert hdr_tag("Dolby Vision", Video.Range.DV, hdr_format="Dolby Vision", profile="dvhe.08") == "DV.HDR"
|
||||
|
||||
|
||||
def test_a_profile_5_dv_track_stays_dv() -> None:
|
||||
assert hdr_tag("Dolby Vision", Video.Range.DV, hdr_format="Dolby Vision", profile="dvhe.05") == "DV"
|
||||
|
||||
|
||||
def test_a_profile_8_4_dv_track_with_an_hlg_base_stays_dv() -> None:
|
||||
assert hdr_tag("Dolby Vision", Video.Range.DV, transfer="HLG", hdr_format="Dolby Vision", profile="dvhe.08") == "DV"
|
||||
|
||||
|
||||
def test_a_dv_track_with_a_named_hdr10_base_keeps_dv_hdr() -> None:
|
||||
assert hdr_tag("HDR10", Video.Range.DV, hdr_format="Dolby Vision", profile="dvhe.08") == "DV.HDR"
|
||||
|
||||
@@ -418,7 +418,7 @@ def test_a_conversion_run_removes_its_rpus_so_the_next_base_makes_its_own(
|
||||
monkeypatch.setattr(Hybrid, "level_6", level_6)
|
||||
monkeypatch.setattr(hybrid_mod.console, "print", lambda *a, **k: None)
|
||||
|
||||
Hybrid([SimpleNamespace(id="b", path=base, height=2160, range=Video.Range.HDR10P)], "TEST")
|
||||
Hybrid([SimpleNamespace(id="b", path=base, width=None, height=2160, range=Video.Range.HDR10P)], "TEST")
|
||||
|
||||
assert seen == {"stale_l6": False, "active_fraction": 0.9}
|
||||
assert not (tmp_path / "RPU.bin").exists() and not (tmp_path / "RPU_L6.bin").exists()
|
||||
@@ -446,8 +446,8 @@ def test_a_failed_resolution_removes_its_crop_edits_and_keeps_the_dv_files(
|
||||
monkeypatch.setattr(Hybrid, "injecting", failed_inject)
|
||||
monkeypatch.setattr(hybrid_mod.console, "print", lambda *a, **k: None)
|
||||
videos = [
|
||||
SimpleNamespace(id="b", path=tmp_path / "base.mkv", height=2160, range=Video.Range.HDR10),
|
||||
SimpleNamespace(id="d", path=tmp_path / "dv.mkv", height=1080, range=Video.Range.DV),
|
||||
SimpleNamespace(id="b", path=tmp_path / "base.mkv", width=None, height=2160, range=Video.Range.HDR10),
|
||||
SimpleNamespace(id="d", path=tmp_path / "dv.mkv", width=None, height=1080, range=Video.Range.DV),
|
||||
]
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
@@ -559,7 +559,7 @@ def test_hybrid_asks_for_hdr10plus_only_for_a_conversion_without_dv(
|
||||
|
||||
|
||||
def test_the_hybrid_step_gets_the_base_alone_when_there_is_no_dv(tmp_path: Path) -> None:
|
||||
base = SimpleNamespace(id="base", path=tmp_path / "base.mp4", height=2160, range=Video.Range.HDR10P)
|
||||
base = SimpleNamespace(id="base", path=tmp_path / "base.mp4", width=None, height=2160, range=Video.Range.HDR10P)
|
||||
|
||||
tracks = hybrid_source_tracks(base, None) # type: ignore[arg-type]
|
||||
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
"""Tests for the standard resolution in the filename context, the export name and -q matching."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import types
|
||||
from typing import Optional
|
||||
|
||||
import pytest
|
||||
|
||||
from unshackle.core.config import config
|
||||
from unshackle.core.export_name import resolution
|
||||
from unshackle.core.titles.movie import Movie
|
||||
from unshackle.core.tracks import Video
|
||||
from unshackle.core.tracks.tracks import at_resolution
|
||||
from unshackle.core.utilities import standard_resolution
|
||||
|
||||
|
||||
class DummyService:
|
||||
pass
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_config(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(config, "tag_rules", [])
|
||||
monkeypatch.setattr(config, "language_tags", {})
|
||||
monkeypatch.setattr(config, "codec_names", {})
|
||||
|
||||
|
||||
def quality(width: Optional[int], height: Optional[int], scan_type: str = "Progressive") -> str:
|
||||
movie = Movie(id_="movie-0001", service=DummyService, name="The Film", year=2024, language="en")
|
||||
video = types.SimpleNamespace(
|
||||
width=width,
|
||||
height=height,
|
||||
format="AVC",
|
||||
scan_type=scan_type,
|
||||
hdr_format_commercial=None,
|
||||
hdr_format=None,
|
||||
transfer_characteristics="BT.709",
|
||||
transfer_characteristics_original=None,
|
||||
frame_rate="25.000",
|
||||
)
|
||||
media_info = types.SimpleNamespace(video_tracks=[video], audio_tracks=[])
|
||||
return str(movie.build_base_template_context(media_info)["quality"])
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("width", "height", "expected"),
|
||||
[
|
||||
(3840, 2160, 2160),
|
||||
(1920, 1080, 1080),
|
||||
(1280, 720, 720),
|
||||
(2560, 1440, 1440),
|
||||
(1920, 1088, 1080), # padded to a multiple of 16
|
||||
(960, 544, 540),
|
||||
(640, 368, 360),
|
||||
(1916, 1076, 1080), # a few lines short
|
||||
(3836, 2156, 2160),
|
||||
(1920, 800, 1080), # scope crop
|
||||
(1920, 1036, 1080), # flat crop
|
||||
(3840, 1600, 2160),
|
||||
(1280, 544, 720),
|
||||
(4096, 2160, 2160), # DCI
|
||||
(4096, 1716, 2160),
|
||||
(2048, 858, 1080),
|
||||
(1440, 1080, 1080), # 4:3 pillarbox
|
||||
(960, 720, 720),
|
||||
(720, 576, 576), # anamorphic SD
|
||||
(704, 576, 576),
|
||||
(720, 480, 480),
|
||||
(854, 480, 480),
|
||||
(1080, 1920, 1080), # portrait
|
||||
(1080, 1350, 1080),
|
||||
(720, 1280, 720),
|
||||
(2560, 1080, 1080), # ultrawide
|
||||
(3440, 1440, 1440),
|
||||
(1600, 900, 900), # no standard height near it
|
||||
],
|
||||
)
|
||||
def test_standard_resolution(width: int, height: int, expected: int) -> None:
|
||||
assert standard_resolution(width, height) == expected
|
||||
|
||||
|
||||
def test_a_picture_with_one_side_uses_it() -> None:
|
||||
assert standard_resolution(None, 1088) == 1080
|
||||
|
||||
|
||||
def test_a_picture_with_no_size_has_no_resolution() -> None:
|
||||
assert standard_resolution(None, None) is None
|
||||
|
||||
|
||||
def test_the_filename_quality_uses_the_standard_resolution() -> None:
|
||||
assert quality(1920, 1088) == "1080p"
|
||||
assert quality(1920, 800) == "1080p"
|
||||
|
||||
|
||||
def test_an_interlaced_picture_keeps_the_i_suffix() -> None:
|
||||
assert quality(1920, 1088, scan_type="Interlaced") == "1080i"
|
||||
|
||||
|
||||
def test_a_video_track_with_no_size_gives_an_empty_quality() -> None:
|
||||
assert quality(None, None) == ""
|
||||
|
||||
|
||||
def video(width: int, height: int) -> Video:
|
||||
return Video(url="https://example.test/v.mpd", language="en", codec=Video.Codec.AVC, width=width, height=height)
|
||||
|
||||
|
||||
def test_the_export_name_uses_the_standard_resolution() -> None:
|
||||
assert resolution(video(1080, 1920)) == 1080
|
||||
assert resolution(video(4096, 2160)) == 2160
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("width", "height"), [(1920, 1080), (1920, 800), (1916, 1076), (1920, 1088)])
|
||||
def test_q_1080_matches_odd_1080p_encodes(width: int, height: int) -> None:
|
||||
assert at_resolution(video(width, height), 1080)
|
||||
|
||||
|
||||
def test_q_1080_does_not_match_720p() -> None:
|
||||
assert not at_resolution(video(1280, 720), 1080)
|
||||
@@ -37,6 +37,10 @@ class Recorder(BaseHTTPRequestHandler):
|
||||
self.send_header("Set-Cookie", "sid=abc123; Path=/")
|
||||
elif self.path.startswith("/set/"):
|
||||
self.send_header("Set-Cookie", f"{self.path[len('/set/') :]}; Path=/")
|
||||
elif self.path.startswith("/scoped/set/"):
|
||||
self.send_header("Set-Cookie", f"{self.path[len('/scoped/set/') :]}; Path=/scoped")
|
||||
elif self.path.startswith("/scoped/expire/"):
|
||||
self.send_header("Set-Cookie", f"{self.path[len('/scoped/expire/') :]}=; Max-Age=0; Path=/scoped")
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
@@ -57,7 +61,7 @@ def server() -> Iterator[str]:
|
||||
httpd.server_close()
|
||||
|
||||
|
||||
def make_cookie(name: str, value: str, domain: str) -> Cookie:
|
||||
def make_cookie(name: str, value: str, domain: str, path: str = "/") -> Cookie:
|
||||
return Cookie(
|
||||
version=0,
|
||||
name=name,
|
||||
@@ -67,7 +71,7 @@ def make_cookie(name: str, value: str, domain: str) -> Cookie:
|
||||
domain=domain,
|
||||
domain_specified=True,
|
||||
domain_initial_dot=domain.startswith("."),
|
||||
path="/",
|
||||
path=path,
|
||||
path_specified=True,
|
||||
secure=False,
|
||||
expires=None,
|
||||
@@ -333,3 +337,91 @@ def test_the_rebuilt_client_is_published_only_when_it_is_ready(server: str, monk
|
||||
|
||||
assert seen and all(client is old for client in seen)
|
||||
assert session._client is not old
|
||||
|
||||
|
||||
def path_session(*cookies: Cookie) -> RnetSession:
|
||||
session = RnetSession()
|
||||
session.cookies.update(list(cookies))
|
||||
return session
|
||||
|
||||
|
||||
def test_same_name_cookies_reach_only_their_own_path(server: str) -> None:
|
||||
"""Two cookies with one name on two paths must each reach only their own path."""
|
||||
session = path_session(make_cookie("h", "A", "127.0.0.1", "/a"), make_cookie("h", "B", "127.0.0.1", "/b"))
|
||||
|
||||
for path, expected in (("/a/seg", "h=A"), ("/b/seg", "h=B"), ("/", "")):
|
||||
session.get(f"{server}{path}")
|
||||
assert Recorder.cookies_sent[-1] == expected
|
||||
|
||||
session.headers["X-Rebuild"] = "1"
|
||||
del session.headers["X-Rebuild"]
|
||||
session.get(f"{server}/a/seg")
|
||||
assert Recorder.cookies_sent[-1] == "h=A"
|
||||
|
||||
|
||||
def test_delete_drops_every_path_copy(server: str) -> None:
|
||||
"""``del cookies[name]`` must reach a cookie stored on a path, not only the root-path one."""
|
||||
session = path_session(make_cookie("h", "A", "127.0.0.1", "/a"), make_cookie("h", "B", "127.0.0.1", "/b"))
|
||||
session.get(f"{server}/a/seg")
|
||||
|
||||
del session.cookies["h"]
|
||||
|
||||
session.get(f"{server}/a/seg")
|
||||
session.get(f"{server}/b/seg")
|
||||
assert Recorder.cookies_sent[-2:] == ["", ""]
|
||||
assert list(session.cookies.jar) == []
|
||||
|
||||
|
||||
def test_clear_by_path_spares_the_other_path(server: str) -> None:
|
||||
session = path_session(make_cookie("h", "A", "127.0.0.1", "/a"), make_cookie("h", "B", "127.0.0.1", "/b"))
|
||||
session.get(f"{server}/a/seg")
|
||||
|
||||
session.cookies.clear(domain="127.0.0.1", path="/a", name="h")
|
||||
|
||||
session.get(f"{server}/a/seg")
|
||||
session.get(f"{server}/b/seg")
|
||||
assert Recorder.cookies_sent[-2:] == ["", "h=B"]
|
||||
assert session.cookies.get("h", path="/b") == "B"
|
||||
assert session.cookies.get("h", path="/a") is None
|
||||
|
||||
|
||||
def test_a_rotated_path_cookie_survives_a_rebuild(server: str) -> None:
|
||||
"""A value the server set for a stored path cookie must win after ``rebuild_client`` and on save."""
|
||||
session = path_session(make_cookie("h", "STALE", "127.0.0.1", "/scoped"))
|
||||
session.headers["X-Auth"] = "secret"
|
||||
|
||||
session.get(f"{server}/scoped/set/h=FRESH")
|
||||
session.headers.pop("X-Auth", None)
|
||||
session.get(f"{server}/scoped/seg")
|
||||
|
||||
assert Recorder.cookies_sent[-1] == "h=FRESH"
|
||||
assert [(c.path, c.value) for c in session.cookies.jar] == [("/scoped", "FRESH")]
|
||||
|
||||
|
||||
def test_a_server_expiry_drops_the_stored_cookie(server: str) -> None:
|
||||
session = path_session(make_cookie("h", "STALE", "127.0.0.1", "/scoped"))
|
||||
session.headers["X-Auth"] = "secret"
|
||||
|
||||
session.get(f"{server}/scoped/expire/h")
|
||||
session.headers.pop("X-Auth", None)
|
||||
session.get(f"{server}/scoped/seg")
|
||||
|
||||
assert Recorder.cookies_sent[-1] == ""
|
||||
assert "h" not in session.cookies
|
||||
|
||||
|
||||
def test_only_a_leading_dot_widens_a_cookie_to_subdomains() -> None:
|
||||
"""A Netscape row with a leading dot covers subdomains; any other domain stays host-only."""
|
||||
assert RnetCookieAdapter.to_rnet(make_cookie("s", "1", ".example.com")).domain == "example.com"
|
||||
assert RnetCookieAdapter.to_rnet(make_cookie("s", "1", "www.example.com")).domain is None
|
||||
|
||||
|
||||
def test_the_latest_write_answers_a_lookup_by_name() -> None:
|
||||
session = path_session(make_cookie("tok", "file", ".example.com"))
|
||||
session.cookies.update({"tok": "dict"})
|
||||
assert session.cookies.get("tok") == "dict"
|
||||
assert session.cookies["tok"] == "dict"
|
||||
session.cookies.set("tok", "set", domain=".example.com")
|
||||
assert session.cookies.get("tok") == "set"
|
||||
assert session.cookies.get("tok", domain="") == "dict"
|
||||
assert [c.value for c in session.cookies.jar] == ["set"]
|
||||
|
||||
@@ -104,7 +104,6 @@ def test_a_quality_rule_matches_the_built_context(monkeypatch: pytest.MonkeyPatc
|
||||
height=2160,
|
||||
format="HEVC",
|
||||
scan_type="Progressive",
|
||||
other_display_aspect_ratio=["16:9"],
|
||||
hdr_format_commercial=None,
|
||||
hdr_format=None,
|
||||
transfer_characteristics=None,
|
||||
|
||||
@@ -47,7 +47,7 @@ def test_rnet_spec_round_trips_cookie_domains() -> None:
|
||||
rebuilt = rebuild_session(pickle.loads(pickle.dumps(spec)), 1)
|
||||
assert rebuilt is not None
|
||||
|
||||
assert rebuilt.cookies.get_dict_by_domain() == ns.cookies.get_dict_by_domain()
|
||||
assert [vars(c) for c in rebuilt.cookies.snapshot()] == [vars(c) for c in ns.cookies.snapshot()]
|
||||
assert rebuilt.cookies.get_dict() == ns.cookies.get_dict()
|
||||
assert rebuilt.cookies.get("cdn_token", domain="cdn.example.com") == "abc"
|
||||
assert rebuilt.cookies.get("api_token", domain="api.example.com") == "def"
|
||||
@@ -71,17 +71,36 @@ def test_rnet_spec_cookies_carry_domain_information() -> None:
|
||||
spec = build_session_spec(ns)
|
||||
assert spec is not None
|
||||
|
||||
cookies = spec["cookies"]
|
||||
assert set(cookies) == {"cdn.example.com", "api.example.com", ""}
|
||||
assert all(isinstance(jar, dict) for jar in cookies.values())
|
||||
assert cookies["cdn.example.com"] == {"cdn_token": "abc"}
|
||||
assert {(c.domain, c.path, c.name, c.value) for c in spec["cookies"]} == {
|
||||
("cdn.example.com", "/", "cdn_token", "abc"),
|
||||
("api.example.com", "/", "api_token", "def"),
|
||||
("", "/", "loose", "ghi"),
|
||||
}
|
||||
|
||||
|
||||
def test_rnet_spec_keeps_same_name_cookies_on_their_paths() -> None:
|
||||
"""Same-name cookies on two paths must reach a child apart, or a stale one goes to every path."""
|
||||
ns = make_session("Chrome131")
|
||||
ns.cookies.set("hdntl", "one", domain=".cdn.example.com", path="/t/1")
|
||||
ns.cookies.set("hdntl", "two", domain=".cdn.example.com", path="/t/2")
|
||||
|
||||
spec = build_session_spec(ns)
|
||||
assert spec is not None
|
||||
rebuilt = rebuild_session(pickle.loads(pickle.dumps(spec)), 1)
|
||||
assert rebuilt is not None
|
||||
|
||||
assert rebuilt.cookies.get("hdntl", path="/t/1") == "one"
|
||||
assert rebuilt.cookies.get("hdntl", path="/t/2") == "two"
|
||||
|
||||
|
||||
def test_rnet_spec_is_picklable() -> None:
|
||||
spec = build_session_spec(rnet_session_with_cookies())
|
||||
assert spec is not None
|
||||
|
||||
assert pickle.loads(pickle.dumps(spec)) == spec
|
||||
loaded = pickle.loads(pickle.dumps(spec))
|
||||
# stdlib Cookie has no __eq__, so compare the cookies by their fields
|
||||
assert [vars(c) for c in loaded.pop("cookies")] == [vars(c) for c in spec.pop("cookies")]
|
||||
assert loaded == spec
|
||||
|
||||
|
||||
def test_rnet_spec_without_impersonate_is_not_rebuildable() -> None:
|
||||
|
||||
@@ -3,11 +3,19 @@ import pytest
|
||||
from unshackle.core.tracks import Subtitle, Tracks
|
||||
|
||||
|
||||
def sub(lang: str, forced: bool = False, sdh: bool = False, cc: bool = False, orig: bool = False) -> Subtitle:
|
||||
def sub(
|
||||
lang: str,
|
||||
forced: bool = False,
|
||||
sdh: bool = False,
|
||||
cc: bool = False,
|
||||
orig: bool = False,
|
||||
codec: Subtitle.Codec = Subtitle.Codec.SubRip,
|
||||
tag: str = "",
|
||||
) -> Subtitle:
|
||||
track = Subtitle(
|
||||
url=f"https://example.test/{lang}-{forced}-{sdh}-{cc}",
|
||||
url=f"https://example.test/{lang}-{forced}-{sdh}-{cc}-{codec.value}{tag}",
|
||||
language=lang,
|
||||
codec=Subtitle.Codec.SubRip,
|
||||
codec=codec,
|
||||
forced=forced,
|
||||
sdh=sdh,
|
||||
cc=cc,
|
||||
@@ -266,3 +274,20 @@ def test_all_sorts_the_original_track_then_its_other_regional_forms() -> None:
|
||||
)
|
||||
t.sort_audio(by_language=["all"])
|
||||
assert [str(a.language) for a in t.audio] == ["fr-CA", "fr-FR", "en"]
|
||||
|
||||
|
||||
def test_dedupe_subtitle_codecs_keeps_the_first_listed_codec_of_each_variant() -> None:
|
||||
codec = Subtitle.Codec
|
||||
vtt = sub("en", codec=codec.WebVTT)
|
||||
wvtt = sub("en", codec=codec.fVTT)
|
||||
stpp = sub("en", codec=codec.fTTML)
|
||||
stpp_cc = sub("en", cc=True, codec=codec.fTTML)
|
||||
srt = sub("en") # unlisted codec: always kept
|
||||
de_wvtt = sub("de", codec=codec.fVTT)
|
||||
vtt_second = sub("en", codec=codec.WebVTT, tag="-second") # same winning codec: kept, it may differ in content
|
||||
tracks = Tracks()
|
||||
tracks.subtitles = [wvtt, stpp, vtt, stpp_cc, srt, de_wvtt, vtt_second]
|
||||
|
||||
tracks.dedupe_subtitle_codecs([codec.WebVTT, codec.fTTML, codec.fVTT])
|
||||
|
||||
assert tracks.subtitles == [vtt, stpp_cc, srt, de_wvtt, vtt_second]
|
||||
|
||||
+12
-28
@@ -120,6 +120,7 @@ from unshackle.core.utilities import (
|
||||
partition_exclusions,
|
||||
plain_subtitle_tags,
|
||||
resolve_sort_langs,
|
||||
standard_resolution,
|
||||
suggest_font_packages,
|
||||
time_elapsed_since,
|
||||
)
|
||||
@@ -2453,6 +2454,10 @@ class dl:
|
||||
level="INFO", operation="get_tracks", service=self.service, context=tracks_info
|
||||
)
|
||||
|
||||
if dedupe_codecs := config.subtitle.get("dedupe_codecs"):
|
||||
codecs = map(SUBTITLE_CODEC.convert, dedupe_codecs)
|
||||
title.tracks.dedupe_subtitle_codecs([c for c in codecs if isinstance(c, Subtitle.Codec)])
|
||||
|
||||
# a plain track covers an SDH track only when it is the same variant, as -sl selects them,
|
||||
# so en-US SDH is stripped next to a plain en-GB: -sl en-US would otherwise keep only the SDH
|
||||
if config.subtitle.get("strip_sdh", True):
|
||||
@@ -2719,9 +2724,7 @@ class dl:
|
||||
v for v in non_hybrid_tracks if v.range in non_hybrid_ranges
|
||||
]
|
||||
for resolution in quality:
|
||||
if any(v.height == resolution for v in deliverable_pool):
|
||||
continue
|
||||
if any(v.width and int(v.width * 9 / 16) == resolution for v in deliverable_pool):
|
||||
if any(at_resolution(v, resolution) for v in deliverable_pool):
|
||||
continue
|
||||
missing_resolutions.append(resolution)
|
||||
|
||||
@@ -2737,12 +2740,7 @@ class dl:
|
||||
|
||||
if non_hybrid_ranges and non_hybrid_tracks:
|
||||
non_hybrid_selected = [
|
||||
v
|
||||
for v in non_hybrid_tracks
|
||||
if any(
|
||||
v.height == res or (v.width and int(v.width * (9 / 16)) == res)
|
||||
for res in quality
|
||||
)
|
||||
v for v in non_hybrid_tracks if any(at_resolution(v, res) for res in quality)
|
||||
]
|
||||
title.tracks.videos = Tracks.merge_video_selections(
|
||||
hybrid_selected, non_hybrid_selected
|
||||
@@ -2754,9 +2752,7 @@ class dl:
|
||||
title.tracks.by_resolutions(quality)
|
||||
|
||||
for resolution in quality:
|
||||
if any(v.height == resolution for v in title.tracks.videos):
|
||||
continue
|
||||
if any(v.width and int(v.width * 9 / 16) == resolution for v in title.tracks.videos):
|
||||
if any(at_resolution(v, resolution) for v in title.tracks.videos):
|
||||
continue
|
||||
missing_resolutions.append(resolution)
|
||||
|
||||
@@ -2814,11 +2810,7 @@ class dl:
|
||||
candidates = [
|
||||
t
|
||||
for t in non_hybrid_tracks
|
||||
if (
|
||||
not resolution
|
||||
or t.height == resolution
|
||||
or (t.width and int(t.width * (9 / 16)) == resolution)
|
||||
)
|
||||
if (not resolution or at_resolution(t, resolution))
|
||||
and (not color_range or t.range == color_range)
|
||||
and (not codec or t.codec == codec)
|
||||
and (vlang is None or str(t.language) == vlang)
|
||||
@@ -2847,11 +2839,7 @@ class dl:
|
||||
candidates = [
|
||||
t
|
||||
for t in title.tracks.videos
|
||||
if (
|
||||
not resolution
|
||||
or t.height == resolution
|
||||
or (t.width and int(t.width * (9 / 16)) == resolution)
|
||||
)
|
||||
if (not resolution or at_resolution(t, resolution))
|
||||
and (not color_range or t.range == color_range)
|
||||
and (not codec or t.codec == codec)
|
||||
and (vlang is None or str(t.language) == vlang)
|
||||
@@ -2889,11 +2877,7 @@ class dl:
|
||||
candidates = [
|
||||
t
|
||||
for t in fallback_pool
|
||||
if (
|
||||
not resolution
|
||||
or t.height == resolution
|
||||
or (t.width and int(t.width * (9 / 16)) == resolution)
|
||||
)
|
||||
if (not resolution or at_resolution(t, resolution))
|
||||
and (not color_range or t.range == color_range)
|
||||
and (not codec or t.codec == codec)
|
||||
and (vlang is None or str(t.language) == vlang)
|
||||
@@ -3649,7 +3633,7 @@ class dl:
|
||||
head = next((v for v in video_tracks if v), None)
|
||||
if head:
|
||||
if len(quality) > 1:
|
||||
task_description += f" {head.height}p"
|
||||
task_description += f" {standard_resolution(head.width, head.height)}p"
|
||||
if len(range_) > 1:
|
||||
task_description += f" {head.range.name}"
|
||||
if len(vcodec) > 1:
|
||||
|
||||
@@ -10,6 +10,7 @@ import time
|
||||
from typing import Any, Callable, Optional
|
||||
|
||||
from unshackle.core.constants import AnyTrack
|
||||
from unshackle.core.utilities import standard_resolution
|
||||
|
||||
JOB_PROGRESS_TERMINAL_STATES = {"Downloaded", "Decrypted", "[yellow]SKIPPED"}
|
||||
PROGRESS_EMIT_INTERVAL = 0.25
|
||||
@@ -26,9 +27,9 @@ def track_progress_label(track: AnyTrack) -> str:
|
||||
track_type = type(track).__name__
|
||||
if track_type == "Video":
|
||||
parts = ["video"]
|
||||
height = getattr(track, "height", None)
|
||||
if height:
|
||||
parts.append(f"{height}p")
|
||||
resolution = standard_resolution(getattr(track, "width", None), getattr(track, "height", None))
|
||||
if resolution:
|
||||
parts.append(f"{resolution}p")
|
||||
track_range = getattr(track, "range", None)
|
||||
if track_range is not None:
|
||||
parts.append(track_range.value)
|
||||
|
||||
@@ -1247,7 +1247,7 @@ def build_session_spec(session: Optional[Any]) -> Optional[dict[str, Any]]:
|
||||
"kind": "rnet",
|
||||
"impersonate": name,
|
||||
"headers": dict(session.headers),
|
||||
"cookies": session.cookies.get_dict_by_domain(),
|
||||
"cookies": session.cookies.snapshot(), # stdlib Cookies pickle with their paths
|
||||
"origin_cookies": session.export_origin_cookies(),
|
||||
"proxy": session.proxies.get("all") or session.proxies.get("https") or session.proxies.get("http"),
|
||||
}
|
||||
@@ -1290,13 +1290,7 @@ def rebuild_session(spec: dict[str, Any], max_workers: int) -> Optional[Any]:
|
||||
ns = make_session(spec["impersonate"])
|
||||
if spec.get("headers"):
|
||||
ns.headers.update(spec["headers"])
|
||||
for domain, jar in spec.get("cookies", {}).items():
|
||||
if domain:
|
||||
for name, value in jar.items():
|
||||
ns.cookies.set(name, value, domain=domain)
|
||||
else:
|
||||
# no domain to scope to; update() reproduces the parent's localhost-scoped state
|
||||
ns.cookies.update(jar)
|
||||
ns.cookies.update(spec.get("cookies") or [])
|
||||
if spec.get("proxy"):
|
||||
ns.proxies.update({"all": spec["proxy"]})
|
||||
ns.import_origin_cookies(spec.get("origin_cookies") or {})
|
||||
|
||||
@@ -15,9 +15,8 @@ from typing import Any, Optional
|
||||
from unshackle.core.constants import AnyTrack
|
||||
from unshackle.core.titles import Episode, Movie, Song, Title_T
|
||||
from unshackle.core.tracks import Audio, Video
|
||||
from unshackle.core.utilities import sanitize_filename
|
||||
from unshackle.core.utilities import sanitize_filename, standard_resolution
|
||||
|
||||
STANDARD_HEIGHTS = (4320, 2160, 1440, 1080, 720, 576, 480, 360)
|
||||
MANIFEST_ORDER = (Video.Descriptor.DASH, Video.Descriptor.HLS, Video.Descriptor.ISM, Video.Descriptor.URL)
|
||||
MAX_SCOPE_PARTS = 3
|
||||
MAX_TITLE_LENGTH = 100
|
||||
@@ -107,13 +106,8 @@ def scope_part(titles: Sequence[Title_T], seasons: Mapping[int, Collection[int]]
|
||||
|
||||
|
||||
def resolution(track: Video) -> Optional[int]:
|
||||
"""The resolution a release name uses: a letterboxed 1920x800 is 1080p, from its width."""
|
||||
if not track.height:
|
||||
return None
|
||||
height = track.height
|
||||
if track.width and track.width > height:
|
||||
height = max(height, track.width * 9 // 16)
|
||||
return next((std for std in STANDARD_HEIGHTS if abs(height - std) <= 16), height)
|
||||
"""The resolution a release name uses, the same as the {resolution} template variable."""
|
||||
return standard_resolution(track.width, track.height) if track.height else None
|
||||
|
||||
|
||||
def audio_token(track: Audio) -> str:
|
||||
|
||||
+167
-138
@@ -10,9 +10,10 @@ import re
|
||||
import sys
|
||||
import time
|
||||
from collections.abc import Iterator, MutableMapping
|
||||
from copy import copy
|
||||
from datetime import datetime, timezone
|
||||
from email.utils import parsedate_to_datetime
|
||||
from http.cookiejar import CookieJar
|
||||
from http.cookiejar import Cookie, CookieJar
|
||||
from typing import Any, Optional, Union
|
||||
from urllib.parse import urlencode, urlparse, urlunparse
|
||||
|
||||
@@ -288,210 +289,231 @@ class RnetSessionHeaders(CaseInsensitiveDict):
|
||||
|
||||
|
||||
class RnetCookieAdapter(MutableMapping):
|
||||
"""Cookie adapter that bridges requests-style cookie access to rnet."""
|
||||
"""Cookie adapter that bridges requests-style cookie access to rnet.
|
||||
|
||||
One store keyed by (domain, path, name) holds every cookie set through this class, in write
|
||||
order. Same-name cookies on different paths stay apart on the wire, and the latest write
|
||||
answers a lookup by name.
|
||||
"""
|
||||
|
||||
def __init__(self, client: Any, session: Optional[RnetSession] = None) -> None:
|
||||
self._client = client
|
||||
self._session = session
|
||||
self._cookies: dict[str, dict[str, str]] = {}
|
||||
self._flat: dict[str, str] = {}
|
||||
self._original_cookies: list[Any] = []
|
||||
self._store: dict[tuple[str, str, str], Cookie] = {}
|
||||
self._from_jar: set[tuple[str, str, str]] = set()
|
||||
|
||||
def set_cookie_on_client(self, url: str, name: str, value: str) -> None:
|
||||
"""Set a cookie on the rnet client, or buffer locally if the client is not yet created."""
|
||||
@staticmethod
|
||||
def client_url(cookie: Cookie) -> str:
|
||||
"""The URL rnet stores *cookie* under; a domain-less cookie goes to localhost."""
|
||||
host = (cookie.domain or "").lstrip(".")
|
||||
return f"https://{host}" if host else "https://localhost"
|
||||
|
||||
@staticmethod
|
||||
def to_rnet(cookie: Cookie) -> rnet.Cookie:
|
||||
"""The rnet form of *cookie*.
|
||||
|
||||
A leading dot marks a cookie for subdomains too, as in a Netscape cookie file; without
|
||||
one the cookie stays on its own host. Expiry stays out, so no layer drops a cookie the
|
||||
cookie loader deliberately kept.
|
||||
"""
|
||||
domain = cookie.domain or ""
|
||||
return rnet.Cookie(
|
||||
cookie.name,
|
||||
cookie.value or "",
|
||||
domain=domain if domain.startswith(".") else None,
|
||||
path=cookie.path or "/",
|
||||
secure=bool(cookie.secure),
|
||||
)
|
||||
|
||||
def push(self, cookie: Cookie) -> None:
|
||||
"""Set *cookie* on the rnet client; before the client exists the store buffers it."""
|
||||
if self._client is not None:
|
||||
try:
|
||||
self._client.set_cookie(url, rnet.Cookie(name, value))
|
||||
self._client.set_cookie(self.client_url(cookie), self.to_rnet(cookie))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def add(self, cookie: Cookie, from_jar: bool = False) -> None:
|
||||
"""Store *cookie* as the latest write for its (domain, path, name) and send it to the client."""
|
||||
key = (cookie.domain or "", cookie.path or "/", cookie.name)
|
||||
self._store.pop(key, None)
|
||||
self._store[key] = cookie
|
||||
if from_jar:
|
||||
self._from_jar.add(key)
|
||||
self.push(cookie)
|
||||
|
||||
def snapshot(self) -> list[Cookie]:
|
||||
"""Every stored cookie, oldest write first. Download threads write the store, so read a copy."""
|
||||
return list(self._store.values())
|
||||
|
||||
def flush_to_client(self) -> None:
|
||||
"""Push all buffered cookies to the rnet client after unshackle makes it."""
|
||||
if self._client is None:
|
||||
return
|
||||
for domain, cookies in self._cookies.items():
|
||||
url = f"https://{domain.lstrip('.')}" if domain else "https://localhost"
|
||||
for name, value in cookies.items():
|
||||
try:
|
||||
self._client.set_cookie(url, rnet.Cookie(name, value))
|
||||
except Exception:
|
||||
pass
|
||||
for cookie in self.snapshot():
|
||||
self.push(cookie)
|
||||
|
||||
def client_urls(self, domain: Optional[str] = None) -> list[str]:
|
||||
def client_urls(self, domain: Optional[str] = None, name: Optional[str] = None) -> list[str]:
|
||||
"""URLs the rnet cookie jar could hold a cookie under.
|
||||
|
||||
rnet keys its jar by URL and cannot list the jar whole, so a removal has to name every
|
||||
URL this class has set a cookie on and every origin the HTTP session has requested.
|
||||
rnet keys its jar by URL, removes a cookie only at its exact path, and cannot list the
|
||||
jar. So a removal names every host this class has set a cookie on, every origin the
|
||||
HTTP session has requested, and every stored path of the cookie *name*.
|
||||
"""
|
||||
urls = {f"https://{d.lstrip('.')}" for d in self._cookies if d}
|
||||
urls.add("https://localhost")
|
||||
cookies = self.snapshot()
|
||||
hosts = {self.client_url(c) for c in cookies} | {"https://localhost"}
|
||||
if self._session is not None:
|
||||
urls |= set(self._session._origins)
|
||||
hosts |= set(self._session._origins)
|
||||
if domain is not None:
|
||||
host = domain.lstrip(".")
|
||||
urls = {url for url in urls if (urlparse(url).hostname or "") == host}
|
||||
urls.add(f"https://{host}")
|
||||
return sorted(urls)
|
||||
hosts = {url for url in hosts if (urlparse(url).hostname or "") == host} | {f"https://{host}"}
|
||||
paths = {c.path for c in cookies if c.path and c.path != "/" and (name is None or c.name == name)}
|
||||
return sorted(hosts | {url + path for url in hosts for path in paths})
|
||||
|
||||
def remove_cookie_on_client(self, name: str, domain: Optional[str] = None) -> None:
|
||||
"""Drop a cookie from the rnet cookie jar.
|
||||
"""Drop a cookie from the rnet cookie jar; call it before the store forgets the cookie's paths.
|
||||
|
||||
The client is built with cookie_store=True, so a cookie the server set lives in that
|
||||
jar alone. Dropping it from the local dicts leaves it going out on the wire.
|
||||
jar alone. Dropping it from the store leaves it going out on the wire.
|
||||
"""
|
||||
if self._client is None:
|
||||
return
|
||||
for url in self.client_urls(domain):
|
||||
for url in self.client_urls(domain, name):
|
||||
try:
|
||||
self._client.remove_cookie(url, name)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def absorb(self, response: Any) -> None:
|
||||
"""Take a value the server set for a stored cookie, so a client rebuild or a cookie save keeps it.
|
||||
|
||||
A server cookie replaces each stored cookie of the same name and path that covers the
|
||||
response host, and an expired one drops it.
|
||||
"""
|
||||
server_cookies = response.cookies if self._store else None
|
||||
if not server_cookies:
|
||||
return
|
||||
parsed = urlparse(response.url)
|
||||
host = parsed.hostname or ""
|
||||
default_path = (parsed.path[: parsed.path.rfind("/")] or "/") if parsed.path.startswith("/") else "/"
|
||||
now = datetime.now(timezone.utc)
|
||||
for server_cookie in server_cookies:
|
||||
path = server_cookie.path if (server_cookie.path or "").startswith("/") else default_path
|
||||
expires, max_age = server_cookie.expires, server_cookie.max_age
|
||||
if expires is not None and expires.tzinfo is None:
|
||||
expires = expires.replace(tzinfo=timezone.utc)
|
||||
expired = (max_age is not None and max_age.total_seconds() <= 0) or (expires is not None and expires <= now)
|
||||
for cookie in self.snapshot():
|
||||
domain = cookie.domain or ""
|
||||
if cookie.name != server_cookie.name or cookie.path != path or not domain:
|
||||
continue
|
||||
if host != domain.lstrip(".") and not (domain.startswith(".") and host.endswith(domain)):
|
||||
continue
|
||||
if expired:
|
||||
self.clear(domain=domain, path=path, name=cookie.name)
|
||||
elif cookie.value != server_cookie.value:
|
||||
fresh = copy(cookie)
|
||||
fresh.value = server_cookie.value
|
||||
self.add(fresh)
|
||||
|
||||
@property
|
||||
def jar(self) -> CookieJar:
|
||||
"""Return a CookieJar with original Cookie objects (requests compat).
|
||||
"""Return a CookieJar of the cookies that arrived as a CookieJar (requests compat).
|
||||
|
||||
``save_cookies`` in dl.py uses this to write cookies back to disk.
|
||||
``save_cookies`` in dl.py uses this to write cookies back to disk, so it carries a value
|
||||
the server rotated for one of those cookies but not the cookies set by name.
|
||||
"""
|
||||
jar = CookieJar()
|
||||
for cookie in self._original_cookies:
|
||||
jar.set_cookie(cookie)
|
||||
for key, cookie in list(self._store.items()):
|
||||
if key in self._from_jar:
|
||||
jar.set_cookie(cookie)
|
||||
return jar
|
||||
|
||||
def update(self, other: Any = None, /, **kwargs: Any) -> None:
|
||||
if other is None:
|
||||
other = {}
|
||||
if isinstance(other, CookieJar):
|
||||
if isinstance(other, (CookieJar, list)):
|
||||
for cookie in other:
|
||||
domain = cookie.domain or ""
|
||||
name = cookie.name
|
||||
value = cookie.value or ""
|
||||
self._flat[name] = value
|
||||
self._cookies.setdefault(domain, {})[name] = value
|
||||
self._original_cookies.append(cookie)
|
||||
url = f"https://{domain.lstrip('.')}" if domain else "https://localhost"
|
||||
self.set_cookie_on_client(url, name, value)
|
||||
elif isinstance(other, dict):
|
||||
for name, value in other.items():
|
||||
self._flat[name] = value
|
||||
self.set_cookie_on_client("https://localhost", name, str(value))
|
||||
self._flat.update(other)
|
||||
self.add(cookie, from_jar=True)
|
||||
elif hasattr(other, "items"):
|
||||
for name, value in other.items():
|
||||
self._flat[name] = str(value)
|
||||
self.set_cookie_on_client("https://localhost", name, str(value))
|
||||
|
||||
self.add(create_cookie(name, str(value)))
|
||||
for name, value in kwargs.items():
|
||||
self._flat[name] = value
|
||||
self.set_cookie_on_client("https://localhost", name, value)
|
||||
self.add(create_cookie(name, str(value)))
|
||||
|
||||
def get( # type: ignore[override]
|
||||
self, name: str, default: Optional[str] = None, domain: Optional[str] = None, path: Optional[str] = None
|
||||
) -> Optional[str]:
|
||||
if domain and domain in self._cookies:
|
||||
return self._cookies[domain].get(name, default)
|
||||
return self._flat.get(name, default)
|
||||
"""The latest value of *name*."""
|
||||
for cookie in reversed(self.snapshot()):
|
||||
if cookie.name == name and domain in (None, cookie.domain) and path in (None, cookie.path):
|
||||
return cookie.value
|
||||
return default
|
||||
|
||||
def set(self, name: str, value: str, domain: str = "localhost") -> None:
|
||||
self._flat[name] = value
|
||||
self._cookies.setdefault(domain, {})[name] = value
|
||||
url = f"https://{domain.lstrip('.')}"
|
||||
self.set_cookie_on_client(url, name, value)
|
||||
def set(self, name: str, value: str, domain: str = "localhost", path: str = "/", **kwargs: Any) -> None:
|
||||
self.add(create_cookie(name, value, domain=domain, path=path, **kwargs))
|
||||
|
||||
def flat(self) -> dict[str, str]:
|
||||
"""Each cookie name with its latest value."""
|
||||
return {cookie.name: cookie.value or "" for cookie in self.snapshot()}
|
||||
|
||||
def __getitem__(self, name: str) -> str:
|
||||
return self._flat[name]
|
||||
return self.flat()[name]
|
||||
|
||||
def __setitem__(self, name: str, value: str) -> None:
|
||||
self.set(name, value)
|
||||
|
||||
def __delitem__(self, name: str) -> None:
|
||||
self._flat.pop(name, None)
|
||||
for domain_cookies in self._cookies.values():
|
||||
domain_cookies.pop(name, None)
|
||||
self._original_cookies = [cookie for cookie in self._original_cookies if cookie.name != name]
|
||||
self.remove_cookie_on_client(name)
|
||||
self.clear(name=name)
|
||||
|
||||
def __contains__(self, name: object) -> bool:
|
||||
return name in self._flat
|
||||
return any(cookie.name == name for cookie in self.snapshot())
|
||||
|
||||
def __iter__(self) -> Iterator:
|
||||
return iter(self._flat)
|
||||
return iter(self.flat())
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._flat)
|
||||
return len(self.flat())
|
||||
|
||||
def __bool__(self) -> bool:
|
||||
return bool(self._flat)
|
||||
return bool(self._store)
|
||||
|
||||
def get_dict(self, domain: Optional[str] = None, path: Optional[str] = None) -> dict[str, str]:
|
||||
"""Return cookies as a plain dict (requests RequestsCookieJar compat).
|
||||
|
||||
With a *domain*, this method returns only the cookies for that domain.
|
||||
It accepts *path* for API compatibility but ignores it (flat storage).
|
||||
"""
|
||||
if domain is not None:
|
||||
return dict(self._cookies.get(domain, {}))
|
||||
return dict(self._flat)
|
||||
|
||||
def get_dict_by_domain(self) -> dict[str, dict[str, str]]:
|
||||
"""Return cookies grouped by domain, domain-less ones under ``""``.
|
||||
|
||||
rnet scopes each cookie to the host of the URL it was set on, so a copy made with
|
||||
``get_dict`` can only ever replay against one host. This class records cookies that
|
||||
arrived as a plain dict flat only, and reports them under the empty domain, to match
|
||||
the localhost fallback in :meth:`flush_to_client`.
|
||||
"""
|
||||
grouped = {domain: dict(cookies) for domain, cookies in self._cookies.items() if cookies}
|
||||
scoped = {name for cookies in grouped.values() for name in cookies}
|
||||
leftovers = {name: value for name, value in self._flat.items() if name not in scoped}
|
||||
if leftovers:
|
||||
grouped[""] = {**grouped.get("", {}), **leftovers}
|
||||
return grouped
|
||||
"""Return cookies as a plain dict (requests RequestsCookieJar compat), filtered by exact domain and path."""
|
||||
return {c.name: c.value or "" for c in self.snapshot() if domain in (None, c.domain) and path in (None, c.path)}
|
||||
|
||||
def clear(self, domain: Optional[str] = None, path: Optional[str] = None, name: Optional[str] = None) -> None:
|
||||
"""Remove cookies (requests RequestsCookieJar compat).
|
||||
|
||||
- ``clear()`` removes all cookies.
|
||||
- ``clear(domain=..., path=..., name=...)`` removes a specific cookie.
|
||||
- ``clear(domain=..., path=..., name=...)`` removes the cookies that match every given filter.
|
||||
"""
|
||||
if name is not None:
|
||||
self._flat.pop(name, None)
|
||||
if domain is not None and domain in self._cookies:
|
||||
self._cookies[domain].pop(name, None)
|
||||
else:
|
||||
for domain_cookies in self._cookies.values():
|
||||
domain_cookies.pop(name, None)
|
||||
self._original_cookies = [
|
||||
cookie
|
||||
for cookie in self._original_cookies
|
||||
if cookie.name != name or (domain is not None and cookie.domain != domain)
|
||||
]
|
||||
self.remove_cookie_on_client(name, domain)
|
||||
elif domain is not None:
|
||||
removed = self._cookies.pop(domain, {})
|
||||
for k in removed:
|
||||
# Only remove from flat if no other domain has same key
|
||||
still_exists = any(k in dc for dc in self._cookies.values())
|
||||
if not still_exists:
|
||||
self._flat.pop(k, None)
|
||||
self._original_cookies = [cookie for cookie in self._original_cookies if cookie.domain != domain]
|
||||
for k in removed:
|
||||
self.remove_cookie_on_client(k, domain)
|
||||
else:
|
||||
self._flat.clear()
|
||||
self._cookies.clear()
|
||||
self._original_cookies.clear()
|
||||
if domain is None and path is None and name is None:
|
||||
self._store.clear()
|
||||
self._from_jar.clear()
|
||||
if self._client is not None:
|
||||
self._client.clear_cookies()
|
||||
return
|
||||
|
||||
def hit(cookie: Cookie) -> bool:
|
||||
return domain in (None, cookie.domain) and path in (None, cookie.path) and name in (None, cookie.name)
|
||||
|
||||
names = {name} if name is not None else {c.name for c in self.snapshot() if hit(c)}
|
||||
for each in names:
|
||||
self.remove_cookie_on_client(each, domain)
|
||||
self._store = {key: c for key, c in list(self._store.items()) if not hit(c)}
|
||||
self._from_jar &= self._store.keys()
|
||||
# removal on the client is by host and name, so put back what a path filter spared
|
||||
for cookie in self.snapshot():
|
||||
if cookie.name in names:
|
||||
self.push(cookie)
|
||||
|
||||
def items(self) -> list[tuple[str, str]]: # type: ignore[override]
|
||||
return list(self._flat.items())
|
||||
return list(self.flat().items())
|
||||
|
||||
def keys(self) -> list[str]: # type: ignore[override]
|
||||
return list(self._flat.keys())
|
||||
return list(self.flat().keys())
|
||||
|
||||
def values(self) -> list[str]: # type: ignore[override]
|
||||
return list(self._flat.values())
|
||||
return list(self.flat().values())
|
||||
|
||||
|
||||
class RnetProxyDict(dict):
|
||||
@@ -671,9 +693,10 @@ class RnetSession:
|
||||
that replaces the impersonate header set instead of overlaying it, which breaks the
|
||||
fingerprint.
|
||||
|
||||
A new client also starts with an empty cookie jar. This method replays the cookies
|
||||
unshackle set itself, then the cookies of every origin this HTTP session has used.
|
||||
That order lets a value the server rotated win over the buffered one. The new client
|
||||
A new client also starts with an empty cookie jar. This method replays the stored
|
||||
cookies, which hold any value the server rotated for them, then the root-path cookies
|
||||
of every origin this HTTP session has used. A cookie the server set on a deeper path
|
||||
that the store never held is lost, because rnet cannot list its jar. The new client
|
||||
goes on the HTTP session last, so a request from another thread keeps the old client
|
||||
until the new one holds the headers and the cookies.
|
||||
"""
|
||||
@@ -739,8 +762,8 @@ class RnetSession:
|
||||
def scoped_cookies(self) -> CookieJar:
|
||||
"""Every cookie this HTTP session sends, each scoped to its domain.
|
||||
|
||||
The live client's cookies of each requested origin come first, then the cookies set
|
||||
through :attr:`cookies` for any other host.
|
||||
The live client's root-path cookies of each requested origin come first, then the
|
||||
cookies stored in :attr:`cookies` with their own paths.
|
||||
"""
|
||||
jar = CookieJar()
|
||||
hosts: dict[str, set[str]] = {}
|
||||
@@ -749,11 +772,11 @@ class RnetSession:
|
||||
for name, value in self.parse_cookie_header(header):
|
||||
jar.set_cookie(create_cookie(name, value, domain=host))
|
||||
hosts.setdefault(name, set()).add(host)
|
||||
for domain, values in self.cookies.get_dict_by_domain().items():
|
||||
bare = domain.lstrip(".")
|
||||
for name, value in values.items():
|
||||
if not any(h == bare or h.endswith("." + bare) for h in hosts.get(name, ())):
|
||||
jar.set_cookie(create_cookie(name, value, domain=domain))
|
||||
for cookie in self.cookies.snapshot():
|
||||
bare = (cookie.domain or "").lstrip(".")
|
||||
live = any(h == bare or h.endswith("." + bare) for h in hosts.get(cookie.name, ()))
|
||||
if not (live and cookie.path in ("", "/")):
|
||||
jar.set_cookie(cookie)
|
||||
return jar
|
||||
|
||||
def import_origin_cookies(self, exported: dict[str, str]) -> None:
|
||||
@@ -813,6 +836,14 @@ class RnetSession:
|
||||
sleep_time = backoff_value + random.uniform(-jitter, jitter)
|
||||
return min(sleep_time, self.max_backoff)
|
||||
|
||||
def absorb_cookies(self, response: RnetResponse) -> RnetResponse:
|
||||
"""Record a value the server set for a stored cookie; a cookie fault must never fail the request."""
|
||||
try:
|
||||
self.cookies.absorb(response)
|
||||
except Exception: # nosec B110 - the store keeps its old value, the rnet jar still has the new one
|
||||
pass
|
||||
return response
|
||||
|
||||
def request(self, method: str, url: str, **kwargs: Any) -> RnetResponse:
|
||||
"""Send a request, retrying on the status forcelist and on the caught exception types.
|
||||
|
||||
@@ -865,16 +896,14 @@ class RnetSession:
|
||||
kwargs["headers"] = dict(kwargs["headers"])
|
||||
|
||||
if method_upper not in self.allowed_methods:
|
||||
raw_resp = client.request(rnet_method, url, **kwargs)
|
||||
return RnetResponse(raw_resp)
|
||||
return self.absorb_cookies(RnetResponse(client.request(rnet_method, url, **kwargs)))
|
||||
|
||||
last_exception: Optional[Exception] = None
|
||||
response: Optional[RnetResponse] = None
|
||||
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
raw_resp = client.request(rnet_method, url, **kwargs)
|
||||
response = RnetResponse(raw_resp)
|
||||
response = self.absorb_cookies(RnetResponse(client.request(rnet_method, url, **kwargs)))
|
||||
|
||||
if config.debug_requests:
|
||||
parsed_url = urlparse(url)
|
||||
|
||||
@@ -14,7 +14,7 @@ from unshackle.core.constants import (
|
||||
VIDEO_CODEC_MAP,
|
||||
)
|
||||
from unshackle.core.tracks import Tracks, Video
|
||||
from unshackle.core.utilities import spoken_language
|
||||
from unshackle.core.utilities import spoken_language, standard_resolution
|
||||
|
||||
|
||||
def rename(name: Optional[str]) -> str:
|
||||
@@ -119,40 +119,14 @@ class Title:
|
||||
context["edition"] = " ".join(first_track.edition)
|
||||
|
||||
if primary_video_track:
|
||||
width = getattr(primary_video_track, "width", primary_video_track.height)
|
||||
resolution = min(width, primary_video_track.height)
|
||||
try:
|
||||
dar = getattr(primary_video_track, "other_display_aspect_ratio", None) or []
|
||||
if dar and dar[0]:
|
||||
aspect_ratio = [int(float(plane)) for plane in str(dar[0]).split(":")]
|
||||
if len(aspect_ratio) == 1:
|
||||
aspect_ratio.append(1)
|
||||
ratio = aspect_ratio[0] / aspect_ratio[1]
|
||||
if ratio not in (16 / 9, 4 / 3, 9 / 16, 3 / 4):
|
||||
if abs(width - 3840) <= 50:
|
||||
width = 3840
|
||||
elif abs(width - 2560) <= 50:
|
||||
width = 2560
|
||||
elif abs(width - 1920) <= 50 or abs(width - 1620) <= 50:
|
||||
width = 1920
|
||||
elif abs(width - 1280) <= 50 or abs(width - 1080) <= 50:
|
||||
width = 1280
|
||||
|
||||
resolution = int(max(width, primary_video_track.height) * (9 / 16))
|
||||
|
||||
track_height = primary_video_track.height
|
||||
if abs(resolution - track_height) <= 10 or track_height in (2160, 1440, 1080, 720, 480):
|
||||
resolution = track_height
|
||||
# garbage DAR strings or a missing height fall back to the plain min(width, height)
|
||||
except (ValueError, TypeError, ZeroDivisionError):
|
||||
pass
|
||||
resolution = standard_resolution(primary_video_track.width, primary_video_track.height)
|
||||
|
||||
scan_suffix = "i" if str(getattr(primary_video_track, "scan_type", "")).lower() == "interlaced" else "p"
|
||||
|
||||
context.update(
|
||||
{
|
||||
"quality": f"{resolution}{scan_suffix}",
|
||||
"resolution": str(resolution),
|
||||
"quality": f"{resolution}{scan_suffix}" if resolution else "",
|
||||
"resolution": str(resolution) if resolution else "",
|
||||
"video": rename(VIDEO_CODEC_MAP.get(primary_video_track.format, primary_video_track.format)),
|
||||
}
|
||||
)
|
||||
@@ -163,6 +137,14 @@ class Title:
|
||||
if (primary_video_track.hdr_format or "").startswith("Dolby Vision"):
|
||||
context["hdr"] = "DV"
|
||||
base_layer = DYNAMIC_RANGE_MAP.get(hdr_format)
|
||||
profile = getattr(primary_video_track, "hdr_format_profile", None) or ""
|
||||
if (
|
||||
base_layer in (None, "DV")
|
||||
and trc
|
||||
and "PQ" in trc
|
||||
and profile.startswith(("dvhe.07", "dvhe.08"))
|
||||
):
|
||||
base_layer = "HDR"
|
||||
if base_layer and base_layer != "DV":
|
||||
context["hdr"] += f".{base_layer}"
|
||||
elif (primary_video_track.hdr_format or "").startswith("HDR Vivid"):
|
||||
@@ -173,6 +155,8 @@ class Title:
|
||||
context["hdr"] = DYNAMIC_RANGE_MAP["HDR10+"]
|
||||
elif trc and "HLG" in trc:
|
||||
context["hdr"] = "HLG"
|
||||
elif trc and "PQ" in trc:
|
||||
context["hdr"] = "HDR"
|
||||
else:
|
||||
context["hdr"] = ""
|
||||
context["hdr"] = ".".join(filter(None, map(rename, context["hdr"].split("."))))
|
||||
|
||||
@@ -14,7 +14,7 @@ from rich.rule import Rule
|
||||
from unshackle.core.binaries import FFMPEG, FFProbe, HDR10PlusTool
|
||||
from unshackle.core.config import config
|
||||
from unshackle.core.console import console
|
||||
from unshackle.core.utilities import log_event
|
||||
from unshackle.core.utilities import log_event, standard_resolution
|
||||
from unshackle.core.utils import dovi
|
||||
from unshackle.core.utils.subprocess import run_step
|
||||
|
||||
@@ -136,7 +136,11 @@ class Hybrid:
|
||||
hdr10_track = next((v for v in videos if v.range == Video.Range.HDR10), None)
|
||||
hdr10p_track = next((v for v in videos if v.range == Video.Range.HDR10P), None)
|
||||
track_for_res = hdr10_track or hdr10p_track
|
||||
self.resolution = f"{track_for_res.height}p" if track_for_res and track_for_res.height else "Unknown"
|
||||
self.resolution = (
|
||||
f"{standard_resolution(track_for_res.width, track_for_res.height)}p"
|
||||
if track_for_res and track_for_res.height
|
||||
else "Unknown"
|
||||
)
|
||||
|
||||
console.print(Padding(Rule(f"[rule.text]HDR10+DV Hybrid ({self.resolution})"), (1, 2)))
|
||||
|
||||
|
||||
@@ -59,22 +59,15 @@ def direct_session(session: AnySession, proxy: Optional[str] = None) -> Session:
|
||||
pass
|
||||
cookies = getattr(session, "cookies", None)
|
||||
if cookies is not None:
|
||||
jar = getattr(cookies, "jar", None)
|
||||
snapshot = getattr(cookies, "snapshot", None)
|
||||
try:
|
||||
new.cookies.update(jar if jar is not None else cookies)
|
||||
if snapshot is None:
|
||||
new.cookies.update(cookies)
|
||||
else:
|
||||
for cookie in snapshot():
|
||||
new.cookies.set_cookie(copy(cookie)) # set_cookie edits a quoted value in place
|
||||
except Exception:
|
||||
pass
|
||||
# RnetCookieAdapter.jar holds only cookies that arrived as a CookieJar; the ones set by name or
|
||||
# received from a server live in its domain map, so copy those across too
|
||||
by_domain = getattr(cookies, "get_dict_by_domain", None)
|
||||
if by_domain is not None:
|
||||
try:
|
||||
for domain, named in by_domain().items():
|
||||
for name, value in named.items():
|
||||
if name not in new.cookies:
|
||||
new.cookies.set(name, value, **({"domain": domain} if domain else {}))
|
||||
except Exception:
|
||||
pass
|
||||
new.mount(
|
||||
"https://",
|
||||
HTTPAdapter(
|
||||
|
||||
@@ -34,6 +34,7 @@ from unshackle.core.utilities import (
|
||||
matching_languages,
|
||||
original_tag,
|
||||
sanitize_filename,
|
||||
standard_resolution,
|
||||
valid_language,
|
||||
)
|
||||
from unshackle.core.utils.collections import as_list, flatten
|
||||
@@ -44,8 +45,12 @@ MP4BOX_OPEN_FAILED = re.compile(r"error while opening|Error opening file|Invalid
|
||||
|
||||
|
||||
def at_resolution(video: Video, res: int) -> bool:
|
||||
"""Whether a track has this height, or this height from a 16:9 width (a letterboxed encode)."""
|
||||
return video.height == res or bool(video.width and int(video.width * 9 / 16) == res)
|
||||
"""Whether a track has this height, this height from a 16:9 width, or this standard resolution (a padded encode)."""
|
||||
return (
|
||||
video.height == res
|
||||
or bool(video.width and int(video.width * 9 / 16) == res)
|
||||
or standard_resolution(video.width, video.height) == res
|
||||
)
|
||||
|
||||
|
||||
def file_track_ids(path: Path) -> dict[str, int]:
|
||||
@@ -308,6 +313,23 @@ class Tracks:
|
||||
kept.append(track)
|
||||
setattr(self, kind, kept)
|
||||
|
||||
def dedupe_subtitle_codecs(self, priority: Sequence[Subtitle.Codec]) -> None:
|
||||
"""
|
||||
Drop a subtitle when the same language, flag set and name exists in a codec earlier in priority.
|
||||
A subtitle in an unlisted codec is always kept.
|
||||
"""
|
||||
rank = {codec: i for i, codec in reversed(list(enumerate(priority)))}
|
||||
|
||||
def variant(sub: Subtitle) -> tuple:
|
||||
signature = self.signature(sub)
|
||||
return signature[:2] + signature[3:] # the signature without its codec
|
||||
|
||||
best: dict[tuple, int] = {}
|
||||
for sub in self.subtitles:
|
||||
if sub.codec in rank:
|
||||
best[variant(sub)] = min(best.get(variant(sub), rank[sub.codec]), rank[sub.codec])
|
||||
self.subtitles = [s for s in self.subtitles if s.codec not in rank or rank[s.codec] == best[variant(s)]]
|
||||
|
||||
def sort_videos(
|
||||
self, by_language: Optional[Sequence[Union[str, Language]]] = None, exact_match: bool = False
|
||||
) -> None:
|
||||
@@ -556,7 +578,7 @@ class Tracks:
|
||||
for group in groups.values():
|
||||
matches = [x for x in group if x.height == resolution]
|
||||
if not matches:
|
||||
matches = [x for x in group if x.width and int(x.width * (9 / 16)) == resolution]
|
||||
matches = [x for x in group if at_resolution(x, resolution)]
|
||||
keep.update(id(x) for x in matches[: per_resolution or None])
|
||||
self.videos = [x for x in self.videos if id(x) in keep]
|
||||
|
||||
|
||||
@@ -122,6 +122,29 @@ def import_module_by_path(path: Path) -> ModuleType:
|
||||
return module
|
||||
|
||||
|
||||
RESOLUTION_TIERS = (4320, 2160, 1440, 1080, 720, 576, 540, 480, 360, 240)
|
||||
|
||||
|
||||
def standard_resolution(width: Optional[int], height: Optional[int]) -> Optional[int]:
|
||||
"""The resolution a release name uses: the nearest standard height within 10%, from the short side or a 16:9 width.
|
||||
|
||||
A cropped 1920x800, a padded 1920x1088 and a 1080x1920 portrait picture are all 1080. A size with no
|
||||
standard height near it keeps its own, and a picture with no size gives None.
|
||||
"""
|
||||
sides = [side for side in (width, height) if side]
|
||||
if not sides:
|
||||
return None
|
||||
short, long = min(sides), max(sides)
|
||||
# naming policy: a short side near 2160, 1440 or 1080 names the picture, so an ultrawide 2560x1080 is 1080
|
||||
# as other media tools name it, even though a 1440 scope crop has the same size
|
||||
by_height = next((tier for tier in (2160, 1440, 1080) if abs(short - tier) <= 16), None)
|
||||
if by_height:
|
||||
return by_height
|
||||
equivalent = max(short, long * 9 / 16)
|
||||
tier = min(RESOLUTION_TIERS, key=lambda t: abs(t - equivalent))
|
||||
return tier if abs(tier - equivalent) <= tier / 10 else round(equivalent)
|
||||
|
||||
|
||||
def sanitize_filename(filename: str, spacer: str = ".", unicode: Optional[bool] = None) -> str:
|
||||
"""
|
||||
Sanitise a string to be filename safe.
|
||||
|
||||
Reference in New Issue
Block a user