forked from kenzuya/unshackle
feat(metadata)!: authoritative id resolver and keyless imdb provider
Supplied IDs resolve directly via a shared resolve_by_ids and never fall back to title search; metadata_providers gains per-kind ordering and a disable_metadata switch stops all automatic lookups while explicit IDs keep working. BREAKING: --tmdb/--imdb/--tvdb are now mutually exclusive and require a usable provider; --enrich also overwrites original language; the imdbapi provider and imdb_api_enabled are replaced by a keyless imdb provider.
This commit is contained in:
@@ -392,9 +392,9 @@ Create a download job. Requires `service` and `title_id`; every other field is a
|
||||
| `best_available` | boolean | `false` | Fall back to best available. |
|
||||
| `repack` | boolean | `false` | Add REPACK tag. |
|
||||
| `tag` | string | `null` | Release group tag. |
|
||||
| `tmdb_id` / `imdb_id` / `tvdb_id` / `animeapi_id` | - | `null` | External ID overrides for tagging. |
|
||||
| `tmdb_id` / `imdb_id` / `tvdb_id` / `animeapi_id` | - | `null` | External ID overrides. Each resolves its metadata directly instead of by a title search, and is used for tagging. Set `enrich` to also take the title, year and original language. Give at most one of `tmdb_id`, `imdb_id` and `tvdb_id`, since unshackle resolves the others from it. Sending two returns `400`. `animeapi_id` still combines with one of them. `tmdb_id` and `tvdb_id` must be positive integers and `imdb_id` must look like `tt1375666`, or the request returns `400`. An ID whose provider is unconfigured, such as `tmdb_id` with no `tmdb_api_key`, fails the job rather than returning `400`. |
|
||||
| `tvdb_order` | `official`, `dvd`, `absolute`, `alternate`, `regional` | `null` | Renumber episodes to a TVDB season order. Falls back to the `tvdb_order` config option. |
|
||||
| `enrich` | boolean | `false` | Override title/year from external source, and fill in the original language. Needs one of `tmdb_id`, `imdb_id`, `tvdb_id` or `animeapi_id`; without one the job fails instead of returning `400`. |
|
||||
| `enrich` | boolean | `false` | Overwrite title, year and original language with the external source's. Needs one of `tmdb_id`, `imdb_id`, `tvdb_id` or `animeapi_id`. Without one the job fails instead of returning `400`. |
|
||||
| `output_dir` | string | `null` | Override output directory. |
|
||||
| `no_cache` / `reset_cache` | boolean | `false` | Title cache controls. |
|
||||
|
||||
|
||||
@@ -150,13 +150,19 @@ Keep only certain track types, or skip certain track types. Attachments are alwa
|
||||
|
||||
| Flag | Description |
|
||||
|---|---|
|
||||
| `--tmdb` | TMDB ID (integer). |
|
||||
| `--imdb` | IMDb ID, e.g. `tt1375666`. |
|
||||
| `--tvdb` | TVDB ID (integer). Skips the series lookup that `--tvdb-order` would otherwise do. `--enrich` reads it too. |
|
||||
| `--tmdb` | TMDB ID (integer). Used for the tags. Skips the title search. `--enrich` reads it too. Needs `tmdb_api_key`. |
|
||||
| `--imdb` | IMDb ID, e.g. `tt1375666`. Used for the tags. Skips the title search. `--enrich` reads it too. Needs no key. |
|
||||
| `--tvdb` | TVDB ID (integer). Used for the tags. Skips the series lookup that `--tvdb-order` would otherwise do. `--enrich` reads it too. Needs `tvdb_api_key`. |
|
||||
| `--animeapi` | AnimeAPI ID, e.g. `mal:12345` or `anilist:98765` (defaults to MAL). Back-fills TMDB/IMDb/TVDB. |
|
||||
| `--enrich` | Override show title/year from the external source, and fill in the original language. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. |
|
||||
| `--enrich` | Overwrite show title, year and original language with the external source's. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. |
|
||||
| `--tvdb-order` | Renumber episodes to a TVDB season order: `official` (aired), `dvd`, `absolute`, `alternate`, or `regional`. Needs `tvdb_api_key`. |
|
||||
|
||||
!!! note "One ID at a time"
|
||||
`--tmdb`, `--imdb` and `--tvdb` cannot be combined. Give one and unshackle resolves the
|
||||
others from it, writing all three to the tags. `--animeapi` still pairs with one of them.
|
||||
An ID with no provider that can resolve it, such as `--tmdb` with no `tmdb_api_key`, fails
|
||||
before the download starts.
|
||||
|
||||
#### Episode ordering
|
||||
|
||||
A service does not always number a series the way TVDB's aired order does. Some services list
|
||||
|
||||
+43
-15
@@ -621,50 +621,78 @@ unshackle dl --skip-dl --export EXAMPLE 81234567
|
||||
## Metadata and tagging
|
||||
|
||||
unshackle looks up metadata automatically, but you can override the identifiers used for
|
||||
tagging and naming:
|
||||
tagging and naming. An ID you give is authoritative: unshackle looks it up directly instead of
|
||||
searching by title, and the ID stays in the tags. An ID only settles *which* title this is.
|
||||
To also take that source's title, year and original language, add `--enrich`. Give at most one
|
||||
of `--tmdb`, `--imdb` and `--tvdb`; unshackle works the rest out from it. To stop the automatic
|
||||
lookups but keep the IDs you give, set
|
||||
[`disable_metadata`](../reference/configuration/misc.md#external-api-keys) in your config.
|
||||
|
||||
| Flag | Example | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `--tmdb` | `--tmdb 27205` | Use this TMDB ID instead of automatic lookup. |
|
||||
| `--imdb` | `--imdb tt1375666` | Use this IMDb ID. |
|
||||
| `--tvdb` | `--tvdb 73871` | Use this TVDB ID instead of looking the series up. |
|
||||
| `--tmdb` | `--tmdb 27205` | Use this TMDB ID instead of an automatic search. Needs `tmdb_api_key`. |
|
||||
| `--imdb` | `--imdb tt1375666` | Use this IMDb ID instead of an automatic search. Needs no key. |
|
||||
| `--tvdb` | `--tvdb 73871` | Use this TVDB ID instead of looking the series up. Needs `tvdb_api_key`. |
|
||||
| `--animeapi` | `--animeapi mal:12345` | Resolve via AnimeAPI (`mal:`/`anilist:` prefix; defaults to MAL). |
|
||||
| `--enrich` | - | Override the show title and year from an external source, and fill in the original language. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. |
|
||||
| `--enrich` | - | Overwrite the show title, year and original language with the external source's. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. |
|
||||
| `--tvdb-order` | `--tvdb-order dvd` | Renumber episodes to a TVDB season order. Needs `tvdb_api_key`. |
|
||||
|
||||
```shell title="Force the right IMDb match and enrich the title"
|
||||
```shell title="Force the right IMDb match, and keep the service's own naming"
|
||||
unshackle dl --imdb tt1375666 EXAMPLE 81234567
|
||||
```
|
||||
|
||||
```shell title="Force the match and take IMDb's title, year and language too"
|
||||
unshackle dl --imdb tt1375666 --enrich EXAMPLE 81234567
|
||||
```
|
||||
|
||||
### What each metadata provider supplies
|
||||
|
||||
Providers differ in what they answer with, so `--enrich` reads different fields depending on
|
||||
which ID you gave it. A provider only runs when its key is configured:
|
||||
Providers differ in what they answer with, so the fields `--enrich` can replace depend on
|
||||
which ID you gave. Every provider except `imdb` only runs when its key is configured:
|
||||
|
||||
| Provider | Config key | Title and year | Original language | External IDs it returns |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| TMDB | `tmdb_api_key` | yes | yes, alpha-2 such as `ko` | IMDb, TMDB, TVDB |
|
||||
| TVDB | `tvdb_api_key` | yes | yes, alpha-3 such as `kor` | TMDB, TVDB, sometimes IMDb |
|
||||
| OMDb | `omdb_api_key` | yes | yes, as an English name such as `Korean` | IMDb |
|
||||
| IMDxAPI | `imdb_api_enabled` | yes | only when looked up by ID | IMDb |
|
||||
| IMDb | *(no key needed)* | yes | yes, alpha-2 such as `ko` | IMDb |
|
||||
| SIMKL | `simkl_client_id` | yes | no, it publishes a country and no language | IMDb, TMDB, TVDB |
|
||||
|
||||
Whatever the tag looks like, unshackle normalises it before use, so `ko`, `kor` and `Korean`
|
||||
all end up as the same language.
|
||||
|
||||
`--enrich` consults these in a fixed order:
|
||||
Each ID goes to the providers that read that kind of ID, in
|
||||
[`metadata_providers`](../reference/configuration/misc.md#external-api-keys) order:
|
||||
|
||||
- `--tmdb` reads TMDB.
|
||||
- `--imdb` tries IMDxAPI, then falls back to OMDb. Since `imdb_api_enabled` is `false` by
|
||||
default, an OMDb key is what makes this path work for most people.
|
||||
- `--imdb` reads IMDb, then falls back to OMDb. IMDb needs no key, so this path works out of
|
||||
the box, and an OMDb key only matters when IMDb has no answer.
|
||||
- `--tvdb` reads TVDB. It needs `tvdb_api_key`.
|
||||
- `--animeapi` supplies a title, plus whichever IDs AnimeAPI knows for the show. If that
|
||||
includes a TMDB, IMDb or TVDB ID, the year and language come from that provider. If it
|
||||
does not, pair it with `--tmdb`, `--imdb` or `--tvdb`.
|
||||
|
||||
Language is the one field `--enrich` will not overwrite. A service that already reported an
|
||||
original language keeps it, because the platform knows its own catalogue better than a
|
||||
third-party match does. Everything else follows the usual override behaviour.
|
||||
Give **one** of `--tmdb`, `--imdb` and `--tvdb`. They cannot be combined, because one ID does
|
||||
the job on its own: unshackle resolves the other two from it and writes all three to the tags.
|
||||
Passing two is an error rather than a silent choice between them. `--animeapi` is the exception
|
||||
and still pairs with one of the three, which is what to do when AnimeAPI knows no IDs for the
|
||||
show.
|
||||
|
||||
The ID you give must have a provider that can actually resolve it, so unshackle checks that
|
||||
before it downloads anything. `--tmdb` without `tmdb_api_key`, or an ID whose providers your
|
||||
`metadata_providers` list leaves out, fails immediately with a message naming what to set.
|
||||
`--imdb` needs no key, since the `imdb` provider is keyless and in the default order.
|
||||
|
||||
With `--enrich`, the title, year and original language are all replaced with the external
|
||||
source's values, whether or not the service already filled them in. A field the source does
|
||||
not answer with is left alone, and unshackle logs which fields those were, so a provider with
|
||||
a thin record cannot blank out what the service told you.
|
||||
|
||||
!!! warning "`--enrich` replaces the original language, which affects more than the filename"
|
||||
Track selection reads the original language, so replacing it changes which audio is
|
||||
treated as the original. That is the point when a service mislabels it, but it means a
|
||||
wrong ID can pick the wrong audio track and not only write a wrong name. Without
|
||||
`--enrich` the service's own value is kept and track selection is untouched.
|
||||
|
||||
### Episode ordering
|
||||
|
||||
|
||||
@@ -36,7 +36,7 @@ Where the file lives, how unshackle discovers it, and how to edit it:
|
||||
| [DRM & CDM](drm.md) | `cdm`, `remote_cdm`, `decryption` |
|
||||
| [Network & proxy](network.md) | `network`, `headers`, `proxy_providers` |
|
||||
| [Key vaults](vaults.md) | `key_vaults`, `vault_timeout` |
|
||||
| [External API keys](misc.md#external-api-keys) | `imdb_api_enabled`, `omdb_api_key`, `tmdb_api_key`, `tvdb_api_key`, `tvdb_pin`, `tvdb_order`, `simkl_client_id`, `metadata_providers`, `decrypt_labs_api_key`, `ipinfo_api_key` |
|
||||
| [External API keys](misc.md#external-api-keys) | `omdb_api_key`, `tmdb_api_key`, `tvdb_api_key`, `tvdb_pin`, `tvdb_order`, `simkl_client_id`, `metadata_providers`, `disable_metadata`, `decrypt_labs_api_key`, `ipinfo_api_key` |
|
||||
| [Caching & updates](misc.md#caching-updates) | `title_cache_enabled`, `title_cache_time`, `title_cache_max_retention`, `update_checks`, `update_check_interval` |
|
||||
| [Logging, privacy & debug](misc.md#logging-privacy-debug) | `redact_paths`, `debug`, `debug_keys`, `debug_requests`, `set_terminal_bg` |
|
||||
| [Deprecated & removed](misc.md#deprecated-removed-keys) | `curl_impersonate`, `downloader`, `scene_naming` |
|
||||
|
||||
@@ -10,14 +10,14 @@ each metadata provider can actually answer with, see
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `imdb_api_enabled` | bool | `false` | Use the free IMDxAPI (api.tiffara.com) metadata provider, which needs no API key. Off by default since the site has been unreliable. |
|
||||
| `omdb_api_key` | str | `""` | [OMDb](https://www.omdbapi.com/) API key for IMDb metadata lookups; a more reliable alternative to IMDxAPI. Free keys are available on the OMDb site. |
|
||||
| `omdb_api_key` | str | `""` | [OMDb](https://www.omdbapi.com/) API key for IMDb metadata lookups; a fallback for when the keyless `imdb` provider has no answer. Free keys are available on the OMDb site. |
|
||||
| `tmdb_api_key` | str | `""` | TMDB API key for metadata enrichment and external-ID tags. |
|
||||
| `simkl_client_id` | str | `""` | SIMKL client ID for metadata lookups; an alternative/fallback source to TMDB. |
|
||||
| `tvdb_api_key` | str | `""` | [TheTVDB v4](https://thetvdb.com/api-information) API key; a fallback source to TMDB, strongest on TV series. Also the source `--enrich --tvdb` reads. Free keys are available on the TVDB site. |
|
||||
| `tvdb_pin` | str | `""` | Subscriber PIN, only needed for a user-supported TVDB key. Leave empty for a normal project key. |
|
||||
| `tvdb_order` | str | `""` | Default for `--tvdb-order`: renumber episodes to a TVDB season order (`official`, `dvd`, `absolute`, `alternate`, `regional`). Empty keeps the numbering the service gives. |
|
||||
| `metadata_providers` | list | *(see below)* | Metadata providers to use, in the order they are tried. |
|
||||
| `metadata_providers` | list or map | *(see below)* | Metadata providers to use, in the order they are tried. |
|
||||
| `disable_metadata` | bool | `false` | Set to `true` to stop all automatic metadata lookups. An ID you give yourself is still looked up. |
|
||||
| `decrypt_labs_api_key` | str | `""` | Global Decrypt Labs API key (used by remote CDM / vault). |
|
||||
| `ipinfo_api_key` | str | `""` | ipinfo.io API key for IP/region lookups. |
|
||||
|
||||
@@ -33,12 +33,61 @@ each metadata provider can actually answer with, see
|
||||
ignored with a warning. Leave it unset for the default order:
|
||||
|
||||
```yaml
|
||||
metadata_providers: [imdbapi, omdb, simkl, tmdb, tvdb]
|
||||
metadata_providers: [imdb, omdb, simkl, tmdb, tvdb]
|
||||
```
|
||||
|
||||
Whatever the order, a provider is skipped when its API key is missing (or, for
|
||||
`imdbapi`, when `imdb_api_enabled` is `false`). Available names are
|
||||
`imdbapi`, `omdb`, `simkl`, `tmdb`, and `tvdb`.
|
||||
Whatever the order, a provider is skipped when its API key is missing. `imdb` needs no
|
||||
key, so it is always available. Available names are `imdb`, `omdb`, `simkl`, `tmdb`, and
|
||||
`tvdb`. The old name `imdbapi` still works and is read as `imdb`.
|
||||
|
||||
The order applies to a title search and to a direct lookup of an ID you give with
|
||||
`--tmdb`, `--imdb`, or `--tvdb`. A supplied ID always goes to the providers that read
|
||||
that kind of ID, and it never falls back to a title search.
|
||||
|
||||
!!! note "What `disable_metadata` turns off"
|
||||
With `disable_metadata: true`, unshackle never contacts a metadata provider on its own.
|
||||
The title search is dead, and that includes the keyless `imdb` provider. Titles, years and
|
||||
languages then come only from the service.
|
||||
|
||||
```yaml
|
||||
disable_metadata: true
|
||||
```
|
||||
|
||||
An ID you give with `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi` is your permission to use
|
||||
that ID, so it still works: the ID is looked up directly, the other IDs are cross
|
||||
referenced from it, and `--enrich` still reads that source. Only the lookups you did not
|
||||
ask for are stopped.
|
||||
|
||||
!!! note "The `imdb` provider needs no key"
|
||||
It replaces the earlier `imdbapi` provider, which read a third-party mirror
|
||||
(api.tiffara.com) and needed the `imdb_api_enabled` switch. That option is gone: delete it
|
||||
from your config, where it is now ignored.
|
||||
|
||||
!!! tip "A different order per title kind"
|
||||
Give a map instead of one list when the best source differs by kind. TVDB is strongest on
|
||||
series, TMDB on films. There are two kinds, `tv` and `movie`, and this example sets both
|
||||
of them and names every provider, so nothing is left to the default:
|
||||
|
||||
```yaml
|
||||
metadata_providers:
|
||||
tv: [tvdb, tmdb, simkl, imdb, omdb]
|
||||
movie: [tmdb, imdb, omdb, simkl, tvdb]
|
||||
```
|
||||
|
||||
Each list is independent: it both orders and filters that kind on its own, so a provider
|
||||
you leave out of `tv` is still used for `movie`. Here TVDB is tried first for a series and
|
||||
last for a film.
|
||||
|
||||
You do not need to give both keys, or a full list in each. A kind you leave out uses the
|
||||
default order:
|
||||
|
||||
```yaml
|
||||
metadata_providers:
|
||||
tv: [tvdb, tmdb] # movies keep the default order
|
||||
```
|
||||
|
||||
`tv` covers every episode and `movie` covers every film. unshackle only looks up
|
||||
metadata for those two kinds, so any other key, such as `anime` or `music`, is ignored.
|
||||
|
||||
!!! note "`ipinfo_api_key` never touches your service sessions"
|
||||
The token is only ever sent to `api.ipinfo.io` as a per-request `Authorization` header; it
|
||||
@@ -109,3 +158,4 @@ calls.
|
||||
| `curl_impersonate` | **Deprecated** → use [`network`](network.md) | Emits a `DeprecationWarning`; still honoured only if `network` is absent. |
|
||||
| `downloader` | **Deprecated** | Any value other than `"requests"` emits a `DeprecationWarning`; the value is otherwise ignored (the unified requests downloader is always used). |
|
||||
| `scene_naming` | **Removed** | If set to any non-null value, unshackle exits with an error directing you to configure [`output_template`](output.md) instead. |
|
||||
| `imdb_api_enabled` | **Removed** | Ignored. It gated the old `imdbapi` provider, which the keyless [`imdb`](#external-api-keys) provider replaces. Safe to delete from your config. |
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
"""The `disable_metadata` config switch stops automatic lookups but keeps supplied IDs working."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Optional, Union
|
||||
|
||||
import pytest
|
||||
|
||||
from unshackle.core import providers
|
||||
from unshackle.core.config import config
|
||||
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult
|
||||
|
||||
|
||||
class FakeProvider(MetadataProvider):
|
||||
"""Stands in for a real provider; fails the test if it is searched."""
|
||||
|
||||
def __init__(self, name: str, result: Optional[MetadataResult] = None) -> None:
|
||||
self.NAME = name
|
||||
self._result = result
|
||||
self.asked: list[Union[int, str]] = []
|
||||
super().__init__()
|
||||
|
||||
def is_available(self) -> bool:
|
||||
return True
|
||||
|
||||
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
|
||||
raise AssertionError("no provider may be searched while disable_metadata is set")
|
||||
|
||||
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
|
||||
self.asked.append(provider_id)
|
||||
return self._result
|
||||
|
||||
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
|
||||
return ExternalIds()
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _disabled(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Turn metadata off and keep every test off the network."""
|
||||
monkeypatch.setattr(config, "disable_metadata", True)
|
||||
monkeypatch.setattr(config, "metadata_providers", [])
|
||||
monkeypatch.setattr(providers, "fetch_external_ids", lambda *a, **kw: ExternalIds())
|
||||
monkeypatch.setattr(providers, "enrich_ids", lambda result: None)
|
||||
|
||||
|
||||
def test_search_metadata_returns_none(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(providers, "get_provider", lambda name: FakeProvider(name))
|
||||
|
||||
assert providers.search_metadata("Futurama", 1999, "tv") is None
|
||||
|
||||
|
||||
def test_supplied_id_still_resolves(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
tmdb = FakeProvider("tmdb", MetadataResult(title="Inception", year=2010, kind="movie"))
|
||||
monkeypatch.setattr(providers, "get_provider", lambda name: {"tmdb": tmdb}.get(name))
|
||||
|
||||
result = providers.resolve_by_ids(27205, kind="movie")
|
||||
|
||||
assert result is not None
|
||||
assert result.title == "Inception"
|
||||
assert result.external_ids.tmdb_id == 27205
|
||||
assert tmdb.asked == [27205]
|
||||
|
||||
|
||||
def test_no_ids_returns_none(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(providers, "get_provider", lambda name: FakeProvider(name))
|
||||
|
||||
assert providers.resolve_by_ids(title="Futurama", year=1999, kind="tv") is None
|
||||
@@ -1,11 +1,11 @@
|
||||
"""--enrich original-language enrichment."""
|
||||
"""Original-language parsing for the metadata a supplied ID resolves."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from unshackle.commands.dl import parse_language
|
||||
from unshackle.core.providers.imdbapi import primary_language as imdb_language
|
||||
from unshackle.core.providers.imdb import primary_language as imdb_language
|
||||
from unshackle.core.providers.omdb import primary_language as omdb_language
|
||||
from unshackle.core.providers.tvdb import primary_language as tvdb_language
|
||||
|
||||
@@ -28,11 +28,27 @@ def test_parse_language(tag: str | None, expected: str | None) -> None:
|
||||
|
||||
|
||||
def test_imdb_language_takes_the_first_spoken_language() -> None:
|
||||
data = {"spokenLanguages": [{"code": "jpn", "name": "Japanese"}, {"code": "eng", "name": "English"}]}
|
||||
assert imdb_language(data) == "jpn"
|
||||
data = {"spokenLanguages": {"spokenLanguages": [{"id": "ja", "text": "Japanese"}, {"id": "en", "text": "English"}]}}
|
||||
assert imdb_language(data) == "ja"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("data", [{}, {"spokenLanguages": []}, {"spokenLanguages": None}])
|
||||
def test_imdb_language_is_alpha_2() -> None:
|
||||
"""IMDb's GraphQL language ids are alpha-2."""
|
||||
data = {"spokenLanguages": {"spokenLanguages": [{"id": "ko", "text": "Korean"}]}}
|
||||
assert parse_language(imdb_language(data)) is not None
|
||||
assert str(parse_language(imdb_language(data))) == "ko"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"data",
|
||||
[
|
||||
{},
|
||||
{"spokenLanguages": None}, # search hits can come back with the whole block null
|
||||
{"spokenLanguages": {}},
|
||||
{"spokenLanguages": {"spokenLanguages": []}},
|
||||
{"spokenLanguages": {"spokenLanguages": None}},
|
||||
],
|
||||
)
|
||||
def test_imdb_language_missing(data: dict) -> None:
|
||||
assert imdb_language(data) is None
|
||||
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
"""IMDb GraphQL provider: persisted-query flow, field mapping, and search constraints."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any, Optional
|
||||
|
||||
import pytest
|
||||
|
||||
from unshackle.core.providers.imdb import IMDBProvider, escape_graphql
|
||||
|
||||
|
||||
class _Response:
|
||||
def __init__(self, payload: dict) -> None:
|
||||
self._payload = payload
|
||||
|
||||
def json(self) -> dict:
|
||||
return self._payload
|
||||
|
||||
|
||||
def _node(
|
||||
imdb_id: str = "tt3581920",
|
||||
text: str = "The Last of Us",
|
||||
year: Optional[int] = 2023,
|
||||
title_type: str = "tvSeries",
|
||||
languages: Any = ({"id": "en", "text": "English"}, {"id": "id", "text": "Indonesian"}),
|
||||
) -> dict:
|
||||
return {
|
||||
"id": imdb_id,
|
||||
"titleText": {"text": text},
|
||||
"originalTitleText": {"text": text},
|
||||
"releaseYear": {"year": year} if year else None,
|
||||
"titleType": {"id": title_type},
|
||||
"spokenLanguages": {"spokenLanguages": list(languages)} if languages is not None else None,
|
||||
}
|
||||
|
||||
|
||||
class _FakeSession:
|
||||
"""Records GET/POST calls and replays queued payloads."""
|
||||
|
||||
def __init__(self, responses: list[dict]) -> None:
|
||||
self._responses = responses
|
||||
self.gets: list[dict] = []
|
||||
self.posts: list[dict] = []
|
||||
|
||||
def get(self, url: str, params: dict, headers: dict, timeout: int) -> _Response:
|
||||
self.gets.append(params)
|
||||
return _Response(self._responses.pop(0))
|
||||
|
||||
def post(self, url: str, json: dict, headers: dict, timeout: int) -> _Response: # noqa: A002
|
||||
self.posts.append(json)
|
||||
return _Response(self._responses.pop(0))
|
||||
|
||||
|
||||
def _provider(monkeypatch: pytest.MonkeyPatch, responses: list[dict]) -> tuple[IMDBProvider, _FakeSession]:
|
||||
p = IMDBProvider()
|
||||
session = _FakeSession(responses)
|
||||
monkeypatch.setattr(type(p), "session", property(lambda _self: session))
|
||||
return p, session
|
||||
|
||||
|
||||
NOT_FOUND = {"errors": [{"message": "PersistedQueryNotFound", "extensions": {"code": "PERSISTED_QUERY_NOT_FOUND"}}]}
|
||||
|
||||
|
||||
def test_is_available_without_any_key() -> None:
|
||||
assert IMDBProvider().is_available() is True
|
||||
|
||||
|
||||
def test_cache_hit_uses_one_get(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
p, session = _provider(monkeypatch, [{"data": {"title": _node()}}])
|
||||
result = p.get_by_id("tt3581920", "tv")
|
||||
assert result is not None and result.title == "The Last of Us"
|
||||
assert len(session.gets) == 1 and session.posts == []
|
||||
|
||||
|
||||
def test_persisted_query_miss_falls_back_to_one_post(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
p, session = _provider(monkeypatch, [NOT_FOUND, {"data": {"title": _node()}}])
|
||||
result = p.get_by_id("tt3581920", "tv")
|
||||
assert result is not None and result.external_ids.imdb_id == "tt3581920"
|
||||
assert len(session.gets) == 1 and len(session.posts) == 1
|
||||
# the POST registers the query by sending its text, keyed by the same hash the GET sent
|
||||
assert "query UnshackleTitle" in session.posts[0]["query"]
|
||||
sent_hash = json.loads(session.gets[0]["extensions"])["persistedQuery"]["sha256Hash"]
|
||||
assert session.posts[0]["extensions"]["persistedQuery"]["sha256Hash"] == sent_hash
|
||||
|
||||
|
||||
def test_get_params_carry_no_spaces(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""IMDb's GraphQL gateway decodes "+" literally, so spaced JSON would come back as a 400."""
|
||||
p, session = _provider(monkeypatch, [{"data": {"title": _node()}}])
|
||||
p.get_by_id("tt3581920", "tv")
|
||||
assert " " not in session.gets[0]["extensions"]
|
||||
assert " " not in session.gets[0]["variables"]
|
||||
|
||||
|
||||
def test_get_by_id_maps_every_field(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
p, _ = _provider(monkeypatch, [{"data": {"title": _node()}}])
|
||||
result = p.get_by_id("tt3581920", "tv")
|
||||
assert result is not None
|
||||
assert (result.title, result.year, result.kind) == ("The Last of Us", 2023, "tv")
|
||||
assert result.original_language == "en"
|
||||
assert result.source == "imdb"
|
||||
assert result.external_ids.imdb_id == "tt3581920"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("title_type", ["movie", "tvSeries", "videoGame"])
|
||||
def test_caller_kind_wins_over_the_response_title_type(monkeypatch: pytest.MonkeyPatch, title_type: str) -> None:
|
||||
"""kind stays the caller's, so it cannot desync from the tmdb_kind resolve_by_ids seeds."""
|
||||
p, _ = _provider(monkeypatch, [{"data": {"title": _node(title_type=title_type)}}])
|
||||
result = p.get_by_id("tt1375666", "tv")
|
||||
assert result is not None and result.kind == "tv"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("payload", [{"data": {"title": None}}, {"errors": [{"message": "boom"}]}, {"data": {}}])
|
||||
def test_get_by_id_returns_none_on_no_answer(monkeypatch: pytest.MonkeyPatch, payload: dict) -> None:
|
||||
p, _ = _provider(monkeypatch, [payload])
|
||||
assert p.get_by_id("tt3581920", "tv") is None
|
||||
|
||||
|
||||
def _edges(*nodes: dict) -> dict:
|
||||
return {"data": {"advancedTitleSearch": {"edges": [{"node": {"title": n}} for n in nodes]}}}
|
||||
|
||||
|
||||
def test_search_uses_its_own_operation_and_one_get_on_a_cache_hit(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
p, session = _provider(monkeypatch, [_edges(_node())])
|
||||
result = p.search("The Last of Us", 2023, "tv")
|
||||
assert result is not None and result.external_ids.imdb_id == "tt3581920"
|
||||
assert session.gets[0]["operationName"] == "UnshackleTitleSearch"
|
||||
assert session.posts == []
|
||||
|
||||
|
||||
def test_search_query_text_carries_the_constraints(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
p, session = _provider(monkeypatch, [NOT_FOUND, _edges(_node())])
|
||||
p.search("The Last of Us", 2023, "tv")
|
||||
query = session.posts[0]["query"]
|
||||
assert 'searchTerm: "The Last of Us"' in query
|
||||
assert 'start: "2023-01-01", end: "2023-12-31"' in query
|
||||
assert '["tvSeries", "tvMiniSeries"]' in query or '["tvSeries","tvMiniSeries"]' in query
|
||||
|
||||
|
||||
def test_search_omits_the_year_range_when_unknown(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
p, session = _provider(monkeypatch, [NOT_FOUND, _edges(_node(title_type="movie"))])
|
||||
p.search("Inception", None, "movie")
|
||||
query = session.posts[0]["query"]
|
||||
assert "releaseDateConstraint" not in query
|
||||
assert '["movie", "tvMovie"]' in query or '["movie","tvMovie"]' in query
|
||||
|
||||
|
||||
def test_search_picks_the_closest_title_not_the_first(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
noise = _node(imdb_id="tt27478132", text="Making of: The Last of Us")
|
||||
exact = _node()
|
||||
p, _ = _provider(monkeypatch, [_edges(noise, exact)])
|
||||
result = p.search("The Last of Us", 2023, "tv")
|
||||
assert result is not None and result.external_ids.imdb_id == "tt3581920"
|
||||
|
||||
|
||||
def test_search_rejects_a_title_that_is_not_a_close_match(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
p, _ = _provider(monkeypatch, [_edges(_node(imdb_id="tt999", text="Something Else Entirely"))])
|
||||
assert p.search("The Last of Us", 2023, "tv") is None
|
||||
|
||||
|
||||
def test_search_survives_a_null_language_block(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
p, _ = _provider(monkeypatch, [_edges(_node(languages=None))])
|
||||
result = p.search("The Last of Us", 2023, "tv")
|
||||
assert result is not None and result.original_language is None
|
||||
|
||||
|
||||
def test_search_returns_none_when_nothing_comes_back(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
p, _ = _provider(monkeypatch, [_edges()])
|
||||
assert p.search("The Last of Us", 2023, "tv") is None
|
||||
|
||||
|
||||
def test_escape_graphql_closes_the_string_injection() -> None:
|
||||
assert escape_graphql('a"b\\c') == 'a\\"b\\\\c'
|
||||
|
||||
|
||||
def test_search_term_with_a_quote_stays_inside_the_string(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
p, session = _provider(monkeypatch, [NOT_FOUND, _edges(_node(text='The "Best" Show'))])
|
||||
p.search('The "Best" Show', None, "tv")
|
||||
assert 'searchTerm: "The \\"Best\\" Show"' in session.posts[0]["query"]
|
||||
|
||||
|
||||
def test_get_external_ids_echoes_the_id() -> None:
|
||||
assert IMDBProvider().get_external_ids("tt1375666", "movie").imdb_id == "tt1375666"
|
||||
@@ -0,0 +1,114 @@
|
||||
"""Input validation for the --tmdb / --imdb / --tvdb flags."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import click
|
||||
import pytest
|
||||
|
||||
from unshackle.commands.dl import validate_metadata_ids
|
||||
from unshackle.core.api.handlers import validate_download_parameters
|
||||
from unshackle.core.config import config
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _default_order(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(config, "metadata_providers", [])
|
||||
|
||||
|
||||
def _with_keys(monkeypatch: pytest.MonkeyPatch, **keys: str) -> None:
|
||||
for name in ("tmdb_api_key", "tvdb_api_key", "omdb_api_key", "simkl_client_id"):
|
||||
monkeypatch.setattr(config, name, keys.get(name, ""))
|
||||
|
||||
|
||||
# ---------- one ID at a time ----------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("tmdb", "imdb", "tvdb"),
|
||||
[(27205, "tt1375666", None), (27205, None, 73871), (None, "tt1375666", 73871), (27205, "tt1375666", 73871)],
|
||||
)
|
||||
def test_two_ids_are_refused(tmdb: int | None, imdb: str | None, tvdb: int | None) -> None:
|
||||
with pytest.raises(click.UsageError) as exc:
|
||||
validate_metadata_ids(tmdb, imdb, tvdb)
|
||||
assert "cannot be used together" in str(exc.value)
|
||||
|
||||
|
||||
def test_no_ids_is_fine() -> None:
|
||||
assert validate_metadata_ids(None, None, None) is None
|
||||
|
||||
|
||||
def test_imdb_alone_passes_on_the_keyless_provider(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
_with_keys(monkeypatch)
|
||||
assert validate_metadata_ids(None, "tt1375666", None) is None
|
||||
|
||||
|
||||
# ---------- the ID must have a provider that can resolve it ----------
|
||||
|
||||
|
||||
def test_tmdb_without_its_key_names_the_key(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
_with_keys(monkeypatch)
|
||||
with pytest.raises(click.UsageError) as exc:
|
||||
validate_metadata_ids(27205, None, None)
|
||||
assert "--tmdb" in str(exc.value) and "tmdb_api_key" in str(exc.value)
|
||||
|
||||
|
||||
def test_tvdb_without_its_key_names_the_key(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
_with_keys(monkeypatch)
|
||||
with pytest.raises(click.UsageError) as exc:
|
||||
validate_metadata_ids(None, None, 73871)
|
||||
assert "--tvdb" in str(exc.value) and "tvdb_api_key" in str(exc.value)
|
||||
|
||||
|
||||
def test_tmdb_with_its_key_passes(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
_with_keys(monkeypatch, tmdb_api_key="k")
|
||||
assert validate_metadata_ids(27205, None, None) is None
|
||||
|
||||
|
||||
def test_a_provider_filtered_out_of_the_order_points_at_the_config(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
_with_keys(monkeypatch, tmdb_api_key="k")
|
||||
monkeypatch.setattr(config, "metadata_providers", ["imdb", "omdb"])
|
||||
with pytest.raises(click.UsageError) as exc:
|
||||
validate_metadata_ids(27205, None, None)
|
||||
assert "metadata_providers" in str(exc.value)
|
||||
|
||||
|
||||
def test_imdb_is_unresolvable_when_the_order_drops_both_of_its_providers(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
_with_keys(monkeypatch)
|
||||
monkeypatch.setattr(config, "metadata_providers", ["tmdb", "tvdb"])
|
||||
with pytest.raises(click.UsageError) as exc:
|
||||
validate_metadata_ids(None, "tt1375666", None)
|
||||
assert "metadata_providers" in str(exc.value)
|
||||
assert "imdb" in str(exc.value) and "omdb" in str(exc.value)
|
||||
|
||||
|
||||
def test_one_kind_keeping_the_provider_is_enough(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""A per-kind order can drop tvdb for movies and keep it for tv, so this must not error."""
|
||||
_with_keys(monkeypatch, tvdb_api_key="k")
|
||||
monkeypatch.setattr(config, "metadata_providers", {"tv": ["tvdb"], "movie": ["tmdb"]})
|
||||
assert validate_metadata_ids(None, None, 73871) is None
|
||||
|
||||
|
||||
# ---------- the REST path rejects the same combinations ----------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"payload",
|
||||
[
|
||||
{"tmdb_id": 27205, "imdb_id": "tt1375666"},
|
||||
{"tmdb_id": 27205, "tvdb_id": 73871},
|
||||
{"imdb_id": "tt1375666", "tvdb_id": 73871},
|
||||
],
|
||||
)
|
||||
def test_api_refuses_two_ids(payload: dict) -> None:
|
||||
err = validate_download_parameters(payload)
|
||||
assert err and "multiple external IDs" in err
|
||||
|
||||
|
||||
@pytest.mark.parametrize("payload", [{"tmdb_id": 27205}, {"imdb_id": "tt1375666"}, {"tvdb_id": 73871}, {}])
|
||||
def test_api_accepts_one_id(payload: dict) -> None:
|
||||
assert validate_download_parameters(payload) is None
|
||||
|
||||
|
||||
def test_api_still_reports_a_malformed_id_first() -> None:
|
||||
err = validate_download_parameters({"tmdb_id": "12345", "imdb_id": "tt1"})
|
||||
assert err and "positive integer" in err
|
||||
@@ -0,0 +1,131 @@
|
||||
"""Precedence of user-supplied external IDs in providers.resolve_by_ids."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Optional, Union
|
||||
|
||||
import pytest
|
||||
|
||||
from unshackle.core import providers
|
||||
from unshackle.core.config import config
|
||||
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult
|
||||
|
||||
|
||||
class FakeProvider(MetadataProvider):
|
||||
"""Stands in for a real provider; records the ID it was asked for."""
|
||||
|
||||
def __init__(self, name: str, result: Optional[MetadataResult] = None, raises: bool = False) -> None:
|
||||
self.NAME = name
|
||||
self._result = result
|
||||
self._raises = raises
|
||||
self.asked: list[Union[int, str]] = []
|
||||
super().__init__()
|
||||
|
||||
def is_available(self) -> bool:
|
||||
return True
|
||||
|
||||
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
|
||||
raise AssertionError("resolve_by_ids must not search when an ID was supplied")
|
||||
|
||||
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
|
||||
self.asked.append(provider_id)
|
||||
if self._raises:
|
||||
raise ValueError("provider is down")
|
||||
return self._result
|
||||
|
||||
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
|
||||
return ExternalIds()
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _isolate(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Keep every test off the network and on the default provider order."""
|
||||
monkeypatch.setattr(config, "metadata_providers", [])
|
||||
monkeypatch.setattr(providers, "fetch_external_ids", lambda *a, **kw: ExternalIds())
|
||||
monkeypatch.setattr(providers, "enrich_ids", lambda result: None)
|
||||
monkeypatch.setattr(
|
||||
providers,
|
||||
"search_metadata",
|
||||
lambda *a, **kw: pytest.fail("search_metadata must not run when an ID was supplied"),
|
||||
)
|
||||
|
||||
|
||||
def _patch_providers(monkeypatch: pytest.MonkeyPatch, available: dict[str, FakeProvider]) -> None:
|
||||
monkeypatch.setattr(providers, "get_provider", lambda name: available.get(name))
|
||||
|
||||
|
||||
def test_supplied_ids_survive_a_providers_own_answer(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
# tmdb answers with a different imdb id; the one the user gave must win
|
||||
tmdb = FakeProvider(
|
||||
"tmdb",
|
||||
MetadataResult(title="Inception", year=2010, kind="movie", external_ids=ExternalIds(imdb_id="tt9999999")),
|
||||
)
|
||||
_patch_providers(monkeypatch, {"tmdb": tmdb})
|
||||
|
||||
result = providers.resolve_by_ids(27205, "tt1375666", kind="movie")
|
||||
|
||||
assert result is not None
|
||||
assert result.external_ids.tmdb_id == 27205
|
||||
assert result.external_ids.tmdb_kind == "movie"
|
||||
assert result.external_ids.imdb_id == "tt1375666"
|
||||
|
||||
|
||||
def test_imdb_falls_through_to_omdb(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
imdb = FakeProvider("imdb", raises=True)
|
||||
omdb = FakeProvider("omdb", MetadataResult(title="Inception", year=2010, kind="movie"))
|
||||
_patch_providers(monkeypatch, {"imdb": imdb, "omdb": omdb})
|
||||
|
||||
result = providers.resolve_by_ids(imdb_id="tt1375666", kind="movie")
|
||||
|
||||
assert result is not None
|
||||
assert result.title == "Inception"
|
||||
assert imdb.asked == ["tt1375666"]
|
||||
assert omdb.asked == ["tt1375666"]
|
||||
|
||||
|
||||
def test_all_providers_down_synthesizes_a_result(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
_patch_providers(monkeypatch, {})
|
||||
|
||||
result = providers.resolve_by_ids(tvdb_id=73871, title="Futurama", year=1999, kind="tv")
|
||||
|
||||
assert result is not None
|
||||
assert result.title == "Futurama"
|
||||
assert result.year == 1999
|
||||
assert result.external_ids.tvdb_id == 73871
|
||||
|
||||
|
||||
def test_no_ids_searches(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
calls: list[tuple] = []
|
||||
monkeypatch.setattr(providers, "search_metadata", lambda *a, **kw: calls.append(a) or None)
|
||||
_patch_providers(monkeypatch, {})
|
||||
|
||||
providers.resolve_by_ids(title="Futurama", year=1999, kind="tv")
|
||||
|
||||
assert calls and calls[0][:3] == ("Futurama", 1999, "tv")
|
||||
|
||||
|
||||
def test_no_ids_and_no_title_returns_none(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
_patch_providers(monkeypatch, {})
|
||||
assert providers.resolve_by_ids(kind="tv") is None
|
||||
|
||||
|
||||
def test_per_kind_order_puts_tvdb_first_for_tv(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(config, "metadata_providers", {"tv": ["tvdb", "tmdb", "imdb", "omdb"]})
|
||||
tvdb = FakeProvider("tvdb", MetadataResult(title="Futurama", year=1999, kind="tv"))
|
||||
tmdb = FakeProvider("tmdb", MetadataResult(title="Wrong Show", year=2020, kind="tv"))
|
||||
_patch_providers(monkeypatch, {"tvdb": tvdb, "tmdb": tmdb})
|
||||
|
||||
result = providers.resolve_by_ids(615, tvdb_id=73871, kind="tv")
|
||||
|
||||
assert result is not None
|
||||
assert result.title == "Futurama"
|
||||
assert tmdb.asked == []
|
||||
# movies keep the default order, so tmdb is consulted before tvdb there
|
||||
assert [cls.NAME for cls in providers.provider_order("movie")] == list(providers.DEFAULT_ORDER)
|
||||
|
||||
|
||||
def test_flat_list_config_applies_to_both_kinds(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(config, "metadata_providers", ["tvdb", "tmdb"])
|
||||
assert [cls.NAME for cls in providers.provider_order("tv")] == ["tvdb", "tmdb"]
|
||||
assert [cls.NAME for cls in providers.provider_order("movie")] == ["tvdb", "tmdb"]
|
||||
assert [cls.NAME for cls in providers.provider_order()] == ["tvdb", "tmdb"]
|
||||
+108
-56
@@ -134,6 +134,43 @@ def normalize_dl_config(dl_config: dict[str, Any]) -> dict[str, Any]:
|
||||
return {DL_OPTION_ALIASES.get(key, key): value for key, value in dl_config.items()}
|
||||
|
||||
|
||||
ID_PROVIDER_KEYS = {"tmdb": "tmdb_api_key", "tvdb": "tvdb_api_key", "omdb": "omdb_api_key"}
|
||||
|
||||
|
||||
def validate_metadata_ids(tmdb_id: Optional[int], imdb_id: Optional[str], tvdb_id: Optional[int]) -> None:
|
||||
"""Reject ID flags that conflict, or that no configured provider could ever resolve.
|
||||
|
||||
Only ever called with the IDs the user gave, never with the ones --animeapi back-fills.
|
||||
"""
|
||||
supplied = [
|
||||
(flag, value) for flag, value in (("--tmdb", tmdb_id), ("--imdb", imdb_id), ("--tvdb", tvdb_id)) if value
|
||||
]
|
||||
if len(supplied) > 1:
|
||||
raise click.UsageError(
|
||||
f"{', '.join(flag for flag, _ in supplied)} cannot be used together. "
|
||||
"Give one ID and unshackle resolves the others from it."
|
||||
)
|
||||
if not supplied:
|
||||
return
|
||||
|
||||
flag, _ = supplied[0]
|
||||
id_kind = flag.lstrip("-")
|
||||
ordered = {cls.NAME: cls for kind in ("tv", "movie") for cls in providers.provider_order(kind)}
|
||||
consumers = [cls for cls in ordered.values() if cls.ID_KIND == id_kind]
|
||||
if any(cls().is_available() for cls in consumers):
|
||||
return
|
||||
|
||||
if consumers:
|
||||
keys = sorted({ID_PROVIDER_KEYS[cls.NAME] for cls in consumers if cls.NAME in ID_PROVIDER_KEYS})
|
||||
raise click.UsageError(
|
||||
f"{flag} needs the {consumers[0].NAME} provider. Set {' or '.join(keys)} in your config."
|
||||
)
|
||||
names = sorted({cls.NAME for cls in providers.REGISTRY.values() if cls.ID_KIND == id_kind})
|
||||
raise click.UsageError(
|
||||
f"{flag} needs the {' or '.join(names)} provider, which your metadata_providers config leaves out."
|
||||
)
|
||||
|
||||
|
||||
def group_videos_by_variant(videos: list[Video], *, merge: bool) -> list[list[Video]]:
|
||||
"""Group video tracks for muxing.
|
||||
|
||||
@@ -617,14 +654,16 @@ class dl:
|
||||
"tmdb_id",
|
||||
type=int,
|
||||
default=None,
|
||||
help="Use this TMDB ID for tagging instead of automatic lookup.",
|
||||
help="Use this TMDB ID for tagging instead of an automatic search. Add --enrich to also take "
|
||||
"its title, year and original language.",
|
||||
)
|
||||
@click.option(
|
||||
"--tvdb",
|
||||
"tvdb_id",
|
||||
type=int,
|
||||
default=None,
|
||||
help="Use this TVDB ID for --tvdb-order and --enrich instead of looking the series up automatically.",
|
||||
help="Use this TVDB ID for --tvdb-order and tagging instead of looking the series up "
|
||||
"automatically. Add --enrich to also take its title, year and original language.",
|
||||
)
|
||||
@click.option(
|
||||
"--tvdb-order",
|
||||
@@ -638,15 +677,16 @@ class dl:
|
||||
"animeapi_id",
|
||||
type=str,
|
||||
default=None,
|
||||
help="Anime database ID via AnimeAPI (e.g. mal:12345, anilist:98765). Defaults to MAL if no prefix.",
|
||||
help="Anime database ID via AnimeAPI (e.g. mal:12345, anilist:98765). Defaults to MAL if no prefix. "
|
||||
"Back-fills the TMDB, IMDB and TVDB IDs it knows for the show.",
|
||||
)
|
||||
@click.option(
|
||||
"--enrich",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help=(
|
||||
"Override show title and year from external source, and fill in the original language "
|
||||
"if the service did not provide one. Requires --tmdb, --imdb, --tvdb, or --animeapi."
|
||||
"Overwrite the show title, year and original language with the external source's. "
|
||||
"Requires --tmdb, --imdb, --tvdb, or --animeapi."
|
||||
),
|
||||
)
|
||||
@click.option(
|
||||
@@ -654,7 +694,8 @@ class dl:
|
||||
"imdb_id",
|
||||
type=str,
|
||||
default=None,
|
||||
help="Use this IMDB ID (e.g. tt1375666) for tagging instead of automatic lookup.",
|
||||
help="Use this IMDB ID (e.g. tt1375666) for tagging instead of an automatic search. Add --enrich "
|
||||
"to also take its title, year and original language.",
|
||||
)
|
||||
@click.option(
|
||||
"--sub-format",
|
||||
@@ -857,6 +898,7 @@ class dl:
|
||||
|
||||
self.profile = profile
|
||||
self.proxy_requested = bool(proxy)
|
||||
validate_metadata_ids(tmdb_id, imdb_id, tvdb_id)
|
||||
self.tmdb_id = tmdb_id
|
||||
self.imdb_id = imdb_id
|
||||
self.tvdb_id = tvdb_id
|
||||
@@ -1576,48 +1618,32 @@ class dl:
|
||||
if self.animeapi_title:
|
||||
enrich_title = self.animeapi_title
|
||||
|
||||
if self.tmdb_id:
|
||||
enrich_result = providers.resolve_by_ids(
|
||||
self.tmdb_id,
|
||||
self.imdb_id,
|
||||
self.tvdb_id,
|
||||
kind=kind,
|
||||
title_cacher=title_cacher,
|
||||
cache_title_id=cache_title_id,
|
||||
cache_region=cache_region,
|
||||
cache_account_hash=cache_account_hash,
|
||||
)
|
||||
if enrich_result:
|
||||
if not enrich_title:
|
||||
enrich_title = providers.get_title_by_id(
|
||||
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
||||
)
|
||||
enrich_year = providers.get_year_by_id(
|
||||
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
||||
)
|
||||
enrich_lang = parse_language(
|
||||
providers.get_language_by_id(
|
||||
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
||||
)
|
||||
)
|
||||
elif self.imdb_id:
|
||||
for name in ("imdbapi", "omdb"):
|
||||
provider = providers.get_provider(name)
|
||||
imdb_result = provider.get_by_id(self.imdb_id, kind) if provider else None
|
||||
if imdb_result:
|
||||
if not enrich_title:
|
||||
enrich_title = imdb_result.title
|
||||
enrich_year = imdb_result.year
|
||||
enrich_lang = parse_language(imdb_result.original_language)
|
||||
break
|
||||
elif self.tvdb_id:
|
||||
provider = providers.get_provider("tvdb")
|
||||
tvdb_result = provider.get_by_id(self.tvdb_id, kind) if provider else None
|
||||
if tvdb_result:
|
||||
if not enrich_title:
|
||||
enrich_title = tvdb_result.title
|
||||
enrich_year = tvdb_result.year
|
||||
enrich_lang = parse_language(tvdb_result.original_language)
|
||||
enrich_title = enrich_result.title
|
||||
enrich_year = enrich_result.year
|
||||
enrich_lang = parse_language(enrich_result.original_language)
|
||||
|
||||
if enrich_lang:
|
||||
self.log.info(f"Original language from metadata: {enrich_lang}")
|
||||
if not (enrich_title or enrich_year or enrich_lang):
|
||||
self.log.warning("--enrich found no metadata; is the provider's API key configured?")
|
||||
self.log.warning("--enrich found no metadata. Is the provider's API key configured?")
|
||||
elif missing := [
|
||||
name
|
||||
for name, value in (("title", enrich_title), ("year", enrich_year), ("language", enrich_lang))
|
||||
if not value
|
||||
]:
|
||||
self.log.warning(f"--enrich source did not provide {', '.join(missing)}; those fields are unchanged.")
|
||||
self.log.warning(f"--enrich source gave no {', '.join(missing)}. Those fields are unchanged.")
|
||||
|
||||
if enrich_title or enrich_year or enrich_lang:
|
||||
for t in titles if isinstance(titles, (Series, Movies)) else [titles]:
|
||||
@@ -1626,9 +1652,9 @@ class dl:
|
||||
t.title = enrich_title
|
||||
else:
|
||||
t.name = enrich_title
|
||||
if enrich_year and not t.year:
|
||||
if enrich_year:
|
||||
t.year = enrich_year
|
||||
if enrich_lang and not t.language:
|
||||
if enrich_lang:
|
||||
t.language = enrich_lang
|
||||
|
||||
if self.tvdb_order and isinstance(titles, Series):
|
||||
@@ -2332,20 +2358,28 @@ class dl:
|
||||
if isinstance(title, Episode) and not self.tmdb_searched:
|
||||
kind = "tv"
|
||||
tmdb_title: Optional[str] = None
|
||||
if self.tmdb_id:
|
||||
tmdb_title = providers.get_title_by_id(
|
||||
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
||||
)
|
||||
else:
|
||||
result = providers.search_metadata(
|
||||
title.title, title.year, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
||||
)
|
||||
if result and result.title and providers.fuzzy_match(result.title, title.title):
|
||||
has_ids = bool(self.tmdb_id or self.imdb_id or self.tvdb_id)
|
||||
result = providers.resolve_by_ids(
|
||||
self.tmdb_id,
|
||||
self.imdb_id,
|
||||
self.tvdb_id,
|
||||
title=title.title,
|
||||
year=title.year,
|
||||
kind=kind,
|
||||
title_cacher=title_cacher,
|
||||
cache_title_id=cache_title_id,
|
||||
cache_region=cache_region,
|
||||
cache_account_hash=cache_account_hash,
|
||||
)
|
||||
if result:
|
||||
if has_ids:
|
||||
tmdb_title = result.title
|
||||
if not self.tmdb_id:
|
||||
self.tmdb_id = result.external_ids.tmdb_id
|
||||
elif result.title and providers.fuzzy_match(result.title, title.title):
|
||||
self.tmdb_id = result.external_ids.tmdb_id
|
||||
tmdb_title = result.title
|
||||
self.search_source = result.source
|
||||
else:
|
||||
self.tmdb_id = None
|
||||
if list_ or list_titles:
|
||||
if self.tmdb_id:
|
||||
console.print(
|
||||
@@ -2359,8 +2393,17 @@ class dl:
|
||||
self.tmdb_searched = True
|
||||
|
||||
if isinstance(title, Movie) and (list_ or list_titles) and not self.tmdb_id:
|
||||
movie_result = providers.search_metadata(
|
||||
title.name, title.year, "movie", title_cacher, cache_title_id, cache_region, cache_account_hash
|
||||
movie_result = providers.resolve_by_ids(
|
||||
None,
|
||||
self.imdb_id,
|
||||
self.tvdb_id,
|
||||
title=title.name,
|
||||
year=title.year,
|
||||
kind="movie",
|
||||
title_cacher=title_cacher,
|
||||
cache_title_id=cache_title_id,
|
||||
cache_region=cache_region,
|
||||
cache_account_hash=cache_account_hash,
|
||||
)
|
||||
if movie_result and movie_result.external_ids.tmdb_id:
|
||||
console.print(
|
||||
@@ -2373,7 +2416,7 @@ class dl:
|
||||
else:
|
||||
console.print(Padding("Search -> [bright_black]No match found[/]", (0, 5)))
|
||||
|
||||
if self.tmdb_id and getattr(self, "search_source", None) not in ("simkl", "imdbapi"):
|
||||
if self.tmdb_id and getattr(self, "search_source", None) not in ("simkl", "imdb"):
|
||||
kind = "tv" if isinstance(title, Episode) else "movie"
|
||||
providers.fetch_external_ids(
|
||||
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
||||
@@ -3869,8 +3912,17 @@ class dl:
|
||||
tvdb_id = self.tvdb_id
|
||||
if not tvdb_id:
|
||||
show = titles[0]
|
||||
result = providers.search_metadata(
|
||||
show.title, show.year, "tv", title_cacher, cache_title_id, cache_region, cache_account_hash
|
||||
result = providers.resolve_by_ids(
|
||||
self.tmdb_id,
|
||||
self.imdb_id,
|
||||
None,
|
||||
title=show.title,
|
||||
year=show.year,
|
||||
kind="tv",
|
||||
title_cacher=title_cacher,
|
||||
cache_title_id=cache_title_id,
|
||||
cache_region=cache_region,
|
||||
cache_account_hash=cache_account_hash,
|
||||
)
|
||||
tvdb_id = result.external_ids.tvdb_id if result else None
|
||||
if not tvdb_id:
|
||||
|
||||
@@ -926,6 +926,23 @@ def validate_download_parameters(data: Dict[str, Any]) -> Optional[str]:
|
||||
if not isinstance(data["downloads"], int) or data["downloads"] <= 0:
|
||||
return "downloads must be a positive integer"
|
||||
|
||||
for name in ("tmdb_id", "tvdb_id"):
|
||||
if data.get(name) is not None:
|
||||
value = data[name]
|
||||
if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
|
||||
return f"{name} must be a positive integer"
|
||||
|
||||
if data.get("imdb_id") is not None:
|
||||
if not isinstance(data["imdb_id"], str) or not re.fullmatch(r"tt\d+", data["imdb_id"]):
|
||||
return "imdb_id must be an IMDB ID like 'tt1375666'"
|
||||
|
||||
supplied_ids = [name for name in ("tmdb_id", "imdb_id", "tvdb_id") if data.get(name)]
|
||||
if len(supplied_ids) > 1:
|
||||
return (
|
||||
f"Cannot use multiple external IDs: {', '.join(supplied_ids)}. "
|
||||
"Give one ID and unshackle resolves the others from it."
|
||||
)
|
||||
|
||||
exclusive_flags = []
|
||||
if data.get("video_only"):
|
||||
exclusive_flags.append("video_only")
|
||||
|
||||
@@ -758,13 +758,13 @@ async def download(request: web.Request) -> web.Response:
|
||||
description: Set the group tag to be used (default - None)
|
||||
tmdb_id:
|
||||
type: integer
|
||||
description: Use this TMDB ID for tagging (default - None)
|
||||
description: Use this TMDB ID for tagging instead of a title search. Set enrich to also take its title, year and original language. Mutually exclusive with imdb_id and tvdb_id. Needs tmdb_api_key (default - None)
|
||||
animeapi_id:
|
||||
type: string
|
||||
description: Anime database ID via AnimeAPI, e.g. mal:12345 (default - None)
|
||||
enrich:
|
||||
type: boolean
|
||||
description: Override show title and year from external source, and fill in the original language. Requires one of tmdb_id, imdb_id, tvdb_id or animeapi_id (default - false)
|
||||
description: Overwrite show title, year and original language with the external source's. Requires one of tmdb_id, imdb_id, tvdb_id or animeapi_id (default - false)
|
||||
no_folder:
|
||||
type: boolean
|
||||
description: Disable folder creation for TV shows (default - false)
|
||||
@@ -791,10 +791,10 @@ async def download(request: web.Request) -> web.Response:
|
||||
description: Add REPACK tag to the output filename (default - false)
|
||||
imdb_id:
|
||||
type: string
|
||||
description: Use this IMDB ID (e.g. tt1375666) for tagging (default - None)
|
||||
description: Use this IMDB ID (e.g. tt1375666) for tagging instead of a title search. Set enrich to also take its title, year and original language. Mutually exclusive with tmdb_id and tvdb_id (default - None)
|
||||
tvdb_id:
|
||||
type: integer
|
||||
description: Use this TVDB ID for tagging, episode ordering and enrichment (default - None)
|
||||
description: Use this TVDB ID for tagging and episode ordering instead of a series lookup. Set enrich to also take its title, year and original language. Mutually exclusive with tmdb_id and imdb_id. Needs tvdb_api_key (default - None)
|
||||
tvdb_order:
|
||||
type: string
|
||||
enum: [official, dvd, absolute, alternate, regional]
|
||||
|
||||
@@ -3,7 +3,7 @@ from __future__ import annotations
|
||||
import re
|
||||
import warnings
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
from typing import Any, Optional, Union
|
||||
|
||||
import yaml
|
||||
from appdirs import AppDirs
|
||||
@@ -114,13 +114,13 @@ class Config:
|
||||
self.tag: str = kwargs.get("tag") or ""
|
||||
self.tag_group_name: bool = kwargs.get("tag_group_name", True)
|
||||
self.tag_imdb_tmdb: bool = kwargs.get("tag_imdb_tmdb", True)
|
||||
self.imdb_api_enabled: bool = kwargs.get("imdb_api_enabled", False)
|
||||
self.omdb_api_key: str = kwargs.get("omdb_api_key") or ""
|
||||
self.tmdb_api_key: str = kwargs.get("tmdb_api_key") or ""
|
||||
self.tvdb_api_key: str = kwargs.get("tvdb_api_key") or ""
|
||||
self.tvdb_pin: str = kwargs.get("tvdb_pin") or ""
|
||||
self.tvdb_order: str = (kwargs.get("tvdb_order") or "").lower()
|
||||
self.metadata_providers: list = kwargs.get("metadata_providers") or []
|
||||
self.metadata_providers: Union[list, dict] = kwargs.get("metadata_providers") or []
|
||||
self.disable_metadata: bool = kwargs.get("disable_metadata", False)
|
||||
self.simkl_client_id: str = kwargs.get("simkl_client_id") or ""
|
||||
self.decrypt_labs_api_key: str = kwargs.get("decrypt_labs_api_key") or ""
|
||||
self.ipinfo_api_key: str = kwargs.get("ipinfo_api_key") or ""
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Optional
|
||||
from typing import TYPE_CHECKING, Optional, Union
|
||||
|
||||
import requests
|
||||
|
||||
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, fuzzy_match, log
|
||||
from unshackle.core.providers.imdbapi import IMDBApiProvider
|
||||
from unshackle.core.providers.imdb import IMDBProvider
|
||||
from unshackle.core.providers.omdb import OMDBProvider
|
||||
from unshackle.core.providers.simkl import SimklProvider
|
||||
from unshackle.core.providers.tmdb import TMDBProvider
|
||||
@@ -15,18 +15,28 @@ if TYPE_CHECKING:
|
||||
from unshackle.core.title_cacher import TitleCacher
|
||||
|
||||
REGISTRY: dict[str, type[MetadataProvider]] = {
|
||||
cls.NAME: cls for cls in (IMDBApiProvider, OMDBProvider, SimklProvider, TMDBProvider, TVDBProvider)
|
||||
cls.NAME: cls for cls in (IMDBProvider, OMDBProvider, SimklProvider, TMDBProvider, TVDBProvider)
|
||||
}
|
||||
|
||||
# legacy `metadata_providers` names, still accepted
|
||||
ALIASES: dict[str, str] = {"imdbapi": "imdb"}
|
||||
|
||||
# used when `metadata_providers` is unset
|
||||
DEFAULT_ORDER: tuple[str, ...] = ("imdbapi", "omdb", "simkl", "tmdb", "tvdb")
|
||||
DEFAULT_ORDER: tuple[str, ...] = ("imdb", "omdb", "simkl", "tmdb", "tvdb")
|
||||
|
||||
|
||||
def provider_order() -> list[type[MetadataProvider]]:
|
||||
"""Provider classes in the order configured by `metadata_providers`."""
|
||||
def provider_order(kind: Optional[str] = None) -> list[type[MetadataProvider]]:
|
||||
"""Provider classes in the order configured by `metadata_providers`.
|
||||
|
||||
`metadata_providers` is either a flat list applying to both kinds, or a mapping of
|
||||
kind ("tv"/"movie") to its own list. A kind the mapping omits uses `DEFAULT_ORDER`.
|
||||
"""
|
||||
from unshackle.core.config import config
|
||||
|
||||
names = [str(n).lower() for n in (config.metadata_providers or DEFAULT_ORDER)]
|
||||
configured = config.metadata_providers
|
||||
selected = (configured.get(kind) if kind else None) if isinstance(configured, dict) else configured
|
||||
|
||||
names = [ALIASES.get(str(n).lower(), str(n).lower()) for n in (selected or DEFAULT_ORDER)]
|
||||
unknown = [n for n in names if n not in REGISTRY]
|
||||
if unknown:
|
||||
log.warning("Ignoring unknown metadata_providers entries: %s", ", ".join(unknown))
|
||||
@@ -60,7 +70,15 @@ def search_metadata(
|
||||
cache_account_hash: Optional[str] = None,
|
||||
) -> Optional[MetadataResult]:
|
||||
"""Search all available providers for metadata. Returns best match."""
|
||||
ordered = provider_order()
|
||||
from unshackle.core.config import config
|
||||
|
||||
# the one gate for `disable_metadata`: every automatic lookup reaches a provider through
|
||||
# here, while a user-supplied ID goes straight to `resolve_by_ids` and stays allowed
|
||||
if config.disable_metadata:
|
||||
log.debug("Metadata lookups are disabled by config; not searching for %r", title)
|
||||
return None
|
||||
|
||||
ordered = provider_order(kind)
|
||||
|
||||
# Check cache first
|
||||
if title_cacher and cache_title_id:
|
||||
@@ -268,6 +286,78 @@ def fetch_external_ids(
|
||||
return ext
|
||||
|
||||
|
||||
def resolve_by_ids(
|
||||
tmdb_id: Optional[int] = None,
|
||||
imdb_id: Optional[str] = None,
|
||||
tvdb_id: Optional[int] = None,
|
||||
*,
|
||||
title: Optional[str] = None,
|
||||
year: Optional[int] = None,
|
||||
kind: str = "movie",
|
||||
title_cacher: Optional[TitleCacher] = None,
|
||||
cache_title_id: Optional[str] = None,
|
||||
cache_region: Optional[str] = None,
|
||||
cache_account_hash: Optional[str] = None,
|
||||
) -> Optional[MetadataResult]:
|
||||
"""Resolve metadata from user-supplied external IDs, falling back to search only without them.
|
||||
|
||||
A supplied ID is authoritative. It is looked up directly through the providers that
|
||||
consume its namespace, in `metadata_providers` order for `kind`, and it always survives
|
||||
into `external_ids` whatever a provider answers. A fuzzy title search only runs when no
|
||||
ID was supplied at all.
|
||||
"""
|
||||
supplied: dict[str, Union[int, str]] = {}
|
||||
if tmdb_id is not None:
|
||||
supplied["tmdb"] = tmdb_id
|
||||
if imdb_id:
|
||||
supplied["imdb"] = imdb_id
|
||||
if tvdb_id is not None:
|
||||
supplied["tvdb"] = tvdb_id
|
||||
|
||||
if not supplied:
|
||||
if not title:
|
||||
return None
|
||||
return search_metadata(title, year, kind, title_cacher, cache_title_id, cache_region, cache_account_hash)
|
||||
|
||||
result: Optional[MetadataResult] = None
|
||||
for cls in provider_order(kind):
|
||||
provider_id = supplied.get(cls.ID_KIND or "")
|
||||
if provider_id is None:
|
||||
continue
|
||||
p = get_provider(cls.NAME)
|
||||
if not p:
|
||||
continue
|
||||
try:
|
||||
result = p.get_by_id(provider_id, kind)
|
||||
except Exception as exc:
|
||||
log.debug("%s lookup of %s failed: %s", cls.NAME, provider_id, exc)
|
||||
continue
|
||||
if result:
|
||||
break
|
||||
|
||||
if not result:
|
||||
result = MetadataResult(title=title, year=year, kind=kind)
|
||||
|
||||
ids = result.external_ids
|
||||
if tmdb_id is not None:
|
||||
ids.tmdb_id = tmdb_id
|
||||
ids.tmdb_kind = kind
|
||||
if imdb_id:
|
||||
ids.imdb_id = imdb_id
|
||||
if tvdb_id is not None:
|
||||
ids.tvdb_id = tvdb_id
|
||||
|
||||
if ids.tmdb_id and not (ids.imdb_id and ids.tvdb_id):
|
||||
ext = fetch_external_ids(ids.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash)
|
||||
if ext.imdb_id and not ids.imdb_id:
|
||||
ids.imdb_id = ext.imdb_id
|
||||
if ext.tvdb_id and not ids.tvdb_id:
|
||||
ids.tvdb_id = ext.tvdb_id
|
||||
enrich_ids(result)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
# -- Internal helpers --
|
||||
|
||||
|
||||
@@ -276,9 +366,9 @@ def fetch_external_ids(
|
||||
_ENRICHMENT_AUTHORITY: tuple[str, ...] = ("tmdb", "simkl", "tvdb")
|
||||
|
||||
|
||||
def _enrichment_providers() -> list[str]:
|
||||
def _enrichment_providers(kind: Optional[str] = None) -> list[str]:
|
||||
"""Names of configured providers that can resolve an IMDB ID, most trusted first."""
|
||||
configured = {cls.NAME for cls in provider_order() if hasattr(cls, "find_by_imdb_id")}
|
||||
configured = {cls.NAME for cls in provider_order(kind) if hasattr(cls, "find_by_imdb_id")}
|
||||
return [name for name in _ENRICHMENT_AUTHORITY if name in configured]
|
||||
|
||||
|
||||
@@ -298,7 +388,7 @@ def enrich_ids(result: MetadataResult) -> None:
|
||||
kind = result.kind or "movie"
|
||||
|
||||
# Step 1: Collect enrichment results from all available providers
|
||||
authority = {name: i for i, name in enumerate(_enrichment_providers())}
|
||||
authority = {name: i for i, name in enumerate(_enrichment_providers(kind))}
|
||||
enrichments: list[tuple[str, ExternalIds]] = []
|
||||
for provider_name in authority:
|
||||
p = get_provider(provider_name)
|
||||
@@ -477,9 +567,11 @@ def _cached_to_result(cached: dict, provider_name: str, kind: str) -> Optional[M
|
||||
source="tvdb",
|
||||
raw=cached,
|
||||
)
|
||||
elif provider_name == "imdbapi":
|
||||
title = cached.get("primaryTitle") or cached.get("originalTitle")
|
||||
year = cached.get("startYear")
|
||||
elif provider_name == "imdb":
|
||||
from unshackle.core.providers.imdb import primary_language
|
||||
|
||||
title = (cached.get("titleText") or {}).get("text") or (cached.get("originalTitleText") or {}).get("text")
|
||||
year = (cached.get("releaseYear") or {}).get("year")
|
||||
imdb_id = cached.get("id")
|
||||
# Restore enriched IDs that were saved alongside the raw data
|
||||
enriched = cached.get("_enriched_ids", {})
|
||||
@@ -493,7 +585,8 @@ def _cached_to_result(cached: dict, provider_name: str, kind: str) -> Optional[M
|
||||
tmdb_kind=enriched.get("tmdb_kind"),
|
||||
tvdb_id=enriched.get("tvdb_id"),
|
||||
),
|
||||
source="imdbapi",
|
||||
original_language=primary_language(cached),
|
||||
source="imdb",
|
||||
raw=cached,
|
||||
)
|
||||
return None
|
||||
@@ -513,5 +606,6 @@ __all__ = [
|
||||
"get_title_by_id",
|
||||
"get_year_by_id",
|
||||
"provider_order",
|
||||
"resolve_by_ids",
|
||||
"search_metadata",
|
||||
]
|
||||
|
||||
@@ -37,7 +37,7 @@ class MetadataResult:
|
||||
kind: Optional[str] = None # "movie" or "tv"
|
||||
external_ids: ExternalIds = field(default_factory=ExternalIds)
|
||||
original_language: Optional[str] = None # alpha-2 or alpha-3, whichever the provider speaks
|
||||
source: str = "" # provider name, e.g. "tmdb", "simkl", "imdbapi"
|
||||
source: str = "" # provider name, e.g. "tmdb", "simkl", "imdb"
|
||||
raw: Optional[dict] = None # original API response for caching
|
||||
|
||||
|
||||
@@ -46,6 +46,7 @@ class MetadataProvider(metaclass=ABCMeta):
|
||||
|
||||
NAME: str = ""
|
||||
REQUIRES_KEY: bool = True
|
||||
ID_KIND: Optional[str] = None
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.log = logging.getLogger(f"METADATA.{self.NAME.upper()}")
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from difflib import SequenceMatcher
|
||||
from typing import Optional, Union
|
||||
|
||||
import requests
|
||||
|
||||
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, _clean, fuzzy_match
|
||||
|
||||
GRAPHQL_URL = "https://caching.graphql.imdb.com/"
|
||||
|
||||
GRAPHQL_HEADERS = {
|
||||
"Accept": "application/graphql+json, application/json",
|
||||
"Content-Type": "application/json",
|
||||
"Origin": "https://www.imdb.com",
|
||||
"X-Imdb-Client-Name": "imdb-web-next-localized",
|
||||
"X-Imdb-User-Country": "US",
|
||||
}
|
||||
|
||||
KIND_TO_TYPES: dict[str, list[str]] = {
|
||||
"movie": ["movie", "tvMovie"],
|
||||
"tv": ["tvSeries", "tvMiniSeries"],
|
||||
}
|
||||
|
||||
TITLE_FIELDS = (
|
||||
"id titleText { text } originalTitleText { text } releaseYear { year } "
|
||||
"spokenLanguages { spokenLanguages { id text } }"
|
||||
)
|
||||
|
||||
|
||||
def escape_graphql(value: str) -> str:
|
||||
return value.replace("\\", "\\\\").replace('"', '\\"')
|
||||
|
||||
|
||||
def primary_language(data: dict) -> Optional[str]:
|
||||
"""IMDb orders spokenLanguages most prominent first. Either level can come back null."""
|
||||
spoken = data.get("spokenLanguages") or {}
|
||||
langs = spoken.get("spokenLanguages") or []
|
||||
return langs[0].get("id") if langs else None
|
||||
|
||||
|
||||
def _title_to_result(node: dict, kind: str) -> Optional[MetadataResult]:
|
||||
title = (node.get("titleText") or {}).get("text") or (node.get("originalTitleText") or {}).get("text")
|
||||
if not title:
|
||||
return None
|
||||
return MetadataResult(
|
||||
title=title,
|
||||
year=(node.get("releaseYear") or {}).get("year"),
|
||||
kind=kind,
|
||||
external_ids=ExternalIds(imdb_id=node.get("id")),
|
||||
original_language=primary_language(node),
|
||||
source="imdb",
|
||||
raw=node,
|
||||
)
|
||||
|
||||
|
||||
class IMDBProvider(MetadataProvider):
|
||||
"""IMDb metadata provider using IMDb's own GraphQL edge cache (no API key)."""
|
||||
|
||||
NAME = "imdb"
|
||||
REQUIRES_KEY = False
|
||||
ID_KIND = "imdb"
|
||||
|
||||
def is_available(self) -> bool:
|
||||
return True
|
||||
|
||||
def _graphql(self, operation: str, query: str) -> Optional[dict]:
|
||||
"""Run a persisted query, registering it with one POST the first time it is seen."""
|
||||
extensions = {"persistedQuery": {"version": 1, "sha256Hash": hashlib.sha256(query.encode()).hexdigest()}}
|
||||
# IMDb's GraphQL gateway decodes "+" literally, so these JSON params must carry no spaces
|
||||
params = {
|
||||
"operationName": operation,
|
||||
"variables": "{}",
|
||||
"extensions": json.dumps(extensions, separators=(",", ":")),
|
||||
}
|
||||
|
||||
try:
|
||||
r = self.session.get(GRAPHQL_URL, params=params, headers=GRAPHQL_HEADERS, timeout=30)
|
||||
body = r.json()
|
||||
if _persisted_query_missing(body):
|
||||
self.log.debug("Registering persisted query %s", operation)
|
||||
payload = {"operationName": operation, "variables": {}, "query": query, "extensions": extensions}
|
||||
r = self.session.post(GRAPHQL_URL, json=payload, headers=GRAPHQL_HEADERS, timeout=30)
|
||||
body = r.json()
|
||||
except (requests.RequestException, ValueError) as exc:
|
||||
self.log.debug("IMDb GraphQL %s failed: %s", operation, exc)
|
||||
return None
|
||||
|
||||
if body.get("errors"):
|
||||
self.log.debug("IMDb GraphQL %s errors: %s", operation, [e.get("message") for e in body["errors"]][:3])
|
||||
return None
|
||||
return body.get("data")
|
||||
|
||||
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
|
||||
self.log.debug("Searching IMDb for %r (%s, %s)", title, kind, year)
|
||||
|
||||
constraints = [f'titleTextConstraint: {{searchTerm: "{escape_graphql(title)}"}}']
|
||||
if year:
|
||||
constraints.append(
|
||||
f'releaseDateConstraint: {{releaseDateRange: {{start: "{year}-01-01", end: "{year}-12-31"}}}}'
|
||||
)
|
||||
types = KIND_TO_TYPES.get(kind)
|
||||
if types:
|
||||
constraints.append(f"titleTypeConstraint: {{anyTitleTypeIds: {json.dumps(types)}}}")
|
||||
|
||||
query = (
|
||||
"query UnshackleTitleSearch { advancedTitleSearch(first: 5, constraints: {%s}) "
|
||||
"{ edges { node { title { %s } } } } }" % (", ".join(constraints), TITLE_FIELDS)
|
||||
)
|
||||
|
||||
data = self._graphql("UnshackleTitleSearch", query)
|
||||
edges = ((data or {}).get("advancedTitleSearch") or {}).get("edges") or []
|
||||
nodes = [edge["node"]["title"] for edge in edges if (edge.get("node") or {}).get("title")]
|
||||
if not nodes:
|
||||
self.log.debug("IMDb returned no results for %r", title)
|
||||
return None
|
||||
|
||||
best = max(nodes, key=lambda n: _match_ratio(n, title))
|
||||
result = _title_to_result(best, kind)
|
||||
if not result or not result.title or not fuzzy_match(result.title, title):
|
||||
self.log.debug("IMDb title mismatch: searched %r, got %r", title, result.title if result else None)
|
||||
return None
|
||||
|
||||
self.log.debug("IMDb -> %s (ID %s)", result.title, result.external_ids.imdb_id)
|
||||
return result
|
||||
|
||||
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
|
||||
"""Fetch metadata by IMDB ID (e.g. 'tt1375666')."""
|
||||
imdb_id = str(provider_id)
|
||||
self.log.debug("Fetching IMDb title %s", imdb_id)
|
||||
|
||||
query = 'query UnshackleTitle { title(id: "%s") { %s } }' % (escape_graphql(imdb_id), TITLE_FIELDS)
|
||||
data = self._graphql("UnshackleTitle", query)
|
||||
node = (data or {}).get("title")
|
||||
if not node:
|
||||
return None
|
||||
return _title_to_result(node, kind)
|
||||
|
||||
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
|
||||
"""Return external IDs. For IMDB, the provider_id IS the IMDB ID."""
|
||||
return ExternalIds(imdb_id=str(provider_id))
|
||||
|
||||
|
||||
def _match_ratio(node: dict, title: str) -> float:
|
||||
names = [(node.get("titleText") or {}).get("text"), (node.get("originalTitleText") or {}).get("text")]
|
||||
return max((SequenceMatcher(None, _clean(title), _clean(n)).ratio() for n in names if n), default=0.0)
|
||||
|
||||
|
||||
def _persisted_query_missing(body: dict) -> bool:
|
||||
for error in body.get("errors") or []:
|
||||
code = (error.get("extensions") or {}).get("code")
|
||||
if code == "PERSISTED_QUERY_NOT_FOUND" or error.get("message") == "PersistedQueryNotFound":
|
||||
return True
|
||||
return False
|
||||
@@ -1,132 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from difflib import SequenceMatcher
|
||||
from typing import Optional, Union
|
||||
|
||||
import requests
|
||||
|
||||
from unshackle.core.config import config
|
||||
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, _clean, fuzzy_match
|
||||
|
||||
# Mapping from our kind ("movie"/"tv") to IMDxAPI title types
|
||||
KIND_TO_TYPES: dict[str, list[str]] = {
|
||||
"movie": ["movie"],
|
||||
"tv": ["tvSeries", "tvMiniSeries"],
|
||||
}
|
||||
|
||||
|
||||
def primary_language(data: dict) -> Optional[str]:
|
||||
"""IMDb orders spokenLanguages most prominent first, and only detail responses carry it."""
|
||||
langs = data.get("spokenLanguages") or []
|
||||
return langs[0].get("code") if langs else None
|
||||
|
||||
|
||||
class IMDBApiProvider(MetadataProvider):
|
||||
"""IMDb metadata provider using IMDxAPI (api.tiffara.com, free, no API key)."""
|
||||
|
||||
NAME = "imdbapi"
|
||||
REQUIRES_KEY = False
|
||||
BASE_URL = "https://api.tiffara.com"
|
||||
|
||||
def is_available(self) -> bool:
|
||||
return config.imdb_api_enabled
|
||||
|
||||
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
|
||||
self.log.debug("Searching IMDBApi for %r (%s, %s)", title, kind, year)
|
||||
|
||||
try:
|
||||
params: dict[str, str | int] = {"query": title, "limit": 20}
|
||||
r = self.session.get(
|
||||
f"{self.BASE_URL}/search/titles",
|
||||
params=params,
|
||||
timeout=30,
|
||||
)
|
||||
r.raise_for_status()
|
||||
data = r.json()
|
||||
except (requests.RequestException, ValueError) as exc:
|
||||
self.log.debug("IMDBApi search failed: %s", exc)
|
||||
return None
|
||||
|
||||
results = data.get("titles") or data.get("results") or []
|
||||
if not results:
|
||||
self.log.debug("IMDBApi returned no results for %r", title)
|
||||
return None
|
||||
|
||||
# Filter by type if possible
|
||||
type_filter = KIND_TO_TYPES.get(kind, [])
|
||||
filtered = [r for r in results if r.get("type") in type_filter] if type_filter else results
|
||||
candidates = filtered if filtered else results
|
||||
|
||||
# Find best fuzzy match, optionally filtered by year
|
||||
best_match: Optional[dict] = None
|
||||
best_ratio = 0.0
|
||||
|
||||
for candidate in candidates:
|
||||
primary = candidate.get("primaryTitle") or ""
|
||||
original = candidate.get("originalTitle") or ""
|
||||
|
||||
for name in [primary, original]:
|
||||
if not name:
|
||||
continue
|
||||
ratio = SequenceMatcher(None, _clean(title), _clean(name)).ratio()
|
||||
if ratio > best_ratio:
|
||||
# If year provided, prefer matches within 1 year
|
||||
candidate_year = candidate.get("startYear")
|
||||
if year and candidate_year and abs(year - candidate_year) > 1:
|
||||
continue
|
||||
best_ratio = ratio
|
||||
best_match = candidate
|
||||
|
||||
if not best_match:
|
||||
self.log.debug("No matching result found in IMDBApi for %r", title)
|
||||
return None
|
||||
|
||||
result_title = best_match.get("primaryTitle") or best_match.get("originalTitle")
|
||||
if not result_title or not fuzzy_match(result_title, title):
|
||||
self.log.debug("IMDBApi title mismatch: searched %r, got %r", title, result_title)
|
||||
return None
|
||||
|
||||
imdb_id = best_match.get("id")
|
||||
result_year = best_match.get("startYear")
|
||||
|
||||
self.log.debug("IMDBApi -> %s (ID %s)", result_title, imdb_id)
|
||||
|
||||
return MetadataResult(
|
||||
title=result_title,
|
||||
year=result_year,
|
||||
kind=kind,
|
||||
external_ids=ExternalIds(imdb_id=imdb_id),
|
||||
original_language=primary_language(best_match),
|
||||
source="imdbapi",
|
||||
raw=best_match,
|
||||
)
|
||||
|
||||
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
|
||||
"""Fetch metadata by IMDB ID (e.g. 'tt1375666')."""
|
||||
imdb_id = str(provider_id)
|
||||
self.log.debug("Fetching IMDBApi title %s", imdb_id)
|
||||
|
||||
try:
|
||||
r = self.session.get(f"{self.BASE_URL}/titles/{imdb_id}", timeout=30)
|
||||
r.raise_for_status()
|
||||
data = r.json()
|
||||
except (requests.RequestException, ValueError) as exc:
|
||||
self.log.debug("IMDBApi get_by_id failed: %s", exc)
|
||||
return None
|
||||
|
||||
title = data.get("primaryTitle") or data.get("originalTitle")
|
||||
result_year = data.get("startYear")
|
||||
|
||||
return MetadataResult(
|
||||
title=title,
|
||||
year=result_year,
|
||||
kind=kind,
|
||||
external_ids=ExternalIds(imdb_id=data.get("id")),
|
||||
original_language=primary_language(data),
|
||||
source="imdbapi",
|
||||
raw=data,
|
||||
)
|
||||
|
||||
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
|
||||
"""Return external IDs. For IMDB, the provider_id IS the IMDB ID."""
|
||||
return ExternalIds(imdb_id=str(provider_id))
|
||||
@@ -39,6 +39,7 @@ class OMDBProvider(MetadataProvider):
|
||||
|
||||
NAME = "omdb"
|
||||
REQUIRES_KEY = True
|
||||
ID_KIND = "imdb"
|
||||
BASE_URL = "https://www.omdbapi.com/"
|
||||
|
||||
def is_available(self) -> bool:
|
||||
|
||||
@@ -14,6 +14,7 @@ class TMDBProvider(MetadataProvider):
|
||||
|
||||
NAME = "tmdb"
|
||||
REQUIRES_KEY = True
|
||||
ID_KIND = "tmdb"
|
||||
BASE_URL = "https://api.themoviedb.org/3"
|
||||
|
||||
def is_available(self) -> bool:
|
||||
|
||||
@@ -71,6 +71,7 @@ class TVDBProvider(MetadataProvider):
|
||||
|
||||
NAME = "tvdb"
|
||||
REQUIRES_KEY = True
|
||||
ID_KIND = "tvdb"
|
||||
BASE_URL = "https://api4.thetvdb.com/v4"
|
||||
|
||||
def __init__(self) -> None:
|
||||
|
||||
@@ -12,13 +12,9 @@ from unshackle.core import binaries
|
||||
from unshackle.core.config import config
|
||||
from unshackle.core.providers import (
|
||||
ExternalIds,
|
||||
MetadataResult,
|
||||
enrich_ids,
|
||||
fetch_external_ids,
|
||||
fuzzy_match,
|
||||
get_available_providers,
|
||||
get_provider,
|
||||
search_metadata,
|
||||
resolve_by_ids,
|
||||
)
|
||||
from unshackle.core.titles.episode import Episode
|
||||
from unshackle.core.titles.movie import Movie
|
||||
@@ -118,53 +114,13 @@ def tag_file(
|
||||
|
||||
if config.tag_imdb_tmdb:
|
||||
try:
|
||||
providers = get_available_providers()
|
||||
if not providers and tvdb_id is None:
|
||||
has_ids = tmdb_id is not None or imdb_id is not None or tvdb_id is not None
|
||||
if not get_available_providers() and not has_ids:
|
||||
log.debug("No metadata providers available; skipping tag lookup")
|
||||
apply_tags(path, custom_tags)
|
||||
return
|
||||
|
||||
result: Optional[MetadataResult] = None
|
||||
|
||||
# Direct ID lookup path
|
||||
if imdb_id:
|
||||
imdbapi = get_provider("imdbapi")
|
||||
if imdbapi:
|
||||
result = imdbapi.get_by_id(imdb_id, kind)
|
||||
if not result:
|
||||
# IMDxAPI is off or had no answer; the ID alone still resolves the others
|
||||
result = MetadataResult(title=name, year=year, kind=kind)
|
||||
result.external_ids.imdb_id = imdb_id
|
||||
enrich_ids(result)
|
||||
elif tmdb_id is not None:
|
||||
tmdb = get_provider("tmdb")
|
||||
if tmdb:
|
||||
result = tmdb.get_by_id(tmdb_id, kind)
|
||||
if result:
|
||||
ext = tmdb.get_external_ids(tmdb_id, kind)
|
||||
result.external_ids = ext
|
||||
enrich_ids(result)
|
||||
else:
|
||||
# Search across providers in priority order
|
||||
result = search_metadata(name, year, kind)
|
||||
|
||||
# If we got a TMDB ID from search but not the other IDs, fetch them
|
||||
if (
|
||||
result
|
||||
and result.external_ids.tmdb_id
|
||||
and not (result.external_ids.imdb_id and result.external_ids.tvdb_id)
|
||||
):
|
||||
ext = fetch_external_ids(result.external_ids.tmdb_id, kind)
|
||||
if ext.imdb_id and not result.external_ids.imdb_id:
|
||||
result.external_ids.imdb_id = ext.imdb_id
|
||||
if ext.tvdb_id and not result.external_ids.tvdb_id:
|
||||
result.external_ids.tvdb_id = ext.tvdb_id
|
||||
enrich_ids(result)
|
||||
|
||||
if tvdb_id is not None:
|
||||
if result is None:
|
||||
result = MetadataResult(kind=kind)
|
||||
result.external_ids.tvdb_id = tvdb_id
|
||||
result = resolve_by_ids(tmdb_id, imdb_id, tvdb_id, title=name, year=year, kind=kind)
|
||||
|
||||
if result and result.external_ids:
|
||||
standard_tags = _build_tags_from_ids(result.external_ids, kind)
|
||||
|
||||
Reference in New Issue
Block a user