feat(metadata)!: authoritative id resolver and keyless imdb provider

Supplied IDs resolve directly via a shared resolve_by_ids and never fall back to title search; metadata_providers gains per-kind ordering and a disable_metadata switch stops all automatic lookups while explicit IDs keep working.
BREAKING: --tmdb/--imdb/--tvdb are now mutually exclusive and require a usable provider; --enrich also overwrites original language; the imdbapi provider and imdb_api_enabled are replaced by a keyless imdb provider.
This commit is contained in:
imSp4rky
2026-08-11 15:07:24 -06:00
parent 0cec6c9686
commit ebb4e74318
22 changed files with 1035 additions and 293 deletions
+2 -2
View File
@@ -392,9 +392,9 @@ Create a download job. Requires `service` and `title_id`; every other field is a
| `best_available` | boolean | `false` | Fall back to best available. | | `best_available` | boolean | `false` | Fall back to best available. |
| `repack` | boolean | `false` | Add REPACK tag. | | `repack` | boolean | `false` | Add REPACK tag. |
| `tag` | string | `null` | Release group tag. | | `tag` | string | `null` | Release group tag. |
| `tmdb_id` / `imdb_id` / `tvdb_id` / `animeapi_id` | - | `null` | External ID overrides for tagging. | | `tmdb_id` / `imdb_id` / `tvdb_id` / `animeapi_id` | - | `null` | External ID overrides. Each resolves its metadata directly instead of by a title search, and is used for tagging. Set `enrich` to also take the title, year and original language. Give at most one of `tmdb_id`, `imdb_id` and `tvdb_id`, since unshackle resolves the others from it. Sending two returns `400`. `animeapi_id` still combines with one of them. `tmdb_id` and `tvdb_id` must be positive integers and `imdb_id` must look like `tt1375666`, or the request returns `400`. An ID whose provider is unconfigured, such as `tmdb_id` with no `tmdb_api_key`, fails the job rather than returning `400`. |
| `tvdb_order` | `official`, `dvd`, `absolute`, `alternate`, `regional` | `null` | Renumber episodes to a TVDB season order. Falls back to the `tvdb_order` config option. | | `tvdb_order` | `official`, `dvd`, `absolute`, `alternate`, `regional` | `null` | Renumber episodes to a TVDB season order. Falls back to the `tvdb_order` config option. |
| `enrich` | boolean | `false` | Override title/year from external source, and fill in the original language. Needs one of `tmdb_id`, `imdb_id`, `tvdb_id` or `animeapi_id`; without one the job fails instead of returning `400`. | | `enrich` | boolean | `false` | Overwrite title, year and original language with the external source's. Needs one of `tmdb_id`, `imdb_id`, `tvdb_id` or `animeapi_id`. Without one the job fails instead of returning `400`. |
| `output_dir` | string | `null` | Override output directory. | | `output_dir` | string | `null` | Override output directory. |
| `no_cache` / `reset_cache` | boolean | `false` | Title cache controls. | | `no_cache` / `reset_cache` | boolean | `false` | Title cache controls. |
+10 -4
View File
@@ -150,13 +150,19 @@ Keep only certain track types, or skip certain track types. Attachments are alwa
| Flag | Description | | Flag | Description |
|---|---| |---|---|
| `--tmdb` | TMDB ID (integer). | | `--tmdb` | TMDB ID (integer). Used for the tags. Skips the title search. `--enrich` reads it too. Needs `tmdb_api_key`. |
| `--imdb` | IMDb ID, e.g. `tt1375666`. | | `--imdb` | IMDb ID, e.g. `tt1375666`. Used for the tags. Skips the title search. `--enrich` reads it too. Needs no key. |
| `--tvdb` | TVDB ID (integer). Skips the series lookup that `--tvdb-order` would otherwise do. `--enrich` reads it too. | | `--tvdb` | TVDB ID (integer). Used for the tags. Skips the series lookup that `--tvdb-order` would otherwise do. `--enrich` reads it too. Needs `tvdb_api_key`. |
| `--animeapi` | AnimeAPI ID, e.g. `mal:12345` or `anilist:98765` (defaults to MAL). Back-fills TMDB/IMDb/TVDB. | | `--animeapi` | AnimeAPI ID, e.g. `mal:12345` or `anilist:98765` (defaults to MAL). Back-fills TMDB/IMDb/TVDB. |
| `--enrich` | Override show title/year from the external source, and fill in the original language. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. | | `--enrich` | Overwrite show title, year and original language with the external source's. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. |
| `--tvdb-order` | Renumber episodes to a TVDB season order: `official` (aired), `dvd`, `absolute`, `alternate`, or `regional`. Needs `tvdb_api_key`. | | `--tvdb-order` | Renumber episodes to a TVDB season order: `official` (aired), `dvd`, `absolute`, `alternate`, or `regional`. Needs `tvdb_api_key`. |
!!! note "One ID at a time"
`--tmdb`, `--imdb` and `--tvdb` cannot be combined. Give one and unshackle resolves the
others from it, writing all three to the tags. `--animeapi` still pairs with one of them.
An ID with no provider that can resolve it, such as `--tmdb` with no `tmdb_api_key`, fails
before the download starts.
#### Episode ordering #### Episode ordering
A service does not always number a series the way TVDB's aired order does. Some services list A service does not always number a series the way TVDB's aired order does. Some services list
+43 -15
View File
@@ -621,50 +621,78 @@ unshackle dl --skip-dl --export EXAMPLE 81234567
## Metadata and tagging ## Metadata and tagging
unshackle looks up metadata automatically, but you can override the identifiers used for unshackle looks up metadata automatically, but you can override the identifiers used for
tagging and naming: tagging and naming. An ID you give is authoritative: unshackle looks it up directly instead of
searching by title, and the ID stays in the tags. An ID only settles *which* title this is.
To also take that source's title, year and original language, add `--enrich`. Give at most one
of `--tmdb`, `--imdb` and `--tvdb`; unshackle works the rest out from it. To stop the automatic
lookups but keep the IDs you give, set
[`disable_metadata`](../reference/configuration/misc.md#external-api-keys) in your config.
| Flag | Example | Purpose | | Flag | Example | Purpose |
| --- | --- | --- | | --- | --- | --- |
| `--tmdb` | `--tmdb 27205` | Use this TMDB ID instead of automatic lookup. | | `--tmdb` | `--tmdb 27205` | Use this TMDB ID instead of an automatic search. Needs `tmdb_api_key`. |
| `--imdb` | `--imdb tt1375666` | Use this IMDb ID. | | `--imdb` | `--imdb tt1375666` | Use this IMDb ID instead of an automatic search. Needs no key. |
| `--tvdb` | `--tvdb 73871` | Use this TVDB ID instead of looking the series up. | | `--tvdb` | `--tvdb 73871` | Use this TVDB ID instead of looking the series up. Needs `tvdb_api_key`. |
| `--animeapi` | `--animeapi mal:12345` | Resolve via AnimeAPI (`mal:`/`anilist:` prefix; defaults to MAL). | | `--animeapi` | `--animeapi mal:12345` | Resolve via AnimeAPI (`mal:`/`anilist:` prefix; defaults to MAL). |
| `--enrich` | - | Override the show title and year from an external source, and fill in the original language. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. | | `--enrich` | - | Overwrite the show title, year and original language with the external source's. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. |
| `--tvdb-order` | `--tvdb-order dvd` | Renumber episodes to a TVDB season order. Needs `tvdb_api_key`. | | `--tvdb-order` | `--tvdb-order dvd` | Renumber episodes to a TVDB season order. Needs `tvdb_api_key`. |
```shell title="Force the right IMDb match and enrich the title" ```shell title="Force the right IMDb match, and keep the service's own naming"
unshackle dl --imdb tt1375666 EXAMPLE 81234567
```
```shell title="Force the match and take IMDb's title, year and language too"
unshackle dl --imdb tt1375666 --enrich EXAMPLE 81234567 unshackle dl --imdb tt1375666 --enrich EXAMPLE 81234567
``` ```
### What each metadata provider supplies ### What each metadata provider supplies
Providers differ in what they answer with, so `--enrich` reads different fields depending on Providers differ in what they answer with, so the fields `--enrich` can replace depend on
which ID you gave it. A provider only runs when its key is configured: which ID you gave. Every provider except `imdb` only runs when its key is configured:
| Provider | Config key | Title and year | Original language | External IDs it returns | | Provider | Config key | Title and year | Original language | External IDs it returns |
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
| TMDB | `tmdb_api_key` | yes | yes, alpha-2 such as `ko` | IMDb, TMDB, TVDB | | TMDB | `tmdb_api_key` | yes | yes, alpha-2 such as `ko` | IMDb, TMDB, TVDB |
| TVDB | `tvdb_api_key` | yes | yes, alpha-3 such as `kor` | TMDB, TVDB, sometimes IMDb | | TVDB | `tvdb_api_key` | yes | yes, alpha-3 such as `kor` | TMDB, TVDB, sometimes IMDb |
| OMDb | `omdb_api_key` | yes | yes, as an English name such as `Korean` | IMDb | | OMDb | `omdb_api_key` | yes | yes, as an English name such as `Korean` | IMDb |
| IMDxAPI | `imdb_api_enabled` | yes | only when looked up by ID | IMDb | | IMDb | *(no key needed)* | yes | yes, alpha-2 such as `ko` | IMDb |
| SIMKL | `simkl_client_id` | yes | no, it publishes a country and no language | IMDb, TMDB, TVDB | | SIMKL | `simkl_client_id` | yes | no, it publishes a country and no language | IMDb, TMDB, TVDB |
Whatever the tag looks like, unshackle normalises it before use, so `ko`, `kor` and `Korean` Whatever the tag looks like, unshackle normalises it before use, so `ko`, `kor` and `Korean`
all end up as the same language. all end up as the same language.
`--enrich` consults these in a fixed order: Each ID goes to the providers that read that kind of ID, in
[`metadata_providers`](../reference/configuration/misc.md#external-api-keys) order:
- `--tmdb` reads TMDB. - `--tmdb` reads TMDB.
- `--imdb` tries IMDxAPI, then falls back to OMDb. Since `imdb_api_enabled` is `false` by - `--imdb` reads IMDb, then falls back to OMDb. IMDb needs no key, so this path works out of
default, an OMDb key is what makes this path work for most people. the box, and an OMDb key only matters when IMDb has no answer.
- `--tvdb` reads TVDB. It needs `tvdb_api_key`. - `--tvdb` reads TVDB. It needs `tvdb_api_key`.
- `--animeapi` supplies a title, plus whichever IDs AnimeAPI knows for the show. If that - `--animeapi` supplies a title, plus whichever IDs AnimeAPI knows for the show. If that
includes a TMDB, IMDb or TVDB ID, the year and language come from that provider. If it includes a TMDB, IMDb or TVDB ID, the year and language come from that provider. If it
does not, pair it with `--tmdb`, `--imdb` or `--tvdb`. does not, pair it with `--tmdb`, `--imdb` or `--tvdb`.
Language is the one field `--enrich` will not overwrite. A service that already reported an Give **one** of `--tmdb`, `--imdb` and `--tvdb`. They cannot be combined, because one ID does
original language keeps it, because the platform knows its own catalogue better than a the job on its own: unshackle resolves the other two from it and writes all three to the tags.
third-party match does. Everything else follows the usual override behaviour. Passing two is an error rather than a silent choice between them. `--animeapi` is the exception
and still pairs with one of the three, which is what to do when AnimeAPI knows no IDs for the
show.
The ID you give must have a provider that can actually resolve it, so unshackle checks that
before it downloads anything. `--tmdb` without `tmdb_api_key`, or an ID whose providers your
`metadata_providers` list leaves out, fails immediately with a message naming what to set.
`--imdb` needs no key, since the `imdb` provider is keyless and in the default order.
With `--enrich`, the title, year and original language are all replaced with the external
source's values, whether or not the service already filled them in. A field the source does
not answer with is left alone, and unshackle logs which fields those were, so a provider with
a thin record cannot blank out what the service told you.
!!! warning "`--enrich` replaces the original language, which affects more than the filename"
Track selection reads the original language, so replacing it changes which audio is
treated as the original. That is the point when a service mislabels it, but it means a
wrong ID can pick the wrong audio track and not only write a wrong name. Without
`--enrich` the service's own value is kept and track selection is untouched.
### Episode ordering ### Episode ordering
+1 -1
View File
@@ -36,7 +36,7 @@ Where the file lives, how unshackle discovers it, and how to edit it:
| [DRM & CDM](drm.md) | `cdm`, `remote_cdm`, `decryption` | | [DRM & CDM](drm.md) | `cdm`, `remote_cdm`, `decryption` |
| [Network & proxy](network.md) | `network`, `headers`, `proxy_providers` | | [Network & proxy](network.md) | `network`, `headers`, `proxy_providers` |
| [Key vaults](vaults.md) | `key_vaults`, `vault_timeout` | | [Key vaults](vaults.md) | `key_vaults`, `vault_timeout` |
| [External API keys](misc.md#external-api-keys) | `imdb_api_enabled`, `omdb_api_key`, `tmdb_api_key`, `tvdb_api_key`, `tvdb_pin`, `tvdb_order`, `simkl_client_id`, `metadata_providers`, `decrypt_labs_api_key`, `ipinfo_api_key` | | [External API keys](misc.md#external-api-keys) | `omdb_api_key`, `tmdb_api_key`, `tvdb_api_key`, `tvdb_pin`, `tvdb_order`, `simkl_client_id`, `metadata_providers`, `disable_metadata`, `decrypt_labs_api_key`, `ipinfo_api_key` |
| [Caching & updates](misc.md#caching-updates) | `title_cache_enabled`, `title_cache_time`, `title_cache_max_retention`, `update_checks`, `update_check_interval` | | [Caching & updates](misc.md#caching-updates) | `title_cache_enabled`, `title_cache_time`, `title_cache_max_retention`, `update_checks`, `update_check_interval` |
| [Logging, privacy & debug](misc.md#logging-privacy-debug) | `redact_paths`, `debug`, `debug_keys`, `debug_requests`, `set_terminal_bg` | | [Logging, privacy & debug](misc.md#logging-privacy-debug) | `redact_paths`, `debug`, `debug_keys`, `debug_requests`, `set_terminal_bg` |
| [Deprecated & removed](misc.md#deprecated-removed-keys) | `curl_impersonate`, `downloader`, `scene_naming` | | [Deprecated & removed](misc.md#deprecated-removed-keys) | `curl_impersonate`, `downloader`, `scene_naming` |
+57 -7
View File
@@ -10,14 +10,14 @@ each metadata provider can actually answer with, see
| Key | Type | Default | Description | | Key | Type | Default | Description |
|-----|------|---------|-------------| |-----|------|---------|-------------|
| `imdb_api_enabled` | bool | `false` | Use the free IMDxAPI (api.tiffara.com) metadata provider, which needs no API key. Off by default since the site has been unreliable. | | `omdb_api_key` | str | `""` | [OMDb](https://www.omdbapi.com/) API key for IMDb metadata lookups; a fallback for when the keyless `imdb` provider has no answer. Free keys are available on the OMDb site. |
| `omdb_api_key` | str | `""` | [OMDb](https://www.omdbapi.com/) API key for IMDb metadata lookups; a more reliable alternative to IMDxAPI. Free keys are available on the OMDb site. |
| `tmdb_api_key` | str | `""` | TMDB API key for metadata enrichment and external-ID tags. | | `tmdb_api_key` | str | `""` | TMDB API key for metadata enrichment and external-ID tags. |
| `simkl_client_id` | str | `""` | SIMKL client ID for metadata lookups; an alternative/fallback source to TMDB. | | `simkl_client_id` | str | `""` | SIMKL client ID for metadata lookups; an alternative/fallback source to TMDB. |
| `tvdb_api_key` | str | `""` | [TheTVDB v4](https://thetvdb.com/api-information) API key; a fallback source to TMDB, strongest on TV series. Also the source `--enrich --tvdb` reads. Free keys are available on the TVDB site. | | `tvdb_api_key` | str | `""` | [TheTVDB v4](https://thetvdb.com/api-information) API key; a fallback source to TMDB, strongest on TV series. Also the source `--enrich --tvdb` reads. Free keys are available on the TVDB site. |
| `tvdb_pin` | str | `""` | Subscriber PIN, only needed for a user-supported TVDB key. Leave empty for a normal project key. | | `tvdb_pin` | str | `""` | Subscriber PIN, only needed for a user-supported TVDB key. Leave empty for a normal project key. |
| `tvdb_order` | str | `""` | Default for `--tvdb-order`: renumber episodes to a TVDB season order (`official`, `dvd`, `absolute`, `alternate`, `regional`). Empty keeps the numbering the service gives. | | `tvdb_order` | str | `""` | Default for `--tvdb-order`: renumber episodes to a TVDB season order (`official`, `dvd`, `absolute`, `alternate`, `regional`). Empty keeps the numbering the service gives. |
| `metadata_providers` | list | *(see below)* | Metadata providers to use, in the order they are tried. | | `metadata_providers` | list or map | *(see below)* | Metadata providers to use, in the order they are tried. |
| `disable_metadata` | bool | `false` | Set to `true` to stop all automatic metadata lookups. An ID you give yourself is still looked up. |
| `decrypt_labs_api_key` | str | `""` | Global Decrypt Labs API key (used by remote CDM / vault). | | `decrypt_labs_api_key` | str | `""` | Global Decrypt Labs API key (used by remote CDM / vault). |
| `ipinfo_api_key` | str | `""` | ipinfo.io API key for IP/region lookups. | | `ipinfo_api_key` | str | `""` | ipinfo.io API key for IP/region lookups. |
@@ -33,12 +33,61 @@ each metadata provider can actually answer with, see
ignored with a warning. Leave it unset for the default order: ignored with a warning. Leave it unset for the default order:
```yaml ```yaml
metadata_providers: [imdbapi, omdb, simkl, tmdb, tvdb] metadata_providers: [imdb, omdb, simkl, tmdb, tvdb]
``` ```
Whatever the order, a provider is skipped when its API key is missing (or, for Whatever the order, a provider is skipped when its API key is missing. `imdb` needs no
`imdbapi`, when `imdb_api_enabled` is `false`). Available names are key, so it is always available. Available names are `imdb`, `omdb`, `simkl`, `tmdb`, and
`imdbapi`, `omdb`, `simkl`, `tmdb`, and `tvdb`. `tvdb`. The old name `imdbapi` still works and is read as `imdb`.
The order applies to a title search and to a direct lookup of an ID you give with
`--tmdb`, `--imdb`, or `--tvdb`. A supplied ID always goes to the providers that read
that kind of ID, and it never falls back to a title search.
!!! note "What `disable_metadata` turns off"
With `disable_metadata: true`, unshackle never contacts a metadata provider on its own.
The title search is dead, and that includes the keyless `imdb` provider. Titles, years and
languages then come only from the service.
```yaml
disable_metadata: true
```
An ID you give with `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi` is your permission to use
that ID, so it still works: the ID is looked up directly, the other IDs are cross
referenced from it, and `--enrich` still reads that source. Only the lookups you did not
ask for are stopped.
!!! note "The `imdb` provider needs no key"
It replaces the earlier `imdbapi` provider, which read a third-party mirror
(api.tiffara.com) and needed the `imdb_api_enabled` switch. That option is gone: delete it
from your config, where it is now ignored.
!!! tip "A different order per title kind"
Give a map instead of one list when the best source differs by kind. TVDB is strongest on
series, TMDB on films. There are two kinds, `tv` and `movie`, and this example sets both
of them and names every provider, so nothing is left to the default:
```yaml
metadata_providers:
tv: [tvdb, tmdb, simkl, imdb, omdb]
movie: [tmdb, imdb, omdb, simkl, tvdb]
```
Each list is independent: it both orders and filters that kind on its own, so a provider
you leave out of `tv` is still used for `movie`. Here TVDB is tried first for a series and
last for a film.
You do not need to give both keys, or a full list in each. A kind you leave out uses the
default order:
```yaml
metadata_providers:
tv: [tvdb, tmdb] # movies keep the default order
```
`tv` covers every episode and `movie` covers every film. unshackle only looks up
metadata for those two kinds, so any other key, such as `anime` or `music`, is ignored.
!!! note "`ipinfo_api_key` never touches your service sessions" !!! note "`ipinfo_api_key` never touches your service sessions"
The token is only ever sent to `api.ipinfo.io` as a per-request `Authorization` header; it The token is only ever sent to `api.ipinfo.io` as a per-request `Authorization` header; it
@@ -109,3 +158,4 @@ calls.
| `curl_impersonate` | **Deprecated** → use [`network`](network.md) | Emits a `DeprecationWarning`; still honoured only if `network` is absent. | | `curl_impersonate` | **Deprecated** → use [`network`](network.md) | Emits a `DeprecationWarning`; still honoured only if `network` is absent. |
| `downloader` | **Deprecated** | Any value other than `"requests"` emits a `DeprecationWarning`; the value is otherwise ignored (the unified requests downloader is always used). | | `downloader` | **Deprecated** | Any value other than `"requests"` emits a `DeprecationWarning`; the value is otherwise ignored (the unified requests downloader is always used). |
| `scene_naming` | **Removed** | If set to any non-null value, unshackle exits with an error directing you to configure [`output_template`](output.md) instead. | | `scene_naming` | **Removed** | If set to any non-null value, unshackle exits with an error directing you to configure [`output_template`](output.md) instead. |
| `imdb_api_enabled` | **Removed** | Ignored. It gated the old `imdbapi` provider, which the keyless [`imdb`](#external-api-keys) provider replaces. Safe to delete from your config. |
+67
View File
@@ -0,0 +1,67 @@
"""The `disable_metadata` config switch stops automatic lookups but keeps supplied IDs working."""
from __future__ import annotations
from typing import Optional, Union
import pytest
from unshackle.core import providers
from unshackle.core.config import config
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult
class FakeProvider(MetadataProvider):
"""Stands in for a real provider; fails the test if it is searched."""
def __init__(self, name: str, result: Optional[MetadataResult] = None) -> None:
self.NAME = name
self._result = result
self.asked: list[Union[int, str]] = []
super().__init__()
def is_available(self) -> bool:
return True
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
raise AssertionError("no provider may be searched while disable_metadata is set")
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
self.asked.append(provider_id)
return self._result
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
return ExternalIds()
@pytest.fixture(autouse=True)
def _disabled(monkeypatch: pytest.MonkeyPatch) -> None:
"""Turn metadata off and keep every test off the network."""
monkeypatch.setattr(config, "disable_metadata", True)
monkeypatch.setattr(config, "metadata_providers", [])
monkeypatch.setattr(providers, "fetch_external_ids", lambda *a, **kw: ExternalIds())
monkeypatch.setattr(providers, "enrich_ids", lambda result: None)
def test_search_metadata_returns_none(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(providers, "get_provider", lambda name: FakeProvider(name))
assert providers.search_metadata("Futurama", 1999, "tv") is None
def test_supplied_id_still_resolves(monkeypatch: pytest.MonkeyPatch) -> None:
tmdb = FakeProvider("tmdb", MetadataResult(title="Inception", year=2010, kind="movie"))
monkeypatch.setattr(providers, "get_provider", lambda name: {"tmdb": tmdb}.get(name))
result = providers.resolve_by_ids(27205, kind="movie")
assert result is not None
assert result.title == "Inception"
assert result.external_ids.tmdb_id == 27205
assert tmdb.asked == [27205]
def test_no_ids_returns_none(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(providers, "get_provider", lambda name: FakeProvider(name))
assert providers.resolve_by_ids(title="Futurama", year=1999, kind="tv") is None
+21 -5
View File
@@ -1,11 +1,11 @@
"""--enrich original-language enrichment.""" """Original-language parsing for the metadata a supplied ID resolves."""
from __future__ import annotations from __future__ import annotations
import pytest import pytest
from unshackle.commands.dl import parse_language from unshackle.commands.dl import parse_language
from unshackle.core.providers.imdbapi import primary_language as imdb_language from unshackle.core.providers.imdb import primary_language as imdb_language
from unshackle.core.providers.omdb import primary_language as omdb_language from unshackle.core.providers.omdb import primary_language as omdb_language
from unshackle.core.providers.tvdb import primary_language as tvdb_language from unshackle.core.providers.tvdb import primary_language as tvdb_language
@@ -28,11 +28,27 @@ def test_parse_language(tag: str | None, expected: str | None) -> None:
def test_imdb_language_takes_the_first_spoken_language() -> None: def test_imdb_language_takes_the_first_spoken_language() -> None:
data = {"spokenLanguages": [{"code": "jpn", "name": "Japanese"}, {"code": "eng", "name": "English"}]} data = {"spokenLanguages": {"spokenLanguages": [{"id": "ja", "text": "Japanese"}, {"id": "en", "text": "English"}]}}
assert imdb_language(data) == "jpn" assert imdb_language(data) == "ja"
@pytest.mark.parametrize("data", [{}, {"spokenLanguages": []}, {"spokenLanguages": None}]) def test_imdb_language_is_alpha_2() -> None:
"""IMDb's GraphQL language ids are alpha-2."""
data = {"spokenLanguages": {"spokenLanguages": [{"id": "ko", "text": "Korean"}]}}
assert parse_language(imdb_language(data)) is not None
assert str(parse_language(imdb_language(data))) == "ko"
@pytest.mark.parametrize(
"data",
[
{},
{"spokenLanguages": None}, # search hits can come back with the whole block null
{"spokenLanguages": {}},
{"spokenLanguages": {"spokenLanguages": []}},
{"spokenLanguages": {"spokenLanguages": None}},
],
)
def test_imdb_language_missing(data: dict) -> None: def test_imdb_language_missing(data: dict) -> None:
assert imdb_language(data) is None assert imdb_language(data) is None
+183
View File
@@ -0,0 +1,183 @@
"""IMDb GraphQL provider: persisted-query flow, field mapping, and search constraints."""
from __future__ import annotations
import json
from typing import Any, Optional
import pytest
from unshackle.core.providers.imdb import IMDBProvider, escape_graphql
class _Response:
def __init__(self, payload: dict) -> None:
self._payload = payload
def json(self) -> dict:
return self._payload
def _node(
imdb_id: str = "tt3581920",
text: str = "The Last of Us",
year: Optional[int] = 2023,
title_type: str = "tvSeries",
languages: Any = ({"id": "en", "text": "English"}, {"id": "id", "text": "Indonesian"}),
) -> dict:
return {
"id": imdb_id,
"titleText": {"text": text},
"originalTitleText": {"text": text},
"releaseYear": {"year": year} if year else None,
"titleType": {"id": title_type},
"spokenLanguages": {"spokenLanguages": list(languages)} if languages is not None else None,
}
class _FakeSession:
"""Records GET/POST calls and replays queued payloads."""
def __init__(self, responses: list[dict]) -> None:
self._responses = responses
self.gets: list[dict] = []
self.posts: list[dict] = []
def get(self, url: str, params: dict, headers: dict, timeout: int) -> _Response:
self.gets.append(params)
return _Response(self._responses.pop(0))
def post(self, url: str, json: dict, headers: dict, timeout: int) -> _Response: # noqa: A002
self.posts.append(json)
return _Response(self._responses.pop(0))
def _provider(monkeypatch: pytest.MonkeyPatch, responses: list[dict]) -> tuple[IMDBProvider, _FakeSession]:
p = IMDBProvider()
session = _FakeSession(responses)
monkeypatch.setattr(type(p), "session", property(lambda _self: session))
return p, session
NOT_FOUND = {"errors": [{"message": "PersistedQueryNotFound", "extensions": {"code": "PERSISTED_QUERY_NOT_FOUND"}}]}
def test_is_available_without_any_key() -> None:
assert IMDBProvider().is_available() is True
def test_cache_hit_uses_one_get(monkeypatch: pytest.MonkeyPatch) -> None:
p, session = _provider(monkeypatch, [{"data": {"title": _node()}}])
result = p.get_by_id("tt3581920", "tv")
assert result is not None and result.title == "The Last of Us"
assert len(session.gets) == 1 and session.posts == []
def test_persisted_query_miss_falls_back_to_one_post(monkeypatch: pytest.MonkeyPatch) -> None:
p, session = _provider(monkeypatch, [NOT_FOUND, {"data": {"title": _node()}}])
result = p.get_by_id("tt3581920", "tv")
assert result is not None and result.external_ids.imdb_id == "tt3581920"
assert len(session.gets) == 1 and len(session.posts) == 1
# the POST registers the query by sending its text, keyed by the same hash the GET sent
assert "query UnshackleTitle" in session.posts[0]["query"]
sent_hash = json.loads(session.gets[0]["extensions"])["persistedQuery"]["sha256Hash"]
assert session.posts[0]["extensions"]["persistedQuery"]["sha256Hash"] == sent_hash
def test_get_params_carry_no_spaces(monkeypatch: pytest.MonkeyPatch) -> None:
"""IMDb's GraphQL gateway decodes "+" literally, so spaced JSON would come back as a 400."""
p, session = _provider(monkeypatch, [{"data": {"title": _node()}}])
p.get_by_id("tt3581920", "tv")
assert " " not in session.gets[0]["extensions"]
assert " " not in session.gets[0]["variables"]
def test_get_by_id_maps_every_field(monkeypatch: pytest.MonkeyPatch) -> None:
p, _ = _provider(monkeypatch, [{"data": {"title": _node()}}])
result = p.get_by_id("tt3581920", "tv")
assert result is not None
assert (result.title, result.year, result.kind) == ("The Last of Us", 2023, "tv")
assert result.original_language == "en"
assert result.source == "imdb"
assert result.external_ids.imdb_id == "tt3581920"
@pytest.mark.parametrize("title_type", ["movie", "tvSeries", "videoGame"])
def test_caller_kind_wins_over_the_response_title_type(monkeypatch: pytest.MonkeyPatch, title_type: str) -> None:
"""kind stays the caller's, so it cannot desync from the tmdb_kind resolve_by_ids seeds."""
p, _ = _provider(monkeypatch, [{"data": {"title": _node(title_type=title_type)}}])
result = p.get_by_id("tt1375666", "tv")
assert result is not None and result.kind == "tv"
@pytest.mark.parametrize("payload", [{"data": {"title": None}}, {"errors": [{"message": "boom"}]}, {"data": {}}])
def test_get_by_id_returns_none_on_no_answer(monkeypatch: pytest.MonkeyPatch, payload: dict) -> None:
p, _ = _provider(monkeypatch, [payload])
assert p.get_by_id("tt3581920", "tv") is None
def _edges(*nodes: dict) -> dict:
return {"data": {"advancedTitleSearch": {"edges": [{"node": {"title": n}} for n in nodes]}}}
def test_search_uses_its_own_operation_and_one_get_on_a_cache_hit(monkeypatch: pytest.MonkeyPatch) -> None:
p, session = _provider(monkeypatch, [_edges(_node())])
result = p.search("The Last of Us", 2023, "tv")
assert result is not None and result.external_ids.imdb_id == "tt3581920"
assert session.gets[0]["operationName"] == "UnshackleTitleSearch"
assert session.posts == []
def test_search_query_text_carries_the_constraints(monkeypatch: pytest.MonkeyPatch) -> None:
p, session = _provider(monkeypatch, [NOT_FOUND, _edges(_node())])
p.search("The Last of Us", 2023, "tv")
query = session.posts[0]["query"]
assert 'searchTerm: "The Last of Us"' in query
assert 'start: "2023-01-01", end: "2023-12-31"' in query
assert '["tvSeries", "tvMiniSeries"]' in query or '["tvSeries","tvMiniSeries"]' in query
def test_search_omits_the_year_range_when_unknown(monkeypatch: pytest.MonkeyPatch) -> None:
p, session = _provider(monkeypatch, [NOT_FOUND, _edges(_node(title_type="movie"))])
p.search("Inception", None, "movie")
query = session.posts[0]["query"]
assert "releaseDateConstraint" not in query
assert '["movie", "tvMovie"]' in query or '["movie","tvMovie"]' in query
def test_search_picks_the_closest_title_not_the_first(monkeypatch: pytest.MonkeyPatch) -> None:
noise = _node(imdb_id="tt27478132", text="Making of: The Last of Us")
exact = _node()
p, _ = _provider(monkeypatch, [_edges(noise, exact)])
result = p.search("The Last of Us", 2023, "tv")
assert result is not None and result.external_ids.imdb_id == "tt3581920"
def test_search_rejects_a_title_that_is_not_a_close_match(monkeypatch: pytest.MonkeyPatch) -> None:
p, _ = _provider(monkeypatch, [_edges(_node(imdb_id="tt999", text="Something Else Entirely"))])
assert p.search("The Last of Us", 2023, "tv") is None
def test_search_survives_a_null_language_block(monkeypatch: pytest.MonkeyPatch) -> None:
p, _ = _provider(monkeypatch, [_edges(_node(languages=None))])
result = p.search("The Last of Us", 2023, "tv")
assert result is not None and result.original_language is None
def test_search_returns_none_when_nothing_comes_back(monkeypatch: pytest.MonkeyPatch) -> None:
p, _ = _provider(monkeypatch, [_edges()])
assert p.search("The Last of Us", 2023, "tv") is None
def test_escape_graphql_closes_the_string_injection() -> None:
assert escape_graphql('a"b\\c') == 'a\\"b\\\\c'
def test_search_term_with_a_quote_stays_inside_the_string(monkeypatch: pytest.MonkeyPatch) -> None:
p, session = _provider(monkeypatch, [NOT_FOUND, _edges(_node(text='The "Best" Show'))])
p.search('The "Best" Show', None, "tv")
assert 'searchTerm: "The \\"Best\\" Show"' in session.posts[0]["query"]
def test_get_external_ids_echoes_the_id() -> None:
assert IMDBProvider().get_external_ids("tt1375666", "movie").imdb_id == "tt1375666"
+114
View File
@@ -0,0 +1,114 @@
"""Input validation for the --tmdb / --imdb / --tvdb flags."""
from __future__ import annotations
import click
import pytest
from unshackle.commands.dl import validate_metadata_ids
from unshackle.core.api.handlers import validate_download_parameters
from unshackle.core.config import config
@pytest.fixture(autouse=True)
def _default_order(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(config, "metadata_providers", [])
def _with_keys(monkeypatch: pytest.MonkeyPatch, **keys: str) -> None:
for name in ("tmdb_api_key", "tvdb_api_key", "omdb_api_key", "simkl_client_id"):
monkeypatch.setattr(config, name, keys.get(name, ""))
# ---------- one ID at a time ----------
@pytest.mark.parametrize(
("tmdb", "imdb", "tvdb"),
[(27205, "tt1375666", None), (27205, None, 73871), (None, "tt1375666", 73871), (27205, "tt1375666", 73871)],
)
def test_two_ids_are_refused(tmdb: int | None, imdb: str | None, tvdb: int | None) -> None:
with pytest.raises(click.UsageError) as exc:
validate_metadata_ids(tmdb, imdb, tvdb)
assert "cannot be used together" in str(exc.value)
def test_no_ids_is_fine() -> None:
assert validate_metadata_ids(None, None, None) is None
def test_imdb_alone_passes_on_the_keyless_provider(monkeypatch: pytest.MonkeyPatch) -> None:
_with_keys(monkeypatch)
assert validate_metadata_ids(None, "tt1375666", None) is None
# ---------- the ID must have a provider that can resolve it ----------
def test_tmdb_without_its_key_names_the_key(monkeypatch: pytest.MonkeyPatch) -> None:
_with_keys(monkeypatch)
with pytest.raises(click.UsageError) as exc:
validate_metadata_ids(27205, None, None)
assert "--tmdb" in str(exc.value) and "tmdb_api_key" in str(exc.value)
def test_tvdb_without_its_key_names_the_key(monkeypatch: pytest.MonkeyPatch) -> None:
_with_keys(monkeypatch)
with pytest.raises(click.UsageError) as exc:
validate_metadata_ids(None, None, 73871)
assert "--tvdb" in str(exc.value) and "tvdb_api_key" in str(exc.value)
def test_tmdb_with_its_key_passes(monkeypatch: pytest.MonkeyPatch) -> None:
_with_keys(monkeypatch, tmdb_api_key="k")
assert validate_metadata_ids(27205, None, None) is None
def test_a_provider_filtered_out_of_the_order_points_at_the_config(monkeypatch: pytest.MonkeyPatch) -> None:
_with_keys(monkeypatch, tmdb_api_key="k")
monkeypatch.setattr(config, "metadata_providers", ["imdb", "omdb"])
with pytest.raises(click.UsageError) as exc:
validate_metadata_ids(27205, None, None)
assert "metadata_providers" in str(exc.value)
def test_imdb_is_unresolvable_when_the_order_drops_both_of_its_providers(monkeypatch: pytest.MonkeyPatch) -> None:
_with_keys(monkeypatch)
monkeypatch.setattr(config, "metadata_providers", ["tmdb", "tvdb"])
with pytest.raises(click.UsageError) as exc:
validate_metadata_ids(None, "tt1375666", None)
assert "metadata_providers" in str(exc.value)
assert "imdb" in str(exc.value) and "omdb" in str(exc.value)
def test_one_kind_keeping_the_provider_is_enough(monkeypatch: pytest.MonkeyPatch) -> None:
"""A per-kind order can drop tvdb for movies and keep it for tv, so this must not error."""
_with_keys(monkeypatch, tvdb_api_key="k")
monkeypatch.setattr(config, "metadata_providers", {"tv": ["tvdb"], "movie": ["tmdb"]})
assert validate_metadata_ids(None, None, 73871) is None
# ---------- the REST path rejects the same combinations ----------
@pytest.mark.parametrize(
"payload",
[
{"tmdb_id": 27205, "imdb_id": "tt1375666"},
{"tmdb_id": 27205, "tvdb_id": 73871},
{"imdb_id": "tt1375666", "tvdb_id": 73871},
],
)
def test_api_refuses_two_ids(payload: dict) -> None:
err = validate_download_parameters(payload)
assert err and "multiple external IDs" in err
@pytest.mark.parametrize("payload", [{"tmdb_id": 27205}, {"imdb_id": "tt1375666"}, {"tvdb_id": 73871}, {}])
def test_api_accepts_one_id(payload: dict) -> None:
assert validate_download_parameters(payload) is None
def test_api_still_reports_a_malformed_id_first() -> None:
err = validate_download_parameters({"tmdb_id": "12345", "imdb_id": "tt1"})
assert err and "positive integer" in err
+131
View File
@@ -0,0 +1,131 @@
"""Precedence of user-supplied external IDs in providers.resolve_by_ids."""
from __future__ import annotations
from typing import Optional, Union
import pytest
from unshackle.core import providers
from unshackle.core.config import config
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult
class FakeProvider(MetadataProvider):
"""Stands in for a real provider; records the ID it was asked for."""
def __init__(self, name: str, result: Optional[MetadataResult] = None, raises: bool = False) -> None:
self.NAME = name
self._result = result
self._raises = raises
self.asked: list[Union[int, str]] = []
super().__init__()
def is_available(self) -> bool:
return True
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
raise AssertionError("resolve_by_ids must not search when an ID was supplied")
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
self.asked.append(provider_id)
if self._raises:
raise ValueError("provider is down")
return self._result
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
return ExternalIds()
@pytest.fixture(autouse=True)
def _isolate(monkeypatch: pytest.MonkeyPatch) -> None:
"""Keep every test off the network and on the default provider order."""
monkeypatch.setattr(config, "metadata_providers", [])
monkeypatch.setattr(providers, "fetch_external_ids", lambda *a, **kw: ExternalIds())
monkeypatch.setattr(providers, "enrich_ids", lambda result: None)
monkeypatch.setattr(
providers,
"search_metadata",
lambda *a, **kw: pytest.fail("search_metadata must not run when an ID was supplied"),
)
def _patch_providers(monkeypatch: pytest.MonkeyPatch, available: dict[str, FakeProvider]) -> None:
monkeypatch.setattr(providers, "get_provider", lambda name: available.get(name))
def test_supplied_ids_survive_a_providers_own_answer(monkeypatch: pytest.MonkeyPatch) -> None:
# tmdb answers with a different imdb id; the one the user gave must win
tmdb = FakeProvider(
"tmdb",
MetadataResult(title="Inception", year=2010, kind="movie", external_ids=ExternalIds(imdb_id="tt9999999")),
)
_patch_providers(monkeypatch, {"tmdb": tmdb})
result = providers.resolve_by_ids(27205, "tt1375666", kind="movie")
assert result is not None
assert result.external_ids.tmdb_id == 27205
assert result.external_ids.tmdb_kind == "movie"
assert result.external_ids.imdb_id == "tt1375666"
def test_imdb_falls_through_to_omdb(monkeypatch: pytest.MonkeyPatch) -> None:
imdb = FakeProvider("imdb", raises=True)
omdb = FakeProvider("omdb", MetadataResult(title="Inception", year=2010, kind="movie"))
_patch_providers(monkeypatch, {"imdb": imdb, "omdb": omdb})
result = providers.resolve_by_ids(imdb_id="tt1375666", kind="movie")
assert result is not None
assert result.title == "Inception"
assert imdb.asked == ["tt1375666"]
assert omdb.asked == ["tt1375666"]
def test_all_providers_down_synthesizes_a_result(monkeypatch: pytest.MonkeyPatch) -> None:
_patch_providers(monkeypatch, {})
result = providers.resolve_by_ids(tvdb_id=73871, title="Futurama", year=1999, kind="tv")
assert result is not None
assert result.title == "Futurama"
assert result.year == 1999
assert result.external_ids.tvdb_id == 73871
def test_no_ids_searches(monkeypatch: pytest.MonkeyPatch) -> None:
calls: list[tuple] = []
monkeypatch.setattr(providers, "search_metadata", lambda *a, **kw: calls.append(a) or None)
_patch_providers(monkeypatch, {})
providers.resolve_by_ids(title="Futurama", year=1999, kind="tv")
assert calls and calls[0][:3] == ("Futurama", 1999, "tv")
def test_no_ids_and_no_title_returns_none(monkeypatch: pytest.MonkeyPatch) -> None:
_patch_providers(monkeypatch, {})
assert providers.resolve_by_ids(kind="tv") is None
def test_per_kind_order_puts_tvdb_first_for_tv(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(config, "metadata_providers", {"tv": ["tvdb", "tmdb", "imdb", "omdb"]})
tvdb = FakeProvider("tvdb", MetadataResult(title="Futurama", year=1999, kind="tv"))
tmdb = FakeProvider("tmdb", MetadataResult(title="Wrong Show", year=2020, kind="tv"))
_patch_providers(monkeypatch, {"tvdb": tvdb, "tmdb": tmdb})
result = providers.resolve_by_ids(615, tvdb_id=73871, kind="tv")
assert result is not None
assert result.title == "Futurama"
assert tmdb.asked == []
# movies keep the default order, so tmdb is consulted before tvdb there
assert [cls.NAME for cls in providers.provider_order("movie")] == list(providers.DEFAULT_ORDER)
def test_flat_list_config_applies_to_both_kinds(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(config, "metadata_providers", ["tvdb", "tmdb"])
assert [cls.NAME for cls in providers.provider_order("tv")] == ["tvdb", "tmdb"]
assert [cls.NAME for cls in providers.provider_order("movie")] == ["tvdb", "tmdb"]
assert [cls.NAME for cls in providers.provider_order()] == ["tvdb", "tmdb"]
+108 -56
View File
@@ -134,6 +134,43 @@ def normalize_dl_config(dl_config: dict[str, Any]) -> dict[str, Any]:
return {DL_OPTION_ALIASES.get(key, key): value for key, value in dl_config.items()} return {DL_OPTION_ALIASES.get(key, key): value for key, value in dl_config.items()}
ID_PROVIDER_KEYS = {"tmdb": "tmdb_api_key", "tvdb": "tvdb_api_key", "omdb": "omdb_api_key"}
def validate_metadata_ids(tmdb_id: Optional[int], imdb_id: Optional[str], tvdb_id: Optional[int]) -> None:
"""Reject ID flags that conflict, or that no configured provider could ever resolve.
Only ever called with the IDs the user gave, never with the ones --animeapi back-fills.
"""
supplied = [
(flag, value) for flag, value in (("--tmdb", tmdb_id), ("--imdb", imdb_id), ("--tvdb", tvdb_id)) if value
]
if len(supplied) > 1:
raise click.UsageError(
f"{', '.join(flag for flag, _ in supplied)} cannot be used together. "
"Give one ID and unshackle resolves the others from it."
)
if not supplied:
return
flag, _ = supplied[0]
id_kind = flag.lstrip("-")
ordered = {cls.NAME: cls for kind in ("tv", "movie") for cls in providers.provider_order(kind)}
consumers = [cls for cls in ordered.values() if cls.ID_KIND == id_kind]
if any(cls().is_available() for cls in consumers):
return
if consumers:
keys = sorted({ID_PROVIDER_KEYS[cls.NAME] for cls in consumers if cls.NAME in ID_PROVIDER_KEYS})
raise click.UsageError(
f"{flag} needs the {consumers[0].NAME} provider. Set {' or '.join(keys)} in your config."
)
names = sorted({cls.NAME for cls in providers.REGISTRY.values() if cls.ID_KIND == id_kind})
raise click.UsageError(
f"{flag} needs the {' or '.join(names)} provider, which your metadata_providers config leaves out."
)
def group_videos_by_variant(videos: list[Video], *, merge: bool) -> list[list[Video]]: def group_videos_by_variant(videos: list[Video], *, merge: bool) -> list[list[Video]]:
"""Group video tracks for muxing. """Group video tracks for muxing.
@@ -617,14 +654,16 @@ class dl:
"tmdb_id", "tmdb_id",
type=int, type=int,
default=None, default=None,
help="Use this TMDB ID for tagging instead of automatic lookup.", help="Use this TMDB ID for tagging instead of an automatic search. Add --enrich to also take "
"its title, year and original language.",
) )
@click.option( @click.option(
"--tvdb", "--tvdb",
"tvdb_id", "tvdb_id",
type=int, type=int,
default=None, default=None,
help="Use this TVDB ID for --tvdb-order and --enrich instead of looking the series up automatically.", help="Use this TVDB ID for --tvdb-order and tagging instead of looking the series up "
"automatically. Add --enrich to also take its title, year and original language.",
) )
@click.option( @click.option(
"--tvdb-order", "--tvdb-order",
@@ -638,15 +677,16 @@ class dl:
"animeapi_id", "animeapi_id",
type=str, type=str,
default=None, default=None,
help="Anime database ID via AnimeAPI (e.g. mal:12345, anilist:98765). Defaults to MAL if no prefix.", help="Anime database ID via AnimeAPI (e.g. mal:12345, anilist:98765). Defaults to MAL if no prefix. "
"Back-fills the TMDB, IMDB and TVDB IDs it knows for the show.",
) )
@click.option( @click.option(
"--enrich", "--enrich",
is_flag=True, is_flag=True,
default=False, default=False,
help=( help=(
"Override show title and year from external source, and fill in the original language " "Overwrite the show title, year and original language with the external source's. "
"if the service did not provide one. Requires --tmdb, --imdb, --tvdb, or --animeapi." "Requires --tmdb, --imdb, --tvdb, or --animeapi."
), ),
) )
@click.option( @click.option(
@@ -654,7 +694,8 @@ class dl:
"imdb_id", "imdb_id",
type=str, type=str,
default=None, default=None,
help="Use this IMDB ID (e.g. tt1375666) for tagging instead of automatic lookup.", help="Use this IMDB ID (e.g. tt1375666) for tagging instead of an automatic search. Add --enrich "
"to also take its title, year and original language.",
) )
@click.option( @click.option(
"--sub-format", "--sub-format",
@@ -857,6 +898,7 @@ class dl:
self.profile = profile self.profile = profile
self.proxy_requested = bool(proxy) self.proxy_requested = bool(proxy)
validate_metadata_ids(tmdb_id, imdb_id, tvdb_id)
self.tmdb_id = tmdb_id self.tmdb_id = tmdb_id
self.imdb_id = imdb_id self.imdb_id = imdb_id
self.tvdb_id = tvdb_id self.tvdb_id = tvdb_id
@@ -1576,48 +1618,32 @@ class dl:
if self.animeapi_title: if self.animeapi_title:
enrich_title = self.animeapi_title enrich_title = self.animeapi_title
if self.tmdb_id: enrich_result = providers.resolve_by_ids(
self.tmdb_id,
self.imdb_id,
self.tvdb_id,
kind=kind,
title_cacher=title_cacher,
cache_title_id=cache_title_id,
cache_region=cache_region,
cache_account_hash=cache_account_hash,
)
if enrich_result:
if not enrich_title: if not enrich_title:
enrich_title = providers.get_title_by_id( enrich_title = enrich_result.title
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash enrich_year = enrich_result.year
) enrich_lang = parse_language(enrich_result.original_language)
enrich_year = providers.get_year_by_id(
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
)
enrich_lang = parse_language(
providers.get_language_by_id(
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
)
)
elif self.imdb_id:
for name in ("imdbapi", "omdb"):
provider = providers.get_provider(name)
imdb_result = provider.get_by_id(self.imdb_id, kind) if provider else None
if imdb_result:
if not enrich_title:
enrich_title = imdb_result.title
enrich_year = imdb_result.year
enrich_lang = parse_language(imdb_result.original_language)
break
elif self.tvdb_id:
provider = providers.get_provider("tvdb")
tvdb_result = provider.get_by_id(self.tvdb_id, kind) if provider else None
if tvdb_result:
if not enrich_title:
enrich_title = tvdb_result.title
enrich_year = tvdb_result.year
enrich_lang = parse_language(tvdb_result.original_language)
if enrich_lang: if enrich_lang:
self.log.info(f"Original language from metadata: {enrich_lang}") self.log.info(f"Original language from metadata: {enrich_lang}")
if not (enrich_title or enrich_year or enrich_lang): if not (enrich_title or enrich_year or enrich_lang):
self.log.warning("--enrich found no metadata; is the provider's API key configured?") self.log.warning("--enrich found no metadata. Is the provider's API key configured?")
elif missing := [ elif missing := [
name name
for name, value in (("title", enrich_title), ("year", enrich_year), ("language", enrich_lang)) for name, value in (("title", enrich_title), ("year", enrich_year), ("language", enrich_lang))
if not value if not value
]: ]:
self.log.warning(f"--enrich source did not provide {', '.join(missing)}; those fields are unchanged.") self.log.warning(f"--enrich source gave no {', '.join(missing)}. Those fields are unchanged.")
if enrich_title or enrich_year or enrich_lang: if enrich_title or enrich_year or enrich_lang:
for t in titles if isinstance(titles, (Series, Movies)) else [titles]: for t in titles if isinstance(titles, (Series, Movies)) else [titles]:
@@ -1626,9 +1652,9 @@ class dl:
t.title = enrich_title t.title = enrich_title
else: else:
t.name = enrich_title t.name = enrich_title
if enrich_year and not t.year: if enrich_year:
t.year = enrich_year t.year = enrich_year
if enrich_lang and not t.language: if enrich_lang:
t.language = enrich_lang t.language = enrich_lang
if self.tvdb_order and isinstance(titles, Series): if self.tvdb_order and isinstance(titles, Series):
@@ -2332,20 +2358,28 @@ class dl:
if isinstance(title, Episode) and not self.tmdb_searched: if isinstance(title, Episode) and not self.tmdb_searched:
kind = "tv" kind = "tv"
tmdb_title: Optional[str] = None tmdb_title: Optional[str] = None
if self.tmdb_id: has_ids = bool(self.tmdb_id or self.imdb_id or self.tvdb_id)
tmdb_title = providers.get_title_by_id( result = providers.resolve_by_ids(
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash self.tmdb_id,
) self.imdb_id,
else: self.tvdb_id,
result = providers.search_metadata( title=title.title,
title.title, title.year, kind, title_cacher, cache_title_id, cache_region, cache_account_hash year=title.year,
) kind=kind,
if result and result.title and providers.fuzzy_match(result.title, title.title): title_cacher=title_cacher,
cache_title_id=cache_title_id,
cache_region=cache_region,
cache_account_hash=cache_account_hash,
)
if result:
if has_ids:
tmdb_title = result.title
if not self.tmdb_id:
self.tmdb_id = result.external_ids.tmdb_id
elif result.title and providers.fuzzy_match(result.title, title.title):
self.tmdb_id = result.external_ids.tmdb_id self.tmdb_id = result.external_ids.tmdb_id
tmdb_title = result.title tmdb_title = result.title
self.search_source = result.source self.search_source = result.source
else:
self.tmdb_id = None
if list_ or list_titles: if list_ or list_titles:
if self.tmdb_id: if self.tmdb_id:
console.print( console.print(
@@ -2359,8 +2393,17 @@ class dl:
self.tmdb_searched = True self.tmdb_searched = True
if isinstance(title, Movie) and (list_ or list_titles) and not self.tmdb_id: if isinstance(title, Movie) and (list_ or list_titles) and not self.tmdb_id:
movie_result = providers.search_metadata( movie_result = providers.resolve_by_ids(
title.name, title.year, "movie", title_cacher, cache_title_id, cache_region, cache_account_hash None,
self.imdb_id,
self.tvdb_id,
title=title.name,
year=title.year,
kind="movie",
title_cacher=title_cacher,
cache_title_id=cache_title_id,
cache_region=cache_region,
cache_account_hash=cache_account_hash,
) )
if movie_result and movie_result.external_ids.tmdb_id: if movie_result and movie_result.external_ids.tmdb_id:
console.print( console.print(
@@ -2373,7 +2416,7 @@ class dl:
else: else:
console.print(Padding("Search -> [bright_black]No match found[/]", (0, 5))) console.print(Padding("Search -> [bright_black]No match found[/]", (0, 5)))
if self.tmdb_id and getattr(self, "search_source", None) not in ("simkl", "imdbapi"): if self.tmdb_id and getattr(self, "search_source", None) not in ("simkl", "imdb"):
kind = "tv" if isinstance(title, Episode) else "movie" kind = "tv" if isinstance(title, Episode) else "movie"
providers.fetch_external_ids( providers.fetch_external_ids(
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
@@ -3869,8 +3912,17 @@ class dl:
tvdb_id = self.tvdb_id tvdb_id = self.tvdb_id
if not tvdb_id: if not tvdb_id:
show = titles[0] show = titles[0]
result = providers.search_metadata( result = providers.resolve_by_ids(
show.title, show.year, "tv", title_cacher, cache_title_id, cache_region, cache_account_hash self.tmdb_id,
self.imdb_id,
None,
title=show.title,
year=show.year,
kind="tv",
title_cacher=title_cacher,
cache_title_id=cache_title_id,
cache_region=cache_region,
cache_account_hash=cache_account_hash,
) )
tvdb_id = result.external_ids.tvdb_id if result else None tvdb_id = result.external_ids.tvdb_id if result else None
if not tvdb_id: if not tvdb_id:
+17
View File
@@ -926,6 +926,23 @@ def validate_download_parameters(data: Dict[str, Any]) -> Optional[str]:
if not isinstance(data["downloads"], int) or data["downloads"] <= 0: if not isinstance(data["downloads"], int) or data["downloads"] <= 0:
return "downloads must be a positive integer" return "downloads must be a positive integer"
for name in ("tmdb_id", "tvdb_id"):
if data.get(name) is not None:
value = data[name]
if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
return f"{name} must be a positive integer"
if data.get("imdb_id") is not None:
if not isinstance(data["imdb_id"], str) or not re.fullmatch(r"tt\d+", data["imdb_id"]):
return "imdb_id must be an IMDB ID like 'tt1375666'"
supplied_ids = [name for name in ("tmdb_id", "imdb_id", "tvdb_id") if data.get(name)]
if len(supplied_ids) > 1:
return (
f"Cannot use multiple external IDs: {', '.join(supplied_ids)}. "
"Give one ID and unshackle resolves the others from it."
)
exclusive_flags = [] exclusive_flags = []
if data.get("video_only"): if data.get("video_only"):
exclusive_flags.append("video_only") exclusive_flags.append("video_only")
+4 -4
View File
@@ -758,13 +758,13 @@ async def download(request: web.Request) -> web.Response:
description: Set the group tag to be used (default - None) description: Set the group tag to be used (default - None)
tmdb_id: tmdb_id:
type: integer type: integer
description: Use this TMDB ID for tagging (default - None) description: Use this TMDB ID for tagging instead of a title search. Set enrich to also take its title, year and original language. Mutually exclusive with imdb_id and tvdb_id. Needs tmdb_api_key (default - None)
animeapi_id: animeapi_id:
type: string type: string
description: Anime database ID via AnimeAPI, e.g. mal:12345 (default - None) description: Anime database ID via AnimeAPI, e.g. mal:12345 (default - None)
enrich: enrich:
type: boolean type: boolean
description: Override show title and year from external source, and fill in the original language. Requires one of tmdb_id, imdb_id, tvdb_id or animeapi_id (default - false) description: Overwrite show title, year and original language with the external source's. Requires one of tmdb_id, imdb_id, tvdb_id or animeapi_id (default - false)
no_folder: no_folder:
type: boolean type: boolean
description: Disable folder creation for TV shows (default - false) description: Disable folder creation for TV shows (default - false)
@@ -791,10 +791,10 @@ async def download(request: web.Request) -> web.Response:
description: Add REPACK tag to the output filename (default - false) description: Add REPACK tag to the output filename (default - false)
imdb_id: imdb_id:
type: string type: string
description: Use this IMDB ID (e.g. tt1375666) for tagging (default - None) description: Use this IMDB ID (e.g. tt1375666) for tagging instead of a title search. Set enrich to also take its title, year and original language. Mutually exclusive with tmdb_id and tvdb_id (default - None)
tvdb_id: tvdb_id:
type: integer type: integer
description: Use this TVDB ID for tagging, episode ordering and enrichment (default - None) description: Use this TVDB ID for tagging and episode ordering instead of a series lookup. Set enrich to also take its title, year and original language. Mutually exclusive with tmdb_id and imdb_id. Needs tvdb_api_key (default - None)
tvdb_order: tvdb_order:
type: string type: string
enum: [official, dvd, absolute, alternate, regional] enum: [official, dvd, absolute, alternate, regional]
+3 -3
View File
@@ -3,7 +3,7 @@ from __future__ import annotations
import re import re
import warnings import warnings
from pathlib import Path from pathlib import Path
from typing import Any, Optional from typing import Any, Optional, Union
import yaml import yaml
from appdirs import AppDirs from appdirs import AppDirs
@@ -114,13 +114,13 @@ class Config:
self.tag: str = kwargs.get("tag") or "" self.tag: str = kwargs.get("tag") or ""
self.tag_group_name: bool = kwargs.get("tag_group_name", True) self.tag_group_name: bool = kwargs.get("tag_group_name", True)
self.tag_imdb_tmdb: bool = kwargs.get("tag_imdb_tmdb", True) self.tag_imdb_tmdb: bool = kwargs.get("tag_imdb_tmdb", True)
self.imdb_api_enabled: bool = kwargs.get("imdb_api_enabled", False)
self.omdb_api_key: str = kwargs.get("omdb_api_key") or "" self.omdb_api_key: str = kwargs.get("omdb_api_key") or ""
self.tmdb_api_key: str = kwargs.get("tmdb_api_key") or "" self.tmdb_api_key: str = kwargs.get("tmdb_api_key") or ""
self.tvdb_api_key: str = kwargs.get("tvdb_api_key") or "" self.tvdb_api_key: str = kwargs.get("tvdb_api_key") or ""
self.tvdb_pin: str = kwargs.get("tvdb_pin") or "" self.tvdb_pin: str = kwargs.get("tvdb_pin") or ""
self.tvdb_order: str = (kwargs.get("tvdb_order") or "").lower() self.tvdb_order: str = (kwargs.get("tvdb_order") or "").lower()
self.metadata_providers: list = kwargs.get("metadata_providers") or [] self.metadata_providers: Union[list, dict] = kwargs.get("metadata_providers") or []
self.disable_metadata: bool = kwargs.get("disable_metadata", False)
self.simkl_client_id: str = kwargs.get("simkl_client_id") or "" self.simkl_client_id: str = kwargs.get("simkl_client_id") or ""
self.decrypt_labs_api_key: str = kwargs.get("decrypt_labs_api_key") or "" self.decrypt_labs_api_key: str = kwargs.get("decrypt_labs_api_key") or ""
self.ipinfo_api_key: str = kwargs.get("ipinfo_api_key") or "" self.ipinfo_api_key: str = kwargs.get("ipinfo_api_key") or ""
+109 -15
View File
@@ -1,11 +1,11 @@
from __future__ import annotations from __future__ import annotations
from typing import TYPE_CHECKING, Optional from typing import TYPE_CHECKING, Optional, Union
import requests import requests
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, fuzzy_match, log from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, fuzzy_match, log
from unshackle.core.providers.imdbapi import IMDBApiProvider from unshackle.core.providers.imdb import IMDBProvider
from unshackle.core.providers.omdb import OMDBProvider from unshackle.core.providers.omdb import OMDBProvider
from unshackle.core.providers.simkl import SimklProvider from unshackle.core.providers.simkl import SimklProvider
from unshackle.core.providers.tmdb import TMDBProvider from unshackle.core.providers.tmdb import TMDBProvider
@@ -15,18 +15,28 @@ if TYPE_CHECKING:
from unshackle.core.title_cacher import TitleCacher from unshackle.core.title_cacher import TitleCacher
REGISTRY: dict[str, type[MetadataProvider]] = { REGISTRY: dict[str, type[MetadataProvider]] = {
cls.NAME: cls for cls in (IMDBApiProvider, OMDBProvider, SimklProvider, TMDBProvider, TVDBProvider) cls.NAME: cls for cls in (IMDBProvider, OMDBProvider, SimklProvider, TMDBProvider, TVDBProvider)
} }
# legacy `metadata_providers` names, still accepted
ALIASES: dict[str, str] = {"imdbapi": "imdb"}
# used when `metadata_providers` is unset # used when `metadata_providers` is unset
DEFAULT_ORDER: tuple[str, ...] = ("imdbapi", "omdb", "simkl", "tmdb", "tvdb") DEFAULT_ORDER: tuple[str, ...] = ("imdb", "omdb", "simkl", "tmdb", "tvdb")
def provider_order() -> list[type[MetadataProvider]]: def provider_order(kind: Optional[str] = None) -> list[type[MetadataProvider]]:
"""Provider classes in the order configured by `metadata_providers`.""" """Provider classes in the order configured by `metadata_providers`.
`metadata_providers` is either a flat list applying to both kinds, or a mapping of
kind ("tv"/"movie") to its own list. A kind the mapping omits uses `DEFAULT_ORDER`.
"""
from unshackle.core.config import config from unshackle.core.config import config
names = [str(n).lower() for n in (config.metadata_providers or DEFAULT_ORDER)] configured = config.metadata_providers
selected = (configured.get(kind) if kind else None) if isinstance(configured, dict) else configured
names = [ALIASES.get(str(n).lower(), str(n).lower()) for n in (selected or DEFAULT_ORDER)]
unknown = [n for n in names if n not in REGISTRY] unknown = [n for n in names if n not in REGISTRY]
if unknown: if unknown:
log.warning("Ignoring unknown metadata_providers entries: %s", ", ".join(unknown)) log.warning("Ignoring unknown metadata_providers entries: %s", ", ".join(unknown))
@@ -60,7 +70,15 @@ def search_metadata(
cache_account_hash: Optional[str] = None, cache_account_hash: Optional[str] = None,
) -> Optional[MetadataResult]: ) -> Optional[MetadataResult]:
"""Search all available providers for metadata. Returns best match.""" """Search all available providers for metadata. Returns best match."""
ordered = provider_order() from unshackle.core.config import config
# the one gate for `disable_metadata`: every automatic lookup reaches a provider through
# here, while a user-supplied ID goes straight to `resolve_by_ids` and stays allowed
if config.disable_metadata:
log.debug("Metadata lookups are disabled by config; not searching for %r", title)
return None
ordered = provider_order(kind)
# Check cache first # Check cache first
if title_cacher and cache_title_id: if title_cacher and cache_title_id:
@@ -268,6 +286,78 @@ def fetch_external_ids(
return ext return ext
def resolve_by_ids(
tmdb_id: Optional[int] = None,
imdb_id: Optional[str] = None,
tvdb_id: Optional[int] = None,
*,
title: Optional[str] = None,
year: Optional[int] = None,
kind: str = "movie",
title_cacher: Optional[TitleCacher] = None,
cache_title_id: Optional[str] = None,
cache_region: Optional[str] = None,
cache_account_hash: Optional[str] = None,
) -> Optional[MetadataResult]:
"""Resolve metadata from user-supplied external IDs, falling back to search only without them.
A supplied ID is authoritative. It is looked up directly through the providers that
consume its namespace, in `metadata_providers` order for `kind`, and it always survives
into `external_ids` whatever a provider answers. A fuzzy title search only runs when no
ID was supplied at all.
"""
supplied: dict[str, Union[int, str]] = {}
if tmdb_id is not None:
supplied["tmdb"] = tmdb_id
if imdb_id:
supplied["imdb"] = imdb_id
if tvdb_id is not None:
supplied["tvdb"] = tvdb_id
if not supplied:
if not title:
return None
return search_metadata(title, year, kind, title_cacher, cache_title_id, cache_region, cache_account_hash)
result: Optional[MetadataResult] = None
for cls in provider_order(kind):
provider_id = supplied.get(cls.ID_KIND or "")
if provider_id is None:
continue
p = get_provider(cls.NAME)
if not p:
continue
try:
result = p.get_by_id(provider_id, kind)
except Exception as exc:
log.debug("%s lookup of %s failed: %s", cls.NAME, provider_id, exc)
continue
if result:
break
if not result:
result = MetadataResult(title=title, year=year, kind=kind)
ids = result.external_ids
if tmdb_id is not None:
ids.tmdb_id = tmdb_id
ids.tmdb_kind = kind
if imdb_id:
ids.imdb_id = imdb_id
if tvdb_id is not None:
ids.tvdb_id = tvdb_id
if ids.tmdb_id and not (ids.imdb_id and ids.tvdb_id):
ext = fetch_external_ids(ids.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash)
if ext.imdb_id and not ids.imdb_id:
ids.imdb_id = ext.imdb_id
if ext.tvdb_id and not ids.tvdb_id:
ids.tvdb_id = ext.tvdb_id
enrich_ids(result)
return result
# -- Internal helpers -- # -- Internal helpers --
@@ -276,9 +366,9 @@ def fetch_external_ids(
_ENRICHMENT_AUTHORITY: tuple[str, ...] = ("tmdb", "simkl", "tvdb") _ENRICHMENT_AUTHORITY: tuple[str, ...] = ("tmdb", "simkl", "tvdb")
def _enrichment_providers() -> list[str]: def _enrichment_providers(kind: Optional[str] = None) -> list[str]:
"""Names of configured providers that can resolve an IMDB ID, most trusted first.""" """Names of configured providers that can resolve an IMDB ID, most trusted first."""
configured = {cls.NAME for cls in provider_order() if hasattr(cls, "find_by_imdb_id")} configured = {cls.NAME for cls in provider_order(kind) if hasattr(cls, "find_by_imdb_id")}
return [name for name in _ENRICHMENT_AUTHORITY if name in configured] return [name for name in _ENRICHMENT_AUTHORITY if name in configured]
@@ -298,7 +388,7 @@ def enrich_ids(result: MetadataResult) -> None:
kind = result.kind or "movie" kind = result.kind or "movie"
# Step 1: Collect enrichment results from all available providers # Step 1: Collect enrichment results from all available providers
authority = {name: i for i, name in enumerate(_enrichment_providers())} authority = {name: i for i, name in enumerate(_enrichment_providers(kind))}
enrichments: list[tuple[str, ExternalIds]] = [] enrichments: list[tuple[str, ExternalIds]] = []
for provider_name in authority: for provider_name in authority:
p = get_provider(provider_name) p = get_provider(provider_name)
@@ -477,9 +567,11 @@ def _cached_to_result(cached: dict, provider_name: str, kind: str) -> Optional[M
source="tvdb", source="tvdb",
raw=cached, raw=cached,
) )
elif provider_name == "imdbapi": elif provider_name == "imdb":
title = cached.get("primaryTitle") or cached.get("originalTitle") from unshackle.core.providers.imdb import primary_language
year = cached.get("startYear")
title = (cached.get("titleText") or {}).get("text") or (cached.get("originalTitleText") or {}).get("text")
year = (cached.get("releaseYear") or {}).get("year")
imdb_id = cached.get("id") imdb_id = cached.get("id")
# Restore enriched IDs that were saved alongside the raw data # Restore enriched IDs that were saved alongside the raw data
enriched = cached.get("_enriched_ids", {}) enriched = cached.get("_enriched_ids", {})
@@ -493,7 +585,8 @@ def _cached_to_result(cached: dict, provider_name: str, kind: str) -> Optional[M
tmdb_kind=enriched.get("tmdb_kind"), tmdb_kind=enriched.get("tmdb_kind"),
tvdb_id=enriched.get("tvdb_id"), tvdb_id=enriched.get("tvdb_id"),
), ),
source="imdbapi", original_language=primary_language(cached),
source="imdb",
raw=cached, raw=cached,
) )
return None return None
@@ -513,5 +606,6 @@ __all__ = [
"get_title_by_id", "get_title_by_id",
"get_year_by_id", "get_year_by_id",
"provider_order", "provider_order",
"resolve_by_ids",
"search_metadata", "search_metadata",
] ]
+2 -1
View File
@@ -37,7 +37,7 @@ class MetadataResult:
kind: Optional[str] = None # "movie" or "tv" kind: Optional[str] = None # "movie" or "tv"
external_ids: ExternalIds = field(default_factory=ExternalIds) external_ids: ExternalIds = field(default_factory=ExternalIds)
original_language: Optional[str] = None # alpha-2 or alpha-3, whichever the provider speaks original_language: Optional[str] = None # alpha-2 or alpha-3, whichever the provider speaks
source: str = "" # provider name, e.g. "tmdb", "simkl", "imdbapi" source: str = "" # provider name, e.g. "tmdb", "simkl", "imdb"
raw: Optional[dict] = None # original API response for caching raw: Optional[dict] = None # original API response for caching
@@ -46,6 +46,7 @@ class MetadataProvider(metaclass=ABCMeta):
NAME: str = "" NAME: str = ""
REQUIRES_KEY: bool = True REQUIRES_KEY: bool = True
ID_KIND: Optional[str] = None
def __init__(self) -> None: def __init__(self) -> None:
self.log = logging.getLogger(f"METADATA.{self.NAME.upper()}") self.log = logging.getLogger(f"METADATA.{self.NAME.upper()}")
+156
View File
@@ -0,0 +1,156 @@
from __future__ import annotations
import hashlib
import json
from difflib import SequenceMatcher
from typing import Optional, Union
import requests
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, _clean, fuzzy_match
GRAPHQL_URL = "https://caching.graphql.imdb.com/"
GRAPHQL_HEADERS = {
"Accept": "application/graphql+json, application/json",
"Content-Type": "application/json",
"Origin": "https://www.imdb.com",
"X-Imdb-Client-Name": "imdb-web-next-localized",
"X-Imdb-User-Country": "US",
}
KIND_TO_TYPES: dict[str, list[str]] = {
"movie": ["movie", "tvMovie"],
"tv": ["tvSeries", "tvMiniSeries"],
}
TITLE_FIELDS = (
"id titleText { text } originalTitleText { text } releaseYear { year } "
"spokenLanguages { spokenLanguages { id text } }"
)
def escape_graphql(value: str) -> str:
return value.replace("\\", "\\\\").replace('"', '\\"')
def primary_language(data: dict) -> Optional[str]:
"""IMDb orders spokenLanguages most prominent first. Either level can come back null."""
spoken = data.get("spokenLanguages") or {}
langs = spoken.get("spokenLanguages") or []
return langs[0].get("id") if langs else None
def _title_to_result(node: dict, kind: str) -> Optional[MetadataResult]:
title = (node.get("titleText") or {}).get("text") or (node.get("originalTitleText") or {}).get("text")
if not title:
return None
return MetadataResult(
title=title,
year=(node.get("releaseYear") or {}).get("year"),
kind=kind,
external_ids=ExternalIds(imdb_id=node.get("id")),
original_language=primary_language(node),
source="imdb",
raw=node,
)
class IMDBProvider(MetadataProvider):
"""IMDb metadata provider using IMDb's own GraphQL edge cache (no API key)."""
NAME = "imdb"
REQUIRES_KEY = False
ID_KIND = "imdb"
def is_available(self) -> bool:
return True
def _graphql(self, operation: str, query: str) -> Optional[dict]:
"""Run a persisted query, registering it with one POST the first time it is seen."""
extensions = {"persistedQuery": {"version": 1, "sha256Hash": hashlib.sha256(query.encode()).hexdigest()}}
# IMDb's GraphQL gateway decodes "+" literally, so these JSON params must carry no spaces
params = {
"operationName": operation,
"variables": "{}",
"extensions": json.dumps(extensions, separators=(",", ":")),
}
try:
r = self.session.get(GRAPHQL_URL, params=params, headers=GRAPHQL_HEADERS, timeout=30)
body = r.json()
if _persisted_query_missing(body):
self.log.debug("Registering persisted query %s", operation)
payload = {"operationName": operation, "variables": {}, "query": query, "extensions": extensions}
r = self.session.post(GRAPHQL_URL, json=payload, headers=GRAPHQL_HEADERS, timeout=30)
body = r.json()
except (requests.RequestException, ValueError) as exc:
self.log.debug("IMDb GraphQL %s failed: %s", operation, exc)
return None
if body.get("errors"):
self.log.debug("IMDb GraphQL %s errors: %s", operation, [e.get("message") for e in body["errors"]][:3])
return None
return body.get("data")
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
self.log.debug("Searching IMDb for %r (%s, %s)", title, kind, year)
constraints = [f'titleTextConstraint: {{searchTerm: "{escape_graphql(title)}"}}']
if year:
constraints.append(
f'releaseDateConstraint: {{releaseDateRange: {{start: "{year}-01-01", end: "{year}-12-31"}}}}'
)
types = KIND_TO_TYPES.get(kind)
if types:
constraints.append(f"titleTypeConstraint: {{anyTitleTypeIds: {json.dumps(types)}}}")
query = (
"query UnshackleTitleSearch { advancedTitleSearch(first: 5, constraints: {%s}) "
"{ edges { node { title { %s } } } } }" % (", ".join(constraints), TITLE_FIELDS)
)
data = self._graphql("UnshackleTitleSearch", query)
edges = ((data or {}).get("advancedTitleSearch") or {}).get("edges") or []
nodes = [edge["node"]["title"] for edge in edges if (edge.get("node") or {}).get("title")]
if not nodes:
self.log.debug("IMDb returned no results for %r", title)
return None
best = max(nodes, key=lambda n: _match_ratio(n, title))
result = _title_to_result(best, kind)
if not result or not result.title or not fuzzy_match(result.title, title):
self.log.debug("IMDb title mismatch: searched %r, got %r", title, result.title if result else None)
return None
self.log.debug("IMDb -> %s (ID %s)", result.title, result.external_ids.imdb_id)
return result
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
"""Fetch metadata by IMDB ID (e.g. 'tt1375666')."""
imdb_id = str(provider_id)
self.log.debug("Fetching IMDb title %s", imdb_id)
query = 'query UnshackleTitle { title(id: "%s") { %s } }' % (escape_graphql(imdb_id), TITLE_FIELDS)
data = self._graphql("UnshackleTitle", query)
node = (data or {}).get("title")
if not node:
return None
return _title_to_result(node, kind)
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
"""Return external IDs. For IMDB, the provider_id IS the IMDB ID."""
return ExternalIds(imdb_id=str(provider_id))
def _match_ratio(node: dict, title: str) -> float:
names = [(node.get("titleText") or {}).get("text"), (node.get("originalTitleText") or {}).get("text")]
return max((SequenceMatcher(None, _clean(title), _clean(n)).ratio() for n in names if n), default=0.0)
def _persisted_query_missing(body: dict) -> bool:
for error in body.get("errors") or []:
code = (error.get("extensions") or {}).get("code")
if code == "PERSISTED_QUERY_NOT_FOUND" or error.get("message") == "PersistedQueryNotFound":
return True
return False
-132
View File
@@ -1,132 +0,0 @@
from __future__ import annotations
from difflib import SequenceMatcher
from typing import Optional, Union
import requests
from unshackle.core.config import config
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, _clean, fuzzy_match
# Mapping from our kind ("movie"/"tv") to IMDxAPI title types
KIND_TO_TYPES: dict[str, list[str]] = {
"movie": ["movie"],
"tv": ["tvSeries", "tvMiniSeries"],
}
def primary_language(data: dict) -> Optional[str]:
"""IMDb orders spokenLanguages most prominent first, and only detail responses carry it."""
langs = data.get("spokenLanguages") or []
return langs[0].get("code") if langs else None
class IMDBApiProvider(MetadataProvider):
"""IMDb metadata provider using IMDxAPI (api.tiffara.com, free, no API key)."""
NAME = "imdbapi"
REQUIRES_KEY = False
BASE_URL = "https://api.tiffara.com"
def is_available(self) -> bool:
return config.imdb_api_enabled
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
self.log.debug("Searching IMDBApi for %r (%s, %s)", title, kind, year)
try:
params: dict[str, str | int] = {"query": title, "limit": 20}
r = self.session.get(
f"{self.BASE_URL}/search/titles",
params=params,
timeout=30,
)
r.raise_for_status()
data = r.json()
except (requests.RequestException, ValueError) as exc:
self.log.debug("IMDBApi search failed: %s", exc)
return None
results = data.get("titles") or data.get("results") or []
if not results:
self.log.debug("IMDBApi returned no results for %r", title)
return None
# Filter by type if possible
type_filter = KIND_TO_TYPES.get(kind, [])
filtered = [r for r in results if r.get("type") in type_filter] if type_filter else results
candidates = filtered if filtered else results
# Find best fuzzy match, optionally filtered by year
best_match: Optional[dict] = None
best_ratio = 0.0
for candidate in candidates:
primary = candidate.get("primaryTitle") or ""
original = candidate.get("originalTitle") or ""
for name in [primary, original]:
if not name:
continue
ratio = SequenceMatcher(None, _clean(title), _clean(name)).ratio()
if ratio > best_ratio:
# If year provided, prefer matches within 1 year
candidate_year = candidate.get("startYear")
if year and candidate_year and abs(year - candidate_year) > 1:
continue
best_ratio = ratio
best_match = candidate
if not best_match:
self.log.debug("No matching result found in IMDBApi for %r", title)
return None
result_title = best_match.get("primaryTitle") or best_match.get("originalTitle")
if not result_title or not fuzzy_match(result_title, title):
self.log.debug("IMDBApi title mismatch: searched %r, got %r", title, result_title)
return None
imdb_id = best_match.get("id")
result_year = best_match.get("startYear")
self.log.debug("IMDBApi -> %s (ID %s)", result_title, imdb_id)
return MetadataResult(
title=result_title,
year=result_year,
kind=kind,
external_ids=ExternalIds(imdb_id=imdb_id),
original_language=primary_language(best_match),
source="imdbapi",
raw=best_match,
)
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
"""Fetch metadata by IMDB ID (e.g. 'tt1375666')."""
imdb_id = str(provider_id)
self.log.debug("Fetching IMDBApi title %s", imdb_id)
try:
r = self.session.get(f"{self.BASE_URL}/titles/{imdb_id}", timeout=30)
r.raise_for_status()
data = r.json()
except (requests.RequestException, ValueError) as exc:
self.log.debug("IMDBApi get_by_id failed: %s", exc)
return None
title = data.get("primaryTitle") or data.get("originalTitle")
result_year = data.get("startYear")
return MetadataResult(
title=title,
year=result_year,
kind=kind,
external_ids=ExternalIds(imdb_id=data.get("id")),
original_language=primary_language(data),
source="imdbapi",
raw=data,
)
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
"""Return external IDs. For IMDB, the provider_id IS the IMDB ID."""
return ExternalIds(imdb_id=str(provider_id))
+1
View File
@@ -39,6 +39,7 @@ class OMDBProvider(MetadataProvider):
NAME = "omdb" NAME = "omdb"
REQUIRES_KEY = True REQUIRES_KEY = True
ID_KIND = "imdb"
BASE_URL = "https://www.omdbapi.com/" BASE_URL = "https://www.omdbapi.com/"
def is_available(self) -> bool: def is_available(self) -> bool:
+1
View File
@@ -14,6 +14,7 @@ class TMDBProvider(MetadataProvider):
NAME = "tmdb" NAME = "tmdb"
REQUIRES_KEY = True REQUIRES_KEY = True
ID_KIND = "tmdb"
BASE_URL = "https://api.themoviedb.org/3" BASE_URL = "https://api.themoviedb.org/3"
def is_available(self) -> bool: def is_available(self) -> bool:
+1
View File
@@ -71,6 +71,7 @@ class TVDBProvider(MetadataProvider):
NAME = "tvdb" NAME = "tvdb"
REQUIRES_KEY = True REQUIRES_KEY = True
ID_KIND = "tvdb"
BASE_URL = "https://api4.thetvdb.com/v4" BASE_URL = "https://api4.thetvdb.com/v4"
def __init__(self) -> None: def __init__(self) -> None:
+4 -48
View File
@@ -12,13 +12,9 @@ from unshackle.core import binaries
from unshackle.core.config import config from unshackle.core.config import config
from unshackle.core.providers import ( from unshackle.core.providers import (
ExternalIds, ExternalIds,
MetadataResult,
enrich_ids,
fetch_external_ids,
fuzzy_match, fuzzy_match,
get_available_providers, get_available_providers,
get_provider, resolve_by_ids,
search_metadata,
) )
from unshackle.core.titles.episode import Episode from unshackle.core.titles.episode import Episode
from unshackle.core.titles.movie import Movie from unshackle.core.titles.movie import Movie
@@ -118,53 +114,13 @@ def tag_file(
if config.tag_imdb_tmdb: if config.tag_imdb_tmdb:
try: try:
providers = get_available_providers() has_ids = tmdb_id is not None or imdb_id is not None or tvdb_id is not None
if not providers and tvdb_id is None: if not get_available_providers() and not has_ids:
log.debug("No metadata providers available; skipping tag lookup") log.debug("No metadata providers available; skipping tag lookup")
apply_tags(path, custom_tags) apply_tags(path, custom_tags)
return return
result: Optional[MetadataResult] = None result = resolve_by_ids(tmdb_id, imdb_id, tvdb_id, title=name, year=year, kind=kind)
# Direct ID lookup path
if imdb_id:
imdbapi = get_provider("imdbapi")
if imdbapi:
result = imdbapi.get_by_id(imdb_id, kind)
if not result:
# IMDxAPI is off or had no answer; the ID alone still resolves the others
result = MetadataResult(title=name, year=year, kind=kind)
result.external_ids.imdb_id = imdb_id
enrich_ids(result)
elif tmdb_id is not None:
tmdb = get_provider("tmdb")
if tmdb:
result = tmdb.get_by_id(tmdb_id, kind)
if result:
ext = tmdb.get_external_ids(tmdb_id, kind)
result.external_ids = ext
enrich_ids(result)
else:
# Search across providers in priority order
result = search_metadata(name, year, kind)
# If we got a TMDB ID from search but not the other IDs, fetch them
if (
result
and result.external_ids.tmdb_id
and not (result.external_ids.imdb_id and result.external_ids.tvdb_id)
):
ext = fetch_external_ids(result.external_ids.tmdb_id, kind)
if ext.imdb_id and not result.external_ids.imdb_id:
result.external_ids.imdb_id = ext.imdb_id
if ext.tvdb_id and not result.external_ids.tvdb_id:
result.external_ids.tvdb_id = ext.tvdb_id
enrich_ids(result)
if tvdb_id is not None:
if result is None:
result = MetadataResult(kind=kind)
result.external_ids.tvdb_id = tvdb_id
if result and result.external_ids: if result and result.external_ids:
standard_tags = _build_tags_from_ids(result.external_ids, kind) standard_tags = _build_tags_from_ids(result.external_ids, kind)