mirror of
https://github.com/unshackle-dl/unshackle.git
synced 2026-11-16 14:05:10 +00:00
feat(metadata)!: authoritative id resolver and keyless imdb provider
Supplied IDs resolve directly via a shared resolve_by_ids and never fall back to title search; metadata_providers gains per-kind ordering and a disable_metadata switch stops all automatic lookups while explicit IDs keep working. BREAKING: --tmdb/--imdb/--tvdb are now mutually exclusive and require a usable provider; --enrich also overwrites original language; the imdbapi provider and imdb_api_enabled are replaced by a keyless imdb provider.
This commit is contained in:
@@ -392,9 +392,9 @@ Create a download job. Requires `service` and `title_id`; every other field is a
|
|||||||
| `best_available` | boolean | `false` | Fall back to best available. |
|
| `best_available` | boolean | `false` | Fall back to best available. |
|
||||||
| `repack` | boolean | `false` | Add REPACK tag. |
|
| `repack` | boolean | `false` | Add REPACK tag. |
|
||||||
| `tag` | string | `null` | Release group tag. |
|
| `tag` | string | `null` | Release group tag. |
|
||||||
| `tmdb_id` / `imdb_id` / `tvdb_id` / `animeapi_id` | - | `null` | External ID overrides for tagging. |
|
| `tmdb_id` / `imdb_id` / `tvdb_id` / `animeapi_id` | - | `null` | External ID overrides. Each resolves its metadata directly instead of by a title search, and is used for tagging. Set `enrich` to also take the title, year and original language. Give at most one of `tmdb_id`, `imdb_id` and `tvdb_id`, since unshackle resolves the others from it. Sending two returns `400`. `animeapi_id` still combines with one of them. `tmdb_id` and `tvdb_id` must be positive integers and `imdb_id` must look like `tt1375666`, or the request returns `400`. An ID whose provider is unconfigured, such as `tmdb_id` with no `tmdb_api_key`, fails the job rather than returning `400`. |
|
||||||
| `tvdb_order` | `official`, `dvd`, `absolute`, `alternate`, `regional` | `null` | Renumber episodes to a TVDB season order. Falls back to the `tvdb_order` config option. |
|
| `tvdb_order` | `official`, `dvd`, `absolute`, `alternate`, `regional` | `null` | Renumber episodes to a TVDB season order. Falls back to the `tvdb_order` config option. |
|
||||||
| `enrich` | boolean | `false` | Override title/year from external source, and fill in the original language. Needs one of `tmdb_id`, `imdb_id`, `tvdb_id` or `animeapi_id`; without one the job fails instead of returning `400`. |
|
| `enrich` | boolean | `false` | Overwrite title, year and original language with the external source's. Needs one of `tmdb_id`, `imdb_id`, `tvdb_id` or `animeapi_id`. Without one the job fails instead of returning `400`. |
|
||||||
| `output_dir` | string | `null` | Override output directory. |
|
| `output_dir` | string | `null` | Override output directory. |
|
||||||
| `no_cache` / `reset_cache` | boolean | `false` | Title cache controls. |
|
| `no_cache` / `reset_cache` | boolean | `false` | Title cache controls. |
|
||||||
|
|
||||||
|
|||||||
@@ -150,13 +150,19 @@ Keep only certain track types, or skip certain track types. Attachments are alwa
|
|||||||
|
|
||||||
| Flag | Description |
|
| Flag | Description |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `--tmdb` | TMDB ID (integer). |
|
| `--tmdb` | TMDB ID (integer). Used for the tags. Skips the title search. `--enrich` reads it too. Needs `tmdb_api_key`. |
|
||||||
| `--imdb` | IMDb ID, e.g. `tt1375666`. |
|
| `--imdb` | IMDb ID, e.g. `tt1375666`. Used for the tags. Skips the title search. `--enrich` reads it too. Needs no key. |
|
||||||
| `--tvdb` | TVDB ID (integer). Skips the series lookup that `--tvdb-order` would otherwise do. `--enrich` reads it too. |
|
| `--tvdb` | TVDB ID (integer). Used for the tags. Skips the series lookup that `--tvdb-order` would otherwise do. `--enrich` reads it too. Needs `tvdb_api_key`. |
|
||||||
| `--animeapi` | AnimeAPI ID, e.g. `mal:12345` or `anilist:98765` (defaults to MAL). Back-fills TMDB/IMDb/TVDB. |
|
| `--animeapi` | AnimeAPI ID, e.g. `mal:12345` or `anilist:98765` (defaults to MAL). Back-fills TMDB/IMDb/TVDB. |
|
||||||
| `--enrich` | Override show title/year from the external source, and fill in the original language. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. |
|
| `--enrich` | Overwrite show title, year and original language with the external source's. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. |
|
||||||
| `--tvdb-order` | Renumber episodes to a TVDB season order: `official` (aired), `dvd`, `absolute`, `alternate`, or `regional`. Needs `tvdb_api_key`. |
|
| `--tvdb-order` | Renumber episodes to a TVDB season order: `official` (aired), `dvd`, `absolute`, `alternate`, or `regional`. Needs `tvdb_api_key`. |
|
||||||
|
|
||||||
|
!!! note "One ID at a time"
|
||||||
|
`--tmdb`, `--imdb` and `--tvdb` cannot be combined. Give one and unshackle resolves the
|
||||||
|
others from it, writing all three to the tags. `--animeapi` still pairs with one of them.
|
||||||
|
An ID with no provider that can resolve it, such as `--tmdb` with no `tmdb_api_key`, fails
|
||||||
|
before the download starts.
|
||||||
|
|
||||||
#### Episode ordering
|
#### Episode ordering
|
||||||
|
|
||||||
A service does not always number a series the way TVDB's aired order does. Some services list
|
A service does not always number a series the way TVDB's aired order does. Some services list
|
||||||
|
|||||||
+43
-15
@@ -621,50 +621,78 @@ unshackle dl --skip-dl --export EXAMPLE 81234567
|
|||||||
## Metadata and tagging
|
## Metadata and tagging
|
||||||
|
|
||||||
unshackle looks up metadata automatically, but you can override the identifiers used for
|
unshackle looks up metadata automatically, but you can override the identifiers used for
|
||||||
tagging and naming:
|
tagging and naming. An ID you give is authoritative: unshackle looks it up directly instead of
|
||||||
|
searching by title, and the ID stays in the tags. An ID only settles *which* title this is.
|
||||||
|
To also take that source's title, year and original language, add `--enrich`. Give at most one
|
||||||
|
of `--tmdb`, `--imdb` and `--tvdb`; unshackle works the rest out from it. To stop the automatic
|
||||||
|
lookups but keep the IDs you give, set
|
||||||
|
[`disable_metadata`](../reference/configuration/misc.md#external-api-keys) in your config.
|
||||||
|
|
||||||
| Flag | Example | Purpose |
|
| Flag | Example | Purpose |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `--tmdb` | `--tmdb 27205` | Use this TMDB ID instead of automatic lookup. |
|
| `--tmdb` | `--tmdb 27205` | Use this TMDB ID instead of an automatic search. Needs `tmdb_api_key`. |
|
||||||
| `--imdb` | `--imdb tt1375666` | Use this IMDb ID. |
|
| `--imdb` | `--imdb tt1375666` | Use this IMDb ID instead of an automatic search. Needs no key. |
|
||||||
| `--tvdb` | `--tvdb 73871` | Use this TVDB ID instead of looking the series up. |
|
| `--tvdb` | `--tvdb 73871` | Use this TVDB ID instead of looking the series up. Needs `tvdb_api_key`. |
|
||||||
| `--animeapi` | `--animeapi mal:12345` | Resolve via AnimeAPI (`mal:`/`anilist:` prefix; defaults to MAL). |
|
| `--animeapi` | `--animeapi mal:12345` | Resolve via AnimeAPI (`mal:`/`anilist:` prefix; defaults to MAL). |
|
||||||
| `--enrich` | - | Override the show title and year from an external source, and fill in the original language. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. |
|
| `--enrich` | - | Overwrite the show title, year and original language with the external source's. **Requires** one of `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi`. |
|
||||||
| `--tvdb-order` | `--tvdb-order dvd` | Renumber episodes to a TVDB season order. Needs `tvdb_api_key`. |
|
| `--tvdb-order` | `--tvdb-order dvd` | Renumber episodes to a TVDB season order. Needs `tvdb_api_key`. |
|
||||||
|
|
||||||
```shell title="Force the right IMDb match and enrich the title"
|
```shell title="Force the right IMDb match, and keep the service's own naming"
|
||||||
|
unshackle dl --imdb tt1375666 EXAMPLE 81234567
|
||||||
|
```
|
||||||
|
|
||||||
|
```shell title="Force the match and take IMDb's title, year and language too"
|
||||||
unshackle dl --imdb tt1375666 --enrich EXAMPLE 81234567
|
unshackle dl --imdb tt1375666 --enrich EXAMPLE 81234567
|
||||||
```
|
```
|
||||||
|
|
||||||
### What each metadata provider supplies
|
### What each metadata provider supplies
|
||||||
|
|
||||||
Providers differ in what they answer with, so `--enrich` reads different fields depending on
|
Providers differ in what they answer with, so the fields `--enrich` can replace depend on
|
||||||
which ID you gave it. A provider only runs when its key is configured:
|
which ID you gave. Every provider except `imdb` only runs when its key is configured:
|
||||||
|
|
||||||
| Provider | Config key | Title and year | Original language | External IDs it returns |
|
| Provider | Config key | Title and year | Original language | External IDs it returns |
|
||||||
| --- | --- | --- | --- | --- |
|
| --- | --- | --- | --- | --- |
|
||||||
| TMDB | `tmdb_api_key` | yes | yes, alpha-2 such as `ko` | IMDb, TMDB, TVDB |
|
| TMDB | `tmdb_api_key` | yes | yes, alpha-2 such as `ko` | IMDb, TMDB, TVDB |
|
||||||
| TVDB | `tvdb_api_key` | yes | yes, alpha-3 such as `kor` | TMDB, TVDB, sometimes IMDb |
|
| TVDB | `tvdb_api_key` | yes | yes, alpha-3 such as `kor` | TMDB, TVDB, sometimes IMDb |
|
||||||
| OMDb | `omdb_api_key` | yes | yes, as an English name such as `Korean` | IMDb |
|
| OMDb | `omdb_api_key` | yes | yes, as an English name such as `Korean` | IMDb |
|
||||||
| IMDxAPI | `imdb_api_enabled` | yes | only when looked up by ID | IMDb |
|
| IMDb | *(no key needed)* | yes | yes, alpha-2 such as `ko` | IMDb |
|
||||||
| SIMKL | `simkl_client_id` | yes | no, it publishes a country and no language | IMDb, TMDB, TVDB |
|
| SIMKL | `simkl_client_id` | yes | no, it publishes a country and no language | IMDb, TMDB, TVDB |
|
||||||
|
|
||||||
Whatever the tag looks like, unshackle normalises it before use, so `ko`, `kor` and `Korean`
|
Whatever the tag looks like, unshackle normalises it before use, so `ko`, `kor` and `Korean`
|
||||||
all end up as the same language.
|
all end up as the same language.
|
||||||
|
|
||||||
`--enrich` consults these in a fixed order:
|
Each ID goes to the providers that read that kind of ID, in
|
||||||
|
[`metadata_providers`](../reference/configuration/misc.md#external-api-keys) order:
|
||||||
|
|
||||||
- `--tmdb` reads TMDB.
|
- `--tmdb` reads TMDB.
|
||||||
- `--imdb` tries IMDxAPI, then falls back to OMDb. Since `imdb_api_enabled` is `false` by
|
- `--imdb` reads IMDb, then falls back to OMDb. IMDb needs no key, so this path works out of
|
||||||
default, an OMDb key is what makes this path work for most people.
|
the box, and an OMDb key only matters when IMDb has no answer.
|
||||||
- `--tvdb` reads TVDB. It needs `tvdb_api_key`.
|
- `--tvdb` reads TVDB. It needs `tvdb_api_key`.
|
||||||
- `--animeapi` supplies a title, plus whichever IDs AnimeAPI knows for the show. If that
|
- `--animeapi` supplies a title, plus whichever IDs AnimeAPI knows for the show. If that
|
||||||
includes a TMDB, IMDb or TVDB ID, the year and language come from that provider. If it
|
includes a TMDB, IMDb or TVDB ID, the year and language come from that provider. If it
|
||||||
does not, pair it with `--tmdb`, `--imdb` or `--tvdb`.
|
does not, pair it with `--tmdb`, `--imdb` or `--tvdb`.
|
||||||
|
|
||||||
Language is the one field `--enrich` will not overwrite. A service that already reported an
|
Give **one** of `--tmdb`, `--imdb` and `--tvdb`. They cannot be combined, because one ID does
|
||||||
original language keeps it, because the platform knows its own catalogue better than a
|
the job on its own: unshackle resolves the other two from it and writes all three to the tags.
|
||||||
third-party match does. Everything else follows the usual override behaviour.
|
Passing two is an error rather than a silent choice between them. `--animeapi` is the exception
|
||||||
|
and still pairs with one of the three, which is what to do when AnimeAPI knows no IDs for the
|
||||||
|
show.
|
||||||
|
|
||||||
|
The ID you give must have a provider that can actually resolve it, so unshackle checks that
|
||||||
|
before it downloads anything. `--tmdb` without `tmdb_api_key`, or an ID whose providers your
|
||||||
|
`metadata_providers` list leaves out, fails immediately with a message naming what to set.
|
||||||
|
`--imdb` needs no key, since the `imdb` provider is keyless and in the default order.
|
||||||
|
|
||||||
|
With `--enrich`, the title, year and original language are all replaced with the external
|
||||||
|
source's values, whether or not the service already filled them in. A field the source does
|
||||||
|
not answer with is left alone, and unshackle logs which fields those were, so a provider with
|
||||||
|
a thin record cannot blank out what the service told you.
|
||||||
|
|
||||||
|
!!! warning "`--enrich` replaces the original language, which affects more than the filename"
|
||||||
|
Track selection reads the original language, so replacing it changes which audio is
|
||||||
|
treated as the original. That is the point when a service mislabels it, but it means a
|
||||||
|
wrong ID can pick the wrong audio track and not only write a wrong name. Without
|
||||||
|
`--enrich` the service's own value is kept and track selection is untouched.
|
||||||
|
|
||||||
### Episode ordering
|
### Episode ordering
|
||||||
|
|
||||||
|
|||||||
@@ -36,7 +36,7 @@ Where the file lives, how unshackle discovers it, and how to edit it:
|
|||||||
| [DRM & CDM](drm.md) | `cdm`, `remote_cdm`, `decryption` |
|
| [DRM & CDM](drm.md) | `cdm`, `remote_cdm`, `decryption` |
|
||||||
| [Network & proxy](network.md) | `network`, `headers`, `proxy_providers` |
|
| [Network & proxy](network.md) | `network`, `headers`, `proxy_providers` |
|
||||||
| [Key vaults](vaults.md) | `key_vaults`, `vault_timeout` |
|
| [Key vaults](vaults.md) | `key_vaults`, `vault_timeout` |
|
||||||
| [External API keys](misc.md#external-api-keys) | `imdb_api_enabled`, `omdb_api_key`, `tmdb_api_key`, `tvdb_api_key`, `tvdb_pin`, `tvdb_order`, `simkl_client_id`, `metadata_providers`, `decrypt_labs_api_key`, `ipinfo_api_key` |
|
| [External API keys](misc.md#external-api-keys) | `omdb_api_key`, `tmdb_api_key`, `tvdb_api_key`, `tvdb_pin`, `tvdb_order`, `simkl_client_id`, `metadata_providers`, `disable_metadata`, `decrypt_labs_api_key`, `ipinfo_api_key` |
|
||||||
| [Caching & updates](misc.md#caching-updates) | `title_cache_enabled`, `title_cache_time`, `title_cache_max_retention`, `update_checks`, `update_check_interval` |
|
| [Caching & updates](misc.md#caching-updates) | `title_cache_enabled`, `title_cache_time`, `title_cache_max_retention`, `update_checks`, `update_check_interval` |
|
||||||
| [Logging, privacy & debug](misc.md#logging-privacy-debug) | `redact_paths`, `debug`, `debug_keys`, `debug_requests`, `set_terminal_bg` |
|
| [Logging, privacy & debug](misc.md#logging-privacy-debug) | `redact_paths`, `debug`, `debug_keys`, `debug_requests`, `set_terminal_bg` |
|
||||||
| [Deprecated & removed](misc.md#deprecated-removed-keys) | `curl_impersonate`, `downloader`, `scene_naming` |
|
| [Deprecated & removed](misc.md#deprecated-removed-keys) | `curl_impersonate`, `downloader`, `scene_naming` |
|
||||||
|
|||||||
@@ -10,14 +10,14 @@ each metadata provider can actually answer with, see
|
|||||||
|
|
||||||
| Key | Type | Default | Description |
|
| Key | Type | Default | Description |
|
||||||
|-----|------|---------|-------------|
|
|-----|------|---------|-------------|
|
||||||
| `imdb_api_enabled` | bool | `false` | Use the free IMDxAPI (api.tiffara.com) metadata provider, which needs no API key. Off by default since the site has been unreliable. |
|
| `omdb_api_key` | str | `""` | [OMDb](https://www.omdbapi.com/) API key for IMDb metadata lookups; a fallback for when the keyless `imdb` provider has no answer. Free keys are available on the OMDb site. |
|
||||||
| `omdb_api_key` | str | `""` | [OMDb](https://www.omdbapi.com/) API key for IMDb metadata lookups; a more reliable alternative to IMDxAPI. Free keys are available on the OMDb site. |
|
|
||||||
| `tmdb_api_key` | str | `""` | TMDB API key for metadata enrichment and external-ID tags. |
|
| `tmdb_api_key` | str | `""` | TMDB API key for metadata enrichment and external-ID tags. |
|
||||||
| `simkl_client_id` | str | `""` | SIMKL client ID for metadata lookups; an alternative/fallback source to TMDB. |
|
| `simkl_client_id` | str | `""` | SIMKL client ID for metadata lookups; an alternative/fallback source to TMDB. |
|
||||||
| `tvdb_api_key` | str | `""` | [TheTVDB v4](https://thetvdb.com/api-information) API key; a fallback source to TMDB, strongest on TV series. Also the source `--enrich --tvdb` reads. Free keys are available on the TVDB site. |
|
| `tvdb_api_key` | str | `""` | [TheTVDB v4](https://thetvdb.com/api-information) API key; a fallback source to TMDB, strongest on TV series. Also the source `--enrich --tvdb` reads. Free keys are available on the TVDB site. |
|
||||||
| `tvdb_pin` | str | `""` | Subscriber PIN, only needed for a user-supported TVDB key. Leave empty for a normal project key. |
|
| `tvdb_pin` | str | `""` | Subscriber PIN, only needed for a user-supported TVDB key. Leave empty for a normal project key. |
|
||||||
| `tvdb_order` | str | `""` | Default for `--tvdb-order`: renumber episodes to a TVDB season order (`official`, `dvd`, `absolute`, `alternate`, `regional`). Empty keeps the numbering the service gives. |
|
| `tvdb_order` | str | `""` | Default for `--tvdb-order`: renumber episodes to a TVDB season order (`official`, `dvd`, `absolute`, `alternate`, `regional`). Empty keeps the numbering the service gives. |
|
||||||
| `metadata_providers` | list | *(see below)* | Metadata providers to use, in the order they are tried. |
|
| `metadata_providers` | list or map | *(see below)* | Metadata providers to use, in the order they are tried. |
|
||||||
|
| `disable_metadata` | bool | `false` | Set to `true` to stop all automatic metadata lookups. An ID you give yourself is still looked up. |
|
||||||
| `decrypt_labs_api_key` | str | `""` | Global Decrypt Labs API key (used by remote CDM / vault). |
|
| `decrypt_labs_api_key` | str | `""` | Global Decrypt Labs API key (used by remote CDM / vault). |
|
||||||
| `ipinfo_api_key` | str | `""` | ipinfo.io API key for IP/region lookups. |
|
| `ipinfo_api_key` | str | `""` | ipinfo.io API key for IP/region lookups. |
|
||||||
|
|
||||||
@@ -33,12 +33,61 @@ each metadata provider can actually answer with, see
|
|||||||
ignored with a warning. Leave it unset for the default order:
|
ignored with a warning. Leave it unset for the default order:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
metadata_providers: [imdbapi, omdb, simkl, tmdb, tvdb]
|
metadata_providers: [imdb, omdb, simkl, tmdb, tvdb]
|
||||||
```
|
```
|
||||||
|
|
||||||
Whatever the order, a provider is skipped when its API key is missing (or, for
|
Whatever the order, a provider is skipped when its API key is missing. `imdb` needs no
|
||||||
`imdbapi`, when `imdb_api_enabled` is `false`). Available names are
|
key, so it is always available. Available names are `imdb`, `omdb`, `simkl`, `tmdb`, and
|
||||||
`imdbapi`, `omdb`, `simkl`, `tmdb`, and `tvdb`.
|
`tvdb`. The old name `imdbapi` still works and is read as `imdb`.
|
||||||
|
|
||||||
|
The order applies to a title search and to a direct lookup of an ID you give with
|
||||||
|
`--tmdb`, `--imdb`, or `--tvdb`. A supplied ID always goes to the providers that read
|
||||||
|
that kind of ID, and it never falls back to a title search.
|
||||||
|
|
||||||
|
!!! note "What `disable_metadata` turns off"
|
||||||
|
With `disable_metadata: true`, unshackle never contacts a metadata provider on its own.
|
||||||
|
The title search is dead, and that includes the keyless `imdb` provider. Titles, years and
|
||||||
|
languages then come only from the service.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
disable_metadata: true
|
||||||
|
```
|
||||||
|
|
||||||
|
An ID you give with `--tmdb`, `--imdb`, `--tvdb`, or `--animeapi` is your permission to use
|
||||||
|
that ID, so it still works: the ID is looked up directly, the other IDs are cross
|
||||||
|
referenced from it, and `--enrich` still reads that source. Only the lookups you did not
|
||||||
|
ask for are stopped.
|
||||||
|
|
||||||
|
!!! note "The `imdb` provider needs no key"
|
||||||
|
It replaces the earlier `imdbapi` provider, which read a third-party mirror
|
||||||
|
(api.tiffara.com) and needed the `imdb_api_enabled` switch. That option is gone: delete it
|
||||||
|
from your config, where it is now ignored.
|
||||||
|
|
||||||
|
!!! tip "A different order per title kind"
|
||||||
|
Give a map instead of one list when the best source differs by kind. TVDB is strongest on
|
||||||
|
series, TMDB on films. There are two kinds, `tv` and `movie`, and this example sets both
|
||||||
|
of them and names every provider, so nothing is left to the default:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
metadata_providers:
|
||||||
|
tv: [tvdb, tmdb, simkl, imdb, omdb]
|
||||||
|
movie: [tmdb, imdb, omdb, simkl, tvdb]
|
||||||
|
```
|
||||||
|
|
||||||
|
Each list is independent: it both orders and filters that kind on its own, so a provider
|
||||||
|
you leave out of `tv` is still used for `movie`. Here TVDB is tried first for a series and
|
||||||
|
last for a film.
|
||||||
|
|
||||||
|
You do not need to give both keys, or a full list in each. A kind you leave out uses the
|
||||||
|
default order:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
metadata_providers:
|
||||||
|
tv: [tvdb, tmdb] # movies keep the default order
|
||||||
|
```
|
||||||
|
|
||||||
|
`tv` covers every episode and `movie` covers every film. unshackle only looks up
|
||||||
|
metadata for those two kinds, so any other key, such as `anime` or `music`, is ignored.
|
||||||
|
|
||||||
!!! note "`ipinfo_api_key` never touches your service sessions"
|
!!! note "`ipinfo_api_key` never touches your service sessions"
|
||||||
The token is only ever sent to `api.ipinfo.io` as a per-request `Authorization` header; it
|
The token is only ever sent to `api.ipinfo.io` as a per-request `Authorization` header; it
|
||||||
@@ -109,3 +158,4 @@ calls.
|
|||||||
| `curl_impersonate` | **Deprecated** → use [`network`](network.md) | Emits a `DeprecationWarning`; still honoured only if `network` is absent. |
|
| `curl_impersonate` | **Deprecated** → use [`network`](network.md) | Emits a `DeprecationWarning`; still honoured only if `network` is absent. |
|
||||||
| `downloader` | **Deprecated** | Any value other than `"requests"` emits a `DeprecationWarning`; the value is otherwise ignored (the unified requests downloader is always used). |
|
| `downloader` | **Deprecated** | Any value other than `"requests"` emits a `DeprecationWarning`; the value is otherwise ignored (the unified requests downloader is always used). |
|
||||||
| `scene_naming` | **Removed** | If set to any non-null value, unshackle exits with an error directing you to configure [`output_template`](output.md) instead. |
|
| `scene_naming` | **Removed** | If set to any non-null value, unshackle exits with an error directing you to configure [`output_template`](output.md) instead. |
|
||||||
|
| `imdb_api_enabled` | **Removed** | Ignored. It gated the old `imdbapi` provider, which the keyless [`imdb`](#external-api-keys) provider replaces. Safe to delete from your config. |
|
||||||
|
|||||||
@@ -0,0 +1,67 @@
|
|||||||
|
"""The `disable_metadata` config switch stops automatic lookups but keeps supplied IDs working."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Optional, Union
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from unshackle.core import providers
|
||||||
|
from unshackle.core.config import config
|
||||||
|
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult
|
||||||
|
|
||||||
|
|
||||||
|
class FakeProvider(MetadataProvider):
|
||||||
|
"""Stands in for a real provider; fails the test if it is searched."""
|
||||||
|
|
||||||
|
def __init__(self, name: str, result: Optional[MetadataResult] = None) -> None:
|
||||||
|
self.NAME = name
|
||||||
|
self._result = result
|
||||||
|
self.asked: list[Union[int, str]] = []
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
|
||||||
|
raise AssertionError("no provider may be searched while disable_metadata is set")
|
||||||
|
|
||||||
|
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
|
||||||
|
self.asked.append(provider_id)
|
||||||
|
return self._result
|
||||||
|
|
||||||
|
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
|
||||||
|
return ExternalIds()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _disabled(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
"""Turn metadata off and keep every test off the network."""
|
||||||
|
monkeypatch.setattr(config, "disable_metadata", True)
|
||||||
|
monkeypatch.setattr(config, "metadata_providers", [])
|
||||||
|
monkeypatch.setattr(providers, "fetch_external_ids", lambda *a, **kw: ExternalIds())
|
||||||
|
monkeypatch.setattr(providers, "enrich_ids", lambda result: None)
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_metadata_returns_none(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
monkeypatch.setattr(providers, "get_provider", lambda name: FakeProvider(name))
|
||||||
|
|
||||||
|
assert providers.search_metadata("Futurama", 1999, "tv") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_supplied_id_still_resolves(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
tmdb = FakeProvider("tmdb", MetadataResult(title="Inception", year=2010, kind="movie"))
|
||||||
|
monkeypatch.setattr(providers, "get_provider", lambda name: {"tmdb": tmdb}.get(name))
|
||||||
|
|
||||||
|
result = providers.resolve_by_ids(27205, kind="movie")
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result.title == "Inception"
|
||||||
|
assert result.external_ids.tmdb_id == 27205
|
||||||
|
assert tmdb.asked == [27205]
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_ids_returns_none(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
monkeypatch.setattr(providers, "get_provider", lambda name: FakeProvider(name))
|
||||||
|
|
||||||
|
assert providers.resolve_by_ids(title="Futurama", year=1999, kind="tv") is None
|
||||||
@@ -1,11 +1,11 @@
|
|||||||
"""--enrich original-language enrichment."""
|
"""Original-language parsing for the metadata a supplied ID resolves."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from unshackle.commands.dl import parse_language
|
from unshackle.commands.dl import parse_language
|
||||||
from unshackle.core.providers.imdbapi import primary_language as imdb_language
|
from unshackle.core.providers.imdb import primary_language as imdb_language
|
||||||
from unshackle.core.providers.omdb import primary_language as omdb_language
|
from unshackle.core.providers.omdb import primary_language as omdb_language
|
||||||
from unshackle.core.providers.tvdb import primary_language as tvdb_language
|
from unshackle.core.providers.tvdb import primary_language as tvdb_language
|
||||||
|
|
||||||
@@ -28,11 +28,27 @@ def test_parse_language(tag: str | None, expected: str | None) -> None:
|
|||||||
|
|
||||||
|
|
||||||
def test_imdb_language_takes_the_first_spoken_language() -> None:
|
def test_imdb_language_takes_the_first_spoken_language() -> None:
|
||||||
data = {"spokenLanguages": [{"code": "jpn", "name": "Japanese"}, {"code": "eng", "name": "English"}]}
|
data = {"spokenLanguages": {"spokenLanguages": [{"id": "ja", "text": "Japanese"}, {"id": "en", "text": "English"}]}}
|
||||||
assert imdb_language(data) == "jpn"
|
assert imdb_language(data) == "ja"
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("data", [{}, {"spokenLanguages": []}, {"spokenLanguages": None}])
|
def test_imdb_language_is_alpha_2() -> None:
|
||||||
|
"""IMDb's GraphQL language ids are alpha-2."""
|
||||||
|
data = {"spokenLanguages": {"spokenLanguages": [{"id": "ko", "text": "Korean"}]}}
|
||||||
|
assert parse_language(imdb_language(data)) is not None
|
||||||
|
assert str(parse_language(imdb_language(data))) == "ko"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"data",
|
||||||
|
[
|
||||||
|
{},
|
||||||
|
{"spokenLanguages": None}, # search hits can come back with the whole block null
|
||||||
|
{"spokenLanguages": {}},
|
||||||
|
{"spokenLanguages": {"spokenLanguages": []}},
|
||||||
|
{"spokenLanguages": {"spokenLanguages": None}},
|
||||||
|
],
|
||||||
|
)
|
||||||
def test_imdb_language_missing(data: dict) -> None:
|
def test_imdb_language_missing(data: dict) -> None:
|
||||||
assert imdb_language(data) is None
|
assert imdb_language(data) is None
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,183 @@
|
|||||||
|
"""IMDb GraphQL provider: persisted-query flow, field mapping, and search constraints."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from typing import Any, Optional
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from unshackle.core.providers.imdb import IMDBProvider, escape_graphql
|
||||||
|
|
||||||
|
|
||||||
|
class _Response:
|
||||||
|
def __init__(self, payload: dict) -> None:
|
||||||
|
self._payload = payload
|
||||||
|
|
||||||
|
def json(self) -> dict:
|
||||||
|
return self._payload
|
||||||
|
|
||||||
|
|
||||||
|
def _node(
|
||||||
|
imdb_id: str = "tt3581920",
|
||||||
|
text: str = "The Last of Us",
|
||||||
|
year: Optional[int] = 2023,
|
||||||
|
title_type: str = "tvSeries",
|
||||||
|
languages: Any = ({"id": "en", "text": "English"}, {"id": "id", "text": "Indonesian"}),
|
||||||
|
) -> dict:
|
||||||
|
return {
|
||||||
|
"id": imdb_id,
|
||||||
|
"titleText": {"text": text},
|
||||||
|
"originalTitleText": {"text": text},
|
||||||
|
"releaseYear": {"year": year} if year else None,
|
||||||
|
"titleType": {"id": title_type},
|
||||||
|
"spokenLanguages": {"spokenLanguages": list(languages)} if languages is not None else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeSession:
|
||||||
|
"""Records GET/POST calls and replays queued payloads."""
|
||||||
|
|
||||||
|
def __init__(self, responses: list[dict]) -> None:
|
||||||
|
self._responses = responses
|
||||||
|
self.gets: list[dict] = []
|
||||||
|
self.posts: list[dict] = []
|
||||||
|
|
||||||
|
def get(self, url: str, params: dict, headers: dict, timeout: int) -> _Response:
|
||||||
|
self.gets.append(params)
|
||||||
|
return _Response(self._responses.pop(0))
|
||||||
|
|
||||||
|
def post(self, url: str, json: dict, headers: dict, timeout: int) -> _Response: # noqa: A002
|
||||||
|
self.posts.append(json)
|
||||||
|
return _Response(self._responses.pop(0))
|
||||||
|
|
||||||
|
|
||||||
|
def _provider(monkeypatch: pytest.MonkeyPatch, responses: list[dict]) -> tuple[IMDBProvider, _FakeSession]:
|
||||||
|
p = IMDBProvider()
|
||||||
|
session = _FakeSession(responses)
|
||||||
|
monkeypatch.setattr(type(p), "session", property(lambda _self: session))
|
||||||
|
return p, session
|
||||||
|
|
||||||
|
|
||||||
|
NOT_FOUND = {"errors": [{"message": "PersistedQueryNotFound", "extensions": {"code": "PERSISTED_QUERY_NOT_FOUND"}}]}
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_available_without_any_key() -> None:
|
||||||
|
assert IMDBProvider().is_available() is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_cache_hit_uses_one_get(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
p, session = _provider(monkeypatch, [{"data": {"title": _node()}}])
|
||||||
|
result = p.get_by_id("tt3581920", "tv")
|
||||||
|
assert result is not None and result.title == "The Last of Us"
|
||||||
|
assert len(session.gets) == 1 and session.posts == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_persisted_query_miss_falls_back_to_one_post(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
p, session = _provider(monkeypatch, [NOT_FOUND, {"data": {"title": _node()}}])
|
||||||
|
result = p.get_by_id("tt3581920", "tv")
|
||||||
|
assert result is not None and result.external_ids.imdb_id == "tt3581920"
|
||||||
|
assert len(session.gets) == 1 and len(session.posts) == 1
|
||||||
|
# the POST registers the query by sending its text, keyed by the same hash the GET sent
|
||||||
|
assert "query UnshackleTitle" in session.posts[0]["query"]
|
||||||
|
sent_hash = json.loads(session.gets[0]["extensions"])["persistedQuery"]["sha256Hash"]
|
||||||
|
assert session.posts[0]["extensions"]["persistedQuery"]["sha256Hash"] == sent_hash
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_params_carry_no_spaces(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
"""IMDb's GraphQL gateway decodes "+" literally, so spaced JSON would come back as a 400."""
|
||||||
|
p, session = _provider(monkeypatch, [{"data": {"title": _node()}}])
|
||||||
|
p.get_by_id("tt3581920", "tv")
|
||||||
|
assert " " not in session.gets[0]["extensions"]
|
||||||
|
assert " " not in session.gets[0]["variables"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_by_id_maps_every_field(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
p, _ = _provider(monkeypatch, [{"data": {"title": _node()}}])
|
||||||
|
result = p.get_by_id("tt3581920", "tv")
|
||||||
|
assert result is not None
|
||||||
|
assert (result.title, result.year, result.kind) == ("The Last of Us", 2023, "tv")
|
||||||
|
assert result.original_language == "en"
|
||||||
|
assert result.source == "imdb"
|
||||||
|
assert result.external_ids.imdb_id == "tt3581920"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("title_type", ["movie", "tvSeries", "videoGame"])
|
||||||
|
def test_caller_kind_wins_over_the_response_title_type(monkeypatch: pytest.MonkeyPatch, title_type: str) -> None:
|
||||||
|
"""kind stays the caller's, so it cannot desync from the tmdb_kind resolve_by_ids seeds."""
|
||||||
|
p, _ = _provider(monkeypatch, [{"data": {"title": _node(title_type=title_type)}}])
|
||||||
|
result = p.get_by_id("tt1375666", "tv")
|
||||||
|
assert result is not None and result.kind == "tv"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("payload", [{"data": {"title": None}}, {"errors": [{"message": "boom"}]}, {"data": {}}])
|
||||||
|
def test_get_by_id_returns_none_on_no_answer(monkeypatch: pytest.MonkeyPatch, payload: dict) -> None:
|
||||||
|
p, _ = _provider(monkeypatch, [payload])
|
||||||
|
assert p.get_by_id("tt3581920", "tv") is None
|
||||||
|
|
||||||
|
|
||||||
|
def _edges(*nodes: dict) -> dict:
|
||||||
|
return {"data": {"advancedTitleSearch": {"edges": [{"node": {"title": n}} for n in nodes]}}}
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_uses_its_own_operation_and_one_get_on_a_cache_hit(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
p, session = _provider(monkeypatch, [_edges(_node())])
|
||||||
|
result = p.search("The Last of Us", 2023, "tv")
|
||||||
|
assert result is not None and result.external_ids.imdb_id == "tt3581920"
|
||||||
|
assert session.gets[0]["operationName"] == "UnshackleTitleSearch"
|
||||||
|
assert session.posts == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_query_text_carries_the_constraints(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
p, session = _provider(monkeypatch, [NOT_FOUND, _edges(_node())])
|
||||||
|
p.search("The Last of Us", 2023, "tv")
|
||||||
|
query = session.posts[0]["query"]
|
||||||
|
assert 'searchTerm: "The Last of Us"' in query
|
||||||
|
assert 'start: "2023-01-01", end: "2023-12-31"' in query
|
||||||
|
assert '["tvSeries", "tvMiniSeries"]' in query or '["tvSeries","tvMiniSeries"]' in query
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_omits_the_year_range_when_unknown(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
p, session = _provider(monkeypatch, [NOT_FOUND, _edges(_node(title_type="movie"))])
|
||||||
|
p.search("Inception", None, "movie")
|
||||||
|
query = session.posts[0]["query"]
|
||||||
|
assert "releaseDateConstraint" not in query
|
||||||
|
assert '["movie", "tvMovie"]' in query or '["movie","tvMovie"]' in query
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_picks_the_closest_title_not_the_first(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
noise = _node(imdb_id="tt27478132", text="Making of: The Last of Us")
|
||||||
|
exact = _node()
|
||||||
|
p, _ = _provider(monkeypatch, [_edges(noise, exact)])
|
||||||
|
result = p.search("The Last of Us", 2023, "tv")
|
||||||
|
assert result is not None and result.external_ids.imdb_id == "tt3581920"
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_rejects_a_title_that_is_not_a_close_match(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
p, _ = _provider(monkeypatch, [_edges(_node(imdb_id="tt999", text="Something Else Entirely"))])
|
||||||
|
assert p.search("The Last of Us", 2023, "tv") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_survives_a_null_language_block(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
p, _ = _provider(monkeypatch, [_edges(_node(languages=None))])
|
||||||
|
result = p.search("The Last of Us", 2023, "tv")
|
||||||
|
assert result is not None and result.original_language is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_returns_none_when_nothing_comes_back(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
p, _ = _provider(monkeypatch, [_edges()])
|
||||||
|
assert p.search("The Last of Us", 2023, "tv") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_escape_graphql_closes_the_string_injection() -> None:
|
||||||
|
assert escape_graphql('a"b\\c') == 'a\\"b\\\\c'
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_term_with_a_quote_stays_inside_the_string(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
p, session = _provider(monkeypatch, [NOT_FOUND, _edges(_node(text='The "Best" Show'))])
|
||||||
|
p.search('The "Best" Show', None, "tv")
|
||||||
|
assert 'searchTerm: "The \\"Best\\" Show"' in session.posts[0]["query"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_external_ids_echoes_the_id() -> None:
|
||||||
|
assert IMDBProvider().get_external_ids("tt1375666", "movie").imdb_id == "tt1375666"
|
||||||
@@ -0,0 +1,114 @@
|
|||||||
|
"""Input validation for the --tmdb / --imdb / --tvdb flags."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import click
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from unshackle.commands.dl import validate_metadata_ids
|
||||||
|
from unshackle.core.api.handlers import validate_download_parameters
|
||||||
|
from unshackle.core.config import config
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _default_order(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
monkeypatch.setattr(config, "metadata_providers", [])
|
||||||
|
|
||||||
|
|
||||||
|
def _with_keys(monkeypatch: pytest.MonkeyPatch, **keys: str) -> None:
|
||||||
|
for name in ("tmdb_api_key", "tvdb_api_key", "omdb_api_key", "simkl_client_id"):
|
||||||
|
monkeypatch.setattr(config, name, keys.get(name, ""))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- one ID at a time ----------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("tmdb", "imdb", "tvdb"),
|
||||||
|
[(27205, "tt1375666", None), (27205, None, 73871), (None, "tt1375666", 73871), (27205, "tt1375666", 73871)],
|
||||||
|
)
|
||||||
|
def test_two_ids_are_refused(tmdb: int | None, imdb: str | None, tvdb: int | None) -> None:
|
||||||
|
with pytest.raises(click.UsageError) as exc:
|
||||||
|
validate_metadata_ids(tmdb, imdb, tvdb)
|
||||||
|
assert "cannot be used together" in str(exc.value)
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_ids_is_fine() -> None:
|
||||||
|
assert validate_metadata_ids(None, None, None) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_imdb_alone_passes_on_the_keyless_provider(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
_with_keys(monkeypatch)
|
||||||
|
assert validate_metadata_ids(None, "tt1375666", None) is None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- the ID must have a provider that can resolve it ----------
|
||||||
|
|
||||||
|
|
||||||
|
def test_tmdb_without_its_key_names_the_key(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
_with_keys(monkeypatch)
|
||||||
|
with pytest.raises(click.UsageError) as exc:
|
||||||
|
validate_metadata_ids(27205, None, None)
|
||||||
|
assert "--tmdb" in str(exc.value) and "tmdb_api_key" in str(exc.value)
|
||||||
|
|
||||||
|
|
||||||
|
def test_tvdb_without_its_key_names_the_key(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
_with_keys(monkeypatch)
|
||||||
|
with pytest.raises(click.UsageError) as exc:
|
||||||
|
validate_metadata_ids(None, None, 73871)
|
||||||
|
assert "--tvdb" in str(exc.value) and "tvdb_api_key" in str(exc.value)
|
||||||
|
|
||||||
|
|
||||||
|
def test_tmdb_with_its_key_passes(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
_with_keys(monkeypatch, tmdb_api_key="k")
|
||||||
|
assert validate_metadata_ids(27205, None, None) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_provider_filtered_out_of_the_order_points_at_the_config(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
_with_keys(monkeypatch, tmdb_api_key="k")
|
||||||
|
monkeypatch.setattr(config, "metadata_providers", ["imdb", "omdb"])
|
||||||
|
with pytest.raises(click.UsageError) as exc:
|
||||||
|
validate_metadata_ids(27205, None, None)
|
||||||
|
assert "metadata_providers" in str(exc.value)
|
||||||
|
|
||||||
|
|
||||||
|
def test_imdb_is_unresolvable_when_the_order_drops_both_of_its_providers(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
_with_keys(monkeypatch)
|
||||||
|
monkeypatch.setattr(config, "metadata_providers", ["tmdb", "tvdb"])
|
||||||
|
with pytest.raises(click.UsageError) as exc:
|
||||||
|
validate_metadata_ids(None, "tt1375666", None)
|
||||||
|
assert "metadata_providers" in str(exc.value)
|
||||||
|
assert "imdb" in str(exc.value) and "omdb" in str(exc.value)
|
||||||
|
|
||||||
|
|
||||||
|
def test_one_kind_keeping_the_provider_is_enough(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
"""A per-kind order can drop tvdb for movies and keep it for tv, so this must not error."""
|
||||||
|
_with_keys(monkeypatch, tvdb_api_key="k")
|
||||||
|
monkeypatch.setattr(config, "metadata_providers", {"tv": ["tvdb"], "movie": ["tmdb"]})
|
||||||
|
assert validate_metadata_ids(None, None, 73871) is None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- the REST path rejects the same combinations ----------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"payload",
|
||||||
|
[
|
||||||
|
{"tmdb_id": 27205, "imdb_id": "tt1375666"},
|
||||||
|
{"tmdb_id": 27205, "tvdb_id": 73871},
|
||||||
|
{"imdb_id": "tt1375666", "tvdb_id": 73871},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_api_refuses_two_ids(payload: dict) -> None:
|
||||||
|
err = validate_download_parameters(payload)
|
||||||
|
assert err and "multiple external IDs" in err
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("payload", [{"tmdb_id": 27205}, {"imdb_id": "tt1375666"}, {"tvdb_id": 73871}, {}])
|
||||||
|
def test_api_accepts_one_id(payload: dict) -> None:
|
||||||
|
assert validate_download_parameters(payload) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_api_still_reports_a_malformed_id_first() -> None:
|
||||||
|
err = validate_download_parameters({"tmdb_id": "12345", "imdb_id": "tt1"})
|
||||||
|
assert err and "positive integer" in err
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
"""Precedence of user-supplied external IDs in providers.resolve_by_ids."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Optional, Union
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from unshackle.core import providers
|
||||||
|
from unshackle.core.config import config
|
||||||
|
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult
|
||||||
|
|
||||||
|
|
||||||
|
class FakeProvider(MetadataProvider):
|
||||||
|
"""Stands in for a real provider; records the ID it was asked for."""
|
||||||
|
|
||||||
|
def __init__(self, name: str, result: Optional[MetadataResult] = None, raises: bool = False) -> None:
|
||||||
|
self.NAME = name
|
||||||
|
self._result = result
|
||||||
|
self._raises = raises
|
||||||
|
self.asked: list[Union[int, str]] = []
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
|
||||||
|
raise AssertionError("resolve_by_ids must not search when an ID was supplied")
|
||||||
|
|
||||||
|
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
|
||||||
|
self.asked.append(provider_id)
|
||||||
|
if self._raises:
|
||||||
|
raise ValueError("provider is down")
|
||||||
|
return self._result
|
||||||
|
|
||||||
|
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
|
||||||
|
return ExternalIds()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _isolate(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
"""Keep every test off the network and on the default provider order."""
|
||||||
|
monkeypatch.setattr(config, "metadata_providers", [])
|
||||||
|
monkeypatch.setattr(providers, "fetch_external_ids", lambda *a, **kw: ExternalIds())
|
||||||
|
monkeypatch.setattr(providers, "enrich_ids", lambda result: None)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
providers,
|
||||||
|
"search_metadata",
|
||||||
|
lambda *a, **kw: pytest.fail("search_metadata must not run when an ID was supplied"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _patch_providers(monkeypatch: pytest.MonkeyPatch, available: dict[str, FakeProvider]) -> None:
|
||||||
|
monkeypatch.setattr(providers, "get_provider", lambda name: available.get(name))
|
||||||
|
|
||||||
|
|
||||||
|
def test_supplied_ids_survive_a_providers_own_answer(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
# tmdb answers with a different imdb id; the one the user gave must win
|
||||||
|
tmdb = FakeProvider(
|
||||||
|
"tmdb",
|
||||||
|
MetadataResult(title="Inception", year=2010, kind="movie", external_ids=ExternalIds(imdb_id="tt9999999")),
|
||||||
|
)
|
||||||
|
_patch_providers(monkeypatch, {"tmdb": tmdb})
|
||||||
|
|
||||||
|
result = providers.resolve_by_ids(27205, "tt1375666", kind="movie")
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result.external_ids.tmdb_id == 27205
|
||||||
|
assert result.external_ids.tmdb_kind == "movie"
|
||||||
|
assert result.external_ids.imdb_id == "tt1375666"
|
||||||
|
|
||||||
|
|
||||||
|
def test_imdb_falls_through_to_omdb(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
imdb = FakeProvider("imdb", raises=True)
|
||||||
|
omdb = FakeProvider("omdb", MetadataResult(title="Inception", year=2010, kind="movie"))
|
||||||
|
_patch_providers(monkeypatch, {"imdb": imdb, "omdb": omdb})
|
||||||
|
|
||||||
|
result = providers.resolve_by_ids(imdb_id="tt1375666", kind="movie")
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result.title == "Inception"
|
||||||
|
assert imdb.asked == ["tt1375666"]
|
||||||
|
assert omdb.asked == ["tt1375666"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_all_providers_down_synthesizes_a_result(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
_patch_providers(monkeypatch, {})
|
||||||
|
|
||||||
|
result = providers.resolve_by_ids(tvdb_id=73871, title="Futurama", year=1999, kind="tv")
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result.title == "Futurama"
|
||||||
|
assert result.year == 1999
|
||||||
|
assert result.external_ids.tvdb_id == 73871
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_ids_searches(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
calls: list[tuple] = []
|
||||||
|
monkeypatch.setattr(providers, "search_metadata", lambda *a, **kw: calls.append(a) or None)
|
||||||
|
_patch_providers(monkeypatch, {})
|
||||||
|
|
||||||
|
providers.resolve_by_ids(title="Futurama", year=1999, kind="tv")
|
||||||
|
|
||||||
|
assert calls and calls[0][:3] == ("Futurama", 1999, "tv")
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_ids_and_no_title_returns_none(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
_patch_providers(monkeypatch, {})
|
||||||
|
assert providers.resolve_by_ids(kind="tv") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_per_kind_order_puts_tvdb_first_for_tv(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
monkeypatch.setattr(config, "metadata_providers", {"tv": ["tvdb", "tmdb", "imdb", "omdb"]})
|
||||||
|
tvdb = FakeProvider("tvdb", MetadataResult(title="Futurama", year=1999, kind="tv"))
|
||||||
|
tmdb = FakeProvider("tmdb", MetadataResult(title="Wrong Show", year=2020, kind="tv"))
|
||||||
|
_patch_providers(monkeypatch, {"tvdb": tvdb, "tmdb": tmdb})
|
||||||
|
|
||||||
|
result = providers.resolve_by_ids(615, tvdb_id=73871, kind="tv")
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result.title == "Futurama"
|
||||||
|
assert tmdb.asked == []
|
||||||
|
# movies keep the default order, so tmdb is consulted before tvdb there
|
||||||
|
assert [cls.NAME for cls in providers.provider_order("movie")] == list(providers.DEFAULT_ORDER)
|
||||||
|
|
||||||
|
|
||||||
|
def test_flat_list_config_applies_to_both_kinds(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
monkeypatch.setattr(config, "metadata_providers", ["tvdb", "tmdb"])
|
||||||
|
assert [cls.NAME for cls in providers.provider_order("tv")] == ["tvdb", "tmdb"]
|
||||||
|
assert [cls.NAME for cls in providers.provider_order("movie")] == ["tvdb", "tmdb"]
|
||||||
|
assert [cls.NAME for cls in providers.provider_order()] == ["tvdb", "tmdb"]
|
||||||
+108
-56
@@ -134,6 +134,43 @@ def normalize_dl_config(dl_config: dict[str, Any]) -> dict[str, Any]:
|
|||||||
return {DL_OPTION_ALIASES.get(key, key): value for key, value in dl_config.items()}
|
return {DL_OPTION_ALIASES.get(key, key): value for key, value in dl_config.items()}
|
||||||
|
|
||||||
|
|
||||||
|
ID_PROVIDER_KEYS = {"tmdb": "tmdb_api_key", "tvdb": "tvdb_api_key", "omdb": "omdb_api_key"}
|
||||||
|
|
||||||
|
|
||||||
|
def validate_metadata_ids(tmdb_id: Optional[int], imdb_id: Optional[str], tvdb_id: Optional[int]) -> None:
|
||||||
|
"""Reject ID flags that conflict, or that no configured provider could ever resolve.
|
||||||
|
|
||||||
|
Only ever called with the IDs the user gave, never with the ones --animeapi back-fills.
|
||||||
|
"""
|
||||||
|
supplied = [
|
||||||
|
(flag, value) for flag, value in (("--tmdb", tmdb_id), ("--imdb", imdb_id), ("--tvdb", tvdb_id)) if value
|
||||||
|
]
|
||||||
|
if len(supplied) > 1:
|
||||||
|
raise click.UsageError(
|
||||||
|
f"{', '.join(flag for flag, _ in supplied)} cannot be used together. "
|
||||||
|
"Give one ID and unshackle resolves the others from it."
|
||||||
|
)
|
||||||
|
if not supplied:
|
||||||
|
return
|
||||||
|
|
||||||
|
flag, _ = supplied[0]
|
||||||
|
id_kind = flag.lstrip("-")
|
||||||
|
ordered = {cls.NAME: cls for kind in ("tv", "movie") for cls in providers.provider_order(kind)}
|
||||||
|
consumers = [cls for cls in ordered.values() if cls.ID_KIND == id_kind]
|
||||||
|
if any(cls().is_available() for cls in consumers):
|
||||||
|
return
|
||||||
|
|
||||||
|
if consumers:
|
||||||
|
keys = sorted({ID_PROVIDER_KEYS[cls.NAME] for cls in consumers if cls.NAME in ID_PROVIDER_KEYS})
|
||||||
|
raise click.UsageError(
|
||||||
|
f"{flag} needs the {consumers[0].NAME} provider. Set {' or '.join(keys)} in your config."
|
||||||
|
)
|
||||||
|
names = sorted({cls.NAME for cls in providers.REGISTRY.values() if cls.ID_KIND == id_kind})
|
||||||
|
raise click.UsageError(
|
||||||
|
f"{flag} needs the {' or '.join(names)} provider, which your metadata_providers config leaves out."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def group_videos_by_variant(videos: list[Video], *, merge: bool) -> list[list[Video]]:
|
def group_videos_by_variant(videos: list[Video], *, merge: bool) -> list[list[Video]]:
|
||||||
"""Group video tracks for muxing.
|
"""Group video tracks for muxing.
|
||||||
|
|
||||||
@@ -617,14 +654,16 @@ class dl:
|
|||||||
"tmdb_id",
|
"tmdb_id",
|
||||||
type=int,
|
type=int,
|
||||||
default=None,
|
default=None,
|
||||||
help="Use this TMDB ID for tagging instead of automatic lookup.",
|
help="Use this TMDB ID for tagging instead of an automatic search. Add --enrich to also take "
|
||||||
|
"its title, year and original language.",
|
||||||
)
|
)
|
||||||
@click.option(
|
@click.option(
|
||||||
"--tvdb",
|
"--tvdb",
|
||||||
"tvdb_id",
|
"tvdb_id",
|
||||||
type=int,
|
type=int,
|
||||||
default=None,
|
default=None,
|
||||||
help="Use this TVDB ID for --tvdb-order and --enrich instead of looking the series up automatically.",
|
help="Use this TVDB ID for --tvdb-order and tagging instead of looking the series up "
|
||||||
|
"automatically. Add --enrich to also take its title, year and original language.",
|
||||||
)
|
)
|
||||||
@click.option(
|
@click.option(
|
||||||
"--tvdb-order",
|
"--tvdb-order",
|
||||||
@@ -638,15 +677,16 @@ class dl:
|
|||||||
"animeapi_id",
|
"animeapi_id",
|
||||||
type=str,
|
type=str,
|
||||||
default=None,
|
default=None,
|
||||||
help="Anime database ID via AnimeAPI (e.g. mal:12345, anilist:98765). Defaults to MAL if no prefix.",
|
help="Anime database ID via AnimeAPI (e.g. mal:12345, anilist:98765). Defaults to MAL if no prefix. "
|
||||||
|
"Back-fills the TMDB, IMDB and TVDB IDs it knows for the show.",
|
||||||
)
|
)
|
||||||
@click.option(
|
@click.option(
|
||||||
"--enrich",
|
"--enrich",
|
||||||
is_flag=True,
|
is_flag=True,
|
||||||
default=False,
|
default=False,
|
||||||
help=(
|
help=(
|
||||||
"Override show title and year from external source, and fill in the original language "
|
"Overwrite the show title, year and original language with the external source's. "
|
||||||
"if the service did not provide one. Requires --tmdb, --imdb, --tvdb, or --animeapi."
|
"Requires --tmdb, --imdb, --tvdb, or --animeapi."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
@click.option(
|
@click.option(
|
||||||
@@ -654,7 +694,8 @@ class dl:
|
|||||||
"imdb_id",
|
"imdb_id",
|
||||||
type=str,
|
type=str,
|
||||||
default=None,
|
default=None,
|
||||||
help="Use this IMDB ID (e.g. tt1375666) for tagging instead of automatic lookup.",
|
help="Use this IMDB ID (e.g. tt1375666) for tagging instead of an automatic search. Add --enrich "
|
||||||
|
"to also take its title, year and original language.",
|
||||||
)
|
)
|
||||||
@click.option(
|
@click.option(
|
||||||
"--sub-format",
|
"--sub-format",
|
||||||
@@ -857,6 +898,7 @@ class dl:
|
|||||||
|
|
||||||
self.profile = profile
|
self.profile = profile
|
||||||
self.proxy_requested = bool(proxy)
|
self.proxy_requested = bool(proxy)
|
||||||
|
validate_metadata_ids(tmdb_id, imdb_id, tvdb_id)
|
||||||
self.tmdb_id = tmdb_id
|
self.tmdb_id = tmdb_id
|
||||||
self.imdb_id = imdb_id
|
self.imdb_id = imdb_id
|
||||||
self.tvdb_id = tvdb_id
|
self.tvdb_id = tvdb_id
|
||||||
@@ -1576,48 +1618,32 @@ class dl:
|
|||||||
if self.animeapi_title:
|
if self.animeapi_title:
|
||||||
enrich_title = self.animeapi_title
|
enrich_title = self.animeapi_title
|
||||||
|
|
||||||
if self.tmdb_id:
|
enrich_result = providers.resolve_by_ids(
|
||||||
|
self.tmdb_id,
|
||||||
|
self.imdb_id,
|
||||||
|
self.tvdb_id,
|
||||||
|
kind=kind,
|
||||||
|
title_cacher=title_cacher,
|
||||||
|
cache_title_id=cache_title_id,
|
||||||
|
cache_region=cache_region,
|
||||||
|
cache_account_hash=cache_account_hash,
|
||||||
|
)
|
||||||
|
if enrich_result:
|
||||||
if not enrich_title:
|
if not enrich_title:
|
||||||
enrich_title = providers.get_title_by_id(
|
enrich_title = enrich_result.title
|
||||||
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
enrich_year = enrich_result.year
|
||||||
)
|
enrich_lang = parse_language(enrich_result.original_language)
|
||||||
enrich_year = providers.get_year_by_id(
|
|
||||||
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
|
||||||
)
|
|
||||||
enrich_lang = parse_language(
|
|
||||||
providers.get_language_by_id(
|
|
||||||
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
|
||||||
)
|
|
||||||
)
|
|
||||||
elif self.imdb_id:
|
|
||||||
for name in ("imdbapi", "omdb"):
|
|
||||||
provider = providers.get_provider(name)
|
|
||||||
imdb_result = provider.get_by_id(self.imdb_id, kind) if provider else None
|
|
||||||
if imdb_result:
|
|
||||||
if not enrich_title:
|
|
||||||
enrich_title = imdb_result.title
|
|
||||||
enrich_year = imdb_result.year
|
|
||||||
enrich_lang = parse_language(imdb_result.original_language)
|
|
||||||
break
|
|
||||||
elif self.tvdb_id:
|
|
||||||
provider = providers.get_provider("tvdb")
|
|
||||||
tvdb_result = provider.get_by_id(self.tvdb_id, kind) if provider else None
|
|
||||||
if tvdb_result:
|
|
||||||
if not enrich_title:
|
|
||||||
enrich_title = tvdb_result.title
|
|
||||||
enrich_year = tvdb_result.year
|
|
||||||
enrich_lang = parse_language(tvdb_result.original_language)
|
|
||||||
|
|
||||||
if enrich_lang:
|
if enrich_lang:
|
||||||
self.log.info(f"Original language from metadata: {enrich_lang}")
|
self.log.info(f"Original language from metadata: {enrich_lang}")
|
||||||
if not (enrich_title or enrich_year or enrich_lang):
|
if not (enrich_title or enrich_year or enrich_lang):
|
||||||
self.log.warning("--enrich found no metadata; is the provider's API key configured?")
|
self.log.warning("--enrich found no metadata. Is the provider's API key configured?")
|
||||||
elif missing := [
|
elif missing := [
|
||||||
name
|
name
|
||||||
for name, value in (("title", enrich_title), ("year", enrich_year), ("language", enrich_lang))
|
for name, value in (("title", enrich_title), ("year", enrich_year), ("language", enrich_lang))
|
||||||
if not value
|
if not value
|
||||||
]:
|
]:
|
||||||
self.log.warning(f"--enrich source did not provide {', '.join(missing)}; those fields are unchanged.")
|
self.log.warning(f"--enrich source gave no {', '.join(missing)}. Those fields are unchanged.")
|
||||||
|
|
||||||
if enrich_title or enrich_year or enrich_lang:
|
if enrich_title or enrich_year or enrich_lang:
|
||||||
for t in titles if isinstance(titles, (Series, Movies)) else [titles]:
|
for t in titles if isinstance(titles, (Series, Movies)) else [titles]:
|
||||||
@@ -1626,9 +1652,9 @@ class dl:
|
|||||||
t.title = enrich_title
|
t.title = enrich_title
|
||||||
else:
|
else:
|
||||||
t.name = enrich_title
|
t.name = enrich_title
|
||||||
if enrich_year and not t.year:
|
if enrich_year:
|
||||||
t.year = enrich_year
|
t.year = enrich_year
|
||||||
if enrich_lang and not t.language:
|
if enrich_lang:
|
||||||
t.language = enrich_lang
|
t.language = enrich_lang
|
||||||
|
|
||||||
if self.tvdb_order and isinstance(titles, Series):
|
if self.tvdb_order and isinstance(titles, Series):
|
||||||
@@ -2332,20 +2358,28 @@ class dl:
|
|||||||
if isinstance(title, Episode) and not self.tmdb_searched:
|
if isinstance(title, Episode) and not self.tmdb_searched:
|
||||||
kind = "tv"
|
kind = "tv"
|
||||||
tmdb_title: Optional[str] = None
|
tmdb_title: Optional[str] = None
|
||||||
if self.tmdb_id:
|
has_ids = bool(self.tmdb_id or self.imdb_id or self.tvdb_id)
|
||||||
tmdb_title = providers.get_title_by_id(
|
result = providers.resolve_by_ids(
|
||||||
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
self.tmdb_id,
|
||||||
)
|
self.imdb_id,
|
||||||
else:
|
self.tvdb_id,
|
||||||
result = providers.search_metadata(
|
title=title.title,
|
||||||
title.title, title.year, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
year=title.year,
|
||||||
)
|
kind=kind,
|
||||||
if result and result.title and providers.fuzzy_match(result.title, title.title):
|
title_cacher=title_cacher,
|
||||||
|
cache_title_id=cache_title_id,
|
||||||
|
cache_region=cache_region,
|
||||||
|
cache_account_hash=cache_account_hash,
|
||||||
|
)
|
||||||
|
if result:
|
||||||
|
if has_ids:
|
||||||
|
tmdb_title = result.title
|
||||||
|
if not self.tmdb_id:
|
||||||
|
self.tmdb_id = result.external_ids.tmdb_id
|
||||||
|
elif result.title and providers.fuzzy_match(result.title, title.title):
|
||||||
self.tmdb_id = result.external_ids.tmdb_id
|
self.tmdb_id = result.external_ids.tmdb_id
|
||||||
tmdb_title = result.title
|
tmdb_title = result.title
|
||||||
self.search_source = result.source
|
self.search_source = result.source
|
||||||
else:
|
|
||||||
self.tmdb_id = None
|
|
||||||
if list_ or list_titles:
|
if list_ or list_titles:
|
||||||
if self.tmdb_id:
|
if self.tmdb_id:
|
||||||
console.print(
|
console.print(
|
||||||
@@ -2359,8 +2393,17 @@ class dl:
|
|||||||
self.tmdb_searched = True
|
self.tmdb_searched = True
|
||||||
|
|
||||||
if isinstance(title, Movie) and (list_ or list_titles) and not self.tmdb_id:
|
if isinstance(title, Movie) and (list_ or list_titles) and not self.tmdb_id:
|
||||||
movie_result = providers.search_metadata(
|
movie_result = providers.resolve_by_ids(
|
||||||
title.name, title.year, "movie", title_cacher, cache_title_id, cache_region, cache_account_hash
|
None,
|
||||||
|
self.imdb_id,
|
||||||
|
self.tvdb_id,
|
||||||
|
title=title.name,
|
||||||
|
year=title.year,
|
||||||
|
kind="movie",
|
||||||
|
title_cacher=title_cacher,
|
||||||
|
cache_title_id=cache_title_id,
|
||||||
|
cache_region=cache_region,
|
||||||
|
cache_account_hash=cache_account_hash,
|
||||||
)
|
)
|
||||||
if movie_result and movie_result.external_ids.tmdb_id:
|
if movie_result and movie_result.external_ids.tmdb_id:
|
||||||
console.print(
|
console.print(
|
||||||
@@ -2373,7 +2416,7 @@ class dl:
|
|||||||
else:
|
else:
|
||||||
console.print(Padding("Search -> [bright_black]No match found[/]", (0, 5)))
|
console.print(Padding("Search -> [bright_black]No match found[/]", (0, 5)))
|
||||||
|
|
||||||
if self.tmdb_id and getattr(self, "search_source", None) not in ("simkl", "imdbapi"):
|
if self.tmdb_id and getattr(self, "search_source", None) not in ("simkl", "imdb"):
|
||||||
kind = "tv" if isinstance(title, Episode) else "movie"
|
kind = "tv" if isinstance(title, Episode) else "movie"
|
||||||
providers.fetch_external_ids(
|
providers.fetch_external_ids(
|
||||||
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
self.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash
|
||||||
@@ -3869,8 +3912,17 @@ class dl:
|
|||||||
tvdb_id = self.tvdb_id
|
tvdb_id = self.tvdb_id
|
||||||
if not tvdb_id:
|
if not tvdb_id:
|
||||||
show = titles[0]
|
show = titles[0]
|
||||||
result = providers.search_metadata(
|
result = providers.resolve_by_ids(
|
||||||
show.title, show.year, "tv", title_cacher, cache_title_id, cache_region, cache_account_hash
|
self.tmdb_id,
|
||||||
|
self.imdb_id,
|
||||||
|
None,
|
||||||
|
title=show.title,
|
||||||
|
year=show.year,
|
||||||
|
kind="tv",
|
||||||
|
title_cacher=title_cacher,
|
||||||
|
cache_title_id=cache_title_id,
|
||||||
|
cache_region=cache_region,
|
||||||
|
cache_account_hash=cache_account_hash,
|
||||||
)
|
)
|
||||||
tvdb_id = result.external_ids.tvdb_id if result else None
|
tvdb_id = result.external_ids.tvdb_id if result else None
|
||||||
if not tvdb_id:
|
if not tvdb_id:
|
||||||
|
|||||||
@@ -926,6 +926,23 @@ def validate_download_parameters(data: Dict[str, Any]) -> Optional[str]:
|
|||||||
if not isinstance(data["downloads"], int) or data["downloads"] <= 0:
|
if not isinstance(data["downloads"], int) or data["downloads"] <= 0:
|
||||||
return "downloads must be a positive integer"
|
return "downloads must be a positive integer"
|
||||||
|
|
||||||
|
for name in ("tmdb_id", "tvdb_id"):
|
||||||
|
if data.get(name) is not None:
|
||||||
|
value = data[name]
|
||||||
|
if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
|
||||||
|
return f"{name} must be a positive integer"
|
||||||
|
|
||||||
|
if data.get("imdb_id") is not None:
|
||||||
|
if not isinstance(data["imdb_id"], str) or not re.fullmatch(r"tt\d+", data["imdb_id"]):
|
||||||
|
return "imdb_id must be an IMDB ID like 'tt1375666'"
|
||||||
|
|
||||||
|
supplied_ids = [name for name in ("tmdb_id", "imdb_id", "tvdb_id") if data.get(name)]
|
||||||
|
if len(supplied_ids) > 1:
|
||||||
|
return (
|
||||||
|
f"Cannot use multiple external IDs: {', '.join(supplied_ids)}. "
|
||||||
|
"Give one ID and unshackle resolves the others from it."
|
||||||
|
)
|
||||||
|
|
||||||
exclusive_flags = []
|
exclusive_flags = []
|
||||||
if data.get("video_only"):
|
if data.get("video_only"):
|
||||||
exclusive_flags.append("video_only")
|
exclusive_flags.append("video_only")
|
||||||
|
|||||||
@@ -758,13 +758,13 @@ async def download(request: web.Request) -> web.Response:
|
|||||||
description: Set the group tag to be used (default - None)
|
description: Set the group tag to be used (default - None)
|
||||||
tmdb_id:
|
tmdb_id:
|
||||||
type: integer
|
type: integer
|
||||||
description: Use this TMDB ID for tagging (default - None)
|
description: Use this TMDB ID for tagging instead of a title search. Set enrich to also take its title, year and original language. Mutually exclusive with imdb_id and tvdb_id. Needs tmdb_api_key (default - None)
|
||||||
animeapi_id:
|
animeapi_id:
|
||||||
type: string
|
type: string
|
||||||
description: Anime database ID via AnimeAPI, e.g. mal:12345 (default - None)
|
description: Anime database ID via AnimeAPI, e.g. mal:12345 (default - None)
|
||||||
enrich:
|
enrich:
|
||||||
type: boolean
|
type: boolean
|
||||||
description: Override show title and year from external source, and fill in the original language. Requires one of tmdb_id, imdb_id, tvdb_id or animeapi_id (default - false)
|
description: Overwrite show title, year and original language with the external source's. Requires one of tmdb_id, imdb_id, tvdb_id or animeapi_id (default - false)
|
||||||
no_folder:
|
no_folder:
|
||||||
type: boolean
|
type: boolean
|
||||||
description: Disable folder creation for TV shows (default - false)
|
description: Disable folder creation for TV shows (default - false)
|
||||||
@@ -791,10 +791,10 @@ async def download(request: web.Request) -> web.Response:
|
|||||||
description: Add REPACK tag to the output filename (default - false)
|
description: Add REPACK tag to the output filename (default - false)
|
||||||
imdb_id:
|
imdb_id:
|
||||||
type: string
|
type: string
|
||||||
description: Use this IMDB ID (e.g. tt1375666) for tagging (default - None)
|
description: Use this IMDB ID (e.g. tt1375666) for tagging instead of a title search. Set enrich to also take its title, year and original language. Mutually exclusive with tmdb_id and tvdb_id (default - None)
|
||||||
tvdb_id:
|
tvdb_id:
|
||||||
type: integer
|
type: integer
|
||||||
description: Use this TVDB ID for tagging, episode ordering and enrichment (default - None)
|
description: Use this TVDB ID for tagging and episode ordering instead of a series lookup. Set enrich to also take its title, year and original language. Mutually exclusive with tmdb_id and imdb_id. Needs tvdb_api_key (default - None)
|
||||||
tvdb_order:
|
tvdb_order:
|
||||||
type: string
|
type: string
|
||||||
enum: [official, dvd, absolute, alternate, regional]
|
enum: [official, dvd, absolute, alternate, regional]
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ from __future__ import annotations
|
|||||||
import re
|
import re
|
||||||
import warnings
|
import warnings
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Optional
|
from typing import Any, Optional, Union
|
||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
from appdirs import AppDirs
|
from appdirs import AppDirs
|
||||||
@@ -114,13 +114,13 @@ class Config:
|
|||||||
self.tag: str = kwargs.get("tag") or ""
|
self.tag: str = kwargs.get("tag") or ""
|
||||||
self.tag_group_name: bool = kwargs.get("tag_group_name", True)
|
self.tag_group_name: bool = kwargs.get("tag_group_name", True)
|
||||||
self.tag_imdb_tmdb: bool = kwargs.get("tag_imdb_tmdb", True)
|
self.tag_imdb_tmdb: bool = kwargs.get("tag_imdb_tmdb", True)
|
||||||
self.imdb_api_enabled: bool = kwargs.get("imdb_api_enabled", False)
|
|
||||||
self.omdb_api_key: str = kwargs.get("omdb_api_key") or ""
|
self.omdb_api_key: str = kwargs.get("omdb_api_key") or ""
|
||||||
self.tmdb_api_key: str = kwargs.get("tmdb_api_key") or ""
|
self.tmdb_api_key: str = kwargs.get("tmdb_api_key") or ""
|
||||||
self.tvdb_api_key: str = kwargs.get("tvdb_api_key") or ""
|
self.tvdb_api_key: str = kwargs.get("tvdb_api_key") or ""
|
||||||
self.tvdb_pin: str = kwargs.get("tvdb_pin") or ""
|
self.tvdb_pin: str = kwargs.get("tvdb_pin") or ""
|
||||||
self.tvdb_order: str = (kwargs.get("tvdb_order") or "").lower()
|
self.tvdb_order: str = (kwargs.get("tvdb_order") or "").lower()
|
||||||
self.metadata_providers: list = kwargs.get("metadata_providers") or []
|
self.metadata_providers: Union[list, dict] = kwargs.get("metadata_providers") or []
|
||||||
|
self.disable_metadata: bool = kwargs.get("disable_metadata", False)
|
||||||
self.simkl_client_id: str = kwargs.get("simkl_client_id") or ""
|
self.simkl_client_id: str = kwargs.get("simkl_client_id") or ""
|
||||||
self.decrypt_labs_api_key: str = kwargs.get("decrypt_labs_api_key") or ""
|
self.decrypt_labs_api_key: str = kwargs.get("decrypt_labs_api_key") or ""
|
||||||
self.ipinfo_api_key: str = kwargs.get("ipinfo_api_key") or ""
|
self.ipinfo_api_key: str = kwargs.get("ipinfo_api_key") or ""
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from typing import TYPE_CHECKING, Optional
|
from typing import TYPE_CHECKING, Optional, Union
|
||||||
|
|
||||||
import requests
|
import requests
|
||||||
|
|
||||||
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, fuzzy_match, log
|
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, fuzzy_match, log
|
||||||
from unshackle.core.providers.imdbapi import IMDBApiProvider
|
from unshackle.core.providers.imdb import IMDBProvider
|
||||||
from unshackle.core.providers.omdb import OMDBProvider
|
from unshackle.core.providers.omdb import OMDBProvider
|
||||||
from unshackle.core.providers.simkl import SimklProvider
|
from unshackle.core.providers.simkl import SimklProvider
|
||||||
from unshackle.core.providers.tmdb import TMDBProvider
|
from unshackle.core.providers.tmdb import TMDBProvider
|
||||||
@@ -15,18 +15,28 @@ if TYPE_CHECKING:
|
|||||||
from unshackle.core.title_cacher import TitleCacher
|
from unshackle.core.title_cacher import TitleCacher
|
||||||
|
|
||||||
REGISTRY: dict[str, type[MetadataProvider]] = {
|
REGISTRY: dict[str, type[MetadataProvider]] = {
|
||||||
cls.NAME: cls for cls in (IMDBApiProvider, OMDBProvider, SimklProvider, TMDBProvider, TVDBProvider)
|
cls.NAME: cls for cls in (IMDBProvider, OMDBProvider, SimklProvider, TMDBProvider, TVDBProvider)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# legacy `metadata_providers` names, still accepted
|
||||||
|
ALIASES: dict[str, str] = {"imdbapi": "imdb"}
|
||||||
|
|
||||||
# used when `metadata_providers` is unset
|
# used when `metadata_providers` is unset
|
||||||
DEFAULT_ORDER: tuple[str, ...] = ("imdbapi", "omdb", "simkl", "tmdb", "tvdb")
|
DEFAULT_ORDER: tuple[str, ...] = ("imdb", "omdb", "simkl", "tmdb", "tvdb")
|
||||||
|
|
||||||
|
|
||||||
def provider_order() -> list[type[MetadataProvider]]:
|
def provider_order(kind: Optional[str] = None) -> list[type[MetadataProvider]]:
|
||||||
"""Provider classes in the order configured by `metadata_providers`."""
|
"""Provider classes in the order configured by `metadata_providers`.
|
||||||
|
|
||||||
|
`metadata_providers` is either a flat list applying to both kinds, or a mapping of
|
||||||
|
kind ("tv"/"movie") to its own list. A kind the mapping omits uses `DEFAULT_ORDER`.
|
||||||
|
"""
|
||||||
from unshackle.core.config import config
|
from unshackle.core.config import config
|
||||||
|
|
||||||
names = [str(n).lower() for n in (config.metadata_providers or DEFAULT_ORDER)]
|
configured = config.metadata_providers
|
||||||
|
selected = (configured.get(kind) if kind else None) if isinstance(configured, dict) else configured
|
||||||
|
|
||||||
|
names = [ALIASES.get(str(n).lower(), str(n).lower()) for n in (selected or DEFAULT_ORDER)]
|
||||||
unknown = [n for n in names if n not in REGISTRY]
|
unknown = [n for n in names if n not in REGISTRY]
|
||||||
if unknown:
|
if unknown:
|
||||||
log.warning("Ignoring unknown metadata_providers entries: %s", ", ".join(unknown))
|
log.warning("Ignoring unknown metadata_providers entries: %s", ", ".join(unknown))
|
||||||
@@ -60,7 +70,15 @@ def search_metadata(
|
|||||||
cache_account_hash: Optional[str] = None,
|
cache_account_hash: Optional[str] = None,
|
||||||
) -> Optional[MetadataResult]:
|
) -> Optional[MetadataResult]:
|
||||||
"""Search all available providers for metadata. Returns best match."""
|
"""Search all available providers for metadata. Returns best match."""
|
||||||
ordered = provider_order()
|
from unshackle.core.config import config
|
||||||
|
|
||||||
|
# the one gate for `disable_metadata`: every automatic lookup reaches a provider through
|
||||||
|
# here, while a user-supplied ID goes straight to `resolve_by_ids` and stays allowed
|
||||||
|
if config.disable_metadata:
|
||||||
|
log.debug("Metadata lookups are disabled by config; not searching for %r", title)
|
||||||
|
return None
|
||||||
|
|
||||||
|
ordered = provider_order(kind)
|
||||||
|
|
||||||
# Check cache first
|
# Check cache first
|
||||||
if title_cacher and cache_title_id:
|
if title_cacher and cache_title_id:
|
||||||
@@ -268,6 +286,78 @@ def fetch_external_ids(
|
|||||||
return ext
|
return ext
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_by_ids(
|
||||||
|
tmdb_id: Optional[int] = None,
|
||||||
|
imdb_id: Optional[str] = None,
|
||||||
|
tvdb_id: Optional[int] = None,
|
||||||
|
*,
|
||||||
|
title: Optional[str] = None,
|
||||||
|
year: Optional[int] = None,
|
||||||
|
kind: str = "movie",
|
||||||
|
title_cacher: Optional[TitleCacher] = None,
|
||||||
|
cache_title_id: Optional[str] = None,
|
||||||
|
cache_region: Optional[str] = None,
|
||||||
|
cache_account_hash: Optional[str] = None,
|
||||||
|
) -> Optional[MetadataResult]:
|
||||||
|
"""Resolve metadata from user-supplied external IDs, falling back to search only without them.
|
||||||
|
|
||||||
|
A supplied ID is authoritative. It is looked up directly through the providers that
|
||||||
|
consume its namespace, in `metadata_providers` order for `kind`, and it always survives
|
||||||
|
into `external_ids` whatever a provider answers. A fuzzy title search only runs when no
|
||||||
|
ID was supplied at all.
|
||||||
|
"""
|
||||||
|
supplied: dict[str, Union[int, str]] = {}
|
||||||
|
if tmdb_id is not None:
|
||||||
|
supplied["tmdb"] = tmdb_id
|
||||||
|
if imdb_id:
|
||||||
|
supplied["imdb"] = imdb_id
|
||||||
|
if tvdb_id is not None:
|
||||||
|
supplied["tvdb"] = tvdb_id
|
||||||
|
|
||||||
|
if not supplied:
|
||||||
|
if not title:
|
||||||
|
return None
|
||||||
|
return search_metadata(title, year, kind, title_cacher, cache_title_id, cache_region, cache_account_hash)
|
||||||
|
|
||||||
|
result: Optional[MetadataResult] = None
|
||||||
|
for cls in provider_order(kind):
|
||||||
|
provider_id = supplied.get(cls.ID_KIND or "")
|
||||||
|
if provider_id is None:
|
||||||
|
continue
|
||||||
|
p = get_provider(cls.NAME)
|
||||||
|
if not p:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
result = p.get_by_id(provider_id, kind)
|
||||||
|
except Exception as exc:
|
||||||
|
log.debug("%s lookup of %s failed: %s", cls.NAME, provider_id, exc)
|
||||||
|
continue
|
||||||
|
if result:
|
||||||
|
break
|
||||||
|
|
||||||
|
if not result:
|
||||||
|
result = MetadataResult(title=title, year=year, kind=kind)
|
||||||
|
|
||||||
|
ids = result.external_ids
|
||||||
|
if tmdb_id is not None:
|
||||||
|
ids.tmdb_id = tmdb_id
|
||||||
|
ids.tmdb_kind = kind
|
||||||
|
if imdb_id:
|
||||||
|
ids.imdb_id = imdb_id
|
||||||
|
if tvdb_id is not None:
|
||||||
|
ids.tvdb_id = tvdb_id
|
||||||
|
|
||||||
|
if ids.tmdb_id and not (ids.imdb_id and ids.tvdb_id):
|
||||||
|
ext = fetch_external_ids(ids.tmdb_id, kind, title_cacher, cache_title_id, cache_region, cache_account_hash)
|
||||||
|
if ext.imdb_id and not ids.imdb_id:
|
||||||
|
ids.imdb_id = ext.imdb_id
|
||||||
|
if ext.tvdb_id and not ids.tvdb_id:
|
||||||
|
ids.tvdb_id = ext.tvdb_id
|
||||||
|
enrich_ids(result)
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
# -- Internal helpers --
|
# -- Internal helpers --
|
||||||
|
|
||||||
|
|
||||||
@@ -276,9 +366,9 @@ def fetch_external_ids(
|
|||||||
_ENRICHMENT_AUTHORITY: tuple[str, ...] = ("tmdb", "simkl", "tvdb")
|
_ENRICHMENT_AUTHORITY: tuple[str, ...] = ("tmdb", "simkl", "tvdb")
|
||||||
|
|
||||||
|
|
||||||
def _enrichment_providers() -> list[str]:
|
def _enrichment_providers(kind: Optional[str] = None) -> list[str]:
|
||||||
"""Names of configured providers that can resolve an IMDB ID, most trusted first."""
|
"""Names of configured providers that can resolve an IMDB ID, most trusted first."""
|
||||||
configured = {cls.NAME for cls in provider_order() if hasattr(cls, "find_by_imdb_id")}
|
configured = {cls.NAME for cls in provider_order(kind) if hasattr(cls, "find_by_imdb_id")}
|
||||||
return [name for name in _ENRICHMENT_AUTHORITY if name in configured]
|
return [name for name in _ENRICHMENT_AUTHORITY if name in configured]
|
||||||
|
|
||||||
|
|
||||||
@@ -298,7 +388,7 @@ def enrich_ids(result: MetadataResult) -> None:
|
|||||||
kind = result.kind or "movie"
|
kind = result.kind or "movie"
|
||||||
|
|
||||||
# Step 1: Collect enrichment results from all available providers
|
# Step 1: Collect enrichment results from all available providers
|
||||||
authority = {name: i for i, name in enumerate(_enrichment_providers())}
|
authority = {name: i for i, name in enumerate(_enrichment_providers(kind))}
|
||||||
enrichments: list[tuple[str, ExternalIds]] = []
|
enrichments: list[tuple[str, ExternalIds]] = []
|
||||||
for provider_name in authority:
|
for provider_name in authority:
|
||||||
p = get_provider(provider_name)
|
p = get_provider(provider_name)
|
||||||
@@ -477,9 +567,11 @@ def _cached_to_result(cached: dict, provider_name: str, kind: str) -> Optional[M
|
|||||||
source="tvdb",
|
source="tvdb",
|
||||||
raw=cached,
|
raw=cached,
|
||||||
)
|
)
|
||||||
elif provider_name == "imdbapi":
|
elif provider_name == "imdb":
|
||||||
title = cached.get("primaryTitle") or cached.get("originalTitle")
|
from unshackle.core.providers.imdb import primary_language
|
||||||
year = cached.get("startYear")
|
|
||||||
|
title = (cached.get("titleText") or {}).get("text") or (cached.get("originalTitleText") or {}).get("text")
|
||||||
|
year = (cached.get("releaseYear") or {}).get("year")
|
||||||
imdb_id = cached.get("id")
|
imdb_id = cached.get("id")
|
||||||
# Restore enriched IDs that were saved alongside the raw data
|
# Restore enriched IDs that were saved alongside the raw data
|
||||||
enriched = cached.get("_enriched_ids", {})
|
enriched = cached.get("_enriched_ids", {})
|
||||||
@@ -493,7 +585,8 @@ def _cached_to_result(cached: dict, provider_name: str, kind: str) -> Optional[M
|
|||||||
tmdb_kind=enriched.get("tmdb_kind"),
|
tmdb_kind=enriched.get("tmdb_kind"),
|
||||||
tvdb_id=enriched.get("tvdb_id"),
|
tvdb_id=enriched.get("tvdb_id"),
|
||||||
),
|
),
|
||||||
source="imdbapi",
|
original_language=primary_language(cached),
|
||||||
|
source="imdb",
|
||||||
raw=cached,
|
raw=cached,
|
||||||
)
|
)
|
||||||
return None
|
return None
|
||||||
@@ -513,5 +606,6 @@ __all__ = [
|
|||||||
"get_title_by_id",
|
"get_title_by_id",
|
||||||
"get_year_by_id",
|
"get_year_by_id",
|
||||||
"provider_order",
|
"provider_order",
|
||||||
|
"resolve_by_ids",
|
||||||
"search_metadata",
|
"search_metadata",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -37,7 +37,7 @@ class MetadataResult:
|
|||||||
kind: Optional[str] = None # "movie" or "tv"
|
kind: Optional[str] = None # "movie" or "tv"
|
||||||
external_ids: ExternalIds = field(default_factory=ExternalIds)
|
external_ids: ExternalIds = field(default_factory=ExternalIds)
|
||||||
original_language: Optional[str] = None # alpha-2 or alpha-3, whichever the provider speaks
|
original_language: Optional[str] = None # alpha-2 or alpha-3, whichever the provider speaks
|
||||||
source: str = "" # provider name, e.g. "tmdb", "simkl", "imdbapi"
|
source: str = "" # provider name, e.g. "tmdb", "simkl", "imdb"
|
||||||
raw: Optional[dict] = None # original API response for caching
|
raw: Optional[dict] = None # original API response for caching
|
||||||
|
|
||||||
|
|
||||||
@@ -46,6 +46,7 @@ class MetadataProvider(metaclass=ABCMeta):
|
|||||||
|
|
||||||
NAME: str = ""
|
NAME: str = ""
|
||||||
REQUIRES_KEY: bool = True
|
REQUIRES_KEY: bool = True
|
||||||
|
ID_KIND: Optional[str] = None
|
||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
self.log = logging.getLogger(f"METADATA.{self.NAME.upper()}")
|
self.log = logging.getLogger(f"METADATA.{self.NAME.upper()}")
|
||||||
|
|||||||
@@ -0,0 +1,156 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from difflib import SequenceMatcher
|
||||||
|
from typing import Optional, Union
|
||||||
|
|
||||||
|
import requests
|
||||||
|
|
||||||
|
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, _clean, fuzzy_match
|
||||||
|
|
||||||
|
GRAPHQL_URL = "https://caching.graphql.imdb.com/"
|
||||||
|
|
||||||
|
GRAPHQL_HEADERS = {
|
||||||
|
"Accept": "application/graphql+json, application/json",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Origin": "https://www.imdb.com",
|
||||||
|
"X-Imdb-Client-Name": "imdb-web-next-localized",
|
||||||
|
"X-Imdb-User-Country": "US",
|
||||||
|
}
|
||||||
|
|
||||||
|
KIND_TO_TYPES: dict[str, list[str]] = {
|
||||||
|
"movie": ["movie", "tvMovie"],
|
||||||
|
"tv": ["tvSeries", "tvMiniSeries"],
|
||||||
|
}
|
||||||
|
|
||||||
|
TITLE_FIELDS = (
|
||||||
|
"id titleText { text } originalTitleText { text } releaseYear { year } "
|
||||||
|
"spokenLanguages { spokenLanguages { id text } }"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def escape_graphql(value: str) -> str:
|
||||||
|
return value.replace("\\", "\\\\").replace('"', '\\"')
|
||||||
|
|
||||||
|
|
||||||
|
def primary_language(data: dict) -> Optional[str]:
|
||||||
|
"""IMDb orders spokenLanguages most prominent first. Either level can come back null."""
|
||||||
|
spoken = data.get("spokenLanguages") or {}
|
||||||
|
langs = spoken.get("spokenLanguages") or []
|
||||||
|
return langs[0].get("id") if langs else None
|
||||||
|
|
||||||
|
|
||||||
|
def _title_to_result(node: dict, kind: str) -> Optional[MetadataResult]:
|
||||||
|
title = (node.get("titleText") or {}).get("text") or (node.get("originalTitleText") or {}).get("text")
|
||||||
|
if not title:
|
||||||
|
return None
|
||||||
|
return MetadataResult(
|
||||||
|
title=title,
|
||||||
|
year=(node.get("releaseYear") or {}).get("year"),
|
||||||
|
kind=kind,
|
||||||
|
external_ids=ExternalIds(imdb_id=node.get("id")),
|
||||||
|
original_language=primary_language(node),
|
||||||
|
source="imdb",
|
||||||
|
raw=node,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class IMDBProvider(MetadataProvider):
|
||||||
|
"""IMDb metadata provider using IMDb's own GraphQL edge cache (no API key)."""
|
||||||
|
|
||||||
|
NAME = "imdb"
|
||||||
|
REQUIRES_KEY = False
|
||||||
|
ID_KIND = "imdb"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def _graphql(self, operation: str, query: str) -> Optional[dict]:
|
||||||
|
"""Run a persisted query, registering it with one POST the first time it is seen."""
|
||||||
|
extensions = {"persistedQuery": {"version": 1, "sha256Hash": hashlib.sha256(query.encode()).hexdigest()}}
|
||||||
|
# IMDb's GraphQL gateway decodes "+" literally, so these JSON params must carry no spaces
|
||||||
|
params = {
|
||||||
|
"operationName": operation,
|
||||||
|
"variables": "{}",
|
||||||
|
"extensions": json.dumps(extensions, separators=(",", ":")),
|
||||||
|
}
|
||||||
|
|
||||||
|
try:
|
||||||
|
r = self.session.get(GRAPHQL_URL, params=params, headers=GRAPHQL_HEADERS, timeout=30)
|
||||||
|
body = r.json()
|
||||||
|
if _persisted_query_missing(body):
|
||||||
|
self.log.debug("Registering persisted query %s", operation)
|
||||||
|
payload = {"operationName": operation, "variables": {}, "query": query, "extensions": extensions}
|
||||||
|
r = self.session.post(GRAPHQL_URL, json=payload, headers=GRAPHQL_HEADERS, timeout=30)
|
||||||
|
body = r.json()
|
||||||
|
except (requests.RequestException, ValueError) as exc:
|
||||||
|
self.log.debug("IMDb GraphQL %s failed: %s", operation, exc)
|
||||||
|
return None
|
||||||
|
|
||||||
|
if body.get("errors"):
|
||||||
|
self.log.debug("IMDb GraphQL %s errors: %s", operation, [e.get("message") for e in body["errors"]][:3])
|
||||||
|
return None
|
||||||
|
return body.get("data")
|
||||||
|
|
||||||
|
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
|
||||||
|
self.log.debug("Searching IMDb for %r (%s, %s)", title, kind, year)
|
||||||
|
|
||||||
|
constraints = [f'titleTextConstraint: {{searchTerm: "{escape_graphql(title)}"}}']
|
||||||
|
if year:
|
||||||
|
constraints.append(
|
||||||
|
f'releaseDateConstraint: {{releaseDateRange: {{start: "{year}-01-01", end: "{year}-12-31"}}}}'
|
||||||
|
)
|
||||||
|
types = KIND_TO_TYPES.get(kind)
|
||||||
|
if types:
|
||||||
|
constraints.append(f"titleTypeConstraint: {{anyTitleTypeIds: {json.dumps(types)}}}")
|
||||||
|
|
||||||
|
query = (
|
||||||
|
"query UnshackleTitleSearch { advancedTitleSearch(first: 5, constraints: {%s}) "
|
||||||
|
"{ edges { node { title { %s } } } } }" % (", ".join(constraints), TITLE_FIELDS)
|
||||||
|
)
|
||||||
|
|
||||||
|
data = self._graphql("UnshackleTitleSearch", query)
|
||||||
|
edges = ((data or {}).get("advancedTitleSearch") or {}).get("edges") or []
|
||||||
|
nodes = [edge["node"]["title"] for edge in edges if (edge.get("node") or {}).get("title")]
|
||||||
|
if not nodes:
|
||||||
|
self.log.debug("IMDb returned no results for %r", title)
|
||||||
|
return None
|
||||||
|
|
||||||
|
best = max(nodes, key=lambda n: _match_ratio(n, title))
|
||||||
|
result = _title_to_result(best, kind)
|
||||||
|
if not result or not result.title or not fuzzy_match(result.title, title):
|
||||||
|
self.log.debug("IMDb title mismatch: searched %r, got %r", title, result.title if result else None)
|
||||||
|
return None
|
||||||
|
|
||||||
|
self.log.debug("IMDb -> %s (ID %s)", result.title, result.external_ids.imdb_id)
|
||||||
|
return result
|
||||||
|
|
||||||
|
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
|
||||||
|
"""Fetch metadata by IMDB ID (e.g. 'tt1375666')."""
|
||||||
|
imdb_id = str(provider_id)
|
||||||
|
self.log.debug("Fetching IMDb title %s", imdb_id)
|
||||||
|
|
||||||
|
query = 'query UnshackleTitle { title(id: "%s") { %s } }' % (escape_graphql(imdb_id), TITLE_FIELDS)
|
||||||
|
data = self._graphql("UnshackleTitle", query)
|
||||||
|
node = (data or {}).get("title")
|
||||||
|
if not node:
|
||||||
|
return None
|
||||||
|
return _title_to_result(node, kind)
|
||||||
|
|
||||||
|
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
|
||||||
|
"""Return external IDs. For IMDB, the provider_id IS the IMDB ID."""
|
||||||
|
return ExternalIds(imdb_id=str(provider_id))
|
||||||
|
|
||||||
|
|
||||||
|
def _match_ratio(node: dict, title: str) -> float:
|
||||||
|
names = [(node.get("titleText") or {}).get("text"), (node.get("originalTitleText") or {}).get("text")]
|
||||||
|
return max((SequenceMatcher(None, _clean(title), _clean(n)).ratio() for n in names if n), default=0.0)
|
||||||
|
|
||||||
|
|
||||||
|
def _persisted_query_missing(body: dict) -> bool:
|
||||||
|
for error in body.get("errors") or []:
|
||||||
|
code = (error.get("extensions") or {}).get("code")
|
||||||
|
if code == "PERSISTED_QUERY_NOT_FOUND" or error.get("message") == "PersistedQueryNotFound":
|
||||||
|
return True
|
||||||
|
return False
|
||||||
@@ -1,132 +0,0 @@
|
|||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from difflib import SequenceMatcher
|
|
||||||
from typing import Optional, Union
|
|
||||||
|
|
||||||
import requests
|
|
||||||
|
|
||||||
from unshackle.core.config import config
|
|
||||||
from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, _clean, fuzzy_match
|
|
||||||
|
|
||||||
# Mapping from our kind ("movie"/"tv") to IMDxAPI title types
|
|
||||||
KIND_TO_TYPES: dict[str, list[str]] = {
|
|
||||||
"movie": ["movie"],
|
|
||||||
"tv": ["tvSeries", "tvMiniSeries"],
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def primary_language(data: dict) -> Optional[str]:
|
|
||||||
"""IMDb orders spokenLanguages most prominent first, and only detail responses carry it."""
|
|
||||||
langs = data.get("spokenLanguages") or []
|
|
||||||
return langs[0].get("code") if langs else None
|
|
||||||
|
|
||||||
|
|
||||||
class IMDBApiProvider(MetadataProvider):
|
|
||||||
"""IMDb metadata provider using IMDxAPI (api.tiffara.com, free, no API key)."""
|
|
||||||
|
|
||||||
NAME = "imdbapi"
|
|
||||||
REQUIRES_KEY = False
|
|
||||||
BASE_URL = "https://api.tiffara.com"
|
|
||||||
|
|
||||||
def is_available(self) -> bool:
|
|
||||||
return config.imdb_api_enabled
|
|
||||||
|
|
||||||
def search(self, title: str, year: Optional[int], kind: str) -> Optional[MetadataResult]:
|
|
||||||
self.log.debug("Searching IMDBApi for %r (%s, %s)", title, kind, year)
|
|
||||||
|
|
||||||
try:
|
|
||||||
params: dict[str, str | int] = {"query": title, "limit": 20}
|
|
||||||
r = self.session.get(
|
|
||||||
f"{self.BASE_URL}/search/titles",
|
|
||||||
params=params,
|
|
||||||
timeout=30,
|
|
||||||
)
|
|
||||||
r.raise_for_status()
|
|
||||||
data = r.json()
|
|
||||||
except (requests.RequestException, ValueError) as exc:
|
|
||||||
self.log.debug("IMDBApi search failed: %s", exc)
|
|
||||||
return None
|
|
||||||
|
|
||||||
results = data.get("titles") or data.get("results") or []
|
|
||||||
if not results:
|
|
||||||
self.log.debug("IMDBApi returned no results for %r", title)
|
|
||||||
return None
|
|
||||||
|
|
||||||
# Filter by type if possible
|
|
||||||
type_filter = KIND_TO_TYPES.get(kind, [])
|
|
||||||
filtered = [r for r in results if r.get("type") in type_filter] if type_filter else results
|
|
||||||
candidates = filtered if filtered else results
|
|
||||||
|
|
||||||
# Find best fuzzy match, optionally filtered by year
|
|
||||||
best_match: Optional[dict] = None
|
|
||||||
best_ratio = 0.0
|
|
||||||
|
|
||||||
for candidate in candidates:
|
|
||||||
primary = candidate.get("primaryTitle") or ""
|
|
||||||
original = candidate.get("originalTitle") or ""
|
|
||||||
|
|
||||||
for name in [primary, original]:
|
|
||||||
if not name:
|
|
||||||
continue
|
|
||||||
ratio = SequenceMatcher(None, _clean(title), _clean(name)).ratio()
|
|
||||||
if ratio > best_ratio:
|
|
||||||
# If year provided, prefer matches within 1 year
|
|
||||||
candidate_year = candidate.get("startYear")
|
|
||||||
if year and candidate_year and abs(year - candidate_year) > 1:
|
|
||||||
continue
|
|
||||||
best_ratio = ratio
|
|
||||||
best_match = candidate
|
|
||||||
|
|
||||||
if not best_match:
|
|
||||||
self.log.debug("No matching result found in IMDBApi for %r", title)
|
|
||||||
return None
|
|
||||||
|
|
||||||
result_title = best_match.get("primaryTitle") or best_match.get("originalTitle")
|
|
||||||
if not result_title or not fuzzy_match(result_title, title):
|
|
||||||
self.log.debug("IMDBApi title mismatch: searched %r, got %r", title, result_title)
|
|
||||||
return None
|
|
||||||
|
|
||||||
imdb_id = best_match.get("id")
|
|
||||||
result_year = best_match.get("startYear")
|
|
||||||
|
|
||||||
self.log.debug("IMDBApi -> %s (ID %s)", result_title, imdb_id)
|
|
||||||
|
|
||||||
return MetadataResult(
|
|
||||||
title=result_title,
|
|
||||||
year=result_year,
|
|
||||||
kind=kind,
|
|
||||||
external_ids=ExternalIds(imdb_id=imdb_id),
|
|
||||||
original_language=primary_language(best_match),
|
|
||||||
source="imdbapi",
|
|
||||||
raw=best_match,
|
|
||||||
)
|
|
||||||
|
|
||||||
def get_by_id(self, provider_id: Union[int, str], kind: str) -> Optional[MetadataResult]:
|
|
||||||
"""Fetch metadata by IMDB ID (e.g. 'tt1375666')."""
|
|
||||||
imdb_id = str(provider_id)
|
|
||||||
self.log.debug("Fetching IMDBApi title %s", imdb_id)
|
|
||||||
|
|
||||||
try:
|
|
||||||
r = self.session.get(f"{self.BASE_URL}/titles/{imdb_id}", timeout=30)
|
|
||||||
r.raise_for_status()
|
|
||||||
data = r.json()
|
|
||||||
except (requests.RequestException, ValueError) as exc:
|
|
||||||
self.log.debug("IMDBApi get_by_id failed: %s", exc)
|
|
||||||
return None
|
|
||||||
|
|
||||||
title = data.get("primaryTitle") or data.get("originalTitle")
|
|
||||||
result_year = data.get("startYear")
|
|
||||||
|
|
||||||
return MetadataResult(
|
|
||||||
title=title,
|
|
||||||
year=result_year,
|
|
||||||
kind=kind,
|
|
||||||
external_ids=ExternalIds(imdb_id=data.get("id")),
|
|
||||||
original_language=primary_language(data),
|
|
||||||
source="imdbapi",
|
|
||||||
raw=data,
|
|
||||||
)
|
|
||||||
|
|
||||||
def get_external_ids(self, provider_id: Union[int, str], kind: str) -> ExternalIds:
|
|
||||||
"""Return external IDs. For IMDB, the provider_id IS the IMDB ID."""
|
|
||||||
return ExternalIds(imdb_id=str(provider_id))
|
|
||||||
@@ -39,6 +39,7 @@ class OMDBProvider(MetadataProvider):
|
|||||||
|
|
||||||
NAME = "omdb"
|
NAME = "omdb"
|
||||||
REQUIRES_KEY = True
|
REQUIRES_KEY = True
|
||||||
|
ID_KIND = "imdb"
|
||||||
BASE_URL = "https://www.omdbapi.com/"
|
BASE_URL = "https://www.omdbapi.com/"
|
||||||
|
|
||||||
def is_available(self) -> bool:
|
def is_available(self) -> bool:
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ class TMDBProvider(MetadataProvider):
|
|||||||
|
|
||||||
NAME = "tmdb"
|
NAME = "tmdb"
|
||||||
REQUIRES_KEY = True
|
REQUIRES_KEY = True
|
||||||
|
ID_KIND = "tmdb"
|
||||||
BASE_URL = "https://api.themoviedb.org/3"
|
BASE_URL = "https://api.themoviedb.org/3"
|
||||||
|
|
||||||
def is_available(self) -> bool:
|
def is_available(self) -> bool:
|
||||||
|
|||||||
@@ -71,6 +71,7 @@ class TVDBProvider(MetadataProvider):
|
|||||||
|
|
||||||
NAME = "tvdb"
|
NAME = "tvdb"
|
||||||
REQUIRES_KEY = True
|
REQUIRES_KEY = True
|
||||||
|
ID_KIND = "tvdb"
|
||||||
BASE_URL = "https://api4.thetvdb.com/v4"
|
BASE_URL = "https://api4.thetvdb.com/v4"
|
||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
|
|||||||
@@ -12,13 +12,9 @@ from unshackle.core import binaries
|
|||||||
from unshackle.core.config import config
|
from unshackle.core.config import config
|
||||||
from unshackle.core.providers import (
|
from unshackle.core.providers import (
|
||||||
ExternalIds,
|
ExternalIds,
|
||||||
MetadataResult,
|
|
||||||
enrich_ids,
|
|
||||||
fetch_external_ids,
|
|
||||||
fuzzy_match,
|
fuzzy_match,
|
||||||
get_available_providers,
|
get_available_providers,
|
||||||
get_provider,
|
resolve_by_ids,
|
||||||
search_metadata,
|
|
||||||
)
|
)
|
||||||
from unshackle.core.titles.episode import Episode
|
from unshackle.core.titles.episode import Episode
|
||||||
from unshackle.core.titles.movie import Movie
|
from unshackle.core.titles.movie import Movie
|
||||||
@@ -118,53 +114,13 @@ def tag_file(
|
|||||||
|
|
||||||
if config.tag_imdb_tmdb:
|
if config.tag_imdb_tmdb:
|
||||||
try:
|
try:
|
||||||
providers = get_available_providers()
|
has_ids = tmdb_id is not None or imdb_id is not None or tvdb_id is not None
|
||||||
if not providers and tvdb_id is None:
|
if not get_available_providers() and not has_ids:
|
||||||
log.debug("No metadata providers available; skipping tag lookup")
|
log.debug("No metadata providers available; skipping tag lookup")
|
||||||
apply_tags(path, custom_tags)
|
apply_tags(path, custom_tags)
|
||||||
return
|
return
|
||||||
|
|
||||||
result: Optional[MetadataResult] = None
|
result = resolve_by_ids(tmdb_id, imdb_id, tvdb_id, title=name, year=year, kind=kind)
|
||||||
|
|
||||||
# Direct ID lookup path
|
|
||||||
if imdb_id:
|
|
||||||
imdbapi = get_provider("imdbapi")
|
|
||||||
if imdbapi:
|
|
||||||
result = imdbapi.get_by_id(imdb_id, kind)
|
|
||||||
if not result:
|
|
||||||
# IMDxAPI is off or had no answer; the ID alone still resolves the others
|
|
||||||
result = MetadataResult(title=name, year=year, kind=kind)
|
|
||||||
result.external_ids.imdb_id = imdb_id
|
|
||||||
enrich_ids(result)
|
|
||||||
elif tmdb_id is not None:
|
|
||||||
tmdb = get_provider("tmdb")
|
|
||||||
if tmdb:
|
|
||||||
result = tmdb.get_by_id(tmdb_id, kind)
|
|
||||||
if result:
|
|
||||||
ext = tmdb.get_external_ids(tmdb_id, kind)
|
|
||||||
result.external_ids = ext
|
|
||||||
enrich_ids(result)
|
|
||||||
else:
|
|
||||||
# Search across providers in priority order
|
|
||||||
result = search_metadata(name, year, kind)
|
|
||||||
|
|
||||||
# If we got a TMDB ID from search but not the other IDs, fetch them
|
|
||||||
if (
|
|
||||||
result
|
|
||||||
and result.external_ids.tmdb_id
|
|
||||||
and not (result.external_ids.imdb_id and result.external_ids.tvdb_id)
|
|
||||||
):
|
|
||||||
ext = fetch_external_ids(result.external_ids.tmdb_id, kind)
|
|
||||||
if ext.imdb_id and not result.external_ids.imdb_id:
|
|
||||||
result.external_ids.imdb_id = ext.imdb_id
|
|
||||||
if ext.tvdb_id and not result.external_ids.tvdb_id:
|
|
||||||
result.external_ids.tvdb_id = ext.tvdb_id
|
|
||||||
enrich_ids(result)
|
|
||||||
|
|
||||||
if tvdb_id is not None:
|
|
||||||
if result is None:
|
|
||||||
result = MetadataResult(kind=kind)
|
|
||||||
result.external_ids.tvdb_id = tvdb_id
|
|
||||||
|
|
||||||
if result and result.external_ids:
|
if result and result.external_ids:
|
||||||
standard_tags = _build_tags_from_ids(result.external_ids, kind)
|
standard_tags = _build_tags_from_ids(result.external_ids, kind)
|
||||||
|
|||||||
Reference in New Issue
Block a user