feat(proxy): try bare --proxy providers in proxy_providers key order

Basic stays first and Control D last; the rest follow their config key order. Geofence and proxy_map lookups go through pick_proxy and skip a provider that fails.

docs/guide/proxies-and-vpn.md

@@ -125,14 +125,18 @@ forwarder for it. See [Control D](#control-d).
 ### Provider load order
 
 When you do a download without `--no-proxy`, unshackle instantiates every configured
-proxy provider once, in this fixed order. It uses that same order for bare-query
+proxy provider once. It uses that same order for bare-query resolution:
-resolution:
 
-**Basic → ExpressVPN → NordVPN → Proton VPN → Surfshark → Windscribe → Gluetun → Hola → Control D**
+1. Basic is always first, so your own proxies win.
+2. The other proxy providers follow in the order of their keys under `proxy_providers` in
+   `unshackle.yaml`. Proxy providers that load without a config key, such as Hola, come
+   after the listed ones in this default order:
+   **ExpressVPN → NordVPN → Proton VPN → Surfshark → Windscribe → Gluetun → Hola**
+3. Control D is always last.
 
-So if you have both Basic and NordVPN configured and use `--proxy us`, a `us` entry in
+So if you list `windscribevpn` above `nordvpn` and use `--proxy us`, a `us` entry in your
-your Basic config wins, because unshackle tries Basic first. To skip ahead to a specific
+Basic config answers first, then Windscribe. To skip ahead to a specific proxy provider,
-proxy provider, prefix the query (`--proxy nordvpn:us`).
+prefix the query (`--proxy nordvpn:us`).
 
 If a proxy provider cannot load, for example because its server list cannot be fetched
 or its config is not valid, unshackle logs a warning and continues with the other proxy
@@ -1025,21 +1029,14 @@ The returned proxy is HTTPS on **port 443**.
 
 ## Developer notes
 
-!!! note "Two resolution paths (developers)"
+!!! note "One resolution path (developers)"
-    unshackle has **two** proxy-resolution code paths that are intentionally not
+    `dl`, `search`, the REST API handlers and the remote-service client all load proxy
-    identical:
+    providers through `load_proxy_providers()` in `unshackle/core/proxies/resolve.py`.
-
+    The REST API and the remote-service client call it through
-    - `unshackle/commands/dl.py` inlines its own logic and loads **all nine** providers,
+    `initialize_proxy_providers()`, which leaves out Gluetun because Gluetun starts a
-      including Windscribe and Gluetun.
+    local Docker container. All paths accept the same query grammar: a direct URI, a
-    - `unshackle/core/proxies/resolve.py::initialize_proxy_providers()` is the shared
+    `provider:query`, a bare query tried against providers in order, or a `controld://`
-      resolver used by the REST API handlers and the remote-service client. It **omits**
+    resolver.
-      Windscribe and Gluetun.
-
-    Both paths accept the same query grammar (a direct URI, a `provider:query`, a bare
-    country query tried against providers in order, or a `controld://` resolver), but the
-    set of available providers differs. Keep this in mind when you add a proxy provider:
-    an entry in `dl.py` does not make it available over the API, and the opposite is also
-    true.
 
     The remote-service client resolves `--proxy` with its own
     `resolve_remote_proxy_arg()`. It changes a `controld:` query to a `controld://`

docs/reference/configuration/network.md

@@ -61,8 +61,11 @@ headers:
 - **Type:** `dict`  ·  **Default:** `{}`
 
 Proxy/VPN provider configuration. Each sub-key names a proxy provider, and unshackle passes its
-block straight to that provider's constructor. See [Proxies & VPN](../../guide/proxies-and-vpn.md)
+block straight to that provider's constructor. A bare `--proxy` query tries Basic first, then
-for the full proxy provider guide. Recognised proxy providers and their exit ports:
+the other proxy providers in the order of their sub-keys, and Control D last (see
+[Provider load order](../../guide/proxies-and-vpn.md#provider-load-order)). See
+[Proxies & VPN](../../guide/proxies-and-vpn.md) for the full proxy provider guide. Recognised proxy
+providers and their exit ports:
 
 | Provider | Config key | Credentials | Proxy scheme/port |
 |----------|-----------|-------------|-------------------|

tests/core/test_proxy_loading.py

@@ -63,6 +63,17 @@ def test_raise_errors_raises_the_load_failure() -> None:
         load_proxy_providers(OTHERS, raise_errors=True)
 
 
+def test_bare_query_order_follows_config_key_order_with_controld_last(monkeypatch: pytest.MonkeyPatch) -> None:
+    from unshackle.core.proxies import ControlD
+
+    order = {"controld": {"token": "t"}, "surfsharkvpn": CREDENTIALS, "nordvpn": CREDENTIALS, "basic": {"us": "x"}}
+    monkeypatch.setattr(config, "proxy_providers", order)
+    monkeypatch.setattr(ControlD, "get_locations", lambda self: [])
+    loaded = load_proxy_providers(("expressvpn", "protonvpn", "hola"), quiet=True)
+    names = [getattr(p, "name", p.__class__.__name__) for p in loaded]
+    assert names == ["Basic", "SurfsharkVPN", "NordVPN", "ControlD"]
+
+
 def test_rest_path_loads_windscribe_but_not_gluetun(monkeypatch: pytest.MonkeyPatch) -> None:
     from unshackle.core.proxies import Gluetun, WindscribeVPN
     from unshackle.core.proxies.resolve import initialize_proxy_providers

unshackle/core/proxies/resolve.py

@@ -54,7 +54,7 @@ def find_provider(proxy_providers: List[Any], name: str, hint: str = "") -> Any:
 
 def load_proxy_providers(exclude: tuple[str, ...] = (), raise_errors: bool = False, quiet: bool = False) -> List[Any]:
     """
-    Build the proxy providers set in config, in the order a bare query tries them.
+    Build the proxy providers set in config: Basic first, then config key order, Control D last.
 
     Each proxy provider loads on its own. One that fails is logged and kept as Unavailable, so a bare
     query skips it and a query that names it reports why. *raise_errors* raises the failure instead.
@@ -75,8 +75,15 @@ def load_proxy_providers(exclude: tuple[str, ...] = (), raise_errors: bool = Fal
     )
 
     settings = config.proxy_providers
-    # Control D is last: a bare region reaches it only when nothing else answers, since it changes the account.
+    rank = {key: i for i, key in enumerate(settings)}
-    classes = (Basic, ExpressVPN, NordVPN, ProtonVPN, SurfsharkVPN, WindscribeVPN, Gluetun, Hola, ControlD)
+    classes = [
+        Basic,
+        *sorted(
+            (ExpressVPN, NordVPN, ProtonVPN, SurfsharkVPN, WindscribeVPN, Gluetun, Hola),
+            key=lambda c: rank.get(c.__name__.lower(), len(rank)),
+        ),
+        ControlD,
+    ]
 
     def build(cls: type) -> Any:
         key = cls.__name__.lower()

unshackle/core/service.py

@@ -25,7 +25,7 @@ from unshackle.core.constants import AnyTrack
 from unshackle.core.credential import Credential
 from unshackle.core.drm import DRM_T
 from unshackle.core.proxies.basic import Basic
-from unshackle.core.proxies.resolve import find_provider
+from unshackle.core.proxies.resolve import pick_proxy
 from unshackle.core.search_result import SearchResult
 from unshackle.core.session import (
     BACKOFF_FACTOR,
@@ -229,40 +229,16 @@ class Service(metaclass=ABCMeta):
                     self.log.info(
                         f"Found service-specific proxy mapping: {full_proxy_key} -> {sanitize_proxy_for_log(mapped_value)}"
                     )
-                    if proxy_provider_name:
+                    try:
-                        # Specific provider requested
+                        proxy_provider, proxy = pick_proxy(ctx.obj.proxy_providers, mapped_value, proxy_provider_name)
-                        try:
+                        self.log.info(
-                            proxy_provider = find_provider(ctx.obj.proxy_providers, proxy_provider_name)
+                            f"Using mapped proxy from {proxy_provider.__class__.__name__}: "
-                        except ValueError:
+                            f"{sanitize_proxy_for_log(proxy, mask_host=isinstance(proxy_provider, Basic))}"
-                            proxy_provider = None
+                        )
-                        if proxy_provider:
+                    except ValueError:
-                            mapped_proxy_uri = proxy_provider.get_proxy(mapped_value)
+                        self.log.warning(
-                            if mapped_proxy_uri:
+                            f"No provider could resolve mapped value '{sanitize_proxy_for_log(mapped_value)}', using default"
-                                proxy = mapped_proxy_uri
+                        )
-                                self.log.info(
-                                    f"Using mapped proxy from {proxy_provider.__class__.__name__}: "
-                                    f"{sanitize_proxy_for_log(proxy, mask_host=isinstance(proxy_provider, Basic))}"
-                                )
-                            else:
-                                self.log.warning(
-                                    f"Failed to get proxy for mapped value '{sanitize_proxy_for_log(mapped_value)}', using default"
-                                )
-                        else:
-                            self.log.warning(f"Proxy provider '{proxy_provider_name}' not found, using default proxy")
-                    else:
-                        for proxy_provider in ctx.obj.proxy_providers:
-                            mapped_proxy_uri = proxy_provider.get_proxy(mapped_value)
-                            if mapped_proxy_uri:
-                                proxy = mapped_proxy_uri
-                                self.log.info(
-                                    f"Using mapped proxy from {proxy_provider.__class__.__name__}: "
-                                    f"{sanitize_proxy_for_log(proxy, mask_host=isinstance(proxy_provider, Basic))}"
-                                )
-                                break
-                        else:
-                            self.log.warning(
-                                f"No provider could resolve mapped value '{sanitize_proxy_for_log(mapped_value)}', using default"
-                            )
 
             if not proxy:
                 # don't override the explicit proxy set by the user, even if they may be geoblocked
@@ -284,12 +260,10 @@ class Service(metaclass=ABCMeta):
                                 self.log.info(
                                     f"Service is Geoblocked in your region, getting a Proxy to {requested_proxy}"
                                 )
-                                for proxy_provider in ctx.obj.proxy_providers:
+                                try:
-                                    proxy = proxy_provider.get_proxy(requested_proxy)
+                                    proxy_provider, proxy = pick_proxy(ctx.obj.proxy_providers, requested_proxy)
-                                    if proxy:
+                                    self.log.info(f"Got Proxy from {proxy_provider.__class__.__name__}")
-                                        self.log.info(f"Got Proxy from {proxy_provider.__class__.__name__}")
+                                except ValueError:
-                                        break
-                                if not proxy:
                                     self.log.warning(
                                         f"No proxy available for {requested_proxy}. "
                                         f"Pass --proxy with a proxy in {requested_proxy}, or the request can fail."