Diagnose Simple API serving
Three things a client reads off a Simple page can surprise it: an api-version lower than expected, a missing GPG
signature marker, and a 301 where it sent a slashless URL. Each is peryx serving only what its bytes back up. This
guide diagnoses all three and shows what to rely on instead. The examples assume peryx at http://127.0.0.1:4433.
Diagnose a mirror that reports api-version 1.0
A client expected PEP 700's versions and size fields, but your mirror's Simple
JSON reports meta.api-version 1.0 and the fields are missing. peryx advertises 1.0 on purpose: it serves the
version the payload guarantees, and 1.0 means it cannot promise those fields.
Inspect the advertised version
Read the served version for the project:
curl -s -H "Accept: application/vnd.pypi.simple.v1+json" \
http://127.0.0.1:4433/{route}/simple/{project}/ \
| python3 -c 'import sys, json; print(json.load(sys.stdin)["meta"]["api-version"])'
1.4 means peryx guarantees versions and size. 1.0 means it does not, and the two causes below are the only ways
it gets there.
Cause 1: the upstream declares no PEP 700 version
peryx serves 1.0 for an upstream that declared 1.0, or that declared no version at all, such as a plain
PEP 503 HTML index. Ask the upstream for the version it sends:
curl -s -H "Accept: application/vnd.pypi.simple.v1+json" \
https://upstream.example/simple/{project}/ \
| python3 -c 'import sys, json; print(json.load(sys.stdin).get("meta", {}).get("api-version"))'
curl -s https://upstream.example/simple/{project}/ | grep -i pypi:repository-version
If the JSON prints None or 1.0, the upstream promises neither field, and peryx is right to serve 1.0. peryx does
not invent versions or size to reach 1.4; it advertises the version the upstream's bytes satisfy. An HTML upstream
prints 1.0 whatever its pypi:repository-version says, because PEP 700 leaves the HTML form unchanged from 1.0.
To serve 1.4, front an upstream that serves PEP 691 JSON declaring 1.1 or higher. pypi.org does; an HTML-only mirror
or an older Artifactory does not.
Cause 2: a virtual layer caps the merged page
A virtual index is only as capable as its weakest layer. If any layer that carries the project serves 1.0, the merged
page drops to 1.0, even when another layer would serve 1.4 on its own. Query each layer on its own route to find the
one holding the version down:
for route in hosted pypi; do
printf '%s: ' "$route"
curl -s -H "Accept: application/vnd.pypi.simple.v1+json" \
"http://127.0.0.1:4433/${route}/simple/{project}/" \
| python3 -c 'import sys, json; print(json.load(sys.stdin)["meta"]["api-version"])'
done
The layer that prints 1.0 is the cap. A hosted layer whose uploads carry no recorded size, or a cached layer fronting
a PEP 503 upstream, will show up here. The merged page cannot promise a field one of its layers omits, so it takes the
lower version by design.
api-version 1.0 client behavior
A 1.0 page is a valid page. It is not missing data that peryx should have sent; it is a page whose format never
guaranteed versions or size in the first place. Adjust the client rather than the mirror:
- Treat per-file
sizeas optional. Read it when present, fall back to aContent-Lengthfrom aHEADon the file, or to no size at all. - Do not rely on a top-level
versionsarray. Derive the release set from the filenames on the page instead.
If the client requires PEP 700 guarantees, point it at a route whose every layer advertises 1.1+, and it will read
1.4 with both fields present.
Rely on hashes instead of the gpg-sig marker
You have a client or a tool that read the gpg-sig marker (data-gpg-sig in HTML, has_sig in the legacy JSON) or
fetched a file's .asc signature. Through peryx that marker is now absent for the files peryx content-addresses onto
its own route, and {file_url}.asc on those files returns 404.
Stop fetching .asc from peryx's file route
If your tool sees the marker and fetches {file_url}.asc, drop that fetch for any file whose URL points at peryx's
/{route}/files/{sha256}/{filename} route. peryx serves the blob and the PEP 658
.metadata sibling there, never the .asc. The signature was never at peryx's route; it lived next to the upstream URL
that peryx replaced. The marker is now dropped so your tool does not chase a 404.
Verify with the hash the index serves
The Simple API provides integrity hashes. Every file object lists a hashes map, and peryx serves a sha256 for each
content-addressed file:
curl -s -H "Accept: application/vnd.pypi.simple.v1+json" \
http://127.0.0.1:4433/root/pypi/simple/requests/ \
| python3 -c "import json,sys; d=json.load(sys.stdin); print(d['files'][0]['hashes'])"
pip and uv already verify this hash on download; peryx verifies the digest as it streams the blob into its store. A
lockfile pins the same sha256, so requiring hashes in your install (pip install --require-hashes, a uv.lock, or
--only-binary with pinned digests) gives you tamper-evidence tied to the file bytes, without a signature.
Required signatures
If a policy requires the OpenPGP signature, fetch it from the origin index that holds it, not through peryx. Read the
original file URL from the upstream index's own Simple page and fetch its .asc there:
curl -sfO https://files.pythonhosted.org/.../requests-2.32.5-py3-none-any.whl.asc
PyPI deprecated GPG signatures in 2023 and no longer serves them. A private signing upstream requires direct network access. Requiring its deprecated signature adds an origin dependency.
Follow the trailing-slash redirect
A tool that requests .../simple/{project} without the trailing slash receives a 301 to the canonical URL from both
pypi.org and peryx. Build the canonical URL to avoid the extra request.
pip, uv, twine, and poetry already append the slash, so this only comes up in custom code: a shell loop, a
health check, a crawler, a language client that assembles URLs itself.
Follow the redirect
The redirect is a plain 301 with a Location header. Any HTTP client can follow it; most need a flag or an option
turned on, since following a redirect after a non-GET is off by default in some clients.
# -L follows the Location header to the slashed, normalized URL
curl -LsS http://127.0.0.1:4433/root/pypi/simple/Flask
import httpx
# follow_redirects is off by default in httpx and requests; turn it on
resp = httpx.get("http://127.0.0.1:4433/root/pypi/simple/Flask", follow_redirects=True)
resp.raise_for_status()
# httpie follows redirects with --follow
http --follow GET http://127.0.0.1:4433/root/pypi/simple/Flask
Each of these lands on /root/pypi/simple/flask/ and reads the project detail. The query string, if any, is carried
across the hop, so parameters survive.
Normalize the name yourself to skip the hop
The redirect also normalizes the project name, so a slashless request for a non-canonical spelling costs two round
trips: the 301, then the page. If you control the URL you build, normalize the name and append the slash yourself, and
the first request hits the page directly.
Normalization is PEP 503: lowercase the name, then collapse every run of ., -,
or _ to a single -.
import re
def normalize(name: str) -> str:
return re.sub(r"[-_.]+", "-", name).lower()
url = f"http://127.0.0.1:4433/root/pypi/simple/{normalize('Flask.Test')}/"
# http://127.0.0.1:4433/root/pypi/simple/flask-test/
name="Flask.Test"
slug=$(printf '%s' "$name" | tr '[:upper:]' '[:lower:]' | sed -E 's/[-_.]+/-/g')
url="http://127.0.0.1:4433/root/pypi/simple/${slug}/"
# http://127.0.0.1:4433/root/pypi/simple/flask-test/
With the name already canonical and the slash in place, no redirect fires.
Reject names containing a slash
Only a single project segment is redirected. A path with an extra slash in it, such as .../simple/some/thing, is not a
project name and is not redirected; it falls through to a 404. A project name never contains a slash, so this only
bites a malformed URL. Build the path from a normalized name and one trailing slash and you stay on the redirected path.
Related
- The exact rules across JSON, HTML, and legacy JSON: Simple API serving
- Serving rationale: Simple API serving
- End-to-end examples: Simple API behavior