Decisions
One line per non obvious choice: the decision, then the reason. The
decisions of the app work live in macos/DECISIONS.md. The 0.5.5, 0.5 and 0.4
runs come first, the 0.3 easy install run follows.
LAN sharing: rooms (2026-11)
- Sharing is copy-only:
share setuprestores a backup intoshare.localhostand hardens that; the site itself is never served to the LAN, so a guest’s mistake or a bug in the gateway reaches a disposable database. - One share per Mac, and the copy’s name is fixed (
share.localhost, no random suffix): one gateway port (8443), one control socket and one launchd job keep the fail-closed checks simple, and a second room would double the attack surface for no use the owner asked for. - In the copy, every user but the seats is disabled and Administrator’s password is replaced by a random one that is never shown: guests sign in through the gateway only, so no password of the copy has to be known by anyone.
- Guests sign in by the gateway asking the copy for a session of the seat’s user (the sign-in snippet factors from upstream’s Open as,
FL_PY_OPEN_AS); the random passwords are discarded, so a seat cannot be used outside the room. - The share job’s label is
com.benchbar-share.*and its plist lives in the share folder, outside~/Library/LaunchAgents, deleted once launchd has loaded it: no bench agent glob,uninstall-service --all, doctor orbenchbar downpattern can see it or touch it, and nothing restarts it at login. - The copy is served by
share_web.py, a shim that replaceswerkzeug.serving.run_simplebeforefrappe.appis imported so the server binds127.0.0.1with the debugger, evalex and reloader off; it runs withCI=1andenv -u DEV_SERVER, because the bench’s.envsetsDEV_SERVER=1, which forces tracebacks into responses. - Rooms replace pairing: a 6 digit room code verified with CPace-X25519-SHA512 (
draft-irtf-cfrg-cpace-21, protocol v3), bound to both TLS certificates by their SPKI fingerprints and the TLS exporter. The commit-reveal SAS, protocol v2 and pin-after-Yes are gone: the host no longer approves each join (the owner accepted the trust change), and the guest pins the host only after the host proved it knows the code (pin-after-proof). - The room code proves membership, not the host’s identity, on a first join (J19): a rogue room that knows the code looks the same to a new guest. The docs and the host card say so; the pin protects rejoins only.
- Ten failed confirmations per code lock the room, and only a new code resets the budget (J3): an attacker gets at most 10 guesses in a million per code, and can never rotate a code someone already handed out.
- The code lives only in the gateway’s memory and leaves through the 0600 control socket’s
codeop.share code --jsonis the one document that carries it (J14), andshare codeis refused underBENCHBAR_NONINTERACTIVE,--for-agentandmcp(the app passesBENCHBAR_NONINTERACTIVE=0). - No session id on disk:
share open-as --jsonnever carries the link or the sid (--print-urlprints upstream’s link, which does); stop, remove and assign end the seat’s sessions in the copy, and a sign-out that could not finish is kept aswipe_failedand retried. share setupstops until--acknowledge-data: the copy holds the site’s real people, customers, contacts and addresses, and the person who shares it must say they know./socket.ioanswers 404 at the gateway in v1: Frappe’s realtime server would need namespace enforcement on every CONNECT packet and an event allowlist (open_in_editoris one of its events) to be safe for guests, so guests get no live updates and reload instead.- One Mac per seat at a time, and a person may switch seats (A15 re-ruled for rooms): a seat taken by another Mac reads “taken by {name}”, and the host can move or remove anyone.
- The activity log keeps only the first two path segments of each request (
/app/sales-order, never a document name), one 0600 file per day, 14 days: “Priya opened Sales Order” is enough for a host, and a customer’s name never lands in a log. - The 5 minute pairing TTL (A17) went with pairing; a join’s start record is kept 30 s, and an unfinished start that expires is charged as a failure (J6).
dns-sddying marks the sharedegradedand is restarted; it never ends the share (A18): the room is still reachable by address, and the Local Network privilege is checked by the app before the first start.- The deadline is capped at 12 hours from the start, and
share extendmoves it within that cap: a room left open overnight ends by itself. - No MCP tool and no
benchbar://route starts or joins a share (U3): sharing a site with real data is a person’s decision. - The gateway forwards only the
sidandpreferred_languagecookies to the copy: any other cookie of the guest’s browser (another site’s, a tracking one) never reaches Frappe. - Sharing fails closed: a gateway that exits, or two failed critical checks in a row, end the job; a probe that cannot run is a failure, never an ok.
- The pf check probes this Mac’s own LAN address with its own throwaway listener inside the blocked range and a control port above it: a refused port proves nothing unless the control answers, and the anchor’s
from ! 127.0.0.1rule blocks the Mac’s own LAN address too, which is what makes the probe possible. - The LAN guard and the firewall allow are the two new sudo exceptions on macOS (
lan-guard install/remove,lan-guard allow-node): a pf anchor and a launch daemon in/etcand/Libraryneed root, and the root script carries both files as literals so root never reads a file the user owns. Both are refused underBENCHBAR_NONINTERACTIVE,--for-agentandmcp, and without a terminal unless the app passesBENCHBAR_SUDO=guiafter its own alert. lan-guard statusand install’s “unchanged” use share preflight’s rule (the anchor’s non-comment rules, whitespace squeezed, and a daemon that loads it): the Phase 0 files written by hand read as installed, and no sudo is asked for a header comment.share teardownkeeps the archived copy’s Time Machine exclusion: the archive still holds the site’s real data.
0.8: the install stream and the privilege path
install --jsonandadopt --jsonshare one emitter inui.sh(fl_jsonl, fd 3) instead of growingpull’s: the phase scripts are child processes and need the same lines, and they inherit fd 3 andFL_JSONLfor free.pullkeeps its own emitter.- The numbered steps announce themselves from
fl_step_beginandfl_step_end(ids inFL_STEP_IDS), so the terminal line and the stream line come from one call and cannot drift; the message of a failed, skipped or warning step is the last[FAIL]or[WARN]line the step wrote to the log. - Sections of the phase scripts are the stream’s nested steps:
fl_sectionends the one before (warning if a[WARN]or[FAIL]was printed in it) and an EXIT trap closes the last. Only a process withFL_JSONL_PARENTset does this, and benchbar sets it for the phase script’s environment only, so benchbar’s ownfl_sectioncalls stay out of the stream. - The repair engine inside install emits its actions as steps with parent
serviceand no number; inside adopt as numbered steps. Its ownplanline is never emitted there (a plan hook is: adopt’s plan is built once the engine knows its actions). - The stream starts before the log and the lock are taken, and
fl_cleanupsendsdoneafter the lock is released, so a refusal, a busy lock, a signal or a failure all end in exactly onedone, and a reader that starts the next command ondonefinds the lock free. --jsonwithout--yes(or--dry-run) is refused before anything is written, and so is a missingADMIN_PASSWORDfor a new site: nothing on the stream can answer a question, and the phases would otherwise stop on one.- A dry run with
--jsongets a log (no other dry run does): the contract puts the human plan there. The plan line turns every later event off (FL_JSONL_QUIET), so a dry run streamsplananddoneonly. - Commands run by
fl_run_longand the setup Redis get no fd 3: a program must not write into the stream, and a daemon must not hold the pipe open and keep the app waiting for the end of the stream. - The port block of a new bench is chosen before the plan (
fl_install_port_plan) and used by step 2 as it is, so the plan’sport_offsetandweb_urlare what the run uses; the human note about taken default ports still prints where it did. BENCHBAR_SUDO=guiruns each root step as one script throughosascript ... with administrator privileges, handed to AppleScript as an argument (never inside a script string), so only shell quoting matters (fl_sq). Root scripts name every tool by absolute path and keep the safety properties of the terminal path: a fresh root-only folder in /tmp for the package, the hash of the copy against the pin, the awk rewrite with the “address names” and line count checks, onemv.- Rosetta 2 is installed inside the package’s root script when it is missing, so the step stays one dialog. The dialog’s title is the step’s name.
- A cancelled dialog (
(-128)in osascript’s error) and a failed one are remembered by title for the run: the service step’s engine would otherwise planhosts_entryandwkhtmltopdf_installagain and show a second dialog. Their nested events are left out of the stream for the same reason. fl_sudo_beginandfl_sudo_dropdo nothing in gui mode (nosudo -v, no keepalive): there is no cached credential to keep or to drop, and the up front prompt is skipped.ports applybatches the hosts lines of all benches into one dialog before the first adopt; a cancelled dialog makes each bench’shosts_entryskip with itssite hosts --bench-dircommand instead of asking again.- The tests’ fake osascript runs the command line with
/bin/shas the test user and maps the absolute tool paths to the mocks: the production script has no environment hook that would let a variable redirect what root runs. - A plain
doctordoes not print the prerequisites: AGENTS.md promises a fix line after every WARN and FAIL and exit 1 on any FAIL, and a group outside the counts broke both (an Intel Mac failed with exit 0). Onlydoctor --jsoncarries them, as a separate array;doctor --prerequisitesgives every non ok check afix_command, a plain instruction where no command exists. install --jsonandadopt --jsonwithoutBENCHBAR_SUDO=guinever prompt:fl_sudo_beginruns onlysudo -n truethere, so a cached credential or a NOPASSWD rule works and anything else skips the privileged steps with their manual command, as a cancelled dialog does.sudo -von a stream with no terminal would hang or fail after a delay.- The dialog memory is per kind of step (
FL_ROOT_KIND,hosts), not per title: the batch dialog ofports applyand each bench’s own hosts step have different titles but are the same question, and a cancelled or failed one must not be asked again for the next bench. doctor --prerequisitescalls nofl_context_initand nofl_require_bench: it must work before a bench exists. On Linux it reportsdisk_free,bench_folderanddefault_portsonly. In a bench’sdoctorthe prerequisites are a separate array and group, not counted in the summary or the exit code.fl_sqquotes with a variable holding the quote (${1//$q/$q\\$q$q}): the form with backslash escaped quotes in the replacement quoted wrongly under bash 3.2 when the root scripts went through it.default_portsfails when no block up to the last offset is free: install would fail only after the privileged and system steps, and no--port-offsetcan help, so the wizard stops on Check Your Mac (Codex review). A dry run that ends with exit 1 is no plan, even when it sent one.
0.7.3: ownership
- One ownership test,
fl_pid_is_bench_own_strict, decides every kill of a process found by its working folder (honcho, socketio, port listeners) in the CLI and the runner: lsof must report the folder as the bench or a folder inside it. An unreadable folder is “not ours”, not “just exited”: the old lenient reading letdownsignal another bench’s honcho when lsof lagged, and a leftover that really is ours costs nothing when it is left for the next cleanup. The one other kill is thepkillof serve, worker and schedule by their command line, which starts with the bench’s own absoluteenv/bin/pythonpath: the path is the proof there. - The runner pauses with
port_conflictinstead of starting honcho when a port is still held after its cleanup: honcho would fail to bind, exit non-zero and trip the crash guard after three rounds, with a log that names no port. The flag is cleared by benchup like the others, so the retry is one command once the port is free. fl_pm_running(up’s “already running”) counts honcho, serve, worker, schedule and socketio only, never a listener: a setup Redis left by a cut-short site add is provably the bench’s and was making up a no-op. The runner’s cleanup stops it since its folder is the bench.- A bench running outside BenchBar (bench start: honcho on the bench’s own Procfile, or bench processes with no benchbar agent running them) gets its agent loaded with a manual stop flag, never a kickstart: that session is a person’s terminal, and the runner’s cleanup would have stopped it. Adopt says so and does not refuse, so the service files are in place for the next benchup.
site dropchecks “only site” before and apart from “default site”: the default test depended onFL_SITE, whichSITE_NAMEor--sitecan point at a site that does not exist, and the only site then fell through to drop-site.- The MCP server checks a
benchargument of a changing tool againstbenchbar list --json(real paths) and refuses the rest: the CLI itself accepts any folder for--bench-dir, which is right for a person in a terminal and wrong for a model that got a path from a log. fgtakes the CLI lock likedowndoes, and releases it just beforeexec honcho: exec skips the EXIT trap, and a lock held for as long as honcho runs in front would block every other benchbar run.
0.7.3: Node for v15
- A failed
node_installstops the repair run like a failed env rebuild: the plist and the shell block that follow would name a Node that is not there, and a working node@20 bench would stop. A formula brew lists whose node is missing getsbrew reinstall(install is a no-op on an installed formula). repairexits 0 when it did everything it could, even while a check without an action still fails (a disabled formula): its exit code says what repair did, anddoctoris the command whose exit code says the state. The verify pass names the leftover once more.v15-ltsmoves fromnode@20tonode@22, not tonode@24: Homebrew disablesnode@20on 2026-10-28 (deprecated a year earlier), Frappe version-15’senginessaysnode >=18, andnode@22is the LTS line with the longest Homebrew life that the v15 asset build is known to accept. It is checked on a Mac before each release; the cloud cannot build a bench.- The move is a repair, not a surprise: the shell block and the agent plist carry the formula in their PATH, so their hash changes and doctor marks them outdated;
toolchain_nodeandtoolchain_yarngained the actionsnode_install(brew install of the profile’s formula) andyarn_install(npm install -g yarn under it), so onebenchbar repaircompletes the move. Nothing uninstallsnode@20: another tool on the Mac may use it, and brew removes it on its own schedule. - The disable dates come from
brew info --json=v2, one call per formula, read from the local tap: three Ruby starts in doctor, no network. One call for all three would be cheaper, but the JSON nestsnamekeys (requirements), so pairing dates with formulae by position is the only way and a brew that reorders them would silently mislabel. - The tests shorten the system part of the bench’s launchd PATH (
FL_LAUNCHD_PATH_SYSTEM, no/usr/local/bin): CI runners carry a node and a yarn there, which stood in for the mocks’ and made the yarn assertions pass or fail by host. - 90 days is the warning horizon in doctor and in the release check alike, so a release cut inside the window fails before users see the warning. The release check lives in
scripts/check-profile-formulae.shand runs from bothrelease-local.shand the signedmacos-release.sh, before any signing or upload.
0.7.3: first install
- Root secured through the OS account’s socket login keeps
unix_socketnext to the new password (IDENTIFIED VIA unix_socket OR mysql_native_password): a fresh Homebrew install and amariadb-secure-installationthat chose socket authentication look alike frommysql.user(both carry the literalinvalid), and replacing the rule would cut off root’s socket clients on the second. The passwordless logins pass--protocol=socket, since an option file withprotocol=tcpwould send them where socket authentication cannot work. - A socket-only root (what Homebrew’s
mysql_install_dbleaves on MariaDB 10.4 and newer) is secured through the macOS user’s own socket account, which that install creates with full privileges, never throughsudo mariadb -u root: the setup already has every privilege it needs without becoming root on the Mac. Only a root whose native password is the literalinvalid(the fresh install’s) is taken as “no password yet”; a failed query, an empty hash (unix_socket only, pam) or a real hash is someone’s choice, and the run keeps saying “root already has a password” and asks, rather than guess. fl_mariadb_root_password_resolvestops with “MariaDB is not running” before it judges any password: against a stopped server every source looks wrong, and the old message blamed the password the user had just passed.- Phase 01 exits 3 on a cancel and
benchbar installmaps that to its own exit 1 with “Cancelled. No bench or site was created; the system dependencies of step 1 stay in place”: 0 made the caller register a bench that does not exist, and 2 is taken (the password), so a cancel needed a code of its own between the phase and its caller. The message says what phase 00 left behind rather than “nothing was changed”, which was false after a first run.installexits by hand (cmd_install || exit $?) so its codes bypass the ERR trap; the “Last command failed” line is phase output the step summary already shows. fl_on_errorturns an incidental exit 2 into 1: the ERR trap exits with the failing command’s status, andgrep,awkand usage errors use 2, which AGENTS.md and the app read as “MariaDB root password unknown”. Onlyfl_die ... 2may say that.fl_mariadb_root_password_resolveverifies the environment password before trusting it, likefl_mariadb_root_setup: an old shell with a staleMARIADB_ROOT_PASSWORDwas failing the site step while the Keychain had the right one.- The up front sudo reason for wkhtmltopdf is added only when the package will reach the installer in this run (
fl_wkhtmltopdf_will_install): without a terminal and--yesevery question in phase 00 is “no”, so the package is skipped and the password prompt was for nothing, on every re-run. - The phase scripts take
OFFLINEfrom the environment (andBENCHBAR_OFFLINE) instead of resetting it, andbenchbar installforwards--offline: one switch for every network check, as the docs already claimed. - A bootstrap that fails because launchd has the label disabled (119) runs
launchctl enableonce and retries: the disabled list survives a plist rewrite, so a reinstall could never load the agent on such a Mac.
0.7.3: profile guards
FL_PROFILE_SOURCE(flag, team, stored, detected, default) is set where the profile is chosen, infl_context_init, and onlydefaultwith anapps/frappepresent counts as a guess (fl_profile_is_guess): a fresh install has no frappe yet and the default is right for it. The guess makes the env checks warn without an action andact_env_rebuildrefuse: an env rebuilt with the wrong Python breaks a working develop or v17 bench, and the fix is a sentence (install --profile), not a guess.- The source is not a new key of
doctor --jsonyet: the app’s fixtures (macos/BenchBarTests/Fixtures) pin the exact key set and this run does not touchmacos/; the Swift follow-up addsprofile_sourceto both sides at once. Until then the header and the env check messages carry it. chk_bench_versionclassifies before it offers:command -v benchempty or exit 126/127 is the CLI’s venv (reinstall with its owner’s tool),No module named 'X'with X insites/apps.txtis that app’s, and only the rest keepsenv_rebuild. The old mapping of every failure to a rebuild moved a healthy env aside for a broken~/.local/bin/bench.act_env_rebuildrefuses while the bench runs rather than stopping and restarting it: a repair that quietly runsdownanduparound a minutes long rebuild is a bigger surprise than “stop it first”, and the three commands are one line in the fix.setuptools<70is installed into a v15 env after the rebuild and offered as theenv_setuptoolsaction: honcho and bench on Frappe v15 importpkg_resources, which setuptools 70 dropped, and a Python 3.12+ venv ships no setuptools at all. v16 does not import it, so the check skips there.bench_minis a new last column ofconfig/release-profiles.tsv(5.22.0 for v15, 5.25.0 for v16, conservative: the bench that first handled each profile’s Python); an older bench is a warning with the owner’s upgrade command, not a stop, since the bench may still work. Abenchon PATH that does not run (bench --versionfails) stops the installer with the reinstall for its owner, where beforebench initfailed with a stack trace.
0.7.3: process groups and locks
- Long and timed commands are started under
set -mand the flag is cleared right after: macOS bash 3.2 has nosetsid, and job control is what gives a background job its own process group; perl’ssetpgrpwould do the same with one more process per command. Signals go to the group (kill -- -PGID) with the leader as the fallback. - benchbar traps TERM, INT and HUP itself, forwards them to the running command’s group and to the phase script in the foreground, and only then exits so the EXIT trap releases the lock: a lock released while pip still ran let the next run start into a half-written env.
- A stale lock is reclaimed by
mvto a private name, neverrm -rfthenmkdir: a rename either succeeds for one process or fails for the rest, so two reclaimers can no longer each delete the other’s fresh lock. An empty lock younger thanFL_LOCK_GRACE_SECS(5) is a run between its mkdir and its pid write, and is refused as “starting” rather than reclaimed. - The bench’s own lock lives at
<bench>/.benchbar.lock, outsidesites/and next to the other files benchbar writes there (Procfile.lean, the runner): the state folder differs by install kind, so only a lock in the bench itself is one every CLI meets. It is taken once the bench is known (fl_context_init) and only by runs that already hold the state lock, so read-only commands and dry runs take nothing. upandrestartdrop both locks after the kickstart and the state write: the wait for the ping only reads, and a lock held for the 45 second wait refused every other bench’srepairandsite addmeanwhile.- The MCP server starts the CLI with
start_new_session=Trueand on a timeout sends SIGTERM to the group, waits a grace period for the CLI’s EXIT trap (the setup Redis stops, the lock goes), then SIGKILL: Python’s own timeout killed benchbar alone and the next tool call reclaimed the lock under a still runningbench get-app. fl_kv_setbuilds the new file withmktempand takesFILE.lck(mkdir) around the read-modify-write, with a short bounded wait: the fixed.tmpname lost keys when the app’s status poll and a repair wrote one file at once. The wait is bounded so a lock left by a killed writer never blocks for good.- The phase scripts of
installrun as a background child the CLI waits for, with<&0so they keep the terminal’s stdin (a background job of a shell without job control gets/dev/null, and the install could not ask for a password); the signal forwarded to them is always TERM, since such a child has SIGINT ignored and could not be interrupted. A signal that lands beforeFL_RUN_PGIDis set falls back to the job table (jobs -p), each job being its own group. - A stale lock is reclaimed under a second mkdir guard (
<lock>.reclaim) and only after reading the pid again: a reclaimer that judged a pid dead a moment ago could otherwise rename away the live lock another reclaimer had just made. A guard older than the grace period is a dead reclaimer’s and is removed. fl_run_longpolls for a stopped command (stateT): in its own process group a command that reads the terminal is stopped by SIGTTIN and would wait for ever behind the spinner;bench get-appruns withGIT_TERMINAL_PROMPT=0for the same reason.- Log and backup names end in the pid: a second run in the same second (the app’s and a terminal’s) truncated the first’s log and overwrote its backups. One stamp names both, so a log and its backups still match.
0.7.3: fail-closed writes
- Write helpers return 1 and leave the target as it was on any failed stage (
fl_backup_file,fl_move_aside,fl_template_apply,fl_rc_block_write): the repair engine runs every action insideif act_...; then, which switches errexit off, so a failure not returned was reported as done. Callers now carry the code with|| return 1; there is no redesign beyond that. - The rc block is written next to the resolved symlink target with the target’s mode and renamed onto it, never
mv TMP FILEonto the link (that replaced a dotfiles repo’s symlink with a 0600 regular file) and nevercat TMP > TARGET(a write cut short left a truncated rc file). The target is resolved link by link in bash (norealpath, which macOS gained only in 13). - The rc file a block was written to is remembered as
RC_FILEin the state:ZDOTDIRis a terminal variable the app and launchd do not have, and without it the app’s doctor and repair would look at~/.zshrcand write a second block there. redis_stopis a question, not a repair: under--yesor without a terminal it is left out of the plan and named as optional, rather than run and reported “done” (or “skipped”) on every run, which kept a second repair from ever saying unchanged.fl_action_is_questionis the one place that knows which actions are questions.- Logs rotate by copy then truncate (
cp -pthen: > file), likefl_arm_startalready did for bench.log: bench.log is launchd’s StandardOutPath and the worker logs are Procfile redirections, so a rename took the writers along and doctor went green over a live file that stayed empty. Three.oldcopies are kept (FL_LOG_KEEP_OLD), oldest removed first; a log copy is the one thing in a bench benchbar deletes, and only its own copies. - The site-under-creation marker lives in
<bench>/logs/.benchbar/creating-<site>, not insidesites/<site>: benchbar never writes intosites/. A marked site makes the next run refuse with the recovery (move the folder aside, drop the half made database), since moving a site folder is also something benchbar never does on its own. lock applykeepsfl_lock_reach_commit’s code in the clone branch and treats anything butequalafter the fallback as a failure, and leaves such a clone out of the build list: the pin is the point of the lock. A detached HEAD that no branch contains is skipped with thegit switch -chint:checkout -bwould have orphaned those commits.- The runner exits 1 when honcho ends with 0 and no stop was requested: honcho’s exit code is its first child’s, and a redis that takes a TERM exits 0, so a bench that lost one process looked cleanly stopped until the next login.
crashedwith exit 1 lets launchd restart it under the crash guard, which is what the guard is for. - The crash guard fails closed: when
.bench-startsor the stop flag cannot be written, the runner pauses withbrokenand exits 0, because a guard that cannot record starts can never stop a restart loop.
0.7.3: secret and input hygiene
benchbar installruns the two sudo steps right after the one prompt and thensudo -k, instead of keeping the credential alive for the whole run: everything after them (brew, pip, npm, yarn, bench get-app) runs third party install scripts on the same terminal, and none of them may find a cached sudo. “Asks once up front” stays true because the two steps are the only ones that need it.- The hosts line is added up front by calling
act_hosts_entryitself, not a copy: the service pass later finds it present and says unchanged, and a declined or refused line is offered again there, as before. - Phase 00 learns the up front result through
BENCHBAR_PDF_STEP(skipped or failed) rather than runningfl_wkhtmltopdf_ensureagain: the second run would ask the same Rosetta and download questions a second time. - The wkhtmltopdf package is copied by root into
sudo mktemp -d /tmp/benchbar-wkhtmltopdf.XXXXXX, hashed there by root and installed from there: the download folder is user writable, so the verified file and the installed file were not necessarily the same./tmpand not$TMPDIR, because the per user$TMPDIRis writable by the user, who could then swap the whole folder;/tmpis sticky, so only root can rename or remove a folder root created there.sudo rm -rfgets only that fresh folder. - The passwords leave the environment with
export -nright after argument parsing inbenchbarand in both phase scripts, andbenchbarpasses them to phase 00 and 01 on their own invocation line: the shell variables stay readable in process (the Keychain, theMYSQL_PWDchecks and the stdin injector keep working) while no other child sees them. bash 3.2 supportsexport -n. FL_ASSUME_YES,FL_MAKE_DEFAULTandFL_PORT_PLAN_APPROVEDstart at 0 inbenchbarand the phase scripts and are set only by flags;benchbarpasses--yesand--make-defaultto the phase scripts as arguments instead of exporting the variables. The in processFL_ASSUME_YES=1 act_hosts_entryofsite hostsis unchanged: it is the CLI answering its own question after asking once for all lines.mariadb-passwordwithout a terminal and without--yesfails with a message naming the flag, instead of the generic “not a terminal” line: an agent or a script that forgot the flag should learn what to pass, and nothing in the environment may stand in for it.- The Keychain item is written through
security -iwith the password quoted for that parser (backslash and double quote escaped) and checked by reading it back, and it names-T /usr/bin/securityexplicitly: the ACL then says what it does, and SECURITY.md documents the limit (any program as the user can read it throughsecurity) and how to remove the trust.-T ''was rejected because every CLI read would then prompt, which--yesruns and the tests cannot answer. - The mock sudo models the cached credential as a file (
-vand a typed command create it,-kremoves it,-nneeds it), and the harness starts with it present, like a developer who typed sudo a moment ago: the hygiene test removes it first, so an “ok” probe from a mock proves that third party code ran while the credential was live. - One set of redaction rules,
fl_redact_streamin report.sh, serves the report andlogs --json: the report applies it one kind at a time (one sed per kind) so REDACTIONS.txt can count each kind; logs —json applies it as one sed. The key rule runs first, because it knows where a value ends; the value shapes catch what has no key. - The email rule keeps
user@host:(an SSH clone URL such asgit@github.com:org/app.git) by requiring what follows the domain not to be a colon: masking those would hide the one thing a failedget-appreport needs. - An unquoted value after a credential key is masked to the end of the line (or to the next
, ; & } " )), so--password=x ymasks the rest of the argv too: over-masking an argv is cheaper than leaking a value with a space in it. logs --jsonis redacted andlogsis not: the JSON goes to scripts and agents (the MCP tool), the plain text is the user’s own file, whichtail -fstreams. The cap of 2000 lines matches the MCP schema’s maximum and is enforced twice: the server refuses a value outside the schema’s 1..2000 before building the argv (a tool error that names the bound, since the MCP hardening), the CLI clamps and saystruncated_to.- URL credentials are masked in
fl_logitself (every line that reaches the run log), inFL_LAST_COMMAND, in the captured output a failed command leaves (git names the URL it could not reach) and infl_die’s hint, rather than at each call site; the check is acaseon://*@, so the common line costs no process. - A bench path (and the default site) is validated once, in
fl_render_all, withadoptand phase 01 checking earlier so the refusal comes before the plan: the runner’sBENCH="...", the plist’s<string>andpkillpatterns all carry it. The refused set is" \ $ \< > &` and control characters; spaces and apostrophes are common on Macs (“Bob’s Bench”) and stay valid, so a site with one renders byte for byte as before. - Template escaping is a second guard, not the guard:
#@quoted KEY...in a template names the values that sit inside bash double quotes (escaped" \ $ \``), a.plisttemplate XML-escapes every value (& < > “). Neither changes a value without those characters, so installed files staycurrent;’is left alone in XML (element text needs no’) so the plist's WorkingDirectory is still the path itself forfl_plist_working_dir`. - The
#@quoteddirective lines bump the template versions (bench-run.sh v7, shell-helpers v2) although a directive is stripped on render and the rendered body and its hash are unchanged for every existing input: a template change bumps its version, without exception, so a reader of the header can tell which template wrote a file.repairrewrites the runner and the helper block once, with identical content. - Every
${var//pat/rep}replacement in templates.sh is a plain variable, never a quoted literal: bash 3.2 to 4.2 keep the quotes of a quoted replacement (BENCH=""...""under/bin/bash). bash 5.2’s patsub_replacement reads an&or\in the replacement, an expanded variable included, as the match or an escape, so ui.sh turns the option off for the whole program (shopt -u patsub_replacement 2>/dev/null || true; bash 3.2 has no such option): every${//}in this bash 3.2 code base means text. test-input-hygiene renders underBASH_COMPAT=32as well to hold both. BENCH_NAMEandLABELare not#@quotedin the runner: they also sit inside write_state’s single quoted printf format, where a backslash would be literal; both are built from a sanitised name, and the path check refuses the characters anyway.- Doctor does not die on a bench path the templates cannot carry: it reports
bench_pathas a FAIL (the app pollsdoctor --json, and AGENTS.md promises doctor is safe any time);fl_require_plain_benchrefuses in the commands that write the files or start the runner (service, repair, adopt, install, up, restart, autostart, ports apply, site add and default). - The hosts validator accepts an IPv6 zone id (
fe80::1%lo0 localhost, shipped by Apple for years) and a CRLF line: the check is for a garbled rewrite, not for policing a file that works. - Token shapes need a word boundary on the left (
sk-,gh?_,github_pat_,xox?-,AKIA):/tmp/task-2026-10-03-worker-default-queueand.desk-sidebar-item-container-wrappercontainsk-and are not secrets. pwdcounts as a credential key only as a keyword argument inside parentheses (connect(pwd=...)):PWD=/Users/...in an environment dump is a path, and masking it would hide what a report is for.- The email rule needs a dotted domain ending in letters and sets
.localaside first:worker@Bobs-MacBook.local(an rq worker name) androot@localhostare not addresses. benchbar install --dry-runhands phase 00BENCHBAR_PDF_STEP=dry-runafter the up front step printed the wkhtmltopdf plan, so the plan appears once; a real run keepsskippedandfailedas before.- The hosts rewrite runs awk as root and pipes into
sudo tee /etc/hosts.benchbar.new, so the new content never sits in a file this user could swap between the write and the copy; the markers and the line go throughENVIRON[]rather than-v, which processes backslash escapes. The check reads the new file as root too (a strict umask could hide it), andchmod 644comes before themv: tee inherits the user’s umask, and an unreadable/etc/hostswould break name resolution for every process on the Mac. - A hosts file with a line that is not “address names” is refused, with the line named: the check exists to catch a garbled rewrite, and a file that fails it is not one to rewrite blind. Appending a new block still uses
sudo tee -a, which has no copy to race. - A
sites/folder whose name is not a site name (evil\nx) is skipped with a warning at the top ofact_hosts_entry,fl_hosts_add_namesandfl_hosts_remove_name, beforefl_confirmandfl_sudo_begin, so the name never reaches sudo, however the function was entered. fl_kv_decode_vdecodesprintf %qwithouteval: plain text as is,$'...'by hand (the octal\NNNof a non-ASCII byte in the C locale,\', which%bdoes not know, and the usual escapes), a backslash form by dropping each backslash,'...'as its inner text, and anything else as the literal line. A hand writtenKEY=~orKEY=$HOMEis therefore the text, not the expansion.
0.7.3: MCP hardening
- Every
tools/callruns on a daemon thread andinitialize,pingandtools/listare answered inline by the reader thread: a one hourbenchbar_app_addotherwise blockedbenchbar_statusand every other call. Replies are written under one lock, one JSON line each, so threads never interleave on stdout. - Actions (
up,down,restart,benchbar_app_add) take one lock in the server, read tools run side by side: the CLI’s own lock would refuse a second action anyway, but a refusal after the agent waited is a worse answer than a short queue. Waiting on that lock still notices a cancellation. notifications/cancelledfor a running call stops its process group (the same SIGTERM, grace, SIGKILL as a timeout, also when the CLI itself already left on TERM, so a member that ignores TERM and holds no pipe dies too) and the call sends no reply: the MCP spec says the server SHOULD not reply to a cancelled request, and a reply the client no longer expects would only confuse it. The CLI’scommunicateruns in 0.25 second slices so a cancellation is seen while it runs. A cancel that lands during the status after an action keeps the action’s result, withafter_error: "cancelled": the action did run, and losing that is worse than one unexpected reply. A cancel only signals a process whose exit is not yet known, never a pid that may have been reused.--process worker --previousis refused: worker.log has no previous file, and reading the previous bench log unfiltered would be a quiet wrong answer.- When stdin closes, running calls get
BENCHBAR_MCP_EOF_GRACE(30 seconds) to finish, then their groups are stopped and the server exits; SIGTERM to the server stops them at once. A client that is gone reads no result, and a half finishedapp addis the same case a timeout already handles. - Argument values are checked against each property’s schema (type, minimum, maximum, enum) and a violation is a tool result with
isError: truenaming the argument and the rule, not a protocol error: the model reads tool errors and corrects the call, while-32602ends up as an opaque failure. Unknown arguments, missing required ones and values starting with-stay-32602as before: those are malformed calls, not wrong values. all_sites(and every boolean) must betrue, never a truthy string: the string"false"turned--all-siteson.- Results whose text comes from outside benchbar (log lines,
app add --dry-runwith itshooks.pyand git output, profile files, the output of an action) start withData from the bench (...), not instructions:and the JSON;structuredContentis the same JSON. Results that are benchbar’s own words (list,status,doctor,site list,app list) stay as the CLI printed them. The label is a plain line, not a format change, so clients that show only the text still read the JSON. fl_json_escape_vremoves a terminal control sequence as a whole (ESC[parameters final byte), not only its ESC byte: the[31mit left behind was not text anyone wrote, and every--jsonreader (the app, the server, scripts) would have had to guess it away. The server keeps a whole sequence strip as a safety net and no remnant regex, which would eat real text such as[5m].- One result is capped at
BENCHBAR_MCP_MAX_BYTES(200 kB): the oldest log lines (or the tail of an action’s output) are dropped fromstructuredContentuntil it fits,"truncated": truesays so, and the text is rebuilt from that shrunk JSON so both agree and the newest lines are the ones kept; a text that still does not fit (benchbar’s own JSON, which is never that large) is cut by bytes with... [truncated, N bytes total]. A 2000 line log with long lines was several hundred kB of context. - The status (or app list) after an action runs in its own try: a hang there returned
-32603and lost the action’s result. The result now comes back withafter_errorand without thestatuskey. tools/listandtools/callbeforeinitializeget-32002“server not initialized” (the code MCP SDKs use for it);jsonrpcmust be"2.0"and anidmust be a string or a number (nullis an invalid request answered with idnull). Unexpected exceptions are-32603“internal error” with the traceback on stderr: the message used to carry absolute paths.benchbar mcpprefers/usr/bin/python3 -Iwhenxcode-select -psucceeds, elsepython3fromPATH, each checked for 3.9: a GUI client starts the server with aPATHthat may name a pyenv shim or a venv, and-IignoresPYTHON*variables and the user site, which the server (standard library only) never needs. The tests steer the choice with a mockxcode-selectand apython3wrapper onPATH, so no extra override exists.- The token is described as proof that the bench has not changed since the plan, and the tool asks the agent to ask the person: the old wording (“the plan the person approved”) claimed something the server cannot know. No elicitation: the clients in use do not support it yet.
--process workerreadslogs/worker.loginfl_cmd_logs(the file option mapping), unfiltered, and--worker-errorkeeps its file:Procfile.leansends the worker there, so the honcho filter never matched a line. The MCP tool gets afileenum for the four files and losesworkerfromprocess, so the schema validator refuses what never worked.site list --jsonaddsdb_nameanddb_portper site, read fromsite_config.jsonandcommon_site_config.jsona line at a time like the other config reads, never the password;listandstatusleave them out because they are polled every few seconds.benchbar_app_listrunsapp list --json --no-sites: a read tool must not ask the database, so the sites come from the last read.
0.7.3: state folder migration
- A new folder that holds nothing but
bin/(the link BenchBar.app makes at launch) and a.DS_Storecounts as not moved yet, and the installer checkout’s state still moves in: the app registers its CLI before the first CLI run, so “the new path exists” alone would leave every installer state behind. The test is a glob over the folder, no process, and runs only while a real.benchbaris still in the checkout. - The state folder is renamed into place whole, not its files moved in one by one: a rename is atomic, a file by file move cut short would pass for the state.
bin/goes into the checkout’s folder first, the emptied new folder is removed withrmdir(atomic; its.DS_Storefirst, a Finder file and an empty folder, never state), and the rename bringsbin/benchbarback with it. Every intermediate layout is one the next run finishes from: abin/already in the checkout’s folder is part of the state folder, never “a state of its own”, and a secondbin/the app made in between waits inside the guard. Armdirthat fails (something appeared) undoes thebin/move and leaves the retry to the next run. - The app’s link is away between the
mvofbin/and the rename: a process already started resolves itself through the retry infl_resolve_self(up to 2 s in 0.1 s steps, and a folder withoutlib/is rejected), but a start that falls inside that window of a few milliseconds fails with “No such file” (observed under the three-at-once test before the retry; a start cannot be made to wait before it runs). A normal start resolves at the first try and starts nothing for it. - The one time move is serialized with
mkdir ~/.local/state/.benchbar-migrating(atomic, a pid file inside, like the lock) taken before any check; losers wait up to 5 s in 0.1 s steps for it to go, then use whatever exists. A guard is stale when its pid is dead, when it is older than 60 s whatever the pid says (a pid number can be reused;stat -c %Y || stat -f %manddate, only when a guard exists), or when it has no pid by the end of the wait. A stale guard is claimed with onemvto.stale.<pid>(atomic, so of two waiters that both saw the dead pid exactly one reclaims it; the other waits once more), then its pid file, the app’s link inbin/(a symlink only) and empty folders go; anything else stays aside, never deleted. An older CLI knows no guard, so the recovery for a link or a folder that landed inside the new folder stays. - The fast path tests
-don the guard once (no process when there is none) and sweeps a stale one the same way: a run killed after the rename would otherwise leave a guard that costs every later first run its wait, and thelnthe fast path re-makes is the only other thing such a run leaves undone. - The app’s helper block prefers a standalone copy that hands off (the copy that handed off to this run, then Homebrew’s, then the installer’s) over the app’s own link: the link goes with the app, the copy runs itself once the app is gone. A recorded copy that hands off is still kept whichever copy runs repair, so the block is rewritten once, not on every switch.
- The app’s Second CLI check warns on a copy newer than the app’s CLI with the app update as the fix, not the copy’s downgrade: every copy hands off to the app’s CLI, so the app is what is behind.
- The split state check for the app offers
mv ~/.local/share/benchbar ~/.Trash/only when the installer’s CLI is still there, andmv .../.benchbar ~/.Trash/otherwise; both say to move out what is still needed first, since the folder may be an unmerged state and not only what an older CLI wrote since the move. where --jsongetsstate_dir_real(cd && pwd -P, noreadlink -f: macOS before 12.3 has none) andhandoff_from_version; doctor —json gets no new top level key, since the app’s test fixtures pin that set.install.sh --uninstallresolves~/.local/state/benchbarand keeps the checkout when the link leads into it (-ef, or the link text underBENCHBAR_HOME): the state of every CLI on the Mac is behind that link, andrm -rfof the checkout would take it.
0.7.0: Homebrew
- The install kind comes from SCRIPT_DIR as a string:
*/opt/benchbar/libexecor*/Cellar/benchbar/*/libexecis homebrew, exactly~/.local/share/benchbaris managed, a folder with.gita checkout, anything else other. No process, because status runs it on every poll of the app, and no variable from the formula’s wrapper, because every child (bench, honcho, a nested benchbar of another kind) would inherit it. - Managed is that exact path, never
BENCHBAR_HOME: that variable belongs to install.sh and self-update, and a test that points it at a checkout must never move the checkout’s state. FL_SELFis the path every generated file and printed command names:<prefix>/opt/benchbar/bin/benchbarunder Homebrew, the Cellar form mapped to it, because brew upgrade keeps the opt path and brew cleanup deletes the Cellar one. For managed, checkout and other it stays${SCRIPT_DIR}/benchbar, so 0.7.0 rewrites no existing helper block.FL_SELF_DIR(<prefix>/opt/benchbar/libexecunder Homebrew) is used for the two fix lines that name a phase script, for the same reason; SCRIPT_DIR stays the folder the running CLI reads lib/, templates/ and config/ from.write_cli_linklinks~/.local/bintoFL_SELFandcli_linkaccepts it: under Homebrew a link into the Cellar would dangle after the next cleanup. For the other kinds the target is unchanged.- Managed and homebrew keep state in
~/.local/state/benchbar; a checkout keeps./.benchbarandFL_STATE_DIRfrom the environment still wins. Not~/Library/Application Support/benchbar: on a case-insensitive disk that is the app’sBenchBarfolder, which the cask’s zap trashes. - The path is fixed, not
${XDG_STATE_HOME:-~/.local/state}: BenchBar.app opened from Finder, an MCP client and launchd start the CLI without the variables of~/.zshrc, so a variable set only in the shell would give them another registry, other settings and another lock.FL_STATE_DIRstays the one override. - The installer checkout’s
.benchbarmoves with onemvand a symlink back, never a copy: an install.sh CLI still on the Mac then reads the same state and takes the same lock, so two CLIs cannot fork the benches. If the symlink cannot be made, the move is undone. - The
mvgoes straight to the new path, not through a temporary name: on one volume it is a rename, so another run sees either the old folder or the new one, never neither, and creates no empty state in between. - When
~/.local/stateand the installer’s folder are on different volumes (onestatof both, on the first run only), nothing moves: the new path becomes a symlink to the old folder. A cross-volumemvcopies, and a copy cut short would pass for the state on every later run. - The move tests again right before the
mv(after themkdirprocess), and amvthat put another first run’s link inside the new folder moves it back: two first runs at once (the app’s status and list polls) leave one state and one link. - When
ln -smeets a folder an older CLI made again in that gap, ln puts the link inside it; that link is removed, the folder stays, this run keeps the moved state, andcli_duplicatenames the split. No lock is created in the old folder before the move: a 0.6.1 run makes its logs folder before it takes its lock, so the lock would not stop it. - A run that finds the new folder but no link at the old path (a run stopped between the
mvand theln) makes the link again when the installer’s folder is there: a few tests on every run, no process. - The move is deferred while
lock/exists in the old folder (the rule of the.frappe-localrename): that run, and every run until the lock is gone, uses the old folder as is. A stale lock is reclaimed and released by the next mutating run, and the run after that moves. - The move happens when state.sh is sourced, like the
.frappe-localrename, so the first status poll after an upgrade may do it; every later run pays one-etest and starts no process. - An existing new folder is never merged into: when both exist, the old one stays where it is and is no longer read.
.gitignoresays.benchbarwithout the slash: a pattern with a slash matches only a directory, and the move leaves a symlink in the installer’s checkout (checked withgit status).~/.local/binstays on the helper block’s PATH under Homebrew too: pipx and uv installbenchthere. The template is unchanged, so no helper block becomes outdated for it.- The runner template marks
CLI_VERSIONwith a new#@unhasheddirective: that key is hashed as its__CLI_VERSION__token, so a new version alone (every brew upgrade) no longer makes every runner outdated. The value still lands in the runner, which writes it asstate.json’scli_version: the version that wrote the runner. The 0.6.1 runners are outdated once, since the hash changed. - The report’s
versions.txtnames the install kind,FL_SELFand the state folder: after the move, support needs to know which CLI ran and where its logs are. tests/test-homebrew.shcopies the CLI into a keg underMOCK_BREW_PREFIX(what the brew mock answers for--prefix): a symlink into the repo would resolve back to the checkout. Both shapes of the formula’sbin/benchbarare covered, the wrapper and a symlink into the Cellar.FL_SELF_PREFIXis the Homebrew prefix cut fromFL_SELF, so self-update and the link checks find the Caskroom, brew and<prefix>/binwithout abrew --prefixprocess.- Under Homebrew
cli_linkis OK without~/.local/binlinks and repair never makes one: brew’sbinis on PATH, and a new link would only be one more thing to go stale. - A link there that leads elsewhere is pointed at
FL_SELF, not removed: the app’s CLILocator, a saved Settings path and user scripts that remember~/.local/bin/benchbarthen reach the Homebrew CLI. - The old link is kept as a symlink with the same target (
fl_backup_link), not withcp, which would copy the file it leads to, or fail when that is gone. - A link to Homebrew’s benchbar by a stable path (opt
bin, optlibexec,<prefix>/bin) is current for every kind, and with the formula installed no kind makes a missing link: otherwise the installer’s copy, or a developer checkout, would point~/.local/binback at itself on its next repair and shadow brew. A Cellar path is never current. - The installer’s copy can still rewrite the helper block for itself when someone runs its repair by full path;
cli_duplicatenames that copy instead of a special case inhelpers. cli_duplicateoffers the Trash for the installer’s folder only when its.benchbaris the symlink or gone: while a run holds the old lock (or the move failed, or the folder is on another volume) that folder holds the state, and the fix says to rerun repair or to keep it.- For a git checkout
cli_duplicateno longer says Homebrew’s takes over: a checkout’s./.benchbaris never moved, so Homebrew’s CLI would start without its benches. The fix isbrew uninstall benchbar. Only the managed kind, or a checkout whoseFL_STATE_DIRis~/.local/state/benchbar, shares the state. - Under Homebrew,
cli_linkandhelpersleave a link or block that runs a checkout with its own real.benchbar(a warning without action, and the repair actions skip it): pointed at Homebrew’s CLI,benchupwould run without that state and could pick another bench. The installer’s folder is not such a checkout, since its state moves. cli_duplicateandapp_copiesare warnings without a repair action: which copy stays is the user’s call, and repair never deletes a CLI or an app. Under Homebrew the other copy is~/.local/share/benchbar/benchbar; for every other kind it is<prefix>/opt/benchbar/bin/benchbar, so a developer checkout next to brew warns too.app_copiesis a system check (it is about the Mac, not the bench) that readsFL_APP_DIRSlikecleanmymac. With the cask’s receipt the fix names the~/Applicationscopy, the installer’s default, as the one to trash.helpersfails when theBENCHBARthe block records is gone, whatever the block’s hash says: every helper fails then. A different path that still exists stays the outdated warning.- self-update takes the homebrew kind from install-kind.sh, before its
pwd -Pcompare, which sees the Cellar folder and says other. It runsbrew upgrade askysh/tap/benchbarafter the usual question, with the prefix’sbrew(else the one on PATH), byexec, since the upgrade replaces the keg it runs from. It never runs install.sh. - Under Homebrew
update_availablecompares only the CLI: the app updates itself or comes from the cask, which this command does not touch, so an older app alone is a note, not an update. The JSON keeps every field;installgainshomebrew,app_diris null andcli_diris the optlibexec. - install.sh finds Homebrew’s copies through
brew --prefixof the brew on PATH only, never a hardcoded/opt/homebrew: a shell without brew cannot have installed benchbar with it, and the suite must never see the real one. The guards apply to--dry-runtoo, so the plan shows what a real run does. - install.sh prints the repair line only while the installer’s CLI is still there, and by its absolute opt path:
~/.local/binmay still lead to the old copy in that shell. - The detached checkout check:
symbolic-ref -qexit 1 is detached (any other failure is left to the pull’s message), and clean meansstatus --porcelain --untracked-files=nois empty, because a 0.7 CLI’s.benchbarsymlink is untracked in a checkout whose.gitignorepredates 0.7. It switches to main, as asked, not to the remote’s default branch. - install.sh
--uninstallnever removes~/.local/state/benchbar, it names it:rm -rfof the checkout removes only the symlink, and Homebrew’s CLI may share the folder. - With Homebrew’s CLI installed but never run,
--uninstallmoves the checkout’s real.benchbarto~/.local/state/benchbarbefore it removes the checkout, through a temporary name in that folder so a cut short cross-volume copy never takes the final name. When Homebrew’s state folder already exists it merges nothing and keeps the checkout. uninstall-service --alltakes the benches from thecom.benchbarplists’ WorkingDirectory, as install.sh--uninstalldoes, not from the bench registry: the agents are whatbrew uninstallleaves running. One question, one lock and one log for the whole run.- Each bench of
--allruns in a subshell betweenset +eandset -e, not underifor||, which would turn errexit off inside it: a bench launchd will not let go of fails alone, and the others are still uninstalled. The ERR trap stays out of it because functions do not inherit it (benchbar sets no errtrace). - With no agent left but a helper block,
--allremoves the block after asking: afterbrew uninstallit would point at a missing CLI. --allalso removes the block once after its loop: a folder that is no bench any more loses only its agent, so with only such folders no per bench run would remove it. It is a no-op when a bench’s run already did.whereprintsinstall,self,state_dir,app_versionandapp_path, not SCRIPT_DIR: under the symlink shape that is the Cellar folder, and the formula’s test asserts thatwhereshows none.- The git mock emulates
symbolic-ref,statusandcheckoutonly in checkouts its ownclonemade (.git/mock-checkout): other tests’ fake apps have a.git/HEADtoo, and their calls must keep going to the real git. - The formula installs
benchbar-cli-<version>.tar.gz, a release asset made bygit archiveof the tag (scripts/cli-tarball.sh), not GitHub’s/archive/refs/tags/tarball, whose bytes GitHub does not promise to keep: a changed sha256 breaks every install. A pathspec keeps it to what the CLI runs (no macos/, site/, docs/, tests/), andgzip -nmakes one commit give one sha256. cli-tarball.shand release.yml refuse a version thatFL_VERSIONdoes not say:brew testand livecheck compare against it, and a formula whose CLI reports the previous version would look like a failed upgrade.- The tap is updated by
homebrew-tap.ymlonrelease: published, not by release.yml: a draft’s asset URLs answer 404, so a formula merged at draft time breaksbrew installuntil someone publishes. - The formula goes to the tap for every release, the cask only when the release has
appcast.xml(the Developer ID path): the CLI needs no signature, and an ad hoc DMG through brew would meet Gatekeeper without the Open Anyway steps. - The workflow hashes the published files after checking them against
SHA256SUMS, then installs both from the tap clone and runsbrew testbefore the pull request: what merges is what was installed. The pull request merges itself (gh pr merge --auto) once the tap’s checks pass. homebrew-render.shtakes--urlso CI can install the formula from afile://tarball of the checkout in a localaskysh/tap(brew tap-new --no-git); the cask’sdepends_on formula: "askysh/tap/benchbar"then resolves to that tap without the real one.homebrew-formula.ymlis path filtered and nightly, never a required check: brew style, audit and install reach the network, and a flaky mirror must not block a pull request.- Its doctor run uses a stub bench (
sites/common_site_config.json) in an empty HOME: doctor stops at “No bench” without one, and only the CLI’s own checks (cli_link, no Cellar path in a fix) matter there. - The formula has no bottle and no
post_install,serviceorcaveatsthat run anything: it copies files intolibexec, and the person runsbenchbar install(sudo once) themselves.
0.6.0: app add over MCP
- The 0.5 rule “no install behind a tool” now has one exception,
benchbar_app_add: Akash’s call on 2026-09-29. The app’s installer already takes a pasted GitHub URL, so an agent should too, gated by a plan and a token instead of a terminal. Repairs, bench installs, services and anything withsudostay out of MCP. - The plan is its own read tool (
benchbar_app_add_plan,app add --dry-run --json); applying needs the plan’s token, likeports apply. The token is a SHA-256 of the plan JSON (withoutcli_version) plussites/apps.txt, theapps/folders and the site list, so a bench that changed between the review and the apply is refused before anything runs. --applyrecomputes the plan under the CLI lock and compares tokens there, so two agents cannot both apply one plan. It runs with stdin on/dev/nulland never asks: the plan already disclosed every required app it clones.- Required apps are read before any change from a
git clone --depth 1 --filter=blob:none --no-checkoutinto a temp folder outside the bench,hooks.pyonly, then the folder is removed. It works for any host the user’s git can read, where the GitHub API or raw URLs would not. Every required app the registry or the team profile resolves is followed the same way; one that neither resolves blocks the plan (can_apply: false). - Without
--name, the app is the package folder that holdshooks.py, since that is the folder bench creates. - Every remote call of the plan runs under
GIT_TERMINAL_PROMPT=0, SSHBatchModeand a timeout (30 seconds forls-remote, 120 for the shallow clone), so an unreachable or private repo givesreachable: falseand a reason instead of a hang. - The steps are: clone the required apps (deepest first), clone the app, one build,
install-appper site, restart only when the bench is running. The restart is always in the plan and decided at apply time, so starting or stopping the bench does not invalidate a token. - The MCP server gives
benchbar_app_adda one hour limit (the plan five minutes, every other tool three) and streams nothing: the call returns the output, the--jsonresult and a freshapp list --jsonat the end. Any argument value that starts with-is refused, so a URL can never become a CLI option. - A URL with a user name or token is still refused, not stripped: it would land in
.git/config.
Folder discovery
- Scanning is read only and registration only remembers canonical paths: finding an existing bench must not silently create a service, start processes, or change its ports.
- Recursive scans stop at a found bench and skip symlinks, hidden folders, dependencies and test fixtures: this avoids cycles, nested example benches and expensive dependency traversal.
- Same-name services use a canonical-path checksum suffix when needed, while retaining labels already owned by the bench: separate projects often use the same
frappe-benchdirectory name. - Port reservations inspect the service’s working directory instead of deriving its label: both legacy names and path-specific labels must reserve the correct bench’s ports.
0.5.5: about, help and docs links
benchbar docschecks the page with onecurl -I(3 seconds) before opening it: a 404 opens the start page with a warning, and no answer (offline, DNS, a slow network) opens the page anyway, so the check can only improve on a plainopen, never block it.benchbar docs --printprints the URL without opening it or touching the network: agents and scripts want the link, not a browser window.- Topics are short names with aliases (
doctorandrepair,teams,profilesandlock), kept in one table inlib/frappe-local/docs.sh; an unknown topic exits 1 with the list, so a typo never opens a wrong page. - The doctor
see:line goes under every[FAIL], afterfix:, and never under[WARN]or in--json: AGENTS.md promises parsers the existing lines byte for byte, and a new line starting withsee:is ignored by anything that reads[FAIL]andfix:. Every check id has an anchor in the doctor guide, so no per check table is needed. --versionkeepsbenchbar X.Y.Zas the first line and addsBenchBar app X.Y.Z (path)only when an app is installed: the app,install.shand the tests read the first line. The app version is read from the XML Info.plist with awk, the helperreportalready used, instead ofplutilordefaults, so the tests can fake an app with a small file.report --jsonsends the human lines to stderr and prints one object on stdout, withschema_versionandcli_versionlike every other document;redactionscounts the lines where something was replaced (the sum of what REDACTIONS.txt lists).--jsonand--printtogether are refused.- The bug report form’s version fields are prefilled by the ids
bug_report.ymluses:macosandversion.
0.5.5: repo foundation
- The Code of Conduct’s contact is a private advisory on GitHub until a contact email exists: the git address is a noreply address that cannot receive mail, and an advisory is private to the maintainers.
- The AI policy welcomes AI assisted pull requests with a one line disclosure instead of banning them: BenchBar is itself built with coding agents (AGENTS.md), and a ban nobody can check only hides the tool.
good first issueitems are kept for people working by hand: they exist to teach the codebase, and an agent solving them teaches nobody.- The bug form requires the
benchbar reportzip and doctor output: those two answer most first questions, and the zip is already redacted. - Blank issues are off and questions go to Discussions Q&A: the issue list stays a list of things to fix.
scripts/social-preview.swiftuses AppKit, not Python: the Mac has no imaging library by default and the repo already builds with Swift;scripts/app-icon.pyonly writes SVG text.- The social preview and the og:image come from one script with a size argument: one design, two sizes (1280x640 for GitHub, 1200x630 for the docs site), no drift between them.
- The runner strip in the preview is cut from
docs/images/runners.png, with the grey tiles made transparent: the frames stay the ones the app draws, without rendering them again. - Dependabot watches
/macosfor Swift: the package list lives inmacos/project.yml(XcodeGen) and the generated.xcodeprojis not committed, so Swift updates only start once aPackage.resolvedis in the repo. - Vouch waits until drive-by pull requests appear: a trust list for a project with one maintainer and few contributors is ceremony.
0.5.5: docs site
- The site reads
docs/in place through a glob loader withbase: ".."insite/src/content.config.ts, not a symlink: Starlight 0.42 supports any Astro loader for thedocscollection, a symlink breaks on Windows clones and in some editors, andentry.filePathstays a real repo path, so “Last updated” comes from the file’s own git history. editLink.baseUrlishttps://github.com/askysh/benchbar/edit/main/site/: Starlight appendsentry.filePath, which is../docs/<page>.mdrelative tosite/, and the URL resolves toedit/main/docs/<page>.md.- Page URLs are the lowercased file paths under
docs/(DECISIONS.mdis/decisions/), set by the loader’sgenerateId, so the URLs the README and the app link to follow the file names. ROADMAP.mdandCONTRIBUTING.mdstay at the repo root with no frontmatter;site/scripts/prepare.mjscopies them intosite/.generated/(ignored by git) before each build with a title from their#heading, their own edit link and last commit date. A missingCONTRIBUTING.mdis skipped, and the sidebar leaves it out.- Links in
docs/stay relative.mdpaths so GitHub renders them; a remark plugin (site/src/remark-repo-links.mjs) turns them into site URLs, and links to other repo files into GitHub links, at build time. - Screenshots in the moved README content use markdown image syntax, not
<img width>: Astro optimises markdown images (webp, a fifth of the size) and ignores raw<img>paths. - The heading slugs come from github-slugger, which keeps underscores, so
### port_clashis#port_clash;tests/test-docs.shchecks a heading for every check id inFL_CHECK_ORDERplusport_block. - The CLI reference documents only flags that
benchbar --helpshows (tests/test-docs.shchecks it).report --out DIRandapp update --jsonwithout--dry-runexist in the code but not in--help, so they are left out until--helpnames them. - Astro and Starlight run under Bun (
bun --bun astro build): Astro 7 needs Node 22.12 or later, and Bun alone avoids a second runtime in CI and on a contributor’s Mac.@astrojs/markdown-remarkis a direct dependency because Astro 7 only runs remark plugins through it. - The docs workflow is the official Astro Pages workflow with its steps written out (
withastro/actionwould runbun installwithout--frozen-lockfileand has no place for the link check). - Docs only changes skip the CLI and app jobs through a
changesjob (dorny/paths-filter) andif:conditions, not a workflowpaths-ignore: a workflow that never starts leaves a required check pending forever, a skipped job counts as a pass. The matrix shards skip step by step on an Ubuntu runner, because a matrix job skipped as a whole reports one check under its unexpanded name. A push tomainskips the same way: the pull request already ran everything, andworkflow_dispatchruns all jobs. - The docs workflow runs on every pull request and skips its build job the same way when no docs file changed, so “Docs build” can be required too. It also runs
tests/test-docs.sh, which the CLI shards skip on a docs only change. - The link check is linkinator over
site/distwith fragment checks on, skipping every external URL: GitHub and the site’s own domain (before DNS) would make the check flaky, and the internal links are the ones a docs change breaks. - The Umami script uses the benchbar.akashmishra.com website of Akash’s Umami, limited with
data-domainsso local previews are not counted, andog:imagepoints athttps://benchbar.akashmishra.com/og.png, which the build copies fromdocs/images/og-image.pngwhen that file exists. - The docs site sends three custom Umami events from
site/public/analytics-events.js:command-copy(the first line of a copied code block),download-click(a link to a GitHub release) andoutbound-click(any other link off the site). They answer what page views cannot: whether a reader copied the install command or went to a download. Query strings are never sent, and no event carries anything a reader typed.
0.5.5: readme
- The README keeps only what a visitor needs to decide and start (why, install, quick start, features) and links four deep pages of the docs site: a 3,200 word README had become the manual, which nobody can navigate on GitHub, and the docs site now has search and a sidebar.
- The README runs about 800 words, not the 1,100 the brief suggested: the fixed order and one line per item leave no room for more without padding.
- The logo and the hero image switch with the reader’s appearance through
<picture>; the icons are exported fromAppIcon.iconwith Icon Composer’sictool(Default and Dark renditions), so they match the app icon exactly.
0.5: app installs
Designed against frappe/bench develop (c9d1250) and frappe version-15; the code moved on since the design, and the code won where they disagreed:
- Everything per bench goes through the existing per bench state (
.benchbar/benches/<name>-<hash>.env,fl_bstate_get/set); the site app cache is a sibling file,<name>-<hash>.site-apps, not the<name>.site-appsof the design. - install-app and migrate run with the bench’s own Redis up (
fl_bench_redis_up/downfrom 0.4), because frappe v16 connects to it during both; the exit handler stops what was started, however the command ends. - Never
get-app --resolve-deps: it readshooks.pythrough the GitHub API (no private repos) and canrmtreean existing app.required_appsis read from the localhooks.pyafter the clone, resolved throughapps.tsv, and cloned after a second plan and question. - Never
get-app --overwrite, and stdin is/dev/null: an app that exists is reported with the manualgit fetchandgit checkout, and a surpriseclick.confirmfails instead of hanging. --skip-assets, then onebench build --app X(or--apps a,bwith the required apps): one build, after every dependency is in place.- Access is checked before any change with
git ls-remoteunderGIT_TERMINAL_PROMPT=0andGIT_SSH_COMMAND='ssh -o BatchMode=yes': a private repo without a key or token fails in a second with a fix line (ssh -T git@HOSTandssh-addfor SSH, which also covers host aliases; the SSH URL orgh auth setup-gitfor HTTPS) instead of waiting on a hidden prompt. - A repo URL with a user name or token is refused: it would land in
.git/config,app list --jsonand later a committed lockfile. Remote URLs are always shown without userinfo. get-appis always given the resolved URL, also for names likeerpnextthat bench could resolve itself: the preflight checked exactly that URL, and apps.tsv entries without a repo map togithub.com/frappe/<name>, which is what bench uses.- The app name is the repo name (lower case,
-as_) or--name; the folder bench actually created is found by comparingapps/before and after, soRavenbecomingravenis followed. - A get-app that fails half way moves the new folder to
.benchbar/backups/<ts>/apps/only whilesites/apps.txtdoes not list it. Once bench has listed it, the folder and the line stay and the fix (bench remove-app NAME) is printed: benchbar never editssites/(AGENTS.md), and doctor’sapps_txtcheck keeps reporting it. The first version removed the line itself; the review before the 0.5 PR reverted that. - A dirty app is left alone by
add(it only installs on sites) but refused byupdate:addnever touches the code,updatemoves it. app updateisgit fetchplusmerge --ff-only, neverbench update: one app changes, nothing is rebased or reset. A diverged branch stops with thegit log --graphto look at; an app ahead of its remote is “unchanged”. On a failure thereset --hard OLDis printed, never run.- A shallow clone is fetched with
--shallow-since=@<HEAD time - 1s>, soold..newexists locally for the changelog and the fast forward check. update --dry-runrunsgit fetch: the changelog needs the new commits. Only the app’s.gitchanges; the working tree, the sites and benchbar’s state do not, and the test snapshots exactly that.- Every site that has the app is backed up (
bench --site S backup) before the merge,--skip-backupopts out: the repo rule is a backup before every change, and migrate is the step that can hurt data. - Which sites have an app comes from
bench --site S list-apps --format jsonwith a 15 second limit per site, cached per bench: it needs MariaDB, and doctor (on a timer in the app),app list --no-sitesand the lockfile check must never need it. A failed read keeps the last good list and says so insites_error. app update --jsonis only the plan (--dry-run --json): the app shows the changelog, then runs the update with--yesand a live log, and a JSON stream of an apply would need its own format.- The doctor checks
apps_txtandapp_branch_policyhave no repair action: fixing either means changing which code runs, a person’s call. --sitestays the global option, soapp install NAME --site Sandapp add X --site Sparse like every other command.- The fake bench of the tests now has an
apps/erpnextfolder: it always listed erpnext inapps.txt, which the newapps_txtcheck rightly fails. app addtakes the app name from the repo name (or--name), never fromapps.tsv, when given a URL:apps.tsvis only the catalog for names.- The git mock passes everything to the real git under
MOCK_GIT_REAL=1, and the bench mock’sget-appthen really clones (--origin upstream, shallow withMOCK_BENCH_SHALLOW=1) from bare repos in the test folder overfile://.MOCK_GIT_LSREMOTE_EXITfakes a private repo. No test touches the network.
0.5: team profiles
- A team profile lives outside BenchBar:
~/.config/benchbar/profiles/NAME.toml, then each folder ofBENCHBAR_PROFILE_PATH(colon separated), so an organisation’s repos and branches never land inconfig/or in this repo, and a team shares its recipe through its own config repo. - Built in profiles are looked up first, and a team file named like one (
v15-lts.toml) is reported as invalid instead of loaded:--profile v15-ltsmust mean the same thing on every Mac. A later file with an earlier file’s name is hidden, andprofile listsays so. - A team profile names a built in
baseand never its own Python, Node or MariaDB: those formulae, the MariaDB range and the doctor checks are tested per built in profile. Onlyfrappe_branchmay be overridden. PROFILEin the bench’s state stays the base, andTEAM_PROFILEnames the team profile: every daily command, doctor included, works from the base alone, and follows the team’s app branches (theapp_branch_policycheck,app add NAME) while the file exists and parses. A missing or broken team file falls back to the base instead of breaking doctor.- The team’s apps plug into the existing policy lookup (
fl_lookup_app_policyasks the loaded team profile first), so phase 01,app add NAMEand required apps all resolve a team app the same way. - The phases get the team profile’s name (
--profile acme) and load it themselves: phase 00 needs only the base, phase 01 the apps, site and pins, andbenchbar installstays a thin wrapper. bundleand[[apps]]exclude each other: a bundle is the built in app list,[[apps]]the team’s own, and mixing them would make the order ofapps.txtdepend on rules nobody wrote down.- An optional
commitin[[apps]]goes through phase 01’s existing pin (fetch --all --tags,checkout COMMIT), which leaves a detached HEAD; exact pins for identical benches are the lockfile’s job, a profile is the recipe. profile create --from-benchreads the bench only: remote URLs without user info, current branches (the policy branch for a detached app, else the app is skipped with a warning), the base fromapps/frappe’s version, the default site name and the scheduler choice. It writes no commits and nothing fromsites/except that name; the written file is parsed back before it is offered, and an existing file is shown as a diff and backed up.- The format is the strict TOML subset of
lib/frappe-local/toml.sh, shared with the lockfile: one parser, and a file this parser accepts is valid TOML.schema = 1is optional in a profile (written bycreate) and refused when it is another number. profile createtakes the checkout’s lock like other writing commands;profile listandshowdo not.
0.5: the team lockfile
- The lockfile is an awk parsed TOML subset (
toml.sh, shared with team profiles), nottomllib: the Command Line Tools’/usr/bin/python3is 3.9 without it, the bench’s env Python is missing exactly when a check matters, doctor stays pure shell, and a fallback would mean two parsers that can disagree. Every refusal names the line (benchbar.toml:3: not supported: inline tables). - The code lives in
teamlock.sh:lock.shalready is the checkout’s run lock, and the two must not be confused. - Lookup is
--lock PATH,BENCHBAR_LOCK,LOCK_FILEin the bench’s own state file (fl_bstate_set, not the<name>.envof the design), then<bench>/benchbar.toml. A relative--lockis taken from the current folder when it exists there, else from the bench, so--lock apps/acme/benchbar.tomlworks from anywhere. The path is remembered by write, check and apply (not in a dry run);BENCHBAR_LOCKnever is, andlist --jsonignores it since it names one bench, not all. - The app that holds the lockfile gets no
commit: committing the lockfile moves that app’s HEAD, so its pin would always be one commit behind and every teammate would seecommit_ahead. - Frappe is an ordinary
[[app]]: one code path, and the entry order isapps.txt’s, soapplyclones in dependency order. - Repos compare as host and path (
git@github.com:acme/x.gitequalshttps://github.com/acme/x): teammates clone over SSH or HTTPS as they like. An SSH host alias from~/.ssh/configis not resolved and reads as another repo (a warning only). lock checkis read only and offline: git reads,apps.txt, and the site app cache thatapp listfills, so it works with MariaDB stopped. The one outside call isbench --version(15 seconds at most) forbench_version_mismatch; doctor’slock_driftleaves that kind out and calls nothing.app_missingis the onlyfaillevel: the bench lacks code the team’s sites need. Everything else is a warning, and doctor reports the whole lock as onelock_driftwarning pointing atlock check.- Drift exits 1 like a failing doctor, through the dispatcher (
|| exit 1), solock check --jsonprints only the JSON. lock applystops at code: nonew-site,install-app,migrateorbench update. Missing sites and site apps are printed asbenchbar site addandbenchbar app installlines, and a changed app ends with the reminder to migrate: site data is the user’s to change.- A pinned commit is reached with
git merge --ff-only, aftergit fetch REMOTE SHAwhen it is not local (protocol v2 serves any reachable commit), and--unshallowfor a shallow clone as the fallback. The design’sfetch --depth 1 SHAwas dropped: a depth 1 fetch cuts the history between HEAD and the pin, so the fast forward check fails. - An app that is ahead of its pin, diverged from it, or dirty is skipped with a warning and apply still exits 0: local work outranks the lock, and a second apply must say
unchanged, not fail forever. - A branch switch happens only on a clean tree: the locked branch is fetched into its remote tracking ref (
--depth 1for a shallow clone), then checked out as the local branch when there is one, else created with--track.checkout -Bis never used on an existing branch, since it would drop that branch’s local commits. - A freshly cloned app whose branch tip is ahead of the pin is moved to the pin with
checkout -B BRANCH SHA: the clone is seconds old and holds no local work, so this is not the reset the rules forbid. - Cloning goes through
app add’s own steps (access check,get-app --skip-assets, the half clone rollback);setup requirements --pythonand--noderun only for apps whose code changed in place (get-app already did it for new clones), and onebench build --apps a,bcovers all of them. lock writerefreshes the site app lists from bench (it is the one lock command that may ask the database) and falls back to the cache with a warning. A dirty or detached app is refused unless--allow-dirty, which pins the commit as it is (a detached app then takes its policy branch).- The written file is parsed back before it is shown, so
writecan never produce a filecheckrejects. decisions of the app work live inmacos/DECISIONS.md. The 0.5 and 0.4 runs come first, the 0.3 easy install run follows.
0.5: benchbar pull
Upstream facts behind the design, frappe version-15 and version-16 (the same in both), from the design notes:
- F1
frappe/commands/site.pyrestore:--encryption-keyis the gpg passphrase for-encbackup files, not the siteencryption_key; without it restore readsbackup_encryption_keyfrom the site config (get_or_generate_backup_encryption_key()infrappe/utils/backups.py). - F2
decrypt_backupinfrappe/utils/backups.pyrunsgpg --yes --passphrase {passphrase} ...in a shell, so the passphrase shows in the process list. - F3
frappe/utils/password.py:get_encryption_key()generates and saves a new key whenencryption_keyis missing, anddecrypt()then fails with “Encryption key is invalid! Please check site_config.json”. Restore never copies the key. - F4
restore_backupcalls_new_site(..., force=True): on an existing site it recreates the database; a fresh name is created with a generateddb_name(get_sitesinfrappe/utils/bench_helper.pyaccepts any name). - F5
_new_siteinstalls only frappe,install_appsand--install-app; restore never checks that the dumped apps exist on the bench. - F6 restore
--forceonly skips the downgrade question and turns a missing__Authinto a warning; it does nothing for missing apps. - F7
bench backupcallsnew_backup(force=True), which first runsdelete_temp_backups()and removes files inprivate/backupsolder thankeep_backups_for_hours(default 23). - F8 migrate’s
pre_schema_updatesandafter_migratecallget_hooks(app_name=app)for every installed app;_load_app_hooksre-raisesImportErrorfor an app not on the bench.remove-from-installed-appsedits only the list. - F9
are_emails_muted()isflags.mute_emails or cint(conf.get("mute_emails")); there is nodisable_emailkey. The scheduler stops withpause_schedulerin the config ordisable-scheduler.
Decisions:
- The default source is the latest existing backup: a plain pull writes nothing on the server.
--new-backupneeds the production site name typed, even with--yes, becausebench backupalso deletes older backups (F7). Without a terminal the name comes from--confirm-site SITE, still typed by whoever runs the command;--yesalone never takes a backup.- Restore goes only into a new site name;
--replacefirst runsbench --site NAME backup --with-fileslocally, because restore over an existing site recreates its database (F4). - The keys come from the live production
site_config.json, read by a Python snippet on the server that prints only the one value asked for, not from the downloadedsite_config_backup.json(the design): the database password and Redis settings of production never reach the Mac, and it also works when the config backup is encrypted.--from-dirhas no server and reads thesite_config_backup.jsonin the folder. - Only
encryption_keyis written into the new site (F3);backup_encryption_keyis used only as the gpg passphrase (F1), so the copy’s own backups are not encrypted with the production key. - The key flows through pipes only (ssh stdout into a Python snippet’s stdin); it is never on a command line, in a shell variable, on disk outside
site_config.json, in the output or in the log. The snippet writes the file the wayupdate_site_configdoes (indent 1, sorted keys, temp file and rename) with plain Python, so it needs no frappe import and runs under the test mocks. - The MariaDB root password and the Administrator password reach frappe on stdin: a small wrapper reads the secret, puts it into
sys.argvinside the process and callsfrappe.utils.bench_helper.main(), which is whatbenchitself runs fromsites/. The design passed--db-root-passwordon the command line (accepted in 0.3 fornew-site); the wrapper keeps it out ofpsat no cost. The option used is--mariadb-root-password, the spellingnew-sitealready uses. - Encrypted backups are decrypted here with
gpg --batch --pinentry-mode loopback --passphrase-fd 0(F2); gpg is needed only then, and a missing one stops before the download withbrew install gnupg. - Restore
--forceis never passed: it does not help with missing apps (F6) and would hide a downgrade. Instead pull stops before the download when production frappe is newer than the bench’s. - Missing apps stop the run before the download with the
bench get-app --branch B URLcommands (F5, F8). The design offeredbenchbar app add; that command is being built separately, so pull only prints the commands for now. --skip-apprunsremove-from-installed-appsand prints that its doctypes and tables stay as orphans (F8). frappe cannot be skipped.- Tokens in an app’s git remote URL (
https://user:token@host/...) are cut before the URL is shown or logged. - The production app list comes from the text form of
list-apps(name, version, branch), parsed by its first columns;--from-dirreads theinstalled_appsglobal from the dump (tabDefaultValue, whatfrappe.get_installed_apps()reads), after decryption when the dump is encrypted. - Migrate runs when any app version differs from production, is unknown, or an app was skipped; the design ran it only when local code was newer, but an older local app with the same schema is the rare case and a needless migrate is cheap.
- Email is muted (
mute_emails), the scheduler paused (pause_scheduleranddisable-scheduler) andhost_nameset before the first start (F9): a copy must never mail customers or poll inboxes.--keep-schedulerleaves the scheduler alone. - The Administrator password is reset only with
ADMIN_PASSWORDor after a yes at the prompt; otherwise the production password keeps working. The design’s--admin-password-promptflag was left out: without--yespull asks anyway. - A decrypt probe after the restore counts the encrypted
__Authrows that decrypt and that fail and prints only the counts, so a wrong key shows up at once, not at the first email sync. - Restore and migrate run with the bench’s own Redis (
fl_bench_redis_up, fromsite add): frappe v16 connects to it during site setup, and the code has moved on since the design. - The staging folder is
<bench>/.benchbar/pulls/<site>-<backup timestamp>/, not<timestamp of the run>: a rerun of the same pull finds the partial files and resumes. It is mode 0700, removed file by file after a successful run, kept after a failed one. - rsync gets
--partial --append-verify --info=progress2only when the local rsync is 3.x; macOS ships openrsync (2.6.9 compatible), which has neither, so it gets--partial --progress, and the partial file is the basis of the next transfer.scpis used when the server has no rsync; it cannot resume. - A file whose local size already matches the server’s is not downloaded again.
- The free space check wants 3x the backup (download, decrypted copy or database, extracted files) on the bench’s volume, before any transfer.
- SSH settings come only from
~/.ssh/config(ProxyJump, keys, ports); benchbar addsBatchMode=yesunder--yesor--jsonand one control connection (ControlMaster=auto,ControlPersist=60) closed with-O exitwhen the run ends. Its socket is in/tmp/benchbar-ssh.XXXXXX, because socket paths are limited to 104 bytes and bench paths can be long. - The host must look like a Host alias (
[A-Za-z0-9@._-], not starting with-) and every remote word is single quoted, so no argument becomes an ssh option or remote shell code. - Remote commands are an allowlist (the site check,
list-apps,git rev-parseandremote get-url,ls,wc -c, the key read,command -v rsync, andbench backupafter the gate); the test fails on anything else. - Remote frappe runs through
env/bin/python -m frappe.utils.bench_helperfromsites/: non login SSH shells often lackbenchon PATH. pull --jsonstreams one object per line on stdout and moves every human line to stderr;doneis always the last line, written by the exit handler after a refusal or a failure too.- Only the source (
PULL_SOURCE) and the remote bench are remembered, per bench, sobenchbar pullalone repeats the last pull; no password is stored. - test-pull fails first when an ssh, rsync, scp, gpg or df mock is not the first one on PATH: during development a mock that was not yet executable let
/usr/bin/sshlook up the hostprod, which did not resolve, so nothing connected.
0.4: roadmap
-
Work happens in a second git worktree (
~/dev/benchbar-work):~/.local/bin/benchbarand the app run the checkout in~/dev/benchbar, so a feature branch checked out there would change the CLI in daily use. That checkout stays on main and is fast forwarded after each merge. -
Pulling a production site moved from the 0.7 plan to Ideas, as a wizard that also handles the encryption key and the app list mismatch: without those two it restores a site that cannot decrypt its passwords or fails on missing apps, and it is not scheduled yet.
-
The old 0.4 items the brief did not schedule (worker restart on Python changes, more speed sources, a runner gallery, running one scheduler event) moved to Ideas; runner import from a zip was dropped from the list because 0.3 already ships it.
-
App installs, team profiles, the lockfile and
pullwere scheduled as 0.6, then folded into 0.5 (Akash, 2026-09-26): they are being built at the same time as the 0.5 app work, and one bigger release is less to write up and tag than two. Patch numbers stay for fixes.
0.4: the v16 profile
Checked in frappe version-16 at 012667b and bench develop at c9d1250 (September 2026), not from memory:
- v16 needs Python 3.14 exactly (
requires-python = ">=3.14,<3.15") and Node 24 (engines.node >=24); frappe itself only warns below Node 18 at build time, so the profile’s pin is what enforces 24. - frappe v16 refuses no MariaDB version:
check_compatible_versionsinfrappe/database/mariadb/setup_db.pyonly warns below 10.6 or above 11.8, at new-site and restore. 10.11 gets no warning at all. The docs’ 11.8 is a recommendation. - wkhtmltopdf stays installed on v16:
pdf_generatoron Print Format defaults towkhtmltopdfand a v16 patch sets every existing format to it; Chromium is opt in per format. So the check becamepdf_engine(wkhtmltopdf everywhere, plus Chromium on v16) instead of swapping one tool for the other. - Chromium is looked up the way frappe does (
find_or_download_chromium_executable):chromium_pathfrom common_site_config, else<bench>/chromium/chrome-mac/headless_shell. A missing one is a warning withbench setup-chrome, not a repair action: frappe downloads it on first use anyway, and its download removes<bench>/chromiumfirst, which benchbar should not trigger on its own. pkgconfandmariadb-connector-care system dependencies of every profile: v16 pinsmysqlclient==2.2.7, which does not build without them.pkg-configis only an alias ofpkgconfin Homebrew, andbrew list pkg-configdoes not follow it, so the formula is namedpkgconf.- bench sets
PKG_CONFIG_PATHitself frombrew --prefix mariadb-connector-c, only for frappe 16 or newer on darwin, when it builds the env or installs an app. benchbar adds the same folder to its own exports (shell block, repair, phase 01) so a pip build outside bench finds it too. The launchd plist and the runner set noPKG_CONFIG_PATH, so nothing undoes bench’s value; the running bench needs none. - Missing build formulae are a warning, not a failure: a v15 bench runs without them and a v15 machine set up before 0.4 should not turn red.
- bench 5.31 depends on uv and uses it for the env by default (
BENCH_DISABLE_UV=1turns that off), so “uv or pipx” only decides how thebenchcommand itself is installed. uv wins when it is on PATH, as in the Frappe docs; an existing pipxbenchis reported and never moved. - honcho 2.0.0 imports
importlib.metadata, notpkg_resources; only honcho 1.x breaks on Python 3.12 or newer without setuptools. - The v16 CI row is a separate job running
tests/test-profile-v16.shon macOS/bin/bash3.2: the rest of the suite asserts v15 strings on purpose, and a second full run would only repeat them.
0.4: doctor hardening
- Full Disk Access: frappe/bench#1730 (crontab “Operation not permitted” during
bench init) is still open with no fix merged as of 2026-09-25; the rollback bench offers is a prompt (click.confirm), not automatic. benchbar passes--no-backups, which skips python-crontab entirely, and keeps a doctor check plus a warning beforebench init, sincebench setup backupswould still need it. - The probe is
crontab -l: “no crontab for” and exit 1 is a normal answer, only “Operation not permitted” counts. Run from BenchBar the check reports ok without probing: the access that matters is the Terminal’s, not the app’s. - The toolchain is four checks (
toolchain_node,toolchain_yarn,toolchain_pkgconfig, plusmariadb_version), one line each in doctor’s existing one line per check format. Python is not repeated:env_pythonalready compares the env’s version with the profile. - Tools are resolved on the launchd PATH the agent uses, not the caller’s: the forum case “nvm’s node is not seen by bench” is exactly a node that only exists on the shell’s PATH.
env/bin/nodewins when bench put one there. - The MariaDB server version comes from the binary of the process listening on 3306 (
ps), else the installed formula’s client; doctor never logs in, so it never reads the Keychain. The range is frappe’s own: below 10.6 unsupported, above 11.8 untested, both warnings. honcho_setuptoolsimportshoncho.commandwith the interpreter in honcho’s shebang instead of importingpkg_resourcesdirectly: honcho 2.0 does not need it, and a barepkg_resourcesprobe would warn on every healthy Python 3.12 venv. The repair installs setuptools into that venv (pipx or uv), never into the bench env, and adopt skips it likehoncho_install.fork_safetyreads the plist only: the variables reach honcho and every worker through the agent’s environment, andProcfile.leandoes not need to repeat them.benchfgsets them itself.orphanscounts listeners on the bench’s ports only when neither the agent nor a honcho runs the bench, so a bench started withbenchfgis not reported.- Found while testing:
doctor --jsonwith a failing check went through the ERR trap on its way to exit 1 and printed[FAIL] Last command failedon stdout after the JSON. The dispatcher now exits 1 itself, and test-json parses a failing report.
0.4: several benches
- Per bench settings live in the checkout,
.benchbar/benches/<name>.env, not in the bench: benchbar writes only its service files into a bench, and a folder name is already the agent’s unique key (com.benchbar.<name>), so it is a safe file name too. - Keys that were per bench but global before 0.4 (
PROFILE,SITE_NAME,AUTOSTART,HONCHO_BIN,APP_BUNDLE,APPS) are read fromstate.envfor the default bench until its own file has them, and moved by the next writing command. Doctor and status stay read only, and an upgrade needs no migration step. - When the default bench changes, its old settings move into its own file first, so
AUTOSTART=offof the old default never leaks to the new one. - A second
install,adoptorservicekeeps the default bench (--make-defaultchanges it): “set up a v16 bench to try something” must not change whatbenchupstarts. - A bench with no stored profile gets the one whose Frappe branch matches
apps/frappe/frappe/__init__.py(__version__ = "16.x"isv16-lts), before the default profile: doctor on a freshly found v16 bench would otherwise judge it by v15 rules. - The shell block’s PATH lines follow the default bench’s profile, and every bench renders the block the same way. Before, a v16 bench would have put Python 3.14 and Node 24 first in the user’s shell, and the two benches’ doctors would have called each other’s block outdated forever.
- Phase 00 writes the shell block only when it is missing: it knows a profile but not which bench is the default.
serviceandrepairkeep it current. - honcho and socketio are matched by their working folder (
lsof -a -p PID -d cwd): both start with a relative path (honcho start -f Procfile.lean,node apps/frappe/socketio.js), so the command line is identical in every bench. A process whose folder cannot be read is still matched, the pre 0.4 behaviour, since it is usually exiting. - The runner (template v3) clears only its own bench’s stale socketio at start, with the same working folder test; before, starting a second bench killed the first bench’s socketio.
- The launchctl mock now stops only the agent’s own honcho, so the tests can run two benches at once.
0.4: Codex review findings on PRs 10 to 12
- Bench paths are resolved once, in
fl_abs_path(symlinks,.and..), so state, ports, the default bench and agents see one spelling per bench; a storedBENCH_DIRfrom before is compared throughfl_same_path. A local review before pushing found a symlinked spelling clashing with its own ports and getting its own state file. - The per bench state file is
<name>-<8 hex of the canonical path>.env(cksum ofpwd -P, so every spelling of a bench maps to one file; a plain<name>.envfrom the first 0.4 builds is read and renamed on the next write, by the default bench, or by a bench no other known bench shares the folder name with):~/frappe-benchand~/dev/frappe-benchare both detected candidates and would otherwise share one file. The agent label still uses the folder name only; two benches with the same folder name remain unsupported, as before 0.4. - A bench installed with uv is looked up in
uv tool dir --bin, which followsUV_TOOL_BIN_DIRandXDG_BIN_HOME, instead of assuming~/.local/bin. - Phase 01 records the default bench only after the site is verified, so a failed
install --make-defaultleavesbenchuppointing at the bench that worked. - With
--make-default, the shell block is rendered for the bench being set up even in a dry run, so the plan shows the PATH change the real run makes. - The
orphansfinding (a machine widepgrepfor honcho) was already fixed by the working folder match in the multi bench PR.
0.4: port blocks
bench initalready chooses ports:make_portsin bench’sconfig/common_site_config.pytakes max+1 over the benches in the same parent folder. benchbar keeps that and only moves a bench when it clashes with another bench it knows (anywhere under~or~/dev, or registered by an agent) or a port is taken by a listener.- The block is written with frappe’s own
bench set-config -g(-pfor the two integer ports) and the Redis files withbench setup redis, not by editingcommon_site_config.jsonorconfig/redis_*.conf: the AGENTS.md rule “never edit sites/” holds, and the files look exactly as bench would write them. The config is backed up first. redis_socketiois written equal toredis_cacheand no12000 + nport is reserved: bench keeps the two equal “for backward compatibility”, and neither frappe v15 nor v16 connects to it (v16 realtime usesredis_queue).- Only a newcomer moves: the default bench (or the one about to become it) never changes ports on its own, and a bench moves only when it clashes with an established bench, one with a benchbar agent or the default. Found by the tests: without that rule, adopting the first bench on a Mac with two more unregistered benches moved the first bench off 8000.
- Moving ports is the
port_blockaction of the same plan as the service files (a check that exists only while a plan moves ports, never in doctor): one confirmation covers it, and a cancelled adopt really changed nothing. Codex found the first version asking separately and moving the ports before the plan could still be cancelled. - Listener ownership has two strictness levels. For the bench’s current ports an unreadable working folder counts as its own, so it never moves on a guess; for a block it does not use yet, only a listener known to run inside the bench is its own (Codex found the first version trusting every listener on the current ports, which could then be killed by the runner’s cleanup).
service --port-offsetvalidates the block before remembering the bench, so a refused block never makes it the default (Codex).- The action rewrites
Procfile.leanand the runner itself after the move: they carry the ports, and when the plan was made they may have been current and therefore not in it. --port-offsetchecks the block for conflicts even when it is the current one, so two benches already sharing block 0 are not confirmed by asking for block 0 (Codex).- A newcomer also moves when a foreign process listens on its own ports, not only when a known bench has them (Codex). “Foreign” is decided by working folder: honcho starts redis, web and socketio inside the bench, and an unreadable folder counts as the bench’s own, so nothing moves on a guess. The default bench is never moved for a listener either; it gets a warning.
- The port clash check now also reports a bench that is only configured with the same ports, since only one of them can run;
benchbar upstill asks only about a running one, so the app (which cannot answer) can start either bench. - MariaDB ranges live in the profile (
mariadb_min,mariadb_max). v16 uses frappe’s own bounds, 10.6 to 11.8. v15’s code warns above 10.8, but the v15 docs and this installer use 10.11, so v15’s range ends at 10.11; the check is a warning either way, as in frappe. - The toolchain’s MariaDB check was named
mariadb_versionbefore 0.4 shipped, to match the brief; nothing released used the other name.
0.4: the MariaDB decision
- Option (a), decided with Akash on 2026-09-26 (“let’s struggle and find out”): the v16 bench uses the
mariadb@10.11server the v15 bench already runs on 3306. frappe v16 accepts it without a warning, and it keeps one server, one data folder and one Keychain password. Option (b) (a secondmariadb@11.8service on 3307) stays possible later if v16 turns out to need 11.8. - The supported path: when a MariaDB server runs on 3306 and its version is inside the profile’s range, phases 00 and 01 and every command use that server’s formula instead of the profile’s (named from the server binary’s
opt/<formula>orCellar/<formula>path, read withps), and the formula is stored per bench (MARIADB_FORMULA) by phase 01 or the firstservice,adoptorinstall. Daily commands read only the stored value:test-processguards thatdownnever touches port 3306, and a detection in every command’s context broke that guarantee. So a v16 install on this Mac never installs or startsmariadb@11.8, and a machine without MariaDB still gets the profile’s formula.mariadb_versionin doctor checks the range per profile.
0.4: sites and the scheduler
- The default site is the one benchbar remembers per bench;
site defaultalso runsbench use, socurrentsite.txtagrees, and rewrites the runner, whose ping embeds the site.benchup’s wait,statusand the app all use it; a second site never changes it on its own. site addcreates a new site only, on the same MariaDB server as the bench (option (a) of the MariaDB decision), with the Keychain password; it never drops or overwrites a site, and there is nosite dropin 0.4.- Apps for a new site come only from
apps/(--bundleor--apps); a missing app is refused with thebench get-appcommand. Fetching apps belongs to the 0.6 app commands. - A site’s
ping_codeis only tried when something listens on the web port, solist --jsonon stopped benches costs no three second timeouts per site. site hostsasks once for all missing lines and uses one sudo prompt, through the existing marker block logic.site listtakes no lock (it is read only);add,defaultandhoststake the checkout’s lock like every other writing command.- The scheduler line is an optional template line: a line that is only a token rendering to nothing is dropped. With the scheduler off,
Procfile.leanrenders byte for byte as before 0.4 (hash 92548cf35913 checked against the real bench), so no existing bench sees an outdated Procfile. The file’s comment still says “no schedule” when it is on: changing it would change every existing Procfile’s hash. - The scheduler choice is per bench state (
SCHEDULERin its state file), sorepairrenders the same Procfile and never undoes it;status --jsonreportsscheduler.
0.4: found on a real Mac (v16 bench, 2026-09-26)
- The MariaDB root password of a bench set up before 0.3 was never in the Keychain, so phase 00 stopped with exit 2 as designed; Akash reset it with the documented recipe (databases kept, the v15 bench stopped around it with
benchbar downand brought back withbenchbar up), and the next run verified the new password and saved it to the Keychain. bench initin~/dev/v16-benchchose 8001, 9001, 11001, 13001 (and file watcher 6788) by itself, as a sibling of~/dev/frappe-bench; benchbar’s port check agreed and moved nothing.- On v16,
bench --site v16dev install-app erpnextfailed with “Error 61 connecting to 127.0.0.1:11001. Connection refused”: frappe v16 connects to the bench’s Redis during site setup, and a fresh bench has none running. Phase 01 andsite addnow start the bench’s own Redis servers fromconfig/redis_*.confwhen nothing listens on their ports, and stop only the ones they started: the queue withshutdown save, so jobs an app install enqueued wait for the first worker, the cache withnosave(Codex). Redis startup errors go to the run log, whose path is now exported to the phase scripts. A listener on the bench’s Redis ports is used only when it runs inside the bench (working folder); another process there, such as another bench on the same default ports, stops the setup with the--port-offsetfix, so this bench’s cache and install jobs never land in someone else’s Redis (Codex). A running bench’s Redis is used as it is.
0.4: the v16 bench on a real Mac, verified (2026-09-26, macOS 27, Apple Silicon)
- Installed
~/dev/v16-bench(profilev16-lts, bundleminimal, sitev16dev) withbenchbar installnext to the running v15 bench: frappe and erpnextversion-16, Python 3.14.7, Node 24.21.0, the existingmariadb@10.11server (10.11.19, option (a)), ports 8001, 9001, 11001, 13001. The profile is nowsupported. - Both benches up at once:
macdevon 8000 andv16devon 8001 each answer/api/method/pingwith 200 (Host header);benchbar listshows both,frappe-benchas the default; the shell block still puts python@3.11 and node@20 first. benchbar downon v16 left v15 running (ping 200), anddownon v15 left v16 running (ping 200).upon v15 then crashed v16 once: the v15 bench still had runner template v2 (Akash has not runrepairsince 0.4’s runner v3), whose start up cleanup kills everysocketio.json the Mac; honcho in v16 exited with 241, launchd restarted it after about 20 seconds and it answered 200 again. This is the bug the working folder match fixes;benchbar repairon the v15 bench installs runner v3. The crash guard counted one start and did not trip.- Doctor: v15 30 ok, 2 warn (CleanMyMac, the outdated runner). v16 28 ok, 4 warn: CleanMyMac, the missing
v16devhosts line (sudo, pending Akash), Chromium not downloaded yet (expected: frappe fetches it on first chrome PDF), and the python formula check, which was wrong (next section). - The resumed install ran with
--yesand no terminal: phase 01 installed erpnext on the existing site with the setup Redis, the service files were written and the agent loaded with the bench stopped, and the hosts step was skipped with its manual command. The run ended with the verify warnings and a misleading[FAIL] Last command failed ... brew tabfrom the ERR trap; the trap now stays quiet when the last command is a finished verify pass. - Then, with Akash:
benchbar repairon the v15 bench (runner v3; doctor 31 ok, 1 warn),benchbar site add v16two(his Administrator password, the hosts line with sudo) andbenchbar site hostsforv16dev;macdev:8000,v16dev:8001andv16two:8001all answer ping 200 by hostname, and the hosts block holds exactly one line per v16 site. - frappe v16’s
bench new-sitecalls--no-mariadb-socketdeprecated in favour of--mariadb-user-host-login-scope='%'; v16 profiles now pass the new flag, v15 keeps the old one, which it still needs.
0.4: python formula check
python_leavesasksbrew list --formula --installed-on-request, notbrew leaves: on the real Macpython@3.14was installed on request (andbrew tabset it again), but pipx, uv and ollama depend on it, andbrew leaveshides every formula another formula depends on, so the check could never pass. What protects a formula frombrew autoremoveis “installed on request”, which is what the check now reads.
0.5: benchbar mcp
- The server is stdlib only Python 3.9 (
lib/frappe-local/mcp.py), run with whateverpython3is on PATH: the Command Line Tools’ 3.9 is enough, so CLI only users need nothing new. The Swift MCP SDK would tie it to the app. - Every tool shells out to
benchbar ... --jsonand returns its output as text and asstructuredContent; the server holds no bench logic. Actions (up,down,restart) return their text output plus a freshstatus --json, so an agent sees the outcome without a second call. - No repair, install, service, site add or anything with sudo behind a tool: those change files or need a password, and belong in a terminal with the user. Read tools carry
readOnlyHint: true, actionsfalse. - stdin of every CLI call is closed, so a question (a port clash on
up) is answered no instead of hanging the session; one bad message gets a JSON-RPC error and the session goes on. - Protocol versions 2025-06-18, 2025-03-26 and 2024-11-05 are accepted as asked; anything else gets 2025-06-18. The transport is newline delimited JSON on stdio, as the spec says for stdio.
logs --jsonexists forbenchbar_logs_tail: the brief keeps the server free of logic, so the process filter (honcho’sHH:MM:SS name.N |prefix, with unprefixed lines such as tracebacks kept under their process) lives in the CLI.fl_json_escapenow drops control characters, since log lines may carry terminal colors.
0.5: repair —json
- The events go to fd 3 (the command’s stdout) and every human line to the run’s log: the engine prints with the same helpers as always, so the text and the JSON can never disagree, and the log keeps the full story for a failed step.
- Without
--yesa JSON run applies nothing: the confirmation would be invisible, so stdin is closed and the plan is answered no (plan, then done with 1). The app shows the plan from--dry-run --json, asks, and then passes--yes. sudoactions are marked in the plan ("sudo": true): without a terminal they are skipped with their manual command as the message, which the app shows as “run in Terminal”.- A step’s message is its own last
[FAIL]line, else its[WARN]line, from the log lines written during the step; the step summary line is ignored.
0.5: local review before the pull request
bench new-sitegets both passwords on stdin, not in its arguments: the bench’s own Python reads them and swaps@secret0@,@secret1@in its argv before handing over to frappe’sbench_helper, so the passwords never appear inpsor the run log.- Every
git ls-remoteputs--before the repository, and a profile or lockfile entry whose repo, branch, commit or name starts with-is refused: a team file must not be able to pass git an option. app list --jsonon a bench without sites reads its cache withgrep ... || true, so pipefail does not end the run.benchbar mcpanswers a message that is not an object with -32600 and parameters that are not objects with -32602, and keeps the session going.- The app’s repair sheet applies stream events on the main actor in order, through one
AsyncStream, so a late step event can never overwrite a failed run. - Codex on the pull request:
app updaterefuses to run when any site’slist-appsfails, since a stale list could skip a site’s backup or migrate.pullinto a running bench setspause_schedulerincommon_site_config.jsonbefore the restore and removes it once the copy has its ownpause_schedulerandmute_emails, so the bench’s scheduler never sees the production copy unguarded; a bench already paused by hand is left alone, and a pull that stops halfway leaves the bench paused and says how to resume. The test runner kills a timed out test’s whole process tree, collected before anything dies.
0.5: found on the real Mac before the release
app add URLfor an appconfig/apps.tsvknows follows that branch when the repository has it (Raven added by its URL came in ondevelop, the repository default, and doctor then warned that it was offmain). A URL for an unknown app still takes the default branch;--branchalways wins.- Raven moved to
github.com/frappe/raven; the registry points there (the old URL redirects). - The app reads
app listwith the 90 second limit, not 20: at app start it runs next to status and doctor for every bench, and it makes several git calls per app.
The easy install run (v0.3)
Setup and environment
- Work happened in a Linux cloud container, not on a Mac: there is no Xcode, launchctl, codesign or hdiutil here. Everything Mac only (the app build, the Swift tests,
release-local.sh) is exercised on GitHub Actions macOS runners; the CLI is tested locally under mocks and in CI under macOS/bin/bash3.2. - Branch
feat/easy-installis pushed and a draft pull request opened againstmain: the brief for this environment allows exactly that remote write and nothing else (no tags, no releases, no settings). - Conflict with AGENTS.md: it says never write LaunchAgents or
Procfileyourself; this run changes the code that generates them (adopt,install). The brief wins; nothing was loaded on a real machine. - bash 3.2 could not be built locally: the proxy blocks the GNU mirrors and only the project repository is reachable on GitHub. CI’s macOS
/bin/bash3.2 is the bash 3.2 check. - The mocked test suite was made to run on Linux too (GNU stat and sed, a
unamemock, a fake uid): it gives fast local feedback here and a quick Linux job in CI, while the macOS job stays the one that counts.
Stage 1: harden
benchbar reportmasks by key name (password, secret, token, key, api, auth, case insensitive, anywhere in the key) rather than by value shape: values are unpredictable, key names are not. Over masking (for exampleencryption_key,Authorization:headers) is accepted.- Header style
Key: valuemasking runs to the end of the line, or to the next quote, comma or brace inside a one line JSON document, so a bearer token afterAuthorization:is gone but a JSON file is not wiped. - The username and hostname are replaced by simple string substitution (any occurrence, no word boundaries, names shorter than three characters skipped): BSD sed has no
\b, and replacing too much is safer than too little. - Site config files are never copied;
site-config-keys.txtlists their key names so a helper can still see whether, say,developer_modeordb_hostis set. REDACTIONS.txtreports what was replaced per file, so the sender can check the bundle before attaching it.reportworks without a bench (versions only) so a tester whose install failed early can still send something.- The CLI job in CI runs on
macos-latest, the app jobs onmacos-26: the project uses Xcode 26 build settings (default MainActor isolation) that the Xcode onmacos-latestdoes not have. The old release workflow already targetedmacos-26. - CI also runs the CLI tests on
ubuntu-latest: it finishes in a minute and catches most breakage before the macOS runner is even assigned. - The old
macos-release.ymlfailed on every push with “workflow file issue”: its job levelenvread${{ runner.temp }}, and therunnercontext is not available there. The mergedrelease.ymluses$RUNNER_TEMPin steps. release-local.shis the ad hoc path of the release and is also what CI runs on every pull request to produce a downloadable app;macos-release.shstays the Developer ID path.macos-build.shtakesBENCHBAR_VERSIONandBENCHBAR_BUILDand passes them to xcodebuild instead of editingproject.ymlin CI, so the checkout stays clean and the tag decides the version.
Stage 2: the manual steps
- The MariaDB root password goes into the macOS Keychain (service
benchbar-mariadb, accountroot) throughsecurity add-generic-password -U: the only non interactive way to write a Keychain item. The password is an argument of that one call; everywhere else it travels inMYSQL_PWD, never on a command line. The mocked test asserts that no call log ever contains it. - Order of password sources:
MARIADB_ROOT_PASSWORD, then the Keychain, then a prompt. A password that verifies against the server is always saved, whatever its source, so the next run needs no source at all. - The secure installation is plain SQL (
DELETE FROM mysql.global_privfor anonymous and remote root,DROP DATABASE test,ALTER USER ... IDENTIFIED VIA mysql_native_password):mariadb-secure-installationis interactive and the answer table in the old README was the whole reason it was manual. Native password auth is kept on purpose: Frappe needs it. - Phase 00 exits 2 only when MariaDB already has a root password that no source knows and there is no terminal to ask; every other former manual step is automated or skipped with a warning, so
benchbar install --yesnormally finishes in one run. - A generated password is 24 letters and digits: safe in SQL, in a shell and on bench’s own
--mariadb-root-passwordargument, which cannot be avoided. - The pinned wkhtmltopdf is 0.12.6-2 from github.com/wkhtmltopdf/packaging, sha256
81a66b77...94f8, computed from a download made during this run. The package’s binary is x86_64 only (checked by reading the Mach-O header inside the nested tarball: cputype 0x01000007, no arm64 slice), so on Apple Silicon Rosetta 2 is required and offered first; declining it or the download only costs PDFs, and the run continues. - Rosetta is detected with
arch -x86_64 /usr/bin/true, not by looking for the daemon: it is what the binary will actually need. - Downloads land in
.benchbar/downloads/and are reused when their checksum matches, so a re-run after a failedinstallerdoes not fetch 50 MB again. - One sudo prompt per run:
benchbar installlooks ahead (patched wkhtmltopdf present? hosts line present?) and runssudo -vwith a keepalive before phase 00; the phase scripts inherit the session throughFL_SUDO_SESSIONand sudo’s own per terminal timestamp. Standalone00-mac-system-deps.shasks itself, only when it gets to the package. - The
/etc/hostsline sits inside# >>> benchbar >>>markers. A missing block is appended withsudo tee -a; an existing block gets the new line inside it through a temp copy andsudo cp, afterfl_backup_file. Rewriting the whole file is limited to the second case. - The utf8mb4 drop-in and wkhtmltopdf became doctor checks with repair actions (
mariadb_utf8,wkhtmltopdf_install): the same code serves phase 00 andbenchbar repair, and the tester guide can say “run doctor”. benchbar adoptisbenchbar servicewith a positional path, validation and an explicit safety statement, run through the same engine restricted to the service group: that group has nomigrate,build,updateor env rebuild by construction. It remembers the bench only after the plan was applied or found unchanged, so a cancelled adopt never makes that bench the default (a Codex review finding).benchbar mariadb-passwordasks before printing unless--yes: a terminal print is a deliberate act, and the app or a script can pass--yes.- The mocks gained state (
mariadb_root_pw, a keychain folder,wkhtml_installed,rosetta,sudo_refused,download_payload) instead of environment switches, so a test reads like a machine’s history.
Stage 3: install.sh
install.shis self contained bash 3.2 with its own small output helpers: it runs before the repository exists on the machine, so it cannot sourcelib/.- Prompts read and write
/dev/ttysocurl | bashcan ask; without a usable tty every question takes its default (--yesdoes the same). Defaults are “yes” for the things the person asked for (Homebrew installer, adopt) and “no” for anything destructive (removing agents or the checkout on--uninstall, where--yesflips them to yes because that is what a non interactive uninstall means). - The CLI is cloned into
~/.local/share/benchbarand updated withgit pull --ff-only;--versionpins the app only. Pinning the CLI to a tag would leave a detached checkout that the nextgit pullcannot update. benchbarandfrappe-macare symlinks into the checkout, sogit pullis the whole CLI update; a regular file at either path is left alone with a warning.- The PATH block in
~/.zshrcuses its own markers (# >>> benchbar-path >>>): the CLI’s# >>> benchbar >>>block is regenerated byrepairfrom a template and only exists afterinstalloradopt. - The app zip is checked against the release’s
SHA256SUMS; a release without that file is refused, not installed unchecked. The installed version is read fromInfo.plistwith awk (noplutil, so the same code runs in the Linux tests). - The app is unpacked with
ditto -x -kand copied withditto: the tools macOS uses for bundles. Becausecurlwrites the download without thecom.apple.quarantineattribute, Gatekeeper never assesses the app on first open. That is the reason the one liner is the recommended install for an ad hoc signed build; the DMG path gets the “Open Anyway” steps in README instead. - A found bench is offered to
benchbar adopt, which keeps its own plan and question;install.shonly passes--yeson when it was given--yes.benchbar installis never started without a terminal: it needs the Administrator password. - On an Intel Mac the app is skipped with a warning (it is built for arm64 only); the CLI installs.
--uninstallremoves only what the installer made (app, links, PATH block), offersbenchbar uninstall-servicepercom.benchbar.*agent and the removal of the checkout, and never lists or touches a bench folder.- The tests drive
install.shagainst a fake HOME with agitmock that copies this checkout, acurlmock that serves a release JSON, zip andSHA256SUMSfrom the mock state, and adittomock on top ofunzip; scenarios: no release, fresh, unchanged rerun, upgrade with a running app, version pin, tampered zip,--no-app,--app-only,--dry-run, adopt offer, Intel,--uninstalltwice.
Stage 4: releases
- One
release.ymlreplacesmacos-release.yml: acheckjob reads the secrets and picks the path; the build job runsscripts/macos-release.sh(Developer ID, notarized, Sparkle, cask) orscripts/release-local.sh(ad hoc). Everything after the build is shared: notes from the CHANGELOG section, a workflow artifact, a draft release, uploads with--clobberso a re-run replaces files. - The version comes from the tag and is passed to xcodebuild (
BENCHBAR_VERSION,BENCHBAR_BUILD= commit count) instead of editingproject.yml; the workflow still refuses a tag that disagrees withMARKETING_VERSIONor has no CHANGELOG section, so the source of truth stays in the repo. - Releases are created as drafts: the person publishes after looking at the files.
workflow_dispatchwith a version builds the artifact without touching releases, for a rehearsal. - Sparkle stays out of the ad hoc path: unsigned updates would defeat its purpose, and the ad hoc app contains no update code.
- The DMG carries an Applications symlink and is not signed on the ad hoc path (signing a DMG ad hoc buys nothing); Gatekeeper’s warning is expected and documented with the macOS 15 “Open Anyway” steps in README.
Found on the macOS runners
- A generated password reads a bounded 4 KB of
/dev/urandom: BSDtron an endless stream never exits whenheadcloses the pipe and SIGPIPE is ignored, as it is under GitHub Actions. The CLI job hung on that for 45 minutes before the cause was found. - The sudo keepalive owns no stdio and sleeps in five second slices: with the caller’s stdout inherited, every
$(...)capture of a run waited for itssleep 50, and on macOS never returned. The tests capture output, so this showed up only in CI. - The
gitmock’s pass through to the real git picks the firstgiton PATH outsidetests/mocksand refuses to exec itself: on macOS/bin/bash3.2,command -v -p gitstill returned the mock, which then exec’d itself in a loop at full CPU. This was the hang behind three timed out macOS CLI jobs; the Linux job never saw it. The test runner now kills a test after ten minutes and prints its process tree, which is how this was found.
Review findings on the pull request
-
fl_wkhtmltopdf_ensurereturns 2 for a deliberate skip (Rosetta or the package declined, no sudo) and 1 for an error; the repair action reports a skip asskipped, and the engine’s verify pass ignores the optional actionswkhtmltopdf_installandredis_stop, so a bench install without PDFs exits 0. -
The report’s assignment style masking takes a quoted value whole (
key="secret",key='secret'): the unquoted form stopped at the quote and left the secret in place. -
install.sh --app-onlyskips the Command Line Tools and Homebrew checks: the prebuilt app needs onlycurl,shasumandditto; macOS and the architecture are still checked. -
The installer’s PATH block is generated from the configured bin folder (
BENCHBAR_BIN_DIR), written as$HOME/...when it lives under the home folder. -
A refused up-front
sudo -vsetsFL_SUDO_REFUSED=1, exported to the phase scripts, so an install asks for sudo exactly once whatever the answer and skips every sudo step with a message after a refusal. -
A hosts line that cannot be written because sudo was refused is a skipped step with the manual command printed, and
hosts_entryjoins the optional actions, so a run the user chose to keep sudo free still ends with exit 0 and warnings. -
The repair engine resets
FL_STEP_RESULTbefore each action and reports it, so a skipped wkhtmltopdf or hosts step printsskipped, the documented step status, notdone. -
The report’s assignment style masking also treats
?as a key boundary, so a credential in a URL query string (?api_key=...&x=1) is masked like any other. -
release-local.shno longer re-signs the app:macos-build.shalready signed it ad hoc with the Hardened Runtime and the entitlements, and a plaincodesign --force -s -dropped both. The script verifies the signature and the runtime flag instead. -
Phase 01 no longer requires
wkhtmltopdf: a bench without it works, only PDF printing does not, and phase 00 already said so when the package was declined. -
The report’s quoted value patterns accept backslash escaped characters inside the quotes, so
"api_key":"abc\"tail"is masked whole. -
The utf8mb4 doctor check also requires the
!includedirline inmy.cnf(a missingmy.cnfcounts as missing). The livecharacter_set_serverquery that 0.3.0 added was removed in 0.3.1: it read the root password from the Keychain, and doctor is read only and runs on a timer in the app, so it must never touch the Keychain. A test asserts that doctor makes nosecuritycall. -
adopt --dry-runnever touched state (fl_state_setis a no-op underFL_DRY_RUN); a test now proves it. -
The report also masks Python repr mappings (
'password': 'x'): worker logs print dicts that way. -
adoptruns the engine withFL_ENGINE_SKIP_ACTIONS=honcho_install: a missing honcho is reported withpipx install honchoorbenchbar repairas the fix, and nothing is ever installed into the bench’senv/by adopt. -
Phase 00 tells a failed wkhtmltopdf install (download, checksum, installer) apart from a deliberate skip: it prints FAILED and a manual step, but still exits 0, since PDFs are optional and the bench can be created.
-
A generated MariaDB password is written to the Keychain before it is applied to the server; when the Keychain refuses (locked), MariaDB is left unchanged and the run stops with the fix, so no password ever exists only in a dying process.
-
Phase 01 exits 2, the documented “root password unknown” code, when a fresh site needs the MariaDB password and no source has it;
benchbar installreports it as a pending manual step like phase 00 does. -
wkhtmltopdf detection prefers the package binary at
/usr/local/bin/wkhtmltopdfwhen it is the patched build, and warns withbrew uninstall wkhtmltopdfwhen an unpatched build earlier on PATH would shadow it: Frappe finds the binary through PATH, and the launchd PATH puts Homebrew’s bin before /usr/local/bin. -
A missing
my.cnfcounts as a missing!includedirin the utf8mb4 check: without it the drop-in folder is never read.
Found on a real Mac (macOS 27, Apple Silicon)
- Verified read only on the real bench:
install.sh --dry-runand--dry-run --uninstallwrite nothing;doctorandreport --printrun clean (no home path, username, hostname or site config value in the report);adopton an already adopted bench is a no-op that only remembers the path.release-local.shbuilds, passes the 114 Swift tests, andshasum -a 256 -c dist/SHA256SUMSpasses; the DMG (UDZO, read-only, Applications symlink) mounts and the unpacked app carries a valid ad hoc signature. Every macOS flag the mocks assume was checked against the man pages:security add-generic-password -U,arch -x86_64 prog,softwareupdate --install-rosetta --agree-to-license,installer -pkg X -target /,ditto -x -kand-c -k --keepParent,hdiutil create -format UDZO,sw_vers -productVersion(the man page lists--productVersion; the single dash form still works and is what macOS 12 and older knew). benchbar reportreplaces every name the Mac goes by, not onlyhostname: on the test Machostnamereturned the router’s name (Mac.lan) while the Bonjour name fromscutil --get LocalHostNamewas the identifying one. The computer name (scutil --get ComputerName) is included too, and the sed patterns built from those names,$HOMEand the username are escaped, since a computer name can hold brackets or a dot.install.sh --uninstall --dry-runended with “BenchBar removed”: the uninstall summary has its own dry-run branch now, andquit_appprints what it would do instead of hiding the line behind the redirectedosascript.- The Gatekeeper dialog for a quarantined ad hoc build reads “BenchBar” Not Opened, Apple could not verify it is free of malware, with Done and Move to Trash (Move to Bin in British English) where Move to Trash is the highlighted default. README says so. The System Settings steps were not clicked: Open Anyway changes a security setting, which the brief reserves for the user.
fl_password_generateand the sudo keepalive were exercised on macOS/bin/bash3.2: the generator returns 24 characters at once, also with SIGPIPE ignored, and 4 KB of urandom never yields fewer than about 900 usable characters; a capturedfl_sudo_beginreturns immediately and its keepalive exits within one five second slice of the parent leaving.
CI speed
Measured on five PR runs (September 2026): 11 to 14 minutes wall clock, all of it the macOS CLI job (10.7 to 13.5 min; its test step 605 to 763s). Linux CLI took 7.5 to 8.8 min, the app job 1.4 to 1.7 min, the release bundle 1.2 to 1.8 min after it, v16 0.3 to 0.7 min, shellcheck 22 to 32s, brew install 2 to 4s (bottle cache hits), DerivedData restores from its restore key in about 5s.
tests/run-tests.shruns the test files in a bash 3.2 worker pool,PARALLELat a time (default: the CPU count), output buffered per test and printed whole in list order, the ten minute deadline kept per test (polled every 0.2s instead of 1s), and every failure reported at the end instead of stopping at the first: the files were already independent (each builds its own HOME and mock state under mktemp; the realpsfallback only reads, the mocks only read from the checkout). Local, this Mac: 512s one at a time, 233s with 3, 113s with 10.- The deadline’s process dump lists only the stuck test’s process tree: with tests side by side, the old
ps | grep tests/would name every running test. - The list is ordered longest first by measured time and a test file missing from it fails the run: the pool then never waits on a slow test started last, and a new file cannot be silently skipped.
SHARD=i/ntakes every nth test of that list: shards are balanced by construction (local, 3 workers each: 75s, 65s, 70s).- The macOS CLI job is a matrix of three shards, each running its share 3 at a time (macos-latest has 3 CPUs): the 11 minute test step should drop to about 2 minutes per shard. Three shards plus the app job stay under the five concurrent macOS jobs of the free plan.
- shellcheck runs as one more pool job in shard 1 (and on Linux), over the full file list the CI step had (
scripts/*.shandinstall.shincluded): it overlaps the tests instead of adding 25s in front of them, and needs no extra runner. - The separate v16 job is gone:
run-tests.shhas runtest-profile-v16.shsince 4d666ee, so it ran twice per PR. - The release bundle is built in the app job (
release-local.sh --skip-testsreplaces the plain Release build step, then the Swift tests run on the warm DerivedData): its Release build is a superset of the old unsigned build (it also signs ad hoc and verifies), and it saves a second runner, Xcode selection, XcodeGen install, cache restore and a full Release build (about 1.5 min on the critical path), while still producing the bundle on every PR. - The Linux job is gone (Akash, 0.5 pull request): BenchBar only runs on macOS, the macOS shards already run the suite on the
/bin/bash3.2 users have, and the Linux job was the slowest one left (4.5 minutes) while only ever finding GNU tool differences no user hits. The suite still aims to run on Linux for a Linux dev container, without a CI guarantee. brew installis skipped when the tool is already on the runner, and the Homebrew and DerivedData caches stay as they were: both already hit (restore key for DerivedData, which is expected when sources change).- Estimate after: about 3 to 4 minutes wall clock per PR, bounded by the slowest shard. First run on the 0.5 pull request: shards 2m28s to 3m50s, the app job 1m50s.
Batch port allocation
- Preserve usable allocations before choosing replacement blocks, so an early conflicting entry cannot displace another selected bench’s valid address. Fixed claims take priority, then managed benches, then newcomers.
- Approval uses a fingerprint of the complete plan and relevant configuration and ownership state. Apply recomputes under the existing CLI lock, then checks listeners and running state again per entry. Foreign processes do not share that lock, so launch also checks; no unrelated process is killed for a port.
- Persist reservations in existing per-bench state after successful setup. Registration alone remains read-only with respect to bench configuration.
- Apply reuses the backed-up adoption engine. This is sequential execution, not an all-or-nothing transaction: report completed entries on a later failure.
- Ignore stale saved automatic reservations on read; fixed reservations remain pinned, and current configuration always participates without duplicate claims.
- CLI starts distinguish hard live-owner conflicts from stopped overlaps that can be confirmed for one-at-a-time operation; positively owned running benches keep idempotent
up. - Include the adoption dry-run service plan in the approval fingerprint;
ports setuphandles human preview and confirmation while token-based apply remains available for clients. - Reject site/profile overrides for port planning and apply so the adopted service always matches the reviewed preview.
0.5.8: site backup and drop
site dropwrapsbench drop-siteinstead of dropping the database itself: bench’s own backup with files comes first and stops the drop when it fails, and the folder is moved toarchived/sites/, never deleted. benchbar only adds the refusals, the password on stdin and the hosts line.- The typed confirmation is checked by the CLI (
--confirm-sitemust equal the name), not only by the app, so no caller can drop a site by passing the name it already has; the app passes what the user typed. - The default site cannot be dropped without
--new-default, which runssite defaultfirst: dropping it would leave the runner pinging andbenchupwaiting for a site that is gone. The only site of a bench is never dropped. - A backup “set” is the files sharing the timestamp prefix, read from the folder, not from bench’s output: bench prints paths in formats that change between versions, and the folder also holds backups taken outside benchbar.
- The hosts line is removed only inside benchbar’s marker block and only when no other known bench has a site with that name; a line outside the block is the user’s and is only reported with the command to remove it.
- Without a terminal and a cached sudo credential the hosts step is not attempted and comes back as
manual_step: the app never asks for a password, as withsite hosts. site backupsis read only and takes no lock, likesite list, so the Sites tab can read it while another change runs.uninstall-serviceon a path that is not a bench falls back to the com.benchbar agents whose WorkingDirectory is that path, and touches nothing else there: the folder may be half deleted or reused, so only the agent is ours to remove.dead_agentslooks at every com.benchbar agent, not only the current bench’s: the bench whose runner is gone cannot run doctor any more, so another bench’s doctor is the only place the warning can show.benchbar consoleanddbexec bench (bench --site S console|mariadb) instead of running it as a child: the terminal, signals and exit code are bench’s, and no benchbar lock or trap outlives the shell.dbrelies on bench reading the site’s own credentials, so the root password is never involved.doctor --fix-hintslists failures before warnings and prints each fix once: several checks sharebenchbar repairas their fix, and an agent should run it once. Everything but the hints goes to stderr, so the output can be piped as it is.
0.6.0: update path
benchbar self-updateis the name, notupdate:app updateandprofile updateexist andbench updateis the command users fear, so the word alone would be ambiguous. It runs the one line installer instead of its own git pull, so the CLI, the app’s Update Now and the release notes all run the same command; once a release is offered, the installer comes from that release’s tag with--version, so the version confirmed is the one installed.self-updateexecs the installer pipeline:install.shpulls the checkout the running script lives in, and nothing of benchbar should run after that.install.sh --yeson a Mac whose CLI checkout already exists is an update: it adopts or installs no bench and never starts the Homebrew installer (the one step that asks for a password). An interactive run still offers both.- A CLI that is not
~/.local/share/benchbargets--app-only: a git checkout is the developer’s own branch, and a pull the installer did not start could fail on local changes or move them. The same rule is inlib/frappe-local/selfupdate.shand the app’sUpdatePlan, tested on both sides. - The release notes are built by
scripts/release-notes.sh(tested) instead of inline workflow shell, with an Update section first: 0.5.x apps open the release page from Check for Updates, so the copy paste command has to be at the top of that page.
0.6.0: dependency freshness
- Doctor warns about the dependencies of focus apps, never about a focus app: the developer pulls the app they work on themselves, so a warning about it is noise, while the apps it needs go stale unnoticed.
- Focus is inferred from local reads only (a dirty tree, a branch other than the profile’s or, without a policy, the remote’s default, a commit whose author is
git config user.emailin the last 14 days, read from HEAD and the local branches), so doctor needs no network to decide it. A pin (focus,ignore) wins;ignoreonly means “not a focus app”, the app is still checked when a focus app needs it. - Pins live in one per bench key (
APP_FOCUS="a=focus b=ignore") in the bench’s state file, not in the app’s repo or the lockfile: focus is about one developer on one bench, not the team. - The graph is
required_appsfrom hooks.py (asfl_app_required_appsreads it), followed transitively, frappe left out: frappe moves under every app, and a warning for it on every bench would drown the ones that matter. - “Behind” is counted against the remote tracking ref of the branch the app follows (
branch.B.mergeon its remote, else the same name), and the days are the age of the oldest commit HEAD lacks: that is how long the bench has been missing changes, not how old HEAD is. - The count and that oldest commit are taken on the first parent line (
git rev-list --count --first-parent HEAD..REF,git log --first-parent HEAD..REF): a merge of other history (india_compliance took in a repository with commits from 2011, and its version branch merges hotfix branches) put commits inHEAD..REFthat are years older than the gap and thousands more than the branch moved, so the days read 5439 instead of 92 and the commits 5541 instead of 14. A merged release counts as its merge commit, not as the commits inside it. - Doctor stays read only (Akash, 2026-09-29): no automatic fetch, not even once a day. Only
doctor --fetchorapp focus --fetch(one globalFL_FETCH, parsed like--fix-hintsand ignored by other commands) fetches the focus apps’ dependencies, side by side, 20 seconds each, with no prompt (GIT_TERMINAL_PROMPT=0, ssh BatchMode) and an explicit refspec into the tracking ref, so only.gitchanges. Without it the numbers are whatever git last fetched, the user’s owngit fetchincluded; FETCHED_AT is kept only to say how old benchbar’s last fetch is. The other apps are never fetched. - Offline is never a FAIL: a failed fetch keeps the last numbers, an app never fetched is unknown in the OK line, and
OFFLINE=1or--dry-runnever fetches. MCP’sbenchbar_doctorand the app’s doctor runs never pass--fetch. dependency_behindis the one check id that can appear several times (one row per stale dependency, each with its own fix); the doctor runner takes extra rows fromCHK_MORE, so the other checks are unchanged.- The fix is
benchbar app update NAME, which already fast forwards with a changelog, backups and migrate; a dependency with local changes getsgit statusfirst. Neverbench update. benchbar app focuswithout a name lists, instead of a new top level command, andapp list --jsoncarries the same fields, so the app gets them from the read it already does.
0.6.0: profile sharing
- Export reads each default branch through the URL as it is on this Mac and writes the portable one: an SSH alias such as
github-exponentexists because it carries another account’s key, andgit@github.comwith the default key is refused for exactly those repos (found on Akash’s Expo profile, 15 of 23 apps). Any app on the base’s release branch keeps it, not only registry apps: india_compliance onversion-15would otherwise be exported ondevelop. profile list --jsonkeepssourceas the kind (addingimportedandsubscribed) and puts an import’s or subscription’s URL in the newsource_url: changing whatsourcemeans would break readers under schema 1, adding a field does not.- A reviewed plan is bound to the content it showed: the app add token hashes each planned repo’s commit (and apply checks the clone is at it before build and install), and profile import, update and export plans carry a
digestthat--expectchecks (the app also passes every export branch, not only edited ones), so a source that moves between review and apply is refused instead of applied unseen. downkills a port listener only when its folder is readably inside the bench (fl_pid_is_bench_own_strict); the lenient check stays for conflict detection, where an unknown folder should not block a start.- Update Now and Copy Command fetch
install.shfrom the offered release’s tag and pass--version, so the app installed is the one the prompt named; the CLI still followsmainwithgit pull --ff-only, since a tag would leave the managed checkout detached. - Freshness keeps one
FETCHED_ATper bench and advances it only when every fetch of the run worked, rather than one time per app: the age note is a caution, and a partly failed run should keep it showing. - The anonymous access probe runs git with
GIT_CONFIG_GLOBAL=/dev/null, no system config and an empty credential helper: a globalurl.insteadOfthat rewrites https to SSH would otherwise make every private repo look public. personalneeds both a failed anonymous read andapi.github.com/users/OWNERsayingUser: a public repo in a personal account is fine to share, so it stayspublic.- Export keeps the release branch of a registry app (erpnext on
version-15for a v15 base): the repo’s default branch isdevelop, and following it would put a v15 bench on the development branch. Every other app follows its default branch, as agreed. - Export drops a pinned
commitwhen it changes the app’s branch: the pin belongs to the old branch. - Import adds
schema = 2andsourceat the top of the stored file and removes the file’s own lines for them, so what is stored always says where it came from and parses as schema 2. profile checkexits 0 whatever it finds: the answer is in the output, and the app and MCP read the JSON. A missing or invalid profile still exits 1.- Reachability runs
ls-remote --exit-code --heads --tags REPO BRANCH: exit 2 (no such branch) isfalse, like a denied read, because install would fail on it too. A resolve or connect error and the 10 second limit arenull, notfalse, so offline never removes apps. install --profileskips onlyfalserepos. With a repo that isnullit goes on and fails at get-app as before, rather than silently build a smaller bench offline.- A subscription’s profiles are
profiles/*.tomlwhen that folder exists, else*.tomlat the root: a config repo often keeps its ownbenchbar.tomllockfile or other TOML at the root. - Clones and fetches run with
core.hooksPath=/dev/nullandGIT_LFS_SKIP_SMUDGE=1, and subscribe clones into a temporary folder first, so a repo that holds no valid profile leaves nothing behind. profile_outdatedfollows the read only doctor rule of dependency freshness: the subscription is fetched only withdoctor --fetch;.git/benchbar-fetchedin the clone dates the last good fetch for the age note.- The
--planforms of export, import, subscribe and update take no CLI lock, likelistandcheck, so the app can preview while an install runs. profile removerefuses a profile that is your own file or aBENCHBAR_PROFILE_PATHfolder: it only undoes what import and subscribe did.
0.7.1: the app’s CLI
- BenchBar.app carries the CLI (
Contents/Resources/cli, install kindapp) and Homebrew’s and the installer’s copy exec it through~/.local/state/benchbar/bin/benchbar: the app and its CLI are one version whichever channel updated what, without removing a channel or rewriting anyone’s PATH. The app always wins over a standalone copy, even a newer one, so the CLI never drifts from the app. - The hand off is decided in
install-kind.shfrom a fixed path and-x, with no process: status runs it on every poll. A dangling link (the app in the Trash) means no hand off.BENCHBAR_HANDOFF_FROMstops a second hand off and is unset at once, so no child sees it. - The app’s CLI records the link as
FL_SELF, the same path whether the app or a handed off copy ran it, so the helper block does not flip between the two. A block naming Homebrew’s or the installer’s copy that hands off is kept as current: rewriting it would only churn~/.zshrc. - An app copy that is not the registered one (its
FL_SELFis its own path) never puts that path in the helper block: it keeps the recorded benchbar unless that is gone or an app’s internal CLI, else names Homebrew’s copy, the installer’s or the link (-x, no version test: each is whatbenchbarin Terminal runs). Its own path is used only when nothing else is installed. - Only an app in
/Applicationsor~/Applicationsmakes the link: an Xcode build, the DMG and a translocated copy would leave it leading somewhere that goes away. - Only Homebrew and the installer hand off, not
other: a copy of unknown origin (a test’s temporary tree) keeps running itself. - The CLI is embedded by
macos-build.shbefore signing, not by an Xcode build phase: the files are sealed with the bundle, user script sandboxing stays on, and Xcode’s own Debug builds simply carry no CLI. Tracked files only, as in the tarball.
Cloud sessions
- Cloud only agent notes live in .claude/cloud-context.md and reach Claude Code cloud sessions through a SessionStart hook gated on CLAUDE_CODE_REMOTE; local sessions and AGENTS.md are unchanged.
Linux and Windows
- The SessionStart hook prints linux-context.md in a local Linux session (WSL included) and windows-context.md when OS is Windows_NT; the Mac still prints nothing and cloud sessions still get cloud-context.md, checked first.
- “rm -rf outside the repo and the state folders” is a PreToolUse hook (.claude/guard-rm.sh), not a deny rule: permission rules match a command’s prefix and cannot express “outside”, so a deny rule would either block the repo too or miss
rm -fr. A variable target is blocked because the hook cannot know where it points. It fails closed: without python3 a call mentioning rm is blocked, and a relative target resolves against the call’s cwd and every directory a cd or pushd before it may have left the shell in (a cd that may fail keeps the old directory possible; one it cannot follow blocks relative targets). - The
cli-linuxCI job is back (it was dropped in 0.5 when BenchBar was Mac only): the Linux platform layer is a supported target now, so a Linux regression is a user bug, not a GNU tool curiosity. One unsharded job on ubuntu-latest; the macOS shards are unchanged. - CI speed with Linux back (PR runs had grown to 10 to 13 minutes, the unsharded Linux job the slowest at 9.5 to 13): the Linux tests run in four shards like macOS, the macOS tests in four instead of three (four plus the app job are the five concurrent macOS jobs of the free plan), and shellcheck is one job of its own instead of a pool job in both the Linux job and macOS shard 1 (85s each, the same 0.11.0). The macOS shards leave out the Linux platform tests (
SKIP_TESTS): Linux users run that code on bash 5, which the Linux shards cover; locally on a Mac they still run. The test list is re-sorted by measured CI times. The required checks in themainruleset name the macOS shards, so they change with the shard count. - The Windows self hosted runner is a logon task, not a service: it runs only while Akash is logged on, so a laptop that is off or logged out simply queues jobs, and nothing runs repo code as SYSTEM. Public repo safety comes from the job’s
if(pushes to main, same repo PRs), not from the runner. run_shellcheckreturns 1 when shellcheck fails: it runs inside anifin the job pool, where errexit is off, so a failing shellcheck still printed “shellcheck: ok” and the job passed. That is how SC2153 and SC2017 sat on main with a green macOS shard.- Linux breakage list (2026-10-10, Ubuntu 24.04 in WSL): the mock suite fails only 3 of 47 tests on Linux (two need
zip, one assumes an en_US.UTF-8 locale), because the mocks stand in for every Mac tool. The real CLI stops at its first step,fl_platform_init. Behind that gate the code read finds about 110 sites in six groups: Homebrew for every dependency and path (the largest), launchd for the whole service lifecycle and bench discovery (~/Library/LaunchAgentsglobs), lsof and BSDpgrep -q/pgrep -lfor processes and ports (procps rejects-q, so “is it running” is always no), the Keychain for the MariaDB root password,/etc/hostsand Mac only doctor checks, and a handful of path and text spots. GNU versus BSD tool differences are nearly absent (earlier work already added GNU fallbacks). The fix is one Linux platform file with the same function names plus a systemd —user backend behind the existingfl_agent_*names; the full table is in the “docs: Linux breakage list” pull request. - One CLI for both platforms:
fl_platform_loadpicks macOS or Linux once at startup from bash’s$OSTYPE(nounameprocess, status polls are cost tested), overridable withFL_PLATFORM, and on Linux sourcesplatform-linux.shandsystemd.shover the macOS functions with the same names. Callers never branch;fl_is_linuxis only for skipping a whole Mac feature (a doctor check, the hosts step). - The test harness exports
FL_PLATFORM=macosby default, so a Linux machine running the suite still tests the Mac path;use_linux(orBENCHBAR_TEST_PLATFORM=linux) switches a test to the Linux layer and its mocks (ss, systemctl, loginctl, apt-get, dpkg-query, uv, fnm, wslview, xdg-open, getent). - Ports and process folders on Linux come from one
ss -Hltnpparser and/proc/PID/cwd, normalised to the lines lsof gave (*:PORTfor any address), so the callers that parse them are unchanged. A socket whose owner ss cannot show (another user’s) counts as a listener with pid 0, never as a bench process. - The
stat -c ... || stat -f ...fallbacks take each form’s output only when it is a number. GNUstat -f %m PATHreads%mas a file name: it prints file system text for PATH and fails, so the joined output made a lock’s age 0 on Linux and once reached the arithmetic offl_state_guard_stale. A guard whose time cannot be read is not stale: the guard is never dropped on a guess. - On Linux the MariaDB root password lives in
$FL_STATE_DIR/secrets/mariadb-root(folder 0700, file 0600, written under umask 077 through a temp file and read back), behind the samefl_keychain_*names;MARIADB_ROOT_PASSWORDstill wins. - On Linux the state folder follows
$XDG_STATE_HOME(absolute values only, per the spec), the helper block goes to~/.bashrcunless the login shell is zsh, and the next steps saywslview(WSL) orxdg-openinstead ofopen. - On Linux each bench is one
systemd --userunit,benchbar-<name>.service, rendered fromtemplates/systemd-user.service.tmplwith the same version hash header and driven through the samefl_agent_*names (systemd.sh overrides launchd.sh).Restart=on-failurewithRestartSec=20mirrors KeepAlive SuccessfulExit=false and ThrottleInterval 20, andStartLimitIntervalSec=0leaves the decision to stop restarting to the runner’s crash guard, as on the Mac. - Autostart is a
# benchbar-autostart:comment in the unit plussystemctl --user enableordisable, so toggling it changes the hash and repair rewrites the unit, as RunAtLoad does in the plist.loginctl enable-lingerruns when a bench starts with autostart on; a refusal is a warning, not a failure. - The runner stays one template: its three macOS-only lines (notify, a process’s folder, a port’s listeners) are
#@defaultvariables whose defaults are the old macOS text, so the macOS runner renders byte for byte as before (same hash, no “outdated runner” warning) and Linux passes:,/procandssinstead. - A bench’s unit that is moved aside goes to
systemd/user-disabledin the config folder, the counterpart ofLaunchAgents-disabled; nothing in the bench is deleted. - Linux phase 00 runs its sudo sections (apt, MariaDB admin over
sudo mariadb, the drop-in, the .deb) before the uv and fnm installers and drops the credential in between, so third party installer scripts never find a cached sudo; a fresh root is changed only when its hash isinvalid, empty or NULL, and keeps unix_socket next to the new native password. - Doctor on Linux leaves out the checks that only mean something on a Mac (formula dates, Python leaves, legacy agents, CleanMyMac, Mole, app copies, Full Disk Access, fork safety, a Redis on 6379, which is apt’s own service there) instead of reporting them as OK: an
[OK]line for something that cannot exist would read as a check that ran. The rest keep their ids, sodoctor --jsonreaders andrepairactions need no Linux case. wsl_time_syncwarns in WSL when NTP runs in the distro: the Hyper-V host sync cannot be turned off from the guest, so two clocks fight and each timesyncd poll steps the clock when Windows has drifted (a step ended a worker’s Redis wait and stopped the bench). Its fix issudo timedatectl set-ntp falsewith no repair action, becausesudoon Linux is kept for apt, the wkhtmltopdf .deb and MariaDB; outside WSL the check is left out.hostson Linux checks that the site name resolves to loopback (getent hosts), and the fix names a*.localhostsite, which needs no/etc/hostsline; benchbar never edits/etc/hostson Linux.fl_sudo_beginstarts the session withoutsudo -vwhensudo -n -k truesucceeds, that is, when a NOPASSWD rule applies: with Ubuntu’s%sudoline also matching, sudo’s defaultverifypw=allmakessudo -vask for a password that no command needs, and without a terminal the install stopped there.-kmakes the probe ignore a cached credential, so a cached one still goes throughsudo -vas before. Found on the first real install in WSL.*.localhostdoes not resolve through glibc on Ubuntu in WSL (nsswitch isfiles dns, and WSL’s DNS proxy does not answer.localhost), although browsers on Windows resolve it themselves. The Linux install addslibnss-myhostname, whose own install script putsmyhostnameon the hosts line of/etc/nsswitch.conf, so wkhtmltopdf and Python inside the distro reach the site by name; benchbar edits neither nsswitch.conf nor/etc/hostsitself (WSL rewrites/etc/hostson every start). Doctor’s fix for a*.localhostname that does not resolve names the package (Akash chose this over a hosts line, 2026-10-10).- v16-lts on Linux is refused for now: it needs MariaDB 11.8 from the vendor repository and Python 3.14, which this session did not set up or test. Later.
cli-linux-realruns on pushes to main and on pull requests labelledtest-linux, but ci.yml keeps the default pull_request event types, so the label takes effect on the next push (or a re-run), not the moment it is added: addinglabeledto the types would rerun every CI job on any label change.cli-linux-realstarts a systemd user manager on GitHub’s runner itself (loginctl enable-lingerplussystemctl start user@UID, thenXDG_RUNTIME_DIR): the hosted Ubuntu VMs boot systemd, but the runner’s job is not a login session, sosystemctl --userhas no bus until linger starts one. It also purges the image’s MySQL, whichmariadb-servercannot install over.- The unit uses
KillMode=mixed: a stop sends SIGTERM to the runner alone, which forwards it to honcho as launchd’s stop does on the Mac, and whatever is left in the unit’s cgroup afterTimeoutStopSec=30is killed.control-groupwould signal every bench process at once, before the runner could write its stopped state. - fnm stays outside PATH (phase 00 installs it with
--skip-shell, as uv’s installer runs withUV_NO_MODIFY_PATH=1): benchbar edits only its own block of~/.bashrc. Repair and doctor’s fix lines call fnm by its full path when it is not on PATH, and the helper block adds fnm’s folder. loginctl --no-ask-password enable-linger: polkit allows a user to enable their own linger without a password only in an active session; elsewhere it would put an admin password prompt in the middle ofbenchbar up. Refused, it is a warning with thesudo loginctl enable-lingerfix.- The Linux helper block prepends fnm’s folder and
~/.local/binfirst and the profile’s Python and Node last, so an oldernodeorpython3.11in~/.local/bincannot shadow the profile’s (eachexport PATH="X:$PATH"line wins over the ones before it). Doctor reads the tools through the unit’s own PATH (fl_launchd_path_value), the sharedfl_bench_which, not a copy of the shell block. - A start job systemd refuses (a masked unit, a start limit) fails
fl_agent_bootstrapinstead of counting as “agent loaded”, soserviceandrepairreport it. A runner that starts and then fails is the crash guard’s andstatus’s to show, as on the Mac.
Windows shim (2026-10-10)
benchbar.exerunswsl.exe -d D --cd ~ --exec /bin/sh -c WRAPPER benchbar ARGS, notwsl.exe -d D -- benchbar ARGS: with--, wsl.exe joins the arguments into one line for the user’s shell, which expands$and splits quoted spaces, and a non-login shell has no~/.local/binon PATH, sobenchbarwas “command not found” on a fresh install.--exechands the arguments straight to the program after wsl.exe splits the Windows command line, and the wrapper (PATH="$HOME/.local/bin:$PATH"; command -v "$0" || exit 127; exec "$0" "$@") adds the installer’s folder and passes"$@"on unchanged. It sources no profile, so nothing a profile prints can corrupt JSON or MCP output.- The shim hands its own stdin, stdout and stderr handles to wsl.exe instead of copying them, so there is no buffering and no line ending translation: JSON and MCP’s stdio framing pass byte for byte.
- A missing distro is caught from the Lxss registry key before wsl.exe starts (no extra process on every call). wsl.exe’s own error for it is UTF-16 text with exit code 127, which cannot be told apart from a missing command. Wrapper exit 127 means
benchbaris not installed in the distro: one line with the fix, exit 1. Exit codes outside 0 to 255 (wsl.exe’s own failures) become 1. - Ctrl+C: the console sends it to every process attached to it, so wsl.exe gets it and passes SIGINT into Linux. The shim ignores it and waits for the CLI’s exit code, and does not send it again. wsl.exe runs in a job object that kills it when the shim dies, so a client that kills
benchbar.exe mcpdoes not leave a wsl.exe behind. The job allows silent breakaway: without it, every Windows program the CLI starts through interop (the browserwslviewopens) joined the job and was killed when the command returned. - One dependency,
golang.org/x/sys(from the Go team): registry reads and writes, the job object, the WM_SETTINGCHANGE broadcast and the memory size would otherwise be hand written syscall tables. doctor --jsonanddoctor --fix-hintsare pure passthrough; the Windows checks print only in the text form, so a JSON reader sees the same output as on the Mac.%LOCALAPPDATA%\BenchBar\config.jsonkeeps keys the shim does not know when it writes, so the tray can share the file.cli_pathpoints at anotherbenchbarinside the distro; the integration tests also use it to send exit codes and quoting through the real hop.- The keepalive is a Task Scheduler logon task, the runner’s pattern (interactive only, hidden through
conhost --headless, restarted on failure), registered from XML withschtasks /Create /XMLas the user, without an elevated prompt. Its action isbenchbar.exe keepalive run, which boots the distro with/bin/trueand keeps onesleep infinityalive, restarting it with a backoff. Awsl --shutdowntherefore stops the benches for at most the backoff time. That is the purpose: an idle distro is what WSL shuts down. schtasks /Endstops only the task’s conhost, notkeepalive run, so the run identifies itself instead. A named mutex (Local\BenchBarKeepalive) keeps it to one run. A pid file holds its pid and process creation time, andremoveandinstallterminate that pid only when the image is benchbar.exe and the creation time matches. A reused pid (abenchbar.exe mcp, say) is never killed.installover a running keepalive stops the old run, so changing the distro takes effect at once.wslconfig --applychanges onlymemory(only when below 4 GB),vmIdleTimeoutandinstanceIdleTimeout, after a timestamped backup beside the file. It never runswsl --shutdown, which would stop running benches; it says that the new values apply at the next start.- Smart App Control in enforce mode (the default on a fresh Windows 11 install) blocks an unsigned
benchbar.exeoutright, as it blocked every build and test binary on the development laptop until Akash turned it off (2026-10-10). The CI artifact is for testing only. A release needs a trusted Authenticode signature, which this session leaves to the release work, together with the installer. - The job object is attached right after
Start, not with a suspended start: wsl.exe starts no child of its own before it talks to the WSL service, and a suspended start would need hand writtenCreateProcesscode. adopt-distrochanges nothing in the distro except linger, and only after a yes: turning on systemd needs root and a distro restart, and the install is interactive, so both stay fix lines..claude/guard-rm.shnow lets a variable target through only when it resolves to a path inside a/tmp/claude-*session scratchpad. The value must come from the environment or a literal assignment earlier in the same command, and the name must not be set any other way in it (declare, read, for, eval). The resolved path must contain no whitespace and no symlink that leads out. The command must not mentionIFS. A glob may appear only in the last component, never inside the variable’s value:rm -rfon a symlink removes only the link, but a glob before a slash could pass through one.find -exec rmis blocked, and so is anyrm -rftarget with a brace expansion ({a,../..},{1..3}), which bash expands before rm runs, for plain paths too. It reacts tormas a word, includingrm''and a tab after it. It blocks, instead of skipping, a piece it cannot parse, rm insidesh -coreval, andxargs rm -rf, and it followsthen,doandif. Both guards compare command names without case and without a.exesuffix: Git Bash runsrm.exeandgh.exe, and Windows findsRMasrm. It looks forrmanywhere in a command, so a wrapper with options (nice -n 5,sudo -u x,timeout 5) cannot hide it. It skipscdoptions before the folder, and counts an assignment only when it is sure to run (not after&&,||, a pipe or inside parentheses). It also blocks when python3 does not run or crashes: on Windowspython3is the Store stub, which is on PATH but exits 9009, and the guard used to let every call through.- GitHub API writes and
schtasks /Createare checked in.claude/guard-writes.sh(pure bash, so it also runs in Git Bash without python3), not by deny rules. Besides the review thread resolve route it allows a thread reply on this repo and GraphQL that only reads or resolves review threads (Akash, 2026-10-10, so Autofix can answer and close Codex threads), each only as the whole command with the text from a file (-F body=@FILE,-F query=@FILE): free text on the command line could chain another command. A deny rule always wins over an allow rule, so it cannot express these exceptions, and a call with a field flag is a POST that the old-Xdeny rules missed. It reads only the command, never the description, and checks eachgh apiorschtaskscall on its own text up to the next one. It handles combined short flags (-iX,-fbody=), and schtasks switches with/or-in any case. A command chained after a call is part of that call’s text, sogh api ... && tail -f logis blocked: it errs toward blocking. - The settings keep the command specific
~/.sshand/mnt/cdeny rules and the read only allow list Akash chose over the old wildcard denies. That trades some cover for fewer prompts:python3 *,find *andawk *can run code that no rule sees. Narrow denies close the plain ways around: reading~/.sshwith grep, sed, awk, diff, python3 or find, andfind -delete.
Recovery and agent contracts (2026-10-29)
site restoreis built on pull’s restore steps (fl_pull_step_*), not a copy: it fills pull’s globals from a local backup set and runs the same backup, quiesce, new-site, restore, key, hook, migrate and cache steps. What it adds is the plan, the token, the local source and its own verify. A restored site is the same site, so it gets nomute_emailsand no paused scheduler (pull’sdev_safetystep is left out).- The plan’s token hashes the plan body and the bench state it was made against: the site list, the default site, the hook’s SHA-256, the backup set (size and time) and whether the bench is running.
--applyrecomputes the plan and refuses a different token, so a set rewritten, a site added or a hook edited since the review voids it. - The quiesce step of site restore looks at honcho, socketio and the workers (
fl_pm_running), not at listeners (pull keepsfl_bench_is_running): a Redis started byup --services-onlywould otherwise make the bench look “running”, add a step and change the token between a plan and its apply. - The backup an existing target gets first is checked against the set being restored: bench names a backup by the second, and one taken in the second of the set’s stamp would overwrite it. A changed size or time of the set after that backup fails the run before the restore.
- The root password is judged in the plan without being tried (
fl_mariadb_root_password_known: the environment or the store), so a plan says “unknown to benchbar” (exit 2 on apply) instead of passing with the dry run’s placeholder; the message never names where the password lives. Apply tries it against the server before anything changes. - A new site is created with
ADMIN_PASSWORDor a generated password that is never shown or kept: the restore brings the Administrator password of the backup, so the onenew-siteneeds lives only until then. Pull’s ownadmin_passwordstep still sets one on request. - The
--before-migratehook must be an executable file (nobash -c, no shell text). It runs with the bench as its working folder andBENCHBAR_SITE,BENCHBAR_BENCHandBENCHBAR_DB_NAMEset, stdin on/dev/nulland fd 3 (the JSON stream) closed, andenv -ufor benchbar’s own passwords,MYSQL_PWD, every variable named like a secret (PASSWORD, PASSWD, SECRET, TOKEN, PASSPHRASE, CREDENTIAL, APIKEY, or ending in_KEY,_PASS,_PWD,_PAT) and every variable whose value is a URL with a user name and password. Without--no-migrateit runs before migrate; with it, at the place migrate would be. The scrub is hygiene, not a security boundary: the hook is the user’s own script, run as the user. - The default site is recorded before the run and compared at the end. If a step moved it (a hook, a bench command), it is put back with
bench useand a warning says so; benchbar never editssites/currentsite.txtitself. pull --dry-runused to print a full plan for--replaceon a folder that is not a site and then fail atbench backup, its first step. The check now refuses a target without asite_config.json, an unfinishednew-site(its marker) and an unknown root password, in the dry run too. Pull creates a missing target withbench new-siteas a plan step, because a restore needs the site to be there.status --jsonreads the Redis ports and asks the other sites forservedwith bash’s/dev/tcp, not lsof or curl: the status path of a running bench is held to its program budget (test-status-cost), and a connect or one GET from bash starts no new program (a subshell for the probe). Each site probe waits one second and all share a 2 second budget, and none is made when the default site did not answer its ping; what is left is null. Only a 404 means “not served” (Frappe’s answer for a Host with no site folder); no answer isnull, neverfalse. A stopped bench is not asked at all.FL_LOCAL_PROBES=0turns both probes off, and the test harness sets it so no test meets a service of the machine it runs on.up --services-onlywrites no stop flag and nostate.jsonand needs no installed service: it is for a bench that is being repaired. The Redis stays up untildown, which already stops the listeners that run in the bench folder.- A pinned app is cloned with
--skip-assets, checked for the commit (git cat-file, afterfetch --all --tagsand, for a shallow clone,--unshallow), checked out, and built once.bench initgets--skip-assetsfor a frappe pin the same way. So an unreachable commit is refused before the first build, not after it. The pin comes from the flag, else the environment (FRAPPE_COMMIT, as before), else the team profile’sfrappe_commitanderpnext_commit. benchbar.tomlalready carriedcommitfor every app, frappe and erpnext among them, inlock write,lock checkandlock apply; only its output changed (lock checknames the pins).lock check --jsongets no new key, because the app’s fixture is compared key for key.site restoreinto an existing site restores from a copy of the set under the state backups (<state>/backups/<stamp>/restore-set/, hard links, else copies, sizes checked), made before the backup first: frappe’s backup deletes backups older thankeep_backups_for_hoursinsites/TARGET/private/backups, which is where--from STAMPreads. The copy stays after the run and its path is printed and in the result (set_copy).- The backup taken first must be a new stamp with its database and both file archives; otherwise the run stops before the restore, and there is no fallback to the newest older set.
- The plan runs pull’s frappe version gate (
fl_pull_frappe_too_new): a backup of a newer frappe than the code here iscan_apply: false. A local set carries no app versions in its dump yet, so the gate only fires where versions are known; it is there for the day they are. - A dry run (pull and the restore plan) does not exit 2 for an unknown MariaDB root password when a terminal run would ask for it: the plan lists
needs: ["MARIADB_ROOT_PASSWORD"]and stays applicable. Runs that cannot be asked (--json,--yes,--apply, no terminal) keep exit 2 and the error. downwith no agent loaded and no bench process (only a Redis fromup --services-only) stops that Redis, waits for nothing and leaves an existing stop flag (crash, broken, port_conflict) andstate.jsonalone; it writesmanualonly when there was no flag.up --services-onlystops the Redis it started when the ports do not come up.site restorerewritesencryption_keyin an existing site’ssite_config.json(through a temp file and a rename, as pull does) with the one from the set: the key must match the restored data. That is the one write to a file undersites/that benchbar makes outside bench’s own commands, and it is the point of the command.- A pin that is already checked out changes nothing and asks for no rebuild, so a second install is a no-op. A clone fetched with
--skip-assets(bench init or get-app) leaveslogs/.benchbar/needs-build; install builds when it finds it, so a run that stopped at an unreachable pin does not leave a bench that a later run without the pin never builds. A commit missing from a single branch or shallow clone is fetched by its hash from each remote (GitHub hands out reachable commits) before the pin is refused. - The frappe version of a local set is read from the first row of its
tabInstalled Applicationtable, streamed (gzip -dc,grep -m1), so the version gate fires forsite restoreandpull --from-dir. The restore gets stdin on/dev/nullin every mode, so frappe’s downgrade question can never wait on a terminal. downdecides by “no bench process and no running agent”, not by “agent not loaded”: an installed agent stays loaded (launchctl print succeeds, a systemd unit isloadedwhile its file exists), so loaded says nothing. The agent still gets a harmless SIGTERM.site restorecopies the set only when it is in the target’s own backup folder (the one bench’s backup prunes); a set from anywhere else is restored from where it is. A copy that cannot be hard linked is checked against the free space first and removed when it fails. The three newestrestore-setcopies are kept, the older ones go.- A command that already printed one JSON document with
--jsonkeeps printing exactly that one line (the Mac app decodes the whole stdout, and nothing undermacos/changes); thedonefields are added to it.--jsonlasks for the stream instead, so one flag never means two shapes. up,down,restart,service,site add,site default,app update(without--dry-run) andports applyhad no JSON, so plain--jsonstreams for them, as forinstall,adopt,repairandpull; there is no second format, the stream isplan,step,progress,done.--for-agentpicks per command frommutatinginlib/frappe-local/commands.sh: the stream where a command changes things, one document where it reads. One table feeds--help,commands --jsonand this choice, and a test fails when a dispatcher label has no entry.- The table is read into a variable with the
readbuiltin:statusandlistare polled andcatwould be a process on every poll. Those two commands do not even look at the table unless--for-agentis given. - A dry run with
--jsonis oneplan(a stream:plan, thendone). Commands that know their plan late (service,site drop,site backup,app add,app update) send their own; for the others the plan is the command and the dispatcher sends it before anything runs. done.exitis the process exit code (the EXIT trap sees it), so a pipe cannot hide a failure;repairkeepsexit_codeand gainsexitwith the same value.state_afteris thestatus --jsonobject computed in a subshell, without a per site ping;nullin a dry run, without a bench, and forports apply, which changes several benches. A stream reads it after the locks are released; a one document command (andpull) reads it while its lock is still held, which is only a read and keeps the document the command’s last act.changed[]is filled by the code that changes something (fl_changed),next_actions[]from thefix:lines (the why is the last warning or failure before it),warnings[]from[WARN]lines; the engine records every action that endeddone.- The steps of
up,downandrestartgo throughfl_step_beginandfl_step_endwithFL_STEP_QUIET=1: the stream gets the step lines, the terminal keeps its one line each. - A refusal under
--jsonis text on stderr and stdout stays empty (fl_dieswitches stdout to stderr), asRules for readersalways said; before, it was on stdout. BENCHBAR_NONINTERACTIVE=1refuses before the lock, the log and any bench call, from a preflight in the dispatcher that names every missing input in one error (fl_need,fl_need_check). It is not the CLI’s job to guess the answer to a question:fl_confirm,fl_askandfl_ask_secretdie naming the input instead of reading the missing terminal as no or taking a default.MARIADB_ROOT_PASSWORDis declared only bysite addandsite drop, and only when neither the environment nor benchbar’s own store has one;installcannot know (a fresh MariaDB gets a generated one) and keeps its exit 2. The message never says where the password is stored.site addasks forADMIN_PASSWORDfirst, before the apps and MariaDB checks.uninstall-servicewith no terminal and no--yesis refused in any mode (it used to endCancelled, which is a “no” nobody typed);--allwith--bench-dirkeeps its own message.- Steps and sudo under the mode: a plain
installrefuses a sudo that would ask;--jsonskips the root steps with their commands as it did. The hosts line ofsite addneeds--yes. not_adoptedis only decided when nothing runs and no stop flag exists, from file tests and reads in bash (the agent file, the primed default flag, the bench’s state file,registered-benches.txt). Itssiteandweb_urlarenull: the default bench’s site is never named for another folder.bench_bincomes from the PATH baked into the agent file (plist or unit), read in bash and searched with file tests, so the status path starts no process. Without a service it isnull; the PATH of the shell would be a guess about a different process.status --pingand the MCP status ask the sites only while the bench’s processes run; a stopped bench is never pinged, andlistnever pings.site addrecords the default site beforebench new-siteand puts it back if bench moved it; onlysite defaultmoves the default.doctor --ackis stored per bench in its state file asACK_<id>with the message acknowledged, so a changed message (a different warning) lapses it and nothing else needs a clock. A FAIL cannot be acked;repairignores acks (it fixes what is wrong, whoever has seen it).doctor --quickleaves out the checks that start brew, python, node, bench or git or ask the site (FL_QUICK_SKIP), and names them.- The stale lock line names the command and subcommand and the start time, from an
infofile written when the lock is taken; never an argument, which could hold a token. logs --jsonkeepslinesas strings and adds a parallelts, so no reader breaks. Honcho lines carry onlyHH:MM:SS: the day is worked out backwards from the file’s mtime (a time later than the line below it is the day before), and local time is converted with the zone offset of now. Only the newest 20000 lines are looked at, so a 50 MB log costs bounded memory. The calendar arithmetic is awk (BSD awk has nostrftime).- The MCP server sets
BENCHBAR_NONINTERACTIVE=1for every CLI it starts, runs the actions with--jsonand returns thedoneline asresultnext to the oldexit_code,outputandstatus;benchbar_statusand the status after an action use--ping. - The fixture checks in
tests/test-json.shand the install stream test now assert that every key of the app’s fixtures is still printed, not that the sets are equal: fields may be added, never renamed or removed, and nothing undermacos/changes. - A key-value file is created with
: >>(append nothing), never: >: two first writers both found no file, and the slower one’s truncation emptied what the faster one had written under the lock (a lost key in test-run-lock on CI; with a 0.3 s pause before the create, the old code lost a key in 10 of 10 runs, the fix in none). Its lock (FILE.lck) is also broken only when it is 10 seconds old, not after 2 seconds of waiting, so a long queue of live writers is never broken into. - A refusal of the non interactive mode is part of the stream: the stream (or the document) starts before the preflight, so
install --jsonandsite add --for-agentend with adonecarryingexit,errorandmissing, and still take no lock, write no log and call no bench. It exits 2 when the MariaDB root password is among the missing inputs, as AGENTS.md promises forsite add,site dropandpull. changed[]records only what changed: the shared library functions (bench init,get-app,new-site,install-app) callfl_changedthemselves, so a second run is empty. Phase scripts are child processes: they appendkind,name,detaillines to the fileFL_CHANGED_FILEnames and the install folds them in when it builds thedone.doctor --ackwrites under the CLI lock, answers with one small document under--json, and an ack-only state file does not make a benchnot_adoptedlook adopted (fl_bstate_file_substantive).site addputs the default site back only when there was one; it re-readscurrentsite.txtand the remembered site afterwards and fails the step if either is not the old default.state: "not_adopted"stays a new enum value: the Mac app decodes unknown states as unknown and shows such a bench as “State unknown” until it learns the value, which the unknown-enum rule allows.repair --jsonkeeps the real stderr on fd 4: a refusal prints its[FAIL]reason there and ends with adoneline (exit1,error) even in a dry run, because the Mac app summarizes stderr and stdout whenrepair --dry-run --jsonfails and an empty one says nothing. A dry run that works is still the plan alone.- A key-value file’s lock is also broken when its mtime is more than 10 s in the future (the clock stepped back, a VM resumed), and after 600 waits (about 30 s) whatever its age: a state write must not wait forever on a lock nobody holds. An age that cannot be read is never a reason to break it: under contention that is mostly a lock released a moment ago, and breaking then removed the next writer’s lock (with stat failing on purpose, 3 of 4 runs lost a key before the fix, none after).
site addwith neither the Administrator nor the MariaDB root password exits 2 in every mode, assite addalways did when the MariaDB password was the unknown one;ADMIN_PASSWORDalone missing is exit 1.- After a
site addthe default is compared with whatcurrentsite.txtsaid before, not with the remembered site: a user who ranbench use otheris not an error. - A refusal’s
donehasstate_afternull (no status is computed for a run that did nothing) and the reason is also one[FAIL]line on stderr, for a person who ran--jsonin a terminal. tests/test-install-json.shis split in two files of about equal runtime (test-install-json-2.shhas the no gui mode, adopt, ports apply under the dialogs and profile list) so that each stays well under the 600 s a test may take; every assertion is kept and the cap is unchanged.- An
app updateor plannedapp addthat stops part way leaves a pending record (<state>/benches/<bench>.pending/<app>.json, one line of JSON with the steps not yet done), and the next run takes its steps from the record, not from Git: after the fast forward,old == newsays nothing about whether migrate and build ran. - The pending record is read with bash regexes, not a JSON parser: bash 3.2 has none and the CLI path must not need python3. That works because the writer only stores benchbar’s own values (enum kinds, site and app names) and turns quotes and backslashes in a failure reason into
'and/. - The record lives in a
.pendingfolder next to the bench’s.envstate file, one file per app, written through a temp file andmv. Status and doctor read it with a glob andread, so the status path starts no process (test-status-cost holds). app updatewith a record trusts the record’sto, not the remote: the range the person reviewed is the one that finishes, and newer commits are counted (newer_commits) and wait for the next run. A fetch that fails is not fatal then, because finishing a migrate needs no network.- An app whose HEAD is neither the record’s
fromnor itstois refused rather than guessed at;--discard-pendingis the explicit way out, and it logs the record it drops. --skip-backupon a retry removes backup steps still in the record: it is the person’s explicit choice for this run.- A planned
app addkeeps its record when a step fails, but a clone that failed and was rolled back leaves only the build of the apps that did get cloned (or no record): an absent app is simply planned again. Installs on different sites stay independent, and a failed install does not skip the restart. - Dependency order is Kahn’s algorithm with ties broken by the old list order (the walk reversed, the main app last), so a plan that was already in a valid order is byte for byte the same. The app being added is a node of the graph, which is how a cycle through it is found.
- One timeout,
fl_run_capture_timeout, built on the existing process group helpers; the grace between TERM and KILL isFL_KILL_GRACE(default 1 second) so tests need not wait a second per kill. - The plain
app addnow goes through the plan and the shared apply loop. Behavior that changes: required apps are cloned before the app that needs them (apps.txt order), one question instead of two, and a plan that cannot be applied (an unknown required app) clones nothing, where the old flow cloned the app first and failed after. Its error hints (the remote’s branches, the ssh and gh lines) are printed from the plan’s errors. - The mock tests check that every key of the app’s fixtures (macos/BenchBarTests/Fixtures) is still printed, not that the CLI prints exactly those keys: the CLI adds fields this round (
pending_operationsand others) without a change under macos/, and the rule for readers is that fields may be added, never renamed or removed. - The plain
app addalso changes how it reads the world:hooks.pycomes from a filtered shallow clone (--depth 1 --filter=blob:none --no-checkout) before anything is cloned, so a server that rejects--filteror a repo whosehooks.pycannot be found now fails the plan where the old flow cloned first. An app already in the bench whose required apps are missing is planned (clone_required, build, restart) instead of refused, and Restart is always listed; it endsskipped (not running)when the bench is stopped. - An update record accepts HEAD at
fromonly while itsfast_forwardstep is still listed; after it,frommeans someone went back (thereset --hardwe print), and the run refuses and names--discard-pending. A record with no step left is checked once (HEAD atto) and cleared;app update NAME --discard-pendingalso drops anapp_addrecord, and it runs before the dirty, detached and remote checks because it only removes benchbar’s own file. - An
app_addrecord with no step left is verified and cleared when the app passes, otherwise the plain and the planned add fail and name--discard-pending. - A get-app that cloned the app and then failed (its pip install) puts a
requirements_pythonstep for that app in the record, before the build, so the install-app that follows never runs against an env that lacks the package. - Dependency order treats an app already in the bench as a pass-through: if A requires P (in the bench) and P requires B (to clone), B goes before A.
--discard-pendingdrops the record and returns: it never falls through into an update, because the person asked to forget a half done run, not to start another (with--yesthat would have run a backup, merge and migrate). With no record it says so and returns 0.- A record that does not end in
}(cut short) or is from another version is kept, not guessed at:app updateand the plan refuse to run from it, doctor warns, and--discard-pendingis the way out. A record that cannot be written stops the run before the first step (and counts as a failed step later), because a stop could not be finished without it. - A failed
requirements_pythonstep skips build, install and restart like a failed build, and the record keeps them. The record path is built only from a plain app name, so a name with a slash can never reach a file outside the.pendingfolder. app addsyntax for pins, batches and per app sites: the targets are the positional arguments, and--branch,--name,--commitand--install-onbelong to the target before them (to the first one when they come before every target, soapp add --branch B URLstill works);--site,--all-sitesand--applyare for the whole add. Chosen overNAME#commit=SHAorNAME@SHAbecause a URL can hold#and@, and one flag spelling serves one app and a batch alike.--install-onreplaces--siteand--all-sitesfor its app.--committakes 7 to 40 hex characters. The plan clones the branch shallow and blobless, fetches a full hash by id (GitHub hands out reachable commits), and for a short one fetches the rest of the history until it resolves;commitin the plan is always the full hash, so the token pins one commit. The plan readshooks.pyat the pin, not at the branch tip.get-appstill clones the branch (bench has no commit option); the checkout of the pin follows it (fl_commit_reach, then a checkout), before the build, and a clone that does not reach it fails like a moved branch (planned X, cloned Y; nothing was built or installed) and leaves the folder forbench remove-app. A pin on an app already in the bench must match its HEAD, else the plan has an error: benchbar never moves an app. The required apps are not pinnable; they are read and cloned at their branch tip, and checked against the commit the plan read.- A batch is one graph: the required apps of all targets are walked once (an app two targets need is one node and one clone), a target another target requires is a node, not a missing app, and Kahn’s sort orders all of them (ties by list order, targets in the order given). One build step covers every cloned app, one restart ends the run. Installs are one step per app and site, site by site and each site in dependency order, so a site’s first failure shows early; an install is skipped when an app it requires failed to install on the same site, and the other sites carry on.
- The pending record of a batch is one record per target that has work (not one keyed by the first app): doctor’s
benchbar app add NAMEfix andapp update NAME --discard-pendingwork on each app, and a retry of one app plans only its own installs. Each record lists the shared steps (clones, build, restart) and only its own installs. A target with no work (in the bench, installed everywhere, no missing required app) gets no record. A failed clone writes the same short record as before (requirements, build, restart) for every target with work; the installs are planned again from the site lists. - The apply result reports the install and its migrate with one status:
install-appruns the app’s patches and DocTypes sync, and there is no separate migrate step to report, so eachinstalls[]entry carriesmigrateequal to itsstatus.apps[]is always in the result, also for one app; in the plan it appears only when there are several, so the plan of one app keeps today’s shape. - The MCP site and service tools follow app add’s plan, token, apply:
site add --dry-run --jsonbecame a one document plan (likeapp update --dry-run --json), andservice’s plan line gained a token computed in the plan hook from its actions, the bench, the port block and the scheduler, soservice --apply TOKENis refused before any action runs when the plan changed. benchbar_site_addtakes the new site’s Administrator password or generates one (20 letters and digits, never starting with-) and returns it once in the result: a throwaway site needs a password the agent can use, and an agent asking the person for one would stall a background task. The MariaDB root password is never an argument or a result.- The MCP server withholds any line that names where the MariaDB root password is kept (
mariadb-password, thebenchbar-mariadbKeychain item,secrets/mariadb-root,find-generic-password), in one pass over every result and error just before the reply, text and structured content alike: agents that saw such a hint went to read the password themselves (agent review #2). A bare “Keychain” is not enough to withhold a line: a git credential helper error must reach the agent. benchbar_site_restoreoffers no--before-migratehook: a script path from an agent would run arbitrary code as the user inside a benchbar run; the hook stays a terminal flag.- MCP progress: with a
progressToken, an action runs with--jsonland eachsteporprogressline becomes onenotifications/progress(a counter and the step’s name); the result is still built from thedoneline. Without a token nothing changes. benchbar_site_addreturns a generated Administrator password only when its result sayscreated: true: on a site that existed the password was never used, and returning it would hand the person a login that does not work.service --apply TOKENrefuses--with-scheduleand--without-schedule: the scheduler choice is stored before the plan is made, so with a token it would be written even when the token turns out stale. A reviewed plan is applied as it is.- A streamed MCP call is complete when its
doneline has been read, not when stdout closes:up --services-onlystarts a Redis that may keep the pipe open, and waiting for the end of it would hold every such call to its timeout. - The shim’s
wsl_enableis the only elevated action, and the UAC prompt elevates<GetSystemDirectory>\wsl.exe --install --no-distributionitself, neverbenchbar.exe: the default install is per user and user writable, so same user malware could swap the exe before the prompt is accepted, while System32 is writable by administrators only. The window is hidden, so there is no output to relay; the result is the exit code (3010 or 1641 means restart, other non zero fails with the code) pluswsl.exe --statusafterwards (silent WSL means restart), a declined prompt (error 1223) iscancelled, and a named mutexLocal\BenchBarWslEnablestops a second elevation while one runs. If the caller is cancelled while wsl.exe still runs, the install cannot be undone, so the result is not a failure: it says the install keeps running in Windows, asks for a restart and writes the restart marker. setup checknever starts a stopped distro (running state comes fromwsl --list --running --quiet, names only, so it does not depend on the Windows language) and never elevates; the steps inside the distro read “not checked” andreadystays false until--probe.setup checkmarks each stepblocking.linger,keepalive,wslconfigandshim_pathare advisory: they arewarnat worst and never decideready, because a user who cannot fix them (anotherbenchbar.exeearlier on the machine PATH, a unreadable registry) would be stuck in the wizard; with no action the detail carries the exact fix command.report-windowsis a shim command, not a CLI one, because the report has to work where the CLI cannot run (no WSL, no distro, stopped distro). It reads the running state with--list --running --quietand never boots the distro, so a stopped distro simply has nocli-report.zipand the reason is listed underskipped; the CLI’s zip is written to/tmpin the distro and copied out withwsl.exe --exec cat, not through\\wsl.localhost, so the shim never reads files under the distro from Windows. Redaction runs on the text parts only (the CLI redacts its own zip), and the profile folder becomesC:\Users\<user>so the log stays readable.- A pending restart is the marker
setup\reboot-pending.jsonwritten afterwsl_enablesaid so (exit 3010 or 1641, the system text, or WSL still silent;wsl_enableanddistro_install) and counts until Windows has started after its time; the generic Windows Update and CBS flags are used nowhere, not by check and not bywsl_enable, since an unrelated update would block onboarding. - The distro is installed with
wsl --install -d NAME --no-launchand started once as root with--exec: Microsoft runs the first-run user prompt only in an interactive shell, so nothing asks a question;distro_userthen creates the user as root, feeds the password tochpasswdon stdin only, and sets[user] defaultin /etc/wsl.conf after a backup. systemdanddistro_userstop only the one distro withwsl.exe --terminate NAME; the shim never runs--shutdownor--unregister, andwsl_updatewarns that WSL may restart. When the distro already runs (asked first with--list --running --quiet, because the action’s own commands start it) and a stop is needed, they refuse before changing anything unless--forceis passed: exit 1 witherror_id: "distro_running", since a stop ends the person’s benches and MariaDB.- The forwarder adds the five password and sudo variables that are set to the child’s
WSLENVcopy and leaves the user’s ownWSLENVand every other variable alone. BENCHBAR_SUDO=env:sudois a shell function that validates withsudo -S(password on stdin from a printf builtin in a process substitution, never a pipeline: bash 3.2 forks a child inside a pipeline element, so the ticket would belong to the wrong parent) in the same shell that then runs the real sudo (by its path, not “command sudo”: bash 3.2 withset -eends the shell whencommand CMDfails even underifor||), because without a terminal sudo ties its ticket to the parent pid; the command keeps its own stdin, and no keepalive runs.BENCHBAR_SUDO_PASSWORDis taken out of the environment before anything else runs (sudo-env.sh). It reaches only phase 00, as one line on fd 8; phases 01 and 02 and every other child never get it, and install forgets it right after phase 00.- A wrong sudo password is checked up front with
sudo -kfirst, so a cached ticket cannot make it pass. Its id issudo_password_wrong, inerror_idof every done line. - Linux prerequisites read no root-only state: the sudo check uses group membership, not
sudo -n, so the check never runs sudo. - The Linux layer never calls
lsof: listeners and the owner’s working folder come fromfl_port_listener_pid,fl_port_listener_pidsandfl_port_owner_cwdinplatform-linux.sh(ss and /proc), so a Ubuntu without thelsofpackage behaves the same. tests/test-windows-commands.shruns the Windows tray’s command list underBENCHBAR_NONINTERACTIVE=1with a failinglsoffirst on PATH, so a later directlsofcall fails the suite.benchbarstarts itself again once whenBENCHBAR_SUDO_PASSWORDis set (same pid,exec): the kernel keeps the first image’s environment in/proc/PID/environfor the whole run, where the pip and npm scripts of the same user could read it. The second image gets the password on an inherited pipe (fd 9,FL_SUDO_PW_FD) and no variable. The variable being gone is the loop guard.- On Linux, repair runs its sudo actions (
mariadb_bind,mariadb_utf8,wkhtmltopdf_install) first in theFL_ACTION_ORDERofdoctor-linux.sh; macOS keeps its order, wherehosts_entrystays late and nothing is dropped, so the credential and the password can be dropped (fl_sudo_phase_done) before pip, npm and bench steps. They do not depend on the Python, Node or build actions. - The Linux
sudoprerequisite warns, not fails, outside the sudo and admin groups: a sudoers rule can still allow it, and the check never runs sudo to find out. benchbar reportwrites its zip with Python’szipfilewhenzipis missing (a distro with only the CLI, before the system dependencies).install.shon Linux is the CLI half of the macOS flow with the same paths (~/.local/share/benchbar,~/.local/bin, thebenchbar-pathblock), soinstall-kindsaysmanagedthere too. It links onlybenchbar(nofrappe-mac), skips Homebrew detection and the app, and never asks:BENCHBAR_NONINTERACTIVE=1makeshas_ttyfalse, and a found bench is only named, not adopted.- On Linux
--version vX.Y.Zpins the CLI, not an app: a fresh clone usesgit clone --branch, a rerunfetch --tagsandcheckout. This is the one case of a detached checkout the installer creates; the macOS rule that--versionpins the app only is unchanged, and a run without--versiongoes back tomainwith the detached HEAD logic that was already there. The shim passes--version v<its own version>for a release build and nothing for a dev build, so the CLI in the distro matchesbenchbar.exe. benchbar installpassesBENCHBAR_PLAN_STEP1(the phase 00 script) to phase 01 in a dry run only. With it, a tool that step 1 installs and is missing (uv, pipx, Python, Node, MariaDB, Redis) is a note “will install it”, not a warning or a die; without it (01 run alone) the old warning stays, because nothing installs those first.- The port blocker on Linux asks the distro’s own systemd unit (
mariadb,redis-server), notss -p: a normal user sees no owner for mysql’s process, and no owner at all for another WSL distro’s, whose processes live in a PID space of their own. A listener with the unit inactive is foreign; with it active it is ours (a rerun). A dry run with a blocker still prints the plan, withblockers[], and exits 1 like any other refusal; the done line carrieserror_id,errorandfix(FL_ERROR_FIX). open --aslands on/login?redirect-to=PATH&sid=SID&t=SECONDS, not onPATH?sid=asbench browsedoes: for a signed in session frappe’s login page answers 301 to PATH with the cookie, so the session id leaves the address bar. Browsers cache a 301, sot=makes every link new.open --asruns the steps of frappe’sLoginManager.post_loginexceptset_user_info. That one sets the response cookies (the browser’s own/login?sid=request sets them) and finds the default page by importing every app: on a bench with Insights it was six of the seven seconds of a login.on_loginhooks, IP and hour restrictions and the session run as in a real login.open --asneedsdeveloper_modefor Administrator too, wherebench browse --user Administratordoes not: one rule for every user, and a production copy brought in withbenchbar pullnever signs anyone in on its own.- The session id of
open --asis on the browser opener’s command line (open,xdg-open,wslviewtake the URL as an argument; there is no other way to hand it over) and nowhere else: no run log, no file, not in--jsonunless--print-url. There is no MCP tool for it: no tool hands a credential to an agent. site listandstatusreaddeveloper_modefrom the JSON files in bash, without frappe: status is polled, and clients need it to explain why Open as does not work;open --asasks frappe itself before it signs anyone in.open --asrefuses a site other than thedefault_siteofcommon_site_config.json:bench servewithout--siteserves that one site for every Host (bench_helper.get_sites), so a session made on another site is unknown where the browser lands. Plainopenstays as it was.- In WSL
open --asopens the browser itself only forlocalhostand*.localhostnames:wslviewopens a Windows browser, which resolves names through Windows’ hosts file, not the distro’s thatgetentreads.--print-urland--no-browserare not refused, since whoever opens the link may have the name in Windows’ hosts file. - The bash read of
developer_mode(one key per line, as frappe writessite_config.json) only decides what to offer; the Python check before the sign-in is the authority, so a minified file or a value like2can at worst refuse wrongly, never sign in wrongly. open --asandusersneed the person to turn Open as on per site (site open-as on NAME), kept asOPEN_AS_SITES(space separated names) in the bench’s state file: never undersites/, and a site thatpullorsite addcreates is never on by itself.site dropclears the name so a later site of that name does not inherit it. Plainopenneeds no opt-in.site open-as onis refused whenBENCHBAR_NONINTERACTIVE=1(so under--for-agentandbenchbar mcp), checked before the lock: password-less sign in is a person’s decision, and an agent that could turn it on could then pull a session id with--print-url.--yesdoes not stand in for the person:onalso needs a terminal on stdin, orBENCHBAR_SUDO=gui(the app, after its own alert), so an agent’s plain shell with--yesis refused too. This is a guard against a mistake, not a boundary: whatever can run the CLI can also set the variable or edit benchbar’s state. A byte of--pathabove 127 is masked before it is encoded: bash 3.2’sprintf "'c"sign extends it.offandstatusare always allowed.onis TTY yes (it asks,--yesskips) and MUTATING yes (it writes state, takes the lock and has a run log, which holds no session id).site listandstatusgive each siteopen_as_enabled(the opt-in) andopen_as(opt-in, developer mode and the sitebench serveanswers as), so a client offers Open as only where the CLI will sign in.openstays non MUTATING although--aswrites a session in the site’s database: a MUTATING command has a run log, and a log could capture the session id. The sign in changes no bench file.open --asreruns the steps of Frappe’sLoginManager.post_loginexceptset_user_info, so it depends on Frappe internals. Tested on v15; v16 is untested so far. The side effects are those of a real login:on_loginhooks run,last_loginis updated, and Frappe may end the user’s oldest session (simultaneous_sessions).open --help, the CLI docs andsite open-as onsay so.- What a sign in does is listed in full where a person turns Open as on (the prompt,
open --help, the docs, the app’s alert), not onlyon_login:make_sessionalso runson_session_creation, so Frappe writes an Activity Log entry per switch, can mail the System Managers for an Administrator sign in, and ERPNext can create a Customer or Supplier for a Website User. Reported in review by Sumanth Udupi. --pathrefuses a backslash: browsers read/\hostas//host, so/\evil.examplepassed the single slash check and would have sent the browser to another site after the sign in.
Test cost
- The mocks find
_mocklib.shwith${0%/*}and name themselves with${0##*/}, and run under#!/bin/bash, not/usr/bin/env bash: a mock call started env, bash, dirname and basename before it did anything, and now starts bash alone./bin/bashis the 3.2 the CLI targets on macOS and exists on the Linux runners. What a mock logs and prints is unchanged, and test-status-cost no longer needs its dirname exception. test-command-costkeeps a program budget fordoctor,doctor --json,install --dry-run,adopt --dry-runandapp list, counted with test-status-cost’s shims on two registered benches and one to adopt. Each budget is the larger of the macOS and Linux counts plus about 10 percent, so the test documents today’s cost and fails when a change makes a command start more programs; a change that makes one cheaper lowers its budget.- The cheap forks went from the paths these commands take:
$(basename X)and$(dirname X)on a path from a glob or an agent file list (it has a slash) became${X##*/}and${X%/*},fl_bench_canonicalstrips trailing slashes before it does the same for a folder not created yet, and exact line tests inprintf | grep -qbecame[[ ]]patterns.sed | head -n1pairs and the case insensitivegrep -qitests stay: changing them needs a decision on what each pattern should match.
CI speed, round 3
- Linux carries the suite: 8 Linux shards, 4 bash 3.2 shards and 2 macOS shards, not 4 and 4. Linux runners start faster, cost less and run more jobs at once than macOS runners. The slowest shard is what a pull request waits for, and more Linux shards shorten it.
- A job runs the suite under bash 3.2.57 (the
bash:3.2.57image) on Linux runners: the bash-ism risk, a 4.x feature that the macOS/bin/bashdoes not have, is a property of the interpreter, not of the OS, so a Linux runner covers it. The image is Alpine, where JavaScript actions such as checkout do not run in a job container, so the job checks out on the runner and runs the suite withdocker run. It adds the GNU tools the suite expects and a/bin/bashlink for the mocks, and runs as a user from a copy of the checkout. - The macOS shards stay for what only a Mac has: the BSD versions of sed, awk, date, stat and find, and the
/bin/bashusers really run. They skip the Linux platform tests, as before. - macOS stays at 4 shards, not 2: on the first run with 2 the whole run took 17.3 minutes instead of 14.1, because the slowest test files take 8 to 10 minutes each on a macOS runner and one shard held several of them (1,002 s against 552 s for the other). No shard count gets below the slowest single file; splitting those files is what would.
- The heaviest test files are split into parts that each build their own fixture and run side by side (test-app-recovery in 3, test-agent-contracts in 3, test-profiles in 3, test-site-restore by platform), the shared setup in tests/lib. A shard cannot finish before its longest file, and the old 250 to 320 s files set that floor. A part that needs state an earlier part builds repeats the few commands that make it, not the assertions.
SHARD=i/nfills the shards greedily from a time table in tests/run-tests.sh (TEST_SECS: heaviest test first onto the shard with the least work so far), not by taking every nth test of a sorted list. With uneven tests the every nth split gave 361 to 634 s for four macOS shards; the table gives shards within 1% of each other. The cost is that a new or changed test needs its time in the table (an unlisted one counts as 10 s, and the shards stay disjoint and complete either way).tests/affected.shpicks every part of a test (test-X-2, …) withtest-X.
Release branches
- Pull requests merge into
next-release;mainmoves only when a release is cut, so it always equals the last release, and so do the one line installers and the docs site, which read frommain. Work can land at any time without moving what users install. - A release is a squash merge of
next-releaseintomain: themainruleset requires a linear history, which rules out a merge commit, and one commit per release reads well inmain’s log. auto-tag.ymltagsvX.Y.Zwhen a push tomainbrings a version that has no tag yet (FL_VERSION and MARKETING_VERSION must agree, CHANGELOG.md must have the section) and dispatchesrelease.ymlandrelease-windows.ymlwith the tag as their ref: a tag pushed with GITHUB_TOKEN starts no workflow by itself. A push that keeps the version tags nothing.- After tagging it merges
mainback intonext-release. The squash commit’s changes are already onnext-release, so the merge is clean, and the next release pull request shows only what is new. A conflict (a fix made onmainalone) is left to a person, with a warning on the run. chk_fork_safetyreads each plist key and the line after it into a variable and tests it with[[ ]], notgrep -A1 | grep -q: underpipefailthat pipeline failed whengrep -qquit before the first grep finished writing, a false “Fork safety env” warning that failed adopt and repair at random (it showed up on the bash 3.2 shards). The otherproducer | grep -qchecks have the same exposure and are listed for a follow up.
Pipelines and locks
- No
producer | grep -q(also-qx,-qxF,-qi,-qiE) in the shell sources: underpipefailit fails when grep quits at the first match and the producer then dies of SIGPIPE (141). The output is captured first, with|| trueso a failing producer still reads as no match, and tested without a pipe:fl_has_line TEXT LINEfor a whole line (grep -qxF),[[ ]]orcasefor the rest.test-pipelinesfails on any that comes back. - The case insensitive tests (
wkhtmltopdf --version, the offline messages of a profile fetch) usecasewith both case patterns, or the text lowercased bytr, because bash 3.2 has no${var,,}and a here-string writes a temp file. - A key value lock is never stale on age alone: a lock whose token PID is alive is not broken by a 10 second old (or future) mtime, because the mtime lies when the clock steps and a live holder can be stopped for seconds on a loaded runner. The waiter broke it, took the next lock, and the holder then renamed its older copy of the file over the new one, losing a key. The last resort break after 600 rounds waits 6000 when the holder is alive, and a holder renames only while its token is still in the lock, else it locks again and rebuilds.
test-port-managementsourcessudo.shwhere it sourcesport-management.shalone, and asserts the output of the apply path has no “command not found”:fl_cmd_ports applyasksfl_sudo_gui, whichbenchbarloads fromsudo.sh, and the stray error was printed and ignored (a missing function reads as false).producer | head -n Nbecomes| awk 'NR==1'(orNR<=N) where the producer can write more than head reads (find, sort, ls, sed, git, a function printing a list): awk reads to the end, so nothing is left to die of SIGPIPE, and the call keeps one process. A first line of a captured variable is taken with${v%%$'\n'*}. The onehead -n 40over a database dump in pull.sh stays, because reading the whole dump is the cost it avoids and its status is ignored.auto-tag.ymlstartsrelease-windows.ymlwithwait_for_draft, so it waits up to 130 minutes forrelease.yml’s draft as a tag push does. Without it the first auto tagged release (v0.8.1) failed: the Windows packages finished before the Mac draft existed, and a dispatched run only attaches to a draft that is already there.- Share setup keeps the bench wide scheduler pause, with a marker. A pause of the copy alone is too late: frappe’s restore writes a fresh site_config.json, so jobs could run on the copy while the data goes in. The marker (
share/scheduler-pausedin the state folder) is written before the pause and removed when it is lifted; theshare_schedulerdoctor check (actionshare_resume) flags a marker whose pause is still there, and the nextshare setuporshare startresumes it first. A pause without the marker is the user’s and is never touched. lan-guard installkeeps the tokenpfctl -Eprints, and whether pf was already enabled (pfctl -s info) before the first install, inlan-guard/pf.statein the state folder, never the bench. The root script prints both on stdout and the user side writes the file, so root never writes into a folder the user owns.lan-guard removerunspfctl -X TOKENfor each recorded token, except when pf was already on: then pf stays on and nothing is released.- lan-guard stays a global, unconditional pf rule: the bench ports and 3306 are refused from the LAN address even with no room open, which also refuses Docker and VM traffic that dials the LAN IP. The rule is not tied to a room, so it holds before
share startand when a crash leaves no one to remove it. The cost is documented in the share reference (“What lan-guard blocks”);lan-guard removeis the way out, andallow-nodeis not one (it only changes the macOS application firewall). - The CPace session id is empty. The channel identifier CI carries the per room salt (the one the invite carries), the join request id sits in ADb, and a recorded exchange does not replay: every attempt draws fresh scalars and the confirmation tags bind the TLS exporter of the connection. A non empty sid would only repeat what the salt and the request id already bind.
- Ten wrong codes lock the room for everyone until
share code --new, not per guest: a per guest count would need a guest identity before the code is proven, which is what the code is for. The cost is a denial of service by anyone who can reach the gateway; the host sees the locked room, and a new code ends it. Documented in the share reference. - Labs: experiments live behind one global
LABSkey the CLI owns; the app and MCP only read it; an agent can never turn one on (labs onexits 2 underBENCHBAR_NONINTERACTIVE). One gate in the dispatcher hides and refuses the commands of a lab that is off, so a feature needs no check of its own. - The lan-rooms lab gates
shareandlan-guardin the dispatcher (the lab row names both), so no share code checks the lab.labs off lan-roomsrunsfl_lab_off_lan_rooms: the share stop path in a subshell with no prompt, and a warning (never sudo) namingbenchbar lan-guard removewhen the pf files are there. Theshare_schedulerdoctor check runs only while the lab is on orshare/exists in the state folder, so a Mac that never used sharing sees nothing of it. - The lab gate runs after the JSON stream starts and before the non interactive preflight, so
share start --jsonwith the lab off is a done line with exit 2, not the preflight’s “needs a person at a terminal” exit 1. - The key-value lock (
fl__kv_lock) does not trustmkdiralone: after its mkdir succeeds a writer puts its token in the folder and checks that it is the only one there, and backs off when it is not. On GitHub’s Linux runners two writers have both seenmkdirsucceed within milliseconds of the previous holder’srmdir, with both tokens ending up in one folder (test-run-lock, traced on 2026-11-04); one key was lost per collision. A token that cannot be written is no lock, and unlock without a token releases nothing. - The share gateway lets an app front end through only when the copy itself declares it (
add_to_apps_screenroutes, andwebsite_route_rulesof the single page app form/x/<path:app_path>served byx, read bysignin.py routesat share start; portal rules such as/ordersor/rfq/<path:name>stay out) and the route is a plain slug that is not reserved (app,api,assets,files,private,room,pair,consoleand the single-page web routes). Frappe still checks every page and API call; a helper that fails gives no extra routes, never a wider allowlist. copy_presentis a critical check (the job runs it every minute), not only part of the staticcopy_ready: a site drop or cleanup can remove the copy while sharing is on, and the web process keeps answering, soweb_upalone never ends the share.- A removal blocks the device only until the next code (
Room.rotateclears the blocked list), not for the whole room: the new code reaches a removed person only if the host sends it, so a mistaken removal is undone with a new invite instead of stopping the room for everyone.share remove --new-codeand an unknown intruder still end up with a code they do not have. - Shim: a CLI newer than
benchbar.exeis a warning with no action, becausecli_installwould take the CLI back to an older release. An older CLI makes theclistepmissingwith the samecli_installaction and acli_versionfield, so the tray can label it Update. - Shim: a Docker Desktop, Rancher Desktop or Podman distro is never picked automatically, default or not; a configured distro is always honoured. The tray uses the same rule (
DistroName.Pick). - Shim:
distro_osblocks every distro that is not Ubuntu 24.04, other Ubuntu releases included, because the Linux install supports 24.04 only. - Shim: the firmware virtualization flag only warns, because
IsProcessorFeaturePresentcan be false while a hypervisor runs; the hard failure comes from0x80370102at install or boot. - Shim: its own
wsl.execalls have a 60 s deadline (90 s when a distro may boot) and the user’s forwarded command never does; a timeout is never read as “WSL is not installed”, and the advice is to wait and retry beforewsl.exe --shutdown. - Shim: only
setup runlistens for Ctrl+C and console close, so forwarded commands keep the console’s own Ctrl+C behaviour. - Shim: Windows drive paths and same distro
\\wsl.localhostpaths in arguments become Linux paths, also after--, because--precedes bench paths and a drive path is never a valid Linux argument. - Shim:
config.jsonwrites take the named mutexLocal\BenchBarConfig(existence based, never waited on), and the tray takes the same one;adopt-distro NAMEreplaces an invalid file and keeps the old text asconfig.json.invalid. - Shim: the installer line carries
set -o pipefail, so a 404 from curl is the exit code; curl hints apply only when curl printed its owncurl: (N)line. - install.ps1 reads Smart App Control and refuses to install while it is on, because an unsigned program is blocked with no per app exception and an install that cannot start helps nobody.
- install.ps1 removes the Zone.Identifier mark from the installed files after the SHA256SUMS check, because the copy keeps the mark and the checksum is the trust anchor.
- install.ps1
-Uninstalledits the user PATH itself and does not callbenchbar.exe path remove, because an older installed exe would forward that to the distro. It removes a shortcut only when it is a.lnkin the Start menu, Startup or install folder whose target is the installed tray, and with no terminal it does nothing without-Yes. - install.ps1 takes the newest published release that has the Windows zip for this CPU, not
releases/latest, because Windows zips can be attached later than the release. - A non release Windows build gets the shim version
0.0.0-dev-SHA, so it never pins the distro CLI install to a tag that does not exist. - SignPath Foundation declined the signing application on 2026-11-13 (the project needs more public adoption first). The signing step stays in the workflow, dormant, for whichever certificate comes next;
SIGNING_REQUIRED=trueturns an unsigned first party file into a release failure. User docs say “not signed yet” and promise no date. .gitattributeskeeps shell scripts, templates andinstall.ps1at LF, becauseinstall.ps1is served raw from GitHub and works that way today.- Linux install preflight (
preflight-linux.sh): blockers stop before sudo, warnings only inform. github.com blocks; the apt mirror, pypi, npm, nodejs.org and astral.sh warn, because curl does not know apt’s own proxy or a file mirror. - A Windows listener is seen with
netstat.exe -ano(all TCP, language independent) and a 127.0.0.1 connect; a port that also listens in the distro is never a Windows holder (wslrelay). - A half built bench is moved aside by install only when
apps/holds nothing butfrappeand no site exists; anything else keeps the stop with the repair hint. - Lingering is enabled before phase 00, while the sudo credential still exists.
- An apt proxy travels in a 0600
APT_CONFIGfile, never on a command line, because proxy URLs can hold credentials. - The wkhtmltopdf arm64 row is trusted from the release asset (the pinned sha256 was computed from the download, the same way the amd64 one reproduces); not tested on real arm64 hardware.
- Shim:
keepalive removeleaves the task alone when it runs another existing copy ofbenchbar.exe, because the task has one fixed name and an uninstall of a second install must not remove the first one’s keepalive. - The Windows port check starts
netstat.exefrom/, because on some WSL builds a Windows program fails with “Invalid argument” when the working directory is a folder under the Linux home (seen on Windows build 26320 with WSL 3.0.5). - Doctor’s “is MariaDB running” test is whether 3306 listens, not whether the listener’s pid is visible:
sshides the owner of the aptmariadbd(usermysql) from a regular user (found by the first full install on a real WSL distro). - Windows self-update runs the one line installer instead of a downloader in Go, because the installer already verifies the release zip against its SHA256 sums, writes files without the internet mark (no SmartScreen dialog), moves files in use aside and restarts the tray.
- Windows self-update fetches main’s
install.ps1, not the release tag’s, and pins-Version: the installed copy and an old tag lack later installer fixes, and main’s installer can install any release, also an older one. - Windows self-update starts the installer as
[scriptblock]::Create((irm URL))with its parameters on the command line, so a Restricted execution policy does not stop it and no variable is set in the user’s shell; theinstall.ps1variables a shell may still hold (BENCHBAR_UNINSTALLand the rest) are removed from the child’s environment. self-update --checkfinds the latest release with one HTTPS request from Go (env proxy, then the Internet Options proxy), not by starting PowerShell, and mirrors install.ps1’s choice (published, not a prerelease, has the zip for this CPU); a proxy auto config script or a proxy that wants the Windows sign in is not supported there, and--version X.Y.Zstill works through such a proxy because only the installer downloads.self-update --checkasks the CLI in WSL only when the distro already runs, so a poll from the tray never boots WSL.- After a Windows update the CLI part runs through the new
benchbar.exe(setup run cli_install), not the old process, so the CLI is pinned to the release that was just installed. - After a Windows update, a keepalive task of this install is moved to the new program with the new
benchbar.exe keepalive install; the keepalive run check acceptsbenchbar.exe.old, the name the installer gives a running exe it replaced. - The Microsoft Store case of self-update is recognised by the exe path (
\WindowsApps\), because the package detection lives on the MSIX branch; switch to it when that merges. self-updatestarts Windows PowerShell 5.1 withoutPSModulePath(any case of the name): PowerShell 7 sets it to its own module folders, and 5.1 started with that value cannot load its own modules (Get-FileHashis not recognized); without it 5.1 builds its default. Every start of powershell.exe in the shim goes throughpowerShellEnv.- install.ps1 replaces the installed files all or nothing: phase 1 renames every file it replaces or removes to NAME.old (NAME.N.old when taken) inside the install folder, phase 2 copies the new files in; a failure in either phase deletes the new files and renames every backup back before anything else runs, and only a rollback that itself fails reports a mixed install, with the one line installer as the repair. A failed run tells
self-updatein the file named byBENCHBAR_INSTALL_RESULT(unchanged, mixed, changed or unknown, then the error line), so the shim keeps streaming the installer’s output to the console as it is.