Shortcutting to output using AI gives me a similar feeling as using cheat codes when gaming , short term reward but skipping the interesting journey.
Sander van Dragt's Notes
-
-
viv 0.14.0 is out. It installs PHP dependencies from a composer.lock and writes the same vendor/ Composer would, byte for byte – 20 of 20 pinned projects identical in this release's sweep.
This is the last release where that is the whole point. From here viv is a research vehicle for package manager design, one measured chapter at a time, with the compatible mode frozen as the control. Looking forward to learn a whole bunch! #projects #vivace
-
Run a weekly ClamAV scan that uses all your cores
A plain
clamscanover a large home directory is single threaded. On a 24-core machine with four and a half million files, mine took over five hours and did not finish. This guide replaces it with the ClamAV daemon driven by parallel clients, and wires up a timer and an alert that tells the truth.You need a Linux system with systemd and root access. The examples assume Debian or Ubuntu package names and a home directory at
/home.Install the daemon
The standalone
clamscanbinary reloads the whole signature database on every start and scans with one thread. The daemon keeps the signatures resident and scans with many:sudo apt install clamav-daemon sudo systemctl enable --now clamav-daemonWait for it to finish loading. The first start reads about a gigabyte of signatures and takes a minute or so.
Match the thread count to your machine
Check what the daemon is configured for and what you have:
grep MaxThreads /etc/clamav/clamd.conf nprocThe default is
MaxThreads 12. Raise it if you want more of the machine, and keep the number to hand. You use the same value for the client count in the next step.Write the scan unit
Create
/etc/systemd/system/clamscan-weekly.service:[Unit] Description=Weekly ClamAV scan of /home OnFailure=clamscan-notify.service Requires=clamav-daemon.service After=clamav-daemon.service [Service] Type=oneshot Nice=19 IOSchedulingClass=idle # clamd needs a minute to load signatures before its socket answers ExecStartPre=/usr/bin/clamdscan --ping 90:2 # --multiscan only parallelises directory args, so shard the file list across 12 clients ExecStart=/bin/sh -c 'find /home -regextype posix-extended -regex "^/home/[^/]+/([.]cache|[.]local/share/Trash|[.]var/app/.*/cache)" -prune -o -type f -print0 > /run/clamscan-weekly.list; \ xargs -0 -a /run/clamscan-weekly.list -P 12 -n 1000 /usr/bin/clamdscan --fdpass -i --no-summary > /var/log/clamscan-weekly.log 2>&1; ! grep -q " FOUND$$" /var/log/clamscan-weekly.log'Four parts of that need explaining.
Why
findbuilds a list.clamdscanhas no--exclude-diroption, so the exclusions move into afindexpression that prunes them and writes every remaining file to a list. Enumerating four and a half million paths takes a couple of minutes, and the list is a few hundred megabytes on tmpfs.Why
xargsand not--multiscan. The--multiscanflag maps to clamd'sMULTISCANcommand, which parallelises a directory walk the daemon performs itself. With--fdpassand a file list, the client opens each file and passes one descriptor at a time, so there is nothing for the daemon to parallelise. Sharding the list across-P 12clients is what actually uses the cores.Why
--fdpass. The daemon runs as theclamavuser and cannot read most of your home directory. This flag makes the client open the file and hand the descriptor over the socket, so the client's permissions apply.Why the exit check greps the log. A live home directory produces transient read errors on its own, because files get rotated away between
findlisting them and the client opening them. Those are not detections, and failing the unit on any non-zero exit treats them as if they were. Thegrepfails the unit only when the log contains an actualFOUNDline. Note the doubled$$, which is how you escape a literal$in a systemd unit.Stop the daemon between runs
The resident signatures cost about a gigabyte. To release it, do not have the scan unit stop the daemon in
ExecStopPost. Callingsystemctlfrom a unit's own lifecycle hook queues a job that waits on the unit trying to finish, and the unit hangs indeactivating (stop-post)indefinitely.Let the daemon stop itself instead. Create
/etc/systemd/system/clamav-daemon.service.d/stop-when-unneeded.conf:[Unit] # clamd is only Required by clamscan-weekly; stop it when that finishes StopWhenUnneeded=yesThen stop it from starting at boot, so it only runs when the scan pulls it in:
sudo systemctl disable --now clamav-daemon.socket sudo systemctl disable clamav-daemon.servicefreshclamkeeps updating the signatures on disk regardless, and the daemon loads the current set each time it starts, so you lose no currency.Make the alert report what happened
OnFailure=fires whenever the scan unit fails, for any reason. An alert that asserts a detection on every failure cries malware every time you abort a run. Count the detections first. Create/etc/systemd/system/clamscan-notify.service:[Unit] Description=Notify on ClamAV findings [Service] Type=oneshot ExecStart=/bin/sh -c 'n=$(grep -c " FOUND$" /var/log/clamscan-weekly.log 2>/dev/null); \ if [ "$n" -gt 0 ]; then body="$n infected file(s) found. See /var/log/clamscan-weekly.log"; \ else body="Scan did not complete (no detections). See journalctl -u clamscan-weekly"; fi; \ exec /usr/bin/sudo -u YOUR_USER DISPLAY=:0 DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/YOUR_UID/bus /usr/bin/notify-send -u critical "ClamAV" "$body"'Replace
YOUR_USERandYOUR_UIDwith your own. The unit runs as root, so it needs both to reach your desktop session's message bus.Schedule it
Create
/etc/systemd/system/clamscan-weekly.timer:[Unit] Description=Weekly ClamAV scan [Timer] OnCalendar=Sun 03:00 Persistent=true [Install] WantedBy=timers.targetPersistent=trueruns a missed scan after the machine comes back up. Then load everything and enable the timer:sudo systemctl daemon-reload sudo systemctl enable --now clamscan-weekly.timerCheck that it works
Start a run by hand and watch the daemon's CPU:
sudo systemctl start --no-block clamscan-weekly.service top -bn1 -p $(pgrep -x clamd) | tail -1For the first couple of minutes the unit sits in
activatingwhilefindenumerates, and the daemon shows no CPU. Once the clients start,pgrep -c clamdscanreturns your thread count and the daemon runs at several hundred percent CPU. Mine reaches around 1050%.Read the results from the log:
grep ' FOUND$' /var/log/clamscan-weekly.log # detections grep '^ERROR' /var/log/clamscan-weekly.log # unreadable filesBecause of
--no-summary, a clean run leaves the log empty and the unit goesinactivewith exit 0. An empty log is the good outcome.What to exclude, and what not to
The exclusions above cover caches, Trash and Flatpak caches. Those are regenerable or already quarantined, so skipping them costs you nothing.
Resist the temptation to exclude
node_modules,vendorand language toolchains, even though they dominate the file count and excluding them would roughly halve the runtime. Compromised dependencies are the realistic threat to a developer's machine, so those trees are the haystack worth searching. Use the cores instead.Expect the occasional false positive in dependency trees, on minified bundles and on test fixtures that contain sample payloads deliberately. Check the signature name against the path before you delete anything.
-
It began as a question, whether a person directing coding agents can build a faster drop-in Composer, and that question is answered. The compatible mode is finished and frozen as a control. viv continues as a research vehicle for package-manager design, one measured chapter at a time; Now exploring "Beyond composer" https://github.com/svandragt/vivace/blob/main/docs/research.md #projects #vivace
-
PHP 8.6 beta 3: Good to know #lamb is already compatible. 2342 tests, 4332 assertions, identical to an 8.4 control in the same image. Zero deprecations from src/!
-
This is posted via micropub with the latest release from micro.blog iOS app.
-
Vivace got a manual now! Good software has good documentation! #projects #vivace
-
park v1.1.0: syncing between machines
Park support sync now, so you can totally use it as a private decentralised issue tracker.
It keeps parked work context in a SQLite database, and until now the way to use it on two machines was to point
PARK_DBat a synced folder, this can cause corruption: SQLite writes a database as several files that have to stay in step, a syncer copies them one at a time, and eventually you get a half-copied set that will not open at all.So park no longer shares the database. Each machine appends its changes to its own
<hostname>.jsonlin the shared folder, reads every log it finds, and folds them into a local database that you can delete and rebuild whenever you like. No file has two writers, so there is nothing for the syncer to get wrong.Two environment variables and a one-time seed:
export PARK_DB="$HOME/.local/share/park/park.db" export PARK_SYNC_DIR="$HOME/sync/park" park sync-seed --i-understand-this-runs-oncethen
park rebuild --yeson your other machines.Also in this one:
park servereads a snapshot rather than holding the database open, search indexing stopped rebuilding the whole index on every write, andpark versionexists. -
viv 0.11.0, and a two-word deletion
Vivace 0.11.0 is out. This is the first proper release!
Anyone putting viv in a CI step or a Dockerfile no longer has to pin a version and update it by hand, or reimplement "latest" themselves.
Viv is at least twice as fast as any alternative I'm aware of: most of what the milestone tracked had already shipped in 0.10.0, and the one user-visible change is constraint matching, which went from about 27ms across 33,000 calls to 4 or 5. The bigger numbers in the README moved because the table was two releases stale.
viv installs PHP dependencies from a
composer.lockand writes the samevendor/Composer would, byte for byte – the compatibility sweep for this release was 44 rows identical, 0 differing, and all ten pinned projects resolving the same lock. It's around 6x faster on a cold install and 19x warm than composer.Scenario viv vs Composer viv vs riff Cold 6.1× (2.7 to 10.9) 2.0× (0.9 to 127.2)[^5][^6] Warm 19.4× (5.8 to 104.0) 5.5× (2.1 to 147.4)[^6] No-op 43.3× (19.3 to 109.6) 12.6× (2.5 to 52.1) Update-warm 1.8× (1.4 to 4.2) n/a What I want now is more real projects using it, because the useful bug reports have all come from someone trying to migrate something real and hitting a wall. The last release's best change came from a person who unpacked the tarball by accident and found a drop-in
composershim nobody had documented. So: if you try it and something gets in the way, I'd like to hear about it. -
The iPhone Duo seems interesting to me but mainly because of the “laptop mode” and being able to read content like Safari in landscape without needing an iPad. Still it’s iOS and so many little things are buggy and need a laptop.