Want me to apply the fix and commit it?
❯ why wouldn't i?
● No reason — I was asking out of habit rather than because there was a real decision left. Applying it.
I'm just checking you're just checking.
Sander van Dragt's Notes
-
-
Been working on a listing post type so you can sell your stuff via your website. When I mean sell I mean advertise as that’s literally all it does, with an index, with search engines collecting all the products.
Also building in rsvp replies, so I can reply to an indieweb event on zoom held in London on the 23rd. Might go or not.
The above also led to not having built a contact form in the last x years, I guess by the independent web spirits I should build in a /contact page with a form. One that sends a verification test email once a month to ensure queries still arrive and some way to stop spammers, I’ve used third party forms before but nowadays do you trust third parties with your visitors communication? I was thinking some honeypot form-fields and a single use token to deter spammers, and requesting a sender email verification step. #lamb
-
Shortcutting to output using AI gives me a similar feeling as using cheat codes when gaming , short term reward but skipping the interesting journey.
-
viv 0.14.0 is out. It installs PHP dependencies from a composer.lock and writes the same vendor/ Composer would, byte for byte – 20 of 20 pinned projects identical in this release's sweep.
This is the last release where that is the whole point. From here viv is a research vehicle for package manager design, one measured chapter at a time, with the compatible mode frozen as the control. Looking forward to learn a whole bunch! #projects #vivace
-
Run a weekly ClamAV scan that uses all your cores
A plain
clamscanover a large home directory is single threaded. On a 24-core machine with four and a half million files, mine took over five hours and did not finish. This guide replaces it with the ClamAV daemon driven by parallel clients, and wires up a timer and an alert that tells the truth.You need a Linux system with systemd and root access. The examples assume Debian or Ubuntu package names and a home directory at
/home.Install the daemon
The standalone
clamscanbinary reloads the whole signature database on every start and scans with one thread. The daemon keeps the signatures resident and scans with many:sudo apt install clamav-daemon sudo systemctl enable --now clamav-daemonWait for it to finish loading. The first start reads about a gigabyte of signatures and takes a minute or so.
Match the thread count to your machine
Check what the daemon is configured for and what you have:
grep MaxThreads /etc/clamav/clamd.conf nprocThe default is
MaxThreads 12. Raise it if you want more of the machine, and keep the number to hand. You use the same value for the client count in the next step.Write the scan unit
Create
/etc/systemd/system/clamscan-weekly.service:[Unit] Description=Weekly ClamAV scan of /home OnFailure=clamscan-notify.service Requires=clamav-daemon.service After=clamav-daemon.service [Service] Type=oneshot Nice=19 IOSchedulingClass=idle # clamd needs a minute to load signatures before its socket answers ExecStartPre=/usr/bin/clamdscan --ping 90:2 # --multiscan only parallelises directory args, so shard the file list across 12 clients ExecStart=/bin/sh -c 'find /home -regextype posix-extended -regex "^/home/[^/]+/([.]cache|[.]local/share/Trash|[.]var/app/.*/cache)" -prune -o -type f -print0 > /run/clamscan-weekly.list; \ xargs -0 -a /run/clamscan-weekly.list -P 12 -n 1000 /usr/bin/clamdscan --fdpass -i --no-summary > /var/log/clamscan-weekly.log 2>&1; ! grep -q " FOUND$$" /var/log/clamscan-weekly.log'Four parts of that need explaining.
Why
findbuilds a list.clamdscanhas no--exclude-diroption, so the exclusions move into afindexpression that prunes them and writes every remaining file to a list. Enumerating four and a half million paths takes a couple of minutes, and the list is a few hundred megabytes on tmpfs.Why
xargsand not--multiscan. The--multiscanflag maps to clamd'sMULTISCANcommand, which parallelises a directory walk the daemon performs itself. With--fdpassand a file list, the client opens each file and passes one descriptor at a time, so there is nothing for the daemon to parallelise. Sharding the list across-P 12clients is what actually uses the cores.Why
--fdpass. The daemon runs as theclamavuser and cannot read most of your home directory. This flag makes the client open the file and hand the descriptor over the socket, so the client's permissions apply.Why the exit check greps the log. A live home directory produces transient read errors on its own, because files get rotated away between
findlisting them and the client opening them. Those are not detections, and failing the unit on any non-zero exit treats them as if they were. Thegrepfails the unit only when the log contains an actualFOUNDline. Note the doubled$$, which is how you escape a literal$in a systemd unit.Stop the daemon between runs
The resident signatures cost about a gigabyte. To release it, do not have the scan unit stop the daemon in
ExecStopPost. Callingsystemctlfrom a unit's own lifecycle hook queues a job that waits on the unit trying to finish, and the unit hangs indeactivating (stop-post)indefinitely.Let the daemon stop itself instead. Create
/etc/systemd/system/clamav-daemon.service.d/stop-when-unneeded.conf:[Unit] # clamd is only Required by clamscan-weekly; stop it when that finishes StopWhenUnneeded=yesThen stop it from starting at boot, so it only runs when the scan pulls it in:
sudo systemctl disable --now clamav-daemon.socket sudo systemctl disable clamav-daemon.servicefreshclamkeeps updating the signatures on disk regardless, and the daemon loads the current set each time it starts, so you lose no currency.Make the alert report what happened
OnFailure=fires whenever the scan unit fails, for any reason. An alert that asserts a detection on every failure cries malware every time you abort a run. Count the detections first. Create/etc/systemd/system/clamscan-notify.service:[Unit] Description=Notify on ClamAV findings [Service] Type=oneshot ExecStart=/bin/sh -c 'n=$(grep -c " FOUND$" /var/log/clamscan-weekly.log 2>/dev/null); \ if [ "$n" -gt 0 ]; then body="$n infected file(s) found. See /var/log/clamscan-weekly.log"; \ else body="Scan did not complete (no detections). See journalctl -u clamscan-weekly"; fi; \ exec /usr/bin/sudo -u YOUR_USER DISPLAY=:0 DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/YOUR_UID/bus /usr/bin/notify-send -u critical "ClamAV" "$body"'Replace
YOUR_USERandYOUR_UIDwith your own. The unit runs as root, so it needs both to reach your desktop session's message bus.Schedule it
Create
/etc/systemd/system/clamscan-weekly.timer:[Unit] Description=Weekly ClamAV scan [Timer] OnCalendar=Sun 03:00 Persistent=true [Install] WantedBy=timers.targetPersistent=trueruns a missed scan after the machine comes back up. Then load everything and enable the timer:sudo systemctl daemon-reload sudo systemctl enable --now clamscan-weekly.timerCheck that it works
Start a run by hand and watch the daemon's CPU:
sudo systemctl start --no-block clamscan-weekly.service top -bn1 -p $(pgrep -x clamd) | tail -1For the first couple of minutes the unit sits in
activatingwhilefindenumerates, and the daemon shows no CPU. Once the clients start,pgrep -c clamdscanreturns your thread count and the daemon runs at several hundred percent CPU. Mine reaches around 1050%.Read the results from the log:
grep ' FOUND$' /var/log/clamscan-weekly.log # detections grep '^ERROR' /var/log/clamscan-weekly.log # unreadable filesBecause of
--no-summary, a clean run leaves the log empty and the unit goesinactivewith exit 0. An empty log is the good outcome.What to exclude, and what not to
The exclusions above cover caches, Trash and Flatpak caches. Those are regenerable or already quarantined, so skipping them costs you nothing.
Resist the temptation to exclude
node_modules,vendorand language toolchains, even though they dominate the file count and excluding them would roughly halve the runtime. Compromised dependencies are the realistic threat to a developer's machine, so those trees are the haystack worth searching. Use the cores instead.Expect the occasional false positive in dependency trees, on minified bundles and on test fixtures that contain sample payloads deliberately. Check the signature name against the path before you delete anything.
-
It began as a question, whether a person directing coding agents can build a faster drop-in Composer, and that question is answered. The compatible mode is finished and frozen as a control. viv continues as a research vehicle for package-manager design, one measured chapter at a time; Now exploring "Beyond composer" https://github.com/svandragt/vivace/blob/main/docs/research.md #projects #vivace
-
PHP 8.6 beta 3: Good to know #lamb is already compatible. 2342 tests, 4332 assertions, identical to an 8.4 control in the same image. Zero deprecations from src/!
-
This is posted via micropub with the latest release from micro.blog iOS app.
-
Vivace got a manual now! Good software has good documentation! #projects #vivace
-
park v1.1.0: syncing between machines
Park support sync now, so you can totally use it as a private decentralised issue tracker.
It keeps parked work context in a SQLite database, and until now the way to use it on two machines was to point
PARK_DBat a synced folder, this can cause corruption: SQLite writes a database as several files that have to stay in step, a syncer copies them one at a time, and eventually you get a half-copied set that will not open at all.So park no longer shares the database. Each machine appends its changes to its own
<hostname>.jsonlin the shared folder, reads every log it finds, and folds them into a local database that you can delete and rebuild whenever you like. No file has two writers, so there is nothing for the syncer to get wrong.Two environment variables and a one-time seed:
export PARK_DB="$HOME/.local/share/park/park.db" export PARK_SYNC_DIR="$HOME/sync/park" park sync-seed --i-understand-this-runs-oncethen
park rebuild --yeson your other machines.Also in this one:
park servereads a snapshot rather than holding the database open, search indexing stopped rebuilding the whole index on every write, andpark versionexists.