Sander van Dragt's Notes

Run a weekly ClamAV scan that uses all your cores

Friday at 3:09 pm

A plain clamscan over a large home directory is single threaded. On a 24-core machine with four and a half million files, mine took over five hours and did not finish. This guide replaces it with the ClamAV daemon driven by parallel clients, and wires up a timer and an alert that tells the truth.

You need a Linux system with systemd and root access. The examples assume Debian or Ubuntu package names and a home directory at /home.

Install the daemon

The standalone clamscan binary reloads the whole signature database on every start and scans with one thread. The daemon keeps the signatures resident and scans with many:

sudo apt install clamav-daemon
sudo systemctl enable --now clamav-daemon

Wait for it to finish loading. The first start reads about a gigabyte of signatures and takes a minute or so.

Match the thread count to your machine

Check what the daemon is configured for and what you have:

grep MaxThreads /etc/clamav/clamd.conf
nproc

The default is MaxThreads 12. Raise it if you want more of the machine, and keep the number to hand. You use the same value for the client count in the next step.

Write the scan unit

Create /etc/systemd/system/clamscan-weekly.service:

[Unit]
Description=Weekly ClamAV scan of /home
OnFailure=clamscan-notify.service
Requires=clamav-daemon.service
After=clamav-daemon.service

[Service]
Type=oneshot
Nice=19
IOSchedulingClass=idle
# clamd needs a minute to load signatures before its socket answers
ExecStartPre=/usr/bin/clamdscan --ping 90:2
# --multiscan only parallelises directory args, so shard the file list across 12 clients
ExecStart=/bin/sh -c 'find /home -regextype posix-extended -regex "^/home/[^/]+/([.]cache|[.]local/share/Trash|[.]var/app/.*/cache)" -prune -o -type f -print0 > /run/clamscan-weekly.list; \
  xargs -0 -a /run/clamscan-weekly.list -P 12 -n 1000 /usr/bin/clamdscan --fdpass -i --no-summary > /var/log/clamscan-weekly.log 2>&1; ! grep -q " FOUND$$" /var/log/clamscan-weekly.log'

Four parts of that need explaining.

Why find builds a list. clamdscan has no --exclude-dir option, so the exclusions move into a find expression that prunes them and writes every remaining file to a list. Enumerating four and a half million paths takes a couple of minutes, and the list is a few hundred megabytes on tmpfs.

Why xargs and not --multiscan. The --multiscan flag maps to clamd's MULTISCAN command, which parallelises a directory walk the daemon performs itself. With --fdpass and a file list, the client opens each file and passes one descriptor at a time, so there is nothing for the daemon to parallelise. Sharding the list across -P 12 clients is what actually uses the cores.

Why --fdpass. The daemon runs as the clamav user and cannot read most of your home directory. This flag makes the client open the file and hand the descriptor over the socket, so the client's permissions apply.

Why the exit check greps the log. A live home directory produces transient read errors on its own, because files get rotated away between find listing them and the client opening them. Those are not detections, and failing the unit on any non-zero exit treats them as if they were. The grep fails the unit only when the log contains an actual FOUND line. Note the doubled $$, which is how you escape a literal $ in a systemd unit.

Stop the daemon between runs

The resident signatures cost about a gigabyte. To release it, do not have the scan unit stop the daemon in ExecStopPost. Calling systemctl from a unit's own lifecycle hook queues a job that waits on the unit trying to finish, and the unit hangs in deactivating (stop-post) indefinitely.

Let the daemon stop itself instead. Create /etc/systemd/system/clamav-daemon.service.d/stop-when-unneeded.conf:

[Unit]
# clamd is only Required by clamscan-weekly; stop it when that finishes
StopWhenUnneeded=yes

Then stop it from starting at boot, so it only runs when the scan pulls it in:

sudo systemctl disable --now clamav-daemon.socket
sudo systemctl disable clamav-daemon.service

freshclam keeps updating the signatures on disk regardless, and the daemon loads the current set each time it starts, so you lose no currency.

Make the alert report what happened

OnFailure= fires whenever the scan unit fails, for any reason. An alert that asserts a detection on every failure cries malware every time you abort a run. Count the detections first. Create /etc/systemd/system/clamscan-notify.service:

[Unit]
Description=Notify on ClamAV findings
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'n=$(grep -c " FOUND$" /var/log/clamscan-weekly.log 2>/dev/null); \
  if [ "$n" -gt 0 ]; then body="$n infected file(s) found. See /var/log/clamscan-weekly.log"; \
  else body="Scan did not complete (no detections). See journalctl -u clamscan-weekly"; fi; \
  exec /usr/bin/sudo -u YOUR_USER DISPLAY=:0 DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/YOUR_UID/bus /usr/bin/notify-send -u critical "ClamAV" "$body"'

Replace YOUR_USER and YOUR_UID with your own. The unit runs as root, so it needs both to reach your desktop session's message bus.

Schedule it

Create /etc/systemd/system/clamscan-weekly.timer:

[Unit]
Description=Weekly ClamAV scan
[Timer]
OnCalendar=Sun 03:00
Persistent=true
[Install]
WantedBy=timers.target

Persistent=true runs a missed scan after the machine comes back up. Then load everything and enable the timer:

sudo systemctl daemon-reload
sudo systemctl enable --now clamscan-weekly.timer

Check that it works

Start a run by hand and watch the daemon's CPU:

sudo systemctl start --no-block clamscan-weekly.service
top -bn1 -p $(pgrep -x clamd) | tail -1

For the first couple of minutes the unit sits in activating while find enumerates, and the daemon shows no CPU. Once the clients start, pgrep -c clamdscan returns your thread count and the daemon runs at several hundred percent CPU. Mine reaches around 1050%.

Read the results from the log:

grep ' FOUND$' /var/log/clamscan-weekly.log   # detections
grep '^ERROR' /var/log/clamscan-weekly.log    # unreadable files

Because of --no-summary, a clean run leaves the log empty and the unit goes inactive with exit 0. An empty log is the good outcome.

What to exclude, and what not to

The exclusions above cover caches, Trash and Flatpak caches. Those are regenerable or already quarantined, so skipping them costs you nothing.

Resist the temptation to exclude node_modules, vendor and language toolchains, even though they dominate the file count and excluding them would roughly halve the runtime. Compromised dependencies are the realistic threat to a developer's machine, so those trees are the haystack worth searching. Use the cores instead.

Expect the occasional false positive in dependency trees, on minified bundles and on test fixtures that contain sample payloads deliberately. Check the signature name against the path before you delete anything.