Can't figure out what systemd service is failing.
2mon 18d ago by sh.itjust.works/u/Whooping_Seal in linux@sh.itjust.works
Hello everyone,
I recently set up a drop-in systemd unit file that adds an "on failure" command to all user-level services on a home server. Specifically the on failure command sends a notification to ntfy.sh detailing the service that has failed. E.g. if my miniflux container fails, it will send a notification such as Le service miniflux.service a échoué. Vérifier : journalctl --user -u miniflux.service
However, particularly on server startup, I have been getting some weird failure notifications for services I cannot find.
Any ideas on what I can do to troubleshoot this? An example is provided bellow.
Title:
⚠️ ❌ 303a3d11aa5781aaca14faa1e36fbe4465466f74e49821fee252d01227f8fdfb-73342cf5eacdadbd.service a échoué
Body:
Le service 303a3d11aa5781aaca14faa1e36fbe4465466f74e49821fee252d01227f8fdfb-73342cf5eacdadbd.service a échoué. Vérifier : journalctl --user -u 303a3d11aa5781aaca14faa1e36fbe4465466f74e49821fee252d01227f8fdfb-73342cf5eacdadbd.service
The naming convention looks suspiciously like an autogenerated (dynamic) unit; one typically sees these from /etc/fstab being parsed at boot by systemd-fstab-generator into /run/systemd/.
Try perusing https://www.freedesktop.org/software/systemd/man/latest/systemd.generator.htmland /run/systemd/ on your system, see if anything catches your eye...
OP mentionned containers, and Podman/Quadlet generates such services for health-checking a container service, so it might be that.
It's probably most effective at this point to add systemd.log_level=debug systemd.log_target=kmsg to the kernel boot options to ferret out what's creating it then go from there
I havent had the chance to reboot with the parameters mentioned below, but I think it most definitely is the health check (which I have manually configured for everything but syncthing).
Can't figure out what systemd service is failing.
over 2 million lines of slop coding, 2000 plus bugs open.
systemd is a fail