Pushing (volatile journald) system logs to New Relic

August 2023 · Last updated 2023-08-01 · 278 words

I set up New Relic Infrastructure Monitoring for my home router.

To minimize writes to the SSD and hopefully prolong its life, I configured the machine to ship all systemd/journald entries to New Relic instead of saving them to disk. (New Relic offers a generous free tier, just like their competition Grafana Cloud.)

Due to a bit scarce documentation that was harder than it should have been, so here’s what I did - hope this helps someone. The system runs Debian stable, and I put the New Relic Infrastructure Agent’s db directory (/var/db/newrelic-infra) on a tmpfs so it doesn’t write to disk either (see here for an example of how to set up a persistent ram disk).

Next, configure journald to not persist logs to disk (From the journald manual page):



Configure the New Relic Infrastructure Agent to set up and start an instance of Fluent Bit (see its documentation).


  - name: external-fluentbit-config-and-parsers-file
      config_file: /etc/fluent-bit/fluent-bit.conf
      parsers_file: /etc/fluent-bit/parsers.conf

And finally, configure Fluent Bit to forward all journal entries:


    Name  systemd
    Path  /run/log/journal
    Tag   host.*

When restarting the New Relic Infrastructure Agent (sudo systemctl restart newrelic-infra), it should be spawning a Fluent Bit process:

sudo ps axf says:

1743164 ?        Ssl    0:00 /usr/bin/newrelic-infra-service
1743168 ?        Sl     1:35  \_ /usr/bin/newrelic-infra
1743204 ?        Sl     0:23      \_ /opt/fluent-bit/bin/fluent-bit -c /tmp/fb/nr_fb_config2386533890 -e /var/db/newrelic-infra/newrelic-integrations/logging/out_newrelic.so -R /var/db/newrelic-infra/newrelic-integrations/logging/parsers.conf -R /etc/fluent-bit/parsers.conf

From the fluent-bit command line above, /tmp/fb/nr_fb_config2386533890 holds the amended configuration to ship our logs to New Relic. If you’re like me, you might want to have a look at that file, just to see what it would have taken to make it work ;)


Screenshot of logs in the New Relic web interface


Greetings! You are reading the personal web page of Florian Sesser.

I always like to hear from people. Please have a look at the imprint for ways to get in touch.