How to use “grep -f file-with-patterns” in a pipe where “file-with-patterns” changes frequently?

Question

I am trying to do dynamic logging of dropped firewall events whereby I can dynamically specify what events not to log.

I use:

logread -f | grep -v -f file-with-patterns >> logfile

in a script running in the background.

This works fine, it logs everything except those events that I don’t want to be logged by specifying patterns in the file “file-with-patterns”.

However, it appears that when I update the file “file-with-patterns”, grep does not re-read it.

After a change to the file “file-with-patterns” I need to kill and restart the scripts starting all of this which is a bit cumbersome.

Can this be made to work without restarting the script? For instance crontab rereads the crontab file automatically when it changes. Is there some feature in grep (or an equivalent utility) that works like that?

Tried changing the file-with-patterns, grep did not change its filtering.

Answer 1

Simple method would be to load the config on every line, I guess performance would suffer a bit. Something along these lines:

logread -f | while read LINE; do echo "$LINE" | grep -v -f file-with-patterns; done

For ease of (re)use a small prof of concept operator program could be made from the first example:

sudo nano /usr/bin/¬

Remember to make it executable: sudo chmod +x /usr/bin/¬

#! /bin/bash

while read LINE; do echo "$LINE"; done | $@

Then you could run any command in place of grep like this:

logread -f |¬ grep -v -f file-with-patterns

Better trick would be to monitor file for changes and reload if any exist. One way to do this would be with the help of tool called inotifywait. Something along these lines:

while inotifywait -e modify file-with-patterns; do logread -f | grep -v -f file-with-patterns; done

Experiment a bit, if stuck reach out here in comments :)