EOS logs and rotation

#1

Are there any plans to allow for EOS to receive a HUP or similar signal (USR1, etc.) to start a new log file or re-open the log file allowing for a new zero length file to be created and used? I ask because the way logging is implemented you are forced to use copytruncate with logrotate.

On a busy MGM, this causes the loss of possibly thousands of log entries. Even switching to hourly rotation, the copy and truncate can take several minutes, causing the logs written during this transition to be lost.

This is obviously not ideal. Is anyone handling this differently?

–
Dan Szkola FNAL

#2

We use Elastic’s filebeats to follow the file into a centralised log store, and we truncate the file on a regular basis. We also store our logs on NVMe as it makes it a far quicker operation. You do need the tail window set low on filebeats, but that’s rarely a problem even with high log rates on modern hardware.

I imagine quite a few other tools could follow the logs just fine in a similar manner.