# Eos log and logstash log format issues

**URL:** <https://eos-community.web.cern.ch/t/eos-log-and-logstash-log-format-issues/1110>\
**Category:** Site Administrators\
**Created:** [December 4, 2024, 11:37am UTC](https://eos-community.web.cern.ch/t/eos-log-and-logstash-log-format-issues/1110 "2024-12-04T11:37:22Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![evamvako](https://eos-community.web.cern.ch/letter_avatar_proxy/v4/letter/e/77aa72/32.png) [@evamvako](https://eos-community.web.cern.ch/u/evamvako)\
**Post date:** [December 4, 2024, 11:37am UTC](https://eos-community.web.cern.ch/t/eos-log-and-logstash-log-format-issues/1110/1 "2024-12-04T11:37:24Z")

</div>

Dear all  
eos xrdlog.fst| logs looks quite unstructured  
I have 15 group of log length ( by columns) for the 29 different func of FST  
… ( func=name # of colunms in log line)  
func=LogMsg 20  
func=Matches 20  
func=MgmSyncer 17  
func=MoveToOrphans 19  
func=ProcessCapOpaque 24  
func=ProcessReq 22  
func=Put 20  
func=ResyncFileFromQdb 32  
func=ResyncMgm 21  
func=RunDiskScan 32  
func=VerifyChecksum 25  
func=\_close 24  
…

do we have some idea how we could normalized this data ?

I do not think that I can match with efficient way more than 5 col for each log line  
241126 15:42:44 time=1732632164.520784 func=\_close level=INFO logid=b2289e30-ac04-11ef-a452-b0262874ab30

another issue is the timestamp if the 241126 15:42:44 is not the nomalized format  
and the epoch timestamp do not appeared in all logs

any idea or current solution for the eos fst ( and mgm) log parsing is welcome  
thank you in advance  
best  
e.v.

---

<div class="post-metadata">

**Author:** ![evamvako](https://eos-community.web.cern.ch/letter_avatar_proxy/v4/letter/e/77aa72/32.png) [@evamvako](https://eos-community.web.cern.ch/u/evamvako)\
**Post date:** [December 4, 2024, 3:50pm UTC](https://eos-community.web.cern.ch/t/eos-log-and-logstash-log-format-issues/1110/2 "2024-12-04T15:50:44Z")

</div>

match =\> { “message” =\> “%{YEAR:year}%{MONTHNUM:month}%{MONTHDAY:day} %{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second} \btime=%{NUMBER:xrdtime} \bfunc=%{WORD:func} ._?\blevel=%{WORD:level} ._?\blogid=%{UUID:logid} %{GREEDYDATA:msg}”

```
match => { "message" => "%{YEAR:year}%{MONTHNUM:month}%{MONTHDAY:day} %{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second} %{INT} FstOfs_(?<func>[a-zA-Z]+): %{GREEDYDATA:msg}"

```

I conclude to two matching pattern of grok/logstash for unstructured data one for func=\* event log and one for FstOfs\_\* ?  
at least I have those in loglevel=info

FYI  
e.v.
