eos experts, perhaps you can assist us in determining the root cause possible solutions for this new install issue.
We installed two new eos-server-5.1.23 FSTs with identical hardware. One has been operating fine. However, since install the other has had an issue that when the eos@fst service is started /var/log/eos/fst/xrdlog.fst logs msg=“stat.statfs.freebytes has not yet been defined, not setting file system fill status”
If eos@fst is left running on this FST it will cause the mgm to become quite unresponsive, and at times has cause the eos@mgm service on the mgm node to fail, though nothing was logged (outside the prior func=GetStatfs messages)
The reason why the systfs call against the fsids is failing is curious, given EOS runs as user daemon and that user can statfs() the mounted fsids when tested for example with a simple C call to statfs(“/fsidmnt/11000/”, &buf); which returns:
[root@eos-fst-11 ~]# su -c ‘./statfs.test’ daemon
File system size: 17998070939648
Free space: 17798225084416
Similarly with python os.statvfs(f’/fsidmnt/{i}') on the fsids, this also succeeds.
Both FSTs are deployed with ansible with configs identical.
The hardware, OS, eos versions, etc. are also identical.
Any suggestions on what might be causing eos to statfs() calls to fail?
We could nuke it and reinstall fresh but wanted to inquire before such.
Thank you,
Pete