Another key seems missing in our set of FST and the logs say that
key=stat.sys.xrootd.version uses an empty value!
The list of installed package related to xrootd is following:
sh-5.1# dnf list installed \*xrootd\*
eos-xrootd.x86_64 5.6.6-2.el9 @storage-ci.web.cern.ch_storage-ci_eos_diopside-depend_el-9_x86_64_
xrootd-client.x86_64 1:5.6.4-1.el9 @epel
xrootd-client-libs.x86_64 1:5.6.4-1.el9 @epel
xrootd-libs.x86_64 1:5.6.4-1.el9 @epel
xrootd-server-libs.x86_64 1:5.6.4-1.el9 @epel
Would you kindly let me know which essential pkgs are missing?
And while we are looking at FST logs, sometimes it complains that
240131 02:18:54 023 FstOfs_fsctl: root.2364:56@jbod-mgmt-10 Unable to execute fsctl function ; operation not permitted
It will be helpful to know the root cause of this.
There is nothing wrong with your deployment when it comes to the xrootd version, it was just a bug on your side that will be fixed in the next release as it’s already addressed by this commit:
For the log line related to
fsctl function this is normally the result of something trying to do an XRootD query againt the FST daemon. We only support a limited set of query commands that the FST can reply to. For example this query would trigger the same log signature that you have:
sudo xrdfs root://localhost:2001 query space /eos
[ERROR] Server responded with an error:  Unable to execute fsctl function ; operation not permitted
This leaves the following sigature in the logs:
240131 09:09:02 24285 XrootdXeq: root.27136:35@localhost pvt IP64 login as root
240131 09:09:02 24285 FstOfs_fsctl: root.27136:35@localhost Unable to execute fsctl function ; operation not permitted
240131 09:09:02 24285 XrootdXeq: root.27136:35@localhost disc 0:00:00
I have pushed the following commit to actually display the command and the arguments passed to this call so we can more easily identify in the next release the exact command.
Using this the output in the logs looks like this:
240131 09:17:00 31857 XrootdXeq: root.988:60@localhost pvt IP64 login as root
240131 09:17:00 31857 FstOfs_fsctl: root.988:60@localhost Unable to execute fsctl function cmd=3 args="/eos"; operation not permitted
240131 09:17:00 31857 XrootdXeq: root.988:60@localhost disc 0:00:00
Once you’ll have the next release in production (5.2.9) we can more easily identify who/what is calling this API.
Thanks a lot for kind explanation, fix and update. I will try it as soon as it is released.