Xrdfs spaceinfo does not work with full EOS path

1

Hello,

Can you please help me understand why “xrdfs spaceinfo” reports numbers when the path is a single “/”

[root@antares-eos01 ~]# xrdfs antares.stfc.ac.uk:1094 spaceinfo /
Path: /
Total: 172749993541632
Free: 142871550230528
Used: 29878443311104
Largest free chunk: 68719476736

but reports zero sizes when the path is a full path, i.e

[root@antares-eos01 ~]# xrdfs antares.stfc.ac.uk:1094 spaceinfo /eos/antares/prod/alice/tape/
Path: /eos/antares/prod/alice/tape/
Total: 0
Free: 0
Used: 0
Largest free chunk: 68719476736

Thanks,

George

2

Hi,

Can you please help me with this issue? ALICE tape storage accouting relies on it.

George

3

This does not work, because you path is too deep.
Until level 4 we report the physical space, otherwise we display the quota. And probably you don’t have quota.
/eos/antares/prod/alice/tape/

I can push a fix, that you can configure this via an environment variable.

4

commit 683185f26d70d2b0f257f61f882571dc2757bafb
Author: Andreas Joachim Peters andreas.joachim.peters@cern.ch
Date: Fri Oct 21 11:16:02 2022 +0200

MGM: allow to force to show only physical space via 'spaceinfo'/statvfs using EOS_MGM_STATVFS_ONLY_SPACE=1
5

Thanks for this Andreas.

How can we make use of this fix? Is it going to be included in a citrine release?
We are currently running 4.8.78-1 and planning to 4.8.88 or whatever the latest
CTA requires.

George

6

Just talked to Julien, we will make an EOS tag, then he will do a public CTA release, which you can use.

7

Hi Andreas,

Is this feature available in EOS 4.8.88-1 or we need to use the specific tag you made it?

Best,

George

8

This was released with 4.8.93. The latest EOS4 release was 4.8.101. Please note, we are also stopping support on EOS4!

9

Thanks Andreas. When excatly is EOL for EOS4?
Sorry, didn’t manage to find this in the website

10

Hi Andreas,

I added EOS_MGM_STATVFS_ONLY_SPACE=1, in /etc/sysconfig/eos_env, on our dev instance running 4.8.98-1 but the result was the same, i.e. I still get zero values for Total, Free and Used

-bash-4.2$ xrdfs antares-dev.stfc.ac.uk:1094 spaceinfo /eos/antaresdev/et_jasmin/
Path: /eos/antaresdev/et_jasmin/
Total: 0
Free: 0
Used: 0
Largest free chunk: 68719476736

Am I doing some thing wrong?

George

11

Hi George,

Do you have the “default” space defined in your instace?
What is the output of eos space ls?

Thanks,
Elvin

12

Hi Elvin,

Yes, we do. Here is the output of eos space ls

[root@antares-eos14 ~]# eos space ls
┌──────────┬────────────────┬────────────┬────────────┬──────┬─────────┬───────────────┬──────────────┬─────────────┬─────────────┬──────────────┬──────┬──────────┬───────────┬───────────┬──────┬────────┬───────────┬──────┬────────┬───────────┐
│type │ name│ groupsize│ groupmod│ N(fs)│ N(fs-rw)│ sum(usedbytes)│ sum(capacity)│ capacity(rw)│ nom.capacity│sched.capacity│ quota│ balancing│ threshold│ converter│ ntx│ active│ wfe│ ntx│ active│ intergroup│
└──────────┴────────────────┴────────────┴────────────┴──────┴─────────┴───────────────┴──────────────┴─────────────┴─────────────┴──────────────┴──────┴──────────┴───────────┴───────────┴──────┴────────┴───────────┴──────┴────────┴───────────┘
spaceview default 0 0 8 8 396.23 MB 15.36 TB 15.36 TB 0 B 15.36 TB off off 20 off 2 0 on 500 0 off
spaceview retrieve 0 0 7 7 340.93 MB 13.44 TB 13.44 TB 0 B 13.44 TB off off 20 off 2 0 on 500 0 off

Thanks,

George

13

Hi George,

Can we double check that the env variable is properly in place?
Please issue this command:
grep EOS_MGM_STATVFS_ONLY_SPACE /proc/<mgm_pid>/environ; echo $?

Thanks,
Elvin

14

Hi Elvin,

It looks like it is

[root@antares-eos14 ~]# ps aux | grep mgm
daemon 7199 2.5 0.1 17690328 320172 ? SLsl 10:12 4:06 /opt/eos/xrootd/bin/xrootd -n mgm -c /etc/xrd.cf.mgm -l /var/log/eos/xrdlog.mgm -Rdaemon
daemon 7300 0.0 0.0 1051072 12076 ? S 10:12 0:00 /opt/eos/xrootd/bin/xrootd -n mgm -c /etc/xrd.cf.mgm -l /var/log/eos/xrdlog.mgm -Rdaemon
root 43128 0.0 0.0 112820 2376 pts/0 S+ 12:56 0:00 grep --color=auto mgm
[root@antares-eos14 ~]#
[root@antares-eos14 ~]#
[root@antares-eos14 ~]# cat /proc/7199/environ
SHELL=/bin/shEOS_FST_HTTP_PORT=8001EOS_HA_REDIRECT_READS=1USER=rootEOS_BROKER_URL=root://antares-eos14.scd.rl.ac.uk:1097//eos/EOS_MGM_STATVFS_ONLY_SPACE=1EOS_GEOTAG=undefPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/binEOS_QUARKDB_PASSWD=/etc/eos.keytabPWD=/var/eosXRD_ROLES=quarkdb fst mgm mqLANG=en_GB.UTF-8SHLVL=0HOME=/rootEOS_FST_NO_SSS_ENFORCEMENT=1EOS_QUARKDB_HOSTPORT=antares-eos14:9999LOGNAME=rootEOS_INSTANCE_NAME=eosantaresdevEOS_MGM_ALIAS=antares-eos14.scd.rl.ac.ukEOS_FST_DELETE_QUERY_INTERVAL=30EOS_USE_QDB_MASTER=1[root@antares-eos14 ~]#
[root@antares-eos14 ~]#
[root@antares-eos14 ~]#
[root@antares-eos14 ~]# cat /proc/7300/environ
SHELL=/bin/shEOS_FST_HTTP_PORT=8001EOS_HA_REDIRECT_READS=1USER=rootEOS_BROKER_URL=root://antares-eos14.scd.rl.ac.uk:1097//eos/EOS_MGM_STATVFS_ONLY_SPACE=1EOS_GEOTAG=undefPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/binEOS_QUARKDB_PASSWD=/etc/eos.keytabPWD=/var/eosXRD_ROLES=quarkdb fst mgm mqLANG=en_GB.UTF-8SHLVL=0HOME=/rootEOS_FST_NO_SSS_ENFORCEMENT=1EOS_QUARKDB_HOSTPORT=antares-eos14:9999LOGNAME=rootEOS_INSTANCE_NAME=eosantaresdevEOS_MGM_ALIAS=antares-eos14.scd.rl.ac.ukEOS_FST_DELETE_QUERY_INTERVAL=30EOS_USE_QDB_MASTER=1[root@antares-eos14 ~]#
[root@antares-eos14 ~]#

15

Hi George,

It looks like the initial fix is not enough and as this env variable needs to be taken into account also earlier in the decision process. Using the following patch, things work as expected.

--- a/mgm/XrdMgmOfs/Fsctl.cc
+++ b/mgm/XrdMgmOfs/Fsctl.cc
@@ -107,7 +107,8 @@ XrdMgmOfs::fsctl(const int cmd,
     eos_thread_info("path=%s cgi=%s", path.c_str(), opaque.c_str());
 
     if (query_space || 
-       (!getenv("EOS_MGM_STATVFS_ONLY_QUOTA") && ((path == "/") || (path == "")))) {
+       (!getenv("EOS_MGM_STATVFS_ONLY_QUOTA") && ((path == "/") || (path == ""))) ||
+        (getenv("EOS_MGM_STATVFS_ONLY_SPACE"))) {
       if (FsView::gFsView.mSpaceView.count(space.c_str())) {
        freebytes =
          FsView::gFsView.mSpaceView[space.c_str()]->SumLongLong("stat.statfs.freebytes",

I will commit this change to EOS 5 and will backport it to EOS 4 if there will be any new release on that branch.

Thanks,
Elvin

16

Hi Elvin,

Many thanks for this. So, if I am not mistaken, your patch will be included in version 4.8.104 (just to keep an eye).

George

17

Hi George,

Yes, indeed, that would be version 4.8.104.

Cheers,
Elvin

18

Hi Elvin,

Sorry to come bvack to this. I tested “xrdfs spaceinfo” on our upgraded EOS instances (5.1.28) but I still get zero values

xrdfs antares-preprod.stfc.ac.uk spaceinfo /eos/antarespreprodtier1/dteam/
Path: /eos/antarespreprodtier1/dteam/
Total: 0
Free: 0
Used: 0
Largest free chunk: 68719476736

I have included EOS_MGM_STATVFS_ONLY_SPACE=1
in /etc/sysconfig/eos_env

and also

[root@antares-eos94 ~]# ps aux | grep mgm
root 1678 0.0 0.0 112816 2244 pts/0 S+ 15:49 0:00 grep --color=auto mgm
daemon 31378 2.9 0.6 16386212 208540 ? SLsl 15:40 0:16 /opt/eos/xrootd/bin/xrootd -n mgm -c /etc/xrd.cf.mgm -l /var/log/eos/xrdlog.mgm -Rdaemon
[root@antares-eos94 ~]#
[root@antares-eos94 ~]# strings /proc/31378/environ | grep EOS_MGM_STATVFS_ONLY_SPACE
EOS_MGM_STATVFS_ONLY_SPACE=1
[root@antares-eos94 ~]#

Is the above change not included in 5.1.28?

Thanks,

George

19

Hi George,

Checking the git history this seems to be included only starting with 5.1.29.

$ git blame -L 113,+2 ../mgm/XrdMgmOfs/Fsctl.cc
7a27b4fe34 (Elvin Alin Sindrilaru 2023-08-14 11:21:32 +0200 113)         (!getenv("EOS_MGM_STATVFS_ONLY_QUOTA") && ((path == "/") || (path == ""))) ||
7a27b4fe34 (Elvin Alin Sindrilaru 2023-08-14 11:21:32 +0200 114)         (getenv("EOS_MGM_STATVFS_ONLY_SPACE"))) {
$ git tag --contains 7a27b4fe34
5.1.29
5.1.30
5.2.0
5.2.1
5.2.2
5.2.3
5.2.3-2
5.2.4

Cheers,
Elvin

20

Hi Elvin,

Happy New Year,

Many thanks for confirming this. We will stick to 5.1.28 for the time being as we were told by the CTA team at CERN that there issues with 5.1.29 and CTA.

George