Eos-xrootd vs xrootd

1

Hi,

Sorry if this is a trivial question. I am trying to understand the relation of eos-xrootd and xrootd packages in EOS5 - we are planning to upgrade from 4.8.105 to 5.1.28

Until now, on our EOS nodes (EOSCTA instance), the version of eos-xrootd and xrootd was the highest found in the citrine deps repo - 4.12.8-1 which was the same for the two packages.

In EOS5, this is no longer the case: the highest xrootd version in the diopside deps repo (5.2.0-1) is not the same with the highest version of the eos-xrootd package (5.6.2-1). So, I am unsure which one to choose for EOS 5.1.28.

Thanks,

George

2

Hi George,

You no longer need to version lock the xrootd/eos-xrootd packages as the eos packages will come with strict dependencies. We recommend running the EOS services with the xrootd version that comes from the eos-xrootd package. For this you should set the LD_LIBRARY_PATH=/opt/eos/xrootd/lib64 in the /etc/sysconfig/eos_env and then inside the configuration files /etc/xrd.cf.mgm/fst use relative paths for the xrootd specific libraries.

Having said this if you want to know the exact version of the dependencies you can always inspect the rpm package listing the dependencies or you can look in the eos.spec.in file for that particular request in the git repo to see the hard-coded dependencies. As an example for 5.1.30 you can find them here:
https://gitlab.cern.ch/dss/eos/-/blob/5.1.30/eos.spec.in?ref_type=tags#L33

Cheers,
Elvin

3

Hi Evlin,

Thanks for clarifying.

Tried to comment out the following packages in the config management template but I still get yum errors

‘/software/packages’ = pkg_repl(‘eos-server’, EOS_VERSION_PLATFORM, ‘x86_64’);
‘/software/packages’ = pkg_repl(‘eos-client’, EOS_VERSION_PLATFORM, ‘x86_64’);
‘/software/packages’ = pkg_repl(‘eos-archive’, EOS_VERSION_PLATFORM, ‘x86_64’);
#‘/software/packages’ = pkg_repl(‘eos-fuse’, EOS_VERSION_PLATFORM, ‘x86_64’);
#‘/software/packages’ = pkg_repl(‘eos-fuse-core’, EOS_VERSION_PLATFORM, ‘x86_64’);
#‘/software/packages’ = pkg_repl(‘eos-fuse-sysv’, EOS_VERSION_PLATFORM, ‘x86_64’);
#‘/software/packages’ = pkg_repl(‘eos-fusex’, EOS_VERSION_PLATFORM, ‘x86_64’);
#‘/software/packages’ = pkg_repl(‘eos-fusex-core’, EOS_VERSION_PLATFORM, ‘x86_64’);
#‘/software/packages’ = pkg_repl(‘eos-fusex-selinux’, EOS_VERSION_PLATFORM, ‘x86_64’);
‘/software/packages’ = pkg_repl(‘eos-ns-inspect’, EOS_VERSION_PLATFORM, ‘x86_64’);
‘/software/packages’ = pkg_repl(‘eos-test’, EOS_VERSION_PLATFORM, ‘x86_64’);
‘/software/packages’ = pkg_repl(‘eos-testkeytab’, EOS_VERSION_PLATFORM, ‘x86_64’);

#‘/software/packages’ = pkg_repl(‘eos-xrootd’, EOS_XROOTD_VERSION + EOS_PLATFORM, ‘x86_64’);
‘/software/packages’ = pkg_repl(‘eos-folly’, EOS_FOLLY_VERSION + EOS_PLATFORM, ‘x86_64’);
‘/software/packages’ = pkg_repl(‘eos-folly-deps’, EOS_FOLLY_VERSION + EOS_PLATFORM, ‘x86_64’);
#‘/software/packages’ = pkg_repl(‘eos-scitokens’);

#‘/software/packages’ = pkg_repl(‘xrootd’, EOS_XROOTD_VERSION + ‘.el7’, ‘x86_64’);
#‘/software/packages’ = pkg_repl(‘xrootd-client’, EOS_XROOTD_VERSION + ‘.el7’, ‘x86_64’);
#‘/software/packages’ = pkg_repl(‘xrootd-server’, EOS_XROOTD_VERSION + ‘.el7’, ‘x86_64’);
#‘/software/packages’ = pkg_repl(‘python2-xrootd’, EOS_XROOTD_VERSION + ‘.el7’, ‘x86_64’);

Does the upgrade to EOS5 require a downtime for the EOS services running on the host.
I mean, doing

yum remove eos-xrootd-4.12.8-1.el7.cern.x86_64

will remove basically all the installed EOS packages.

George

4

Hi George,

First you don’t need to remove eos-xrootd but just upgrade it. Then, what yum errors did you encounter?

Cheers,
Elvin

5

Hi Elvin,

Yes, sorry, of course we dont have to remove eos-xrootd. I managed to resolve the yum errors I mentioned above. I am testing manually different
“yum install” combinations to get deps right: it seems that the following combination works
(had to remove the existing eos-scitokens-1.2.0-1.el7.cern.x86_64)

yum install eos-server-5.1.28-1.el7.cern.x86_64 eos-client-5.1.28-1.el7.cern.x86_64 eos-archive-5.1.28-1.el7.cern.x86_64 eos-ns-inspect-5.1.28-1.el7.cern.x86_64 eos-test-5.1.28-1.el7.cern.x86_64 eos-testkeytab-5.1.28-1.el7.cern.x86_64 eos-fusex-5.1.28-1.el7.cern.x86_64 eos-fusex-core-5.1.28-1.el7.cern.x86_64 eos-fusex-selinux-5.1.28-1.el7.cern.x86_64

I cant see xrootd packages in the list of new installs or updates, see here
http://www-public.gridpp.rl.ac.uk/filelists/EOS5_upgrade_goodsofar.log

If I try to add, " xrootd xrootd-client xrootd-server python2-xrootd" (without pining the version) into the mix, then I get the following yum error related to eos-grpc, see here
http://www-public.gridpp.rl.ac.uk/filelists/EOS5_upgrade_problemxrootd.log

Finally, do we still need to install the eos-folly and eos-folly-deps packages.

Best,

George

6

Just to add that I get the same yum error if I add into the list of packages the following
“eos-fuse eos-fuse-core eos-fuse-sysv” instead of xrootd

7

Hi George,

Yes, eos-scitokens is not needed anymore and can be removed.

Looking at the first log that you posted, things look fine. You only need eos-xrootd to run the EOS services, so the fact that you don’t have any xrootd packages is fine. One more thing that you can also remove is the grpc and grpc-devel packages. This are now replaced by eos-grpc. This will become more important when you will upgrade to EOS 5.2.x.

Looking at the second log that you pasted, the problem there is that the system tries to pick up the latest eos packages, note that it tries to instal 5.2.2. Which I don’t think you want for the moment. Best would be to put in place the version lock for eos 5.1.28 and things should work fine.

Yes, eos-folly and eos-folly-deps are needed by there was no change to them so this should not cause any issues.

Cheers,
Elvin

8

Hi Elvin,

Thanks for reply. f I try to remove the installed “grpc-1.19.0-1.el7.x86_64”, yum will also remove the existing (4.8.98) eos-server and eos-test (and others) - which brings up the question if we need a downtime for an EOS5 upgrade.

In the both logs, I posted I do version lock all the eos packages that I could see they have 5.1.28 in their name. I am struggling to see why yum picked up lthe 5.2.2 version

George

9

Hi Elvin,

Adding, eos-quarkdb in addition to xrootd and xrootd-voms packages did the trick…!
http://www-public.gridpp.rl.ac.uk/filelists/EOS5_upgrade_good_withxrootd.log

It is just now that EOS services will use the XRootD 5.2.0 libs (latest version found in the diopside deps repo) which will be against your recommendation above (" running the EOS services with the xrootd version that comes from the eos-xrootd package")

The thing is is that with EOS4, we have not so far defined LD_LIBRARY_PATH=/opt/eos/xrootd/lib64 in the /etc/sysconfig/eos_env and I think that worked (in production) because the latest eos-xrootd and xrootd versions in the citrine deps repo were the same (4.12.8)

So now, as I understand we either

  • Do not install xrootd packages on EOS nodes - as they are no needed to run EOS services as you said - at all and defineLD_LIBRARY_PATH=/opt/eos/xrootd/lib64 in the /etc/sysconfig/eos_env

or

  • Install XRootD 5.5.10-1 from our local XRootD repo and leave etc/sysconfig/eos_env without a definition of LD_LIBRARY_PATH.

Best,

George

10

Hi George,

For EOS 4, grpc is still needed. So you can remove the package after updating to eos 5 which comes with the eos-grpc dependency. You will need downtime, in the sense that the MGM will need to be restarted - that’s for sure.

As a long term approach, I would recommend to always use LD_LIBRARY_PATH inside the /etc/sysconfig/eos_env file - this is the safest approach to avoid any surprises. We run in production relying on eos-xrootd and not vanilla xrootd. You can for sure install both eos-xrootd and xrootd packages alonside (that was the intention when we created the eos-xrootd packages) but make sure you run EOS using the eos-xrootd ones.

The thing is that sometimes we will have an eos-xrootd package that does not have a direct correspondent to xrootd packages, as we might find and fix earlier bugs that come from xrootd. That is the entire rationale behind this decision.

Cheers,
Elvin

11

Hi Elvin,

Thanks for the very clear and detailed comments.

One last thing…

Re the XRootD TPC proxy nodes that we have set up to run together with our Tier-1 EOS cluster. They are not running EOS services, only a single standard xrootd-proxy service that redirects external TPC reqs to EOS FSTs.

Based on eos.spec.in · 5.1.28 · dss / eos · GitLab

I willl install XRootD 5.5.5 on these nodes (and eos-xrootd 5.5.10 - which we most probably dont need but I will check). Is this correct?

Best,

George

12

Hi George,

Yes, indeed you can run XRootD 5.5.5 for the TPC proxy service and in this way you would have things nicely aligned. The TPC proxy service comes from vanilla xrootd so you should not need to install eos-xrootd.

Cheers,
Elvin