Subscribe to global / regional redirector (CMS)

Site Administrators
1

Hi all,

We’re trying to finalize our EOS setup, apparently we missed the config to subscribe to the global xroot. redirector. I understand this is probably more xrootd specific than EOS, but I’m not sure if this might be part of the EOS cli client.
There are some generic docs,


So it looks like from Vienna wie should subscribe to xrootd-cms.infn.it:1213
What I dont understand is where the config is done for that,

  1. EOS cli commands, access set redirector host:port
  2. eos@fed service with config /etc/xrd.cf.fed
  3. EOS_FED_MANAGER in /etc/sysconfig/eos_env

Again with the background of running multiple VOs on that instance. Our auth is ok, but will the redirector affect this. Or can I limit the redirector to just a subtree?

Any help is greatly appreciated,
Best,
Erich

2

Hi Erich,

The only thing you need is one xrootd and one cmsd daemons that connect to the rest of the CMS federation and forward the requests to your CMS MGM daemon. You can find an example for the /etc/xrd.cf.fed daemon that we use at CERN, below:

###
# _______  __   __  _______  _______  _______  _______
#|       ||  | |  ||       ||       ||       ||       |
#|    _  ||  | |  ||    _  ||    _  ||    ___||_     _|
#|   |_| ||  |_|  ||   |_| ||   |_| ||   |___   |   |
#|    ___||       ||    ___||    ___||    ___|  |   |
#|   |    |       ||   |    |   |    |   |___   |   |
#|___|    |_______||___|    |___|    |_______|  |___|
# _______  _______  __    _  _______  ______    _______  _______  _______  ______
#|       ||       ||  |  | ||       ||    _ |  |   _   ||       ||       ||      |
#|    ___||    ___||   |_| ||    ___||   | ||  |  |_|  ||_     _||    ___||  _    |
#|   | __ |   |___ |       ||   |___ |   |_||_ |       |  |   |  |   |___ | | |   |
#|   ||  ||    ___||  _    ||    ___||    __  ||       |  |   |  |    ___|| |_|   |
#|   |_| ||   |___ | | |   ||   |___ |   |  | ||   _   |  |   |  |   |___ |       |
#|_______||_______||_|  |__||_______||___|  |_||__| |__|  |___|  |_______||______|
###

set  eospssport = 1098
set  eospssmgm = eoscms.cern.ch:1094
set  eospsspath = /
set  eosfedmanager = xrootd-cms.infn.it+:1213
set  sitename = T2_CH_CERN
xrd.port  $eospssport
ofs.osslib  libXrdPss.so
pss.origin  $eospssmgm $eospsspath
xrootd.redirect  $eospssmgm $eospsspath
xrootd.trace  emsg redirect
cms.trace  all
ofs.trace  most
all.export  $eospsspath
all.role  server
all.manager  $eosfedmanager
all.adminpath  /var/spool/xrootd
all.pidpath  /var/spool/xrootd
all.sitename  $sitename

The relevant bits in /etc/sysconfig/eos_env:

[root@eoscms-qdb-207a975e35 (mgm:master mq:master) ~]$ grep FED /etc/sysconfig/eos_env
EOS_FED_MANAGER="xrootd-cms.infn.it+:1213"
EOS_FED_SITENAME="T2_CH_CERN"

Below you can see the two daemons I mentioned:

[root@eoscms-qdb-207a975e35 (mgm:master mq:master) ~]$ systemctl status xrootd@fed
● xrootd@fed.service - XRootD xrootd daemon instance fed
   Loaded: loaded (/usr/lib/systemd/system/xrootd@.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/xrootd@fed.service.d
           └─custom.conf
   Active: active (running) since Tue 2020-10-27 09:44:41 CET; 1 months 0 days ago
     Docs: man:xrootd(8)
           http://xrootd.org/docs.html
 Main PID: 36368 (xrootd)
   CGroup: /system.slice/system-xrootd.slice/xrootd@fed.service
           └─36368 /usr/bin/xrootd -l /var/log/xrootd/xrootd.log -c /etc/xrootd/xrootd-fed.cfg -k fifo -s /var/run/xrootd/xrootd-fed.pid -n fed -Rdaemon

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
[root@eoscms-qdb-207a975e35 (mgm:master mq:master) ~]$ systemctl status cmsd@fed
● cmsd@fed.service - XRootD cmsd daemon instance fed
   Loaded: loaded (/usr/lib/systemd/system/cmsd@.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/cmsd@fed.service.d
           └─custom.conf
   Active: active (running) since Tue 2020-10-27 09:44:49 CET; 1 months 0 days ago
     Docs: man:cmsd(8)
           http://xrootd.org/docs.html
 Main PID: 36912 (cmsd)
   CGroup: /system.slice/system-cmsd.slice/cmsd@fed.service
           └─36912 /usr/bin/cmsd -l /var/log/xrootd/cmsd.log -c /etc/xrootd/xrootd-fed.cfg -k fifo -s /var/run/xrootd/cmsd-fed.pid -n fed -Rdaemon

These should be all the configs needed for an EOS instance to join the federation.

Cheers,
Elvin

3

Hi Elvin,

I have a config now, that is basically working I believe. Just the file permissions for the cmsd and xrootd took a bit of fiddling.

xrd.port 1098
ofs.osslib libXrdPss.so
pss.origin eos.grid.vbc.ac.at:1094
xrootd.redirect eos.grid.vbc.ac.at:1094 /
xrootd.trace  emsg redirect
cms.trace  all
ofs.trace  most
# specia, because of NAT?
xrd.network rpipa
all.sitename T2_AT_Vienna
all.export /
all.role server
all.manager cms-xrd-transit.cern.ch+:1213
all.adminpath /var/spool/xrootd
all.pidpath /var/run/xrootd

My question now is, since EOS mgm is basically HA, I’d like to have that for fed subscription too. i.e. run this on each of our mgms.
The docs in section “Special configuration for aliased redirector(s)” https://twiki.cern.ch/twiki/bin/view/CMSPublic/RedirectorsSubscription don’t exactly align with the config here - compare the all.manager in the docs vs pss.origin+xrootd.redirect here.

Can I just run this (the identical config) with cmsd+xrootd processes alongside the EOS mgm services on each of my mgm nodes?

Best,
Erich

4

Hi Erich,

I don’t think this will work. In principle the fed daemon is very lightweight and should not crash or be down in general - also there should be just one up and there is no logic to coordinate multiple such daemons. I suggest you have one of them which points to the alias of the MGMs. This is how we’ve run this since the beginning.

Cheers,
Elvin

5

Hello Elvin
on eos version 5 we could deploy the same configuration ?
I do not understand if we need to run cmsd or not
in the case that we need a local redirector in-from of eos - xroot ?
thank you in advance
best
e.v

6

Hi Emmanouil,

Nothing changes with respect to this functionality from EOS 4 to 5. So whatever you were doing for EOS 4 should work the same for 5.

Cheers,
Elvin