EOS 5.5.0 MGM with WLCG token support fails to start

georgep · September 9, 2026, 11:23am

Hello,

I have upgraded one of our preprod instances to EOS 5.5.0 and it works fine with current auth config (x509/GSI). When I add to /etc/xrd.cf.mgm all the directives I think I need for WLCG token authentication the MGM service fails to start with the following message:

60909 12:06:53 1446208 sysEosMgmHttpHandler: Unable to Failed config of EosMgmHttpHandler; invalid argument
------ HTTP protocol initialization failed.
260909 12:06:53 1446208 XrdProtocol: Protocol XrdHttp could not be loaded
------ xrootd mgm@antares-eos94.scd.rl.ac.uk:-1 initialization failed.

I am not quite sure what exactly I am missing. Can you please help?

Thanks,

George

rptaylor · September 14, 2026, 7:01pm

Hi George,

all the directives I think I need for WLCG token authentication

Which ones?

Our v5.5.1 MGM has this and it seems to work:

# grep -i http /etc/xrd.cf.mgm
##### Setup HTTP
http.exthandler EosMgmHttp libEosMgmHttp.so eos::mgm::http::redirect-to-https=0
http.exthandler xrdtpc libXrdHttpTPC.so
#http.gridmap /etc/grid-security/grid-mapfile
http.secxtractor libXrdVoms.so
http.trace all
xrd.protocol XrdHttp:8443 libXrdHttp.so

However the token-related config is a separate thing, we have this:

[root@eos-mgm-1 /]# grep -i sci /etc/xrd.cf.mgm
mgmofs.macaroonslib libXrdMacaroons.so libXrdAccSciTokens.so

[root@eos-mgm-1 /]# cat /etc/xrootd/scitokens.cfg 
[Global]
# If a token is not present or does not authorize the requested action, invoke the next configured authorization plugin.
#onmissing = passthrough
audience = https://wlcg.cern.ch/jwt/v1/any, https://eos-mgm.wlcg.uvic.ca:8443, roots://eos-mgm.wlcg.uvic.ca:1094, eos-mgm.wlcg.uvic.ca

[Issuer ATLAS]
issuer = https://atlas-auth.cern.ch/
base_path = /eos/wlcg.uvic.ca/data/atlas
map_subject = False
name_mapfile = /etc/eos.config/scitokens.map
default_user = nobody
authorization_strategy = capability

[Issuer DTeam]
issuer = https://dteam-auth.cern.ch/
base_path = /eos/wlcg.uvic.ca/data/dteam
map_subject = False
default_user = dteam

[root@eos-mgm-1 /]# cat /etc/eos.config/scitokens.map 
[
  {"path": "/atlasscratchdisk/",    "result": "atlas", "comment": "Owner of ATLASSCRATCHDISK"},
  {"path": "/atlasdatadisk/",       "result": "atprd", "comment": "Owner of ATLASDATADISK"},
  {"path": "/atlaslocalgroupdisk/", "result": "atcan", "comment": "Owner of ATLASLOCALGROUPDISK"}
]

Previously I tried to figure that out here: WLCG Token AuthN/Z for EOS but didn’t get any answers, anyway that is what I came up with.

Thanks.

georgep · September 16, 2026, 9:28am

Thanks Ryan.

We have the following directives for token support

sec.protocol ztn
sec.protbind * only ztn gsi unix
mgmofs.authlib ++ libXrdAccSciTokens.so config=/etc/xrootd/scitokens.cfg
mgmofs.authlib ++ libXrdMacaroons.so

xrootd.tls capable all
http.header2cgi Authorization authz

xrd.tls /etc/grid-security/daemon/hostcert.pem /etc/grid-security/daemon/hostkey.pem
xrd.tlsca certdir /etc/grid-security/certificates/
scitokens.trace all

and the http block has not changed

if exec xrootd
  xrd.protocol XrdHttp:%5$s libXrdHttp.so
  http.cadir /etc/grid-security/certificates/
  http.cert /etc/grid-security/xrootd/hostcert.pem
  http.key /etc/grid-security/xrootd/hostkey.pem
  http.gridmap /etc/grid-security/http-grid-mapfile
  http.trace all
  http.exthandler xrdtpc libXrdHttpTPC.so
  http.exthandler EosMgmHttp libEosMgmHttp.so eos::mgm::http::redirect-to-https=0
  mgmofs.macaroonslib libXrdMacaroons.so
  macaroons.secretkey /etc/xrootd/macaroon-secret
  macaroons.trace all
fi

We are running 5.5.0 though (instead 5.5.1). I wonder if this makes any difference…

Best,

George

georgep · September 16, 2026, 10:09am

Sadly, the upgrade to 5.5.1 did not make a difference, The error remains.

ccaffy · September 16, 2026, 11:02am

Hi @georgep ,

This is what we have for the token authentication (xrd.cf.mgm):

mgmofs.macaroonslib libXrdMacaroons.so libXrdAccSciTokens.so
macaroons.secretkey /etc/eos.macaroon.secret
macaroons.trace all

we do not use mgmofs.authlib.

Can you please try?

georgep · September 22, 2026, 12:55pm

Hi @ccaffy

It works now - many thanks for this!!

For the record, I used mgmofs.authlib because of what is mentioned here xrootd/src/XrdSciTokens at master · xrootd/xrootd · GitHub

georgep · September 22, 2026, 12:58pm

How do you pass the “SciTokens Configuration File” (/etc/xrootd/scitokens.cfg) to the mgm config?

ccaffy · September 22, 2026, 1:54pm

It is picked automatically by the plugin, we do not configure that path

georgep · September 24, 2026, 3:15pm

We get an ztn auth error when we try to stat a file via xrootd (we verified that the token is correct)

260924 16:11:45 2556400 XrootdXeq: User authentication failed; ztn required plugin (libXrdAccSciTokens.so) has not been loaded!

I dont understand why EOS thinks the library is not loaded only when an access request is made,

ccaffy · September 24, 2026, 3:34pm

On the MGM process, can you do a lsof and tell me if indeed libXrdAccSciTokens.so is not loaded?

georgep · September 24, 2026, 3:54pm

Thanks, it looks loaded

[root@antares-eos14 ~]# ps aux | grep mgm
daemon   2555782  1.8  0.4 19325360 839528 ?     SLsl 16:11   0:47 /opt/eos/xrootd/bin/xrootd -n mgm -c /etc/xrd.cf.mgm -l /var/log/eos/xrdlog.mgm -Rdaemon
root     2565017  0.0  0.0   6424  2280 pts/0    S+   16:53   0:00 grep --color=auto mgm
[root@antares-eos14 ~]#
[root@antares-eos14 ~]# lsof -p 2555782  | grep libXrdAccSci
xrootd  2555782 daemon  mem       REG               65,5    136872 201699570 /opt/eos/xrootd/lib64/libXrdAccSciTokens-6.so
[root@antares-eos14 ~]#

ccaffy · September 25, 2026, 7:11am

Thanks, could you please tell me what eos-xrootd version you are running `rpm -qa | grep eos-xrootd` ?

ccaffy · September 25, 2026, 7:17am

actually, maybe the problem is somewhere else. It would be great that you have a look at the startup logs. When you start the MGM, xrootd will print lots of logs from the config and what it configures. The answer maybe in those lines. If you manage to send me those logs, I can have a deeper look.

Thanks

georgep · September 29, 2026, 3:34pm

Thanks for this.

I am running eos-xrootd 6.1.1-1

root@antares-eos14 ~]# rpm -qa | grep eos-xrootd
eos-xrootd-6.1.1-1.el9.x86_64

I put the MGM log lines that follow a restart of the service in http://www-public.gridpp.rl.ac.uk/tape_accounting/antares-eos14-xrdlog.mgm if you want to have look. I cannot see libXrdAccSciTokens being present in these or any other log lines.

ccaffy · October 1, 2026, 7:09am

Hi George,

OK can you please also share the current MGM configuration file?

I think the macaroonslib may be missing something…

Thanks!

georgep · October 1, 2026, 9:25am

Many thanks, please find it here: http://www-public.gridpp.rl.ac.uk/tape_accounting/xrd.cf.mgm

ccaffy · October 1, 2026, 10:03am

Hi George,

Thanks, it looks like this block does not contain the libXrdAccSciToken:

if exec xrootd
  xrd.protocol XrdHttp:9000 libXrdHttp.so
  http.cadir /etc/grid-security/certificates/
  http.cert /etc/grid-security/xrootd/hostcert.pem
  http.key /etc/grid-security/xrootd/hostkey.pem
  http.gridmap /etc/grid-security/http-grid-mapfile
  http.trace all
  http.exthandler xrdtpc libXrdHttpTPC.so
  http.exthandler EosMgmHttp libEosMgmHttp.so eos::mgm::http::redirect-to-https=0
  mgmofs.macaroonslib libXrdMacaroons.so
  macaroons.secretkey /etc/xrootd/macaroon-secret
  macaroons.trace all
fi

Try to add it and tell me what you get? Edit the line “mgmofs.macaroonslib”: mgmofs.macaroonslib libXrdMacaroons.so libXrdAccSciTokens.so

Thanks

CERN Accelerating science

EOS 5.5.0 MGM with WLCG token support fails to start