“XRootD TPC with delegated credentials” and PSS node

I would like to ask particularly for “XRootD TPC with delegated credentials” and PSS node

  • Do you have Alice instance that they use this gateway?

  • How do you configure a load balancer for this node and how many know you use?

  • The redirect-gateway on MGMs ( e.g. ofs.tpc redirect delegated eos-gateway-node.cern.ch:1094 )

can be a round-robin alias over N nodes ?

  • The pss.origin eos-target-instance.cern.ch:1094 on PSS should be to the master MGM?

  • Could the PSS component be co-install in a Fst node?

thank you in advance

best regards

e.v.

Hi Emmanouil,

ALICE does not use delegated credentials, they only use their authz token which is embedded in the URL. For ALICE you need to disable the SSS enforcement on the FST side:
EOS_FST_NO_SSS_ENFORCEMENT="1"
and this allows TPC transfers between your instance and any other instance.

For other use-cases, yes the redirection endpoint can be a round-robin alias. Yes, you can install the PSS gateways on the FSTs if you want and indeed the pss.origin needs to point to the MGM node.

Cheers,
Elvin

hello Elvin
thank you for your reply
at CERN do you use PSS on production?
thank you in advance
best
e.v.

Hi Emmanouil,

Yes, we use such “PSS delegation gateways” for ATLAS and CMS. They use a round-robin alias.

Cheers,
Elvin

Hello @esindril & EOS community,
I’m also in the need to support “XRootD TPC with delegated credentials” (I think) for CMS for RUCIO transfers
I’m currently running EOS 5.2.22 in production.

I read in XRootD TPC with EOS - #2 by esindril that if I setup a proxy as explained in 4.7. Protocols & APIs — EOS DIOPSIDE documentation then I don’t need to set EOS_FST_NO_SSS_ENFORCEMENT=1 in my FSTs. Right ?

I currently have EOS_FST_NO_SSS_ENFORCEMENT=1 set in my /etc/eos/config/fst/fst FSTs config file

and on my MGM I have :

[root@lyoeosmgm1 ~]# grep tpc /etc/eos/config/mgm/mgm |sed -e "/^#/d"
http.exthandler xrdtpc libXrdHttpTPC.so
ofs.tpc  redirect delegated lyoeos-gw.in2p3.fr:1094

on my vanilla XRootD PSS gateway lyoeos-gw.in2p3.fr, I have :

[root@lyoeos-gw ~]# cat /etc/xrootd/xrootd-eos-gw.cfg
ofs.osslib  libXrdPss.so
ofs.ckslib  * libXrdPss.so
xrootd.chksum  adler32
xrootd.seclib  libXrdSec.so
pss.origin  lyoeos.in2p3.fr:1094
all.export  /eos/
all.adminpath  /var/spool/xrootd
all.pidpath  /var/run/xrootd
sec.protocol  gsi -dlgpxy:1 -exppxy:=creds -crl:1 -moninfo:1 -cert:/etc/grid-security/hostcert.pem -key:/etc/grid-security/hostkey.pem -gridmap:/etc/grid-security/grid-mapfile -d:1 -gmapopt:2
sec.protbind  * gsi
ofs.tpc  autorm fcreds gsi =X509_USER_PROXY ttl 60 60 xfr 9 pgm /usr/bin/xrootd-third-party-copy.sh

[root@lyoeos-gw ~]# cat /usr/bin/xrootd-third-party-copy.sh
#!/bin/bash
dst='root://'$XRDXROOTD_ORIGIN'/'$2
/usr/bin/xrdcp --server -d 3 $1 $dst

[root@lyoeos-gw ~]# rpm -qf /usr/bin/xrdcp
xrootd-client-5.4.2-1.el7.x86_64

The command :

XrdSecPROTOCOL=gsi,unix xrdcp --tpc delegate only root://sbgdcache.in2p3.fr:1094///store/mc/Run3Summer22EENanoAODv12/WpWpJJ_EWK-QCD_TuneCP5_13p6TeV_madgraph-pythia8/NANOAODSIM/130X_mcRun3_2022_realistic_postEE_v6-v2/50000/28268227-06eb-41ef-998a-48f90ffaeeeb.root \
    root://lyoeos.in2p3.fr//eos/lyoeos.in2p3.fr/home/pugnere/toto

Gives an error :

Run: [ERROR] Server responded with an error: [3000]  [[x]root[s]://<host>[:<port>]/]<path> | -

I don’t know what is wrong In my config.
Best regards,
Denis

Hi Denis,

If you are using XRootD 5 on the PSS endpoint then your script should look like this:

[root@eoscmsftp21 ~]# cat /usr/local/bin/xrootd-third-party-copy.sh
#! /usr/bin/env bash
export XRD_STREAMTIMEOUT=600

if [[ $(/usr/bin/xrdcp --version 2>&1 | grep -oP "v\K(\d)") -ge 5 ]]; then
    # XRD_CPTARGET should be set
    # add protocol prefix to destination if not present
    ## Run: [ERROR] Server responded with an error: [3000]  [[x]root[s]://<host>[:<port>]/]<path> | -
    if [[ ! "${a}" =~ x?root* ]]; then
        set -- "${@:1:$#-1}" "xroot://${XRDXROOTD_ORIGIN}/${@: -1}"
    fi

    /usr/bin/xrdcp $@
    EXITCODE=$?;

    if [[ ${EXITCODE} -ne 0 ]];then
        /usr/bin/logger -t xrootd-third-party-copy.sh "FAILED xrootd-tpc transfer [${EXITCODE}]: env $(env | grep ^XRD | tr '\n' ' ') /usr/bin/xrdcp $@"
    fi
else
    dst='root://'$XRDXROOTD_ORIGIN'/'$2
    /usr/bin/xrdcp --server $1 $dst
fi

This one is compatible with both XRootD 4 and 5. I will take care of updating also the documentation.

Cheers,
Elvin

Thanks a lot (again) Elvin !
It works.

but for the line 8 of your script :
if [[ ! "${a}" =~ x?root* ]]; then
I don’t understand the ${a}, I would expect a test on arguments, like $1 of $2 ?

Just for double check : With a PSS endpoint, does I still need EOS_FST_NO_SSS_ENFORCEMENT=1 on my FSTs ?
Denis

Hi Denis,

You are absolutely right, this script can be dramatically simplified, plus there is still an issue with the fact that is does not report the error in case one happens. Therefore, I would recommend the following script:

#! /usr/bin/env bash
export XRD_STREAMTIMEOUT=600

if [[ $(/usr/bin/xrdcp --version 2>&1 | grep -oP "v\K(\d)") -ge 5 ]]; then
    /usr/bin/xrdcp $@
else
    dst='root://'$XRDXROOTD_ORIGIN'/'$2
    /usr/bin/xrdcp --server $1 $dst
fi

Yes, if you have the PSS endpoint in place you can remove the SSS env variable from the FST configuration.

Cheers,
Elvin

Thanks Elvin,
I can share the script that worked for me, based on yours.
It adds the xrootd servername origin before the path of the destination i.e.the last argument of the script)

#! /usr/bin/env bash
export XRD_STREAMTIMEOUT=600

if [[ ! "${@: -1}" =~ x?root* ]]; then
    set -- "${@:1:$#-1}" "xroot://${XRDXROOTD_ORIGIN}/${@: -1}"
fi

/usr/bin/xrdcp $@
EXITCODE=$?

if [[ ${EXITCODE} -ne 0 ]]; then
     /usr/bin/logger -t xrootd-third-party-copy.sh "FAILED script arguments : $@"
     /usr/bin/logger -t xrootd-third-party-copy.sh "FAILED xrootd-tpc transfer [${EXITCODE}]: env $(env | grep ^XRD | tr '\n' ' ') /usr/bin/xrdcp $1 $dst"
fi

cheers,
Denis

Hi Denis,

Actually, after discussing with you I had a look at our gateways and realized that some clients depending on their version of the xrootd client that they are using were failing. Therefore, we had another look at the script and improved it. This script will now be shipped as part of vanilla XRootD and can handle TPC for both simple XRootD servers (used as a destination) but also for PSS XRootD servers.

I paste it here and maybe you can give it a try. I hope it works well for you and therefore you won’t need to maintain your own special script:

#!/bin/bash

OPTS=("${@:1:$#-2}")
shift $(($# - 2))

SRC=$1
DST=$2

if [[ -n "${XRDXROOTD_ORIGIN}" ]]; then
        DST="root://${XRDXROOTD_ORIGIN}/${DST}"
fi

xrdcp --server "${OPTS[@]}" "${SRC}" "${DST}"
STATUS=$?

if [[ ${STATUS} -ne 0 ]]; then
        logger -p err  -t xrdcp-tpc "transfer: xrdcp --server ${OPTS[*]} ${SRC} ${DST} FAILED [exit code: ${STATUS}]"
fi

exit ${STATUS}

Looking forward to your feedback on this.

Thanks,
Elvin

Great, I work !
Thanks a lot Elvin