Hi all,
we have xrootd version 4.9.1 with eos version 4.4.23 installed for ALICE.
During tests we keep receiving error:
[2019-06-12 16:58:36.846203 +0200][Error ][XRootDTransport ] [eosalice.cern.ch:1094 #0.0] Authentication with sss failed: Decryption key not found.
Run: [ERROR] Operation not supported: Destination does not support third-party-copy.
We want to enable third-party-copy so we modified /etc/xrd.cf.fst file.
I do not know why, but xrootd is not using /etc/xrd.cf.fst properly when restarting. /etc/xrd.cf.fst looks like this:
###########################################################
set MGM=$EOS_MGM_ALIAS
###########################################################
xrootd.fslib -2 libXrdEosFst.so
xrootd.async off nosf
xrd.network keepalive
xrootd.redirect $(MGM):1094 chksum
###########################################################
xrootd.seclib libXrdSec.so
sec.protocol unix
sec.protocol sss -c /etc/eos.keytab -s /etc/eos.keytab
sec.protbind * only unix sss
###########################################################
all.export / nolock
all.trace none
all.manager localhost 2131
#ofs.trace open
###########################################################
xrd.port 1095
ofs.persist off
ofs.osslib libEosFstOss.so
ofs.tpc pgm /usr/bin/xrdcp
###########################################################
#this URL can be overwritten by EOS_BROKER_URL defined /etc/sysconfig/xrd
fstofs.broker root://localhost:1097//eos/
fstofs.autoboot true
fstofs.quotainterval 10
fstofs.metalog /var/eos/md/
#fstofs.trace client
###########################################################
xrootd.monitor all flush 60s window 30s dest files info user aliceml.cern.ch:9930
When restarted, eos@fst.service is preparing different configuration than required. It is visible in logs:
190612 10:04:27 14093 Starting on Linux 3.10.0-957.21.2.el7.x86_64
Copr. 2004-2012 Stanford University, xrd version v4.9.1
++++++ xrootd fst@a-fst1.grid.icm.edu.pl initialization started.
Config using configuration file /etc/xrd.cf.fst
=====> xrd.network keepalive
=====> xrd.port 1095
Config maximum number of connections restricted to 65000
Copr. 2012 Stanford University, xrootd protocol 4.0.0 version v4.9.1
++++++ xrootd protocol initialization started.
=====> xrootd.fslib -2 libXrdEosFst.so
=====> xrootd.async off nosf
=====> xrootd.redirect a-se.grid.icm.edu.pl:1094 chksum
=====> xrootd.seclib libXrdSec.so
=====> all.export / nolock
=====> xrootd.monitor all flush 60s window 30s dest files info user aliceml.cern.ch:9930
Config exporting /
Plugin loaded
++++++ Authentication system initialization started.
Plugin loaded
=====> sec.protocol unix
Plugin loaded
=====> sec.protocol sss -c /etc/eos.keytab -s /etc/eos.keytab
=====> sec.protbind * only unix sss
Config 3 authentication directives processed in /etc/xrd.cf.fst
------ Authentication system initialization completed.
++++++ Protection system initialization started.
Config warning: Security level is set to none; request protection disabled!
Config Local protection level: none
Config Remote protection level: none
------ Protection system initialization completed.
Config Routing for a-fst1.grid.icm.edu.pl: local pub4 prv4 pub6 prv6
Config Route all4: a-fst1.grid.icm.edu.pl Dest=[::213.135.54.19]:1095
Config Route all6: a-fst1.grid.icm.edu.pl Dest=[2001:6a0:2001:112::19]:1095
Plugin No such file or directory loading fslib libXrdEosFst-4.so
Config Falling back to using libXrdEosFst.so
Plugin loaded
++++++ © 2010 CERN/IT-DSS FstOfs (Object Storage File System) 4.4.23
++++++ File system initialization started.
190612 10:04:27 14093 FstOfs_Config: Configuration file not specified.
++++++ Storage system initialization started.
Config warning: config file not specified; defaults assumed.
Config effective Default oss configuration:
oss.alloc 0 0 0
oss.cachescan 600
oss.fdlimit 32500 65000
oss.maxsize 0
oss.trace 0
oss.xfr 1 deny 10800 keep 1200
oss.memfile off max 8379172864
oss.defaults r/w nocheck nodread nomig norcreate nopurge nostage xattr
------ Storage system initialization completed.
Config effective default ofs configuration:
all.role server
ofs.maxdelay 60
ofs.persist manual hold 600 logdir /tmp/fst/.ofs/posc.log
ofs.trace 0
------ File system server initialization completed.
=====> fstofs does not enforce SSS authentication for XROOT clients - make sure MGM enforces sss for this FST!
=====> fstofs.autoboot : true
=====> fstofs.broker : root://a-se.grid.icm.edu.pl:1097//eos/a-fst1.grid.icm.edu.pl:1095/fst
=====> eoscp-log : /var/log/eos/fst/eoscp.log
=====> fstofs.defaultreceiverqueue : /eos/*/mgm
I have no clue what is the origin of the problem. I am certain xrootd is invoked properly by the system with command:
/usr/bin/xrootd -n fst -c /etc/xrd.cf.fst -l /var/log/eo…
Best,
Karol Niedzielewski