Hello!
Some data source has commas in directory names. Because of that data transfers fail with an error as below:
illegal characters - use only A-Z a-z 0-9 / SPACE .-_~#:^ /some-long-path-here/PDE_0,1pe/dirname/filename.dat; Invalid or incomplete multibyte or wide character
I wonder if it’s EOS limitation by design or it’s a bug?
Hi Nikolay,
The eos instance needs to be configured to accept any UTF8 characters and for this you need to set the following env variable in the /etc/sysconfig/eos_env file:
EOS_UTF8=0
Cheers,
Elvin
Dear Elvin,
Thanks a lot for your reply!
Dear Elvin,
I tried copying file with comma in filename without “EOS_UTF8=0”. With command:
gfal-copy /etc/hostname root://junoeos01.ihep.ac.cn//eos/juno/dirac/juno/user/z/zhangxt/a,a
It took much time so I use Ctrl+C to stop the command. It caused a mgm crash and here is the mgm logs:
211206 10:23:03 191633 XrootdXeq: zhangxua.22084:99@lxslc703 pub IP46 login as sjuno001
211206 10:23:03 time=1638757383.342957 func=stat level=ERROR logid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f090aefd700 source=Stat:78 tident=<single-exec> sec=(null) uid=99 gid=99 name=nobody geo="" illegal character in /eos/juno/dirac/juno/user/z/zhangxt/a,a
211206 10:23:03 time=1638757383.342984 func=Emsg level=ERROR logid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f090aefd700 source=XrdMgmOfs:1207 tident=<single-exec> sec= uid=0 gid=0 name= geo="" Unable to accept path name - illegal characters - use only A-Z a-z 0-9 / SPACE .-_~#:^ /eos/juno/dirac/juno/user/z/zhangxt/a,a; Invalid or incomplete multibyte or wide character
211206 10:23:03 time=1638757383.344466 func=stat level=ERROR logid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f090aefd700 source=Stat:78 tident=<single-exec> sec=(null) uid=99 gid=99 name=nobody geo="" illegal character in /eos/juno/dirac/juno/user/z/zhangxt/a,a
211206 10:23:03 time=1638757383.344484 func=Emsg level=ERROR logid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f090aefd700 source=XrdMgmOfs:1207 tident=<single-exec> sec= uid=0 gid=0 name= geo="" Unable to accept path name - illegal characters - use only A-Z a-z 0-9 / SPACE .-_~#:^ /eos/juno/dirac/juno/user/z/zhangxt/a,a; Invalid or incomplete multibyte or wide character
211206 10:23:03 time=1638757383.350217 func=IdMap level=INFO logid=static.............................. unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f090aefd700 source=Mapping:1042 tident= sec=(null) uid=99 gid=99 name=- geo="" sec.prot=gsi sec.name="sjuno001" sec.host="lxslc703.ihep.ac.cn" sec.vorg="" sec.grps="" sec.role="" sec.info="/C=CN/O=HEP/O=IHEP/OU=CC/CN=Xuantong Zhang" sec.app="" sec.tident="zhangxua.22084:99@lxslc703" vid.uid=58003 vid.gid=1004
211206 10:23:03 time=1638757383.350257 func=open level=ERROR logid=7094ca42-563b-11ec-86e9-0c42a15d0b00 unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f090aefd700 source=XrdMgmOfsFile:405 tident=zhangxua.22084:99@lxslc703 sec=gsi uid=58003 gid=1004 name=sjuno001 geo="" illegal character in /eos/juno/dirac/juno/user/z/zhangxt/a,a
211206 10:23:03 time=1638757383.350270 func=Emsg level=ERROR logid=7094ca42-563b-11ec-86e9-0c42a15d0b00 unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f090aefd700 source=XrdMgmOfsFile:3294 tident=zhangxua.22084:99@lxslc703 sec=gsi uid=58003 gid=1004 name=sjuno001 geo="" Unable to accept path name - illegal characters - use only A-Z a-z 0-9 / SPACE .-_~#:^ /eos/juno/dirac/juno/user/z/zhangxt/a,a; Invalid or incomplete multibyte or wide character
211206 10:23:03 time=1638757383.353361 func=Drop level=INFO logid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f09161fc700 source=Drop:57 tident=daemon.1879:473@junoeos06 sec=sss uid=2 gid=2 name=daemon geo="" drop request for &mgm.pcmd=drop&mgm.fsid=134&mgm.fid=00599923&mgm.localprefix=/data50/eos
error: received signal 11:
/lib64/libXrdEosMgm-4.so(_Z20xrdmgmofs_stacktracei+0x47)[0x7f09a8a151a7]
/lib64/libc.so.6(+0x36400)[0x7f09afdcc400]
/lib64/libc.so.6(+0x16f691)[0x7f09aff05691]
/lib64/libstdc++.so.6(_ZNSsaSEPKc+0x14)[0x7f09b073a384]
211206 10:23:03 time=1638757383.416374 func=Schedule2Balance level=INFO logid=FstOfsStorage unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f09161fc700 source=Schedule2Balance:346 tident=daemon.1879:473@junoeos06 sec=sss uid=2 gid=2 name=daemon geo="" cmd=schedule2balance fsid=146 freebytes=8743358001152 logid=FstOfsStorage
/lib64/libXrdEosMgm-4.so(_ZN9XrdMgmOfs3remEPKcR13XrdOucErrInfoPK12XrdSecEntityS1_+0x2cb)[0x7f09a8a5d5db]
/opt/eos/xrootd/lib64/libXrdServer.so.2(_ZN17XrdXrootdProtocol5do_RmEv+0x193)[0x7f09b12a1a73]
/opt/eos/xrootd/lib64/libXrdUtils.so.2(_ZN7XrdLink4DoItEv+0x19)[0x7f09b1015bb9]
/opt/eos/xrootd/lib64/libXrdUtils.so.2(_ZN12XrdScheduler3RunEv+0x17f)[0x7f09b1018f4f]
/opt/eos/xrootd/lib64/libXrdUtils.so.2(_Z15XrdStartWorkingPv+0x9)[0x7f09b1019099]
/opt/eos/xrootd/lib64/libXrdUtils.so.2(XrdSysThread_Xeq+0x37)[0x7f09b0fdeaa7]
#########################################################################
# stack trace exec=/opt/eos/xrootd/bin/xrootd pid=64019 what='thread apply all bt'
#########################################################################
211206 10:23:03 time=1638757383.431582 func=Schedule2Balance level=INFO logid=FstOfsStorage unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f09161fc700 source=Schedule2Balance:346 tident=daemon.1879:473@junoeos06 sec=sss uid=2 gid=2 name=daemon geo="" cmd=schedule2balance fsid=147 freebytes=8862631002112 logid=FstOfsStorage
211206 10:23:03 time=1638757383.432648 func=Schedule2Balance level=INFO logid=FstOfsStorage unit=mgm@junoeos01.ihep.ac.cn:1094 tid=00007f09161fc700 source=Schedule2Balance:346 tident=daemon.1879:473@junoeos06 sec=sss uid=2 gid=2 name=daemon geo="" cmd=schedule2balance fsid=127 freebytes=8235392634880 logid=FstOfsStorage
My eos instance version is 4.8.62 and gfal-copy version is 2.18.0.
Hi Xuantong,
I think the crash at the MGM is not related to the gfal-copy command. It seems the crash comes from a “rm” operation. You should get a detailed stacktrace after the messages that you pasted. If you can paste that one, we can more easily understand the root cause.
Thanks,
Elvin
Hi Xuantong,
I think this commit fixes the crash that you saw:
https://gitlab.cern.ch/dss/eos/-/commit/a7f3845c25ba5579e69b8ab13b4a65feaf486885
This was released in eos-4.8.63.
Cheers,
Elvin
Dear Elvin,
Thanks for your reply. I will try it when my eos instance upgrades to 4.8.63.
Regards,
Xuantong