Geobalancer gets permission denied

1

Hi,

I’m getting some errors in the logs about the geobalancer being denied access to a file:

Logs 
mgm-0 mgm 240424 18:15:24 time=1713982524.804070 func=DoIt                     level=DEBUG logid=static.............................. unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f86c7be570
0 source=ConversionJob:154              tident= sec=(null) uid=99 gid=99 name=- geo="" msg="starting conversion job" conversion_id=0000000000004b4f:default.0#00610002^geobalancer^
mgm-0 mgm 240424 18:15:24 time=1713982524.804492 func=DoIt                     level=INFO  logid=static.............................. unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f86c7be570
0 source=ConversionJob:238              tident= sec=(null) uid=99 gid=99 name=- geo="" [tpc]: root@mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094@root://mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094//eos/service/svc-
immich/thumbs/e8569b4d-8e2e-452a-ba6b-df9423819d68/60/66/60669374-de5c-40a0-ba1b-a266edc9901a.webp => root@mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094@root://mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094//eos//pro
c/conversion/0000000000004b4f:default.0#00610002^geobalancer^ prepare_msg=[SUCCESS]
mgm-0 mgm 240424 18:15:24 time=1713982524.804824 func=IdMap                    level=DEBUG logid=static.............................. unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f87497fb70
0 source=Mapping:181                    tident= sec=(null) uid=99 gid=99 name=- geo="" msg="XrdSecEntity client" name="(null)" role="(null)" group="(null)" tident="root.1:421@mgm-0" cred="none"
mgm-0 mgm 240424 18:15:24 time=1713982524.804916 func=IdMap                    level=DEBUG logid=static.............................. unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f87497fb70
0 source=Mapping:369                    tident= sec=(null) uid=99 gid=99 name=- geo="" swcuidtident=tident:"*@mgm-0":uid sprotuidtident=tident:"host@mgm-0":uid myrole=root
mgm-0 mgm 240424 18:15:24 time=1713982524.804934 func=IdMap                    level=DEBUG logid=static.............................. unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f87497fb70
0 source=Mapping:453                    tident= sec=(null) uid=99 gid=99 name=- geo="" tuid= tgid=
mgm-0 mgm 240424 18:15:24 time=1713982524.804940 func=IdMap                    level=DEBUG logid=static.............................. unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f87497fb70
0 source=Mapping:536                    tident= sec=(null) uid=99 gid=99 name=- geo="" suidtident:tident:"root@mgm-0":uid sgidtident:tident:"root@mgm-0":gid
mgm-0 mgm 240424 18:15:24 time=1713982524.804975 func=IdMap                    level=DEBUG logid=static.............................. unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f87497fb70
0 source=Mapping:996                    tident= sec=(null) uid=99 gid=99 name=- geo="" selected 99 99 [0 0]
mgm-0 mgm 240424 18:15:24 time=1713982524.805046 func=IdMap                    level=INFO  logid=static.............................. unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f87497fb70
0 source=Mapping:1001                   tident= sec=(null) uid=99 gid=99 name=- geo="" sec.prot=host sec.name="" sec.host="mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel" sec.vorg="" sec.grps="" sec.role="" sec.inf
o="" sec.app="geobalancer" sec.tident="root.1:421@mgm-0" vid.uid=99 vid.gid=99 sudo=0 gateway=0
mgm-0 mgm 240424 18:15:24 time=1713982524.805116 func=PathRemap                level=DEBUG logid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f87497fb70
0 source=PathMap:89                     tident=<single-exec> sec=      uid=0 gid=0 name= geo="" mappath=/eos/service/svc-immich/thumbs/e8569b4d-8e2e-452a-ba6b-df9423819d68/60/66/60669374-de5c-40a0-ba1b-a266edc9901
a.webp ndir=0 dirlevel=7
mgm-0 mgm 240424 18:15:24 time=1713982524.805135 func=open                     level=ERROR logid=9eed53bc-0266-11ef-be86-c6a626795ade unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f87497fb70
0 source=XrdMgmOfsFile:499              tident=root.1:421@mgm-0 sec=host  uid=99 gid=99 name= geo="" user access restricted - unauthorized identity vid.uid=99, vid.gid=99, vid.host="mgm-0.mgm.services-eos.svc.c.k3
s.fsn.lama.tel", vid.tident="root.1:421@mgm-0" for path="/eos/service/svc-immich/thumbs/e8569b4d-8e2e-452a-ba6b-df9423819d68/60/66/60669374-de5c-40a0-ba1b-a266edc9901a.webp" user@domain="nobody@mgm.services-eos.sv
c.c.k3s.fsn.lama.tel"
mgm-0 mgm 240424 18:15:24 time=1713982524.805181 func=Emsg                     level=ERROR logid=9eed53bc-0266-11ef-be86-c6a626795ade unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f87497fb70
0 source=XrdMgmOfsFile:3533             tident=root.1:421@mgm-0 sec=host  uid=99 gid=99 name= geo="" Unable to give access - user access restricted - unauthorized identity used ; Permission denied
mgm-0 mgm 240424 18:15:24 time=1713982524.805371 func=HandleError              level=ERROR logid=static.............................. unit=mgm@mgm-0.mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094 tid=00007f86c7be570
0 source=ConversionJob:379              tident= sec=(null) uid=99 gid=99 name=- geo="" msg="[ERROR] Server responded with an error: [3010] Unable to give access - user access restricted - unauthorized identity use
d ; Permission denied
mgm-0 mgm " tpc_src=root://mgm.services-eos.svc.c.k3s.fsn.lama.tel:1094//eos/service/svc-immich/thumbs/e8569b4d-8e2e-452a-ba6b-df9423819d68/60/66/60669374-de5c-40a0-ba1b-a266edc9901a.webp tpc_dst=root://mgm.servic
es-eos.svc.c.k3s.fsn.lama.tel:1094//eos//proc/conversion/0000000000004b4f:default.0#00610002^geobalancer^ conversion_id=0000000000004b4f:default.0#00610002^geobalancer^

I’m sure I’ve configured something wrong but I can’t figure out what.

eos space status default (only one space) 
balancer                         := on
balancer.node.ntx                := 2
balancer.node.rate               := 25
balancer.threshold               := 20
converter                        := on
converter.ntx                    := 2
drainer.node.nfs                 := 5
drainer.node.ntx                 := 2
drainer.node.rate                := 25
drainperiod                      := 86400
filearchivedgc                   := off
fsck_refresh_interval            := 7200
geobalancer                      := on
geobalancer.ntx                  := 1
geobalancer.threshold            := 5
graceperiod                      := 86400
groupbalancer                    := off
groupbalancer.engine             := std
groupbalancer.file_attempts      := 50
groupbalancer.max_file_size      := 16G
groupbalancer.max_threshold      := 0
groupbalancer.min_file_size      := 1G
groupbalancer.min_threshold      := 0
groupbalancer.ntx                := 10
groupbalancer.threshold          := 5
groupmod                         := 24
groupsize                        := 0
policy.blockchecksum             := crc32c
policy.blocksize                 := 64k
policy.layout                    := plain
policy.recycle                   := on
quota                            := off
scan_disk_interval               := 14400
scan_ns_interval                 := 259200
scan_ns_rate                     := 50
scan_rain_interval               := 2419200
scaninterval                     := 604800
scanrate                         := 100
taperestapi.stage                := off
taperestapi.status               := off
tgc.availbytes                   := 0
tgc.qryperiodsecs                := 320
tgc.totalbytes                   := 1000000000000000000
tracker                          := off
wfe                              := off
wfe.interval                     := 10
wfe.ntx                          := 1
eos vid ls 
krb5:"<pwd>":gid => root
krb5:"<pwd>":uid => root
publicaccesslevel: => 2
sss:"<pwd>":gid => root
sss:"<pwd>":uid => root
sudoer                 => uids(daemon)
tokensudo              => always
xrd.cf.mgm, only sec.* options 
sec.protocol host
sec.protocol unix
sec.protocol sss -c /etc/eos.keytab -s /etc/eos.keytab
sec.protocol krb5 /etc/eos.krb5.keytab eos/eos.lama-corp.space@LAMA-CORP.SPACE

sec.protbind localhost.localdomain host unix sss
sec.protbind localhost host unix sss
sec.protbind * only krb5 sss unix

And the EOS_MGM_URL env var is set to root://localhost on the mgm, but I’m not sure if that matters. I don’t have any other env var that looks suspicious, but I’ll paste them below anyway

EOS_UTF8=1
EOS_HTTP_THREADPOOL=epoll
EOS_FST_TRANSFER_THREAD_POOL=20
EOS_FST_S3_STORAGE_SIZE=20000000000
EOS_MGM_MASTER1=mgm.services-eos.svc.c.k3s.fsn.lama.tel
EOS_MGM_MASTER2=mgm.services-eos.svc.c.k3s.fsn.lama.tel
LD_LIBRARY_PATH=/opt/eos/xrootd/lib64:
EOS_FST_CONNECTION_RETRY=1
EOS_GEOTAG=fsn::nucleus
EOSREPODIR=/repo/eos
EOS_SKIP_SSS_HOSTNAME_MATCH=1
EOS_MGM_URL=root://localhost
EOS_MGM_GRPC_PORT=50051
EOS_FUSE_MGM_ALIAS=mgm.services-eos.svc.c.k3s.fsn.lama.tel
EOS_HTTP_THREADPOOL_SIZE=16
EOS_USE_SHARED_MUTEX=1
EOS_CONVERTER_DRIVER=1
EOS_MGM_ALIAS=mgm.services-eos.svc.c.k3s.fsn.lama.tel
EOS_FUSE_MOUNTDIR=/eos/
EOS_FST_DELETE_QUERY_INTERVAL=5
XRD_ROLES=mgm

Thanks a lot in advance for your help,

2

Hi Marc,

This is most likely due to the host authentication that you use in the /etc/xrd.cf.mgm in the sec.protbind for both localhost and localhost.localdomain. I would try at least to put host as the last option on that line, but if this still does not work just remove it. The host auth mechanism is not very secure, we never use it.

Cheers,
Elvin

3

Yes, that was it! I don’t know how I ended up putting it there, as it’s not present in the default config in the Docker image, nor in the eos-charts repo.

Thanks a lot for the assistance!