Variables in /etc/sysconfig/eos_env : usage, requirement, etc

Site Administrators
1

Hello,
I have questions on the /etc/sysconfig/eos_env content.

a) EOS_MGM_HOST and EOS_MGM_HOST_TARGET:
I have no doubts on EOS_MGM_HOST but I am not sure EOS_MGM_HOST_TARGET is still required when using QuarkDB. I think it is only used by the sync service that we do not use in this case. Correct ?

b) EOS_MGM_MASTER1 and EOS_MGM_MASTER2:
When using QuarkDB, the number of MGM can be greater that 2. What is the use of these 2 variables ?

c) EOS_MGM_ALIAS: I think this is required, at least for FST, but in the case we do not have a DNS alias for several MGMs, how this variable should be defined ?

d) EOS_QUARKDB_PASSWD and EOS_QUARKDB_HOSTPORT:
These variables are used to configure the EOS xrootd quarkdb service when it is not provided by the standard xrootd, but will the be used by the MQ service if they are defined in /etc/sysconfig/eos_env ?

Thank you

JM

2

Hi,

After we moved to QDB we were able to remove the EOS_MGM_HOST and EOS_MGM_HOST_TARGET variables - as you said, they are used for the sync which is not necessary any more.

We also dropped the MASTER 1/MASTER2 variables as QDB takes care of that as well.

Not sure about the others, as we do have a DNS alias for the MGM, and we do the qdb cluster config for mq/mgm/fst without those environment variables within our docker containers, but hope that helps a little.

3

Hi Crystal,
Thank you very much, yes it does help.
JM

4

Dear Crystal and Jean M Barbet,

Thank for your information and knowledge.

There are some question before move to QDB.

  1. Earlier we use same host name certificate in EOS Master as well as EOS slave also. So, what we do in case of QDB? Should we use same host certificate for all 3 QDB nodes or different ?

  2. The variable i.e. EOS_MGM_ALIAS and BROKER URL at eos_env for MASTER/SLAVE is point to alias name (define in DNS) which is different then MASTER and SLAVE hostname.
    What we do in case of QDB?

Thanks

Prasun

5

Hi Prasun,

  1. I think that host certificates are not involved in communications between the MGMs and the quarkdb hosts.

  2. similarly, I think that the EOS_MGM_ALIAS and BROKER URL are not used for quarkdb,

You need to:

a) set xrootd@quarkdb config in /etc/xrootd/xrootd-quarkdb.cfg (EPEL’s xrootd different from xrootd used by EOS), this is for quarkdb itself (this is where you define a password for accessing quarkdb)

b) tell the MGM about quarkdb in /etc/xrd.cf.mgm

[...]
#############################################################
# Ajouts pour quarkdb
# (https://eos-community.web.cern.ch/t/config-in-quarkdb-for-master-slave-s/322
mgmofs.qdbpassword xxxxxlongquarkdbpasswordxxxxxxx
mgmofs.qdbcluster naneosmgr01.in2p3.fr:7777 naneosmgr02.in2p3.fr:7777 naneosmgr03.in2p3.fr:7777
[...]

c) The FSTs also need to know about quarkdb in /etc/xrd.cf.fst

[…]
fstofs.qdbpassword xxxxxlongquarkdbpasswordxxxxxxx
fstofs.qdbcluster naneosmgr01.in2p3.fr:7777 naneosmgr02.in2p3.fr:7777 naneosmgr
03.in2p3.fr:7777
[…]

It happens that the quarkdb service is running on the same hosts as the EOS manager, hence the same hostnames, but it could be separated.

I hope these tips are correct, this is how it is in our cluster.

JM

6

Dear Jean,

Thanks for your tips and help.

We have created quarkdb cluster in 3 identical machines with quarkdb packages. Also, we install eos packages in those 3 machines.

Kolkata QDB cluster information:-

=============
127.0.0.1:7777> raft_info

  1. TERM 5559
  2. LOG-START 0
  3. LOG-SIZE 14
  4. LEADER eos-mgm.tier2-kol.res.in:7777
  5. CLUSTER-ID 2a5ac5bd-628d-41ea-977a-c164ee9fc057
  6. COMMIT-INDEX 13
  7. LAST-APPLIED 13
  8. BLOCKED-WRITES 0
  9. LAST-STATE-CHANGE 76822 (21 hours, 20 minutes, 22 seconds)
  11. MYSELF eos-mgm.tier2-kol.res.in:7777
  12. VERSION 0.4.2
  13. STATUS LEADER
  14. NODE-HEALTH GREEN
  15. JOURNAL-FSYNC-POLICY sync-important-updates
  17. MEMBERSHIP-EPOCH 0
  18. NODES eos-mgm.tier2-kol.res.in:7777,eos-slave.tier2-kol.res.in:7777,eos-qdb.tier2-kol.res.in:7777
  19. OBSERVERS
  20. QUORUM-SIZE 2
  22. REPLICA eos-qdb.tier2-kol.res.in:7777 | ONLINE | UP-TO-DATE | NEXT-INDEX 14 | VERSION 0.4.2
  23. REPLICA eos-slave.tier2-kol.res.in:7777 | ONLINE | UP-TO-DATE | NEXT-INDEX 14 | VERSION 0.4.2
    127.0.0.1:7777>
    =================

As per your reply, we had added entry of quarkdb servers nodes in /etc/xrd.cf.mgm for all 3 quarkdb nodes (Clause b) and /etc/xrd.cf.fst ( Clause c).

But how we define quarkdb and EOS parameters in eos_env and eos for MGM-quarkdb and FSTs?
If possible, share eos_env and eos of mgm and fst also.

For Master-Slave configuration, do we need to install eos packages in all 3 nodes? If yes, How we define Master and Slave machines for eos cluster?

Please help.

Regards
Prasun