3 MGM configuration

Site Administrators
1

Dear all
please could you have an example of 3 MGM configurations?
thank you
e.v.

2

I just following this page to setup 3 (MGM+QuarkDB) configuration.

The quarkdb xrootd config file /etc/xrootd/xrootd-quarkdb.cfg on each MGM node

xrd.port 7777
xrd.protocol redis:7777 libXrdQuarkDB.so
redis.mode raft
redis.database /var/lib/quarkdb/node-1
redis.myself <hostname>:7777

/etc/systemd/system/xrootd@quarkdb.service.d/override.conf

[Service]
User=daemon
Group=daemon

And run the quarkdb service

mkdir /var/lib/quarkdb/
quarkdb-create --path /var/lib/quarkdb/node-1 --clusterID 001234 --nodes <mgm1>:7777,<mgm2>:7777,<mgm3>:7777

chown -R daemon:daemon /var/lib/quarkdb/node-1
systemctl start xrootd@quarkdb

I add the following lines to xrd.cf.mgm on each MGM node

mgmofs.cfgtype quarkdb
mgmofs.qdbcluster <mgm1>:7777 <mgm2>:7777 <mgm3>:7777
mgmofs.qdbpassword_file /etc/eos.keytab
3

thank Chien-De Li for you reply

I mean with 3 mgm ( metadata servers) and not just the 3 quarkdb servers

I think at CERN they have this configuration
or just we can have 2 mgm ( one master and one slave) ?
FYI
best
e.v.

4

Hi,

I just found this thread and would like to leave a comment.

We are running 3 MGMs and they can cover one another.
The only thing we did was to add the following configuration in eos sysconfig file (e.g. /etc/sysconfig/eos)

MGM #1
export EOS_MGM_MASTER1=<MGM #1>.domain
export EOS_MGM_MASTER2=<MGM #2>.domain
MGM #2
export EOS_MGM_MASTER1=<MGM #1>.domain
export EOS_MGM_MASTER2=<MGM #2>.domain
MGM #3
export EOS_MGM_MASTER1=<MGM #1>.domain
export EOS_MGM_MASTER2=<MGM #3>.domain

Basically, MGM #1 is the primary and the two are secondaries. However once MGM #1 lost control for some reason, then anyone of two will take over the master role randomly I presume.

The main configuration of MGM (e.g. /etc/xrd.cf.mgm) should be identical for the three MGMs.

And I think it would be better to have a alias such that a round-robin DNS name points to the three of MGMs. We had tested the access through DNS name and the requests arrived to secondaries are well redirected to FSTs via the primary MGM.

FYI, we are running with QuarkDB clusters (3 nodes in raft mode) and the eos version is 4.8.82 (it also worked with 4.8.31).

Best regards,
Sang-Un