Scale out of EOS FST with existing and new rain6 config

1

We are currently running a test bed setup of 4 fst with mgm, qdb and fst. We had make a two rain6 group i.e. default within 12 disks (9 + 2 +1) and it’s run perfectly.
Attributes configuration for /eos/alicekolkata/grid (9+2+1) as below:-
[root@node-01 ~]# eos attr ls /eos/gridlab/grid
sys.forced.blockchecksum=“crc32c”
sys.forced.blocksize=“1M”
sys.forced.checksum=“adler”
sys.forced.layout=“raid6”
sys.forced.nstripes=“11”
sys.forced.space=“default”
[root@node-01 ~]#
[root@node-01 ~]# eos space ls
┌──────────┬────────────────┬────────────┬────────────┬──────┬─────────┬───────────────┬──────────────┬─────────────┬─────────────┬──────────────┬──────┬──────┬──────────┬───────────┬───────────┬──────┬────────┬───────────┐
│type │ name│ groupsize│ groupmod│ N(fs)│ N(fs-rw)│ sum(usedbytes)│ sum(capacity)│ capacity(rw)│ nom.capacity│sched.capacity│ usage│ quota│ balancing│ threshold│ wfe│ ntx│ active│ intergroup│
└──────────┴────────────────┴────────────┴────────────┴──────┴─────────┴───────────────┴──────────────┴─────────────┴─────────────┴──────────────┴──────┴──────┴──────────┴───────────┴───────────┴──────┴────────┴───────────┘
spaceview default 13 24 24 23 4.95 TB 276.06 TB 275.95 TB 0 B 271.01 TB 1.79 off on 0 off 1 0 on

Now, we planning to scale out the fst with another raid6 group with 12 disks (9 + 2 +1).
As per our plan, we had define another group vecc with attributes for 8 disks (5 + 2 + 1) with same fsts as below:-
[root@node-01 ~]# eos attr ls /eos/gridlab/kolkata
sys.eos.btime=“1787891468.28696031”
sys.forced.blockchecksum=“crc32c”
sys.forced.blocksize=“1M”
sys.forced.checksum=“adler”
sys.forced.layout=“raid6”
sys.forced.nstripes=“7”
sys.forced.space=“vecc”
[root@node-01 ~]#

But on output of “eos space ls” is shows 2 different groups.
[root@node-01 ~]# eos space ls
┌──────────┬────────────────┬────────────┬────────────┬──────┬─────────┬───────────────┬──────────────┬─────────────┬─────────────┬──────────────┬──────┬──────┬──────────┬───────────┬───────────┬──────┬────────┬───────────┐
│type │ name│ groupsize│ groupmod│ N(fs)│ N(fs-rw)│ sum(usedbytes)│ sum(capacity)│ capacity(rw)│ nom.capacity│sched.capacity│ usage│ quota│ balancing│ threshold│ wfe│ ntx│ active│ intergroup│
└──────────┴────────────────┴────────────┴────────────┴──────┴─────────┴───────────────┴──────────────┴─────────────┴─────────────┴──────────────┴──────┴──────┴──────────┴───────────┴───────────┴──────┴────────┴───────────┘
spaceview default 13 24 24 22 4.95 TB 276.06 TB 263.96 TB 0 B 259.17 TB 1.88 off on 0 off 1 0 on
spaceview vecc 8 10 8 8 682.64 GB 95.98 TB 95.98 TB 0 B 95.30 TB 0.71 off on 0 off 1 0 off

[root@node-01 ~]#

As per above "eos space ls ", disk space are divided into two groups.

So, here I want to know, how can we will add two different group in same eos namespace for scale out ??

2

Dear Peter and Elvin,

In the existing Kolkata::EOS namespace we have RAIN-6 (5+2) configuration and each group is under Default “sys.forced.space” attribute with sys.forced.nstripes=“7”. Also in each group added one FS as hot-spare so present configuration is 5 (data) + 2 (parity) + 1 (spare).

Now we want to expand our Kolkata::EOS by adding more storage nodes under same name space (Kolkata::EOS). For expansion, we want each group = 9 (data) + 2 (parity) + 1 (spare) (different nstripes than earlier). But as mentioned by @prasun we could not add it in the same "sys.forced.space. We endup with different "sys.forced.space=“vecc” attribute name. It resulted that “eos space ls” shows two different spaces.

Kindly guide how to scale present RAIN-6 enabled EOS Namespace with different number of File System (FS) in each group for additional nodes.

Regards,

Kolkata Team

3

Dear Vikas, Prasun,

I guess what you mean is that you want just one space “default” with multiple groups 0, 1, … For this you don’t need to create a new space, just move the newly added file systems from “spare” space which should be the first entry point when you add a new file system to the desired group.

Example: eos fs mv 10 default.1

This moved file system 10 to group default.1. I see that you already have the groupsize=13 and groupmod=24 for the default space. So as long as you are adding new file systems to the “default” space they should be nicely included either in existing groups if the necessary conditions are met or into new groups eg. default.3, default.4.

Moreover the “sys.forced.space” attribute of the directories stays always the same, namely “default” space. The scheduler will take care of selecting the proper group from that space later on when a file placement is actually requested.

You can see the existing groups in the default space by doing “eos groups ls default”. Make sure that the file systems are empty when moving them around between groups as this can be dangerous if they hold data!

Let me know if this solves your issue.

Thanks,
Elvin