Scheduling group is full?

Site Administrators
1

Dear Experts,

I wanted to add 8 new FST nodes and their corresponding filesystems to the existing setup. So far we already had 8 FSTs, with 24 disks on each node, with each filesystem belonging to a different group, something like this:

[root@eos-mgm1 ~]# eos group ls
┌──────────┬────────────────┬────────────┬──────┬────────────┬────────────┬────────────┬──────────┬──────────┐
│type      │            name│      status│ N(fs)│ dev(filled)│ avg(filled)│ sig(filled)│ balancing│   bal-shd│
└──────────┴────────────────┴────────────┴──────┴────────────┴────────────┴────────────┴──────────┴──────────┘
 groupview         default.0           on      9         0.42         0.43         0.17       idle          0 
 groupview         default.1           on      9         0.40         0.40         0.16       idle          0 
 groupview        default.10           on      9         0.40         0.40         0.15       idle          0 
 groupview        default.11           on      9         0.40         0.41         0.15       idle          0 
 groupview        default.12           on      9         0.41         0.42         0.16       idle          0 
 groupview        default.13           on      9         0.44         0.44         0.17       idle          0 
 groupview        default.14           on      9         0.42         0.42         0.16       idle          0 
 groupview        default.15           on      9         0.43         0.43         0.16       idle          0 
 groupview        default.16           on      9         0.42         0.42         0.17       idle          0 
 groupview        default.17           on      9         0.41         0.42         0.16       idle          0 
 groupview        default.18           on      9         0.41         0.42         0.17       idle          0 
 groupview        default.19           on      9         0.44         0.44         0.18       idle          0 
 groupview         default.2           on      9         0.44         0.44         0.16       idle          0 
 groupview        default.20           on      9         0.41         0.41         0.16       idle          0 
 groupview        default.21           on      9         0.40         0.41         0.17       idle          0 
 groupview        default.22           on      9         0.40         0.40         0.16       idle          0 
 groupview        default.23           on      9         0.41         0.42         0.16       idle          0 
 groupview         default.3           on      9         0.42         0.42         0.15       idle          0 
 groupview         default.4           on      9         0.41         0.41         0.16       idle          0 
 groupview         default.5           on      9         0.43         0.43         0.17       idle          0 
 groupview         default.6           on      9         0.41         0.41         0.15       idle          0 
 groupview         default.7           on      9         0.39         0.40         0.16       idle          0 
 groupview         default.8           on      9         0.41         0.41         0.18       idle          0 
 groupview         default.9           on      9         0.42         0.43         0.18       idle          0

This shows the filesystems of the 9th FST already added, therefore each group has 9 fs. Now, the problem is that from any other new FST node I receive such messages when I perform eosfstregister:

/data1 : uuid=7e0c5c36-6ec7-4f67-9964-0e3e1cf9dd7d fsid=undef
error: scheduling group default.0 is full
error: no group available for file syst
error: cannot boot filesystem - no filesystem with uuid=7e0c5c36-6ec7-4f67-9964-0e3e1cf9dd7d
...

Is this really a limit, i.e. do I have to define new groups?

Thanks,
Gabor

2

Hi Gabor,

Yes, look at the output of eos space ls and you will see the groupmod and groupsize values corresponding to each space. You can update them with the eos space define command.

Cheers,
Elvin

3

Hi Elvin!

Thanks, this is indeed what I was searching for.

Regarding the performance: is there any reason why I shouldn’t increase the groupsize to larger and larger values in the future with more FST nodes?

Cheers,
Gabor

4

Hi Gabor,

There is no performance impact if you have more file systems in a group. The groups are there just to separate the failure domains - that is why you should not have two file systems from the same machine in the same group. So in principle you can not have more file systems inside a group than you have machines in your cluster.
Hope this helps!

Cheers,
Elvin