Federated or tree structured EOS instances

General Discussion
1

I’ve been looking into how to create a federated or tree structured EOS setup, much like how CERN have deployed their existing namespaces.

There’s no documentation from what I can find. Does anyone know of current instructions for how to join namespaces so we can browse up and down the tree from /eos down to all the instances?

It’d be excellent if it were possible to do this via xrootd or HTTP, rather than just using eosd sub-mounts.

2

Hi David,
there are two things you have to do to split instances into pieces:

Deploy a ‘router’ front-end for XRootd & HTTP traffic.

You can setup a front-end EOS server(s), where you configure routing the MGM for xrootd and http traffic.

[root@ajp console]# eos route -h 
Usage: route [ls|link|unlink]
    namespace routing to redirect clients to external instances

  route ls [<path>]
    list all routings or the one matching for the given path

  route link <path> <dst_host>[:<xrd_port>[:<http_port>]],...
    create routing from <path> to destination host. If the xrd_port
    is ommited the default 1094 is used, if the http_port is ommited
    the default 8000 is used. Several dst_hosts can be specified by
    separating them with ",". The redirection will go to the MGM
    from the specified list
    e.g route /eos/dummy/ foo.bar:1094:8000

  route unlink <path>
    remove routing matching path

For FUSE access we use AUTOFS to map paths to instances. The mounts cannot use the front-end router service, they need to talk directly to instances.

As an example you deploy:

(empty MGM): eosentry.foo
(MGM1): eosbranch1.foo
(MGM2): eosbranch2.foo

Now you define routing like this:

/eos/tree1/ => eosbranch1
/eos/tree2/ => eosbranch2

All Xrootd & Http clients talk always to eosentry.foo

The mounts have to be mounted as the routing table.

The mounts have to be configured like the routing settings. One can actually use the eos route ls command to derive the autofs mount table.

At CERN we are experimenting additionally with bind mounts to avoid to have too many mount points.

E.g. we mount 5 eos instances:

/eos/instance1
/eos/instance2
/eos/instance3/
/eos/intsance4/

and then we define bind mounts on top of them to get more finegrained assignements like

/eos/letter/a => /eos/instance1/
/eos/letter/b => /eos/instance2/

This is not as flexible as splitting the MGM tree internally, but it creates 100% independent trees, which are not sharing the same point of failures.

Cheers Andreas.

3

Thank’s @apeters, that’s similar to I had been thinking based on prior conversations.

Does the FUSEX client follow routes in EOS, or do you have to solve it the same way as for the older eosd fuse mount?

4

Further questions having tested further.

Curiously, eosd allowed me to browse the routed path (although I didn’t test read/write), but I couldn’t follow it in the eos CLI. Is this the expected behaviour?

Additionally, I couldn’t see the route path it if I didn’t have the directory created in /eos. Do I need to create an empty directory in the namespace for the routed path for it to be visible?

I could change into the routed path, it just wasn’t visible in a ls/readdir().

5

eosd will allow that, but every call will go via a redirection and it will be slowish …

As an example in the frontend you create this path

/eos/aarnet/a
and you redirect
eos/aarnet/a => eosaarnet-a:1094:8000

You can browse
/eos/
/eos/aarnet/
(served from the front-end)
and
/eos/aarnet/a/
(served from the target eosaarnet-a)

If you don’t create /eos/aarnet/a in the front-end it is not visable but usable via xrootd and http.

Cheers Andreas.