Is there a full python API for EOS?

1

Some background. As part of the migration to CTA for Fermilab, I need to be able to make placeholder files in EOS (create a file entry with checksum, etc, but with no contents since the contents are just on tape).

I have been using a tool from CTA (made for their migration from CASTOR) which, from what I can tell, uses the GRPC methods to do this. It would be much easier for me if I there was a python API which I could use to do this instead. Does such a thing exist or are there similar such examples?

Cheers,

Eric

2

One can compile the python api from the grpc interface using the rpc/proto files. I did this once, but we dont publish them since we never needed them.

3

Let me revisit this as this is still a problem for us. Is there a way to do what I need to do with EOS absent the RPC interface? Make a placeholder file with no actual content but all the metadata I might need?

I’ve looked through various parts of the eos CLI and I have not found anything that seems to be useful.

Any sort of simple example of this would be very helpful.

Cheers,

Eric

4

create a file without any replica and a given size by doing:

(mynewfile with 64kb):
eos file touch -n /eos/mynewfile 65535

create a layout with single copy and adler checksum

eos file layout /eos/mynewfile -type 00100012

tag the file to be on tape

eos file tag /eos/mynewfile +65535

It will then have if you do “eos file info /eos/mynewfile”
TapeID: undef StorageClass: none

You have to set this attributes doing:

set archive file id

eos attr set sys.archive.file_id=4406944319 /eos/mynewfile

set storage class

eos attr set sys.archive.storage_class=fnal_raw /eos/mynewfile

The missing piece is that:
I have to modify the touch comand to allow to set also a checksum when ‘-n’ is used. Can do that.

BUT, this does not allow to decide the EOS inode number of a file. If you need that, it cannot be done like that! I would crosscheck with the CTA team, if this procedure can work.

Cheers Andreas.

5

Thanks, this is very helpful. So just to make sure

file layout <path>|fid:<fid-dec>|fxid:<fid-hex>  -checksum <checksum-type> :
                                                  change the checksum-type of a file to <checksum-type>
file layout <path>|fid:<fid-dec>|fxid:<fid-hex>  -type <hex-layout-type> :
                                                  change the layout-type of a file to <hex-layout-type> (as shown by file info)

The first is the checksum type and the second is the value?

I don’t think I need to set the inode number and probably prefer not to.

6

Also, my eos doesn’t show (or seem to take a file size) on touch. I’m running version

EOS 4.8.62 (2020)

Is this something available in a later version?

7

Yes, for that you need a newer (latest) version, but anyway I will add the required ‘set checksum’ option and let you know when it is tagged.

8

The size does work on a the latest version (4.8.88). Is there an issue setup for adding the checksum so we can track it? I haven’t been able to find the EOS issue tracker.