Eos 5.1.8 and occ.asize bug fix on http-tpc

hello Elvin and Adreas
I test again 5.1.8 eos version with xrootd 5.5.1 which includes the bugfix with occ.size
does not look like it works
with source DPM or DCACHE and eos as a destination for TPC pull
the xfs preallocation is still there and the file size is greater than the apparent size.

when I made eos to eos, the eos preserve the file size ( if has preallocation copy the full file) if not do not add preallocation and copy the apparent size.

When I upload a file from a UI to eos , also we do not have any preallocation ( but this was also true for 5.0.18 ver)
when I download a file from eos (5.1.8) with preallocation, at UI I have the correct size without preallocation

any idea?
what kind of log you would like to send?
best regards
e.v.
p.s.
the test system is under https://grid23.lal.in2p3.fr:11000/eos/grif/dteam
test took place with gfal-copy --copy-mode pull $SRC $DST
for push mode looks that we do ot have this issue for old 5.0.18 and new version (5.1.8)

Hi Emmanouil,

Sorry, I can’t follow very well your problem as there are many things mixed up in your request. Let’s start with some basic tests, following some of you comments:

  1. When I upload a file from a UI to eos , also we do not have any preallocation
    Can you give an example of what this means? How are you uploading? Can you append the logs of such a transfer or send them to me over email? Then we can understand exactly what information is passed to the FST and we can take it from there. I am interested in the logs from the first FST which is contacted by the client.

  2. when I download a file from eos (5.1.8) with preallocation, at UI I have the correct size without preallocation
    What do you mean download with pre-allocation? I don’t follow this at all … maybe I am missing what UI means in this context.

  3. with source DPM or DCACHE and eos as a destination for TPC pull
    Can you again send the logs from the entry FST which is doing to pull so that we understand exactly if the oss.asize info is properly passed?

Thanks,
Elvin

Hello Elvin

according to your question I have the following simple tests , the size of initial file is 41943040 bytes (ag.img)

  1. gfal-copy ag.img https://grid23.lal.in2p3.fr:11000/eos/grif/dteam/upload.gfal
    in this case the file size is 41943040 bytes on FST filesystem

  2. gfal-copy https://grid23.lal.in2p3.fr:11000/eos/grif/dteam/upload.gfal download.gfal
    in this case the file size of download.gfal i 41943040 bytes, on the machine which I run gfal-copy e.g. a user interface UI

  3. gfal-copy --copy-mode pull https://grid05.lal.in2p3.fr/dpm/lal.in2p3.fr/home/dteam/ag.img https://grid23.lal.in2p3.fr:11000/eos/grif/dteam/ag.gfal.tpc
    in this case the file size of ag.gfal.tpc 65994752 bytes , on FST filesystem due to preallocation of XFS

  4. gfal-copy https://grid23.lal.in2p3.fr:11000/eos/grif/dteam/ag.gfal.tpc download.gfal.tpc

in this case the file size of download.gfal.tpc 41943040 bytes ( on the machine which I run gfal-copy )

please find the logs of fsts for each case on the tar file in attachments in a private email

thank you in advance

best
e.v.

Hi Emmanouil,

Thanks. I replied over email.

Cheers,
Elvin