Fastest way to run file layout -stripes 2 on lots of files

1

Hi All,

Is there a quick (and nice) way to run file layout "${path}" -stripes 2 on a directory recursively and/or on the whole EOS namespace?

All I can think of is crawling the output of find -f --stripediff over the entire namespace and listing each file.

Cheers
Michael

2

Running file layout over 18912 files took around 7 hours, single thread.

3

Do you run this command on the MGM?

time eos file layout /eos/ajp/passwd -stripes 2
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd

real 0m0.037s
user 0m0.014s
sys 0m0.021s

You can write all this comands into a file /tmp/script.eosh
time eos -r 0 0 /tmp/script.eosh
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/ajp/passwd

real 0m0.054s
user 0m0.024s
sys 0m0.023s

In my example the script looks like:
file layout /eos/ajp/passwd -stripes 2
file layout /eos/ajp/passwd -stripes 2
file layout /eos/ajp/passwd -stripes 2
file layout /eos/ajp/passwd -stripes 2
file layout /eos/ajp/passwd -stripes 2
file layout /eos/ajp/passwd -stripes 2
file layout /eos/ajp/passwd -stripes 2
file layout /eos/ajp/passwd -stripes 2
file layout /eos/ajp/passwd -stripes 2
file layout /eos/ajp/passwd -stripes 2

4

Hi, I run the command from inside the MGM docker container.

time eos file layout /eos/a/b/c/d/e/f/g/file -stripes 2
success: setting layout to replica:2[2]:adler:none[4k] for path=/eos/a/b/c/d/e/f/g/file

real 0m2.159s
user 0m0.021s
sys 0m0.016s

Ours seems slower, also getting into the eos console takes over 1s, might be related to that

I will try my next run using an .eosh file

5

Hnm, you should figure out, why this is so extremely slow … that is not normal …