CERN Accelerating science

Group balancer and recycle bin

During some group balancing activity, I realized that the old files that get moved to another scheduling groups are put in recycle bin if the containing folder says so. Is it the desired behaviour ?

The recycle bin might get filled very quickly if a large amount of files are moved. We are planning to create new groups because we we add FSTs with more disks than the current ones. All these new groups will be empty, so they’ll pull many files, potentially with recycle bin policy. Is there a way to force deletion of files when group balancing ?

And when the files are removed from recycle bin after expiring delay, this might unbalance the groups and trigger group balancing again the other way. Did someone experience something like that ?

Hi Franck,
you are right. I think, that we should put files from conversion into the recycle bin. I can change it, anyone has another opinion to keep the original file before conversion in the bin?

HI Andreas,

We don’t use much the conversion, so I’m not sure I can give good advise, but it seems logical to not keep the converted file, as it would mean keep twice the same data, just the layout, or even only scheduling group, would differ.

The impact on operation depends much about the number of conversions operated on an instance. If very few, no problem, as the files will be removed from recycle bin after expiration. But when many conversions are done like in group balancing activity, it would fill up the recycle bin, which would cause issue to users removing files, and I suppose also to conversions themselves.
So at least a parameter that could be changed to inore the recycle bin policy for conversions (or only group balancing) would be nice.