CHPC to Remove Old User Files from Lustre from 24 Jul 2023
14 Jul 2023
Dear CHPC Users
The CHPC Lustre storage is currently at 76% capacity use. Unfortunately the removal of old and unused data by users have not led to a satisfactory reduction in capacity usage of the Lustre parallel file system. The CHPC will proceed with the removal of all old and unused files contained in individual user directories from Monday, 24 Jul 2023. This deletion does not apply to shared directories.
Users are reminded that the CHPC’s user policy states that data older than 90 days may be deleted without warning. Starting on 24 Jul 2023, the CHPC will remove all user files on Lustre that have not been accessed or used in the past 180 days.
It is important to note that the process deals with individual files, not directories or complete data sets. In other words, if you have a data set, consisting of 100 files, of which 25 have not been accessed (read or write) in the preceding 180 days, those 25 older files will be deleted. The process of file removal cannot be reversed – there is no undelete. The process will not delete directories, but all old files in all directories will be deleted. To get a list of your files that will be targeted, use the lfs find command. Assuming that your username is jblogs, run the following command:
lfs find /mnt/lustre/users/jblogs –atime +180 –mtime +180 –type f –print
This will produce a list of your old files. You may re-direct this info to file :
lfs find /mnt/lustre/users/jblogs –atime +180 –mtime +180 –type f –print > myfiles_in_danger_of_deletion.txt
Please take the appropriate action before 24 July to move important old files to a safe long-term storage, bearing in mind that the CHPC’s lustre was never intended to function as long-term storage.
I thank you for your understanding that managing the capacity usage of the lustre parallel file system is essential to ensure continued uninterrupted usage of the CHPC Lengau compute cluster by all users. If you require any assistance with the management of your lustre data, please submit a query to CHPC Helpdesk here:
If you know of users that are not receiving communications from the CHPC user mailing list, please assist the CHPC by requesting those users to SUBSCRIBE here as this is the official communication channel with users from the CHPC:
https://lists.chpc.ac.za/mailman/listinfo/chpc-users
Kind Regards
Werner