How to delete millions of files in a directory quickly?

Moderator: NorbertKrupa

Post Reply
harryrundles
Intermediate
Intermediate
Posts: 96
Joined: Thu Jul 19, 2012 12:33 am

How to delete millions of files in a directory quickly?

Post by harryrundles » Tue Nov 26, 2013 10:04 pm

Hello,

Does anyone have a recommendation on how to delete millions of files in a directory and in sub folders under that directory?

I tried using the rm -rf command but it takes FOREVER!!!
Thanks,
Harry

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1825
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: How to delete millions of files in a directory quickly?

Post by JimKnicely » Wed Nov 27, 2013 3:07 am

Hi Harry,

I use the rsync trick ...

1. Create an empty directory
2. Use the rsync command in conjunction with the --delete option to synchronize the empty directory with the directory you want to empty ...

Example:

Code: Select all

[dbadmin@vertica02 vertica_backups]$ pwd
/home/dbadmin/vertica_backups
[dbadmin@vertica02 vertica_backups]$ mkdir blank
[dbadmin@vertica02 vertica_backups]$ ls -lrt blank
total 0
[dbadmin@vertica02 vertica_backups]$ rsync -a --delete /home/dbadmin/vertica_backups/blank/ /home/dbadmin/vertica_backups/v_snowfall_node0001/
I learned this neat trick from here:

http://www.slashroot.in/which-is-the-fa ... s-in-linux
Jim Knicely

Image

Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.

Post Reply

Return to “Vertica and the Operating System”