Sunday, August 31, 2008

Using find to recursively unrar all files in subfolders

Sometimes when I've downloaded something, it is compressed into several small rar files, divided into lots of subfolders. This can be handy to keep track of files, but at the same time a pain in the ass when uncompressing them. You simply have to go into each subfolder and either run:
$ unrar x *01.rar
or using a gui-tool to unrar/unzip them. A little trick to make this a lot faster is to simply use the "find" command to find all the files you need, and then send them into whatever program you need to decompress them. You can use the same command to pipe files into playlists or even to compress files into several folders.

Now in this example we have a folder structure like the following:
$ ls -l
totalt 2,3G
foo.bar.v1.12.sourcecode/
foo.bar.v1.13.sourcecode/
foo.bar.v1.14.sourcecode/
foo.bar.v1.15.sourcecode/
foo.bar.v1.16.sourcecode/
foo.bar.v1.17.sourcecode/
foo.bar.v1.18.sourcecode/
foo.bar.v1.19.sourcecode/
foo.bar.v1.20.sourcecode/
foo.bar.v1.21.sourcecode/
foo.bar.v1.22.sourcecode/
foo.bar.v1.23.sourcecode/
foo.bar.v1.24.sourcecode/
First we start with the find command.
$ find ./software-foo*/
This will list all files in the folders. Each subdirectory in its turn contains:
$ ls -l
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part01.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part02.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part03.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part04.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part05.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part06.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part07.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part08.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part09.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part10.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part11.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part12.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part13.rar
To unrar, we need to define the first file (or any really, but just one) in each subdirectory. To show just the files we add the "-type f" flag. To then be able to parse the list, we also add a check for filenames. This is done with the "-name" flag.
$ find ./software-foo*/ -type f -name '*01.rar'
This gives us a list of the filenames that match the pattern *01.rar.
$ find ./software-foo*/ -type f -name '*01.rar'
./foo.bar.v1.12.sourcecode/foo.bar.v1.12.sourcecode.part01.rar
./foo.bar.v1.13.sourcecode/foo.bar.v1.13.sourcecode.part01.rar
./foo.bar.v1.14.sourcecode/foo.bar.v1.14.sourcecode.part01.rar
./foo.bar.v1.15.sourcecode/foo.bar.v1.15.sourcecode.part01.rar
./foo.bar.v1.16.sourcecode/foo.bar.v1.16.sourcecode.part01.rar
./foo.bar.v1.17.sourcecode/foo.bar.v1.17.sourcecode.part01.rar
./foo.bar.v1.18.sourcecode/foo.bar.v1.18.sourcecode.part01.rar
./foo.bar.v1.19.sourcecode/foo.bar.v1.19.sourcecode.part01.rar
./foo.bar.v1.20.sourcecode/foo.bar.v1.20.sourcecode.part01.rar
./foo.bar.v1.21.sourcecode/foo.bar.v1.21.sourcecode.part01.rar
./foo.bar.v1.22.sourcecode/foo.bar.v1.22.sourcecode.part01.rar
./foo.bar.v1.23.sourcecode/foo.bar.v1.23.sourcecode.part01.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part01.rar
So here is our list of files that we want to unrar. We then add the flag "-exec" to perform a command on the given files. The command we want is "unrar x", and then we also need to add some extra syntax to give the command its parameters.
$ find ./software-foo*/ -type f -name '*01.rar' -exec unrar x {} \;
Now we just wait! This decompresses all the files in all the subdirectories into the current directory. Perhaps not the best thing if it's hundreds of sourcefiles, so we have one more trick. Instead of using the "-exec" flag, we use the "-execdir" flag. Instead of performing the command from the directory where "find" was invoced, it performs it from each subdirectory. This sorts the files into subfolders for us!
$ find ./software-foo*/ -type f -name '*01.rar' -execdir unrar x {} \;
Hope this helps in the future! This is one reason why the command line is hard to replace fully in a GUI application.

9 comments:

Slinky said...

The "find" command was missing from the beginning of some of the commands, and here I was, scratching my head wondering what the heck is your shell which allows such syntax :)

Anonymous said...

Yes, that is confusing

# btw, are you running as root? :)

Peter said...

slinky: thanks, I huge miss. I fixed it now.

Hehe no I'm not running as root. Maybe $ would have been better ;)

Joe said...

I find myself always doing it like this...

find -name "*.rar" -print0 | xargs -0 -I{} unrar x {} /path/to/output

Anonymous said...

Hi,
Can someone help me, I tried that one:
find ./temp*/ -iname '*.rar' -execdir unrar e {} \;

but I always get the message:

BusyBox v1.1.0 (2008.11.10-03:26+0000) multi-call binary

Usage: find [PATH...] [EXPRESSION]


I am really getting crazy about that.
Could you please help me??

Best,
Lionel

Peter said...

Hi Lionel. I've never seen the busybox-message, but I checked at http://www.busybox.net/about.html and saw that it 'mimics' the GNU functionality but with smaller size.

I don't know if it is just a typo, but I don't use the -iname flag, but instead the -name flag. Could that be it?

A great way to find better ways of using 'find' is really to just type
$ man find
in you terminal! Hope it helps!

Anonymous said...

I find the following useful, paste it into a script in ~/.gnome2/nautilus-scripts

find . -name "*.rar" -exec unrar x {} \; | zenity --progress --pulsate --auto-close --title "Find & unrar" --text "Unpacking..."


Then you can recursively unrar files from within nautilus...

Species8472 said...

I made a script based on your post that unrars and deletes archive files if successful.

Posted it here:
http://blog.itsfortytwo.net/2011/01/auto-unrar-script/

Anonymous said...

thanks man, you are my hero <3