Using find to recursively unrar all files in subfolders

Sometimes when I've downloaded something, it is compressed into several small rar files, divided into lots of subfolders. This can be handy to keep track of files, but at the same time a pain in the ass when uncompressing them. You simply have to go into each subfolder and either run:

$ unrar x *01.rar

or using a gui-tool to unrar/unzip them. A little trick to make this a lot faster is to simply use the "find" command to find all the files you need, and then send them into whatever program you need to decompress them. You can use the same command to pipe files into playlists or even to compress files into several folders.

Now in this example we have a folder structure like the following:

$ ls -l
totalt 2,3G
foo.bar.v1.12.sourcecode/
foo.bar.v1.13.sourcecode/
foo.bar.v1.14.sourcecode/
foo.bar.v1.15.sourcecode/
foo.bar.v1.16.sourcecode/
foo.bar.v1.17.sourcecode/
foo.bar.v1.18.sourcecode/
foo.bar.v1.19.sourcecode/
foo.bar.v1.20.sourcecode/
foo.bar.v1.21.sourcecode/
foo.bar.v1.22.sourcecode/
foo.bar.v1.23.sourcecode/
foo.bar.v1.24.sourcecode/

First we start with the find command.

$ find ./software-foo*/

This will list all files in the folders. Each subdirectory in its turn contains:

$ ls -l
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part01.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part02.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part03.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part04.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part05.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part06.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part07.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part08.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part09.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part10.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part11.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part12.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part13.rar

To unrar, we need to define the first file (or any really, but just one) in each subdirectory. To show just the files we add the "-type f" flag. To then be able to parse the list, we also add a check for filenames. This is done with the "-name" flag.

$ find ./software-foo*/ -type f -name '*01.rar'

This gives us a list of the filenames that match the pattern *01.rar.

$ find ./software-foo*/ -type f -name '*01.rar'
./foo.bar.v1.12.sourcecode/foo.bar.v1.12.sourcecode.part01.rar
./foo.bar.v1.13.sourcecode/foo.bar.v1.13.sourcecode.part01.rar
./foo.bar.v1.14.sourcecode/foo.bar.v1.14.sourcecode.part01.rar
./foo.bar.v1.15.sourcecode/foo.bar.v1.15.sourcecode.part01.rar
./foo.bar.v1.16.sourcecode/foo.bar.v1.16.sourcecode.part01.rar
./foo.bar.v1.17.sourcecode/foo.bar.v1.17.sourcecode.part01.rar
./foo.bar.v1.18.sourcecode/foo.bar.v1.18.sourcecode.part01.rar
./foo.bar.v1.19.sourcecode/foo.bar.v1.19.sourcecode.part01.rar
./foo.bar.v1.20.sourcecode/foo.bar.v1.20.sourcecode.part01.rar
./foo.bar.v1.21.sourcecode/foo.bar.v1.21.sourcecode.part01.rar
./foo.bar.v1.22.sourcecode/foo.bar.v1.22.sourcecode.part01.rar
./foo.bar.v1.23.sourcecode/foo.bar.v1.23.sourcecode.part01.rar
./foo.bar.v1.24.sourcecode/foo.bar.v1.24.sourcecode.part01.rar

So here is our list of files that we want to unrar. We then add the flag "-exec" to perform a command on the given files. The command we want is "unrar x", and then we also need to add some extra syntax to give the command its parameters.

$ find ./software-foo*/ -type f -name '*01.rar' -exec unrar x {} \;

Now we just wait! This decompresses all the files in all the subdirectories into the current directory. Perhaps not the best thing if it's hundreds of sourcefiles, so we have one more trick. Instead of using the "-exec" flag, we use the "-execdir" flag. Instead of performing the command from the directory where "find" was invoced, it performs it from each subdirectory. This sorts the files into subfolders for us!

$ find ./software-foo*/ -type f -name '*01.rar' -execdir unrar x {} \;

Hope this helps in the future! This is one reason why the command line is hard to replace fully in a GUI application.

Using the PlayStation 3 as a media center with Linux

I recently got a PlayStation 3 from my wonderful girlfriend, and one of the reasons I wanted one was for its great possibilities of being a media center. I installed MediaTomb on my Linux machine (Gentoo) and after half an hour of configurations, it was ready to stream content to the PS3.

Installing MediaTomb
Portage is wonderful, and of course contains MediaTomb. It is an open source uPnP media server with DLNA support. DLNA is a protocol that computers and other devices can use to talk to each other, for instance when supplying video files over a network. It can do much more too, just have a look at the link to wikipedia for more info. When I did my install, I followed the guide at gentoo-wiki.com. To get more information (especially regarding transcoding), check the site there, this is my "shortcut" guide to it.

Install it using portage:

# emerge mediatomb

Make sure you have the use flags that you need enabled. I'm using

[I] net-misc/mediatomb
Available versions: 0.11.0 {curl debug exif expat ffmpeg javascript libextractor mysql taglib}
Installed versions: 0.11.0(22.28.13 2008-08-17)(curl exif expat ffmpeg javascript taglib -debug -libextractor -mysql)
Homepage: http://www.mediatomb.cc/
Description: MediaTomb is an open source UPnP MediaServer

and that works well for me. Since I rather use sqlite instead of a full mysql install, I haven't enabled that use flag.

When streaming to the PS3, you also need to enable some transcoding support. This also allows you to get thumbnails on the different video files. I use ffmpeg as my transcoder, since I already have it installed. Make sure you have the "encode" use flag enabled. If you need video thumbnails, also install ffmpegthumbnailer. support.

# emerge ffmgeg
# emerge ffmpegthumbnailer

Then you have to create 2 scripts to launch the transcoding for you. First the script for audio (/usr/bin/mediatomb-transcode-audio):

#!/bin/bash

FFMPEG_PATH="/usr/bin/ffmpeg"
INPUT="$1"
OUTPUT="$2"
VIDEO_CODEC="mpeg2video"
VIDEO_BITRATE="4096k"
VIDEO_FRAMERATE="25"
AUDIO_CODEC="mp2"
AUDIO_BITRATE="192k"
AUDIO_SAMPLERATE="44100"
AUDIO_CHANNELS="2"
FORMAT="dvd"

exec "${FFMPEG_PATH}" -i "${INPUT}" -vcodec ${VIDEO_CODEC} -b ${VIDEO_BITRATE} \
-r ${VIDEO_FRAMERATE} -acodec ${AUDIO_CODEC} -ab ${AUDIO_BITRATE} -ar ${AUDIO_SAMPLERATE} \
-ac ${AUDIO_CHANNELS} -f ${FORMAT} - > "${OUTPUT}" 2>/dev/null

and then the one for video (/usr/bin/mediatomb-transcode-video)

#!/bin/bash

FFMPEG_PATH="/usr/bin/ffmpeg"
INPUT="$1"
OUTPUT="$2"
VIDEO_CODEC="mpeg2video"
VIDEO_BITRATE="4096k"
VIDEO_FRAMERATE="25"
AUDIO_CODEC="mp2"
AUDIO_BITRATE="192k"
AUDIO_SAMPLERATE="44100"
AUDIO_CHANNELS="2"
FORMAT="dvd"

exec "${FFMPEG_PATH}" -i "${INPUT}" -vcodec ${VIDEO_CODEC} -b ${VIDEO_BITRATE} \
-r ${VIDEO_FRAMERATE} -acodec ${AUDIO_CODEC} -ab ${AUDIO_BITRATE} -ar ${AUDIO_SAMPLERATE} \
-ac ${AUDIO_CHANNELS} -f ${FORMAT} - > "${OUTPUT}" 2>/dev/null

We need to make them both executable:

# chmod +x /usr/bin/mediatomb-transcode-audio
# chmod +x /usr/bin/mediatomb-transcode-video

My config file now looks like the following. Copy it to /etc/mediatomb/config.xml and start MediaTomb.

<?xml version="1.0" encoding="UTF-8"?>
<config version="1" xmlns="http://mediatomb.cc/config/1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://mediatomb.cc/config/1
http://mediatomb.cc/config/1.xsd">
<server>
    <ui enabled="yes">
        <accounts enabled="no" session-timeout="30">
            <account user="mediatomb" password="mediatomb"/>
        </accounts>
    </ui>
    <name>MediaTomb</name>
    <udn>uuid:51b10cef-e678-41be-8a0c-6d53e5243b2d</udn>
    <home>/var/lib/mediatomb</home>
    <webroot>/usr/share/mediatomb/web</webroot>
    <storage>
        <sqlite3 enabled="yes">
            <database-file>mediatomb.db</database-file>
        </sqlite3>
        <mysql enabled="no">
            <host>localhost</host>
            <database>mediatomb</database>
            <username>mediatomb</username>
            <password>mediatomb</password>
        </mysql>
    </storage>
    <protocolInfo extend="yes"/>
</server>
<import hidden-files="no">
    <scripting script-charset="UTF-8">
        <common-script>/usr/share/mediatomb/js/common.js</common-script>
        <playlist-script>/usr/share/mediatomb/js/playlists.js</playlist-script>
        <virtual-layout type="builtin">
            <import-script>/usr/share/mediatomb/js/import.js</import-script>
        </virtual-layout>
    </scripting>
    <mappings>
        <extension-mimetype ignore-unknown="no">
            <map from="mp3" to="audio/mpeg"/>
            <map from="ogg" to="application/ogg"/>
            <map from="asf" to="video/x-ms-asf"/>
            <map from="asx" to="video/x-ms-asf"/>
            <map from="wma" to="audio/x-ms-wma"/>
            <map from="wax" to="audio/x-ms-wax"/>
            <map from="wmv" to="video/x-ms-wmv"/>
            <map from="wvx" to="video/x-ms-wvx"/>
            <map from="wm" to="video/x-ms-wm"/>
            <map from="wmx" to="video/x-ms-wmx"/>
            <map from="m3u" to="audio/x-mpegurl"/>
            <map from="pls" to="audio/x-scpls"/>
            <map from="flv" to="video/x-flv"/>
            <map from="avi" to="video/x-divx"/>
            <map from="divx" to="video/x-divx"/>
            <map from="vob" to="video/x-divx"/>
        </extension-mimetype>
        <mimetype-upnpclass>
            <map from="audio/*" to="object.item.audioItem.musicTrack"/>
            <map from="video/*" to="object.item.videoItem"/>
            <map from="image/*" to="object.item.imageItem"/>
        </mimetype-upnpclass>
        <mimetype-contenttype>
            <treat mimetype="audio/mpeg" as="mp3"/>
            <treat mimetype="application/ogg" as="ogg"/>
            <treat mimetype="audio/x-flac" as="flac"/>
            <treat mimetype="image/jpeg" as="jpg"/>
            <treat mimetype="audio/x-mpegurl" as="playlist"/>
            <treat mimetype="audio/x-scpls" as="playlist"/>
            <treat mimetype="audio/x-wav" as="pcm"/>
            <treat mimetype="audio/L16" as="pcm"/>
            <treat mimetype="video/x-msvideo" as="avi"/>
            <treat mimetype="video/divx" as="avi"/>
            <treat mimetype="video/vob" as="avi"/>
        </mimetype-contenttype>
    </mappings>
</import>
<transcoding enabled="yes">
    <mimetype-profile-mappings>
        <transcode mimetype="application/ogg" using="audio-common"/>
        <transcode mimetype="application/ogg" using="video-common"/>
        <transcode mimetype="audio/x-flac" using="audio-common"/>
        <transcode mimetype="video/x-flv" using="video-common"/>
        <transcode mimetype="video/divx" using="video-common"/>
        <transcode mimetype="video/divx" using="video-thumbnail"/>
    </mimetype-profile-mappings>
    <profiles>
        <profile name="audio-common" enabled="yes" type="external">
            <mimetype>audio/L16</mimetype>
            <accept-url>yes</accept-url>
            <first-resource>yes</first-resource>
            <hide-original-resource>yes</hide-original-resource>
            <accept-ogg-theora>no</accept-ogg-theora>
            <agent command="mediatomb-transcode-audio" arguments="%in %out"/>
            <buffer size="1048576" chunk-size="131072" fill-size="262144"/>
        </profile>
        <profile name="video-common" enabled="yes" type="external">
            <mimetype>video/mpeg</mimetype>
            <accept-url>yes</accept-url>
            <first-resource>yes</first-resource>
            <hide-original-resource>yes</hide-original-resource>
            <accept-ogg-theora>yes</accept-ogg-theora>
            <agent command="mediatomb-transcode-video" arguments="%in %out"/>
            <buffer size="10485760" chunk-size="262144" fill-size="524288"/>
        </profile>
        <profile name="video-thumbnail" enabled="yes" type="external">
            <mimetype>image/jpeg</mimetype>
            <accept-url>yes</accept-url>
            <thumbnail>yes</thumbnail>
            <resolution>128x128</resolution>
            <agent command="ffmpegthumbnailer" arguments="-i %in -o %out -s 128"/>
            <buffer size="524288" chunk-size="512" fill-size="1024"/>
        </profile>
    </profiles>
</transcoding>
</config>

Start it using

# /etc/init.d/mediatomb start

Adding files and directories
To add files to the MediaServer, simply point your favourite browser to http://192.168.0.157:49152/ and then press the "Filesystem" link to the left. There you can add the directories you want, and the server parses them for you. If you have inotify enabled in you kernel (which you probably do) it will be possible to let the server automatically know when something has been changed, and it reparses that directory when needed.

Mesa releases v.7.1

On the 26th of August, Mesa released the 7.1 version of their open-source implementation of the OpenGL library.

This is a development release, and warns that users focused on stability should go for the 7.0.4 release instead (or wait for 7.2). There are some interesting features added though, like DRI enhancements, "lots of DRI driver fixes", and GL_EXT_texture_from_pixmap in Xlib driver. They also mention the upcoming change to the GEM Memory Manager. The last is not in this release, but it is in the master branch of the GIT source tree, so I guess now we just wait...

The most interesting for me however is probably the support for the GL shading language on the i965 driver (Intel cards), this might actually mean that the integrated graphics card on my laptop finally supports shaders! Welcome to 2004! I bought the laptop with the intel card because of their great open source support, but was very (to say the least) surprised when it didn't have support of OpenGL2.0 or higher (or at least the shader side of it). Now my masters thesis might actually look a bit better, or at least as crappy as it looked on my desktop computer.

http://sourceforge.net/forum/forum.php?forum_id=860721 + http://mesa3d.sourceforge.net/