iTunes dupe.blaster

December 31st, 2005 Leave a comment Go to comments

If you find this program useful, please send me an email with feedback or just saying it worked for you! Email address found on left navbar at bottom. If you really REALLY find it useful, you could donate this needy student (me) some cash using PayPal (safe and secure) to put toward my next term's textbooks or whatever :)


NOTE - iTunes DupeBlaster will work with any music program: winamp, media player, iTunes -- because it works on the files themselves, not with the player. It is named iTunes DupeBlaster because iTunes was what inspired me to write it.

iTunes is a fantastic program with many wonderful features. At the moment (iTunes 4.7) one feature that's missing is good dupe-detection -- you know what i'm talking about, how come you have a directory with an album of mp3s, but one of the songs is there twice with a " 1" at the end? or a different file extension? or ___ or ___ or... the list goes on. In any case, there aren't that many solutions to the problem that don't either require way too much manual work, screw up your filesystem, require crazy things like perl or python or ruby or worse, Mac OS X!

Enter iTunes DupeBlaster. This is a beautiful ~500-line (big!) perl script, but before you call me a hypocrite all you Windows users will enjoy the .exe executable that was made for you (hint: Windows users, double-click the iTunesDupeBlaster.exe file). Linux users already have perl installed, so no worries there - it uses only modules that come with perl. So why should you use iT DB? Because I spent 3 full 8-hour days designing, re-designing, implementing and starting from scratch until I had something perfect, flawless and useful. I promise this'll be handy if you use iTunes!

What iTunes DupeBlaster DOESN'T do:
  • automatically delete files -- it makes it easy to delete files, but nothing is ever deleted without you expressly saying so
  • require a specific O/S like Windows, Mac OS X, Linux, etc -- it'll run on anything that can execute a perl script
  • try to scam money off you -- this is freeware, no nags / crippled features. If you really like it, hit that donate button...
  • take a long time to run -- on normal scanning mode iTunes DupeBlaster can fly through my 80 gigabyte collection in under a minute!
  • take lots of RAM while it's running -- on my 80 gigabyte collection less than 1.0% of my 512MB of RAM was used
  • get slower if you're scanning a "big" stash of music -- I scanned 80GB with no issues, everything is implemented with the efficiency of perl
  • anything with ID3-tags -- iTunes DupeBlaster doesn't read ID3 tags or use any iTunes metadata (XML file etc). It relies on only filenames and filesizes. Future versions may have ID3-tag stuff, but most people have well-tagged collections anyways (iTunes is good at that)

What iTunes DupeBlaster DOES do:
  • intelligent dupe-checking -- looks for any types of dupes that are common when using iTunes, including various filename and filesize duplications
  • superb console-based user interface -- intelligent groupings, cmd-line parameters, interactive or list mode, multiple ops quickly, and MUCH more! I spent many hours optimizing the user interface so that it'd be not only easy to use, but fast for repeat-users
  • provide you with source-code -- as if i'd release something that wasn't open-source! I keep my code pretty damn clean, so feel free to poke around, make improvements (i'm no perlhacker)
  • run risk-free -- there's no way you'll delete files accidentally

Anyways, give it a go and let me know how it works for you! Here's to hoping it wasn't a waste of 3 days :) See below for the output from my 80GB dupe-blasting session!



h o w   t o   u s e   : :

    You should put this program in the directory which contains your music, and run it from there. It'll recurse into all directories beneath wherever you start it from.

    To find out what the cmd-line options are, type (Windows) iTunesDupeBlaster.exe --help or (Linux/UNIX) ./iTunesDupeBlaster.pl --help. Note that the options are all explained in the help, and you can always ask me questions via email (see left navbar).



d o w n l o a d s   : :


s a m p l e   o u t p u t   : :

    This is output from 1.0.0 (old). Here's what I got when I ran ./iTunesDupeBlaster.pl --list (list-mode) on my iTunes repository of music.




iTunes DupeBlaster version 1.0.0, by Tim Carr

Options being used:
no --m4aisCommon - you have more mp3 files than m4a files 
	(affects only dupe-listing order)
no --dontDoSizeDupes - considering files of same size to be duplicates
no --sizeAllDir - checking for files of same size in same directory only
no --nameAllDir - checking for files of similar name in same directory only
--list - interactive mode turned off, just listing duplicates (no deletions)

Pass 1 (long), finding name dupes.
. = (sub)directory processed
! = name dupe found

................!........... (SNIP)

Pass 2 (short), finding size dupes... done

NOTE: ANY FILES YOU DELETE WILL BE REALLY AND TOTALLY DELETED!
Note: this script does not remove directories that become empty after
deleting the last file.

*** Groups of duplicates found: 45 ***

0 [AC_DC/Dirty Deeds Done Dirt Cheap/02 Ain't No Fun (Waiting Round to Be.mp3] (10782720 bytes)
1 [AC_DC/Dirty Deeds Done Dirt Cheap/07 Ain't No Fun (Waiting Round to Be.mp3] (10782720 bytes)

0 [AC_DC/Dirty Deeds Done Dirt Cheap/03 There's Gonna Be Some Rockin'.mp3] (4737024 bytes)
1 [AC_DC/Dirty Deeds Done Dirt Cheap/06 There's Gonna Be Some Rockin'.mp3] (4737024 bytes)

0 [AC_DC/Dirty Deeds Done Dirt Cheap/05 Problem Child.mp3] (8278782 bytes)
1 [AC_DC/Dirty Deeds Done Dirt Cheap/04 Problem Child.mp3] (8278782 bytes)

0 [AC_DC/Dirty Deeds Done Dirt Cheap/09 Squealer.mp3] (7541497 bytes)
1 [AC_DC/Dirty Deeds Done Dirt Cheap/05 Squealer.mp3] (7541497 bytes)

0 [AC_DC/High Voltage/07 Little Lover.mp3] (8153088 bytes)
1 [AC_DC/High Voltage/03 Little Lover.mp3] (8153088 bytes)

0 [AC_DC/High Voltage/08 She's Got Balls.mp3] (6977536 bytes)
1 [AC_DC/High Voltage/02 She's Got Balls.mp3] (6977536 bytes)

0 [Aerosmith/A Little South of Sanity (disc 1)/01 Eat the Rich.mp3] (5032064 bytes)
1 [Aerosmith/A Little South of Sanity (disc 1)/11 Janie's Got a Gun.mp3] (5032064 bytes)

0 [Air/Talkie Walkie/10 Alone in Kyoto.mp3] (6940672 bytes)
1 [Air/Talkie Walkie/10 - Alone In Kyoto.m4a] (6940672 bytes)

0 [Beck/Sea Change/03 Guess I'm Doing Fine.mp3] (6947595 bytes)
1 [Beck/Sea Change/03 - Guess I'm Doing Fine.mp3] (6946713 bytes)

0 [Broken Social Scene/You Forgot It In People/11 Lover's Spit.mp3] (9176210 bytes)
1 [Broken Social Scene/You Forgot It In People/11 - Lover's Spit.m4a] (6195860 bytes)

0 [Buck 65/Talkin' Honky Blues/16 - Tired Out.m4a] (3402482 bytes)
1 [Buck 65/Talkin' Honky Blues/16 Tired Out.m4a] (3402482 bytes)

0 [Compilations/Rushmore - Original Motion Picture Sound/
	05 Mark Mothersbaugh - Sharp Little.mp3] (681984 bytes)
1 [Compilations/Rushmore - Original Motion Picture Sound/
	08 Mark Mothersbaugh - Edward Appleb.mp3] (681984 bytes)

0 [DJ Krush/Milight/27 From Eri Ohno.mp3] (1519708 bytes)
1 [DJ Krush/Milight/19 From DJ Krush.mp3] (1519708 bytes)

0 [Dinosaur Jr/Without A Sound/10 Seemed Like The Thing To Do.mp3] (8379377 bytes)
1 [Dinosaur Jr/Without A Sound/10 - Seemed Like The Thing To Do.m4a] (8379377 bytes)

0 [Econoline Crush/The Devil You Know/16 [silence].mp3] (81190 bytes)
1 [Econoline Crush/The Devil You Know/13 [silence].mp3] (81190 bytes)
2 [Econoline Crush/The Devil You Know/14 [silence].mp3] (81190 bytes)
3 [Econoline Crush/The Devil You Know/15 [silence].mp3] (81190 bytes)
4 [Econoline Crush/The Devil You Know/17 [silence].mp3] (81190 bytes)
5 [Econoline Crush/The Devil You Know/18 [silence].mp3] (81190 bytes)

0 [Econoline Crush/The Devil You Know/22 [silence].mp3] (81711 bytes)
1 [Econoline Crush/The Devil You Know/20 [silence].mp3] (81711 bytes)
2 [Econoline Crush/The Devil You Know/21 [silence].mp3] (81711 bytes)

0 [Explosions in the Sky/
	The Earth Is Not a Cold Dead Place/02 The Only Moment We Were Alone.mp3] (14741504 bytes)
1 [Explosions in the Sky/
	The Earth Is Not a Cold Dead Place/02 - The Only Moment We Were Alone.mp3] (14741504 bytes)

0 [Fountains of Wayne/Fountains of Wayne/01 Radiation Vibe.mp3] (5299675 bytes)
1 [Fountains of Wayne/Fountains of Wayne/01 - Radiation Vibe.m4a] (3538250 bytes)

0 [Jim Guthrie/Now, More Than Ever/01 Problem With Solutions.mp3] (6636032 bytes)
1 [Jim Guthrie/Now, More Than Ever/01 - Problem With Solutions.m4a] (4452000 bytes)

0 [Jim Guthrie/Now, More Than Ever/03 So Small.mp3] (5811863 bytes)
1 [Jim Guthrie/Now, More Than Ever/03 - So Small.m4a] (5811863 bytes)

0 [Jim Guthrie/Now, More Than Ever/09 The Evangelist.mp3] (7301139 bytes)
1 [Jim Guthrie/Now, More Than Ever/09 - The Evangelist.m4a] (7301139 bytes)

0 [Jimmy Eat World/Bleed American/11 My Sundown.mp3] (8398976 bytes)
1 [Jimmy Eat World/Bleed American/11 - My Sundown.m4a] (5522747 bytes)

0 [Junya Nakano, Masashi Hamauzu & Nobuo Ue/
		Final Fantasy X Original Soundtrack (dis/22 The Trials.mp3] (5134336 bytes)
1 [Junya Nakano, Masashi Hamauzu & Nobuo Ue/
		Final Fantasy X Original Soundtrack (dis/04 Tidus' Theme.mp3] (5134336 bytes)

0 [LCD Soundsystem/LCD Soundsystem/01 - Daft Punk Is Playing At My House.mp3] (7606984 bytes)
1 [LCD Soundsystem/LCD Soundsystem/01 Daft Punk Is Playing at My House.mp3] (7607115 bytes)

0 [LCD Soundsystem/LCD Soundsystem/03 Tribulations 1.mp3] (7178857 bytes)
1 [LCD Soundsystem/LCD Soundsystem/03 Tribulations.mp3] (7178893 bytes)

0 [Metric/Old World Underground, Where Are You Now/05 Calculation Theme.mp3] (5084142 bytes)
1 [Metric/Old World Underground, Where Are You Now/05 - Calculation Theme.m4a] (3440762 bytes)

0 [Midnight Oil/Earth and Sun and Moon/03 Renaissance Man.mp3] (4507775 bytes)
1 [Midnight Oil/Earth and Sun and Moon/09 In the Valley.mp3] (4507775 bytes)

0 [Minus the Bear/They Make Beer Commercials Like This/
	04 I'm Totally Not Down With Rob's A.mp3] (5398528 bytes)
1 [Minus the Bear/They Make Beer Commercials Like This/
	04 - I'm Totally Not Down With Rob's Alien.mp3] (5398528 bytes)

0 [Nobuo Uematsu/Final Fantasy IX - Original Soundtrack (/19 Immoral Melody.mp3] (2914432 bytes)
1 [Nobuo Uematsu/Final Fantasy IX - Original Soundtrack (/16 Queen of the Abyss.mp3] (2914432 bytes)

0 [Paul Simon/Songbook/14 You Don't Know.....MP3] (4528616 bytes)
1 [Paul Simon/Songbook/23 For Emily....MP3] (4528616 bytes)

0 [Ratatat/Ratatat/11 Cherry.mp3] (8123694 bytes)
1 [Ratatat/Ratatat/11 Cherry.m4a] (8123694 bytes)

0 [Simon and Garfunkel/Simon and Garfunkel - 1993-10-/16 50 Ways To Leave Your Lover.MP3] (8064896 bytes)
1 [Simon and Garfunkel/Simon and Garfunkel - 1993-10-/12 Gospel_ Gone At Last.MP3] (8064896 bytes)

0 [Stars/Set Yourself on Fire/03 Ageless Beauty.mp3] (5302307 bytes)
1 [Stars/Set Yourself on Fire/03 Ageless Beauty.m4a] (5302307 bytes)

0 [The Beatles/Revolver/10 For No One.mp3] (1945728 bytes)
1 [The Beatles/Revolver/09 And Your Bird Can Sing.mp3] (1945728 bytes)

0 [The Chemical Brothers/AuGmEnTeD/07 Take Place.mp3] (3598401 bytes)
1 [The Chemical Brothers/AuGmEnTeD/03 Innovation.mp3] (3598401 bytes)

0 [The Chemical Brothers/AuGmEnTeD/09 5-55-555.mp3] (3598399 bytes)
1 [The Chemical Brothers/AuGmEnTeD/04 Who Hero.mp3] (3598399 bytes)

0 [The Get Up Kids/Something to Write Home About/05 Out of Reach.mp3] (7252884 bytes)
1 [The Get Up Kids/Something to Write Home About/05 - Out Of Reach.m4a] (7252884 bytes)

0 [The Mr. T Experience/Revenge Is Sweet, And So Are You/09 Swiss Army Girlfriend.mp3] (2998821 bytes)
1 [The Mr. T Experience/Revenge Is Sweet, And So Are You/13 I Don't Need You Now.mp3] (2998821 bytes)

0 [The Postal Service/Give Up/08 This Place Is a Prison.mp3] (5627608 bytes)
1 [The Postal Service/Give Up/08 - This Place is a Prison.mp3] (5626954 bytes)

0 [The Shins/Chutes Too Narrow/01 Kissing the Lipless.mp3] (4770361 bytes)
1 [The Shins/Chutes Too Narrow/01 - Kissing the Lipless.mp3] (3194130 bytes)

0 [The Stills/Logic Will Break Your Heart/01 Lola Stars and Stripes.m4a] (3753710 bytes)
1 [The Stills/Logic Will Break Your Heart/01 Lola Stars And Stripes 1.m4a] (3735931 bytes)

0 [The Stills/Logic Will Break Your Heart/07 Let's Roll 1.m4a] (4253770 bytes)
1 [The Stills/Logic Will Break Your Heart/07 Let's Roll.m4a] (4236014 bytes)

0 [The Streets/A Grand Don't Come For Free/10 Dry Your Eyes 1.m4a] (4293577 bytes)
1 [The Streets/A Grand Don't Come For Free/10 Dry Your Eyes.m4a] (4293577 bytes)

0 [Treble Charger/Maybe It's Me/05 Red.mp3] (6735598 bytes)
1 [Treble Charger/Maybe It's Me/05 Red.m4a] (6735598 bytes)

0 [Weezer/Pinkerton/10 Butterfly.mp3] (2765416 bytes)
1 [Weezer/Pinkerton/10 Butterfly.m4a] (2765416 bytes)

All done.
  1. No comments yet.
  1. No trackbacks yet.

*
To prove you're a person (not a spam script), type the security word shown in the picture. Click on the picture to hear an audio file of the word.
Anti-spam image