Sunday, May 30, 2010

People and their dogs

I am a dog owner. Like most dog owners, I love my dog despite all of his quirks and bad tendencies. One of our neighbors in our building have a pug that has been inciting my dog into a barking and growling frenzy for a while now. Yesterday, when this occurred once again, I decided to try to speak with the pug owner to see if there was something they might be willing to do to prevent the distracting exchange between our pets. First, a quick description of the problem:

Our dog is a mutt and he is very protective of our apartment. My neighbors dog is a young pug that is a very playful dog, according to its owner. Now the main hallway in our building runs north/south, with the elevator that leads to the main lobby at the south end of the building. The pug neighbors live north of us down the hall and must pass by our door when taking their dog outside. There are plenty of dogs on our floor, and the issue of dogs passing by in the hallway is usually a minor one. Our dog might run to the door and bark once, perhaps sniffing around for good measure, but that's about the extent of it. The pug owners routinely take their dog out and either a) neglect to leash him while in the building, or b) give his lead so much slack that he can go wherever he wants. Well, being a dog the pug knows that another dog lives behind our door. So often times when going outside, because he basically has free roam of the hallway, the pug runs directly to our door and begins sniffing under it. This incites our dog and he barks and growls violently at the door until the pug moves away. There is a direct cause and effect relationship going on here:

P + D = N

Where P=the pug with free reign, D=our apartment door, and N=the noise frenzy from our dog. There is at least one other dog on our floor that used to incite growling and the like from our dog, but it's owner wised up and keeps it under control when walking past our apartment.

So back to last night. When our dog went into a frenzy last night, I stepped out in the hall and politely asked my neighbor if she wouldn't mind leashing her dog when walking by our apartment. Apparently, she took this as a reproach and to keep things succinct absolved herself and her dog of any blame. She came back a few minutes later and we had a civilized conversation about the situation. In hindsight, I can see how my approach may have been seen as confrontational, and I duly apologized as that was not my intent. However, no matter how I tried to explain the above cause/effect relationship it did not sink in. I explained to her several times that if she would simply keep her dog leashed and on the other side of the hallway away from our door when walking down the hall, my dog would not go into such a violent sounding fit. Apparently this logic was too complicated because today the pug ran by our door and my dog was growling and barking again.

What she failed and continues to fail to understand is that using your dog's "personality" to support is a poor argument. Dogs are animals and it is up to their handlers to keep them from misbehaving. Since her dog is the antagonist, it is up to her to correct her dogs animal instinct and keep it away from our door. You may be thinking - why don't you just train your dog not to bark at the door? Well, the rest of the time, it's not a problem. Only when my dog feels that this other dog is impinging on his territory does he get so defensive. Since this is an isolated situation with a simple solution (i.e. keeping the pug away from our door) the onus really falls on the pug owner to simply control and leash their dog (which is building policy).

I have a feeling that despite our conversation, which ended amicably, will not be productive in the end because my arguments were based on logic and reason, her's were simply based on her feelings toward her dog, the breed's natural traits, and the typical American perspective that I can do what I want, when I want, because it's my right. Apparently, being considerate of all her neighbors isn't a priority, so her dog will continue to annoy ours, and in turn, the rest of the floor.

If I'm way off base, tell me in the comments.

Wednesday, May 26, 2010

How To Create A Lossless Music Archive (FLAC)

Some people may be looking for information on how to archive their CDs or wondering why one would even be interested in doing so. This post will try to address those questions as thoroughly as possible. There is a lot of information here and it could probably be broken down into several posts, but that can be harder to keep track of and I think it's better to have everything in one place.

First off, why would one create a lossless audio archive? One reason is convenience. Having a CD quality file on a hard drive allows users to do many things quickly with their music without having to continuously re-rip it. Transcoding, which I will cover below, is much easier and convenient once you already have your files properly archives. One of the primary reasons that I do it, is with the pace of change in terms of digital music formats and technology dealing with the consumption of audio media, archiving your music in FLAC enables me to easily keep pace with these changes with little effort.

Of course for every benefit there are some drawbacks. If you have a lot of CDs or if you do a lot of legal music trading (i.e. Grateful Dead, Phish, etc.) your music library might be very large. This means you will need a sizable hard drive (or perhaps several). This process can be very time consuming as well. Fortunately, for myself, I find this entire process cathartic, and, as you will probably see, I am OCD enough that I enjoy the attention to detail it sometimes requires.

It is probably worthwhile to say something about file formats at this juncture to ensure that there is no confusion for those who are new to the world of digital media. To keep things simple I'll deal with 3 different file formats: WAV, FLAC, and MP3.

WAV is the format that you end up with when you rip a CD to your hard drive. One minute of music in WAV format takes up about 10MB. The complement of WAV on the Mac system is called AIFF. However, Mac has no problem handling WAV and since it is the standard and works across platforms I use WAV as my default raw audio format.

FLAC stands for Free Lossless Audio Codec. When encoding a WAV file to FLAC the same minute of music will take up anywhere from 4.5MB-6MB. Depending on the original WAV files, the FLACs of the same files will use 40-60% of the original drive space. So a 30 minute CD would be around 300MB in WAV format and circa 150MB in FLAC. Although FLAC is the standard in lossless audio compression, Apple's iTunes does not support it. Apple has their own proprietary lossless audio compression format, but I prefer FLAC. I have conducted checksum tests with Apple's lossless codec and they routinely fail whereas I have had no problems with FLAC. (On rare occasion one might see reference to SHN files as well. SHN or Shorten is a lossless file format that pre-dates FLAC, but is not as flexible as FLAC and has been replaced by FLAC in recent years.)

MP3 is the standard lossy audio compression format. The size of MP3 files depends on the bit-rate with which they are encoded. Because I have lossless copies of everything, I encode all of my MP3s at 128kbps, which is the standard rate. The same 1 minute of music encoded in MP3 format at 128kbps takes up about 1MB, or 10% of the original file disc space. This reduced file size does not come without consequences. The audio quality of the MP3 is inferior to both WAV and FLAC. With modern production techniques most people cannot tell the difference, but trust me, it's there and it doesn't sound as good. Therein lies my problem with Apple's iTunes - customers are paying good money for an inferior product. I will always opt to purchase a CD and create my own MP3s. That way I can create a perfect, CD-quality, lossless version of the same album. (While MP3 is the standard in lossy audio compression other formats exist, such as Apple's AAC format. Again, I deal with the standard format because they work error free on any platform.)

Ok, so now that I've talked about the advantages and disadvantages of creating a FLAC audio archive and outlined the three basic file types, it is fit to proceed to discuss the software involved in creating such an archive. My primary machine is an Apple MacBook running SnowLeopard, but I will include some links and comments for a few key PC programs as well.

Audio Rippers:

XLD: The best audio ripper is Exact Audio Copy, and unfortunately there is no version for the Mac. XLD is the closest thing to EAC on the Mac. It's got a simple interface and is pretty straight-forward. XLD can rip to a variety of file formats, not just WAV, and can automatically grab album artwork, too. Most importantly, it taps into the AccuRip database to ensure that rips are 100% accurate. XLD is also actively being developed. Make you have your preferences check for new versions because updates come out all the time for this software (See Right).

Rip/Max: These are two of several tools offered by programmer Stephen Booth. I find Max's ripping interface to be less user-friendly that XLD's, and I like XLD so much that I have not used Rip. However, it's good to know there are options and Rip continues to be developed.

Encoder/Decoder/Transcoder:

xACT: Is able to convert files between multiple formats, although depending on what you're trying to do, transcoding may take several steps. One of the best things about xACT is its ability to generate and verify file checksums (which will be discussed below) including, MD5, ST5, and FLAC Fingerprints.

Max: Supports an even wider spectrum of audio formats and while it lacks the checksum capabilities of xACT as a transcoder, this is my go-to program.

Metadata:

xACT: Also has metadata capabilities, but I find its FLAC Tagging to be rather cumbersome. One of the nice things about xACT's tagging is that it allows users to batch tag the song title and track number of FLAC files. However there is a glitch in the software that I have consistently ran into where, if you are trying to tag more than one album at a time you must tag one. Quit the program, re-open it, and tag the next one, and so on. There seems to be a problem with clearing the clipboard with the auto-title function.

Tag: If you've planned ahead with your original ripping settings the shortcomings of xACT can easily be circumvented with Tag. When setting up XLD to rip users can set the default format for the filenames of the ripped files (See Picture). I use the string "%n - %a - %t" which uses the CD data (which can be edited before ripping) to provide the track number, artist, and track title for each song ripped. This will look like "01 - Grateful Dead - Promised Land." Tag has a feature where the program will "guess" certain criterion based on a similar string. More on this below.

Playback:
As I already mentioned, iTunes does not include FLAC support. Feel free to contact Apple and request FLAC support in the future (http://www.apple.com/feedback/itunesapp.html). While I may not be a huge fan of iTunes as a whole, it is the best music database program and it would be great to be able to apply that organizational structure to a FLAC library.

Cog: A standard drag and drop music player. There's not a whole lot of flash to this program, but it doesn't use a lot of system resources and plays a wide spectrum of audio formats. There are no playlists or equalizers, which would be nice, but for those who just want a solid, reliable music player, Cog is tough to beat.

Play: I fiddled around with Play a while back and found myself appreciating the simplicity of Cog more. However, this is another Stephen Booth program and a definitely a worthwhile audio player.

Miscellaneous:

Album Artwork: Both Amazon.com and Wikipedia are good sources for tracking down album artwork, and I can go into a trick for getting high quality iTunes artwork on your hard drive, but the Album Art Exchange is a wonderful place to find quality album artwork.

Fluke: Fluke claims to enable users to play FLAC files in iTunes. I've never got it to work, and, quite frankly, I'd prefer native support for FLAC so I didn't invest much time in making it work. However, installing Fluke has a positive Finder benefit by allowing users to preview FLAC files in the Finder. Simply select a FLAC file in the Finder and press the space-bar. You'll see a preview screen and the file will start playing (the same work for WAV and MP3, and well all files). Press space-bar again to close the preview.

FLAC & Spotlight: Once you have your FLAC library built installing this little utility will allow Spotlight to search the metadata in your library. A really useful feature.

Hardware: I would recommend an external hard drive. The size depends on how big your music collection is. At present I have my entire FLAC library on a 1TB external hard drive and it takes up almost 700GB. I have the entire library backed up on a second 1TB drive as well. I also have a 320GB external hard drive where I store my iTunes library.

For PCs, the best audio ripper is Exact Audio Copy. A panacea encoding program can be found in Trader's Little Helper. I've found FLAC Frontend and MD5 Summer useful over the years as well.

Archival Workflow

Alright. Now that the software issue has been covered I'll describe my actual archival work flow. I've done this hundreds of times and although it can be a time consuming process, especially if you have scratched CDs, the final product is worth it, in my opinion. I will go through the entire process and explain what all the checksums, transcoding, tagging, etc. is all in the successive section.

Step 1: Insert CD in disc drive. Also create a destination folder. I have a folder for the artist, and a separate folder for each album. If it is a multi-disc set, each disc gets its own folder within the album folder. I prefer to label mine chronologically. In the example pictured, you can see that I label the folder with the Year, Album Title, and the file type. WIthin the parent album folder I have folders for each disc where I keep that disc's relevant data.

Step 2: Open XLD. Edit disc metadata to your liking. Make sure that the drop down menu at the bottom of the window is on "Include pre-gap for all tracks" to ensure that your rips are verified with the AccuRip database. If XLD finds artwork for you, you can save it right from the program in the target folder. When you have everything to your liking, press decode. In the picture you can see that I have XLD set to rip to both WAV and encode those WAV files to FLAC from the get-go. This is something the user can set up in the preferences. Note: Although it is an option, I opt to not add an MP3 file set at this point. This is done on purpose and will be explained later. If you do not have XLD set to automatically save the Rip log, make sure to save a copy of the Log file in the parent album folder. An example of a log report is provided.

Step 3: Once you have your set of WAV and FLAC files, open xACT and choose the "Checksum" tab.

Step 4: Select all of the WAV files and drag them into xACT. Press checksum and you will be prompted with a dialog box. Select "shntool," name the file, and save it to the parent album folder. Once you have generated the ST5 checksum you can delete the WAV files.

Step 5: Select all of the FLAC files and drag them into xACT. Press checksum and you will be prompted with a dialog box. Select "Fingerprint," name the file, and save it in the parent album folder.

Step 6: Open Tag, and drag-and-drop all of the FLAC files into the program. A drawer should pop open and the files should be visible in a list. Select all of the files in the list.

Step 7: Select "Guess Tags" from the Tools menu, or simply press command-G. A dialog box will appear. Fill out the appropriate tags. Remember the formatting must be identical to the actual file name otherwise the tags will not be properly created. Once you have everything properly filled out, press "Guess" and Tag will automatically populate the specified fields.

Step 8: Fill in any remaining metadata fields you desire. Remember to select all of the files to change the metadata for all files at once. When a file's metadata has changed it will be bolded in the tray window. Make sure to save all of your changes before exiting.

At this point you have created a verified and verifiable, losslessly compressed digital copy of your original CD. At this point I like to create an MP3 version of my CD and add it to my iTunes.

Step 9: Open Max. The first time you open Max it's a good idea to set your preferences. Select the appropriate encoding format under "Format" tab. Be sure to set your encoder settings how you like them. Like I mentioned before, I encode MP3s at 128kbps because I can always go back and make a higher quality one now that I have a FLAC copy. You can specify the MP3 default file name structure under the "Output" tab, and you can automatically add encoded files to "iTunes" under the "iTunes" tab.

Once you've established your preferences, select "Convert Files" from the File Menu and add your FLAC files. Press "Convert" and you'll have MP3 files in a few short minutes. Add these files to iTunes if you have not already done so.

Note: Depending on how particular you are about your ID3 tags in iTunes you may need to go and edit the MP3 tags. Fortunately, this is a since in iTunes, and one of the reasons I'd like to see native FLAC integration. iTunes has many more tag fields than the standard FLAC file and you would have to go back and fill in Album Artist field and any other options a particular album might require. Also, if you insert comments in your FLAC tags, for some reason Max does not write those to the MP3 files. If the comment was important, you would have to re-add that. FLAC files can technically have Album Art added, but Tag does not seem to support this function. There is no tag field for lyrics either.

Why did I wait to create MP3 files rather than just have XLD create them for me? The answer is in the metadata. WAV files do not support metadata so the information you edit in XLD only goes so far. For other formats XLD will write metadata. If you have basic tagging needs, this will probably suffice for you and even save a few steps along the way. I tend to be pretty particular about my metadata tagging and prefer the above method because it provides greater flexibility. Using XLD's auto-tagging doesn't make much sense if I plan to revise or add additional tags to my files, so I do not use the option in XLD.



Additional thoughts, loose ends, and explanations

As you might be able to tell, this is somewhat of a cumbersome work flow and unless you work from home and can do all of this in the back ground, or you have a lot of time on your hands, creating a FLAC library, especially of a large CD collection would take a long time. Unfortunately, there is no single program that can do everything I need it to. One of big things seems to be generating ST5 and FLAC Fingerprint (FFP) checksums.

Ideally, there would be one program that would have all of the components of this work flow that could be customizable as the user needs. If that were the case it would be possible for the end user to write a script to cover all the steps laid out above. Because this work flow spans several programs that task becomes increasingly difficult. (If there is anyone reading this who knows about AppleScripts and can help with this, please let me know.)

Throughout all of this I may come off as critical of Apple. Don't get me wrong, I love Apple and most of their products, however, they are not without flaw. I don't understand how they can have multiple lossy codecs native to iTunes, but ignore the accepted standard in lossless compression (FLAC) in lieu of their proprietary ALAC. FLAC is open source, so it shouldn't be an issue of cost. It's really beyond me because they don't do anything with that codec. If they sold music on iTunes in ALAC it would make sense (and do the music a favor, too), but business isn't about what's right, it's more about what will make money.

Some may say, why go through all this trouble when I can just rip a CD with iTunes? Doesn't iTunes have error-correction? The short answer is, yes. However, specialty ripping programs are capable of verifying the integrity of a rip and are much more thorough than iTunes. If you're an audiophile, iTunes is not doing you any favors.

Others might wonder if they can use a program like Max to convert their MP3s to FLAC. Obviously, the answer is, again, yet. However, there really is no point in doing so. Think of the WAV (or FLAC) to MP3 conversion in terms of analog equipment. In the days of making mix tapes if you dub a track from one tape onto another, the audio on the new tape loses a generation of quality, and is inferior to the original. The same thing happens when you encode from WAV/FLAC to MP3. Remember that WAV files process at 44.1kHz/second. That's really fast. Compare that to an MP3 where the number drops from 44,100, to 128. Even a 320kbps MP3 pales in comparison to the original file. And while most people might not be able to tell the difference, if you encode a file to 128 MP3 and back to WAV, you would be able to tell the difference. The situation is the same between MP3 bit rates. A 128 MP3 can be convert to 160, 256, or even 320, but it will not improve in quality at all. Sure, the file size will grow, but the actual audio quality will decrease in this conversion. An MP3s audio quality is, at best, as good as its lowest encoded bit rate.

Finally, let me address the issue of checksums. Many folks on the internet like to use CUE files, which makes no sense to me. A CUE file is only really useful when a CD is ripped as one single track and a playback program needs to know where to insert the track splits. Since most people would never think to rip a CD as one long track, the use of CUE files is moot.

The benefit of having checksums created at the time of the initial ripping/encoding is that you have a means to detect file corruption, and the ability to verify the quality of the files that you have. Originally I used MD5 checksum files, but later learned of their limitations. Once you encode a file and generate a MD5 checksum for it, any change to the file, including metadata, will cause the file to fail it's checksum verification. For this reason, I switched to ST5 checksums, which look at the actual wave form of the audio and not it's container format. I use ST5 checksums for WAV files. Using MD5 for FLACs is a big problem because of metadata tagging. Everyone wants their music tagged their own way, but changing that information causes the MD5 to fail. By using FFPs one can avoid this problem. The FLAC metadata is essentially tacked onto the end of the file and the FFP checks the audio portion, but ignores the metadata section. If you make a typo in your metadata and find it after you generate your checksum you can simply edit the metadata if you have FFPs, whereas it would be necessary to regenerate a new MD5 for the file. The rip LOG simply verifies that a CD was properly ripped.

So used in concert, the LOG, ST5, and FFP verify the quality of your music and make it verifiable at the same time.

For more on checksums, see: http://www.thetradersden.org/forums/showthread.php?t=17951

Have I forgotten something? Is anything unclear? Provide any and all feedback in the comments section.

Tuesday, May 25, 2010

Do people actually like Circus Peanuts?

A recent foray into a Colorado hobby store prompted me to remember how gross Circus Peanuts are. My cousin and I went to the store to buy some model rockets, which are wicked fun I might add. As we were preparing to checkout my cousin quickly perused the candy near the checkout. Seeing that I am never one to deny my sweet tooth, I, too, took in the Hobby Lobby candy selection. As I scanned the available carbohydrate concoctions, I saw what I consider to be one of the grossest candies this side of black licorice: Circus Peanuts.



Not only do Circus Peanuts taste terrible, they have a gross spongy texture. You may be saying, "Well, duh. They're basically marshmallows." Well guess what, I'm not a huge fan of marshmallows either.

As we were checking out, I made a comment about how gross I thought Circus Peanuts. To my amazement, the woman working at the checkout said they actually sold a fair share of Circus Peanuts. Needless to say I was shocked. However, what she said next partially affirmed my sentiments on the red-headed step-child of the candy aisle. She said that one or two on occasion were not bad, but when pushed further she agreed that consuming an entire package in a short span of time would not be a satisfying endeavor. Of course, the problem with a marshmallow candy is that if you let it sit, it will quickly go stale.

What this anecdotal evidence suggests is that Circus Peanuts should be sold in 2 packs, and even that might be too great of a quantity. And because a two-pack of Circus Peanuts does not represent a sound production or marketing strategy, the folks at Brach's or whoever makes those hideous "treats" should simply cease production all together.

Sunday, May 23, 2010

Get "Lost"

Like many other folks my wife and I have spent the past six television seasons watching "Lost" on ABC. This show has been so uneven, so up-and-down that I have to really think about watching anything else produced by J. J. Abrams. I know I'm not the first to say this, but as the series wraps up later tonight, let me join in the horde that says "who cares."

Excuse the pun, but "Lost" lost it a while ago. The writer's staunch refusal to answer questions and the inadequate way in which they answer the ones they do makes the show unpleasant to watch. For instance, the episode that focused on Jacob and his brother (would it really be that hard to give him a name) could have been a 10-15 minute piece within an episode that moved the rest of the story line further. But no, as time wound down on the show the writers decided to dedicate an entire episode to something they continued to not fully explain.

Similarly, the entire "parallel universe" motif that has been going on the entire sixth season is just a ploy. Seriously? The only good J. J. Abrams show is "Fringe" and that show has so much parallel universe stuff it's not even funny - but it's interesting. In "Lost" it seems like the whole plot was thought out willy-nilly. It is as though the writers were planning to write 7 shows but were give 12, and they thought, "oh crap, we better come up with some filler."

Two or three years ago, when "Lost" was spiraling out of control (remember the episode where Paolo died?) and the writers announced that the show would end in 2010, the show actually started to move forward instead of chasing it's proverbial tail. With an end date in sight, the writers seemed to have an idea of where they wanted the story to go and how they wanted to get there. Somewhere along the line, while writing the last season, it appears obvious that the task of answering all the questions they had prompted during their 40 years in the desert period proved too daunting. Having shot themselves in the foot, the writers threw up their hands in frustration, jumped into the safety net of post-modernism and said "We don't have to answer all the questions. Not all questions need to have answers. You have the body of work, what does it mean to you?" To me, it means a total cop out. People can parse and analyze the show all they want, a recent article in the Washington Post suggested that the show was a metaphorical mirror of contemporary America. Hog wash. It's a piece of escapist television that was crushed under its own popularity and the indecisiveness of its writers.

The entire sixth season my wife and I have been watching the show. Like suckers. We figured that since we had already invested so much time into the show we might as well finish it out. However, the entire season has been like pulling teeth. Neither of us really cares what happens at this point, we just want the show to end. If I had to do it over again I would have jumped ship when the show initially went downhill, like we did with "Heroes."

So while many may be going crazy over what will happen in tonight's final episode, like the final episode of the Sopranos many will be content with the ending and many will not. In my household, the feeling will simply be "thank God it's over." We don't care.