This isn't always a bad thing. Closed headphones, open headphones and earbuds are all constructed differently, and thus transmit sound in different ways. Altering the frequency response of a closed headphone design might make bass frequencies sound more natural, for example, which most users would probably prefer over a flat, "authentic" sound. However, it's hard to compensate for all styles and genres of music; altering the bass response might make your Modest Mouse sound great, but your Mozart, not so much.
But while a frequency test is great for testing the setup and theoretical performance of a set of headphones, it's not a good real world test; after all, you'll be listening to music, not artificial tones on a daily basis. Oftentimes, the best way to test the quality of a potential purchase is to use a well-recorded song that exhibits a nice range of qualities and instruments. For example, classical pieces from well-known orchestras are often popular with many reviewers, because they demonstrate a wide range of instruments, encompassing both high to low frequencies. Jazz and classic rock albums are also popular choices.
What you won't find used are albums or songs produced more recently. Today, sound engineers often compress and squish frequencies in order to achieve the "loudest" possible sound without distortion, compromising the quality of the music in the process. In fact, if you're to compare the waveforms of two songs — one from the 70's, and one from today — you'll see what we mean. If there's no distinct frequencies left, it becomes hard to accurately use a song or album to test the frequency response of a pair of headphones.
With all this in mind, what sort of tests do you do when picking out a new set of headphones or earbuds? And more importantly, what are you using now? Let us know!
Images via Flickr users Giandomenico Ricci and Scratchworx.