How to Properly Compare Headphones

Many of us instinctively prefer listening to music through regular loudspeakers, perhaps because it seems more analogous to listening in a real life concert hall.

But using headphones offers three compelling advantages, and sometimes is not only our best choice, but may also be our only choice.  For example, as travelers, headphones give us a convenient ability to enjoy highest quality music when away from home.  For that matter, with less distance involved, headphones allow us to listen to music anywhere at home, rather than just in our main living room where the speakers are located.

A second advantage is if you’re in an apartment and your neighbors don’t want to be treated to your musical experiences too.  A related and equally applicable situation is if everyone else in your house doesn’t want to be hearing your music at the same time also.

The third is if you want to enjoy great music without spending ridiculous amounts of money – generally you’ll find you need to spend ten to one hundred times more for comparable quality sound through loudspeakers as through headphones.

So, no matter which of these factors applies – and of course, it is not an either/or choice; most of us have both speakers and headphones – you’ll probably find yourself wanting to buy a pair of headphones.

We discuss the different types of headphones and your choices here.  This article is all about how you can compare different headphones and confidently select the set that best suits your musical and sound preferences.

Unfortunately, it is very difficult to compare different headphones, and pretty much impossible if you don’t have the different sets you are comparing side by side.  This is because our audio memory is very poor.  Indeed, our overall audio perception is not very good to start with.

Extreme differences between headphones are probably things you’d notice and remember, but the more subtle shadings of tonality between one set and another are things you’d forget, or, if you did remember, you’d find it hard to decide which you preferred, as between, for example, the slightly ‘dark’ sound you heard several days ago and the slightly ‘bright’ sound you were listening to now.  In addition, sometimes a sound which initially appeals can become more tiring after a while, whereas a less colorized sound that first sounds bland and boring subsequently sounds more clean and pure.

Sounds also are perceived differently when they are a change from one set of coloring to another.  If you’ve become used to a certain sound profile, when you hear a different set of headphones, at first the differences may seem stark and objectionable (a bit like sucking on a lemon – it is a very different experience if you’ve just been doing the same with a lime, or if you’ve been eating sugar).  You need to give yourself time to decide if the differences are good differences or bad differences – this is a distinction that will not first be obvious.

So, and unfortunately, comparing headphones side by side either requires the unlikely circumstance of having friends who already own the models you are most interested in, or taking some advantage of sellers with generous return policies.  Hello, Amazon and Costco!

Now for the big thing for when you can compare your short-listed headphones side by side.  You can’t just swap different sets of headphones, alternately plugging and unplugging each set, one at a time, into and out of whatever music source you are using.  Neither could you also use a splitter that simply takes the output and splits it into two outlet plugs to save you the plugging and unplugging process.

This is because we perceive sound quality differently when we listen to it at different volume levels (basically, the louder the sound, the generally better we perceive its quality to be).  Different headphones have different efficiencies, so when they are plugged into the same sound output, some will sound louder than others and therefore will sound better than the others.

This is a nonsense perception that would be corrected if you adjusted the volume to make each set of headphones the same loudness, but by the time you’ve struggled to do that, your audio memory of the previous pair has been totally destroyed.  You need a process to automate this for you.

The Equipment You Need for Headphone Testing

It is very important that you make a good choice when you buy headphones.  That way there’s no danger of suffering ‘buyer’s remorse’ and within a very short time, upgrading your headphones (and probably for no good purpose), or buying a set more expensive than you really need, just to be ‘on the safe side’.  Fortunately, it is simple and inexpensive to put together a high quality testing system so you can know, for sure, which set of headphones really is the best for you.

You need two things.  The first is an amplifier that has multiple outputs, each with its own volume control.  The second is a way to scientifically calibrate the sound of the different headphones you are evaluating to ensure you are indeed comparing apples with apples.  Fortunately, both of these things are much easier than they might seem.

The Behringer HA400 four channel stereo headphone amplifier is an ideal solution to the first part of your problem.  It takes a sound source, splits it into four different outputs, and gives you separate volume controls for each of the four outputs.  So you could potentially be comparing as many as four different headphones simultaneously with this unit.

Available on Amazon for only $25 , this is an essential and excellent unit for your testing.  You’ll get way more than $25 of value from it by being able to confidently select the headphones you truly feel to be best, and enjoy peace of mind with your selection subsequently.

After you’ve used the unit for choosing your preferred headphones, you could either sell it, keep it for a ‘just in case’ future evaluation of more headphones, or use it if there are times when two of you both want to listen to headphones simultaneously, with each of you able to set your own preferred volume levels (this is a huge benefit).

Now for the second part of the puzzle.  You need a sound pressure level meter.  Yes, you could buy one of those from Amazon as well, and they have perfectly acceptable ones for as little as $16.  But if you have a smartphone, you can download a free or nearly free app that will turn your phone into a sound pressure level meter, and that can be fun to have on your phone and to occasionally use in all sorts of other environments (such as seeing how loud it is on a plane, getting some facts to complain about noise from your neighbors, or whatever else).

On Android, there are a number of free apps and also a 99c app which we feel is better than the free apps – SPL Meter from AudioControl.  On iOS, there are again a range of free and paid apps to choose from and in our opinion the SPL Meter from Andrew Smith is the best choice.  (Note that AudioControl and Andrew Smith are one and the same.)

In order to use either a sound pressure level meter, or your smart phone and an SPL app, to measure headphone volume, you need to make a consistent measuring template, but before you do that, set the volume for normal levels and settings.

We suggest you set the volume level control on the Behringer multi-channel headphone amp for the first set of headphones to about the middle point, and use your main music source’s volume control to adjust the headphone volume to a comfortable moderately loud level, first using some of the music you’ll be evaluating.  You don’t want any of the volume levels – on your music player or on the Behringer unit, to be much more than about half way to maximum, but you do want them to be beyond about a quarter way.

Now that you have things set up for normal music, it is time to switch to the test rig.   This need not be complicated.  We suggest a simple piece of stiff cardboard, slightly larger than the headphone cup, with a hole in the middle.  On the back side of the cardboard, tape your phone or meter with the microphone exposed through the hole.  Then press the left earcup (because we are giving you a sound file for calibrating, below, that plays only through the left earcup) from the headphones up against the sheet of cardboard, centered on the hole, hold everything very still (or – much better – use some rubber bands and set it down on a soft rather than hard surface) and read the sound level displayed on the phone or meter.

You need a calibration sound to play through this test rig to calibrate your different headphones.  Not a problem.  We’ve created one for you, for free.  Download this MP3 file.  It is one minute of pink noise, in the left channel only, and is a good sample to use to set the volume levels with.  Play it through your system.

You should do this in a quiet room with hopefully few background noises to interfere with your measurements.

You’ll see the volume level will probably fluctuate a bit, but by less than 1 dB.  Decide what the average number is (use slow response if you have an SPL that offers that option).  Consider whatever sound reading you get from the first set of headphones as your reference number, then swap to the next set, place them in the same position on the cardboard, and adjust the channel two volume control on the Behringer until the volume is reading the same as for the first.  If you are testing more than two sets, do the same for the other units too.

You might notice the pink noise sound varies in timbre or tonal coloring a bit from one set of headphones to another.  That’s only to be expected, because they are of differing characteristics.  Don’t worry too much about these sound changes, and there’s not really any way you can evaluate the quality of the headphones from the changes in pink noise tonality alone.

The Actual Evaluation Process

Once you’ve got your headphones calibrated (and be careful not to bump the volume controls) you can now proceed to listen and evaluate them.  Choose whatever high quality music you have and like – music which you feel is demanding in terms of quality and which you can understand the difference between good and bad quality, and play away.

Some people find it helps to have a loop of music that keeps replaying, swapping from one set of headphones to another and back again, hearing the same piece of music over and over to help make more exact comparisons.  Other people like to listen for an extended time to one set of headphones, then for an extended time to the other set, and so on.

Note that when you swap one set of headphones for another, make sure that you have the new set properly and comfortably on your head and the earcup foam has had a little time to settle and conform to your ears.  It is only when the ‘seal’ has been formed around your ears that the acoustic will be optimized.

We suggest perhaps the ‘best’ approach (if there is such a thing) is a mix of both strategies.  You are really trying to achieve two different things.

The first objective is to be able to describe the difference in sound quality between the different sets of headphones.  There are several elements of this to consider.

• Think about the bass, the mid-range and the treble.  Which parts are more or less pronounced in one set rather than the other?
• Think also about the ‘clarity’ of the music.  Which headphones give you the clearest, purest, least distorted tones (no matter if they are more prominently high or low)?
• Some people also perceive a difference in location/accuracy with different headphones, saying that some create a ‘broader soundstage’ and others have less directional information in them.  Some people perceive some sounds as coming from more clearly perceived exact points, other people perceive the sounds as being more vaguely defined in terms of location information.

It is very helpful to write down your perceptions of the different headphones you are listening to, under these or whatever other categories you wish.  This is a bit like how a wine taster will categorize his impressions under different aspects such as nose, color, initial taste, mouth-feel, after taste, and so on.

This helps you to know exactly what to focus on as you return back to the different sets and switch between them.  Eventually you’ll end up with lists of differences for each set of headphones.  But which is the best?  That brings us to the second part of your evaluation.

After you’ve established the differences between the headphones, you now have to do the harder thing.  Which one do you feel sounds ‘best’ to you?  This is best done by listening to each pair of headphones for an hour or more, to see if you get any audio fatigue, and if some aspect of the headphones which at first appealed starts to overwhelm or become too artificial or gimmicky.

It is helpful for your extended listening test to use several different musical styles (assuming your tastes extend to more than one style of music).  For example, you might want to listen to some instrumental music with just a few clearly recorded instruments, to some vocal music (or even spoken sounds), and to some larger ensemble type music.  You might want some music that has a heavy bass beat, and other music that doesn’t.

Amplifier Related Issues

Different headphones will sound slightly differently when connected to different amplifiers.  One obvious difference is that some high-impedance headphones might not sound loud enough when connected to underpowered amplifiers, such as you can often find in some portable electronic players and cell phones.

Once you’ve established your preferred set of headphones while using the Behringer amplifier, you should be sure to check the headphones will be loud enough and sound good when connected directly to whatever music source (or sources) you’ll most commonly be using with them.

Note – if everything else is suggesting a particular set of headphones are clearly your favorite, but they just won’t go loud enough when you’re playing them through the system you’ll use them with, that’s not necessarily an unsolvable problem. There are tiny portable headphone amplifiers you can connect to your portable electronics if needed.

The most popular headphone amp on Amazon is also a good choice, and bargain priced.  It is the Fiio E6, and costs a mere $28.  So lack of output power doesn’t need to be a deal breaker.

Non-Sound Issues

Although you are buying headphones to listen to music through, there are some non-musical considerations as well.  Price is an obvious one, and please understand that price is totally unrelated to quality.  We could list dozens of examples where markedly lower priced headphones have scored appreciably higher than much more expensive headphones.

A much more important consideration is comfort.  Do the headphones fit your ears well?  If they are in-ear earplug types headphones, comfort can be a huge variable.  For external headphones, there are two basic design styles – those that sit on your ear, and those that surround your ear.  On-ear headphones are slightly smaller, but around the ear headphones are usually more comfortable, particularly for extended listening.

Some people with larger or smaller than standard sized ears will find some headphones are much less well-fitting and comfortable than others.  For around-the-ear headphones in particular, you want to be sure that they truly are big enough to surround your ears, and that they are deep enough to not squash your ears inside their cups.

Some headphones are much heavier than others; some people might find this a comfort issue too.

There are also two design approaches to headphones.  Some have a closed back and others have an open back.  Closed back headphones keep more of the music in, and let less outside noise in.  Open back headphones leak more sound into wherever you are, and also allow more outside sound to permeate in.

Many people feel that open headphones are better, but if you’re in an ultra-quiet environment where other people will object to the leaked sounds from your headphones, they could be a problem.  Similarly, if you want to listen to music in a moderately noisy environment, you’ll prefer closed back headphones.  If you’re in a very noisy environment, you should get noise cancelling headphones – probably the Bose QC15s.  These do not have as good a pure sound capability, but they are unbeatable for getting rid of a huge chunk of the background noise.

Build quality is important, but not always easy to ascertain.  Do the hinges and other connectors look flimsy and breakable, or robustly strong?  Is the connecting cable well made, and with strain reliefs at both ends?  Ideally, the connecting cable should be capable of being replaced (ie unpluggable where it meets the headphones) – you don’t want to have to junk a $150 pair of headphones because of a failure in the $15 cord.  How long is the cord – is it both ‘long enough’ for your typical listening environment, but also not too long as to always be a tangled mess?  Does it end in a big or small plug, and does it have an adapter for the other size plug as well?

A longer warranty and a recognizable brand name give you added comfort in terms of quality and longevity of your headphones.

Is anything else included with the headphones?  Sometimes you’ll get carry bags or cases, which might or might not have any value to you.

Last, but still appreciable, do the headphones look appealing to you?  You don’t want to have to suppress a shudder of revulsion and regret every time you see them.

What to Ignore

Please try, as best you can, to ignore other opinions, reviews, the prices of the different headphones, and, most of all, manufacturer specifications.  None of these should matter to you.

Think of it like choosing a color to paint your house.  One friend might advocate green, another brown, and a third blue.  They each have convincing reasons why their choice is ‘best’.  But the color of your house is all about you and your preferences, and so too, exactly, the type of sound you most like is all about you, your ears, and your personal preferences.

There is no right choice.  Sometimes there is an obvious wrong choice, but there is no right choice.  There is only the set of headphones that you like the most, and which sound best to you.

Headphone Burn-In

There’s a controversial theory that headphones need ten, maybe even one hundred hours of music to play through them before they have been ‘burned in’.  It is thought that just like you formerly needed to carefully drive a car for the first thousand miles or so, while the various moving parts bedded themselves in and moulded to each other, with headphones (and speakers in general) there are definitely moving parts and perhaps it takes some hours for them to evolve from a possibly stiff brand new form to a more used and flexible/pliant form.

We’re unsure about the validity of this theory, but we can conceptually understand the reason for thinking this is necessary.  It does no harm to play the headphones for some time before you start to test and evaluate them, and so on the basis of it not hurting/harming and possibly helping, why not burn your headphones in by just playing them without listening to them for a day or so.

Just connect them up to a radio receiver or something like that with an ongoing continual sound source, and let them run for a while – at least a day, and potentially three or four days to be absolutely sure.

The Final Choice

If you’re like us, you end up with some clear ‘losers’, but no clear winner!  If you’re again like us, you then struggle to choose which of your short-listed headphones you are most comfortable with.

There are several things you can do at this point.

You can buy the most expensive set, in the desperate hope that, for once, there truly is a correlation between cost and quality.  You can decide not to trust your ears and then buy based on manufacturer specifications and third-party reviews.  You can buy the least expensive set and feel triumphant that you saved yourself probably some hundreds of dollars while getting a sound quality that sounds, to you, equally as good as from the more expensive headphones.

Or, of course, buy multiple pairs of headphones and switch between them from time to time, depending on your mood and the style of music and sound coloring you seek!

Summary

All sound systems need headphones so as to extend their functionality.

It costs you only $25 and a bit of time on your part to create a truly professional grade headphone evaluation unit that will allow you to clearly and confidently test and choose between different sets of headphones.  Best of all, there’s every chance that with this high quality test setup, you’ll be able to correctly choose the best set of headphones to suit your preferences, and they will almost certainly save you money compared to other headphones you might otherwise also be considering.  You’ll save more than you spent on the test rig, while also feeling better about your choice.

You no longer have to rely on the totally flawed hope that if you spend some more money, you’ll get better quality headphones.  Now you can know, based on your own testing, which headphones really are the best for you.

We’ve provided a short list of some good choices in headphones here.