We wrote before about how our hearing is imprecise and easily tricked. If you’ve not read that article yet, and not yet done some of the hearing self-tests linked within it, we recommend you should do so first, so as to quantify your present hearing abilities.
So, we will assume that you’ve tested your hearing and can hear up to a high frequency limit of maybe 15 kHz. Actually, it doesn’t really matter if you can hear ‘only’ to 12 kHz, or impressively all the way to 20 kHz, the key assumption here is that you can’t really hear anything over 20 kHz.
The other relevant test is to see how much dynamic range you can perceive. We’ll guess you can hear 60 dB and struggle to hear 66 dB of dynamic range. Again, maybe you can even hear 78 dB of dynamic range, but whatever your result, you can’t hear anything approaching 100 dB.
It is also interesting, but not quite so essential for this discussion, to see how sensitive you are to minor variations in sound levels. This at least helps you to understand the implications of a frequency response curve that might show a variation of +/- 1.5 dB compared to one that shows a variation of +/- 3 dB.
Anyway, at this point, you have hopefully already realized that your hearing fades away somewhere in the mid teens of kHz, and that in your normal listening environment, you can’t hear much more than a 60dB spread between music that starts to become uncomfortably loud and music which is too quiet to be heard.
So, before moving any further, can we ask you this key question : If you can’t hear higher than 15 kHZ or greater than 60 dB of dynamic range, what is the point in paying massive amounts of extra money for frequencies higher than you can hear and dynamic ranges that are greater than you can appreciate? Oh yes, not only are these outside your hearing capabilities, but they are also greater than the music as it was played/recorded, and greater than your amplifier and speakers/headphones can play back!
If you take our point that spending appreciably more money for things you can’t hear or benefit from is pointless and ridiculous, you don’t really need to read any further. But if you’re vaguely thinking that there might be some sort of intangible extra benefit to justify the better specifications of more expensive music recordings, and if you think ‘all these millions of people advocating high-end audio gear and expensive recordings surely can’t be wrong’, please do keep reading.
Double-Blind Testing
What you need to do now is some ‘double-blind testing‘ of music that has been recorded at both ‘greater than CD’ standards and at regular CD standards, so you can see for yourself if you really truly can hear any difference or not. Never mind the abstract theory that we’ve looked at so far, the ultimate test surely has to be a double-blind comparison of real music.
Double blind testing is the ultimate in testing, because it strips out all the possible biases and inadvertent/unconscious preferences that otherwise invariably and subtly influence your perceptions. When double-blind testing, you listen to a series of different music samples without you – or anyone else – knowing which of them are the ‘greater than CD’ samples and which are the standard CD samples, and make your own best choices for which is which.
After having done this, the actual results are revealed, and a statistical analysis tells you if the accuracy of your selection (such as it may be) is something that is significant or if it is merely little more than random guessing.
Just like tossing a coin sometimes sees it landing heads multiple times in a row, so too can guessing sometimes apparently create the appearance of correctly differentiating between the two different recordings, so it is necessary to repeat the evaluation at least ten times to cause random chance to become less a factor.
The good news is you can do your own double-blind testing at home, and you don’t need to spend a penny to do so. You don’t need any special equipment, and neither do you need any costly software, to do so. This article gives you careful step by step procedures that will allow you to have a valid double-blind test.
It is a relatively simple process, but there are some potential pitfalls and traps you need to avoid so as to give both the music and your ears the fairest opportunity to make a correct evaluation.
This fair opportunity is not as obvious as it might seem. You need to have the same identical piece of music – not only the same performance, but the same recording and mixing and mastering, with the only difference being different sample rates. Then you can compare the different pieces to see if you can hear any difference. But – here’s the huge unanswerable question – how do you know that you have the exact same recording, with everything identical except the sample rate, for your different comparison pieces?
We have heard sufficiently many stories, sometimes from seemingly credible sources, as to accept that it may well be regrettably true that some sources of music will deliberately ‘sweeten’ the sound for their higher quality sample rate recordings, so that people listening to the highest sample rate recordings will truly hear a difference and possibly even a genuine quality improvement. The trap is that the true improvement they are hearing is not coming from the sampling rate (although they’ll assume this to be the case), but rather from the recording engineers, who have slightly altered the settings as they mix and create the masters for the different recordings.
Fortunately, there’s a simple solution that works well for the purpose of this testing process. Get a piece of music that is recorded in, say, 192 kHz 24 bit digital format, and then do your own down-sampling of this to create versions at other sampling frequencies and depths.
This will give you the exact same piece of music every time, with the only difference being the changed sampling rates. This is truly the only way you can be sure you are truly comparing apples to apples.
1. Music Samples
First, you need to get some music samples. You can get some 192/24 samples from here and/or from here. A bit of Googling will bring up many more sources of free samples too. Download as many samples as you like, with the types of music you think you’re most likely to hear subtle quality differences in.
We suggest some solo vocal (because our hearing is well calibrated for voices and the differences between them), some solo piano (a very hard instrument to record and play back well) and some of whatever other music you enjoy.
2. Converting to Different Qualities
We’re assuming that you went to what is currently the close to ultimate top end of music files, and have downloaded some 192/24 samples in FLAC format.
Now you need to make lower quality copies of these to do your comparison testing with. If you have a PC, we suggest you use the free program, Audacity, to do this.
After installing Audacity, open one of your downloaded music files, then in the bottom left of the screen, change the project rate from presumably 192000 and make it 44100. Then go to File – Export and save it as a FLAC with a new name, and in the options button, choose 16 bit instead of 24 bit.
You now have the original 192/24 file and the newly created 44.1/16 bit (ie CD standard) file. This is probably all you need. Repeat this for the other test files you also have.
3. The Double Blind Testing
Next, you need a way to play these samples on your computer and to do the double-blind testing. You could certainly play them through Audacity, but you couldn’t conveniently do double-blind testing this way.
Assuming you have a PC, you should now get a copy of the excellent program, Foobar 2000. Then get a ‘plug-in’ for Foobar, the ABX Comparator from this page of the Foobar site and add it to the program. It is a very simple and quick process.
Now, in Audacity, go to File – Add Files and add the two different versions of the first music sample you want to test. Then highlight them both, right-click, and choose ‘Replay Gain’ and scan them both for per-file track gain. This ensures the two samples have exactly the same volume level. Our ears hear sounds differently at different volume levels, so you want to make sure the volume level is the same.
Highlight the two files again, right click, choose Utilities – ABX Two Tracks, then select the ‘Use Replay Gain’ and you are now ready to do your own blind testing.
Use your best headphones or speakers, choose a nice quiet environment, set the volume level to the most comfortable, and first click on the Play A and Play B buttons a bit to make sure it works.
Experiment with some of the options. A useful one is the ‘Keep playback position’ option so you can switch from one track to the other and have the music continue without interruption. Another is to set a start and end position so you can loop over the same piece of music.
Anyway, when you know how it all works, it is time to blind test.
Click on the Play X and Play Y buttons, and now you don’t know whether X is A or B, and whether Y is A or B. After you’ve gone back and forward every which way, you then can ‘put your money where your mouth is’ and click on one of the two choice buttons to see if you correctly identified the two different sounds or not.
Then click the Next Trial button and repeat. Note that the next trial might have the same X/Y and A/B links as before, or – randomly – the program might swap them. Each trial might (or might not!) be different.
You’ll probably not want to hide the results to start with, but after a bit of experimenting, we suggest you then do a full trial of maybe ten or more evaluations, and not look at the results until after you’ve done them all.
If you’re like us, you’ll find it frustrating and difficult and probably absolutely impossible to hear any clear real difference at all, and the results will confirm this. Don’t feel you are failing when you can’t find any difference, because you are not failing at all. Quite the opposite – you are validating the theory that there is no perceptible difference, you are succeeding!
So, end of story. Unless something went wrong, at the end of the testing, you have now confirmed that, at least for you, your ears, and your audio setup, you can’t hear any difference between a regular CD quality setting and an esoterically high-end setting. Sure, the 192/24 has potentially and theoretically 6.5 times more data within its file, but this data has been tested and proven by you to all be useless and meaningless and of no musical value.
Don’t feel cheated by this. CDs are truly stunning in their ability to store and play back amazing quality music. Just because they are ubiquitous and inexpensive in no way limits their ability to give you better quality sound than your ears can appreciate.
Think, as a comparison, of the difference between video tape and DVD or Blu-ray. You probably don’t even remember the appalling video quality of a VHS tape, and you probably also don’t remember how much bigger they were or – here’s our point – how much more expensive they were, either. But just as has been proven with video, there is no correlation because cost and anything else, and this is true too of music.
The only reason a higher resolution music file costs more than a CD resolution music file is because the company selling it believes they can charge more for it. It costs no more at the end of a studio mixing process to save the resulting mix at either CD quality or any other quality, other than perhaps a few more MB of disk space, and how trivial is that these days. The price difference is 99.5% a marketing driven issue only.
It is as astonishing fact that many people happily pay more money for these higher resolution files, but how many of them do you think have done what you just did – double-blind tested themselves and these files to confirm they truly can hear any difference?
Which leads to :
The Most Important Double-Blind Testing of All
Don’t believe and don’t trust anyone who is advocating these higher resolution formats until they have sat themselves down with your double-blind tester and proved to you they truly can hear the differences they are talking about.
Use only your files, because you know their provenance, and insist on these people proving to you their claims. You can load this all onto a laptop and so can travel to meet these people at a place of mutual convenience, you can even confront them at conventions and trade shows and invite them to prove their claims on the spot.
Our guess? Without exception, they’ll either refuse to do so, or else claim that double-blind testing is of no value.
We ourselves have invited leaders in the high-end audio field to double-blind test their claims and prove the validity of the nonsense they assert as truth. Without exception, all have refused.
We’ve heard people say that it is only when you listen to a piece of music for an hour or two that you can really benefit from the higher resolution files. That’s a strange claim to make, and conveniently harder to prove, but tell the person making such a claim that you’re willing to spend the hour or two while they do the double-blind testing.
Advanced Checking for the Best Sound Settings
Some people might claim that a computer and its low-priced sound card is not of sufficiently good quality to allow the subtle extra enhancements of the higher resolution files to become apparent.
There might be one accidental truth in that, but there’s also some nonsense too.
The accidental truth is that Windows might be ‘helpfully’ and automatically resampling the music you are playing through the Foobar ABX comparator. If that is the case, you need to either change the sound drivers in Foobar to avoid Windows interfering in this manner (ie switch away from ‘Direct Sound’ and probably to WASAPI instead), or to do one more thing with your sample sound files.
The easiest approach is to add another step to your sample sound files. After you converted the 192/24 file to 44.1/16, open the 44.1/16 version of the file and reconvert it back to 192/24. Please understand that when you convert the file back up to 192/24 you are not restoring the quality back to the original quality, actually, if anything, the double conversion has slightly harmed the sound quality, making it easier to falsely ‘prove’ the contention that higher resolution music files have a difference in quality that can be heard.
Once you’ve lost the data, through the downconversion, it can never be added back again. You can prove that by opening up the original 192/24 and the new 192/24 files in Audacity, then clicking on Analyse – Plot Spectrum. You’ll see the original file has more sound energy above 20 kHz, the re-re-sampled file has almost none. A simpler test is to look at the file sizes – while the upsampled file will be bigger than the 44.1/16 file, it is also much smaller than the original 192/24 file.
By having Foobar playing two different files with the same ‘built in’ sample rate and depth, Windows will treat them both identically, therefore removing any possible variable in the playback chain.
The nonsense part of blaming your computer’s sound card is to suggest there might actually be any audible difference between your sound card and the electronics in a high-end separate system. You could point to this article which seems to clearly show there is no audible difference between a $2 chip on a built-in sound card and a $2000 external DAC!
On the other hand, you could also feed both files to an external ‘digital analog converter’ and play them back through that. If you have a Fiio personal music player,
you can use that in its DAC mode, and connect it through a USB connection to your computer and listen to the samples through the very high quality DAC in the Fiio player.
We suggest you connect any DAC via the best quality ASIO protocol. In that way, all the sound processing is done through the Fiio or other DAC device and none through the computer (which simply reads the digital file and sends it to the DAC).
This would address any concerns people raise about the quality of the soundcard in your computer preventing you from hearing the differences, whether valid or not. The Fiio also has the benefit of displaying the sampling frequency and bit depth it is playing, so you can see visible proof of the changes in sampling rates mirroring the files you are testing (but make sure you obscure this display during your double-blind testing!).
One more interpretation of the claim that your sound card is harming the music quality. Another way of stating that claim is ‘only really expensive equipment can showcase the sound improvements we suggest might be present, and your equipment isn’t that ‘good’ enough, so with that equipment, you don’t have any need of more expensive audio files’.
Maybe MP3 Files are Good Enough?
Now that you’ve conclusively proved that CD quality is the best you can ever hear, maybe you should do another round of testing to see if you can actually hear the difference between CD quality and MP3 quality. That might be interesting, too.
To do that, simply rip a CD to a FLAC computer file, then use Audacity to export it as an MP3 file, and choose whatever bit rate for the file you want to use. We suggest you experiment with 256 kbps constant bit rate and using stereo rather than joint stereo channel mode (in the options settings when exporting the file). If you can hear a bit of quality loss, try again with 320 kbps constant or 220-260 kbps variable.
On the other hand, if you can’t hear the difference between 256 kbps and the FLAC file, try it again at 192 kbps.
But, as you go down in quality, we’d recommend you choose settings for your music storage that are slightly better than you can hear, rather than decidedly inferior. As we said at the beginning, there’s no longer any real need to force yourself to accept anything less than full CD quality sound.
Summary
So now you know for sure there is no audible difference between a regular CD music file and an esoteric ‘high end’ music file. You know your ears are incapable of hearing the highest frequencies or the greatest dynamic ranges from the direct testing, and you now know there’s no mysterious ‘X factor’ that flows from higher sampling rates and bit depths to the audio quality of the music so recorded.
Now you know these things, spend your audio budget wisely on getting good CD quality recordings, good speakers and headphones, and good music players, and don’t waste any of it on anything where you can’t for sure, double-blind confirmed, hear the difference.
Most of all, marvel at the entire high-end audio industry, and how it seems to be precariously balanced on a myth. How many millions – maybe even how many billions – of dollars each year is wasted on music files and equipment that creates no difference at all? Whatever the answer, there’s one thing you can be sure of. None of that wasted money will be coming out of your pocket!
