Hi all,
I'm conducting a test as part of my degree course to research any perceived differences in analog and digital audio. I'm proposing a double-blind within subjects ABX test with both vinyl and CD stimuli over a number of genres, recorded to 24-bit/96kHz (audible clicks/pops removed) and administered in a controlled listening environment (studio containing Blue Sky 5.1 monitor system and Pro Tools HD with D-Command console).
Like the subject header says, I'm new to all this, so was just wondering if anyone had any advice/constructive criticism for me please?
My advice is don't bother. There will be no problem successfully ABXing vinyl vs, CD, and all you will be left with is asking people's preference for one over the other.
I'm sure this must have been already done, but wouldn't ABXing a digital recording of a vinyl recording against the vinyl recording itself be worthwhile?
I'm sure this must have been already done, but wouldn't ABXing a digital recording of a vinyl recording against the vinyl recording itself be worthwhile?
That's been tried, with an analog signal either going through ADC -> DAC or directly, and they were unable to ABX it.
I'm sure this must have been already done, but wouldn't ABXing a digital recording of a vinyl recording against the vinyl recording itself be worthwhile?
That's been tried, with an analog signal either going through ADC -> DAC or directly, and they were unable to ABX it.
I'm not at all surprised it's been done, and that those were the (worthwhile) results.
I think an interesting research question would be to try to figure out what is behind people's preferences.
Analyze what kinds of distortion are present in playback from Vinyl and simulate each of those with your digital system and see which enhance preference and which detract from it.
It would also be fun to do an experiments where you play back a digitized version of the Vinyl with and without a turntable visible to the listener. I would hypothesize that the visual cue will increase preference for Vinyl.
jj (aka Woodinville on this forum) has spent some time on another forum discussing what must be involved for the proper execution of such a test: http://thewombforums.com/showthread.php?t=14462 (http://thewombforums.com/showthread.php?t=14462)
You should PM him at your earliest convenience.
Thanks for all your replies I really appreciate it.
I'm sure this must have been already done, but wouldn't ABXing a digital recording of a vinyl recording against the vinyl recording itself be worthwhile?
That's been tried, with an analog signal either going through ADC -> DAC or directly, and they were unable to ABX it.
Would you have any idea where I might find info on this?
Quite honestly, the results don't have to be that worthwhile - it's an assignment for my degree, although I'm already beginning to see the advantages of objective testing - it's my first test, and I really just have to come up with A result to support my hypotheses, null or otherwise. Having said that, the test has kind of mutated a bit...it's now looking like a double-blind between vinyl/CD/3 different types of MP3 bitrates, with CD as the reference, and the experimental question being something along the lines of "Have various format changes affected the perceived quality of audio, are we sacrificing quality, and what future implications could this have?"
Obviously articulated a little better in the end I can't stress enough that I'm new to all this!
Thanks again
Thanks for all your replies I really appreciate it.
I'm sure this must have been already done, but wouldn't ABXing a digital recording of a vinyl recording against the vinyl recording itself be worthwhile?
That's been tried, with an analog signal either going through ADC -> DAC or directly, and they were unable to ABX it.
Would you have any idea where I might find info on this?
The JAES paper about listening tests related to high resolution audio (http://drewdaniels.com/audible.pdf)
You should take note that when you compare CD to Vinyl, you're not only comparing the different mediums but also two completely different masters of the same material. This would render any conclusion regarding the effects of the medium invalid.
A solution would be what notat suggested: take the CD as source and simulate the vinyl's distortions. This would require a careful analysis plus the development of a DSP though.
You should take note that when you compare CD to Vinyl, you're not only comparing the different mediums but also two completely different masters of the same material.
Are all CD's "completely different masters of the same material"?
The JAES paper about listening tests related to high resolution audio (http://drewdaniels.com/audible.pdf)
Isn't this the JAES paper about ABXing a digital (SACD/DVD-A) signal going through a 16 / 44.1 ADA process? Is there a JAES study ABXing a CD versus a LP?
You should take note that when you compare CD to Vinyl, you're not only comparing the different mediums but also two completely different masters of the same material.
Are all CD's "completely different masters of the same material"?
I don't know if I understand your question... of course you could take the vinyl master and make a straight digital transfer from that, but noone would ever do that (exept for when the original master hasn't survived), because vinyl masters are designedly distorted to ideally "fit" on the medium. You don't know the exact(!) amount of distortion applied to them, so you can't restore the original tape Master's sound. On the side of the digital master, you have level differences and most probable some EQ (not to mention dynamic compression, digital cleanup etc.) - so you do most certainly come out with a completely different master compared to the vinyl.
The JAES paper about listening tests related to high resolution audio (http://drewdaniels.com/audible.pdf)
From this study;
... one trend became obvious very quickly and held up throughout our testing: virtually all of the SACD and DVD-A recordings sounded better than most CDs—sometimes much better. Had we not “degraded” the sound to CD quality and blind-tested for audible differences, we would have been tempted to ascribe this sonic superiority to the recording processes used to make them.
~ 4 A Note On High-Resolution Recordings
The authors found a difference in the manner in which recordings were engineered. Hi-resolution recordings received much more care and attention. If the analog record was based on a better recording than the digital CD / FLAC / MP3 version, I suspect I would prefer the analog to digital despite the advantages that digital medium has over the analog.
so was just wondering if anyone had any advice/constructive criticism for me please?
May I suggest looking up the scientific method and trying to write down your "scientific hypothesis" and particularly what is being measured. As others have said, if the signal on the record and CD are different then you are not only measuring the influence of more than one parameter you are also probably not in a position to say what they are. This effectively makes the experiment a write off due to poor design. Probably not a good idea if you want reasonable marks.
Recording the vinyl to 24/96 or 16/44 PCM, compressing to MP3 and then comparing record vs MP3 vs PCM would be a more reasonable experiment even though it has probably been done a zillion times before.
A solution would be what notat suggested: take the CD as source and simulate the vinyl's distortions. This would require a careful analysis plus the development of a DSP though.
There are many off-the-shelf plugins that emulate various analog distortions - vinyl, tape, tubes.
You should take note that when you compare CD to Vinyl, you're not only comparing the different mediums but also two completely different masters of the same material.
Are all CD's "completely different masters of the same material"?
*All* is a big word. It's an open, uncontrolled industry. Enough different CDs have been made that every possible combination of mastering steps has been mixed and matched at least once.
If indistry scuttlebutt is true, then some commerical CDs were created from tapes that were intended to be directly cut into LP dies.
This is about as bad as it gets.
The JAES paper about listening tests related to high resolution audio (http://drewdaniels.com/audible.pdf)
Isn't this the JAES paper about ABXing a digital (SACD/DVD-A) signal going through a 16 / 44.1 ADA process? Is there a JAES study ABXing a CD versus a LP?
I don't see why anybody would bother ABXing a CD to a LP because so many variables are uncontrolled.
jj (aka Woodinville on this forum) has spent some time on another forum discussing what must be involved for the proper execution of such a test: http://thewombforums.com/showthread.php?t=14462 (http://thewombforums.com/showthread.php?t=14462)
You should PM him at your earliest convenience.
And such tests are a (*&(*&()&*& to run.
The JAES paper about listening tests related to high resolution audio (http://drewdaniels.com/audible.pdf)
Isn't this the JAES paper about ABXing a digital (SACD/DVD-A) signal going through a 16 / 44.1 ADA process? Is there a JAES study ABXing a CD versus a LP?
I don't see why anybody would bother ABXing a CD to a LP because so many variables are uncontrolled.
Well, unless you make the CD from the LP.
Then you have to get time alignment right to under a millisecond or so, or you'll create cues that people WILL notice sometimes, and you'll get a false positive.
Subjective tests of minute differences tend to be large, time-consuming and difficult.
What about testing for the threshold when e.g. 50%*) of a random selection of listeners can hear any difference when degrading SACD/DVD-A/LP with a CD-like ADC/DAC using a variable number of bits and sampling rate?
If a practical treshold is e.g. 13 bits and 32kHz (using "good" dithering and resampling), then readers of your work can make their choices about how much headroom above that they want.
The fact that 3 persons out there may or may not be able to distinguish 16 bits from 15 bits using specially selected "killer samples" is interesting for some problems. For other problems, knowing that most of us, most of the time cannot reliably distinguish 14 from 13 bits may be most interesting.
If you can do the test PC-based, I really have interest in "ladder-tests", in which the test stimuli are served adaptively based on previous responses, meaning that the "psychometric function" is sampled more heavily close to the point where interesting things happen.
-k
*)I'drather not get into the statistics, but depending on the test, "a significantly larger number of positive outcomes than that suggested by chance"
Subjective tests of minute differences tend to be large, time-consuming and difficult.
Depends on what you call large, time-consuming, and difficult. IME doing the tests is easy. Convincing the audiophiles of the world is another matter.
What about testing for the threshold when e.g. 50%*) of a random selection of listeners can hear any difference when degrading SACD/DVD-A/LP with a CD-like ADC/DAC using a variable number of bits and sampling rate?
If a practical treshold is e.g. 13 bits and 32kHz (using "good" dithering and resampling), then readers of your work can make their choices about how much headroom above that they want.
I don't think it was the first time it was done, but I demonstrated this at my old and now departed www.pcabx.com web site in or about 2002.
The fact that 3 persons out there may or may not be able to distinguish 16 bits from 15 bits using specially selected "killer samples" is interesting for some problems.
Reality is that you can easily do a non-represenative test that shows that most people can distinguish 15 bits from 16 bits, and you can also do any number of representative tests that show that nobody can distinguish 15 bits from 16 bits.
Unfortunately, the world is full of people who *will* do something that amounts to being *no test at all* that "shows" that anybody and everybody can "hear the difference" between 24 bits and 16 bits. Just do a sighted evaluation, prefereably non-level-matched. ;-)
Depends on what you call large, time-consuming, and difficult. IME doing the tests is easy. Convincing the audiophiles of the world is another matter.
Doing a naiive, inconclusive test is often easy, but has limited value.
Doing a conclusive test of something that is allready proven (or easy to prove) may also be easy.
Doing a conclusive test of something that no-one has ever observed before, may be difficult (assuming knowledge Darwinism here).
Doing an inconclusive test that is solid enough to make believers question their beliefs is very hard.
I still believe that the more marginal a phenomenon is (1 in 1000 can hear it in 1 of 1000 sound-clips etc), the more time-consuming it is to prove its existence, and the less value the results will have in real-life (except used as arguments in audiophile debates).
-k
Depends on what you call large, time-consuming, and difficult. IME doing the tests is easy. Convincing the audiophiles of the world is another matter.
Doing a naiive, inconclusive test is often easy, but has limited value.
One would have to be pretty ignorant of the facts to say that the general run of DBT debunking golden ear mysticism has been naive or inconclusive.
Doing a conclusive test of something that is allready proven (or easy to prove) may also be easy.
What rational people find to be highly conclusive tests have no effect on people who are primarily motivated by raw emotion and irrational thinking.
Doing a conclusive test of something that no-one has ever observed before, may be difficult (assuming knowledge Darwinism here).
What is there about audio that has not already been observed?
Doing an inconclusive test that is solid enough to make believers question their beliefs is very hard.
What rational people find to be highly conclusive tests have no effect on people who are primarily motivated by raw emotion and irrational thinking.
There are a virtually unlimited number of examples of irrational behavoir in the world. Mao is dead, but there are still Maoists!
I still believe that the more marginal a phenomenon is (1 in 1000 can hear it in 1 of 1000 sound-clips etc), the more time-consuming it is to prove its existence, and the less value the results will have in real-life (except used as arguments in audiophile debates).
The 1 in 1000 audiophiles and the 1 in 1000 sound clips almost certainly don't exist. By now, 10,000s or 100,000s of each have been tested. It is like tested unplugged toasters. Out of a million conventional toasters, how many will toast bread without being plugged in? None! How many runners can run a 2 minute mile? None! There are things that are simple to observe, even naively observe, and they simply never happen!
If you want to test 2 million toasters looking for that one toaster that will toast bread without being plugged in, please be my guest! But do so with your own time and money. Don't waste my time with your hopes and dreams of being successful at doing the impossible. No matter what your high school coach told you, there are some things that are simply impossible. Actually, a great many of them exist.
What rational people find to be highly conclusive tests have no effect on people who are primarily motivated by raw emotion and irrational thinking.
No matter what my personal beliefs are, and what rules I follow in my day-to-day life, I still think that science should be conducted based on high standards. It is impossible to prove that God does not exist. We sceptics can use all of our brain-power, reasoning, arguments, etc, but proving that something does not exist is impossible.
What we can do is conduct tests that tries to prove Gods existance, and plead with believers that they do the same. I believe that the current state in the "does God exist?" question, is that science cannot prove either, and those that choose to believe should do so based on faith, not science. I have no problem with "faith" and "science" existing as two orthogonal cultural systems that can work independently (
"I dont care about physics, using this expensive cable makes my experience more pleasant"). Problems do occur when people try to mix them in strange ways (
"This cable sounds better than the other due to the skin-effect").
The 1 in 1000 audiophiles and the 1 in 1000 sound clips almost certainly don't exist.
I agree, many "audiophile phenomena" almost certainly dont exist as physical, audiological phenomena. But Sir Rayleigh was almost certain that Darwin was wrong based on physically sound theories of the earth being no more than 6000 years old. Physicists around 1900 was almost certain that all there was to discover in physics was allready discovered. Etc. I think that the sceptic making false claims about what science can disprove is making the same disservice to the public as the believer making false claims about what science has proven.
-k
=But Sir Rayleigh was almost certain that Darwin was wrong based on physically sound theories of the earth being no more than 6000 years old.
What?
Citation please.
I do not believe there has been a "scientific" theory on the age of the earth which placed the formation less than 75,000 years ago - and that was a "weak" experiment based upon cooling (and far and away shorter than any other theory prior or since).
My point, because I am not attempting to drag this thread off topic despite appearances, is that I think you are mischaracterizing science, removing the fulcrum on which your argument rests.
EDIT: You appear to be falsely attributing a
faith in the "6000" year calculation by Bishop Ussher, to a scientist (not sure exactly who "Sir Rayleigh" is - Lord Rayleigh (a contemporary of Darwin) certainly would not have disputed Darwin's back-calculated geological age). Perhaps you mean Lord Kelvin? (William Thomson) He actually disputed Darwin on the age of the earth - but despite his dominance in thermodynamics his "science" on this matter was
not falsifiable and therefore a
weak counter to Darwin - though it did cause Darwin hesitancy. Though for every physicist who had trouble accepting an earth older than ~100,000 years old you had a geologist who had good reason to believe the earth was millions of years old, at least.
I stand corrected.
-k
The 1 in 1000 audiophiles and the 1 in 1000 sound clips almost certainly don't exist. By now, 10,000s or 100,000s of each have been tested. It is like tested unplugged toasters. Out of a million conventional toasters, how many will toast bread without being plugged in? None! How many runners can run a 2 minute mile? None! There are things that are simple to observe, even naively observe, and they simply never happen!
Science prides itself on the foolproofness of repeated, consistent observations. Unfortunately, there are some fields masquerading as science without living up to its reasonable demand of reproducibility, and usually tries to explain away its liability by appealing to the short lifespan of humans.
On the other hand, there are some fields (namely, audio) in which a great many people think are completely exempt from scientific empiricism when they are not. It's a worse-than-inconsistent parade of foolishness.
Kudos to the areas of life in which empiricism doesn't work axiomatically, and they don't pretend that it does. That's another form of consistency.
I agree, many "audiophile phenomena" almost certainly dont exist as physical, audiological phenomena. But Sir Rayleigh was almost certain that Darwin was wrong based on physically sound theories of the earth being no more than 6000 years old. Physicists around 1900 was almost certain that all there was to discover in physics was allready discovered. Etc. I think that the sceptic making false claims about what science can disprove is making the same disservice to the public as the believer making false claims about what science has proven.
The belief that the earth is no more than 6000 years old has never been based on physically sound theories about geology, biology, chemistry, physics or anything else that we would normally call "physical science". The belief that the earth is no more than 6000 years old has *always* been based on speculative analyses of Biblcal geneologies. People simply counted up entries in lists of Biblical names. In some cases they added bibilcal accounts of people's ages and in other cases they used speculative numbers to come up with estimates of the time since Adam and Eve. This is a questionable procedure, even on just the grounds of Biblical scholarship.
At this time we know about some of Darwin's mistakes and some of Rayleigh's mistakes. We're never going to know for sure exactly what the exactly the right way is to treat Biblical geneologies is because the information is so incomplete. We have far better evidence today from the hard sciences, but even much of that is inferential.
It seems to be safe to say that the Earth is far older than 6000 years, and everybody should move on based on that. The creationists should stick to science that they understand and stop piling speculation on top of speculation in order to preserve speculative analyses of geneologies that were never written with the intent that they be proof of the age of the Earth. The Darwinists should stop trying to prove the Bible wrong, and the Biblicists should stop trying to prove Science wrong. It all fits if you take the blinders off!
Hi all,
I'm conducting a test as part of my degree course to research any perceived differences in analog and digital audio. I'm proposing a double-blind within subjects ABX test with both vinyl and CD stimuli over a number of genres, recorded to 24-bit/96kHz (audible clicks/pops removed) and administered in a controlled listening environment (studio containing Blue Sky 5.1 monitor system and Pro Tools HD with D-Command console).
Like the subject header says, I'm new to all this, so was just wondering if anyone had any advice/constructive criticism for me please?
You should take note that when you compare CD to Vinyl, you're not only comparing the different mediums but also two completely different masters of the same material.
Are all CD's "completely different masters of the same material"?
Digital and analog audio go way beyond the scope of LPs and CDs. So you have that.
If you are interested in doing blind listening tests with vinyl and CD I suggest the following. Contact RTI or Sterling Sound or Doug Sax or Bernie Grundman and see if you can set up some sort of listening test with one of them. These are all individuals and facilities that do audiophile mastering in both media. It is *possible* through one of these to facilitate a level matched, time synced double blind ABChr listening test using the master source as a reference. You could also follow it up with a level matched time synced AB preference comparison with no reference as a follow up.
You have to understand though that the making of a commercial CD or LP is not a singular process. There are variables in each path so under any circumstance you are ultimately comparing a CD and vinyl made from particular cutting lathes/AD converters and pressing plants. If you took your master tape to any number of facilities to make a commercial CD or LP there will be differences in the end product. I'm not going to argue whether or not all those differences are audible but they will be physically different. If you have an analog master just the tape deck used at the given facility and how it is set up and maintained will make a difference. A lot of variables here.
The belief that the earth is no more than 6000 years old has never been based on physically sound theories about geology, biology, chemistry, physics or anything else that we would normally call "physical science". The belief that the earth is no more than 6000 years old has *always* been based on speculative analyses of Biblcal geneologies. People simply counted up entries in lists of Biblical names. In some cases they added bibilcal accounts of people's ages and in other cases they used speculative numbers to come up with estimates of the time since Adam and Eve. This is a questionable procedure, even on just the grounds of Biblical scholarship.
My memory served me badly. Instead of going into details about what I do not know very well anyways, I suggest to extract the moral of my story: At the given state of physics at Darwins time, one could not explain the earths age being what we now believe it to be, and what Darwin thought necessary to allow for natural selection to produce the living things that he could see. Some scientists used this as an Argument against Darwinism. Later on, the proper physical mechanisms was found.
We can never now what we dont know. The best we can hope is to make models that fits our (flawed) observations of the world. If we should ever find irrefutable evidence of God, E.T or the utility of hifi-cables made out of kevlar, we should report them. Until that, we should explain that we cannot prove the existence (or the opposite) of either, and those so inclined could announce their scepticism against such evidence ever being presented.
-k