Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: What's the problem with double-blind testing? (Read 244980 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

What's the problem with double-blind testing?

Reply #325
It's probably worth noting that the tenure track can be 25+ years long in some universities. I knew a guy who joined a dept as asst/associate professor (I forget which) in the mid 80s, and only got his tenure after I graduated in '02.

It's a really messed up system and I wouldn't belittle any professor who wasn't tenured.



Especially if they own a gun, these days.

What's the problem with double-blind testing?

Reply #326
Some people might assume from reading the post below that if they placed a PAL DVD of a Hollywood movie onto a standard PAL DVD player it should play at correct speed and pitch, i.e. the speed and pitch at which the actors spoke, or the orchestral instruments were played.

This cannot be assumed.  The technology for motion interpolation to create a relatively artifact-free 25 video frames per second from a source of 24 video frames per second (the traditional speed at which movies are filmed) has only become available in recent years, and is still not perfect.

As an alternative approach, leaving the video as is [i.e. simply leaving each video frame intact and allowing the 24fps to play at 25fps] but processing the audio to reduce the apparent pitch back to original pitch, is complex and causes audible artifacts.  With a complex source, e.g. an orchestral sound track, it may be recommended to leave the pitch 0.7 semitones sharp.  In any event the tempo is still wrong with this approach, even if the pitch has been 'corrected'.

On a related matter, in Australia any number of sitcoms and movies produced in the USA are currently broadcast on digital TV with a simple 25/24 speedup, causing an increase in pitch of the sound.  Perhaps surprisingly, most people do not notice this speedup of just over 4%.

But in an ABX test with a short gap between listening to a correct speed  version and a version that is 0.7 of a semitone sharp, I am sure these same people (most of them anyway) would notice the set of differences.  [The set of differences includes tempo, timbre, and vowel formant frequencies.] 

This is a very interesting practical example of a difference in sound that typically goes unnoticed in real life; despite being quite a big difference in an ABX comparison.  [If anyone reading doubts how different the sound is, use Audacity to speed up an audio clip by 4% and compare the processed sound with the original clip.]

As a result of the many hours in my childhood spent watching movies on television with a 4% speedup, watching a movie at correct speed at the cinema, on an NTSC DVD, or on a Blu-ray DVD, makes the sound seem unusually solid and real.

I do get annoyed with the 4% speedup when watching TV, or my few PAL DVDs, but I am in the minority.  Most people remain blissfully unaware of it.


I finally found an example : in Europe, when we watch a movie in the theatre, then several monthes later on DVD, the sound track is speeded up, because the movie runs at 25 fps (video PAL) instead of 24 fps (theatre).


Just because the FPS change, there's no necessity for the pitch or timing to change. People working with films and video have been managing this situation for several decades.

This article explains how FPS changes are have been managed for decades so that they don't affect the length or running speed of film or video:

http://www.zerocut.com/tech/pulldown.html

In the old days, the production paths for sound and video were separated, the video or film was altered using pull down techniques, and the audio was added back into the finished product with  no changes. Or, the sound was added back into the finished product with a different FPS from a more origional source.

These days, most video production software manages things like this automagically.  You just specify the FPS of the finished product, and it is produced in accordance with your spec under the covers, as it were.

In short, your example has a rather serious flaw.

What's the problem with double-blind testing?

Reply #327
If you've heard the content before, the pitch shift is quite obvious. Even if it's weeks later that you hear it again.

If it's new to you, then far less so - e.g. if you don't know what the actor's voice really sounds like, or the music track really sounds like, then the pitch shift is meaningless to you, and won't be detected.

Cheers,
David.

What's the problem with double-blind testing?

Reply #328
I finally found an example : in Europe, when we watch a movie in the theatre, then several monthes later on DVD, the sound track is speeded up, because the movie runs at 25 fps (video PAL) instead of 24 fps (theatre).


Just because the FPS change, there's no necessity for the pitch or timing to change. People working with films and video have been managing this situation for several decades.

This article explains how FPS changes are have been managed for decades so that they don't affect the length or running speed of film or video:

http://www.zerocut.com/tech/pulldown.html

In the old days, the production paths for sound and video were separated, the video or film was altered using pull down techniques, and the audio was added back into the finished product with  no changes. Or, the sound was added back into the finished product with a different FPS from a more origional source.

These days, most video production software manages things like this automagically.  You just specify the FPS of the finished product, and it is produced in accordance with your spec under the covers, as it were.

In short, your example has a rather serious flaw.

It's in fact your example that has a serious flaw, because the method you posted a link to (2:3 pulldown) is for NTSC and not for PAL. All PAL movie DVDs I've encountered so far (old and new) have been speed-up. This means the tempo of the audio always has been speed-up (with or without pitch correction, personally I prefer without). From capturing I know that long ago movies in PAL were broadcoasted with fieldshifting which didn't require to change the audio tempo, but nowadays this technique isn't used anymore, at least not with the movies I have captured.

If you've heard the content before, the pitch shift is quite obvious. Even if it's weeks later that you hear it again.

Personally, I have to disagree with this. My tonal memory is pretty awful even though I'm playing piano and guitar since I was 8.
"We cannot win against obsession. They care, we don't. They win."

What's the problem with double-blind testing?

Reply #329
This clip Samples combined - HOUSE MD contains three samples of the opening music to the TV series HOUSE MD as broadcast in Australia on Digital TV (terrestrial):

1. Correct speed (as broadcast in 2008)
2. With PAL speedup (as broadcast in 2007)
3. Version 1 again

Although the difference is quite marked when the samples are contrasted with each other without delay (e.g. with an immediate ABX test), most people watching 50fps TV do not notice whether a particular TV episode is being broadcast with PAL speedup or not.

Here is a post I made in 2008:

[blockquote]I settled back to enjoy an episode of House-MD tonight, recorded earlier this week with Vista Media Centre.

Recalling that some of the episodes last year were 24p but broadcast at 25fps (50i), I chose to play the episode with ZoomPlayer, which is now permanently configured on my HTPC to use ReClock, and with a fixed set Reclock rate of 24fps.

I immediately noticed the video was not as smooth as it usually is with Reclock running. After a while I noticed that some of the actors had particularly deep voices that didn't seem quite natural. I then switched over to playing the episode in Vista Media Centre and it sounded natural. A quick check with VideoRedo revealed that the episode was interlaced.

It was clear that the episode this week was a correct speed 50i version, and not a sped-up 24p version.[/blockquote]
Similarly, most people do not notice the 25/24 speedup usually present in PAL DVD releases of Hollywood movies.  When I have raised the topic socially of "PAL speedup" with television and with PAL DVDs, not one person has expressed awareness of the issue.  And yet trying to play a PAL DVD and its soundtrack CD together reveals an obvious difference.  The PAL DVD races ahead, and is higher in pitch. This difference is not hard to notice!

If you've heard the content before, the pitch shift is quite obvious. Even if it's weeks later that you hear it again.

Personally, I have to disagree with this. My tonal memory is pretty awful even though I'm playing piano and guitar since I was 8.

With an isolated note,  0.7 of a semitone is too small a difference for long term memory, other than for the very small percentage of people who sense pitch absolutely and can name any musical note by ear ("perfect pitch").

With connected music [i.e. not just a single test tone] some of us can recall how a piece sounded and a 4% difference will be noticed because of its cumulative effect: tempo and timbre.  With the spoken voice, vowel formant frequencies tend to sound too light and high if the playback is 4% fast.

Some people can perceive the unnatural thinness and hurried quality of a 4% speedup without having heard the speech or music previously, and despite not possessing perfect pitch. I happen to fall into that category.

A verification method if you suspect audio has been subjected to 25/24 speedup, is to subject it to 24/25 slowdown (i.e. a 4.000% slowdown).  It will then either sound right, or too low in pitch and too slow.  Another method is to wait for music in the audio, sing a note of the melody and go to a piano.  The note you sing will lie in between two notes, about 0.7 sharp of one note, and 0.3 of a semitone flat with respect to the note above, if the music has been sped up by 25/24, i.e. by 4.166%.

The PAL speedup is frequently used to accommodate the fact that analogue and digital TV systems in PAL countries operate at 25fps or 50fps, rather than at 24fps or 48fps. Film is conventionally captured at 24fps.

In the United States, Japan and other NTSC countries, a 3:2 pulldown process enables 24fps film to run at 23.976fps, from which is derived "59.94fps interlaced" for broadcast, a standard that has been used since the introduction of color in the 1950s.  59.94fps interlaced is more commonly referred to as 60fps or 30fps for simplicity.  [The speed and pitch difference factor of 1000/1001 is practically negligible for human hearing.]

What's the problem with double-blind testing?

Reply #330
With the spoken voice, vowel formant frequencies tend to sound too light and high if the playback is 4% fast.
Yes - or more simply, it sounds like a different person talking - sometimes a younger person - sometimes just plain wrong.

Quote
A verification method if you suspect audio has been subjected to 25/24 speedup, is to subject it to 24/25 slowdown (i.e. a 4.000% slowdown).  It will then either sound right, or too low in pitch and too slow.
True - because we're supposedly more sensitive to "slightly too slow/low" than we are to "slightly too fast/high".

Quote
Another method is to wait for music in the audio, sing a note of the melody and go to a piano.  The note you sing will lie in between the notes, about 0.7 sharp of one note, and 0.3 of a semitone flat the semitone above, if the music has been sped up by 25/24, i.e.  by 4.166%.
I don't think many people can sing this well!  Other things get in the way with this comparison though - instruments are not guaranteed to be "in tune" (your piano, and/or the ones on the recording) - music is not guaranteed to be played into the soundtrack at the correct speed. Also different releases of the same music (especially older music) sometimes have slightly different speed.

Cheers,
David.

What's the problem with double-blind testing?

Reply #331
Other things get in the way with this comparison though - instruments are not guaranteed to be "in tune" (your piano, and/or the ones on the recording) - music is not guaranteed to be played into the soundtrack at the correct speed. Also different releases of the same music (especially older music) sometimes have slightly different speed.

True. But it is a very quick and useful method if listening to a tv series or movie produced in say the last 20 years, and despite the slighly different pitch for concert A used by some European orchestras. [My piano gets tuned once a year and drifts very little; nothing like 0.3 of a semitone. And many people these days have electronic keybords.]

Another method is to play along with a piano and try to find a matching note. A 0.3 of a semitone discrepancy will frustrate the attempt. 

Also some pieces of music are usually played in a particular key.  If the TV broadcast is about three-quarters of a semitone sharp of the usual key, this will be a strong indicator.

Unforunately PAL DVDs of movies include no warning about the speedup.  It is presumably considered too minor a matter to comment on ...

What's the problem with double-blind testing?

Reply #332
I generally don't notice PAL speedup at all. The only times I notice it are when watching favourite TV episodes that I have been used to in PAL then getting the NTSC version.

What's the problem with double-blind testing?

Reply #333
For TV shows it depends, because often they are shot in 30i and then converted via field blending which doesn't affect the audio.
"We cannot win against obsession. They care, we don't. They win."

What's the problem with double-blind testing?

Reply #334
Hi everybody -

Thanks to Kees for letting me know this thread came back to life - though I didn't know that my academic job was under discussion. Hah. Needless to say, I have been doing other stuff. My brother ("bryant" the developer of WavPack) and I still plan on doing the set of experiments we talked about here 3 years ago. Apparently, we like to think about these things for awhile! Truth is, we are just busy. I'm quite active in research but that work is not on psychoacoustic testing methods and auditory processing.

In the experiments we are devising, all we intend to demonstrate is that decision-based paradigms such as ABX might suggest perceptual equality between different stimuli at the level of conscious awareness, while implicit measures such as reaction time tasks might reveal consequential processing differences in those same stimuli. Additionally, long term exposures and subsequent judgments might reveal preference differences between stimuli that are also not perceptually different using ABX. If this is the case (and I suspect it very well could be), then short term and/or long-term processing differences might be at the root of some people's apparent discomfort associated with lossy codecs, and even 44.1/16 bit digitization. I have already explained the ideas quite thoroughly before, so check out the early pages of this thread. But my links were dead. I copied one post from Nov 2005 and fixed the links (I just learned you can't edit after 1 hour now!), so now those papers are available if you'd like to read them.

Keep in mind that this is not some ultimate rejection of ABX, and admittedly the effect (if there) will be small. But a small effect is all you need to provide some explanation for various audiophile claims. It's just an attempt at providing experimental support for the idea that conscious decision processes that ABX paradigms rely on might not always be the last word in auditory perception. A similar effect happens in visual perception. I explained the flicker rate studies before. Same idea: subjects can't detect the difference in flicker rates in two CRT presentation conditions, but in one condition the rate disrupted eye movements and caused subjects to re-read many characters. No awareness of the experimental manipulation, significant difference in reaction time tests, and we know flicker rates have measurable effects on people's comfort. The relevant information is below threshold, but has an impact that is only measurable using certain tasks.

Someone mentioned doing brain scans - our expectation would be that given the right measurement (tall order given the sophistication of such technology even now), one might find neural differences. As I described earlier, there are neural correlates to distinct sensory and decision processes. I think it is quite plausible that greater neural activity could be associated with processing compressed audio relative to uncompressed counterparts, and ABX decision tasks would not reflect it. Just saying it's possible. And if so, rather than calling all audiophile stupid nutcases, we would have at least one explanation that could help resolve this long standing debate.


I have compiled a number of articles that provide anybody here resources and empirical support for the various claims David and I have been making. I claimed that there is a distinction between sensory and decision processes. This recent Nature article provides empirical support for this. Cognitive scientists have understood this distinction for decades, but here is recent evidence demonstrating the neurological basis for it. This is a fundamental issue in our criticism of the validity of ABX testing as the last word on ultimate differences in the auditory processing of lossy audio. Acoustic differences that could matter for a listener’s overall experience might not be audible.

Here are two articles that show the difference between decision processes linked to the discrimination of stimuli versus sensory processes that are affected when there is no discrimination. There is clear enhanced brain activity that occurs during the presentation of noisier stimuli even when it cannot be distinguished from a less noisy counterpart.

Effects of Low-Pass Noise Masking on Auditory Event-Related Potentials to Speech
The Effects of Decreased Audibility Produced by High-Pass Noise

Woodinside claimed that no filtering processes happen after the ear. This is definitely incorrect, as filtering processes do happen in the auditory cortex (one of the many analogues between auditory and visual processing), but I’ll assume he meant that the relevant filtering for audio codecs involves processes that happen only in the ear. This is fine, as there’s no reason to distinguish peripheral (inner and outer ear) processes from central processes (auditory cortex) when talking about metabolic costs of neural effort. The neurons must work harder (wherever this happens) to resolve the signal and encode it for later processing. And this does not deal with the likely possibility that the resulting representations might differ depending on stimulus quality (not a necessary feature of our argument).

There has been quite a bit of skepticism regarding the claim that compressed or degraded stimuli require more effort to process. I could cite a list of papers a mile long showing reaction time increases as a function of stimulus complexity (which heavily imply that processing difficulty increases), but there is also work showing that neurons respond to increased attention demands. Additionally, there seem to be specialized neural systems for resolving degraded signals, which in turn contribute to learning. These processes require effort, and have associated metabolic costs. This is exactly the sort of thing we have been arguing.




What's the problem with double-blind testing?

Reply #335
I think it is quite plausible that greater neural activity could be associated with processing compressed audio relative to uncompressed counterparts, and ABX decision tasks would not reflect it. Just saying it's possible.

Consider also that the reverse might apply: lesser neural activity.

I find if listening to conventional Dolby 5.1 from a DVD that the sound is "neater" than an uncompressed version.  The compression removes less important sounds (potential distractions) and may thus help guide human hearing towards perception of the "significant and relevant" audio content.

What's the problem with double-blind testing?

Reply #336
I think it is quite plausible that greater neural activity could be associated with processing compressed audio relative to uncompressed counterparts, and ABX decision tasks would not reflect it. Just saying it's possible.

Consider also that the reverse might apply: lesser neural activity.

I find if listening to conventional Dolby 5.1 from a DVD that the sound is "neater" than an uncompressed version.  The compression removes less important sounds (potential distractions) and may thus help guide human hearing towards perception of the "significant and relevant" audio content.


You never know, but... the Dolby 5.1 differences are above threshold -- we're talking about differences you can't necessarily hear. Still, I agree that it's unclear what the relationship is between subjective sound judgment and brain activity.



What's the problem with double-blind testing?

Reply #338
Assuming there actually is one?
For simplicity let's assume there is a brain
What puzzles me is that training is an essential aspect of perceptual DBT (audio, wine etc.) and can make a difference in the results. An athlete can train his/her body for (visibly) better performance, but what makes a trained ear ? We probably can't train to hear things beyond the physical limitations of our ear. I've always assumed that the trained ear has to do with brain activity. What else can it be ?

What's the problem with double-blind testing?

Reply #339
Assuming there actually is one?
For simplicity let's assume there is a brain
What puzzles me is that training is an essential aspect of perceptual DBT (audio, wine etc.) and can make a difference in the results. An athlete can train his/her body for (visibly) better performance, but what makes a trained ear ? We probably can't train to hear things beyond the physical limitations of our ear. I've always assumed that the trained ear has to do with brain activity. What else can it be ?


IME ear training is a misnomer. What needs training is the brain.

What's the problem with double-blind testing?

Reply #340
I finally found an example : in Europe, when we watch a movie in the theatre, then several monthes later on DVD, the sound track is speeded up, because the movie runs at 25 fps (video PAL) instead of 24 fps (theatre).


Just because the FPS change, there's no necessity for the pitch or timing to change. People working with films and video have been managing this situation for several decades.

This article explains how FPS changes are have been managed for decades so that they don't affect the length or running speed of film or video:

http://www.zerocut.com/tech/pulldown.html

In the old days, the production paths for sound and video were separated, the video or film was altered using pull down techniques, and the audio was added back into the finished product with  no changes. Or, the sound was added back into the finished product with a different FPS from a more origional source.

These days, most video production software manages things like this automagically.  You just specify the FPS of the finished product, and it is produced in accordance with your spec under the covers, as it were.

In short, your example has a rather serious flaw.

It's in fact your example that has a serious flaw, because the method you posted a link to (2:3 pulldown) is for NTSC and not for PAL. All PAL movie DVDs I've encountered so far (old and new) have been speed-up. This means the tempo of the audio always has been speed-up (with or without pitch correction, personally I prefer without). From capturing I know that long ago movies in PAL were broadcoasted with fieldshifting which didn't require to change the audio tempo, but nowadays this technique isn't used anymore, at least not with the movies I have captured.


You've missed my point. My point is that in 2010 there is no need to accept a pitch shift in the accompanying audio when changing the frame rate of A/V media. The general problem has been around for decades, and the means for solving it have only become more sophisticated, economical and undetectable over the years.  You have fastened your attention on a particular implementation (2:3 pulldown) of a particular technique (pulldown) and mistakenly invalidated the general comment because of some variations in impementations of that technique. There are also other techniques.

If perchance a piece of video actually has  its running time changed, there is still no need to change the pitch of the accompanying audio. For decades there have been means for changing the running time of audio without changing its pitch. Currently, this is a standard feature of a number of reasonably-priced audio editing programs. I personally do this sort of thing every once in a while.

So, if you are getting PAL media whose pitch is different from the corresponding NTSC media, the core problem is that the production people did not avail themselves of any number of different approaches that are available for a reasonable cost and in general use, at least by people who have a clue and have the resources to exercise that clue.

Furthermore if this problem really bugs you, you could fix it for yourself because the tools for doing so aren't all that expensive or hard to use.

This seems like a good time to shoot or at least beat up the messenger - the messenger being the video media distributor who is foisting these A/V kludges off on you. He's taking your money and feeding you crap. Tell him that you are madder than %$## and won't take it any more! ;-)

What's the problem with double-blind testing?

Reply #341
I think it is quite plausible that greater neural activity could be associated with processing compressed audio relative to uncompressed counterparts, and ABX decision tasks would not reflect it. Just saying it's possible.

Consider also that the reverse might apply: lesser neural activity.

I find if listening to conventional Dolby 5.1 from a DVD that the sound is "neater" than an uncompressed version.  The compression removes less important sounds (potential distractions) and may thus help guide human hearing towards perception of the "significant and relevant" audio content.


You never know, but... the Dolby 5.1 differences are above threshold -- we're talking about differences you can't necessarily hear. Still, I agree that it's unclear what the relationship is between subjective sound judgment and brain activity.


Right, Dolby 5.1 can cause audible degradation all by itself. It is not as sonically accurate as 48 KHz 16 bit PCM.

Dolby digital is itself a perceptual coding technique.  It is not necessarily even a outstandingly good perceptual coding technique. It is reliatively old, but apparently the coders in general use have been upgraded somewhat over the years. There are blind tests that show that audible degradation due to it can be heard, at least in some cases.

Cascading different percpetual coding techniques can lead to unexpectedly bad sound quality losses.

What's the problem with double-blind testing?

Reply #342
So, if you are getting PAL media whose pitch is different from the corresponding NTSC media, the core problem is that the production people did not avail themselves of any number of different approaches that are available for a reasonable cost and in general use, at least by people who have a clue and have the resources to exercise that clue.
No, most mastering houses have at least one implementation - some have several.

It's just that sometimes none of them are judged to be good enough.

Also, for cheaper releases, a simple resample (changing time and pitch) is a known quantity - whereas an algorithm which changes time but maintains pitch may introduce content-specific artefacts. If there's no time to check this carefully, it's better to introduce a known flaw (wrong pitch), rather than introduce a potential serious unknown flaw (e.g. mangling the sound of some strings).

It's a bit like noise reduction - it is possible to remove hiss from older recordings - but sometimes it's better not to, given what's available. Just because someone chooses not to doesn't mean that they can't - not doing something can be a sign of great care, rather than great incompetence.


Anyway, many PC-based home theatre enthusiasts are thankful for 25p masters with a speed up - they can easily slow them down again and get the original sound - something which is impossible with pitch "corrected" versions!

Doesn't matter with BluRay - most are 24p even in "PAL" countries.

Cheers,
David.

What's the problem with double-blind testing?

Reply #343
@duff,

The correlate of the flicker example you give would be finding a task which was impaired by lossy coding, despite the lossy coding being inaudible in an ABX test. e.g. "write down the words that are spoken", or "identify the instrument", or "sing the tune" etc.

I'm not convinced by the flicker example anyway - whether you see a difference between different flicker rates depends on whether the flickering device is in the centre or periphery of your field of view, whether your eyes are moving or stationary, and whether your eyes are following something (eye-tracked motion), or moving across something that isn't itself moving.

Hence if you can't ABX a difference when simply staring at a screen, but can perceive a difference when reading from that screen, it doesn't necessarily tell you anything about ABX vs reading - because you've introduced another variable (eye movement) which you would expect to interact with the flicker anyway.


Not sure about the ba/da examples you posted. I get the point, but it could be more complex than is suggested. After all, no one is claiming that the subjects can't consciously differentiate between the other variable in the test - the LPF of the noise - though I can't think why this should matter.

There are (possibly) four categories:
1. differences we can short-term ABX
2. differences we cannot short-term ABX directly, but which yield different results in some task
3. differences we cannot short-term ABX directly, which do not yield different results in any known task, but which reveal consistent differences in some kind of brain respond
4. differences which are undetectable on all three counts

The thing with (2) is that the task itself becomes the ABX test - you merely have to ask the correct question.
The thing with (3) is that, if there really is no question or task or anything else that can reveal any kind of conscious knowledge of the difference, over any time scale, why would any one care when listening to music? (It's nice research, but...!)
Obviously (4) self evidently doesn't matter.

Cheers,
David.

What's the problem with double-blind testing?

Reply #344
For simplicity let's assume there is a brain

Right about now I'm wondering where mine was when I posted that.

What's the problem with double-blind testing?

Reply #345
@duff,

The correlate of the flicker example you give would be finding a task which was impaired by lossy coding, despite the lossy coding being inaudible in an ABX test. e.g. "write down the words that are spoken", or "identify the instrument", or "sing the tune" etc.

I'm not convinced by the flicker example anyway - whether you see a difference between different flicker rates depends on whether the flickering device is in the centre or periphery of your field of view, whether your eyes are moving or stationary, and whether your eyes are following something (eye-tracked motion), or moving across something that isn't itself moving.

Hence if you can't ABX a difference when simply staring at a screen, but can perceive a difference when reading from that screen, it doesn't necessarily tell you anything about ABX vs reading - because you've introduced another variable (eye movement) which you would expect to interact with the flicker anyway.

There are (possibly) four categories:
1. differences we can short-term ABX
2. differences we cannot short-term ABX directly, but which yield different results in some task
3. differences we cannot short-term ABX directly, which do not yield different results in any known task, but which reveal consistent differences in some kind of brain respond
4. differences which are undetectable on all three counts

The thing with (2) is that the task itself becomes the ABX test - you merely have to ask the correct question.
The thing with (3) is that, if there really is no question or task or anything else that can reveal any kind of conscious knowledge of the difference, over any time scale, why would any one care when listening to music? (It's nice research, but...!)
Obviously (4) self evidently doesn't matter.

Cheers,
David.


The point of the flicker example is just that a reaction time test (and analysis of eye movements) revealed processing differences that did not apparently affect conscious awareness - but conscious decision processes are what ABX tests rely on. Subjects' eyes were moving because they were reading - but regardless, the idea is just that they didn't notice differences even though there were measurable differences in their behavior - and those measurable differences can have long-term impacts such as increased fatigue and discomfort that short term tests won't capture.

Nevertheless, it is only analogous, and they didn't do the ABX test. It's just an example to give you an idea of how processing can have consequences that are only measurable through implicit means. The correlate of the flicker example you suggest is right along the lines of what we would like to try. A positive result in something like that would be an issue for those who claim ABX testing is the end all.

As for your categories, we are after #2. But I wouldn't quite say the task "becomes the ABX test" unless you mean the task becomes the means by which we can uncover any kind of processing difference (which is different from perceptual difference). The limitation of ABX in this sense is that it's reliant on conscious reports. ABX might fail to reveal a processing difference - and any processing differences might be contributing to various phenomena people associate with lossy codecs and 44.1/16 bit digitization.

What's the problem with double-blind testing?

Reply #346
You've missed my point. My point is that in 2010 there is no need to accept a pitch shift in the accompanying audio when changing the frame rate of A/V media. The general problem has been around for decades, and the means for solving it have only become more sophisticated, economical and undetectable over the years.  You have fastened your attention on a particular implementation (2:3 pulldown) of a particular technique (pulldown) and mistakenly invalidated the general comment because of some variations in impementations of that technique. There are also other techniques.

I'd rather accept a pitch shift than a ratty pitch correction, and this still does not solve tempo.

Quote
If perchance a piece of video actually has  its running time changed, there is still no need to change the pitch of the accompanying audio. For decades there have been means for changing the running time of audio without changing its pitch. Currently, this is a standard feature of a number of reasonably-priced audio editing programs. I personally do this sort of thing every once in a while.

So, if you are getting PAL media whose pitch is different from the corresponding NTSC media, the core problem is that the production people did not avail themselves of any number of different approaches that are available for a reasonable cost and in general use, at least by people who have a clue and have the resources to exercise that clue.

Sorry, but it just seems you are clueless about the PAL world. Sped-up DVDs are not an anomaly at all, they are the standard.
"We cannot win against obsession. They care, we don't. They win."

What's the problem with double-blind testing?

Reply #347
This seems like a good time to shoot or at least beat up the messenger - the messenger being the video media distributor who is foisting these A/V kludges off on you. He's taking your money and feeding you crap. Tell him that you are madder than %$## and won't take it any more! ;-)

Unfortunately, it is standard industry practice to release PAL DVDs with a simple 25/24 speedup.  It is also standard practice in PAL countries to broadcast TV series and movies with 25/24 speedup.

For example the lush orchestral sound accompanying Harry Potter movies will simply be broadcast 0.7 semitone sharp, 4.16% fast; and the timbre of the instruments and "gravitas" of the performance are significantly affected.  Around 98% of the population will not notice.  But an immediate ABX test will reveal the marked differences.

If there's no time to check this carefully, it's better to introduce a known flaw (wrong pitch), rather than introduce a potential serious unknown flaw (e.g. mangling the sound of some strings).

It's a bit like noise reduction - it is possible to remove hiss from older recordings - but sometimes it's better not to, given what's available. Just because someone chooses not to doesn't mean that they can't - not doing something can be a sign of great care, rather than great incompetence.
Indeed.

Quote
Anyway, many PC-based home theatre enthusiasts are thankful for 25p masters with a speed up - they can easily slow them down again and get the original sound - something which is impossible with pitch "corrected" versions!

Doesn't matter with BluRay - most are 24p even in "PAL" countries.
Yes.  I can and do use Reclock to play back selected TV recordings I capture in Australia, and the occasional PAL DVDs, at correct speed, so am thankful that no attempt has been made to "correct the pitch", as I would then have to perhaps "re-correct the pitch".  Pitch correction without a corresponding speed change is a complex and imperfect exercise.  You may be able to get away with it when editing the sound of a solo instrument, or some dialogue, but it will compromise the sound of an orchestra.  In fact I prefer the sound to remain 0.7 of a semitone sharp, than to be "pitch corrected".

What's the problem with double-blind testing?

Reply #348
Dolby digital is itself a perceptual coding technique.  It is not necessarily even a outstandingly good perceptual coding technique. It is reliatively old, but apparently the coders in general use have been upgraded somewhat over the years. There are blind tests that show that audible degradation due to it can be heard, at least in some cases.

I notice differences when comparing sound tracks on HD-DVD and Blu-ray discs [adjusted to match levels - though this matching can be quite difficult as lossy codecs alter the waveforms so drastically].  The difference between lossless and high bitrate lossy (e.g. DD+ 1536Kbps) is extremely subtle and I find with my hearing that I have to listen very intently in an ABX comparison.  There is very little in it.

It is interesting how well 448kbps Dolby 5.1 as found on an NTSC DVD discs performs, if not compared with another source.  It is quite pleasant, if a little bland.  Without an immediate ABX test available, many listeners would not notice the audible degradation relative to a lossless, or high definition lossy, version.