Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: FAQ on HA Wiki (Read 7883 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

FAQ on HA Wiki

I vaguely recall someone offering to dump the HA FAQ into the HA Wiki (this page)... any progress?

If you find difficulty in doing so, you can start by writing a .doc or .rtf document of the messy HA FAQ and email it to me. Please PM me for my email address.

Thanks.

FAQ on HA Wiki

Reply #2 –
I think explanation about how to use java ABC/HR may be helpful for novice tester. ABX is well-known though, ABC/HR isn't.

FAQ on HA Wiki

Reply #3 –
It exists in french... http://chaud7.forumactif.fr/1ere-categorie...eek-end-t27.htm

Here is a very rough translation.
Explanations are sometimes detailed because I had to explain the english buttons for french readers.

Feel free to rewrite this tutorial in a better english way.


Load files into Foobar2000
Select them all

Right click / convert / settings. Préférences appear.
Output presets -> WAV, PCM fixed point.
Output bit depth -> 16 bits.
Processing, verify that Use DSP is NOT checked.

Close préférences.

On mp3 sélected, right clic / convert / run conversion.
Confirm format, tell destination folder (same as mp3's).

Download ABC/HR beta 1.1 : http://ff123.net/abchr/abchr.html

File / Setup test
Orig Wav -> Original wav.
Other Wavs... tell the previously decompressed wavs.  Those you want to compare. Ideal is to compare several encoders at same bitrate.

In the main window, the columns turn on.

In order to ABX, click ABX. In select A, choose original. In select B, the sample in the column that you want to test.

WARNING, Files entered are randomly distributed among the columns. If you want to perform an ABX with a given file, you must "setup test" with only one file besides the original.

Start some trials with buttons ABX.
You can choose the part to playback by sliding the ends or the bottom grey bar.
Click on "X is A" or "X is B" in order to answer. Then next trial to perform another randomization.

Once you have found a good part to ABX, switch from training to normal. And perform at least 8 trials. (the more the better)
Once finished, click finish.
Your score and the probability to get it by chance are displayed. (probability of falsely accepting A is different from B).

Easier way : stay in training mode, click reset, and choose a total number of trials to do.
The result is the same, but you can see your result as and when you proceed.

Once ABX done. Close. In the main windows, in the column of the sample that you tested, give a mark from 1 to 5 for sound quality.
You can add comments in order to describe the sound by clicking the small sheets above the columns.
Warning, the slider of the original is not always the left hand one ! This is randomized for each column.
However, the Ref button is always the original.

If you succeeded ABX, only the tested sample slider remains active. This allows not to rank the original by mistake.

Once done, file / save test results.

This allows not only to ABX, but also to blind test your preferences.

Edit : one more thing. Setting up a blind test, you should always check that the decompressed files do not clip, and that they are in synch with the original on the time axis.

FAQ on HA Wiki

Reply #4 –
Thanks, this is informative and mainly explains what to do with ff123's software. Despite the fact that all listener must go with ABC/HR in public test, I couldn't find any related entry in HA Wiki. Your writing about blind test and read me about java ABC/HR will best suit to that.

FAQ on HA Wiki

Reply #5 –
Uh... so, Pio, what *exactly* do you want me to do?

I got this impression (CMIIW) that you want me (or someone else) to make a HA-wiki article about "How to perform an ABC/HR listening test."

If I intuit right, then would someone also do a short writeup on the definition of an ABC/HR listening test?

Edit: If you search for ABC/HR on HA-wiki or use the Topic Index, you can see that there's no article for ABC/HR yet.

FAQ on HA Wiki

Reply #6 –
Now, now, I do not "want" anybody to do anything... that's not the way community works.

It's just that I don't have the time or motivation to write wiki articles about this. But I've got some links and some bits of info, like the ABC/HR summary above.
So I can't let them rest in my Favorites folder while some people could use them.

FAQ on HA Wiki

Reply #8 –
To clarify my post above:

The ABC/HR article wasn't there. So, I started writing it and stumbled on a writer's block.

Can someone fix the page for us all?

Thx.

FAQ on HA Wiki

Reply #9 –
I guess it would be good to rough our a structure and get an idea of the references and hyperlinks that are appropriate too.

Feel free to comment and correct me, debate points or suggest something different, but here's a start at a structure that may break the writer's block and get you going.

What does ABC/HR stand for?
(You've done this)

What is it used for?
Blind comparison and blind quality rating to remove the effects of subjective bias and the placebo effect.

How is it different to ABX?
ABX is used to detect audible differences blindly, thereby removing subjective bias or the placebo effect and, over multiple trials, estimating the probability that the tester was guessing. While an ABC/HR tool can do the same, it adds to that the ability to provide a quality rating on a standardised scale, and multiple participants' results can be statistically evaluated to estimate error bars and statistical significance of differences between encoders or encoder parameters in ranking their quality or tying them. ABC/HR tends to find particular application in low-to-medium bitrate listening tests (below the quality expected to constitute transparency). As part of a well-designed listening test, useful quality comparison can be made between a selection of encoders plus a high-quality anchor (high anchor) and low-quality anchor (low anchor) without the tester being aware of which encoder is being evaluated at any time.

What is the standardised quality scale
It is most common to use the 1.0 to 5.0 scale defined by ????? (provide reference citation in footnotes, e.g. ISO/ITU/MPEG standard for quality evaluation in formal listening tests). Any value (including fractions) between 1.0 and 5.0 is valid, with the exact whole number representing the following definitions.

That's all I have time for...

More questions / headings

References

Links
Dynamic – the artist formerly known as DickD

FAQ on HA Wiki

Reply #10 –
Okay, followed your suggestion. Please check out the page again.

FAQ on HA Wiki

Reply #12 –
Quote from: Dynamic on
What is it used for?
Blind comparison and blind quality rating to remove the effects of subjective bias and the placebo effect.

I object to using the word "subjective" in this context. It would be more precise and more robust, to only describe the "observer-expectancy-effect" and its variants (of which placebo is one).

The reason for why i object to this has to do with a deep fundamental misunderstanding in our civilization about what "subjective" and what "objective" means. Part of this myth is the scientific illusion that it is possible (or even desirable) to remove ALL subjective influences. This is per definition impossible, because the observer - the subject - is always there - without an observer, there would be no one who perceives, and without someone who subjectively interpretes the information (i.e. test-results) there would be no meaning of the test - there even would be no one doing the test.

It is probably too complex and difficult, to explain the whole subject-object-model in full here - to keep a long story short: you can avoid the myth AND make the article more precise, by describing only what you actually mean - which in this case is the observer-expectancy effect.

- Lyx
I am arrogant and I can afford it because I deliver.
 

FAQ on HA Wiki

Reply #14 –
This would change the wording, but not the meaning. The problem is that double-blind-testing cannot (completely) exclude subjective/personal bias. Especially not when it comes to interpreting the results (and uninterpreted data is worthless.... its just random numbers without any relations unless you interprete them).

Off course one could make the argument, that this is less of concern when it comes to the *measurement* itself(ignoring the test-setup and result-interpretation). While this may be technically more near the truth, it would be very misleading. When you write an FAQ about DBTs on the wiki, then the audience which you are addressing most probably has no clue about the theoretical background-details how science and experimentation works - and quite possibly neither they are interested in learning them just to understand why DBTs are necessary, nor you are interested in writing a kickstart-manual on science theory. When people read that article, then they do not make such differentiations. They only want to know "if it works, why it works". They see all three components - the setup, measurement and interpretation - as one thing.

So, if you write "removes all personal bias", then their interpretation will be "it will magically eradicate all possible error-sources and subjective influences, so that the result is ABSOLUTELY OBJECTIVE. And since they associate objectivity with truth, the final meaning is "this test tells you the absolute truth". This would even be the case, if you insert the word "measurement" somewhere - they will not understand its relevance (because they lack information) and therefore just filter it out. On the other hand - giving them a quick halfassed rundown on the background theory may actually irritate them more, than educate them (sometimes, no information is better than incomplete information - which is why i proposed to only focus on the observer/subject-expectancy effect - which is the main issue why DBTs are done anyways).

- Lyx
I am arrogant and I can afford it because I deliver.

FAQ on HA Wiki

Reply #15 –
While I don't argue the fact that to properly listen, one must be subjective, I do think that double-blind removes personal bias, i.e. it removes the personal preference one has on a certain encoding. E.g. a tester may be biased personally for AAC, to the point of subconsciously thinking/opinioning that AAC is always better than other encoding, no matter the bitrate. Double-blind ensures that despite one's preference encoding, there is no bias against the actual sound quality that one's listened to.