Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: ABX Comparator version 2.0 (Read 50692 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

ABX Comparator version 2.0

Reply #50
Well my example was hypothetical. In truth I don't think I have ever used more than one phrase repeat segment per test. I do hunt around for several when in training mode though and when the cumulative score tells me I've hit "pay dirt" I then commence testing on that segment I've latched onto by switching to actual test mode. Unfortunately when I do so the segment is purged from memory so I have to find it again, but knowing this will happen I now know to memorize the start and end time stamps so I can quickly reinstate the loop section. This is a slight annoyance however looking at the big picture I think it is the correct way to do it and I am NOT suggesting that the A to B loop be maintained when switching from training mode to actual test mode.

ABX Comparator version 2.0

Reply #51
Logging segments used in the test doesn't seem very 'solid'.
Anyone could play the whole track (which is quite short usually) but listen only for a single artifact, e.g. at the beginning or end.


And then miss the issue when switching A->B sounds different from B->A or B->B or A->A, right?

ABX Comparator version 2.0

Reply #52
Eh.. yes? No switching during the track means no switching artifacts, but I wasn't talking about ABX artifacts but e.g. audible differences at the beginning/end of the file due to small time delays, etc.
"I hear it when I see it."

ABX Comparator version 2.0

Reply #53
Although I agree seeing what segment(s) a test listener is looping per trial might be of interest, another problem with creating a log to show this, as Arny has suggested, is that the listener still gets to use an extremely useful yet completely undocumented one; namely by removing the check mark in the box "Keep playback position when changing track" one can manually listen to the opening one second [or whatever length they want] over and over again by switching between A, B, X, and Y, every one second, without it being documented as a "phrase repeat was engaged".

Listening to the opening second is exactly how I discriminated between MLXXX's new time aligned and level matched conversions of the Hi-res song "On the street where you live" he submitted at the AVS thread "AVS/AIX High-Resolution Audio Test: Take 2" a couple of days ago, by listening at elevated levels and concentrating on the tonality of the noise floor. [The music itself sounded identical to me. As I mention there, the ONLY reason I did this was to document how an unscrupulous con artist, who has infected that thread,  might "pass" the test without detection.]

ABX Comparator version 2.0

Reply #54
Arny, all within the confines of one trial, in real test mode, one may listen to a 5 second segment looped from early in the song, but then a minute later switch to a 10 second segment from the end of the song, etc., so Foobar ABX would have to keep track and list ALL of them, every time the phrase repeat loop feature is activated, plus it won't understand when you are trying to isolate a particular telling section precisely, and keep moving the start and stop points around slightly by knudging them up and down the time scale slightly, that each attempt isn't really the loop you focused on. A final report would look like this:

"Trial 1 segment repeats used: 10.1 seconds to 15.3 seconds, 10.0 seconds to 15.3 seconds, 9.9 seconds to 15.3 seconds, 1m08.7 seconds to 1m19.2 seconds, 1m08.7 seconds to 1m20.0seconds, 1m08.7seconds to 1m19.7 seconds, 1m08.8seconds to 1m19.7 seconds, 1m08.9seconds to 1m19.7 seconds

Trial 2 segment repeats used..."

That's just how my trials would look, especially if the difference is at the outer limit of my hearing capacity. I frequently cross-check by choosing two or even three different segments for one trial. Only if my tentative answers for different segments are all the same, e.g.  X is A, X is A, X is A, do I actually enter a response. (X is A!)

One reason I need to do this is that my hearing can change within 30 seconds to a reversal. I should explain this. At the beginning of the session, A may sound darker in sound quality than B and that is the basis I use for testing X. This may remain the case for say 5 minutes. But then within 30 seconds B may start sounding the darker. I cannot trust my ears to remain stable and rely on my memory of which of A and B had the darker sound. I must retest A and B.

And of course in such a transition phase, A and B for a while may sound as dark as each other. If that situation persists, I may have to come back later and see whether A and B have once again become capable of being distinguished. If they still sound as dark as each other, I will go hunting in the files under comparison for some other characteristic to distinguish them.

All of this to-ing and fro-ing can involve dozens of different segments being listened as part of just one trial. Some are not really listened to but are mere adjustments to the starting and ending points of a segment until it is found that the segment sits well as to length and content.

Another method which works very nicely if the transition between A, B, X, and Y can be near instantaneous is to listen to long sections of the files observing whether A to X represents a change, or A to Y. I can find this very helpful if I've been listening for a long time and my brain has become fatigued.

So I tend to use the flexibility and freedom of the foobar ABX tool to the fullest.

If the approach I have just outlined is not considered acceptable ABX practice, then perhaps acceptable practice guidelines should be determined and published restricting the number of different segments a user can carefully listen to for a single trial.






ABX Comparator version 2.0

Reply #55
Current progress (no new component update yet, expect that soon)-

200ms fade is used for: mid-track beginning, mid-track end, track switch, seek.
Why 200ms? Short fades are perceived as audio dropouts / glitches and therefore tedious. In my opinion longer fade is less annoying despite of theoretically presenting a longer audio dropout.

Sine ramp used for all fades.

Tracks with different sample rates or channel counts = fadeout + forced output reinit + fadein on each track switch. Must reinit output forcibly or else it becomes a cheat factor as output reinit lag on sample rate change is easy to notice.

PROPOSED crossfade logic-
On track change, analyse the 10ms chunk that we're at; calculate RMS( chunkA - chunkB ) to determine crossfade plausibility and crossfade only if RMS of signal difference is low enough.
Comments please.
Microsoft Windows: We can't script here, this is bat country.

ABX Comparator version 2.0

Reply #56
If possible I'd be inclined to keep the swapping quick and simple, e.g. 10mS fade out, 10mS silence, 10mS fade in. This would however require a tight timing alignment in the files under comparison.

ABX Comparator version 2.0

Reply #57
I have my DS fade times all set to much shorter than 200ms. It doesn't produce glitches for me, but then again I'm not A-B testing with foobar core itself. But even on seek 50ms still seems to work fine.


The problem with the RMS calculation I see is this: consistency.
What if at one part of the track A<->B can be crossfaded, but at another part of the track A<->B difference is too big to crossfade? Does this give away too much information regarding the similarity of the files?

I guess you would do the same processing regardless if there is an actual underlying track switch (A->X and X=B) or not (X=A)?


edit: Wouldn't it make more sense to analyze files beforehand and decide whether crossfading is enable or disabled for the whole ABX session?
Also, I'm not sure about RMS of a 10ms chunk. Isn't the problem that the two waveforms don't overlap neatly?

edit2: How about subtracting the chunks from each other and calculating the RMS of that? Would have to test if this is too sensitive for lossy compression..
"I hear it when I see it."

ABX Comparator version 2.0

Reply #58
edit2: How about subtracting the chunks from each other and calculating the RMS of that? Would have to test if this is too sensitive for lossy compression..

This is precisely what I'm suggesting....
If the RMS comes out low, it means that the crossfade ought to appear unnoticeable to the user.
Microsoft Windows: We can't script here, this is bat country.

ABX Comparator version 2.0

Reply #59
200ms fade is used for: mid-track beginning, mid-track end, track switch, seek.

No problem with 200ms; in fact, this should go a long way towards eliminating the temporal-masking issue I mentioned.

Quote
Sine ramp used for all fades.

If it looks like this, great!:


Quote
PROPOSED crossfade logic-
On track change, analyse the 10ms chunk that we're at; calculate RMS( chunkA - chunkB ) to determine crossfade plausibility and crossfade only if RMS of signal difference is low enough.
Comments please.

This could get round the problem that xnor mentioned, but at the expense of introducing inconsistency in behaviour, which could be confusing. So I'm for not cross-fading at all: always fade out then in.

Though if folk have a legitimate use for the cross-fades, perhaps it could be restricted to practice mode? And even then (to allow consistency), it should perhaps be opt-in.

ABX Comparator version 2.0

Reply #60
Beta 5 posted.

Crossfading is currently disabled in favor of forced 200ms fades.
Microsoft Windows: We can't script here, this is bat country.

ABX Comparator version 2.0

Reply #61
Beta 6 posted, fixed wrong fade curve used in beta 5.
Microsoft Windows: We can't script here, this is bat country.

ABX Comparator version 2.0

Reply #62
Testing with beta-6, the transitions between A/B/X/Y seem good, as do the transitions between silence and a truncated signal (i.e. with Set start or Set End).

However, when transitioning between an un-truncated signal and silence (i.e. simply playing the signal without having done Set start or Set end), it seems that there is no fade being applied at the ends, and as a result, I can still ABX silence vs. a 20kHz tone.

ABX Comparator version 2.0

Reply #63
I'd call it a bug if it altered file playback when playing the full file. Prepare such test files so that they include fades of their own so they work without glitching everywhere. Like this:
[attachment=8081:22000Hz_48kHz_1ch.flac]
[attachment=8082:silence_48kHz_1ch.flac]

Unfortunately people can cheat with this test by using Windows or external volume control during the high frequency tone playback. It produces glitching.

I'd like to see a version with shorter fades to see if that makes noticing small differences easier.

ABX Comparator version 2.0

Reply #64
Feature request: keep segment selection when starting the test from training mode, so the user can continue with as little interruption as possible.

"I hear it when I see it."

ABX Comparator version 2.0

Reply #65
Bug report: why are there no SHA1 sums for the files when you start the test from within training mode?
"I hear it when I see it."

ABX Comparator version 2.0

Reply #66
Feature request: keep segment selection when starting the test from training mode, so the user can continue with as little interruption as possible.

Noted.

Bug report: why are there no SHA1 sums for the files when you start the test from within training mode?

Noted also, thanks for reporting.
Microsoft Windows: We can't script here, this is bat country.

ABX Comparator version 2.0

Reply #67
Background:
For many years, foobar ABX reports using the earler version of the plug-in have been accepted on HydrogenAudio as a valid form of ABX test.

Feature request:
That an option be provided for the formal ABX test stage of the new plug-in to display progressive results (as the traditional plug-in did). [The current beta version provides no indication of the progressive result during the formal test.]

Reasons:
It is not unusual for an ABX test to require a fine discrimination at the outer limits of a test subject's ability to hear.  The investment of time and effort can be considerable. The traditional ABX plug-in gave the test subject the oportunity to be alerted to the negative outcome that their error rate was so high that it would be futile to proceed further. The test subject could abandon the test and report that the difference was beyond their ability to discriminate reliably.  Conversely, if a test subject had feedback that they were doing well, they could feel encouraged to persevere, despite the time and effort involved.  This tended to lead, overall, to an efficient use of the test subject's listening time.

Unless a difference were obvious to my ears, I personally would be reluctant to embark on a formal foobar ABX exercise [of a minimum of 8 trials, as seems to be imposed in the beta version], without progressive feedback.

ABX Comparator version 2.0

Reply #68
Beta 7 up.

Crossfading is back, as an option.
Microsoft Windows: We can't script here, this is bat country.

ABX Comparator version 2.0

Reply #69
Beta 8: fade duration set to 40ms instead of 200ms.

If there are no further issues, this will be the last beta before 2.0 is posted on the components site, replacing the old component.
Microsoft Windows: We can't script here, this is bat country.

ABX Comparator version 2.0

Reply #70
I like the choice to crossfade or not. Perfect. Thank you.

I don't realistically expect this, but there are alternative stats that apply if you do watch your progress (pass/fail) as you go along and choose to stop when it's good enough. One day, it would be cool to have this mode, with logs too.


Is this of any interest?
http://www.hydrogenaud.io/forums/index.php...st&p=884060

You've already fixed some of those issues, and some of the other things aren't really improvements IMO, but one comment is true: the time resolution of the slider is low if you choose to ABX very long tracks.

Cheers,
David.

ABX Comparator version 2.0

Reply #71
Nice.
Only small "cosmetic" things I've noticed:
a) When nothing is playing seek to a random position, press stop. Internally the position seems to be reset to the start, but the slider doesn't get visually reset.

b) When nothing is playing seek to a random position, press "Set end" and then "Play A". The seekbar will jump to the start and not start playing. Ok, that is acceptable.
But this also happens if the slider/seekbar is past the marked end and you press play.
If the slider is before the marked start it will happily start playing there, outside the marked segment.

I've always wondered if this is intentional.

c) The seekbar seems to be divided into 5 equal-length page sizes (that's what Microsoft calls it anyway). Whenever one clicks next to the slider (or press pgup/down), the slider will jump to that "nearest page" which can actually be pretty far away.
How about seeking to the precise mouse cursor location?

d) Play within a segment, press pause, click the slider. You can now move the slider to the left with the left/up key, but not right with the right/down key. Page Down works fine however.

"I hear it when I see it."


ABX Comparator version 2.0

Reply #73
Beta 9 posted.

Finishing touch before 2.0 stable: you can now manually edit the current timecode.

I know that the slider sucks, but that is standard Windows control to you. I will be sure to refresh it when I have a custom slider substitute class handy but don't have time to fix this right now.

Playback looping, noted, there will be other updates after 2.0 too.
Microsoft Windows: We can't script here, this is bat country.

ABX Comparator version 2.0

Reply #74
Sorry, but this comparator isn't any use for me now.

8 trials (p=0,39%) is too high minimal number.

For the most of cases (90%+) I use 5 trials (p=3%) and in very rare, specific cases go to 8.



Edit: Sure, it's nice to have a high number of trials but whoever has suggested 8 trials hasn't in account  that a fatigue grows with a number of trials ... and not linearly.