Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Multiformat@128 public listening test - CANCELLED (Read 43289 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Multiformat@128 public listening test - CANCELLED

Reply #75 –
Quote
This post is not just about the current test, but about level-matching and ABXing in general...

OK, it doesn't really matter how you measure it (ReplayGain, average or total of 50ms RMS values with or without a bandpass filter), the volume differences between the files are...

1. almost nothing
2. almost nothing
3. almost nothing
4. about 0.1dB
5. almost nothing
6. 3dB

"almost nothing" means about 0.01 to (at the very most) 0.04 dB, which can be ignored.

File 6 needs fixing, and strictly speaking file 4 needs fixing, since true ABX tests try to match levels to within 0.1dB - but with a 16-bit output, you might do more harm than good by making the adjustment.

The file numbers are from the .wav files I have from the Kraftwork sample - I haven't attempted to identify them.

[...]

Using ReplayGain when ABXing and not time-aligning and length-matching the files could introduce an audible level difference where none existed before - don't do it!

Look at the config files. ABC/HR for Java calculated almost exactly the same values (less than 0.05dB difference). And it does time-align before calculating gain.

Multiformat@128 public listening test - CANCELLED

Reply #76 –
I have been thinking about this, and I really wouldn't like to give up ABC/HR Java in this test, specially because of encryption. I never conduced a test with so much potential of attracting people interested in messing with the results, specially with the high level of zealousy and bias surrounding some of the formats featured - bias towards formats and against formats, mind you. Giving up encryption and dealing with plain text only is almost an invitation to tarnish the results.

And, before someone suggests it: No, I won't calculate two result sets, one with every result and other with results submitted by "trusted" people. I abhor this kind of elitism, so don't even get me started. This is an open, public test, if I wanted to take into consideration only the results of a few people I would counduce a closed test and invite them individually.

I keep wondering, why there wasn't as much criticism towards Schnofler's app at the AAC test? Is there a particular reason?

Regards;

Roberto.

Multiformat@128 public listening test - CANCELLED

Reply #77 –
I also concur that the best of everything should be used here and that means the best AAC coder, as determined by the last listening test.  I think we're all pretty surprised at the latest iTunes encoder with its weird high frequency behaviour.

Multiformat@128 public listening test - CANCELLED

Reply #78 –
Quote
I keep wondering, why there wasn't as much criticism towards Schnofler's app at the AAC test? Is there a particular reason?

The clicking noises were discovered after the start of the AAC test and we had talked about how to best mitigate the problem.  I discussed the volume issue with schnofler off-line.

If abchr-java is used, I would think you would want to make very sure that novice listeners, especially, do not base their decisions on the clicking sounds.  Knowing which slider is the codec without having to discern the real artifacting may tarnish the results.

But I understand the concern about people trying to stack the deck.  Actually, I'm more worried that normal listeners peek at the results file and make decisions based on what they see.  So I agree that encryption is good in that it keeps non-cheaters honest.

There are problems either way.  So choose your poison.

ff123

Edit:  It should be relatively easy to eliminate the problem where a listener can just look at the playing time to discover which sample is the codec.  I think this should be taken care of before the start of the test.

Multiformat@128 public listening test - CANCELLED

Reply #79 –
I found the reason for the clicking noises, they're my own fault, not Java's. They were not present in the AAC test, because they are a consequence of my rewrite of the playback code (which was supposed to eliminate deadlock issues).
I apologize for this mistake. I will fix it and make a new version of the application available soon.

Multiformat@128 public listening test - CANCELLED

Reply #80 –
Quote
I found the reason for the clicking noises, they're my own fault, not Java's. They were not present in the AAC test, because they are a consequence of my rewrite of the playback code (which was supposed to eliminate deadlock issues).
I apologize for this mistake. I will fix it and make a new version of the application available soon.

Nice.  One problem solved then 

Multiformat@128 public listening test - CANCELLED

Reply #82 –
Quote
There are problems either way.  So choose your poison.

My dream solution would be encryption support in your app

Multiformat@128 public listening test - CANCELLED

Reply #83 –
Quote
Quote
There are problems either way.  So choose your poison.

My dream solution would be encryption support in your app

Why isn't that an option? Does Java have an advantage when it comes to encryption? Or is it just lack of time?

Just curious, that's all

Multiformat@128 public listening test - CANCELLED

Reply #84 –
Quote
Quote
Quote
There are problems either way.  So choose your poison.

My dream solution would be encryption support in your app

Why isn't that an option? Does Java have an advantage when it comes to encryption? Or is it just lack of time?

Just curious, that's all 

Well, lack of time is the main issue.  I figure I'd have to install a java compiler to watch what's going on in schnofler's program so I could copy it in mine, and make sure that we do the same thing.  I don't know java, and despite having written abchr in C++, I don't know that language very well either!

I briefly looked at the encryption java code, and I suspect it would take me a while to figure out.

ff123

Multiformat@128 public listening test - CANCELLED

Reply #85 –
a few suggestions from a long time lurker:

for the restart of the test, could you please put the torrent-links directly on the website? because its very inconvenient to copy&paste 18 textlinks from the readme into the torrent-prog. as is understand it, you refrained from making the links public because of bandwidth issues, but with torrent links this isn't a problem anymore (is it?)

also it would be great to provide an alternative torrent with all the samples in them. i believe most people who use torrent do have broadband, and don't care so much how big the download is. personally i always download all packages, and decide later during testing which samples i rate (depending on the music-style and the difficulty to hear differences). and when you are one it, include the package with abc/hr too, so inexperienced people don't have to take care of extracting into the right folder with pathnames etc.. (you know, one possibility less to make errors is always a good thing )

as i think of it, it probably would be better to exclusively provide the one big torrent, otherwise people who want to help seeding would have to download the big torrent plus the 18 single ones. also people which are using the official bittorrent client and want to seed wouldn't have to clutter their taskbar with 19 instances of bittorrent, instead just with one.

people one modem can still download single samples per http from the links in the readme. which brings me to another point: could you please provide the readme in html format? aside from the sample-download-links there are various other links (java-download, ff123's page, your email, etc.), and it would be much more convenient if you could click on them directly instead of copy&pasting.

otherwise, your tests are great, and i'm looking forward to the outcome of this one. i think it will be *very* interesting in many ways, regardless of the result

Multiformat@128 public listening test - CANCELLED

Reply #86 –
Quote
for the restart of the test, could you please put the torrent-links directly on the website? because its very inconvenient to copy&paste 18 textlinks from the readme into the torrent-prog. as is understand it, you refrained from making the links public because of bandwidth issues, but with torrent links this isn't a problem anymore (is it?)

Right. I'll put the torrents on the test presentation page.

Quote
also it would be great to provide an alternative torrent with all the samples in them. i believe most people who use torrent do have broadband, and don't care so much how big the download is. personally i always download all packages, and decide later during testing which samples i rate (depending on the music-style and the difficulty to hear differences). and when you are one it, include the package with abc/hr too, so inexperienced people don't have to take care of extracting into the right folder with pathnames etc.. (you know, one possibility less to make errors is always a good thing )


Hehe. That's true, but I personally prefer to have abc-hr_bin as a separate download because often I need to change stuff there while the test is running. Correct batch files, add some important information or updated links to the readme, etc. So, I will provide a big Samples.torrent, but I won't include ABC/HR on it.

Quote
as i think of it, it probably would be better to exclusively provide the one big torrent, otherwise people who want to help seeding would have to download the big torrent plus the 18 single ones. also people which are using the official bittorrent client and want to seed wouldn't have to clutter their taskbar with 19 instances of bittorrent, instead just with one.


That's OK. Still, I prefer to seed separate torrents as well. Some people really only want to test one sample, because they are low on time or don't care much about results but would like to help anyway. And, if these people download through torrent instead of http, all the better.

Quote
people one modem can still download single samples per http from the links in the readme. which brings me to another point: could you please provide the readme in html format? aside from the sample-download-links there are various other links (java-download, ff123's page, your email, etc.), and it would be much more convenient if you could click on them directly instead of copy&pasting.


OK, I will provide the readme in both formats.

Thank-you for your suggestions

Regards;

Roberto.

Multiformat@128 public listening test - CANCELLED

Reply #87 –
Hello.

Fate seems to be set on ruining my test (I arrived home on monday to see most of my NTFS partition destroyed), but justice shall prevail! or something.

I would like to invite people with some free time tonight to test some of the packages I uploaded. I worked hard to avoid the same problems creeping in this time (I hopefully downloaded the right version of aoTuV, fixed volume differences on Atrac3, and double checked that the m4a files were encoded with the correct iTunes version). Also, Schnofler worked on fixing the noises in ABC/HR java. Still, I would like to be a little more on the safe side and have some people check the samples and batch files for problems before the official announcement tomorrow.

The sample packages and abc-hr_bin are available here:

http://www.rarewares.org/pretest/

Don't download them if you don't plan to test for errors, because these packages might change until tomorrow, when the test will be officially announced.

Please report in this thread if you get problems or not.

Thank-you very much.

Regards;

Roberto.

Multiformat@128 public listening test - CANCELLED

Reply #88 –
There's no problem with rosemary_1, rosemary_2, rosemary_3 and rosemary_4. (bit identical to my files)
BTW, what tool did you use for WMA en/decoding?

Multiformat@128 public listening test - CANCELLED

Reply #89 –
Quote
BTW, what tool did you use for WMA en/decoding?

For encoding Windows Media Encoder 9

For decoding straight to FLAC, dbPowerAmp.

Multiformat@128 public listening test - CANCELLED

Reply #90 –
Quote
For encoding Windows Media Encoder 9

For decoding straight to FLAC, dbPowerAmp.

Thanks. I found rosemary_6 is bit identical to my file as well.

Edit: It seems that each sample has decent replaygain value. (nearly same volume)

rosemary.wav:track_gain = -3.22 dB
rosemary_1.wav:track_gain = -3.20 dB
rosemary_2.wav:track_gain = -3.31 dB
rosemary_3.wav:track_gain = -3.24 dB
rosemary_4.wav:track_gain = -3.15 dB
rosemary_5.wav:track_gain = -3.16 dB
rosemary_6.wav:track_gain = -3.17 dB

Edit2:fixed some values.

Multiformat@128 public listening test - CANCELLED

Reply #91 –
downloaded sample06 (getiton)

version of abchr-java is ok.
time offsets appear to be ok (I can't tell tracks apart because of differences in time).

lame encoding is fine (3.96 -V5 --athaa-sensitivity 1)
mpc encoding is fine (1.14 beta --quality 4.15 --xlevel)
ogg encoding is fine (aotuvb2 -q 4.35)
wma encoding is fine (bitrate vbr, 128)

ff123

Multiformat@128 public listening test - CANCELLED

Reply #92 –
Wonderful! Thank-you very much for testing

Multiformat@128 public listening test - CANCELLED

Reply #93 –
One last thing: the volume is still softer in abchr-java than it is in the native windows version.  Did you use the abchr-java gain calculator or did you make the wav gains equal externally and bypass the gain calculator?  The abchr-java gain calculator, in addition to making the gains the same, also brings the volume up.

ff123

Multiformat@128 public listening test - CANCELLED

Reply #94 –
Quote
Did you use the abchr-java gain calculator or did you make the wav gains equal externally and bypass the gain calculator?

I used ABC/HR Java's gain calculator. There's no way to make the wave gains equal externally because M4A, MPC and Vorbis are distributed in encoded form.

Multiformat@128 public listening test - CANCELLED

Reply #95 –
Quote
Quote
Did you use the abchr-java gain calculator or did you make the wav gains equal externally and bypass the gain calculator?

I used ABC/HR Java's gain calculator. There's no way to make the wave gains equal externally because M4A, MPC and Vorbis are distributed in encoded form.

What did you do differently with the atrac3 encode?  Wasn't it 3 dB lower than the others before?  Everything seems to be about equally loud this time around, even before it gets to abchr-java.

ff123

Multiformat@128 public listening test - CANCELLED

Reply #96 –
Quote
What did you do differently with the atrac3 encode?  Wasn't it 3 dB lower than the others before?  Everything seems to be about equally loud this time around, even before it gets to abchr-java.

Ah, I changed the volume of the Atrac3 sample with CoolEdit, more or less following 2BDecided's guidelines he posted earlier in this thread.

But I didn't mess with the volumes of the other samples with CoolEdit.

And, even after fixing Atrac3, I used ABC/HR Java's gain calculator on it, same as the other files.

Multiformat@128 public listening test - CANCELLED

Reply #97 –
Hello,

I don't understand why this thread title ends with the word "Cancelled", as I recall reading the results of this test before, so I don't think the test was cancelled.

I saw a lot of discussion about which version of LAME to use, which preset to use, or tweaked command line. What was finally used for the LAME encoding in the test? I seem to have missed that. Which version and which setting?

Also, for the WMA encoding in the test, exactly what was the setting? I know that WMA VBR has quality settings like 50, 75, 90, 95, etc. (not settings by average bitrate). I forget which is closest to 128 kbps. (Is it level 75? 90?)

Multiformat@128 public listening test - CANCELLED

Reply #98 –
Quote
I don't understand why this thread title ends with the word "Cancelled", as I recall reading the results of this test before, so I don't think the test was cancelled.

Read this thread, and you'll understand why the 1st incarnation of this test was cancelled

Quote
I saw a lot of discussion about which version of LAME to use, which preset to use, or tweaked command line. What was finally used for the LAME encoding in the test? I seem to have missed that. Which version and which setting?


Lame 3.96 -V 5 --athaa-sensitivity 1

Quote
Also, for the WMA encoding in the test, exactly what was the setting? I know that WMA VBR has quality settings like 50, 75, 90, 95, etc. (not settings by average bitrate). I forget which is closest to 128 kbps. (Is it level 75? 90?)


WMA VBR varies too wildly in bitrate (some samples get bitrate lower than 100kbps, others are higher than 150...), so I used Two-pass VBR ("Bitrate VBR")  at 128kbps.

Multiformat@128 public listening test - CANCELLED

Reply #99 –
Quote
Lame 3.96 -V 5 --athaa-sensitivity 1

So you decided not to use a preset after all? Is that the version and command line that you found to get the best sound quality with LAME at a bit rate of around 128 kbps?