Skip to main content

Topic: Public MP3 Listening Test @ 128 kbps - CLOSED (Read 128706 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #50
I downloaded ABC/HR, I also downloaded a sample package "Sample07.zip". I ran Sample07.bat, then executed abchr.jar, clicked Open ABC/HR config, loaded Sample07.ecf.

The problem: Clicking play on any sample doesn't produce any sound.


From the presentation page:

Quote
Please note: if you don't hear any sound when using ABC/HR, simply change the device from ABC/HR's settings. In some cases, ABC/HR might not default to the primary audio driver.


This sample package happened to be the eig sample


Impossible - eig isn't even used in this test.

  • halb27
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #51
My sample07 is Suzanne Vega (Tom's Diner) as well.
lame3995n -Q0.5

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #52
Just downloaded Sample07 from both mirrors and it's definitely Tom's Diner. As I mentioned already, eig isn't even featured in this test.

The samples included are (alphabetically sorted):
atrain
Castanets_Original
chariotsoffire
fatboy_30sec
finalfantasy
Hypnotize
inthenight
Layla
linchpin
macabre
sfbay
TomsDiner
velvet
Waiting

Edit: Wondering why IPB sent me 3 mails for the same reply from Bodhi.
  • Last Edit: 26 October, 2008, 06:09:34 PM by Sebastian Mares

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #53
Problem #2. This sample package happened to be the eig sample, which produces easily noticable artifacts. Just out of curiosity, I tried to encode the sample with FHG surround encoder "-br 0 -m 4 -q 1 -vbri -ofl" and compared my encoding to the test samples. It sounded different from all of them, and it shouldn't.


I tried to encode samples #2 and #12 myself. My files were bit-identical to downloaded ones. (By the way, l3enc encodes raw files, not wavs). And sample 7 is definitely "Tom's Diner", not "Everything is green".

  • /mnt
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #54
My Sample 7 file is Tom's Diner anyway and not eig. I have to admit i fail to ABX Tom's Diner on the non low anchor encodes (l3ence is very easy to ABX though). And now for some reason after i tried to ABX Tom's Diner. i need to watch a episode of Seinfeld  .
  • Last Edit: 26 October, 2008, 09:09:19 PM by /mnt
"I never thought I'd see this much candy in one mission!"

  • Neasden
  • [*][*][*]
  • Banned
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #55
If one encoder SURPRISES you then this definitely should not be LAME, because if it were it, it wouldn't be a surprise, would it?
  • Last Edit: 26 October, 2008, 10:00:29 PM by Neasden

  • halb27
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #56
... I have to admit i fail to ABX Tom's Diner on the non low anchor encodes ...

I have to admit that there are many samples where only the very obvious low anchor has an issue for me. Shows that even 128 kbps mp3 usually is fine (to me), and it's personal properties to which kind of artifact we are sensitive.
  • Last Edit: 27 October, 2008, 05:06:35 AM by halb27
lame3995n -Q0.5

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #57
I have to admit that there are many samples where only the very obvious low anchor has an issue for me. Shows that even 128 kbps mp3 usually is fine (to me).

Me too.

Quote
[...] and it's personal properties to which kind of artifact we are sensitive.

I’m more sensitive to warbling, hi-hat chirping and warped ride cymbal (I play the drums) and also distorted voices or trumpets but not so much to pre-echo.

P.S. : I really liked the Vangelis sample, interesting one IMO.
  • Last Edit: 02 November, 2008, 06:44:35 AM by melomaniac

  • Alex B
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #58
I have to admit i fail to ABX Tom's Diner on the non low anchor encodes (l3ence is very easy to ABX though).


That's surprising. There's another Tom's Diner sample that can be easily distinguished.

IMHO, it has obvious problems and ABXing is not necessary. The four other samples are closer to the original, but not fully transparent.

I wonder if you simply tried to hear other kinds of artifacts and didn't notice this problem because it is different.

Would you mind testing the two samples I uploaded here: http://www.hydrogenaudio.org/forums/index....showtopic=66815

  • /mnt
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #59
I have to admit i fail to ABX Tom's Diner on the non low anchor encodes (l3ence is very easy to ABX though).

That's surprising. There's another Tom's Diner sample that can be easily distinguished.

IMHO, it has obvious problems and ABXing is not necessary. The four other samples are closer to the original, but not fully transparent.

I wonder if you simply tried to hear other kinds of artifacts and didn't notice this problem because it is different.

Would you mind testing the two samples I uploaded here: http://www.hydrogenaudio.org/forums/index....showtopic=66815

I have found some issues with that sample. The mp3 version has a skratchy artifact on the vocals, abit like fatboy at V5.

Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.6 beta 1
2008/10/27 10:30:58

File A: C:\Downloads\sample07_source.flac
File B: C:\Downloads\sample07_mp3.flac

10:30:58 : Test started.
10:31:08 : 01/01  50.0%
10:31:11 : 02/02  25.0%
10:31:16 : 03/03  12.5%
10:31:19 : 04/04  6.3%
10:31:22 : 05/05  3.1%
10:31:30 : 06/06  1.6%
10:31:37 : 07/07  0.8%
10:31:42 : 08/08  0.4%
10:31:48 : 09/09  0.2%
10:31:53 : 10/10  0.1%
10:32:00 : 11/11  0.0%
10:32:05 : 12/12  0.0%
10:32:11 : Test finished.

 ----------
Total: 12/12 (0.0%)
  • Last Edit: 27 October, 2008, 06:38:57 AM by /mnt
"I never thought I'd see this much candy in one mission!"

  • jmartis
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #60
Just downloaded Sample07 from both mirrors and it's definitely Tom's Diner. As I mentioned already, eig isn't even featured in this test.

OK, sorry then, I must have gotten a wrong (old) sample... Thanks for the help.

  • Alex B
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #61
I have found some issues with that sample. The mp3 version has a skratchy artifact on the vocals, abit like fatboy at V5.


The sample is a direct clip from one of the test samples (and it is not from the low anchor).

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #62
I have found some issues with that sample. The mp3 version has a skratchy artifact on the vocals, abit like fatboy at V5.


The sample is a direct clip from one of the test samples (and it is not from the low anchor).

I've just made an ABC/HR test on sample07 and I can only spot the one you're talking about right now (like /mnt, no problem ABX it) and the low anchor, but not the other four.

  • /mnt
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #63
I have found some issues with that sample. The mp3 version has a skratchy artifact on the vocals, abit like fatboy at V5.


The sample is a direct clip from one of the test samples (and it is not from the low anchor).


That was embarrassing , i did not know what was really wrong with that sample, i knew its imfamus for tuning. When i was trying to ABX Tom's Diner last week; I thought all the non low anchor samples sounded ok.
  • Last Edit: 27 October, 2008, 07:18:15 AM by /mnt
"I never thought I'd see this much candy in one mission!"

  • Alex B
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #64
Normally we should not discuss about the test samples before the test is over, but I think this is a special case. The problem is quite obvious and it's worrying if someone cannot spot it.

So please be careful when starting a new test sample. Try to clear your mind from the previously listened samples and the possibly different problems you may have spotted. It might be a good idea to have a short break and do something completely different before opening the next test configuration file.
  • Last Edit: 27 October, 2008, 07:24:07 AM by Alex B

  • halb27
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #65
In case I  know what to listen for (and I'm sensitive for the kind of problem with the sample) in many cases it's rather easy to ABX the differences.
With an unknown sample it's a lot more difficult. It's also a question of how much time is used on the sample. I remember having considered the harp40_1 Helix ~200 kbps result perfect, whereas after another day of intensive listening tests with other encoders at even 320 kbps I could easily distinguish all the mp3 results from the original. We are able to learn. But is hard learning significant? We do not listen to music this way when enjoying it.
So I think if we have given care with the listening test (while avoiding spending hours on a sample) we should consider an encoder sample transparent if we can't hear a difference. We should accept the fact that this may be just our (maybe temporary) inability to hear the issue. We shouldn't feel challenged too much in finding differences. A little bit of a challenge is necessary to do a good job, but we shouldn't go very far. After all it's about perfectness of encoders under real life conditions, especially at 128 kbps.
  • Last Edit: 27 October, 2008, 07:48:30 AM by halb27
lame3995n -Q0.5

  • Lyx
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #66
Adding to halb's comments:

This may also be relevant to testing-bias. In tests like these, we focus on short samples, with each of them typically representing one single issue. But the purpose of doing that isn't knowing how transparent those test-samples sound like - the purpose is extrapolating the scores to generic encoder quality (so, music in general). As we know, tuning a lossy encoder also is a matter of balance - what improves one issue, may worsen other issues. It is therefore very important, that the test-samples cover a wide range of "scenarios" evenly. Else, the following could happen: testers focus on specific issues and get accustomed to noticing them. The encoder then gets tuned to improve those cases, while unintentionally creating new different problems... but since the testers are focussed on the biased sample pool, the regression stays unnoticed. As visible in this thread, this may happen even if the testers also listen to music with those regressions, but don't recognize them because of being focussed on looking for issues related to the biased sample pool.
I am arrogant and I can afford it because I deliver.

  • Alex B
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #67
Many of us already knew that a MP3 VBR test @ 128 kbps would be quite difficult for the testers. I'd say that all contenders are usable for casual listening. Spotting the small differencies and occasional obvious artifacts needs concentration.

Particularly in this test we should not too easily let the contenders to pass any of the tests "as transparent" if we want to get any meaningful test results. Otherwise the test would be unnecessary. We could just say that VBR MP3 @ 128 kbps is good enough for casual listening and there is no matter which encoder is used.

So I think if we have given care with the listening test (while avoiding spending hours on a sample) we should consider an encoder sample transparent if we can't hear a difference.

I think at least half an hour would be a reasonable time for each test configuration. That would give only 5 minutes on average for each encoder. It is useful to try several different short passages when trying to ABX them and that takes some time.

I spent about an hour with Tom's Diner. The four near transparent encoders were difficult to ABX. I have yet to decide if I give them all the same rating or if I try to put them in some order.
  • Last Edit: 27 October, 2008, 08:34:57 AM by Alex B

  • halb27
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #68
Particularly in this test we should not too easily let the contenders to pass any of the tests "as transparent" if we want to get any meaningful test results. Otherwise the test would be unnecessary. ... I think at least half an hour would be a reasonable time for each test configuration. ...

Sure we shouldn't let the contenders pass too easily, and about half an hour per sample with all 6 encoder results sounds reasonable to me. In case there are suspicious spots 5 minutes per encoder result quickly pass by. But I admit in those cases where I didn't even have a suspicion I didn't spend half an hour.

BTW I'm just through with the first 9 samples, and my primary concern is with having a really silent home when I'm testing and being not pressed to finish a test because of other things waiting for me. So usually I manage to go through 1 or 2 tests per day.
lame3995n -Q0.5

  • Bodhi
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #69

  • halb27
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #70
sample12: for a change a sample with 4 encoders having obvious issues. What a relief!
lame3995n -Q0.5

  • /mnt
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #71
sample12: for a change a sample with 4 encoders having obvious issues. What a relief!

Thats fatboy, alot of codecs struggle at the vocals on this sample and very easy to ABX.
"I never thought I'd see this much candy in one mission!"

  • Alexxander
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #72
Just sent my results. I had a nice and sometimes hard time spotting problems of this nice selection of samples. Curious to see that a few samples were easy to ABX on speakers (no matter what encoder excluding low-anchor). But generally spoken using headphones is a must.

I want to thank those who made possible this mp3 listening test and especially Sebastian.

  • lexor
  • [*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #73
Having trouble running the test here. I've downloaded the zip with abc-hr and java (from the link in the readme). The directory structure is preserved, I can unarchive samples no problem using the appropriate .bat file. Then it breaks.

I get this at start up.
Code: [Select]
D:\Program Files\Java\jre1.5.0_15\bin>java -jar E:\ABC-HR\abchr.jar
java.io.FileNotFoundException: clicodecs.cfg (The system cannot find the file sp
ecified)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(Unknown Source)
        at abchr.settings.DecoderSettings.<init>(Unknown Source)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at java.lang.Class.newInstance0(Unknown Source)
        at java.lang.Class.newInstance(Unknown Source)
        at abchr.gui.ProjectFrame.<clinit>(Unknown Source)
        at abchr.gui.Main.main(Unknown Source)


and this, when I press play.
Code: [Select]
Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException: Line
unsupported: interface SourceDataLine supporting format PCM_SIGNED 44100.0 Hz, 1
6 bit, stereo, 4 bytes/frame, little-endian, and buffers of 88200 to 88200 bytes

        at com.sun.media.sound.PortMixer.getLine(Unknown Source)
        at abchr.audio.PlaybackThread.play(Unknown Source)
        at abchr.gui.PlayAction.actionPerformed(Unknown Source)
        at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
        at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
        at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
        at java.awt.AWTEventMulticaster.mouseReleased(Unknown Source)
        at java.awt.Component.processMouseEvent(Unknown Source)
        at javax.swing.JComponent.processMouseEvent(Unknown Source)
        at java.awt.Component.processEvent(Unknown Source)
        at java.awt.Container.processEvent(Unknown Source)
        at java.awt.Component.dispatchEventImpl(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Window.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.EventQueue.dispatchEvent(Unknown Source)
        at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown Source)

        at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.run(Unknown Source)


What should I do?
  • Last Edit: 29 October, 2008, 09:55:54 AM by lexor
The Plan Within Plans

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #74
I think I know what the problem is... What happens if you do this:

cmd
E:
cd \ABC-HR
D:\Program Files\Java\jre1.5.0_15\bin\java.exe -jar abchr.jar