Skip to main content

Topic: Pre-Test discussion (Read 32093 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • ff123
  • [*][*][*][*][*]
  • Developer (Donating)
Pre-Test discussion
Reply #25
I vote both for keeping Musepack and for using WMA9pro.

ff123

1. BTW, I verified that the time offset of WMA9pro using dbpoweramp to decode is 0.

2. The filesizes of the downloads will be much larger this time around, one reason is the higher bitrate, the other reason is the proprietary formats.  You'll probably have to flac the wma9 and the atrac3.  Are you going to batch encode the ogg, lame, and other encodeable files as well as batch decode?  That would make things smaller.

3.  On classical, fossiles.wav is a possibility, although Filburt once told me he couldn't stand to listen to it because he said it was recorded so badly (I personally am not annoyed by this clip).  So I would vote to replace LifeShatters with fossiles.wav or macabre.wav, and also second guruboolez's suggestion to replace beautyslept.wav with bachpshichord.wav.

  • ErikS
  • [*][*][*][*][*]
Pre-Test discussion
Reply #26
Quote
I suggest to replace BeautySlept - harpsichord sound, but maybe synthetic - by a real instrument recording (as Bachpsichord, or many others).

Just FYI: BeutySlept is one sample where MPC:s PNS screws up. Except a trumpet/horn from the SQAM collection, I haven't heard that as clearly as with this sample, so I think it would be nice to leave it in. Maybe the other "real" harpsichord is as tough though... I wouldn't know. Would you mind doing a quick test, Guruboolez? Could seem unfair to the other codecs to remove a problemsampe fro MPC...?
  • Last Edit: 16 July, 2003, 12:28:31 PM by ErikS

  • boojum
  • [*][*][*][*][*]
  • Members (Donating)
Pre-Test discussion
Reply #27
Ah, there's that harpsichord again.  B)  I agree with Guruboolez that there should be some classical sounds in the test.  Choral would be great: voice, as we all know what that sounds like, and the mass of the symphony orchestra.  How about something from Beethoven's 9th?  Freude! 
Nov schmoz kapop.

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
Pre-Test discussion
Reply #28
One 'problem' with bachpsichord (and with beautyslept too, but at lower degree), is the bitrate reached with --radio profile : near 200 kbps...

P.S. What to you mean by "screw up" ?

  • ErikS
  • [*][*][*][*][*]
Pre-Test discussion
Reply #29
Quote
One 'problem' with bachpsichord (and with beautyslept too, but at lower degree), is the bitrate reached with --radio profile : near 200 kbps...

P.S. What to you mean by "screw up" ?

That it's a clear difference with and without PNS active.

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
Pre-Test discussion
Reply #30
Quote
Quote
One 'problem' with bachpsichord (and with beautyslept too, but at lower degree), is the bitrate reached with --radio profile : near 200 kbps...

P.S. What to you mean by "screw up" ?

That it's a clear difference with and without PNS active.

Is PNS lowering sound quality ? Or is it an exemple of the benefit of Klemm PNS algorithm ?

  • ErikS
  • [*][*][*][*][*]
Pre-Test discussion
Reply #31
Quote
Is PNS lowering sound quality ? Or is it an exemple of the benefit of Klemm PNS algorithm ?

With PNS sounds worse than without. For me I should add... But I don't always trust my own ears - will you try it?

  • bond
  • [*][*][*][*][*]
Pre-Test discussion
Reply #32
if it is generally accepted that wma9 pro will produce better results than wma9 standard then i will vote for using wma9 pro (although hm i dont really want that the codec is getting pushed if it is really good compared to the others  )
I know, that I know nothing (Socrates)

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
Pre-Test discussion
Reply #33
No problem. But for the moment, I must shut my computer down : thunder is more than menacing...

  • Gecko
  • [*][*][*][*][*]
Pre-Test discussion
Reply #34
I'm not in favor of adding any other form of lowpassing other than the codecs use themselves. Most importantly we should be testing real world performance imo. Also expect alot of people who mistake hifi with lots of high frequencies to complain about overall dull sound. It will be a mess to explain this step to all those self appointed experts (especially outside of HA) who will disregard such a testing method. It all depends on what you want this test to be.

Another problem of artificial lowpassing would be readjusting the bitrate of the vbr codecs.

One valid reason for an artificial lowpass might be to reduce ogg Vorbis' hf boosting bug.

I'm not sure about adding mpc. People will extrapolate the results of the 128k test to other bitrates and this might shed a wrong light on mpc. Mpc doesn't suck at this bitrate, but I expect ogg or aac to sound better by quite a margin. Mpc is not intended to be used at 128 and probably noone will, so why bother testing? On the other hand, maybe mpc will present itself as a viable alternative and this is a good chance to find out.

Oh yeah, I think trance and celtic music is definately underrepresented in this test.

  • rjamorim
  • [*][*][*][*][*]
Pre-Test discussion
Reply #35
@Gecko: I think you didn't understand the purpose of the anchor. The anchor is only one file, and it suffers lowpass (or any other process to degrade quality) in order to put things into perspective and to avoid codecs being rated too low.

The samples encoded with the competing codecs suffer no preprocessing like lowpass.

About MPC: From some preliminary tests conduced by Den (and Guruboolez?), I would expect MPC to win this test, or at least come close to the winner. :B

Regards;

Roberto.
  • Last Edit: 16 July, 2003, 02:49:34 PM by rjamorim
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • rjamorim
  • [*][*][*][*][*]
Pre-Test discussion
Reply #36
Quote
I vote both for keeping Musepack and for using WMA9pro.

OK... unless someone comes with good reasons to use WMA std or to sack Musepack, I'll go with your vote.

Quote
1. BTW, I verified that the time offset of WMA9pro using dbpoweramp to decode is 0.


OK. Thanks a lot.

Quote
2. The filesizes of the downloads will be much larger this time around, one reason is the higher bitrate, the other reason is the proprietary formats.  You'll probably have to flac the wma9 and the atrac3.  Are you going to batch encode the ogg, lame, and other encodeable files as well as batch decode?  That would make things smaller.


Yeah, I would also batch process the anchor, now I only need to know if I'll use Blade or Sox for that. >_<  (blade = encode, sox = lowpass)

If we use blade, I would also have to find out if there's any offset introduced when decoding it's encodes with Lame. <hint>

Quote
3.  On classical, fossiles.wav is a possibility, although Filburt once told me he couldn't stand to listen to it because he said it was recorded so badly (I personally am not annoyed by this clip).


So, I think we can use Macabre instead, just to be safe.

Quote
So I would vote to replace LifeShatters with fossiles.wav or macabre.wav, and also second guruboolez's suggestion to replace beautyslept.wav with bachpshichord.wav.


Good. It's settled then.

@ErikS: I'm using (mostly) the same sample suite because it's a good suite (representing several different styles) and because I'm lazy.

I'm thinking about calling for new samples for the 64kbps extension test, but it's still too early to talk about that.
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • rjamorim
  • [*][*][*][*][*]
Pre-Test discussion
Reply #37
Ok, so what's (nearly) settled (unless someone comes with good arguments)

- Atrac3 will be tested
- Musepack will be tested
- WMA PRO will be tested instead ow WMA std
- LifeShatters will be replaced with macabre and beautyslept with bachpshichord

What's not settled:

- Will the anchor use lowpass or Blade? If it uses lowpass, one lowpass for all samples or one for each sample? If it uses Blade, plain 128kbps CBR?

Regards;

Roberto.
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • JohnV
  • [*][*][*][*][*]
  • Developer
Pre-Test discussion
Reply #38
Quote
Quote
I vote both for keeping Musepack and for using WMA9pro.

OK... unless someone comes with good reasons to use WMA std or to sack Musepack, I'll go with your vote.

What kind of "voting" that is? 
If this will be solved by voting, you should create a poll...

Anyway, I vote exactly opposite to ff123: sack wma9pro and MPC and keep Atrac. 
Juha Laaksonheimo

  • JohnV
  • [*][*][*][*][*]
  • Developer
Pre-Test discussion
Reply #39
Quote
Ok, so what's (nearly) settled (unless someone comes with good arguments)

- Atrac3 will be tested
- Musepack will be tested
- WMA PRO will be tested instead ow WMA std

6 codecs+anchor+original is hell of a job for those who want to do the full test with all samples.
Juha Laaksonheimo

  • ErikS
  • [*][*][*][*][*]
Pre-Test discussion
Reply #40
Where can I find this Bachpsichord samle?

  • rjamorim
  • [*][*][*][*][*]
Pre-Test discussion
Reply #41
Quote
sack wma9pro and MPC and keep Atrac. 

Sack WMA and MPC, favouring Atrac??? Heck, Atrac3 loses to other codecs (in features) in nearly every imaginable way (not multi platform (only Windows)), only CBR, no open decoder or encoder, not usable in media players like Winamp or Foobar, not usable in movies besides Real ones (unless you do hacks)...

I don't understand what is going on here. :B

And I agree 6 codecs is nearly too much, but, IMO, most of them aren't "sackeable". I see good reasons for including MPC, but, still, I think a poll would be the way to go.



@ ErikS: Ask Guruboolez
  • Last Edit: 16 July, 2003, 03:39:03 PM by rjamorim
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • ff123
  • [*][*][*][*][*]
  • Developer (Donating)
Pre-Test discussion
Reply #42
I'll try out blade tonight to see what it will sound like on the test suite and also to see if it has an offset

ff123

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Pre-Test discussion
Reply #43
Too bad Lifeshatters is out ... I found the clip 'interesting'..

  • bond
  • [*][*][*][*][*]
Pre-Test discussion
Reply #44
Quote
Atrac3 ... not usable in media players like Winamp

no legal way

Quote
not usable in movies besides Real ones

should be possible with gabest's filter, i think...
I know, that I know nothing (Socrates)

Pre-Test discussion
Reply #45
 I would like Atrac3 comparision!

  • ScorLibran
  • [*][*][*][*][*]
  • Banned
Pre-Test discussion
Reply #46
Is the testing something a newbie like myself could help with?  I actually tried to get in on the bitrate analysis phase, and had everything I needed to do it downloaded and ready to go.  Then I got really busy re-encoding my collection and didn't have the time or HD space to follow through.

Now that I'm finishing up all my encoding, I'd like to take part in the ABX phase, unless a group has already been identified to do the testing.


  • Dibrom
  • [*][*][*][*][*]
  • Administrator
Pre-Test discussion
Reply #47
I've only briefly skimmed this thread, so I could be missing something, but here's my opinion:  I would personally like to see MPC included in this test, especially if this test is going to be geared more towards the users of this board than perhaps the general populace (and let's face it, it probably will be even if designed otherwise).  There are a lot of MPC users here, and there has always been an open ended question about it's low bitrate performance.  It'd be nice to see it put to the test once again.

I think the same argument above probably would work against atrac.  While it is fairly widely used overall, I don't think it is particularly widely used on this board.  Furthermore, it'd almost seem like a purely academic excercise to include it since I figure that it doesn't have a good chance of doing very well at all considering the more advanced and more highly tuned codecs it will be competing with.  It could perhaps provide some useful results to point people to, but I believe most people who are serious about audio compression (which are the people who will appreciate these results the most) already realize that the alternatives are better on just about every front.

As for WMA vs WMA Pro, I'm not really sure.  If we test WMA Pro, it has to be made very clear that there is a distinction between this and normal WMA, and that WMA Pro will not work on most (any?) portable devices.  I think maybe it'd be best to actually test both, but that might push the codec count too high.

Personally, I'd favor dropping Atrac first out of those available.

Perhaps it'd be best to really define the target audience of the test before deciding which to include and which not to.  Are we testing for the masses here?  If so, then MPC doesn't make sense and WMA Pro probably doesn't either.  Atrac would though.  If we are testing for the more enthusiastic and technically minded users (like the people here on HA  -- again, those who I think would appreciate these results the most also), then WMA Pro makes sense over normal WMA and MPC makes more sense than Atrac I think.  If we're trying to strike a balance, then maybe it'd be best to go with my original suggestion:  WMA/WMA Pro and MPC instead of Atrac.

Just my $0.02...
  • Last Edit: 16 July, 2003, 04:34:35 PM by Dibrom

  • bond
  • [*][*][*][*][*]
Pre-Test discussion
Reply #48
imho the audience will be all people who transcode audio cause it will be the first independent representative codec comparison @128 (as i know)

on any discussion on the net (professional or not) about codec quality this test could be quoted (which makes it very usefull)

so including the most used codecs will be necessary to keep the test as much usefull for as most people as possible (not only for some professional super-minds  )
I know, that I know nothing (Socrates)

  • Gecko
  • [*][*][*][*][*]
Pre-Test discussion
Reply #49
Quote
@Gecko: I think you didn't understand the purpose of the anchor. The anchor is only one file, and it suffers lowpass (or any other process to degrade quality) in order to put things into perspective and to avoid codecs being rated too low.
The samples encoded with the competing codecs suffer no preprocessing like lowpass.

Ok, thx for clearing that up. I guess this was confusing me. Heh, actually, it still is:
Quote
b) JohnV votes for one lowpass for each sample, else some samples might sound better lowpassed than their original versions. ff123 votes for one fixed lowpass for all samples, to put things into perspective.