Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: New Listening Test (Read 107092 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

New Listening Test

Reply #150 –
Quote from: digitalradiotech on
Quote from: Ivan Dimkovic on

Quote

I agree with you, range 0-100 is silly.


It is the range standardized in ITU for the ITU-R BS.1534 recommendation - testing of audio signals with large impairment, such as codecs at 48 kbps.



I don't see the point in the MUSHRA test, to be honest. It seems to be more of an excuse to allow mediocre-performing codec/bit rate combinations to get high scores. If a codec/bit rate combination does cause a large impairment, then I think it should be scored as a large impairment on the original BS.1116 test, and the MUSHRA test just makes it more difficult to compare the performance of codecs/bit rates because it creates a totally new scale and confuses lay people, because they see a high score (e.g. CT 48kbps HE AAC getting a score of 88 on a MUSHRA test) and think it means that the codec provides high quality at that bit rate, when the recent HE AAC test showed that it's not all that good in absolute terms. I think it's best to stick with using BS.1116.


So, your only argument not to use MUSHRA is that you suppose that it could be an "excuse" to get high scores that might not be warranted in absolute terms? Do you know what these listening tests are used for in the ITU? First and foremoest, to find the best encoding solution, with the best possible accuracy and discriminating power.

Did you even read Gabriel's post?

http://www.hydrogenaudio.org/forums/index....ndpost&p=381952

New Listening Test

Reply #151 –
I edited my post while you were replying, so the edited version might cast more light on why I think changing to the 0-100 scale isn't a good idea. Basically, it's to avoid confusing the results from BS.1116 and MUSHRA tests, because they do use very different testing methodologies.

Maybe I was being a bit harsh when I described 48kbps HE AAC as "mediocre", but do you think the CT 48kbps HE AAC deserved a score as high as 88? In absolute terms I don't think it deserves 88.

As I said in my edited post, I think it would be better to either stick with the 0-5 scale or change completely to MUSHRA, because you can just imagine the confusion you'd get when a MUSHRA test gives a score of 88 and then a BS.1116 test with a scale of 0-100 only gives it 70.

And fair enough, I can see the reasoning behind MUSHRA, and saying that it's just an excuse to allow very high scores for mediocre-performing codecs/bit rate combinations is a bit harsh, but given some of the claims made about 48kbps HE AAC, such as "CD-quality stereo down to 48 kbps", I think a 48kbps test should use BS.1116.

New Listening Test

Reply #153 –
OK, here is what I thought...
I sent MS a last reminder and told them to reply by the end of next week (21.05 the latest). If not, I told them I am going to test 9.1.
Now, the Vista EULA says I am not allowed to disclose the results of benchmarks - benchmarking itself is allowed. If MS doesn't give me the permission to publish the results yet, I could simply include the codec in the test, but exclude it from the results until the EULA changes. In that way, I am not breaking EULA and I also tested it against HE-AAC and Vorbis. I am doing this because I am not sure when the next multiformat test at 48 kbps is going to take place. Also, only testing WMA 10 against WMA 9.1 later in order be able to compare the results will not work since the testing environment changed between the upcoming multiformat test and the WMA vs. WMA test (even if the same samples are used, maybe other listeners will participate and even if I also have the same testers, they might be in a different mood or whatever). I hope you understand what I mean.

Edit: What I mean is this... Let's say we run the multiformat test and the following results are available:

WMA 9.1: 3
HE-AAC: 3.5
Vorbis: 4

If we later test WMA 9.1 against WMA 10 and have the following results:

WMA 9.1: 2.5
WMA 10: 4

we cannot say that WMA 10 is as good as Vorbis, because the testing environment changed between the two tests. Even if WMA 9.1 would still get 3 in the second test, it wouldn't mean anything.

New Listening Test

Reply #154 –
Quote from: Sebastian Mares on
Now, the Vista EULA says I am not allowed to disclose the results of benchmarks - benchmarking itself is allowed. If MS doesn't give me the permission to publish the results yet, I could simply include the codec in the test, but exclude it from the results until the EULA changes.


That would still be illegal.

The EULA applies to the program you are using. If you encode samples with a version which EULA forbids benchmark disclosure, you can never disclose said benchmarks, even if a later version of the same program changes EULA, since that new EULA will only apply to the later version, and not the old version.

New Listening Test

Reply #155 –
Microsoft says that WMA10Pro+ will give better quality @64 Kb/s than aacPlus v2 so take WMA10Pro+ to test  is it true that new WMP11 (with new WMA10 codecs) will be to download on 17 mai? When you install WMP11 then in XP system in Windows Media Encoder you will see new codecs to converting (WMA10Voice, WMA10, WMA10Pro and WMA10Pro+)
 

New Listening Test

Reply #156 –
Quote from: Garf on
Quote from: digitalradiotech on
Quote from: Ivan Dimkovic on

Quote

I agree with you, range 0-100 is silly.


It is the range standardized in ITU for the ITU-R BS.1534 recommendation - testing of audio signals with large impairment, such as codecs at 48 kbps.



I don't see the point in the MUSHRA test, to be honest. It seems to be more of an excuse to allow mediocre-performing codec/bit rate combinations to get high scores. If a codec/bit rate combination does cause a large impairment, then I think it should be scored as a large impairment on the original BS.1116 test, and the MUSHRA test just makes it more difficult to compare the performance of codecs/bit rates because it creates a totally new scale and confuses lay people, because they see a high score (e.g. CT 48kbps HE AAC getting a score of 88 on a MUSHRA test) and think it means that the codec provides high quality at that bit rate, when the recent HE AAC test showed that it's not all that good in absolute terms. I think it's best to stick with using BS.1116.


So, your only argument not to use MUSHRA is that you suppose that it could be an "excuse" to get high scores that might not be warranted in absolute terms? Do you know what these listening tests are used for in the ITU? First and foremoest, to find the best encoding solution, with the best possible accuracy and discriminating power.

Did you even read Gabriel's post?

http://www.hydrogenaudio.org/forums/index....ndpost&p=381952


I think there isn't even a point discussing wether to use MUSHRA or not, since we have no access to a working comparator.

Maybe somebody interested in it could contact Schnofler?

New Listening Test

Reply #157 –
Quote from: Speechless on
Microsoft says that WMA10Pro+ will give better quality @64 Kb/s than aacPlus v2 so take WMA10Pro+ to test  is it true that new WMP11 (with new WMA10 codecs) will be to download on 17 mai? When you install WMP11 then in XP system in Windows Media Encoder you will see new codecs to converting (WMA10Voice, WMA10, WMA10Pro and WMA10Pro+)


I don't have any official information about the release date of Windows Media Player 11.

New Listening Test

Reply #158 –
Quote from: rjamorim on
I think there isn't even a point discussing wether to use MUSHRA or not, since we have no access to a working comparator.

Maybe somebody interested in it could contact Schnofler?

http://www.hydrogenaudio.org/forums/index....ndpost&p=382210
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

New Listening Test

Reply #159 –
Microsoft WMP 11 released, and can be downloaded here:
http://www.9down.com/story.php?sid=6574

Don't know if this package include the new codec or not.

New Listening Test

Reply #160 –
Don't know if it's already been mentioned, but does the Microsoft EULA even apply in Germany? Wikipedia says it does not: http://de.wikipedia.org/wiki/EULA (german)

Edit: Should have read the Wikipedia article to the end. It seems, Microsoft EULA does indeed apply here, because the software needs to be downloaded from the internet. It it was bought in a store it would not apply, since the contract about the use of the software is made while buying it, and as you cannot read the EULA before buying it would not be valid.
So...
Blubb

New Listening Test

Reply #162 –
Quote from: Enig123 on
Microsoft WMP 11 released, and can be downloaded here:
http://www.9down.com/story.php?sid=6574

Don't know if this package include the new codec or not.


Can also download from http://www.shooter468.com/ccount/click.php?id=32

Quote from: Enig123 on
Don't know if this package include the new codec or not.


WMP11:



The Good
Writes ID3 tags to the physical file regardless where the rip folder is
Windows Media Audio 10 Professional codec
Windows Media Audio 9.2 codec
Expanded tile view
Live search
Very responsive media library

The Bad
Aggressively replaces my high quality Folder.jpg with a low res Folder.jpg
Does not notify the user when ID3 tag writing fails
Does not let remove monitor folders (only ignore)


WME9:



Two new codecs:

Windows Media Audio 9.2
Windows Media Audio 10 Professional

Cheers,
McoreD

New Listening Test

Reply #164 –
Quote from: guruboolez on
http://www.hydrogenaudio.org/forums/index....ndpost&p=382210


Hrm... OK, but custom scale is not enough to be a MUSHRA test. There must also be only one hidden reference for the whole test, and not one for each sample.

New Listening Test

Reply #165 –
Also to be true MUSHRA, there must be at least one anchor, and the recommendation for that first anchor is 3.5 kHz lowpassed.  After all, the procedure is termed MUlti Stimulus test with Hidden Reference and Anchors.  I think this part is probably more important than the 100-interval scale or the labels or maybe even eliminating the side-by-side reference for each codec, because the anchors are what standardizes the ratings.

I wonder if a current 48 kHz test isn't becoming too good for the original intent of MUSHRA?

ff123

New Listening Test

Reply #166 –
Quote from: Enig123 on
Microsoft WMP 11 released, and can be downloaded here:
http://www.9down.com/story.php?sid=6574

Don't know if this package include the new codec or not.

It works. WMAStd 9.2 & WMAPro 10 (or 11) is automatically working with different audio tools (like dBpoweramp). WMAPro is now working with very low bitrate (dbPowerAmp crashed with bitrate < 48 kbps). It shows interesting things like very high lowpass value, similar to SBR encoders but without SBR artefacts (but with other ones, clearly more irritating IMO).
The decoding side is less complex than SBR encoders: WMAPro at low bitrate is twice faster on decoding. Encoding speed has been clearly lowered with WMAPro.
I noticed strong regressions with WMA 9.2 Std ~128 kbps with harpsichord.

I didn't played that much this week-end with this encoder.
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

New Listening Test

Reply #167 –
Both new WMA work fine for me.

New Listening Test

Reply #168 –
I feel compelled to point out that there are 2 versions of WMA 10 Pro included with WMP11.  The first version--"WMA10Pro+"--is available only within WMP (under "ripping options") as Windows Media Audio Pro using SBR and PS (or direct clones of these technologies), which is quite obvious not because of the artifacts and similar sound (that would be TOS #8 ) but becuase when played on my pocket pc, it plays as mono 22khz. 

The second version of WMA 10 Pro (no plus) is usable via WME and external applications, does not use SBR or PS and is playable on my pocket pc flawlessly.

Just thought I'd point out that piece of important info

edit: Also, I'd like to refer your attention to the following post from Doom9's forum: http://forum.doom9.org/showpost.php?p=827660&postcount=2, which states:
Quote
That's a leaked build 11.0.5358.4826 from April 19th. I'm pretty sure that wasn't the final beta build...
Copy Restriction, Annulment, & Protection = C.R.A.P. -Supacon

New Listening Test

Reply #169 –
The Sheep of DEATH, thanks for that info. Unfortunately "WMA10Pro+" version still displays as "Windows Media Audio 10 Professional". Hope they make the difference clear.

WMA ripped from WMP11:

Windows Media Audio 10 Professional
128 kbps, 44 kHz, 2 channel 16 bit 1-pass CBR

WMA converted using WME9 while having WMP11:

Windows Media Audio 10 Professional
128 kbps, 44 kHz, 2 channel 16 bit 1-pass CBR

New Listening Test

Reply #172 –
Did you guys read point 6 of the EULA?

Quote
6. SCOPE OF LICENSE.  The software is licensed, not sold. This agreement only gives you some rights to use the software.  Microsoft reserves all other rights.  Unless applicable law gives you more rights despite this limitation, you may use the software only as expressly permitted in this agreement.  In doing so, you must comply with any technical limitations in the software that only allow you to use it in certain ways.    You may not

* disclose the results of any benchmark tests of the software to any third party without Microsoft’s prior written approval;
* work around any technical limitations in the software;
* reverse engineer, decompile or disassemble the software, except and only to the extent that applicable law expressly permits, despite this limitation;
* make more copies of the software than specified in this agreement or allowed by applicable law, despite this limitation;
* publish the software for others to copy;
* rent, lease or lend the software;
* transfer the software or this agreement to any third party; or
* use the software for commercial software hosting services.

New Listening Test

Reply #173 –
Did this version of wma 10 use SBR+PS because sound's very good and suppored by and hardware player.

New Listening Test

Reply #174 –
As far as everyone here knows - there is no public information that Microsoft is using SBR and PS technologies in their codecs, so I don't think it is a good idea anyway to call these technologies with those names.

For sure, it seems from the evidence on this forum, that Microsoft is indeed using some frequency-enhancing technology, maybe similar to HE-AAC's SBR - but what kind of technology is that exactly it is still unknown.