Skip to main content

Topic: SV8 beta speed testing (Read 9976 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • Alexxander
  • [*][*][*][*]
SV8 beta speed testing
Congrats for the new release.

After at least three years I played again with mpc. Besides confirming that this Musepack version encodes and decodes faster than ogg (ogg2.83-aoTuVb5), aac (Nero Digital™ Audio+ 1.1.34.2) and mp3 (lame3.98b5) by at least 20% (Intel Core2Duo at 3 GHz) I have compared mentioned encoders at bitrates around 130-135 kbps (q 0.4 and -V4 -V5) using 4 albums of pop and disco genre.

aac -> not abx-able
ogg -> nearly never abx-able
lame -V4 -V5 and mpc Beta 1.16 (SV8) -> often abx-able

I'm impressed by aac, but if you don't pay much attention when listening, all encoded music in this bitrange sound terrific.

Question: has mpc room for tuning in the 128 kbps range?
  • Last Edit: 26 September, 2007, 09:30:49 AM by Alexxander

  • shadowking
  • [*][*][*][*][*]
SV8 beta speed testing
Reply #1
Congrats for the new release.

After at least three years I played again with mpc. Besides confirming that this Musepack version encodes and decodes faster than ogg (ogg2.83-aoTuVb5), aac (Nero Digital™ Audio+ 1.1.34.2) and mp3 (lame3.98b5) by at least 20% (Intel Core2Duo at 3 GHz) I have compared mentioned encoders at bitrates around 130-135 kbps (q 0.4 and -V4) using 4 albums of pop and disco genre.

aac -> not abx-able
ogg -> nearly never abx-able
lame -V4 and mpc Beta 1.16 (SV8) -> often abx-able

I'm impressed by aac, but if you don't pay much attention when listening, all encoded music in this bitrange sound terrific. Question: has mpc room for tuning in the 128 kbps range?


Since when is Lame -V4 in the 130 k range ??
I average around 160 k with 20 GB collection.
  • Last Edit: 26 September, 2007, 07:50:29 AM by shadowking
wavpack -b4x4s1c

  • Alexxander
  • [*][*][*][*]
SV8 beta speed testing
Reply #2
Since when is Lame -V4 in the 130 k range ??
I average around 160 k with 20 GB collection.


You're right, I was talking about -V5 and have corrected it in my post. The -V4 average I have is 156 kbps (50 GB mp3 encoded from lossless with lame3.98b5).

  • sld
  • [*][*][*][*][*]
SV8 beta speed testing
Reply #3
Congrats for the new release.

After at least three years I played again with mpc. Besides confirming that this Musepack version encodes and decodes faster than ogg (ogg2.83-aoTuVb5)

On my Core Duo (Yonah), mppenc SV8 encodes at 17x while the SSE3MT Lancer build of aoTuVb5 encodes at 40x.

But no other familiar lossy codec beats Musepack at decoding, it's the only one that does 200+x on my setup.
  • Last Edit: 28 September, 2007, 05:15:10 AM by sld

  • GenjuroXL
  • [*]
SV8 beta speed testing
Reply #4
On my Core Duo (Yonah), mppenc SV8 encodes at 17x while the SSE3 Lancer build of aoTuVb5 encodes at 40x.

But no other familiar lossy codec beats Musepack at decoding, it's the only one that does 200+x on my setup.


Something's not right with that 17x number. Even I get 26.5x on my Athlon64 3500+

  • sld
  • [*][*][*][*][*]
SV8 beta speed testing
Reply #5

On my Core Duo (Yonah), mppenc SV8 encodes at 17x while the SSE3 Lancer build of aoTuVb5 encodes at 40x.

Something's not right with that 17x number. Even I get 26.5x on my Athlon64 3500+

Can you test your processor with Lancer Vorbis SSE3MT? It may simply be due to architectural differences. My Yonah's a 1.8 GHz and I'm using a laptop.
  • Last Edit: 28 September, 2007, 05:15:31 AM by sld

  • vlada
  • [*][*][*][*]
SV8 beta speed testing
Reply #6

On my Core Duo (Yonah), mppenc SV8 encodes at 17x while the SSE3 Lancer build of aoTuVb5 encodes at 40x.

But no other familiar lossy codec beats Musepack at decoding, it's the only one that does 200+x on my setup.


Something's not right with that 17x number. Even I get 26.5x on my Athlon64 3500+



Dual core CPUs are known to be much slower then single cores when you use programs which are not optimized for multithreading.

  • GenjuroXL
  • [*]
SV8 beta speed testing
Reply #7
Can you test your processor with Lancer Vorbis SSE3MT? It may simply be due to architectural differences. My Yonah's a 1.8 GHz and I'm using a laptop.

Test result:
mpcenc --quality 7: 26.52x (exact number)
lancer SSE3MT -q 7: 27,95x (rounded up)

(Note that I have a single core Athlon64.)

  • sld
  • [*][*][*][*][*]
SV8 beta speed testing
Reply #8

Can you test your processor with Lancer Vorbis SSE3MT? It may simply be due to architectural differences. My Yonah's a 1.8 GHz and I'm using a laptop.

Test result:
mpcenc --quality 7: 26.52x (exact number)
lancer SSE3MT -q 7: 27,95x (rounded up)

(Note that I have a single core Athlon64.)

Haha, I've been spoilt silly by Lancer SSE3MT. Musepack is really THE speed king, though I still want a SSE3MT version of mppenc!

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
SV8 beta speed testing
Reply #9
Dual core CPUs are known to be much slower then single cores when you use programs which are not optimized for multithreading.


This should get a TOS8 warning, somehow.

  • echo
  • [*][*][*]
SV8 beta speed testing
Reply #10


Dual core CPUs are known to be much slower then single cores when you use programs which are not optimized for multithreading.


This should get a TOS8 warning, somehow.

What does this have to do with sound quality?

Quote
8. All members that put forth a statement concerning subjective sound quality, must -- to the best of their ability -- provide objective support for their claims. Acceptable means of support are double blind listening tests (ABX or ABC/HR) demonstrating that the member can discern a difference perceptually, together with a test sample to allow others to reproduce their findings. Graphs, non-blind listening tests, waveform difference comparisons, and so on, are not acceptable means of providing support.



  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
SV8 beta speed testing
Reply #11
Besides confirming that this Musepack version encodes and decodes faster than ogg (ogg2.83-aoTuVb5), aac (Nero Digital™ Audio+ 1.1.34.2) and mp3 (lame3.98b5) by at least 20% (Intel Core2Duo at 3 GHz) I have compared mentioned encoders at bitrates around 130-135 kbps (q 0.4 and -V4 -V5) using 4 albums of pop and disco genre.


My own tests:

HARDWARE SETTINGS:
• Intel Core2Duo E6300
• MSI-7318 motherboard
• 2x512 MB DDR (533 Mhz?)
• 320 GB HDD, partitioned and defragmented

SOFTWARE SETTINGS:
• test file : 9:58.933 (26412960 samples)
• encoding GUI & speed values: foobar2000 0.9.4.5b
• encoders: see complete log files for information


NOTE:
The test is restricted to the encoders I already have on my harddrive. Three different categories of settings:
- portable bitrate (~130 kbps)
- ultra-high lossy bitrate (~320 kbps)
- lossless (only WavPack & FLAC)

I performed three encodings for each setting and I kept the fastest one.

Most important: these results applies for one single file encoding. One core is used (excepted for Vorbis LANCER, which is the only multithreaded audio encoder available).

RESULTS:

Code: [Select]
          FRIENDLY BITRATE

ENCODER           SPEED   BASIS¹ BITRATE
-------------------------------------------
AAC Nero          21.97x    93   129 kbps
MP3 Fraunhofer    43.75x   185   144 kbps
MP3 LAME          20.03x    85   139 kbps
MPC SV8           23.57x   100   160 kbps
Vorbis Lancer MT  72.32x   307   138 kbps
Vorbis Lancer     50.90x   216   138 kbps


          PARANOIAC BITRATE

ENCODER           SPEED   BASIS¹ BITRATE
-------------------------------------------
AAC Nero          17.77x    75   332 kbps
MP3 Fraunhofer   109.83x   466   320 kbps
MP3 LAME          17.34x    74   320 kbps
MPC SV8           21.57x    92   385 kbps
Vorbis Lancer MT  69.57x   295   312 kbps
Vorbis Lancer     48.39x   205   312 kbps
WavPack lossy     68.93x   292   343 kbps


          LOSSLESS

ENCODER           SPEED   BASIS¹ BITRATE
-------------------------------------------
FLAC -0          206.03x   874   760 kbps
FLAC -5           99.82x   424   723 kbps
FLAC -8           29.08x   123   719 kbps
WavPack -f       126.09x   535   747 kbps
WavPack          106.19x   451   726 kbps
WavPack -h        80.87x   343   720 kbps
WavPack -hh       65.30x   277   713 kbps


¹ MPC SV8 beta --quality 4 = 100 basis (example: AAC Nero (~130 kbps) is ~7% [0,93 or 100-93] slower than musepack (~130 kbps) but WavPack lossy is 2,92x faster.

MPC is slightly faster (7...15%) than Nero AAC and LAME, but much more slower than Vorbis (up to 3 time faster), Fraunhofer MP3 (4.66x faster at high bitrate), WavPack lossy (~3x faster) - and abysmally slower than true audiophile (lossless) encoder (up to ~9x slower).

In other words LAME, Nero Digital and MPC's are more comparable to turtles than rabbits. Musepack is in 2007 anything but a fast encoder¹ (unlike in 2001, golden age of the format) but is indeed less slow -consolation- than two big competitors

Musepack is really THE speed king
...Yes, in the kingdom of turtles and snails 



Anyway, congrats to the MPC developers - though SV8 only adds some fundamental stuff (seeking, possible video muxing) other formats got for a decade.


N.B. information about encoders'version & settings used in my test here


___
¹: as already shown in the amazing test performed by Nyaochi.
  • Last Edit: 29 September, 2007, 07:27:54 AM by guruboolez

  • GenjuroXL
  • [*]
SV8 beta speed testing
Reply #12
Just a reminder, guys. The current SV8 encoder doesn't use any ASM optimizations. Speed isn't the top priority (yet). We'll have to wait until the optimizations are implemented. Right now the encoder is plain C. Comparing it to Lancer, which is optimized to hell and back, is not quite fair .
  • Last Edit: 29 September, 2007, 09:40:15 AM by GenjuroXL

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
SV8 beta speed testing
Reply #13
Just a reminder, guys. The current SV8 encoder doesn't use any ASM optimizations. (...) Comparing it to Lancer, which is optimized to hell and back, is not quite fair .

I made and published this table/test in order to break the neck to some wrong statements about SV8's position in speed hierarchy (see some messages above).
We will see in the future how much speed mpcenc will gain with optimisations.

  • sld
  • [*][*][*][*][*]
SV8 beta speed testing
Reply #14
Comparing it to Lancer, which is optimized to hell and back, is not quite fair .

I don't see anything unfair with it, if a person making decisions due to speed (which IMO is quite superficial) has to decide NOW. Most of us don't decide on that basis anyway, or we'd be using the Gogo(?) version of LAME instead of the latest one.

My use of SV8 will be limited to the SV7 to SV8 conversion process. I'm encoding new CDs to TAK nowadays.

...Yes, in the kingdom of turtles and snails

Well, I'm sorry, I have my own informal results which conclude like yours, but since they are informal, I'll trust someone else's especially if their explanation for their results are convincing. It just so happens that they have a pro-Musepack agenda, perhaps?
  • Last Edit: 29 September, 2007, 01:19:35 PM by sld

  • Destroid
  • [*][*][*][*][*]
SV8 beta speed testing
Reply #15
I always thought MPC decoding speed was the advantage over MP3 and Vorbis 
"Something bothering you, Mister Spock?"

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
SV8 beta speed testing
Reply #16
It is one advantage - probably the last remaining one over LAME and Vorbis.

  • niktheblak
  • [*][*][*][*]
  • Members (Donating)
SV8 beta speed testing
Reply #17
Dual core CPUs are known to be much slower then single cores when you use programs which are not optimized for multithreading.

That's only true if you compare a slow dual core Pentium D against a faster single core Pentium 4. Core 2 Duo will beat a single core Pentium 4 in pretty much any computation, regardless of whether it's optimized for multi-threading or not.

You could say that a Core 2 Duo beats a Pentium 4 left-handed, i.e. using only one core

  • Alexxander
  • [*][*][*][*]
SV8 beta speed testing
Reply #18
I took some time to write down speed measurements to backup my statements in the starting posts after reading guruboolez measurements and comments. 

CPU Intel E6600 at 3.0 GHz with FSB at 334 GHz on Asus P5B motherboard
RAM DDR2 at 833 GHz

flac v1.2.0
lame3.98b5
ogg2.83-aoTuVb5
Nero Digital™ Audio+ 1.1.34.2
Musepack-SV8-beta

Test file is a single wavfile made of 4 pop albums: 3:22:53.943 (536870900 samples) 2.753.241.836 bytes
En- and decoding were done from the commandline. Speed- and timemeasurements were copied when coder displayed these information, if not I used a stopwatch to measure encoding time (Nero and FLAC). I know, a bit primitive!    Tests were done twice and afterwards I eliminated the slowest measurements.

Source files are on a WD5000AAKS and resulting files are written to an other WD5000AAKS hard disk. OS Windows XP is on an other drive and both WD hard drives had no activity other then that caused by the encoding or decoding process. I wanted to ensure maximum hard disk speeds (especially flac speeds can decrease a lot caused by other disk activity).

Code: [Select]
           avg.bitrate  enc.speed  dec.speed
flac -0     1022 kbps     295x       358x
flac -5      956 kbps     152x       338x
mp3 -V5      136 kbps      31x        49x
aac -q 0.4   136 kbps      32x       137x
ogg -q4      132 kbps      26x        78x
mpc -q4      134 kbps      41x       489x

The decoding speed of mpc is impressive. Maybe that's the reason why they say batterylife of Rockbox firmware is longer with mpc then with mp3 or ogg. 

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
SV8 beta speed testing
Reply #19
x49 decoding speed for MP3 on a C2D E6600?! My Duron 800 used to decode MP3 faster.
What OS are you running? If you're on a Win32 platform I strongly suggest to use foobar2000's "decoding speed test" embedded tool rather than a stopwatch. My C2D E6300 just measure LAME -V5 decoding at 175x.

  • Alexxander
  • [*][*][*][*]
SV8 beta speed testing
Reply #20
x49 decoding speed for MP3 on a C2D E6600?! My Duron 800 used to decode MP3 faster.
What OS are you running? If you're on a Win32 platform I strongly suggest to use foobar2000's "decoding speed test" embedded tool rather than a stopwatch. My C2D E6300 just measure LAME -V5 decoding at 175x.

I run WinXP SP2 up-to-date. I didn't know about the mentioned optional plugin. I installed it and got this:

Code: [Select]
            Speed (x realtime)
flac -0         335.022
flac -5         299.440
mp3 -V5         281.873
aac -q 0.4      362.783
ogg -q4         298.766
mpc -q4         415.586

Compared with my results measured with commandline encoding flac and mpc show lower decoding speeds (mpc still being the winner) but mp3, aac and ogg are significantly higher. This is strange, I will look into it in a few days. An explanation will be hard to find I suppose.

And: measuring with a stopwatch isn't the point because tenths of seconds or even a deviation of two seconds doesn't differ too much the result (and I'm not drunk or whatever  )

  • haregoo
  • [*][*][*]
SV8 beta speed testing
Reply #21
You'd be suprised if you try Lancer build of oggdec. Don't forget to set -t option for benchmark.

  • capma
  • [*]
SV8 beta speed testing
Reply #22
I took some time to write down speed measurements to backup my statements in the starting posts after reading guruboolez measurements and comments. 

CPU Intel E6600 at 3.0 GHz with FSB at 334 GHz on Asus P5B motherboard
RAM DDR2 at 833 GHz

flac v1.2.0
lame3.98b5
ogg2.83-aoTuVb5
Nero Digital™ Audio+ 1.1.34.2
Musepack-SV8-beta

Test file is a single wavfile made of 4 pop albums: 3:22:53.943 (536870900 samples) 2.753.241.836 bytes
En- and decoding were done from the commandline. Speed- and timemeasurements were copied when coder displayed these information, if not I used a stopwatch to measure encoding time (Nero and FLAC). I know, a bit primitive!    Tests were done twice and afterwards I eliminated the slowest measurements.

Source files are on a WD5000AAKS and resulting files are written to an other WD5000AAKS hard disk. OS Windows XP is on an other drive and both WD hard drives had no activity other then that caused by the encoding or decoding process. I wanted to ensure maximum hard disk speeds (especially flac speeds can decrease a lot caused by other disk activity).

Code: [Select]
           avg.bitrate  enc.speed  dec.speed
flac -0     1022 kbps     295x       358x
flac -5      956 kbps     152x       338x
mp3 -V5      136 kbps      31x        49x
aac -q 0.4   136 kbps      32x       137x
ogg -q4      132 kbps      26x        78x
mpc -q4      134 kbps      41x       489x

The decoding speed of mpc is impressive. Maybe that's the reason why they say batterylife of Rockbox firmware is longer with mpc then with mp3 or ogg. 


Try Timer for measuring encoding and decoding time. Process Time is relevant value.
--------------------

  • capma
  • [*]
SV8 beta speed testing
Reply #23
Decoding speed test:

mpcdec v0.9.0 - 201.155 x
oggdec 1.9.3 Lancer SSE - 152.969 x

Processor: AMD Sempron 2200+
--------------------