Skip to main content

Topic: My Observations between Nero AAC and Opus (Read 8801 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
My Observations between Nero AAC and Opus
Hello everybody.
I have noticed something that may be of interest to you all.
I am very interested with lossy audio/video codecs and I like playing around with them.
I mostly (85%) listen to chiptune/video game music so I always encode from the native chip tune format as emulated by my preferred program (mostly XMPlay plugins), so I never encode lossy to lossy.
I have adopted several formats over 3 years for my music collection as follows, in chronological order:
1. Vorbis @ q.2 mainly, but I often used up to q.4 for chip tunes that encoded with audible artifacts.
2. Noticing that my collection is taking up too much space on my 8GB SD card, and noticing that AAC sounds acceptable at Q0.2 (48kbps) I switched to it. Very soon after, I am not satisfied with the outcome. It may have been acceptable considering the low bitrate for general music, but not for chiptune. The trademark square-waves lost their timbre, and sounds different. The noise also lost the electric feel to it and it sounds like a bunch of random sine waves. All in all, it's acceptable, but annoying.
3. I moved to Opus at 48kbps, and am satisfied with its quality. It recreated the chip tune feel very well at low bitrates. BUT! There are times when I would notice a sudden degradation of sound quality for around half a second, mostly at the beginning of a song. Upon further inspection, those are times when the encoder used either SILK or hybrid modes, before switching to CELT. I found no way to force use of CELT only, except for raising the bitrate up.
4. I went back to AAC @ q0.2. And this is where something interesting happened.
I was listening to my newly encoded tunes from the SNES game Spanky's Quest (available at Zophar.net). Then I noticed that the Introduction Part 3 tune (wrongly labeled Part 2), with the sharp closed high-hats, sounded incredibly horrible with AAC! It lost ALL its sharpness, and sounds like sandpapers scratching together. Extremely annoying.
I tried encoding it at 48kbps Opus and the result was very acceptable considering the bitrate.
I tried finding other culprit songs that bring down q2 AAC to its knees, and here's another one, Grzegorz Kwiatek (Greyscale)'s "Alive" from the Atari SAP Music Archive. The intro features several noise sweeps, and at Q2 AAC, it sounds extremely unpleasant. Opus at 48kbps, on the other hand, sounds also bad, but somewhat better than AAC.
Increasing Opus' bitrate, I noticed that 96kbps offers almost transparent results on those 2 tunes. Seeing that those 2 were problematic tunes, other tunes should sound extremely good on that bitrate. As for AAC, I increased its bitrate to Q4, but found that the hi-hat from Spanky's quest artifact was NOT eliminated. It now sounds more like a hi-hat, but the sharp attack was not recreated faithfully. Further increasing to 190kbps (AAC), the difference is still easily distinguished, as ABX tests with Foobar revealed a 100% certainty. 190kbps AAC was EASILY beaten by a 96kbps Opus.
And so, I now have my collection in 96kbps Opus.

I have some questions. Seeing how good Opus is, I wonder if it has its own flaws. And what is your personal opinion of AAC vs Opus on very low bitrates (like 48kbps) and low bitrates (like 96kbps)? Finally, did I choose right? I now have an Android phone, and GMMP plays Opus extremely well. I don't plan to use an iPhone.
Playing chiptune files on Android without encoding lossly first is NOT a choice for me. C64 SID sounds horrible on available Android SID players, and only XMPlay plays them well (seems like Foobar uses the same engine as Android, so it sounds horrible too).

Anyway, here's just my observation, and I hope that I have contributed to the community somehow, to chiptune listeners and lossy codec enthusiasts.

Thanks for your time reading this

  • eahm
  • [*][*][*][*][*]
My Observations between Nero AAC and Opus
Reply #1
Thanks for this. First of all, every codec has its own killer sample/s so... you will find a music that works much better somewhere else, even at much lower bitrates.

Few questions:

1) AAC, are you using LC, HE or HEv2? LC doesn't perform well below 64 kbps, you will certainly find artifacts. To me, for my use, it's transparent around 100 kbps.

2) AAC, can you test again with Apple, FhG, FDK (still FhG but open source) instead of the "obsolete" Nero?

3) Can you upload pieces of these files?
  • Last Edit: 18 May, 2014, 02:22:01 AM by eahm

  • saratoga
  • [*][*][*][*][*]
My Observations between Nero AAC and Opus
Reply #2
If you are using AAC-HE, I could see that performing oddly with chiptunes.  Perhaps try forcing LC and see if it works better if you aren't already.

My Observations between Nero AAC and Opus
Reply #3
Thank you very much for the reply, guys. I really appreciate it, talking to people with similar interests
First, yeah, I use Nero AAC, and it automatically chooses which profile to use depending on the quality setting (HEv2 for q2, HE for q4, I guess, and LC for above). I also wanted to try other AAC encoders, but I can't find a way to get the Apple encoder (I'm on Windows XP SP3). And sure, thanks for suggesting other encoders, I'll try them straight away!
I can upload, but I only have 3.5G mobile internet, and I'm on a very limited data limit, and so, uploading files would kill my allowance immediately, so sorry... You may encode them yourself, though  The original files are on zophar.net (Spanky's Quest) and the Atari SAP Music Archive (Alive).
Knowing how HEv2 technically works, sharp high-frequency attacks seem to be affected the most.

Edit: I found out that those other encoders are not standalone, and with such big downloads, my internet data allocation will be over immediately before finishing such downloads. Can't try them right now, sorry.
  • Last Edit: 18 May, 2014, 03:19:07 AM by TarShoduze

  • lvqcl
  • [*][*][*][*][*]
  • Developer
My Observations between Nero AAC and Opus
Reply #4
First, yeah, I use Nero AAC, and it automatically chooses which profile to use depending on the quality setting (HEv2 for q2, HE for q4, I guess, and LC for above).


IIRC it uses:
HE-AAC v2 for q = 0-0.15
HE-AAC for  q = 0.16-0.3
LC-AAC for  q = 0.31-1.0
  • Last Edit: 18 May, 2014, 03:47:04 AM by lvqcl

  • lithopsian
  • [*][*][*][*]
My Observations between Nero AAC and Opus
Reply #5
In terms of audio quality, you should find Opus to be superior to anything else for most music in the 48-96 kbps range you are looking at.  The killer tracks for Opus that I have found are harpsichord test tracks (probably something in the codec is just plain wrong for those, but I don't know what).  That seems about as far removed from the type of music you have as it is possible to get, but you are the best judge of that.  It does have some other limitations for a music codec, relating to gain control, tagging, and currently limited tool support.

One limitation you are coming across is that, while the codec has a vast range of potential internal configurations, the current encoders tend to pick what they think is best and not give you a lot of choice.  Works well most of the time, but inevitably you will find situations where a different setting would be better.  On the plus side, Android is all set up for Opus.

  • nu774
  • [*][*][*][*][*]
  • Developer
My Observations between Nero AAC and Opus
Reply #6
The killer tracks for Opus that I have found are harpsichord test tracks (probably something in the codec is just plain wrong for those, but I don't know what).

As far as I understand, It is not Opus implementation is wrong (in some ways) but it basically comes from various kind of technical limitations due to being a low delay codec. Especially, it's MDCT window size is limited. I'm surprised how well Opus is doing despite of those limitations.
Read chapter "Tonality Estimation" at  http://people.xiph.org/~xiphmont/demo/opus/demo3.shtml

My Observations between Nero AAC and Opus
Reply #7
Quote
. And what is your personal opinion of AAC vs Opus on very low bitrates (like 48kbps) and low bitrates (like 96kbps)? Finally, did I choose right? I now have an Android phone, and GMMP plays Opus extremely well. I don't plan to use an iPhone.


Personal Opinion? Well I could contribute with that:

Recently I've been messing around with new FhG/Fdk encoders (old Nero user here) and I got pretty excited with it's quality.
As I understand, setting ACC at 48kbps means you are using HE mode. I particularly HATE the SBR artifacts and would avoig going bellow 64kbps with AAC with nero. Depending of the source I'd stick with either 64 or 80kbps for very low bitrate encodes with AAC forcing always LC mode. But then again, that is my personal opinion.
Recently I did some new tests with Fdk HE mode at 48kbps, HE/LC comparison at 64 kbps and the equivalent attainable bitrate with OPUS I discovered OPUS tends to deviate and inflate from it's target bitrate MUCH more than the FhG codec.
I can´t comment on chiptune music in particular, but with some hard rock samples I've recently tested I had some interesting results, results. I tested with half a dozen samples but I'll coment on Two particular two:

-----------------------------
First a typical Result, Opus Outperforming AAC Fhg, sample in question is From the Angel Beats Little Braver Single,

1) The song is: Answer Song, This song is not particularly "hard" and has more tranquil parts.

Opus 1.1

Set -48 VBR, Actual bitrate: 50kbps: Subjective quality: 2-3 Annoying, easily perceivable artifacts, but quality would oscilate considerable and get worse at harder/noisier parts, becoming considerably annoying. The result is still surprisingly good for only -50kbps overall tough, but not enough for me to use it on a noisy bus/car
Set -64 VBR, Actual bitrate: 64kbps: Subjective quality: 3  Annoying,  easily perceiveable artifacts, I couldn´t perceive gain in quality in the quieter parts, and just a little bit more on the noisiers one. I'd have expected a better result with 64 kbps after i lissened for the 48kbps one.
Set -96 VBR, Actual bitrate: 102kbps: Subjective quality: 3.75 Slightly Annoying, perceivable artifacts. Quality gets much better at 96 kbps, Maybe could use this setting on a bus if I was running out of space, but not at home, there is still a lot of perceivable artifacts on the hit-hat.

AAC Fdk:
Set -48 VBR Forced LC, Actual bitrate: 48: Subjective quality 1.5 Bad, easily discernible artifacts with badly muffled sound. This is pretty much unusable
Set -48 VBR HE2, Actual bitrate: 48: Subjective quality: 2, Bad. Not as muffled as the LC version but the SBR characteristic aritfact is a completely turn off to me, even when watching movies with mainly voice only this level of quality annoys me too much to be acceptable.
Set -64 VBR LC, Actual bitrate: 65:Subjective quality 2.5, Annoying, easily perceivable artifacts, not as annoying as the HE version but a little bit more muffled, also more muffled than the Opus equivalent which is still a little bit better, quality gain however is far superior from 48 to 64 compared from the same increase in bitrate in opus.
Set -96 VBR. Actual bitrate: 97 Subjective quality: 3.5 Slightly Annoying, perceivable artifacts: Similar results to Opus at same quality setting, just barely more distinguishable high-pitch hit-hat artifacts, however There is still a little bit of muffling occuring tough, which is a turn off. I would avoid having to use it even on a noisy bus.

I'll try and find a "transparent" sweet spot for this sample when I get bet home and do some proper ABX's to post it here later
==========================================================================================

Now a more surprisingly result almost the opposite of the previous one, This happened with two songs that I tested which remained pretty much noisier during all length, the song in question reported bellow is also from Angel Beats,

Song 2:  Crow Song (Marina version):

Opus 1.1:
Set -48 VBR, Actual bitrate: 98kbps: Subjective quality: 3.5 Slightly Annoying, easily perceiveable artifacts. Would not use even in a noisy train.
Set -64 VBR, Actual bitrate: 112kbps: Subjective quality: 3.75 Fine, perceivable artifacts but not annoying, would use on a noisy enviroment but not at the silence of my room.
Set -20 VBR, Actual bitrate: -67, Subjective quality: 1 Abismal, can´t even describle how bad it is.  (I tried to enforce the closest bitrate possible to 64kbps in this file to compare with AAC)

AAC Fdk:
Set - 48 VBR HE2 Actual bitrate: 67 kbps: Subjective quality: 2 Bad, easily discernible artifacts, SBR still annoys me a lot
Set - 48 VBR forced LC Actual bitrate: 68 kbps: Subjective quality: 1.5 Very bad, artifacts so strong I couldn´t even tell this isn´t SBR, the sound is much more muffled than the HE version.
Set - 80 VBR forced LC Actual bitrate: 98: Subjective quality 3.5: Slightly Annoying, easily perceivable artifacts. (this setting was chosen to match the -48 VBR actual 98kbps Opus file)  Slightly less muffled but with a little bit more high-pitched (hit-hat) artifacts than the Opus 98kbps version. As a curiosity I ABX'd the 98 aac vs the 99kbps Opus and could with a certain effort discern between the two tracks but couldn´t pick for sure which One I personally preferred. I could repeat the ABX test and provide it here if anyone gets interested together with a 30sec sample of this particular song.

I anyway quality wise neither of these settings were satisfactory to my particular taste even with cheap fones and noisy background. The point which Fhg and Opus become transparent or at least not annoying for this particular sample for me however would take more time to test. Usually, however, in the distant past when I did a series of ABX and subjective tests with nero years ago when first ditching Mp3, I decided to stick with q0.55~q0.6 for noisy hit-hat rich hard rock tracks, which would result between 180~190kbps, so that would be my start with FhG and go down step by step for optimal size on the go in a smartphone/iphone.

This kind of result was similar with more similar samples, what I found interesting is that Opus usually fair better than FhG at lower bitrates but AAC escalates better overall, specially at beyond medium bitrates (100+), Opus is still quite unreliable with noisier high-volume samples. I did some quick tests with anime audio at 48kbps and Opus was actually watchable on a smartphone with cheap earbuds. Video OST typically has much less audio information compared to rock music, voices and background noises didn´t have that typical annoying artifacts of equivalent bitrate HE AAC, however, since opus is still poorly supported outside the windowns enviroment, specially for video, I guess I'll never actually adopt it and stick with Fhg instead, at least for video, a shame really, but absolute compatibility > small gain in quality, at least for audio that is just a small fraction of a overall videofile size.
Those where my 2 cents, Hope this could contribute.
  • Last Edit: 28 June, 2014, 03:24:51 AM by sephirotic

  • lithopsian
  • [*][*][*][*]
My Observations between Nero AAC and Opus
Reply #8
Something very strange going on there.  I tried with the same track at 64kbps in Opus and it didn't kick up the requested bitrate and sounded as expected.  So perhaps not exactly the same track, but probably very similar audio.

My Observations between Nero AAC and Opus
Reply #9
Something very strange going on there.  I tried with the same track at 64kbps in Opus and it didn't kick up the requested bitrate and sounded as expected.  So perhaps not exactly the same track, but probably very similar audio.


When I was testing Opus I also find it very intriguing how on a same album with the same style of music sometimes in two different tracks, one would be inflated from the target bitrate to almost double and the other similar song would keep the exactly target bitrate, quality-wise, they would both sound on par tough.

  • lithopsian
  • [*][*][*][*]
My Observations between Nero AAC and Opus
Reply #10
Something very strange going on there.  I tried with the same track at 64kbps in Opus and it didn't kick up the requested bitrate and sounded as expected.  So perhaps not exactly the same track, but probably very similar audio.


When I was testing Opus I also find it very intriguing how on a same album with the same style of music sometimes in two different tracks, one would be inflated from the target bitrate to almost double and the other similar song would keep the exactly target bitrate, quality-wise, they would both sound on par tough.


That's what I'd hope for with a proper variable bitrate encoding.  Regardless of the actual bitrate, the audible quality should be similar.

Your results with The Crow Song don't look right though.  The bitrate inflation is massive and I don't see similar results here for the same track, even if I don't have exactly the same file you do.  There are some types of music that don't encode well in Opus (see nu774's link) but this isn't one of them.

My Observations between Nero AAC and Opus
Reply #11
Something very strange going on there.  I tried with the same track at 64kbps in Opus and it didn't kick up the requested bitrate and sounded as expected.  So perhaps not exactly the same track, but probably very similar audio.


When I was testing Opus I also find it very intriguing how on a same album with the same style of music sometimes in two different tracks, one would be inflated from the target bitrate to almost double and the other similar song would keep the exactly target bitrate, quality-wise, they would both sound on par tough.


That's what I'd hope for with a proper variable bitrate encoding.  Regardless of the actual bitrate, the audible quality should be similar.

Your results with The Crow Song don't look right though.  The bitrate inflation is massive and I don't see similar results here for the same track, even if I don't have exactly the same file you do.  There are some types of music that don't encode well in Opus (see nu774's link) but this isn't one of them.


I tried to look further into this matter to see what part of the song was bloating the bitrate. Strangely the bitrate never jumped above 51 kbps during the whole music. I then further looked into it and discovered that the ID3 tag where I added quite a large text with both japanese and translational lyrics of the music was bloating the overall file size, I used the Mediainfo tool to see the average bitrate and it seems it calculates the average data from the total filesize including container and metadata, not only the raw audio stream. So yes, the Opus track was actually 49kbps not 98 on the first test I performed, so disregard the results with the enforced filesize of the crow-song file, it's faulty.
  • Last Edit: 30 June, 2014, 02:04:39 PM by sephirotic

  • lithopsian
  • [*][*][*][*]
My Observations between Nero AAC and Opus
Reply #12
Good sleuthing!  I have previously complained to Xiph that their op_bitrate function returned a bitrate that was based on the total file size including metadata.  I imagine Mediainfo is just doing the same thing, possibly just copied the same code or at least the idea.  Seems dumb to me.  Xiph calls it a bitrate "with overhead", but I don't call metadata overhead.

The Xiph library also calculates the bitrate based on the size including a multiplexed video (!), but that is at least acknowledged as an issue.  I wonder if Mediainfo does that too?  There isn't currently a good (ie. quick) way to determine the size of just one audio stream in a multiplexed Opus file, so no good way to find a real bitrate.  It isn't stored anywhere.

  • lithopsian
  • [*][*][*][*]
My Observations between Nero AAC and Opus
Reply #13
It may be worth mentioning that the Opus codec did not really have complete audio transparency very high on its list of design goals.  Compromises to meet other goals mean that while quality is excellent at surprisingly low bitrates, even at very high bitrates there are occasional jarring artefacts.

  • IgorC
  • [*][*][*][*][*]
My Observations between Nero AAC and Opus
Reply #14
Any lossy format has its own difficult samples. It's not only Opus. 
Opus performs on par with the best LC-AAC, Vorbis and Muspack encoders at 128 kbps and higher.

  • lithopsian
  • [*][*][*][*]
My Observations between Nero AAC and Opus
Reply #15
Any lossy format has its own difficult samples. It's not only Opus. 
Opus performs on par with the best LC-AAC, Vorbis and Muspack encoders at 128 kbps and higher.

Opus performs *close* to the best of the other codecs on *most* music at 128 kbps.  However it shows considerably more spread between its best, or perhaps typical, performance and some difficult tracks.  This performance is at odds with its class-leading performance at lower bitrates (although still poor on particular killer tracks, other codecs are equally poor on some tracks).  The ongoing SoundExpert test @ 128 kbps is not looking good for Opus.  I'm not sure why, perhaps a higher proportion of tracks that it doesn't work well with?

The technical reasons for this are known and it seems strange to argue that it isn't the case.  It would be nice if Opus outperformed as well at 160 kbps as it does at 64 kbps, but unfortunately not.  What this means in practice is that while it is quite passable for casual listening at 64 kbps, something I don't find with any other codec, it is not transparent in all cases at levels where I cannot find fault with codecs such as AAC and Vorbis (eg. 160 kbps where Opus still sounds nasty with Harpsichord and similar tones).

  • IgorC
  • [*][*][*][*][*]
My Observations between Nero AAC and Opus
Reply #16
Dear lithopsian, I suggest you to search for people's reaction on "Soundexrert" test. And You will understand everyrhing.

According to his  64 kbps "test" http://soundexpert.org/encoders-64-kbps
MP3 22,050 Hz - 3.10
Vorbis - 2.87 
Opus - 2.77

Halooooo?!

Opus scores very well in every personal test at high bitrates. Look at Kamedo2 (old version of Opus though), Steve Forte Rio and my current personal tests.

In the end You can always do your own test ... before making claims.
  • Last Edit: 01 July, 2014, 03:46:21 PM by IgorC

  • lithopsian
  • [*][*][*][*]
My Observations between Nero AAC and Opus
Reply #17
Quote
In the end You can always do your own test ... before making claims.

You think I didn't?  But you wouldn't care anyway since you only pay attention to test results that fit your pre-conceived notions.  Hardly scientific

  • IgorC
  • [*][*][*][*][*]
My Observations between Nero AAC and Opus
Reply #18
Quote
In the end You can always do your own test ... before making claims.

You think I didn't? 

Sorry, have You posted your results anywhere? I didn't saw them anywhere.
And making it easier, one single result of  killer sample doesn't count.


But you wouldn't care anyway since you only pay attention to test results that fit your pre-conceived notions.

False.

Hardly scientific

I have provided you 3 independent personal tests by listeners who know how perform ABC/HR tests.  Perfectly scientific.


It's You who provide the results of Soundexpress tests those aren't even ABC/HR either ABX. 
Please inform yourself how those tests were done http://soundexpert.org/testing-room
That's not valid blind test. That's not even a blind test itself. So are You here to talk about science, right?
  • Last Edit: 01 July, 2014, 05:19:23 PM by IgorC

  • saratoga
  • [*][*][*][*][*]
My Observations between Nero AAC and Opus
Reply #19
Opus performs *close* to the best of the other codecs on *most* music at 128 kbps.  However it shows considerably more spread between its best, or perhaps typical, performance and some difficult tracks.  This performance is at odds with its class-leading performance at lower bitrates (although still poor on particular killer tracks, other codecs are equally poor on some tracks).  The ongoing SoundExpert test @ 128 kbps is not looking good for Opus.


I don't think people on this forum consider the sound expert tests acceptable measures of perceived audio quality.  I certainly don't at least. 


Re: My Observations between Nero AAC and Opus
Reply #20
Hello, this is the original poster (TarShoduze) here :D

It's been 3 years already and my further research for comparing AAC and Opus has led me again here (I've been reading my own post without knowing I wrote it, thinking "wow, this guy is just like me!" LOL )

After three years, I now have decent hardware (Windows 10 64-bit) and internet access (3Mbps but still...) and also learned a lot more during the past years.

I ended up using Opus at 64kbps for my phone's music collection (my computer only has FLAC) and my music preferences also changed, going from chiptune to soft rock, to rock, then to heavy metal / metalcore / death metal and a bit of trance (yeah--weird, huh?), and I always used Opus @ 64bit, sometimes going as low as 48kbps (I use earbuds with my phone).

I don't know if this will be of any relevance, but here are modern encodings of the music that I mentioned in my original post:

1. Gregorz Kwiatek - "Alive" (self-released chiptune track for the Atari XL/XE (if I remember correctly)
2. Kyouhei Sada - "Intro 3" (from the SNES game Spanky's Quest)

This link includes
1. the lossless FLAC copy (emulated by XMPlay ASAP 3.1.1 and Alpha-II SPC Player 3.3.4)
2. the lossy 48kbps Opus copy (encoded by libopus 1.2-alpha2-31-g8e19536b)
3. the lossy 48kbps AAC-HE copy (encoded by qaac 2.62 using the CoreAudioToolbox 7.10.9.0, AAC-HE Encoder)

https://mega.nz/#F!EINn1bCA!WbGCK7wXu9IgSZUTrKFo2w

Nowadays, I mostly listen to rock and heavy metal, but I still enjoy chiptune and VGM from time to time :D

I've been also listening to some anime soundtracks XD

Anyways, it's likely that no one will even read this anyway, after 3 years, but I'm thankful for your replies to this little post of mine.

Until next time :D