We recently tested a few software players on Windows to see how fast they could access RAM. Some player makers claim it matters. We don't think so. But the results were interesting.
http://wiki.jriver.com/index.php/Audio_Testing (http://wiki.jriver.com/index.php/Audio_Testing)
We recently tested a few software players on Windows to see how fast they could access RAM. Some player makers claim it matters. We don't think so. But the results were interesting.
http://wiki.jriver.com/index.php/Audio_Testing (http://wiki.jriver.com/index.php/Audio_Testing)
How did you test this? You make no mention of that in the article or here.
How did you test this? You make no mention of that in the article or here.
Sorry. It was a special ASIO driver. We've modified the wiki topic to add more detail. Here's what it says:
"It's easy to measure how fast different players are at this. This is done by compiling an ASIO driver that includes instrumented timing of its calls back to to the player for data (the ASIO call bufferSwitchTimeInfo).
"The tests below time the average buffer fill performance during the course of a five minute song. The test machine is an i7 running Windows 7 x64.
"Here is how several players stack up in this regard, testing ASIO output as 32-bit integer (a common hardware format)"
What about Winamp?
What about Winamp?
Media Monkey uses this Winamp plugin:
http://otachan.com/out_asio(dll).html (http://otachan.com/out_asio(dll).html)
So I would expect Winamp to benchmark the same as Media Monkey.
We recently tested a few software players on Windows to see how fast they could access RAM. Some player makers claim it matters. We don't think so. But the results were interesting.
http://wiki.jriver.com/index.php/Audio_Testing (http://wiki.jriver.com/index.php/Audio_Testing)
Let us see, the maximum number of samples per microsecond of 192 KHz sampling is 0.2, right? Seems like you would need a ton of channels to tax any of the players you tested, right?
Here's another memory related test we did this week:
http://www.computeraudiophile.com/content/...st-audio-player (http://www.computeraudiophile.com/content/Engineering-best-audio-player)
and a listening test of JRiver vs jplay:
http://yabb.jriver.com/interact/index.php?topic=70043.0 (http://yabb.jriver.com/interact/index.php?topic=70043.0)
All of the testing was part of our response to the questionable claims jplay has made about their software.
In this thread, memory movement performance was also discussed:
http://www.computeraudiophile.com/content/...gement-be-heard (http://www.computeraudiophile.com/content/Can-memory-management-be-heard)
The author of HQPlayer made condescending remarks about the use of memcpy(...) to fill sound card buffers.
The author claimed to write hand-tuned assembly that was twice as fast.
HQPlayer performed at the bottom of the performance test:
http://wiki.jriver.com/index.php/Audio_Testing (http://wiki.jriver.com/index.php/Audio_Testing)
Apparently you can have your cake and eat it too. He claims to not believe that faster processing makes a difference, then goes on to show that JRiver performance is superior. I guess that appeases those on both sides.
JRiver believes all modern computers are fast enough at #2, and that more speed is not relevant. But some companies make claims to the contrary, so let's test performance.
Samples per µs (higher is better)
Results
Player Samples per µs (higher is better)
JRiver 1019.2
MediaMonkey 1013.8
cPlay 864.0
Foobar2000 364.9
HQPlayer 167.4
JPlay No ASIO
XXHighEnd No ASIO
In this thread, memory movement performance was also discussed:
http://www.computeraudiophile.com/content/...gement-be-heard (http://www.computeraudiophile.com/content/Can-memory-management-be-heard)
The author of HQPlayer made condescending remarks about the use of memcpy(...) to fill sound card buffers.
The author claimed to write hand-tuned assembly that was twice as fast.
I like how when people point out how stupid that is, he refuses to say what he measured or how he did it and instead links the Intel reference manual. Because I guess he wants to make it known that hes aware that the Intel manual exists.
(higher is better)
Yeah, JimH is unawaringly falling for the competition's marketing nonsense. He.Will.Be.Assimilated.
Let us see, the maximum number of samples per microsecond of 192 KHz sampling is 0.2, right? Seems like you would need a ton of channels to tax any of the players you tested, right?
All players are fast enough on a modern machine.
I suppose you could imagine a scenario where a slower machine might encounter a buffer shortfall, which would play as a stutter or silence, under system load in one player but not another player.
However, this type of thing is obvious and normally fixed by using a larger hardware buffer.
To be clear, we're not claiming this performance matters to sound quality. The competitors are claiming it matters.
So we're trying to give the claims a fair technical testing and see how they hold up.
(higher is better)
Yeah, JimH is unawaringly falling for the competition's marketing nonsense. He.Will.Be.Assimilated.
That should have said "higher is faster". We've also said that faster doesn't matter, doesn't produce better sound quality.
Someone needs to convince me this is not a totally flawed test (comparing apples and oranges), I am thinking you are testing players which pre-buffer the whole track (so there is a large HDD / CPU hit at the start of the track) and compare to other players which read in realtime (HDD / CPU hit is spread out over the length of the track)...
Can someone explain to me why this matters at all? As long as you get no stutters during playback/the buffer fills up nicely I see no problem with access speeds. Even the thought of assuming the speed of loading samples to RAM having an impact on playback fidelity absolutely dumbfounds me. Just because no tests exist doesn't mean that testing something like this makes any sense. I don't see the point in testing unfounded claims by "audiophiles" to prove them wrong, it is their responsibility to prove that they're right.
Someone needs to convince me this is not a totally flawed test (comparing apples and oranges), I am thinking you are testing players which pre-buffer the whole track (so there is a large HDD / CPU hit at the start of the track) and compare to other players which read in realtime (HDD / CPU hit is spread out over the length of the track)...
The test measures ASIO buffer fill performance, spread over real-time playback of a 5 minute song.
Pre-buffering is not particularly relevant to the speed with which device buffers are filled unless you were reading from disk in the ASIO fill callback, which I can't imagine anyone would do (certainly not JRiver or f2k).
Said another way, what happens in the reading threads, processing threads, system threads, loading before playing threads, etc. is not measured. The only measurement is where the rubber meets the road -- filling the device buffers.
As long as you get no stutters during playback/the buffer fills up nicely I see no problem with access speeds.
I agree with you. I've described this as "just in time". Beyond that, any increase in the speed by which you fill RAM should not matter at all. But... there are some other software developers claiming otherwise. You could read a little here:
http://www.computeraudiophile.com/Forums/Equipment/Software (http://www.computeraudiophile.com/Forums/Equipment/Software)
This was moved because it had "nothing to do with audio performance".
The tests were conducted because other players, principally jplay, were claiming that memory access speed did affect sound quality. Silly, I know, but true. Read a few of the posts at comuteraudiophile.com to learn more.
This forum demands data, and that's what was provided. I'm sorry you don't agree.
JimH:
Reading the discussion at http://www.computeraudiophile.com/Forums/Equipment/Software (http://www.computeraudiophile.com/Forums/Equipment/Software), I still don't see the point in even investigating that matter. Just because you can create some data doesn't mean that they have any meaning at all. The only thing I see is that there is now an article on this on your wiki, which is prominently linked to HA, with statements like "Samples per µs (higher is better)" and "JRiver still believes that speed doesn't matter." So you did not reach a conclusion about the effect on audio quality? The last sentence can be stated without even testing anything at all. Either you did prove that access speed does affect audio quality, or you didn't. Your data is not helpful in any way to prove or falsify that claim. So the only thing I see now is a wiki entry where JRiver achieved the highest number in some meaningless test, and statements about belief. Cynical people, like me, might see an angle there to attract the sort of crowd most people here at HA despise.
Of course you are free to do any tests and post them on your site, but I don't see how this warrants an HA discussion. Moreso since your test didn't provide any result regarding the claims. Does or does not RAM access affect playback quality?
EDIT: I can accept that this is pure advertising, if this thread stays in Off-Topic.
The issue at heart here is two fold:
1) HA becomes a battle ground for nonsense Audiofile claims, users who hear differences will popup, and HA will descend into trash...
2) Authors will clamber to embrace such nonsense, a modern system can transfer 16GB/second from memory, CD quality audio requires just 176KB, or 0.0001 GB/s - see the issue, everyone who writes a player will feel the need to get closer to a value which is meaningless. You know that it would be possible to better the values presented here?, but doing so would create a player which impacts the system, so suddenly all our favorite players will have this negative impact? a pointless arms race.
That is why such threads as this need to be removed from circulation, as it is not about free speech, this thread is as loaded as they come and full over overtones.
Software players can be "optimized" to score "better" at such pseudobenchmarks, except it will hamper other functionality (say, out-of-process ASIO to cope with buggy drivers more nicely surely degrades the score here).
Some of the best minds in digital media have weighed in on this thread and think the JPlay claims are ridiculous. I agree.
As a community of developers trying to make honest products, what's the best course of action?
Is a technical trial of ridiculous claims helpful (this thread being one imperfect attempt)? Should other software actively lock JPlay out?
Thank you for any advice.
HA I hope will always remain a fortress of common sense, outside of HA there are often huge debates about how one USB cable sounds different to another, or two bit identical files sound different to another, when faced with such, all scientific reasoning goes out of the Window. People are invested into such claims, because $ is spent achieving it, no matter what it happens to be.
Notice how the author of Jplay did not say around HA too long?, perhaps he prefers less critical climes, but there are plenty of places he would be welcomed. The emperors new clothes exists today and is centered around audio reproduction.
The issue at heart here is two fold:
1) HA becomes a battle ground for nonsense Audiofile claims, users who hear differences will popup, and HA will descend into trash...
Having spent - and having been locked out or removed myself voluntarily from other sites that are full bore into audiophile bullshit - I find it very important that this site stays open to critically investigate any such claims and shoot them down with all the weaponry needed.
There has to be a site one can point to and proclaim that not all interested in good audio reproduction are idiots of the special audiophile kind, where counterarguments that destroy such bullshit can be posted without the typical response by the audio idiots to squash any attempt of such discussions and push dissenter out to then freely engage in the religious aspects of audiophily....faith in your senses without proper testing is about the worst of the offenses
.
Please so not tread the path to eliminate any possibility to discuss audio fraud and ridiculous claims. This is the only site I know of that is willing to let them be taken apart piece by piece and dump on the trash heap of falsified ideas and claims.
OK, sounds good.
So, playing devils advocate, though we mostly seem to agree that JPlay does not seem to do much, how is it there has not been a definative post to show that.
Just because the author was not able to show what improvement it DOES make, can someone not post what difference it DOES NOT make?
OK, sounds good.
So, playing devils advocate, though we mostly seem to agree that JPlay does not seem to do much, how is it there has not been a definative post to show that.
Just because the author was not able to show what improvement it DOES make, can someone not post what difference it DOES NOT make?
As mentioned before, the reason it's not up to HA to disprove the claim comes down to Russel's teapot. The burden of proof is on the claimant.
We tried to test that. We tested the "Large Page" memory access they touted was better. It wasn't.
One of their claims is that their memory access is faster in general. Because they don't support ASIO, we were unable to test it, but our own ASIO write speeds did well tested against other players.
Another player claims that special assembler code gave them an advantage in speed, but our tests showed that it was at the slow end of players we tested.
AS I'VE SAID BEFORE, WE DON'T THINK MEMORY ACCESS SPEED CAN AFFECT SOUND QUALITY. But we felt compelled to test.
WE DON'T THINK MEMORY ACCESS SPEED CAN AFFECT SOUND QUALITY. But we felt compelled to test.
Jim,
I think the main reason this got moved to Off-Topic is because you haven't addressed the sound quality question. You tested buffers, and showed your product at the top, but you did not show a correlation or non-correlation with sound quality.
our own ASIO write speeds did well tested against other players.
So?
We have NEVER claimed that memory access speed mattered, but it was a principal claim made by JPlay. Trying to be open minded, we tested.
I agree with you, greynol, that it seems pointless.
I posted this link in the JPlay thread, but in case you missed it....
At ComputerAudiophile.com, user mitchco did a nice test essentially disproving the JPlay claim of any difference in sound produced:
http://www.computeraudiophile.com/blogs/JR...AY-Test-Results (http://www.computeraudiophile.com/blogs/JRiver-vs-JPLAY-Test-Results)
Jim,
I think the main reason this got moved to Off-Topic is because you haven't addressed the sound quality question. You tested buffers, and showed your product at the top, but you did not show a correlation or non-correlation with sound quality.
I have to agree with JimH.
The reading comprehension abilities of some posting at certain topics at HA seem to be severely limited - or is it they cannot comprehend context?
Several times he made it clear that this post is a response to claims made by Jplay, and that that IS ALL THAT HE IS DOING....
This posting by JH is not about addressing sound quality issues or anything else beside this precise and very limited issue - speed comparisons to refute certain claims made by Jplay.
How hard is that to understand?
For what it’s worth, I’m inclined to agree with JimH and kraut.
The relation to sound quality lies in the test’s debunking absurd claims about sound quality by the author of jplay. Does that qualify this thread for a place in General Audio? Maybe; but I’m not completely sure, and I don’t want to start a game of ping-pong with it.
What I think is worthy of credit in either case, at least as a token gesture, is the fact that someone did something to obtain concrete evidence against such claims. Sure: we all know they’re nonsense right off the bat, there’s not enough time in the universe to disprove every nonsensical claim ever made, and there have been some quibbles with some facets of the test’s publicity (some of which appear to have been genuine mistakes)—but isn’t more evidence better evidence, at least when the nonsensical claim in question is preying on gullibility at €99 a time? It’s always good to have something else to offer those who are in danger of being misled and/or the most stubborn of fence-sitters for whom ‘common’ sense isn’t quite enough. Isn’t that right in line with the ethos of this site?
I appreciate the editing that db1989 undertook.. the value of points I made would have been much diminished by my display of anger and the tone I used.
Those points have now been even more clearly laid out by him.
The simple fact is that Jplay made certain claims; be they as nonsensical as they may, they are used to make a product whose utility is rather doubtful look "better" or "good" in comparison to products who actually deliver the goods.
That is why I appreciate the work done by Jim to prove those developers wrong in that particular claim while at the same time emphasizing the nonsense of the importance of memory access speed in the present computer environment as it relates to audio reproduction off files on harddrive.
>"The simple fact is that Jplay made certain claims;"
They were not even on the test as they do not use ASIO, so the test was against other players, many on that list make no claims about better quality, that is my issue with the testing.
If HQPlayer were touting faster transfers, then fine, just include them.
We tested the "Large Page" memory access they touted was better. It wasn't.
To substantiate this, here are the results from several runs comparing normal memory against large page memory (again, testing ASIO buffer fill performance):
Normal Large Page
1045.6 1145.6
1079.1 1137.1
1162.3 1159.1
942.3 1196.5
1207.2 1076.5
1215.1 1148.4
1148.8 1177.4
1047.2 905.2
1161.9 864.3
1138.4 1164.6
1175.8 1101.1
1169.9 1124.8
Average:
1124.4 1100.0
There's some variability, but overall there is no statistically significant performance difference between the two methods when filling soundcard buffers.
And again, we believe this performance is not related to sound quality. It is simply an empirical refutation of a claim made by JPlay.
>"The simple fact is that Jplay made certain claims;"
They were not even on the test as they do not use ASIO, so the test was against other players, many on that list make no claims about better quality, that is my issue with the testing.
Hi spoon. We didn't realize that until we had tested a couple of other players. We tested other players because we were curious and it was fairly easy. If we had left JPlay off the list, we probably would have been criticized for not testing it.
Damned if you do, damned if you don't.
I find it very important that this site stays open to critically investigate any such claims and shoot them down with all the weaponry needed.
Kraut,
Measurements were taken. But I find no critical analysis, as you mention above.