A quick look around the forum seemed to indicate that no-one has ever attempted to build Vorbis with Lancer optimisations for Linux ?!
So I spent a few hours fiddling and produced an executable:
- Linux
- aoTuV-beta5
- Lancer optimisations for SSE
- 32 bit (x86)
- statically linked
- around 3x faster compared to the standard aoTuV-beta5 build
I did not attempt to port Lancer's SSE2/SSE3 instructions to Linux because the difference between SSE/SSE2/SSE3 is minimal on the PCs I use.
(Intel Celeron M and AMD Turion 64x2)
Any interest? If so, I will post the binary.
Better yet, post your (I assume) cleaned-up sources, so that we can build it ourselves.
Better yet, post your (I assume) cleaned-up sources, so that we can build it ourselves.
Excellent idea! I don't know how things work around here or in the Linux world, source-wise, as I'm very much a Windows person.
Usually I would commit my changes into a source code repository, perhaps with optional build switches, and then add a few more test cases to the regression tests to ensure that my changes won't break anything.
What I did to arrive at the current source code:
1. Downloaded aoTuV-beta5 source.
2. Applied latest Lancer patches.
3. Made mods so these patches would build on Linux.
I guess I would be able to produce a patch against the Lancer patches - but would this be at all useable to anyone?
Is it just me, or is distributing a set of patches just an absolutely horrid way to do things?
Is it just me, or is distributing a set of patches just an absolutely horrid way to do things?
Only if your changes are the only changes someone wants.
Any interest? If so, I will post the binary.
Yes and possible have rarewares distribute the binary or host it externally. I will place a link to it in the wiki page when I get the opportunity! Thanks.
Yes and possible have rarewares distribute the binary or host it externally. I will place a link to it in the wiki page when I get the opportunity! Thanks.
Do you have rarewares' contact info ?
I guess I would be able to produce a patch against the Lancer patches - but would this be at all useable to anyone?
Rather, against either vanilla libvorbis-1.2.3 or aotuv's latest beta source tarball (libvorbis-aotuv_b5.7).
diff -Naur aotuv-b5.7_20090301/ aotuv-b5.7_20090301-lancer/ > aotuv-b5.7_20090301-lancer.diff
That would absolutely work for me.
Do you have rarewares' contact info ?
I don't I am sure they wouldn't' mind hosting it though! Contact Rjamorim on here. I am not sure who else has access to the website?
I put the required patches and modifications under source control, so everyone can:
- build
- look at the diffs
- generate patches
Use the repository's web interface to browse it. You can also download the accelerated SSE Linux executable there.
http://repo.or.cz/w/vorbis-lancer-gcc.git/tree (http://repo.or.cz/w/vorbis-lancer-gcc.git/tree)
Thank you for the binary. It is twice as fast as mainstream oggenc.
Unfortunately it accepts only wav files as input. Could you please add flac support?
You used aotuv-b5 version of 2006-10-24. Do you plan to make a binary of the latest aotuv-b5.7 (2009-03-01)?
It is twice as fast as mainstream oggenc.
Funny, it's
exactly twice as fast as aotuv b5.7 on my netbook (Arch Linux, GCC 4.4.2 on an Intel Atom N270 1.6GHz): 14.88x vs. 7.44x. Very nice!
Ideally Aoyumi would incorporate the optimizations…
Very nice!
The credit goes to the author of the Lancer Vorbis acceleration project:
http://homepage3.nifty.com/blacksword/ (http://homepage3.nifty.com/blacksword/)
I just made a few patches to make it work on Linux.
Unfortunately it accepts only wav files as input. Could you please add flac support?
FLAC support now added. Get the updated binary direct from the repository:
http://repo.or.cz/w/vorbis-lancer-gcc.git/blob/HEAD:/oggenc (http://repo.or.cz/w/vorbis-lancer-gcc.git/blob/HEAD:/oggenc)
I will try to encorporate Aoyoumi's latest version. It will be a good test for Git ;-)
I will try to encorporate Aoyoumi's latest version. It will be a good test for Git ;-)
Gotta love Free Software. One guy invents a cool codec, another guy improves it, yet another guy makes it faster, and a fourth party makes the latter improvements work under linux. Granted, it took some time, but better late than never. Closed source software developers, take notice!
Thanks for your work.
FLAC support now added. I will try to encorporate Aoyoumi's latest version. It will be a good test for Git ;-)
Thanks for both. Flac support works OK.
You used aotuv-b5 version of 2006-10-24. Do you plan to make a binary of the latest aotuv-b5.7 (2009-03-01)?
Unfortunately I can't. Aoyumi has updated the very same algorithms which the Lancer author optimised. Put another way, a plain old conflict.
So unless the Lancer author 'wakes up' again (he hasn't worked on it since 2006 ?), then beta 5 is the latest accelerated version we're ever going to get. Not bad, as it is the current peer-reviewed Hydrogen-audio recommended version. (See http://wiki.hydrogenaudio.org/index.php?ti...ded_Ogg_Vorbis) (http://wiki.hydrogenaudio.org/index.php?title=Recommended_Ogg_Vorbis))
I compiled your snapshot vorbis-lancer-gcc-master.tar.gz locally by running ./build-script. The result here are not a static build and seems to have compiled my systems libogg in dynamically:
~/src/lancer/vorbis-lancer-gcc$ file oggenc
oggenc: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.8, dynamically linked (uses shared libs), not stripped
~/src/lancer/vorbis-lancer-gcc$ ldd oggenc
linux-gate.so.1 => (0xb7fae000)
libogg.so.0 => /usr/lib/libogg.so.0 (0xb7f63000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7f3d000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7ddf000)
/lib/ld-linux.so.2 (0xb7f94000)
~/src/lancer/vorbis-lancer-gcc$
I compiled your snapshot vorbis-lancer-gcc-master.tar.gz locally by running ./build-script. The result here are not a static build and seems to have compiled my systems libogg in dynamically:
You're absolutely right: my build script doesn't generate a completely static binary. To do that, I would have to modify the autoconf build script and I don't know how, being a Windows person. I've posted a question about it on LinuxQuestions:
http://www.linuxquestions.org/questions/li...utoconf-769088/ (http://www.linuxquestions.org/questions/linux-general-1/how-to-build-statically-linked-binary-with-autoconf-769088/)
If you want to get a statically linked binary, do the following from the ogg-aoTuVb5-lancer directory (i.e. the root):
1. $ ./build-script (if you haven't done this already)
2. $ cd vorbis-tools-1.2.0/oggenc/
3. $ make clean; make
4. Copy the last gcc line on your console, paste it on your command line and add the -static switch. Execute it.
5. $ cd ..
6. $ make install
7. The binary is now in ogg-aoTuVb5-lancer/out/bin
Any help with fixing this is very welcome!!
If you want to get a statically linked binary, do the following from the ogg-aoTuVb5-lancer directory (i.e. the root):
1. $ ./build-script (if you haven't done this already)
2. $ cd vorbis-tools-1.2.0/oggenc/
3. $ make clean; make
4. Copy the last gcc line on your console, paste it on your command line and add the -static switch. Execute it.
5. $ cd ..
6. $ make install
7. The binary is now in ogg-aoTuVb5-lancer/out/bin
That last gcc line also had a pointer to /usr/lib/libogg.so instead of to ${WORKDIR}/out/lib/libbogg.a which made it fail with:
/usr/bin/ld: attempted static link of dynamic object `/usr/lib/libogg.so'
collect2: ld returned 1 exit status
Correcting that bit I succeeded in making the build static.
Neither I know enough to fix this...
EDIT:
My build fails anyway, somethings wrong with it, trying to encode a file:
$ ./oggenc -q5 Luckie.wav -o Luckie.ogg
Open with wav-module: WAV file reader
Encoding "Luckie.wav" to
"Luckie.ogg"
at quality 5,00
Mode initialisation failed: invalid parameters for quality
Well, well...
Well, well...
Does your build machine have libogg, liborbis and libflac installed? Mine doesn't. I suspect that vorbis-tools's makefile prefers installed libraries, and simply ignores the --with-ogg, --with-vorbis switches. I already had to do a work-around for the --with-flac switch, which didn't work.
As an aside: I'm a Windows person, and know very little of the Linux dev world. It's completely baffling to me why *anyone* would use a tool like autoconf/automake. For starters, I haven't yet found a single software developer who understands these tools. Is this really the state-of-the-art in Linux? With Visual Studio, build maintenance consists of a mouse click here and there. Time saved there is time gained for important things, such as code development/debugging/test etc.
The --with-flac switch bug was discussed by its developers on a Xiph.org forum as early as 2007(!!). Are we the only ones struggling with the build system? I doubt it...
Does your build machine have libogg, liborbis and libflac installed? Mine doesn't.
Yes. I'm pretty sure problems stems from it compiling against system-installed libraries instead of the provided ones.
autoconf/automake works in linux x86, linux ARM, UNIX x86, UNIX Sparc, .... list goes on.
Not only that, but autoconf is old and tested.
Visual studio needs visual studio, and everything that visual studio provides (runtimes, etc..).
The truth is that nowadays, autoconf is still used (because it exists everywhere), but more and more projects use other tools, like scons or other python based ones.
Now, about your comment of not being able to update b5 to b5.7... Is that because you think you cannot do as good as lancer is, or because you don't know about intrinsics?
I haven't looked at the code, but did some work on intrinsics a few years ago (for a windows/linux project, dsp related, but mostly mixing)
autoconf/automake works in linux x86, linux ARM, UNIX x86, UNIX Sparc, .... list goes on.
Not only that, but autoconf is old and tested.
vi and ed are also old and tested, work in Linux, Sparc, HP-UX, Irix, etc. Why not use those? :-) Same with bash, you could write shell scripts to calculate your tax returns...
Anyways, let's not fight over this. Rather, let's work together to find an alternative that works!
Now, about your comment of not being able to update b5 to b5.7... Is that because you think you cannot do as good as lancer is, or because you don't know about intrinsics?
I haven't looked at intrinsics yet. All I did, this far, was to 'port' Lancer to Linux. Would you like to look for accelerations? If so, it'd be best to get cooperation from Aoyumi.
First thing is to profile OggEnc - applications use 5% of the code 95% of the time. We should find out where the 'expensive' functions are, and concentrate on those.
What do you think?
vi and ed are also old and tested, work in Linux, Sparc, HP-UX, Irix, etc. Why not use those? :-)
I do use vi from time to time... well, vim.
First thing is to profile OggEnc - applications use 5% of the code 95% of the time. We should find out where the 'expensive' functions are, and concentrate on those.
Yes, that would be adequate. Anyway, the answer most probably is the mdct.
I am comparing sources right now to get a clear idea of what does lancer modify. I am also downloading the snapshot from your git sources to see what you've changed.
I am PM'ing you my e-mail.
My build fails anyway, somethings wrong with it
I'm sorry to hear it doesn't work on your build machine. If you really want to build it, you can try setting up an Ubuntu 9.10 installation from scratch. That's what I am using.
That's great
Thank you very much.
I was listening to some songs which go into another, and to my surprise gapless playback was not gapless. I found that about 0,03 s silence was added to the end of the song, whereas in the following song for about the same lenght the right channel was missing.
Could this be fixed?
Hi I just tried your build and it works perfectly. Thx for your work The_Sven! (btw it's amazing to think that all this is possible thanks to you and BlackSword and Aoyumi and Xiph!)
But doing my tests some weird things happened and I have some questions.
I wanted to test Lancer SSE3MT optimizations so I went get the files on BlackSword's website. I didn't pay attention to the fact the oggenc file I got had .exe as extension and I just replaced /usr/bin/oggenc with this one and tried. And it worked! And I also remarked a sensible speed-up (about 20% better) compared to the simple SSE optimization.
Then I realized what I did and I don't understand why it worked. I checked the properties of the file /usr/bin/oggenc and the type was indeed DOS/Windows executable (application/x-ms-dos-executable).
So I'm wondering how was it possible to work? I thought maybe because of Wine but I didn't ask in the terminal to run this program via Wine, or maybe would it be possible my system (I'm using Ubuntu 11.04) recognized type of the program and run it via Wine?
Also is there some cons / known problems for using thus the .exe in my Linux system?
Anyway I think I'm gonna use the version compiled by The_Sven but I'm curious about this strange thing "using a Windows program under Linux working so good I could have never suspected there was something wrong".
Are the latest aoTuV available to compile? Regards.