Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Streaming Ogg Packets (Read 28524 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Streaming Ogg Packets

I am writing a program to stream audio via TCP to a browser. I am encoding buffers of PCM audio to a compressed format and streaming the encoded files one by one to the browser which then plays them back one after the other. This scheme works fine if I use mp3 files. The browser plays back the individual packets one after the other for a continuous stream.

The problem arises when I try to stream ogg files, either ogg/vorbis or ogg/opus. The browser (Mozilla Firefox) plays back the first file and stops. It does not accept any more packets. It's behaving as if the network connection has been dropped. I am using HTML 1.1 so it is a keep-alive connection by default.

What's puzzling is that, using the exact same code, mp3 files are played back as a continuous stream while ogg files cause the browser to quit and possibly drop the connection after the first packet.

Why is this so? Am I taking the entirely wrong approach to streaming? Is there something about an ogg file that would trigger this behavior? Is it a bug in the browser? I have tried myriad combinations of header fields and have tried sending the data chunked, to no avail. I have examined the headers of an opus demo stream and can't find anything they're doing differently with the headers.

Streaming Ogg Packets

Reply #1
Have you tried any other browsers?  Also, which version of Firefox?

I think this used to be standard Firefox behaviour, but has been fixed since Firefox 20.  Ogg streams effectively are multiple files chained together continuously, whereas MP3s really are streaming continuously with the server perhaps throwing out different metadata from time to time.  I just tested and Firefox 17 stops after each track.  Firefox 29 plays continuously.

So your code may be fine.  Try streaming direct to a music player that does this properly.

Streaming Ogg Packets

Reply #2
This is Firefox v28.0.

I can't imagine any other way of doing LIVE streaming than by sending individual encoded buffers/packets, or what some call progressive download.

How are you playing back these tracks? Are they packets in a live stream or separate song tracks?

Streaming Ogg Packets

Reply #3
Firefox 28 should be able to play a chained Ogg stream continuously.  You can test your browser against one of these streams.  Note that not all of them make track changes.  A track change in an Ogg stream is effectively a separate file, although the packets are sent continuously.  As you probably know if you wrote it  The track change interrupts the continuous flow of audio packets and send some header packets describing the new track, then back to the flow of audio packets.  I'm not sure what you would break doing this, but perhaps something is getting truncated (or duplicated?) at the file switchover.

Do you have an example I can connect to and examine?

Streaming Ogg Packets

Reply #4
The streams you linked to play fine on Firefox 28.0. The question is, what are they doing differently to make them stream continuously? As mentioned previously, I am able to stream mp3 files continuously but not ogg files. I have copied the HTTP headers from those streams into my code and it didn't solve the problem. The fact that those streams play fine suggests that there isn't a problem with the browser but perhaps with the way the files are encoded or the way I'm sending them?

I have tried sending header - file - header - file, and header - file - file - file but no matter -- I never make it past the first header and file pair. The browser plays the first ogg file and thinks it's done and accepts no more data, suggesting the connection may be broken.

By "header" I mean HTTP header.

Here are the HTTP headers from my code. They were copied from the headers linked to above. #CRLF$ is a carriage return/line feed string constant.

header$="HTTP/1.1 200 OK"+#CRLF$
header$+"Server: 127.0.0.1:8000" + #CRLF$
header$+"Content-Type: application/ogg"+#CRLF$
header$+"Cache-Control: no-cache"+#CRLF$
header$+"icy-pub: 1"+#CRLF$
header$ +#CRLF$

Streaming Ogg Packets

Reply #5
Correct content-type is audio/ogg.  No matter, almost everyone including Firefox accepts application/ogg.

I don't think the HTTP headers are the problem.  More likely you are transmitting something between the EOS packet and the BOS packet of the next link.  There isn't really anything fancy to worry about.  So long as Firefox (or any other player) gets a page marked EOS and immediately gets a page marked BOS then it should keep playing.

Serial numbers must be valid, but if you have a valid Ogg chain then that should be true already.  Are you streaming a bunch of separate files or from a single Ogg chain file?  Might be worth making a chain file and streaming that.  Just concatenate a bunch of individual tracks togther and that's a chain.  Make sure it plays as a file, then stream it.

Streaming Ogg Packets

Reply #6
I concur that it's probably not the headers. Again, it works fine with mp3 files. The application/ogg was copied directly from the HTML5 opus streams you linked to, but it really should be audio/ogg.

I emphasize that this is a LIVE stream so the individual audio buffers are encoded in real time on the fly. I don't see how you could concatenate them into a single chain file. You would have to keep building these concatenated files and that's more overhead. I am not familiar with ogg BOS and EOS so I will have to research those.

Streaming Ogg Packets

Reply #7
OK, LIVE stream.  Build straight from PCM?  Or equivalent?  Doesn't matter I suppose.

So what is your definition of the end of a track?  An Ogg stream consists of a BOS page, which is a header defining the stream, then a series of pages.  Or packets if you like.  The first packet must be a comment packet.  In Vorbis the second packet must be a codebooks packet, but Opus doesn't have that.  Then audio packets with no special markings until the last one, which is marked EOS (end of stream).

I assume you must be creating the header packets somehow if you are encoding live?  Are you doing this yourself or calling a library?  What happens when you decide it is time to change tracks?  Are you coding the EOS page correctly?  Are you then coding a new set of headers correctly?  You can stream standard audio packets forever if you want, but a track change must have the defined EOS and BOS and other header packets or it will be considered as an EOF/error by the receiver.  Without the correct header, which contains things like channel info and serial number for the stream, subsequent audio packets can't be understood.

Streaming Ogg Packets

Reply #8
Here is the basic scheme:

1. PCM data from sound card/interface

2. write PCM data to temporary raw file

3. encode raw file to opus using opusenc

4. send encoded file on network (localhost for testing)

Running opusenc with its options is the extent of my knowledge of opus encoding. Do I need to manipulate the serial number when encoding?

The opus documentation mentions oggfwd but this does not seem to be an option for non-Linux setups (running Windows 7). I am not using Icecast for this.

https://mf4.xiph.org/jenkins/view/opus/job/...an/opusenc.html

The end of the stream would occur when either my program or the browser closes the network connection.

Streaming Ogg Packets

Reply #9
opusenc should take care of everything for you, including all the serial numbers.  It outputs a mass of data which can either be stored as a file or streamed directly, no difference in the format.  You might try outputting a few streams into files instead of sending them over the network.  Then I could examine the results.  Or if you could log the stream that you are receiving.

I'm not sure what tool might be available that would give you more information about what is happening on your stream when the track changes.  Firefox doesn't really say anything useful.  On Linux I could offer ways to get some information at least that might tell you why the stream breaks.

I'm still not clear what your program is going at a track change.  Does it do anything that just streaming the last byte opusenc sends from one track and then the first byte that it sends for the next track?  Is there a pause between availability of the TCP packets for the two streams?

Streaming Ogg Packets

Reply #10
Here is some actual code written in PureBasic. It is executed in a loop.

It works. The first packet plays OK, then the browser stops accepting data.

Code: [Select]
Procedure AudioLoop()
Pa_ReadStream(*my_stream, *writeBuffer, framesPerBuffer)
CreateFile(10, "temp.raw"):
WriteData(10, *writeBuffer, framesPerBuffer*bytesPerFrame): CloseFile(10)

RunProgram("\Programs\Opus\opusenc.exe" , "--max-delay 0 --quiet --comp 0 --raw-rate 44100 --raw-chan 1 temp.raw  temp.opus", "./", #PB_Program_Wait)

header$="HTTP/1.1 200 OK"+#CRLF$
header$+"Server: 127.0.0.1:8000" + #CRLF$
header$+"Content-Type: audio/ogg"+#CRLF$
header$+"Cache-Control: no-cache"+#CRLF$
header$ +#CRLF$

ReadFile(10, "temp.opus"): ReadData(10, *writeBuffer+Len(header$), leng): : CloseFile(10)

If ClientID <> 0

SendNetworkString(ClientID, header$)
result = SendNetworkData(ClientID, *tempBuffer, leng)

EndIf

EndProcedure


If I join the header and the data and send them together, the result is the same.

Streaming Ogg Packets

Reply #11
I can't imagine that ever working well.  opusenc takes many seconds, maybe even minutes, to run.  You will have that gap between your tracks and I suspect most players will just hang up.  Streams have to stream  opusenc can stream, or at least it can on Linux.  I imagine there is a similar solution where you can take the data direct without an intermediate file.  Or maybe the browser would actually wait patiently for the next track to arrive?

Seems like you also send the (HTTP) header before every file.  I don't think Firefox will like that.  You already told it what it is getting, now you just send data forever.  It would be fine the first time, but I think sending it again will break the playback.

Last thing I'm a little confused about.  You have tempbuffer.  Is that a typo or do I not understand the Basic syntax.  And you put the Ogg data into writebuffer after a gap (as long as the HTTP header, but is the HTTP header actually in there?)  Again I might be misunderstanding the language, but it seems like that is making a mess between tracks.

Probably some misunderstandings of the Basic from me, but that should give you a few ideas of where to look for issues.

Streaming Ogg Packets

Reply #12
writeBuffer is where PortAudio puts the PCM data. It gets written to a file which is the file opusenc encodes. The opus file is read into tempBuffer which is sent to the network.

I'm prepared for there to be gaps, but haven't gotten that far yet.

I don't know of a better way of doing this without using Linux and oggfwd. I don't know of a non-disk solution that would work with Windows.

I'll try again but I think the mp3 version needed a header before every packet.

For testing purposes, I can open a pre-made file containing a sine wave, read the data to writeBuffer, encode that and send it, rather than getting the audio data from PortAudio.

Streaming Ogg Packets

Reply #13
Is the opusenc call async?  Can it run in the background while the program streams from the file it is writing?  Maybe it actually does that already?  Maybe worry about that once other issues are addressed.  I think there are definite problems with the way the HTTP headers are being sent and the way the writebuffer is populated.  Garbage at the start of a stream is generally tolerated and receivers will just seek (or read) through until they find a valid Ogg stream, but between streaming tracks they aren't so tolerant.

 

Streaming Ogg Packets

Reply #14
You've seen the code and I've said the program works with mp3 files. Why a browser stops accepting data after the first buffer/packet is the mystery. I'm not going to worry about the rest of it until that mystery is solved.

Streaming Ogg Packets

Reply #15
You've seen the code and I've said the program works with mp3 files. Why a browser stops accepting data after the first buffer/packet is the mystery. I'm not going to worry about the rest of it until that mystery is solved.

Stop putting garbage between the end of one track and the start of the next.  Saying it works for MP3 is irrelevant.  Don't do it for Ogg, then see where to go from there.  If you get a significant pause between the last packet from one track and the first packet from the next, then that will also need to be addressed.

Streaming Ogg Packets

Reply #16
So you're saying to send the header only once. OK, I'll try it.

Streaming Ogg Packets

Reply #17
OK I tried sending the HTTP header only once, then sending successive packets. No change. Firefox plays through the first packet and quits.

Streaming Ogg Packets

Reply #18
OK I tried sending the HTTP header only once, then sending successive packets. No change. Firefox plays through the first packet and quits.

Seems like you want me to psychically spot the bug in your program.  I've offered you several possibilities and several avenues for investigation.  You ignored all that and didn't even try to fix any of the obvious bugs I pointed out until pressed.  Just what is it you want me to do if you won't do anything yourself?  Have you examined the generated stream to see if there is garbage between the tracks (still my #1 guess at the problem).  Have you even tried to log the stream so I can examine it and save you the trouble?  Have you tried feeding the output to a tool like ogginfo which will tell you what is wrong with it?

Streaming Ogg Packets

Reply #19
I wasn't familiar with ogginfo, which appears to be a Linux program.

Here is a link to a file with a sample stream if you want to examine it with ogginfo:

www.gameshowforum.org/audio/OpusTestStreamChris

Streaming Ogg Packets

Reply #20
ogginfo and opusinfo *really* don't like that file and neither does anything else.  I'm amazed Firefox would play it.  Are you sure that is an accurate reflection of what you're streaming?

The first obvious thing I see wrong is the serial number.  It is the same for every link in the stream.  That confuses decoders because they can't tell whether a packet is part of a new stream or not.  The tools are also complaining about holes all over the place, basically on every page.  That generally means your page structure is messed up, the segment table doesn't match the actual data segments.

Streaming Ogg Packets

Reply #21
I've looked at the file pages in a little more detail.  Looks like they're all truncated at 4500 bytes?  Does that ring any bells for you?  That doesn't seem like a normal fixed page size that oggenc would output.  I don't see anything in the code snippet that you post, but presumably there is more.  Is one of your buffers too small to contain a page?  An Ogg page can be up to 64k long although more typically it is 4k-16k.

Streaming Ogg Packets

Reply #22
Also checked the standards.  application/ogg used to be the mime type for all Ogg containers.  However the 2008 standards update reserved this for custom Ogg containers and created audio/ogg and video/ogg to be used in the vast majority of cases.  Obviously most tools will still support application/ogg, but in time new ones may not.

Streaming Ogg Packets

Reply #23
I've been using audio/opus as the mime type, which I believe is now the standard for opus.

I've reworked my code some and will have another capture of my stream shortly. If the opus file itself lacks integrity, that raises flags. I will also upload two opus files, one generated by Goldwave and the other generated by opusenc which is invoked from my program.

That the code streams mp3 continuously tells me something. The mystery is why the same code (with the audio/opus mime type) doesn't stream opus continuously. I've tried encoding both wav and raw files with the same result.

Streaming Ogg Packets

Reply #24
I've been using audio/opus as the mime type, which I believe is now the standard for opus.


Not really.  The standard says audio/ogg for all predominantly-audio Ogg files, or audio/ogg; codecs=opus if you want to make it obvious that the content is Opus.  Can't say I've seen audio/opus in the wild, but I wouldn't be surprised.  It seems a pretty likely answer if someone was to just guess at a mime type.  I notice that Wikipedia gives audio/opus as an alternative and references an obsolete draft specification.  I might have to get my wiki-hat out of the cupboard ...