HLS-13: Adaptive Bitrates from Named Pipe

Issue Information

Issue Type: New Feature
Priority: Major
Status: Open

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: HLS Stream Creator (HLS)
Resolution: Unresolved
Target version: VNEXT,
Components: Transcoding , Adaptive Streams ,

Created: 2015-06-15 15:37:06
Time Spent Working

Where the input is a FIFO it's obviously only possible for one process to retrieve data.

As the ABR functionality fires multiple ffmpeg processes, it obviously won't work with a FIFO.

Will need to implement a wrapper to read from the FIFO and make it available to the processes. At a very, very basic level, simply _cat_ting to a temp file may be sufficient (if a bit hacky).

Toggle State Changes


It still feels a bit hacky (though better than using a temp file - especially for a linear stream) but if the input is a fifo, could look at creating additional fifo's and using tee to push to them - each ffmpeg process would then read from it's own fifo

Something along the lines of
for i in bitrates
mknod $TMPDIR/pip.encode.$i p
# fire the ffmpeg process

# Start reading from the original fifo into the new pipes
cat $fifo | tee (for i in bitrates; do echo $TMPDIR/pip.encode.$i; done)

I don't like it as a long-term solution, but it should work (once the crappy pseudo-code has been fixed).
Commit a077490 implements a fairly rough work-around to this issue - if the input file is a FIFO then a named pipe is created for each bitrate, and the original named pipe is read into all the bitrate specific pipes.

It has the positive benefit of ensuring that all bitrates read from the source at the same speed, so the bitrate specific manifests stay more or less in sync. This does also mean that the encode is capped to the speed of the slowest encode.

Video performance seems OK based on a few quick tests, but it needs more testing to be sure

Repo: HLS-Stream-Creator
Commit: a077490bebcd5a3e57214548806c07d39a8a6778
Author: Ben Tasker <github@<Domain Hidden>>

Date: Tue Jun 16 16:47:59 2015 +0100
Commit Message: Temporary solution to using FIFO input with ABR streams. See HLS-13

Modified (-)(+)

Webhook User-Agent


View Commit

Commands used for initial testing were
$ ./HLS-Stream-Creator.sh -i /tmp/test.fifo.avi -s 10 -o /tmp/fifointest -b 64,128,512 -l

# Start writing into the FIFO
mplayer -dumpstream -dumpfile test.fifo.avi  ~/Downloads/The_Inbetweeners/Season_1/The.Inbetweeners.S01E06.avi

It might be a player thing, but when testing with VLC it's starting part way through the playlist - roughly at the same point the encode is up to (though the manifest contains segments for the entire stream).

Looking at a packet capture, VLC retrieves each of the manifests (strangely by requesting range:0-, resulting in a 206). The full manifest is returned, but VLC then requests a segment 5 segments from the end.

I think it's actually the player being smart - each time it requested the manifest the Last-Modified date has changed (because the encode was still running) so the player appears to have decided it's a live stream.

Now the encode has finished, the player starts from the beginning of the manifest.

Slightly unexpected, but doesn't seem to be an issue with the encoding process
btasker added 'VNEXT' to Fix Version
btasker removed '1.0' from Fix Version