HLS-6: Adaptive Streams



Issue Information

Issue Type: New Feature
 
Priority: Minor
Status: Closed

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: HLS Stream Creator (HLS)
Resolution: Done (2015-06-14 23:50:04)
Target version: 1.0,

Created: 2015-06-10 23:38:00
Time Spent Working


Description
The PR merged in HLS-5 simplify something I've been debating implementing for a while.

If the script is configured to accept a comma seperated list of bitrates, we can generate multiple bitrate HLS streams and then create a top level playlist pointing to the relevant m3u8s.

Players that support adaptive streaming can then choose (and switch between) playlists in response to changing network conditions (e.g. going from wifi to 3G).


Issue Links

Toggle State Changes

Activity


Have marked this issue as blocked by HLS-7 as whilst we could adjust to take the input as an additional argument, it'll potentially get quite messy.

Probably better to wait until getopt support has been written in so that they information can be provided in a command line option.
When doing this, will also need to take into account that the flow will likely be very different with linear video (as implemented by HLS-8)

With VoD we could potentially do the following

- build a top-level playlist containing references to each of the bitrate playlists
- Iterate through each of the bitrates, calling ffmpeg for each and then .

With Live that won't work, as the stream won't end, so the bitrates need to be generated in parallel.

The top-level playlist needs to be generated before calling ffmpeg so that it'll be present whether live or VoD.
Have moved the ffmpeg call into a function and adjusted some of it's inputs to use function arguments rather than globals ready for introducing support for encoding multiple bitrates.

It's occurred that if we can get multiple bitrates transcoding at the same time (which we'll need to do anyway for live adaptive streams) there's no real benefit in drawing a distinction between VoD and Live in terms of whether we fork or not. It'd mean that for a VoD encode the different bitrates could be created in parallel

Though, probably worth providing a command line option so that the user can specify VoD should be transcoded sequentially (in case the hardware in use simply isn't capable enough to handle it)

Repo: HLS-Stream-Creator
Commit: a8c86ff7b82883f67ec42ab61b246d1dd07a55ab
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jun 12 12:01:52 2015 +0100
Commit Message: Moved ffmpeg call into a function ready for adding support for adaptive streams. See HLS-6



Modified (-)(+)
-------
HLS-Stream-Creator.sh




Webhook User-Agent

GitHub-Hookshot/99ae70d


View Commit


Repo: HLS-Stream-Creator
Commit: eaa186962c26a9f13ec9bd3d30ac4562f60b8358
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jun 12 12:57:40 2015 +0100
Commit Message: Implemented sequential transcoding of adaptive streams for HLS-6



Modified (-)(+)
-------
HLS-Stream-Creator.sh




Webhook User-Agent

GitHub-Hookshot/99ae70d


View Commit

Have implemented and tested sequential transcoding of each bitrate and creation of the Variant/master playlist. Testing has obviously only been with VoD as already know transcoding sequentially isn't an option for linear video.

As that's working I'm going to look at implementing the parallel transcoding now, there will be a few other minor tweaks needed to be able to test it properly with linear (essentially finishing up HLS-8)
So, for parallelisation we have have the following requirements

- Must be able to disable it
- Script shouldn't exit until the jobs themselves are all complete
- must be able to be run as a background script (i.e. if we're going to be using fg bg etc need to do a set -m first)

Have a test of the parallelisation changes running at the moment, but assuming that succeeds, the following is true

- Added a new Command line option -f so that the parallelisation can be disabled
- Script will wait for each bitrate to finish (and report whenever it sees a bitrate reach completion)
- Interrupting the script will stop the encoding process

The script doesn't do any kind of tweaking on the ffmpeg flags (such as NUMTHREADS ) which may, potentially want to be tweaked given you'll be running number-of-bitrates transcodes simultaneously (for example, perhaps let them use a core each rather than all trying to thread across all cores?). Might be best left to the kernel to handle scheduling though tbh.

But the basic parrallelisation functionality is largely in place


Repo: HLS-Stream-Creator
Commit: 4f47a0940f89adf834ca845d8034b2096daec2e6
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jun 12 16:53:33 2015 +0100
Commit Message: Enabled parallelisation of Adaptive stream encoding. See HLS-6



Modified (-)(+)
-------
HLS-Stream-Creator.sh




Webhook User-Agent

GitHub-Hookshot/99ae70d


View Commit


Repo: HLS-Stream-Creator
Commit: a9dec3d3fd1917cbbb1f46c22baf2c73c7644e75
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jun 12 16:53:52 2015 +0100
Commit Message: Updated readme with section on Adaptive streams. See HLS-6



Modified (-)(+)
-------
README.md




Webhook User-Agent

GitHub-Hookshot/99ae70d


View Commit

There is a minor status output bug in commit 4f47a09 but decided not to let it block merging (will raise a bug for it in a moment - HLS-10 ).

Other than that, the encode works correctly, and insists on parallelisation if the stream is live (as of commit e7ffea7)

Repo: HLS-Stream-Creator
Commit: e7ffea78e16eb356b7ce51b2055206e05bf3ed7c
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jun 12 16:57:06 2015 +0100
Commit Message: Correcting a live stream check. See HLS-6



Modified (-)(+)
-------
HLS-Stream-Creator.sh




Webhook User-Agent

GitHub-Hookshot/99ae70d


View Commit

As the adaptive streams work correctly for VoD, and Live streams won't officially be supported until HLS-8 is done, I'm going to close this FR as complete.
btasker changed status from 'Open' to 'Resolved'
btasker added 'Done' to resolution
btasker changed status from 'Resolved' to 'Closed'