HLS-22: IV will be wrong for some bitrates in ABR streams



Issue Information

Issue Type: Task
 
Priority: Major
Status: Closed

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: HLS Stream Creator (HLS)
Resolution: Fixed (2017-04-29 12:07:25)
Components: Encryption ,

Created: 2017-04-27 09:19:05
Time Spent Working
Estimated:
 
120 minutes
Remaining:
  
60 minutes
Logged:
  
60 minutes


Description
In function encrypt, the for loop is written on the basis of there only being one bitrate, and so generates an IV based upon the segment number
    count=0
    for file in ${OUTPUT_DIRECTORY}/*.ts
    do
        ENC_FILENAME="$OUTPUT_DIRECTORY/${SEGMENT_PREFIX}_enc_${count}".ts

	INIT_VECTOR=$(printf '%032x' $count)

If we've got an ABR stream with 3 bitrates, containing 5 segments each, then when we move onto the second bitrate count is going to be at 6 (whilst we'll want it to be 0).

So the function almost certainly needs adapting to take bitrates into account rather than simply iterating over files.
Apple's technote ( https://developer.apple.com/library/content/technotes/tn2288/_index.html ) specifies that

The default Initialization Vector for media encryption (if none is specified) is the sequence number of the media file. You should specify an Initialization Vector value, and not rely on sequence numbers. The main reason for this is portability. For example, if you change where the segment appears in the playlist (e.g. inserting an ad), that changes its sequence number, requiring a re-encrypt.

So this is almost certainly going to be an issue. Longer term, it'd be better to follow their advice and set a specific IV (which will mean rewriting the manifest), but in the short term need to make the sequence numbers work as a minimum


Issue Links

Toggle State Changes

Activity


As the codebase currently stands, segment file names for a multi bitrate stream will always be
[arbitrary strings]_[bitrate]_[segment number].ts


For a single bitrate (where the -b argument hasn't been used at all) it'll be
[arbitrary strings]_[segment number].ts


When generating the IV, we're only interested in the segment number, so we can probably just iterate over the files and extract the segment number from the filename to then use as the IV.
Triggering a 'before' run for comparison while I make the changes
ben@milleniumfalcon:/tmp/hlstest$ ./HLS-Stream-Creator/HLS-Stream-Creator.sh -i big_buck_bunny_720p_stereo.avi -s 10 -e -o before -b 272,872,1372


Doi
[aac @ 0xa40a60] The encoder 'aac' is experimental but experimental codecs are not enabled, add '-strict -2' if you want to use it.

ben@milleniumfalcon:/tmp/hlstest$ FFMPEG_FLAGS='-strict -2'
ben@milleniumfalcon:/tmp/hlstest$ export FFMPEG_FLAGS
ben@milleniumfalcon:/tmp/hlstest$ ./HLS-Stream-Creator/HLS-Stream-Creator.sh -i big_buck_bunny_720p_stereo.avi -s 10 -e -o before -b 272,872,1372


I'm gonna drop a check for that in (HLS-23) once I've got this sorted, it was introduced in https://github.com/bentasker/HLS-Stream-Creator/commit/0796febbc73d9b96c7ee58864e6ce79c9ff71b4d - I'm running a fairly recent install, so I suspect a few people may hit up against that
btasker changed status from 'Open' to 'In Progress'
Changes are made, have just triggered a test run so can test.
For some reason, when running, the following can't be used to strip leading 0's off the segment number (needs doing so printf doesn't assume the number is octal)
SEG_NO=${SEG_NO##+(0)}


For now, I've done it with sed, but it's irritating. Want to get around to having a look at shopt and the like when running to see if I can figure out why. Running the same loop manually in a shell, it works fine.

Repo: HLS-Stream-Creator
Commit: 7e3fdefffd16f9051dba6a41100f36dd7ca4cc0a
Author: B Tasker <github@<Domain Hidden>>

Date: Sat Apr 29 11:08:42 2017 +0100
Commit Message: Adjust encryption loop to ensure the IV is based on the segment number rather than a counter. HLS-22



Modified (-)(+)
-------
HLS-Stream-Creator.sh




Webhook User-Agent

GitHub-Hookshot/2cc07f9


View Commit

btasker changed status from 'In Progress' to 'Open'
btasker changed timespent from '0 minutes' to '60 minutes'
btasker changed status from 'Open' to 'Resolved'
btasker added 'Fixed' to resolution
btasker changed status from 'Resolved' to 'Closed'

Work log


Ben Tasker
Permalink
2017-04-29 11:13:32

Time Spent: 60 minutes
Log Entry: Experimenting, implementing and testing