Github #34 reported that it wasn't possible to playback an encrypted stream using Safari or VLC on a Mac.
Reproducing the info in that ticket here for posterity, but this also needs fixing.
User Report
created encrypted parts using
hls-stream-creater.sh -i ./small.mp4 -e -s 10 ./
it created successfully but now its not playing.
m3u8 file contents:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-KEY:METHOD=AES-128,URI=small.mp4.key
#EXT-X-TARGETDURATION:6
#EXTINF:5.600000,
small.mp4_00000.ts
#EXT-X-ENDLIST
Developer tools in Safari simply reports
Failed to load resource: Plug-in handled load
Investigation details are provided in comments below, but the root cause has boiled down to the keyfile not being in quotes.
Activity
2019-11-16 12:16:18
2019-11-16 12:17:24
Key-length is correct
Segment decrypts happily enough using the key
ben@milleniumfalcon:~/tmp/HLS$ cp small.mp4_00000.ts test.ts ben@milleniumfalcon:~/tmp/HLS$ IV=$(printf '%032x' 00000) ben@milleniumfalcon:~/tmp/HLS$ KEY=$(cat small.mp4.key | hexdump -e '16/1 "%02x"') ben@milleniumfalcon:~/tmp/HLS$ openssl aes-128-cbc -d -in test.ts -out test.dc.ts -K $KEY -iv $IV ben@milleniumfalcon:~/tmp/HLS$ ffprobe -hide_banner -i test.dc.ts Input #0, mpegts, from 'test.dc.ts': Duration: 00:00:05.53, start: 1.466667, bitrate: 350 kb/s Program 1 Metadata: service_name : Service01 service_provider: FFmpeg Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 560x320, 30 fps, 30 tbr, 90k tbn, 60 tbc Stream #0:1[0x101](eng): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, mono, fltp, 72 kb/sand actually, VLC playback works fine for me:
2019-11-16 12:19:09
It's truncating letters off the key-filename:
I noticed truncation in users output but assumed it might've been copy paste.
Verified on my webserver that it is requesting the truncated version
Dumping the file out as hex it looks ok
But, if we take the relevant line and try and convert it back the issue becomes apparent
The end of that looks ok, hex `79` is dec 121 ("y" in ascii) and `0a` is a line break. Let's iterate over the hex and convert to dec then
ben@milleniumfalcon:~/tmp/HLS$ echo "382c 5552 493d 736d 616c 6c2e 6d70 342e 6b65 790a" | tr -d ' ' | python3 -c 'import sys,re; p=re.findall("..",sys.stdin.read()); [print(int(x,16))for x in p]' 56 44 85 82 73 61 115 109 97 108 108 46 109 112 52 46 107 101 121 10Dumping those to chars
ben@milleniumfalcon:~/tmp/HLS$ echo "382c 5552 493d 736d 616c 6c2e 6d70 342e 6b65 790a" | tr -d ' ' | python3 -c 'import sys,re; p=re.findall("..",sys.stdin.read()); [print(chr(int(x,16)))for x in p]' 8 , U R I = s m a l l . m p 4 . k e yLooks ok...
Converting one way and back works fine
Perhaps the earlier test was a false positive then?
This is ridiculous. I hate macs.
Going back to basics, what if we quote the keyfile name
VLC works. And so does Safari....
FFS
2019-11-16 12:27:22
Making the change
ben@milleniumfalcon:~/Documents/src.old/HLS-Stream-Creator$ git diff diff --git a/HLS-Stream-Creator.sh b/HLS-Stream-Creator.sh index d88fa6b..1d6a6c3 100755 --- a/HLS-Stream-Creator.sh +++ b/HLS-Stream-Creator.sh @@ -322,7 +322,7 @@ function encrypt(){ for manifest in ${OUTPUT_DIRECTORY}/*.m3u8 do # Insert the KEY at the 5'th line in the m3u8 file - $SED -i "5i #EXT-X-KEY:METHOD=AES-128,URI="${KEY_PREFIX}${KEY_NAME}.key "$manifest" + $SED -i "5i #EXT-X-KEY:METHOD=AES-128,URI=\"${KEY_PREFIX}${KEY_NAME}.key\"" "$manifest" done }Running a new mux
Tested playback on a Mac
- Playback of http://scratch.holly.home/before/lua_string_split.mp4.m3u8 fails in the same manner.
- Playback of http://scratch.holly.home/after/lua_string_split.mp4.m3u8 fails in the same manner.
Committing as
2019-11-16 12:28:58
Webhook User-Agent
View Commit
2019-11-16 12:36:49
URI The value is a quoted-string containing a URI that specifies how to obtain the key. This attribute is REQUIRED unless the METHOD is NONE.Looks like the initial draft said the same, so this has probably always been wrong.
It was unquoted when added back in https://github.com/bentasker/HLS-Stream-Creator/commit/bb3cce4febdbcc59ca33c304ae34a6168d310169
So, the codebase wasn't spec compliant (although blindly trimming first and last char is a bloody stupid way to be dealing with a quoted string....)
2019-11-16 12:38:29
2019-11-16 12:38:36
2019-11-16 12:38:36
2019-11-16 12:38:40
2019-11-17 10:10:31
We see the key get extracted in file
case AttributesTag::EXTXKEY: { const AttributesTag *keytag = static_cast<const AttributesTag *>(tag); if( keytag->getAttributeByName("METHOD") && keytag->getAttributeByName("METHOD")->value == "AES-128" && keytag->getAttributeByName("URI") ) { encryption.method = SegmentEncryption::AES_128; encryption.key.clear(); Url keyurl(keytag->getAttributeByName("URI")->quotedString()); if(!keyurl.hasScheme()) { keyurl.prepend(Helper::getDirectoryPath(rep->getPlaylistUrl().toString()).append("/")); } M3U8 *m3u8 = dynamic_cast<M3U8 *>(rep->getPlaylist()); if(likely(m3u8)) encryption.key = m3u8->getEncryptionKey(keyurl.toString()); if(keytag->getAttributeByName("IV")) { encryption.iv.clear(); encryption.iv = keytag->getAttributeByName("IV")->hexSequence(); } } else { /* unsupported or invalid */ encryption.method = SegmentEncryption::NONE; encryption.key.clear(); encryption.iv.clear(); }So, the key's URI ends up as a
Url keyurl(keytag->getAttributeByName("URI")->quotedString());Although we don't currently write any others in with HLS-Stream-Creator, now would be a good time to see what other attributes will get first/last chars chomped off by VLC
ben@milleniumfalcon:/tmp/VLC/vlc-3.0.8/modules/demux/hls$ grep -R "quotedString()" * playlist/Tags.cpp: return Attribute(this->name, quotedString()); playlist/Tags.cpp:std::string Attribute::quotedString() const playlist/Parser.cpp: uri = uriAttr->quotedString(); playlist/Parser.cpp: Url keyurl(keytag->getAttributeByName("URI")->quotedString()); playlist/Parser.cpp: initSegment->setSourceUrl(uriAttr->quotedString()); playlist/Parser.cpp: std::pair<std::string, AttributesTag *> pair(tag->getAttributeByName("URI")->quotedString(), tag); playlist/Parser.cpp: desc = pair.second->getAttributeByName("GROUP-ID")->quotedString(); playlist/Parser.cpp: desc += pair.second->getAttributeByName("NAME")->quotedString(); playlist/Parser.cpp: std::string lang = pair.second->getAttributeByName("LANGUAGE")->quotedString(); playlist/Tags.hpp: std::string quotedString() const;So, thats
-
-
-
-
And now, getting back on topic, let's verify how
std::string Attribute::quotedString() const { if(value.length() < 2) return ""; std::istringstream is(value.substr(1, value.length() - 2)); std::ostringstream os; char c; while(is.get(c)) { if(c == '\\') { if(!is.get(c)) break; } os << c; } return os.str(); }So, breaking that down
- If length < 2 just return an empty string
- create
- create
- Iterate over each char, if the char is a backslash, check whether there's a subsequent char, if not stop iterating
- push the char into
- Turn the output stream back into type
So the codebase is definitely stripping off the first and last char (assuming that they'll be quotes). Not particularly defensive there, but ok.
But why didn't my Linux box also break?
Ahh, it's really old. But, it achieved playback so there must be something in there. Worth a quick gander - sources for VLC 2.2.2 are at http://download.videolan.org/pub/videolan/vlc/2.2.2/vlc-2.2.2.tar.xz
Completely different location in this one, function
static int parse_Key(stream_t *s, hls_stream_t *hls, char *p_read) { assert(hls); /* #EXT-X-KEY:METHOD=<method>[,URI="<URI>"][,IV=<IV>] */ int err = VLC_SUCCESS; char *attr = parse_Attributes(p_read, "METHOD"); if (attr == NULL) { msg_Err(s, "#EXT-X-KEY: expected METHOD=<value>"); return err; } if (strncasecmp(attr, "NONE", 4) == 0) { char *uri = parse_Attributes(p_read, "URI"); if (uri != NULL) { msg_Err(s, "#EXT-X-KEY: URI not expected"); err = VLC_EGENERIC; } free(uri); /* IV is only supported in version 2 and above */ if (hls->version >= 2) { char *iv = parse_Attributes(p_read, "IV"); if (iv != NULL) { msg_Err(s, "#EXT-X-KEY: IV not expected"); err = VLC_EGENERIC; } free(iv); } } else if (strncasecmp(attr, "AES-128", 7) == 0) { char *value, *uri, *iv; if (s->p_sys->b_aesmsg == false) { msg_Dbg(s, "playback of AES-128 encrypted HTTP Live media detected."); s->p_sys->b_aesmsg = true; } value = uri = parse_Attributes(p_read, "URI"); if (value == NULL) { msg_Err(s, "#EXT-X-KEY: URI not found for encrypted HTTP Live media in AES-128"); free(attr); return VLC_EGENERIC; } /* Url is put between quotes, remove them */ if (*value == '"') { /* We need to strip the "" from the attribute value */ uri = value + 1; char* end = strchr(uri, '"'); if (end != NULL) *end = 0; } /* For absolute URI, just duplicate it * don't limit to HTTP, maybe some sanity checking * should be done more in here? */ if( strstr( uri , "://" ) ) hls->psz_current_key_path = strdup( uri ); else hls->psz_current_key_path = relative_URI(hls->url, uri); free(value); value = iv = parse_Attributes(p_read, "IV"); if (iv == NULL) { /* * If the EXT-X-KEY tag does not have the IV attribute, implementations * MUST use the sequence number of the media file as the IV when * encrypting or decrypting that media file. The big-endian binary * representation of the sequence number SHALL be placed in a 16-octet * buffer and padded (on the left) with zeros. */ hls->b_iv_loaded = false; } else { /* * If the EXT-X-KEY tag has the IV attribute, implementations MUST use * the attribute value as the IV when encrypting or decrypting with that * key. The value MUST be interpreted as a 128-bit hexadecimal number * and MUST be prefixed with 0x or 0X. */ if (string_to_IV(iv, hls->psz_AES_IV) == VLC_EGENERIC) { msg_Err(s, "IV invalid"); err = VLC_EGENERIC; } else hls->b_iv_loaded = true; free(value); } } else { msg_Warn(s, "playback of encrypted HTTP Live media is not supported."); err = VLC_EGENERIC; } free(attr); return err; }It's a long function, but it includes an explicit check to see whether it's a quoted string (actually, technically just whether it starts with a double quote, there's no check for the last char)
value = uri = parse_Attributes(p_read, "URI"); if (value == NULL) { msg_Err(s, "#EXT-X-KEY: URI not found for encrypted HTTP Live media in AES-128"); free(attr); return VLC_EGENERIC; } /* Url is put between quotes, remove them */ if (*value == '"') { /* We need to strip the "" from the attribute value */ uri = value + 1; char* end = strchr(uri, '"'); if (end != NULL) *end = 0; }So, it'll work quite happily with an unquoted string as we'll never enter that conditional block. Means they've made the later "improved" versions more fragile... sadness
2019-11-17 10:25:01
2019-11-22 09:11:43