HLS-18: FFMPEG_FLAGS not being honoured

Issue Information

Issue Type: Bug
 
Priority: Major
Status: Closed

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: HLS Stream Creator (HLS)
Resolution: Cannot Reproduce (2015-12-14 12:12:53)
Affects Version: 1.0,
Target version: 1.0,
Components: Transcoding ,

Created: 2015-12-09 08:47:04
Time Spent Working
Estimated:
 
60 minutes
Remaining:
  
35 minutes
Logged:
  
25 minutes


Description
It's possible it may just be that ffmpeg requires specific ordering for some flags, but I was asked how to have segment filenames prefixed with a FQDN within the generated manifest.

The answer should be 
FFMPEG_FLAGS='-segment_list_entry_prefix "http://foo.bar/"'
export FFMPEG_FLAGS
./HLS-Stream-Creator.sh -i example.avi -s 10


Which should lead to the segment filenames within the manifest being prefixed with the FQDN.

For some reason it didn't work, though manually adding that option to ffmpeg within the script worked fine.


Toggle State Changes

Activity


2015-12-14 11:57:30
Setting up, using latest version of HLS-Stream-Creator 
ben@milleniumfalcon:~$ cd /tmp/
ben@milleniumfalcon:/tmp$ mkdir HLS18test
ben@milleniumfalcon:/tmp$ cd HLS18test/
ben@milleniumfalcon:/tmp/HLS18test$ git clone https://github.com/bentasker/HLS-Stream-Creator.git
ben@milleniumfalcon:/tmp/HLS18test$ ffmpeg -version
ffmpeg version N-74248-g107026e Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04)
configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libdcadec --enable-libfreetype --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvo-aacenc --enable-libvidstab
libavutil      54. 30.100 / 54. 30.100
libavcodec     56. 56.101 / 56. 56.101
libavformat    56. 40.101 / 56. 40.101
libavdevice    56.  4.100 / 56.  4.100
libavfilter     5. 32.100 /  5. 32.100
libavresample   2.  1.  0 /  2.  1.  0
libswscale      3.  1.101 /  3.  1.101
libswresample   1.  2.101 /  1.  2.101
libpostproc    53.  3.100 / 53.  3.100




First run, without flags 
ben@milleniumfalcon:/tmp/HLS18test$ ./HLS-Stream-Creator/HLS-Stream-Creator.sh -i The.Inbetweeners.S01E01.avi -s 10
ben@milleniumfalcon:/tmp/HLS18test$ head -n 15 output/The.Inbetweeners.S01E01.avi.m3u8 
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:17
#EXTINF:10.600000,
The.Inbetweeners.S01E01.avi_00000.ts
#EXTINF:9.760000,
The.Inbetweeners.S01E01.avi_00001.ts
#EXTINF:9.880000,
The.Inbetweeners.S01E01.avi_00002.ts
#EXTINF:10.480000,
The.Inbetweeners.S01E01.avi_00003.ts
#EXTINF:10.280000,
The.Inbetweeners.S01E01.avi_00004.ts


Adding flags to try and repro reported behaviour 
ben@milleniumfalcon:/tmp/HLS18test$ rm -f output/*
ben@milleniumfalcon:/tmp/HLS18test$ FFMPEG_FLAGS='-segment_list_entry_prefix "http://foo.bar.invalid/"'
ben@milleniumfalcon:/tmp/HLS18test$ export FFMPEG_FLAGS
ben@milleniumfalcon:/tmp/HLS18test$ ./HLS-Stream-Creator/HLS-Stream-Creator.sh -i The.Inbetweeners.S01E01.avi -s 10
ben@milleniumfalcon:/tmp/HLS18test$ ps aux | grep ffmpeg
ben       8254  157  1.9 226264 77084 pts/31   Sl+  11:49   0:12 ffmpeg -i The.Inbetweeners.S01E01.avi -loglevel error -y -vcodec libx264 -acodec libfdk_aac -threads 0 -map 0 -flags -global_header -f segment -segment_list ./output/The.Inbetweeners.S01E01.avi.m3u8 -segment_time 10 -segment_format mpeg_ts -segment_list_entry_prefix "http://foo.bar/" ./output/The.Inbetweeners.S01E01.avi_%05d.ts


Looks like it's being passed through by HLS-Stream-Creator at least to me. Checking Manifest output 
ben@milleniumfalcon:/tmp/HLS18test$ head -n 15 output/The.Inbetweeners.S01E01.avi.m3u8 
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:12
#EXTINF:10.600000,
"http://foo.bar.invalid/"The.Inbetweeners.S01E01.avi_00000.ts
#EXTINF:9.760000,
"http://foo.bar.invalid/"The.Inbetweeners.S01E01.avi_00001.ts
#EXTINF:9.880000,
"http://foo.bar.invalid/"The.Inbetweeners.S01E01.avi_00002.ts
#EXTINF:10.480000,
"http://foo.bar.invalid/"The.Inbetweeners.S01E01.avi_00003.ts
#EXTINF:10.280000,
"http://foo.bar.invalid/"The.Inbetweeners.S01E01.avi_00004.ts


Syntax used was wrong though, so retrying with valid flags 
ben@milleniumfalcon:/tmp/HLS18test$ FFMPEG_FLAGS='-segment_list_entry_prefix http://foo.bar.invalid/'
ben@milleniumfalcon:/tmp/HLS18test$ export FFMPEG_FLAGS
ben@milleniumfalcon:/tmp/HLS18test$ rm -f output/*
ben@milleniumfalcon:/tmp/HLS18test$ ./HLS-Stream-Creator/HLS-Stream-Creator.sh -i The.Inbetweeners.S01E01.avi -s 10
ben@milleniumfalcon:/tmp/HLS18test$ head -n 15 output/The.Inbetweeners.S01E01.avi.m3u8 
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:12
#EXTINF:10.600000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_00000.ts
#EXTINF:9.760000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_00001.ts
#EXTINF:9.880000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_00002.ts
#EXTINF:10.480000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_00003.ts
#EXTINF:10.280000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_00004.ts


Looks good to me

2015-12-14 11:57:42
btasker changed timespent from '0 minutes' to '15 minutes'

2015-12-14 12:03:57
Trying without an export 
ben@milleniumfalcon:/tmp/HLS18test$ FFMPEG_FLAGS=''
ben@milleniumfalcon:/tmp/HLS18test$ export FFMPEG_FLAGS
ben@milleniumfalcon:/tmp/HLS18test$ FFMPEG_FLAGS='-segment_list_entry_prefix http://foo.bar.invalid/'
ben@milleniumfalcon:/tmp/HLS18test$ ps aux | grep ffmpeg
ben       8540  176  1.9 226372 77084 pts/31   Rl+  12:01   0:08 ffmpeg -i The.Inbetweeners.S01E01.avi -loglevel error -y -vcodec libx264 -acodec libfdk_aac -threads 0 -map 0 -flags -global_header -f segment -segment_list ./output/The.Inbetweeners.S01E01.avi.m3u8 -segment_time 10 -segment_format mpeg_ts -segment_list_entry_prefix http://foo.bar.invalid/ ./output/The.Inbetweeners.S01E01.avi_%05d.ts

So, that seems to have worked too.

Looking at where the flags are inserted though, I wonder if we need to take argument order into account 
$FFMPEG -i "$infile" \
    -loglevel error -y \
    -vcodec "$VIDEO_CODEC" \
    -acodec "$AUDIO_CODEC" \
    -threads "$NUMTHREADS" \
    -map 0 \
    -flags \
    -global_header \
    -f segment \
    -segment_list "$playlist_name" \
    -segment_time "$SEGLENGTH" \
    -segment_format mpeg_ts \
    $bitrate \
    $FFMPEG_ADDITIONAL \
    $FFMPEG_FLAGS \
    "$OUTPUT_DIRECTORY/$output_name"

What happens if FFMPEG_ADDITIONAL is non-empty?

2015-12-14 12:12:28
Let's try treating it as a live stream

Without export 
ben@milleniumfalcon:/tmp/HLS18test$ FFMPEG_FLAGS='-segment_list_entry_prefix http://foo.bar.invalid/'
ben@milleniumfalcon:/tmp/HLS18test$ ./HLS-Stream-Creator/HLS-Stream-Creator.sh -i The.Inbetweeners.S01E01.avi -s 10 -l
ben@milleniumfalcon:/tmp/HLS18test$ ps aux | grep ffmpeg
ben       8583  139  1.7 226256 72820 pts/31   Rl+  12:04   0:02 ffmpeg -i The.Inbetweeners.S01E01.avi -loglevel error -y -vcodec libx264 -acodec libfdk_aac -threads 0 -map 0 -flags -global_header -f segment -segment_list ./output/The.Inbetweeners.S01E01.avi.m3u8 -segment_time 10 -segment_format mpeg_ts -segment_list_flags +live -segment_list_entry_prefix http://foo.bar.invalid/ ./output/The.Inbetweeners.S01E01.avi_%05d.ts

ben@milleniumfalcon:/tmp/HLS18test$ cat output/*m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:11
#EXTINF:10.600000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_00000.ts
#EXTINF:9.760000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_00001.ts
#EXTINF:9.880000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_00002.ts

Seems to work.

Multiple bitrates? 
ben@milleniumfalcon:/tmp/HLS18test$ ./HLS-Stream-Creator/HLS-Stream-Creator.sh -i The.Inbetweeners.S01E01.avi -s 10 -l -b 128,256,512
ben@milleniumfalcon:/tmp/HLS18test$ cat output/The.Inbetweeners.S01E01.avi_256.m3u8 
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:11
#EXTINF:10.600000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_256_00000.ts
#EXTINF:9.760000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_256_00001.ts


Non-live ABR? 
ben@milleniumfalcon:/tmp/HLS18test$ ./HLS-Stream-Creator/HLS-Stream-Creator.sh -i The.Inbetweeners.S01E01.avi -s 10  -b 128,256,512
ben@milleniumfalcon:/tmp/HLS18test$ ps aux | grep ffmpeg
ben       8852 58.9  1.8 226452 75724 pts/31   Sl+  12:07   0:09 ffmpeg -i The.Inbetweeners.S01E01.avi -loglevel error -y -vcodec libx264 -acodec libfdk_aac -threads 0 -map 0 -flags -global_header -f segment -segment_list ./output/The.Inbetweeners.S01E01.avi_128.m3u8 -segment_time 10 -segment_format mpeg_ts -b:v 128k -segment_list_entry_prefix http://foo.bar.invalid/ ./output/The.Inbetweeners.S01E01.avi_128_%05d.ts
ben       8854 52.3  1.9 226256 77068 pts/31   Rl+  12:07   0:08 ffmpeg -i The.Inbetweeners.S01E01.avi -loglevel error -y -vcodec libx264 -acodec libfdk_aac -threads 0 -map 0 -flags -global_header -f segment -segment_list ./output/The.Inbetweeners.S01E01.avi_256.m3u8 -segment_time 10 -segment_format mpeg_ts -b:v 256k -segment_list_entry_prefix http://foo.bar.invalid/ ./output/The.Inbetweeners.S01E01.avi_256_%05d.ts
ben       8856 46.6  1.8 226404 76932 pts/31   Sl+  12:07   0:07 ffmpeg -i The.Inbetweeners.S01E01.avi -loglevel error -y -vcodec libx264 -acodec libfdk_aac -threads 0 -map 0 -flags -global_header -f segment -segment_list ./output/The.Inbetweeners.S01E01.avi_512.m3u8 -segment_time 10 -segment_format mpeg_ts -b:v 512k -segment_list_entry_prefix http://foo.bar.invalid/ ./output/The.Inbetweeners.S01E01.avi_512_%05d.ts
ben@milleniumfalcon:/tmp/HLS18test$ cat output/The.Inbetweeners.S01E01.avi_256.m3u8 
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:11
#EXTINF:10.600000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_256_00000.ts

ben@milleniumfalcon:/tmp/HLS18test$ cat output/The.Inbetweeners.S01E01.avi_128.m3u8 
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:11
#EXTINF:10.600000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_128_00000.ts
#EXTINF:9.760000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_128_00001.ts
#EXTINF:9.880000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_128_00002.ts

ben@milleniumfalcon:/tmp/HLS18test$ cat output/The.Inbetweeners.S01E01.avi_512.m3u8 
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:11
#EXTINF:10.600000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_512_00000.ts
#EXTINF:9.760000,
http://foo.bar.invalid/The.Inbetweeners.S01E01.avi_512_00001.ts


Seems to work without issue.

It should work even when video and audio are split out into fifos as the arguments related to those come at the beginning of the arg list.

For the time being, I'm going to close this as cannot reproduce.

2015-12-14 12:12:43
btasker changed timespent from '15 minutes' to '25 minutes'

2015-12-14 12:12:53
btasker changed status from 'Open' to 'Resolved'

2015-12-14 12:12:53
btasker added 'Cannot Reproduce' to resolution

2015-12-14 12:12:58
btasker changed status from 'Resolved' to 'Closed'

Work log

Ben Tasker
 Permalink
2015-12-14 11:57:42

Time Spent: 15 minutes
Log Entry: Trying to repro
Ben Tasker
 Permalink
2015-12-14 12:12:43

Time Spent: 10 minutes
Log Entry: Further testing