HLSDVD-3: Build Reference Platform

Issue Information

Issue Type: Task
 
Priority: Major
Status: Closed

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: HLSDVD (HLSDVD)
Resolution: Won't Fix (2019-09-09 15:59:52)

Created: 2015-06-15 15:03:56
Time Spent Working


Description
Although the eventual solution is supposed to be relatively platform agnostic, need to have a base platform to build on/test against.

The most flexible approach is probably to go with a VM with an eye to porting over to a Raspberry Pi afterwards. The VM should be relatively memory/CPU constrained to help pick up on expected issues on the Pi.

Base specs for the VM are

- 1 vCore
- 512MB RAM
- Debian 8 minimal install
- 1 NIC (to begin with)
- 20GB HD

One limitation of the Pi that will be a little hard to emulate is the effect of the NIC and USB sharing bandwidth. It's unlikely you'd want to write video segments to an SD card given the Pi's propensity for VM corruption, so the output would need to be written to either a USB stick/drive or a RAM disk.

Will start by writing direct to disk until we've got an idea of the memory usage. The Pi2 has 1GB of RAM, so depending on processing requirements and segment size, it may be feasible to create a RAM disk for output to be written to.


Toggle State Changes

Activity


2015-06-16 02:33:28
Default passwords configured as

- encoder/encoder
- root/toor

Before the VM image is published, will add a script to force a reset at first login.
root is not currently permitted to login via SSH, encoder can do so and then use su or sudo

2015-06-16 02:53:38
ffmpeg (for transcoding), NGinx (for HLS Egress) and Python are installed

2015-06-17 02:13:08
Still waiting for the HDMI extender to turn up, so in the meantime have started adding a new feature to the VM 
root@hlsenc:/home/encoder# apt-get install proftpd incron
root@hlsenc:/home/encoder# mkdir /var/ingress
root@hlsenc:/home/encoder# echo root > /etc/incron.allow 
root@hlsenc:/home/encoder# incrontab -e
root@hlsenc:/home/encoder# incrontab -l
/var/ingress IN_CLOSE_WRITE /bin/bash /home/encoder/encoding/FTP/process_ingress.sh $#

root@hlsenc:/home/encoder# echo "DefaultRoot /var/ingress encoder" > /etc/proftpd/conf.d/encoder.conf
root@hlsenc:/home/encoder# service proftpd restart
root@hlsenc:/home/encoder# mkdir /var/www/html/output
root@hlsenc:/home/encoder# chown encoder:encoder /var/www/html/output

The process_ingress script moves the file from the ingress directory into a processing directory (currently /var/inprogress) and then calls HLS-Stream-Creator against it to create an Adaptive stream in /var/www/html/output

2015-06-17 03:11:45
The process_ingress.sh script pulls in a global config to set a few bits 
encoder@hlsenc:~$ cat ~/encoding/config 
OUTPUT_DIRECTORY="/var/www/html/output/"
FFMPEG_FLAGS="-map 0 -map -0:s"

export OUTPUT_DIRECTORY
export FFMPEG_FLAGS

The FFMPEG_FLAGS set tell ffmpeg to disregard any subtitles (useful if uploading a Makostra - mkv).

Because the action is triggered by incron the process also works if you scp a file to /var/ingress.

2015-06-17 07:11:12
A couple of other tweaks

tmpfs egress directory
The VM is configured with a dynamically allocated disk image, so was causing a bit of a bottleneck in I/O speeds. Have edited fstab to make /var/www/html/output a tmpfs mount for faster reads/writes (though the contents are obviously lost if the system reboots).

Remote source support
Pushing a file with extension .remote will cause the file to be read. Contents are currently expected to be 1 or 2 lines: 
scheme:url
output bitrates

Output bitrates are a comma seperated list (in K). If not present, a default of 128,1024 is used (arbitrary numbers picked to prove the concept).
The location string needs to be in a format that ffmpeg understands, e.g. 
rtmp://fms.12E5.edgecastcdn.net/0012E5/mp4:videos/8Juv1MVa-485.mp4
64,512

The uploaded file will remain in the ingress directory until the stream ends

2015-06-18 03:49:20
Have built basic support for the HDMI adapter capture in, control is currently via the filename extension 'lenkeng'

- start.lenkeng - start capturing and streaming
- stop.lenkeng - stop any running streams

It's not a particularly clean way of doing it, but it proves the concept until I get a web interface/API in place.

Output is to /var/www/html/output/dvd/

2015-06-18 23:15:28
As both HLS-4 and HLS-5 will benefit from it's presence, seems worth putting memcached onto the system. Technically, both could be achieved using MySQL/MariaDB but should get better response times from memcached . 
root@hlsenc:/home/encoder# apt-get install memcached

2016-05-21 11:37:27
Repo: HLS_Encoder
Host:Rimmer

commit f2a23076d338f09f488bace7cf885cc4b11c460a
Author: B Tasker <github@<Domain Hidden>>
Date: Thu Jun 18 04:09:24 2015 +0100

Commit Message: Streaming from HDMI extender now part supported. See HLSDVD-3

FTP/process_ingress.sh | 42 ++++++++++++++++++++++++++++++++++++++++++
Lenkeng/hdmicap.py | 4 +++-
2 files changed, 45 insertions(+), 1 deletions(-)


View Commit | View Changes

2019-09-09 15:59:52
Bulk Change

Realistically, if I wanted to restart work on this, I'd probably start from scratch, so marking as Won't Fix

2019-09-09 15:59:52
btasker changed status from 'Open' to 'Closed'

2019-09-09 15:59:52
btasker added 'Won't Fix' to resolution