########################################################################################## HLSDVD-3: Build Reference Platform ########################################################################################## Issue Type: Task ----------------------------------------------------------------------------------------- Issue Information ==================== Priority: Major Status: Closed Resolution: Won't Fix (2019-09-09 15:59:52) Project: HLSDVD (HLSDVD) Reported By: btasker Assigned To: btasker Time Estimate: 0 minutes Time Logged: 0 minutes ----------------------------------------------------------------------------------------- Issue Description ================== Although the eventual solution is supposed to be relatively platform agnostic, need to have a base platform to build on/test against. The most flexible approach is probably to go with a VM with an eye to porting over to a Raspberry Pi afterwards. The VM should be relatively memory/CPU constrained to help pick up on expected issues on the Pi. Base specs for the VM are - 1 vCore - 512MB RAM - Debian 8 minimal install - 1 NIC (to begin with) - 20GB HD One limitation of the Pi that will be a little hard to emulate is the effect of the NIC and USB sharing bandwidth. It's unlikely you'd want to write video segments to an SD card given the Pi's propensity for VM corruption, so the output would need to be written to either a USB stick/drive or a RAM disk. Will start by writing direct to disk until we've got an idea of the memory usage. The Pi2 has 1GB of RAM, so depending on processing requirements and segment size, it may be feasible to create a RAM disk for output to be written to. ----------------------------------------------------------------------------------------- Activity ========== ----------------------------------------------------------------------------------------- 2015-06-16 02:33:28 btasker ----------------------------------------------------------------------------------------- Default passwords configured as - encoder/encoder - root/toor Before the VM image is published, will add a script to force a reset at first login. _root_ is not currently permitted to login via SSH, _encoder_ can do so and then use _su_ or _sudo_ ----------------------------------------------------------------------------------------- 2015-06-16 02:53:38 btasker ----------------------------------------------------------------------------------------- _ffmpeg_ (for transcoding), _NGinx_ (for HLS Egress) and _Python_ are installed ----------------------------------------------------------------------------------------- 2015-06-17 02:13:08 btasker ----------------------------------------------------------------------------------------- Still waiting for the HDMI extender to turn up, so in the meantime have started adding a new feature to the VM -- BEGIN SNIPPET -- root@hlsenc:/home/encoder# apt-get install proftpd incron root@hlsenc:/home/encoder# mkdir /var/ingress root@hlsenc:/home/encoder# echo root > /etc/incron.allow root@hlsenc:/home/encoder# incrontab -e root@hlsenc:/home/encoder# incrontab -l /var/ingress IN_CLOSE_WRITE /bin/bash /home/encoder/encoding/FTP/process_ingress.sh $# root@hlsenc:/home/encoder# echo "DefaultRoot /var/ingress encoder" > /etc/proftpd/conf.d/encoder.conf root@hlsenc:/home/encoder# service proftpd restart root@hlsenc:/home/encoder# mkdir /var/www/html/output root@hlsenc:/home/encoder# chown encoder:encoder /var/www/html/output -- END SNIPPET -- The _process\_ingress_ script moves the file from the ingress directory into a processing directory (currently /var/inprogress) and then calls HLS-Stream-Creator against it to create an Adaptive stream in /var/www/html/output ----------------------------------------------------------------------------------------- 2015-06-17 03:11:45 btasker ----------------------------------------------------------------------------------------- The _process\_ingress.sh_ script pulls in a global config to set a few bits -- BEGIN SNIPPET -- encoder@hlsenc:~$ cat ~/encoding/config OUTPUT_DIRECTORY="/var/www/html/output/" FFMPEG_FLAGS="-map 0 -map -0:s" export OUTPUT_DIRECTORY export FFMPEG_FLAGS -- END SNIPPET -- The _FFMPEG\_FLAGS_ set tell _ffmpeg_ to disregard any subtitles (useful if uploading a Makostra - mkv). Because the action is triggered by _incron_ the process also works if you scp a file to /var/ingress. ----------------------------------------------------------------------------------------- 2015-06-17 07:11:12 btasker ----------------------------------------------------------------------------------------- A couple of other tweaks *tmpfs egress directory* The VM is configured with a dynamically allocated disk image, so was causing a bit of a bottleneck in I/O speeds. Have edited _fstab_ to make /var/www/html/output a _tmpfs_ mount for faster reads/writes (though the contents are obviously lost if the system reboots). *Remote source support* Pushing a file with extension .remote will cause the file to be read. Contents are currently expected to be 1 or 2 lines: -- BEGIN SNIPPET -- scheme:url output bitrates -- END SNIPPET -- Output bitrates are a comma seperated list (in K). If not present, a default of 128,1024 is used (arbitrary numbers picked to prove the concept). The location string needs to be in a format that _ffmpeg_ understands, e.g. -- BEGIN SNIPPET -- rtmp://fms.12E5.edgecastcdn.net/0012E5/mp4:videos/8Juv1MVa-485.mp4 64,512 -- END SNIPPET -- The uploaded file will remain in the ingress directory until the stream ends ----------------------------------------------------------------------------------------- 2015-06-18 03:49:20 btasker ----------------------------------------------------------------------------------------- Have built basic support for the HDMI adapter capture in, control is currently via the filename extension 'lenkeng' - _start.lenkeng_ - start capturing and streaming - _stop.lenkeng_ - stop any running streams It's not a particularly clean way of doing it, but it proves the concept until I get a web interface/API in place. Output is to /var/www/html/output/dvd/ ----------------------------------------------------------------------------------------- 2015-06-18 23:15:28 btasker ----------------------------------------------------------------------------------------- As both HLS-4 and HLS-5 will benefit from it's presence, seems worth putting _memcached_ onto the system. Technically, both could be achieved using MySQL/MariaDB but should get better response times from _memcached_. -- BEGIN SNIPPET -- root@hlsenc:/home/encoder# apt-get install memcached -- END SNIPPET -- ----------------------------------------------------------------------------------------- 2016-05-21 11:37:27 git ----------------------------------------------------------------------------------------- -- BEGIN QUOTE -- Repo: HLS_Encoder Host:Rimmer commit f2a23076d338f09f488bace7cf885cc4b11c460a Author: B Tasker