##########################################################################################

       HLS-33: Brew no longer supports --with-default-names

##########################################################################################


Issue Type: Bug 
-----------------------------------------------------------------------------------------

Issue Information
====================

Priority: Major					Status:      Closed
Resolution:  Fixed (2019-03-29 13:09:12)
Project: HLS Stream Creator (HLS)


Reported By: btasker					
Assigned To: btasker


Affected Versions:		
	- 1.0  			


Targeted for fix in version: 
	- 1.0


Time Estimate: 0 minutes
Time Logged:   0 minutes


-----------------------------------------------------------------------------------------

Issue Description
==================

It looks like brew has removed support for the --with-default-names argument. So when
trying to install GNU sed Mac users will get


-- BEGIN SNIPPET --

Error: invalid option: --with-default-names

 -- END SNIPPET --

There are a bunch of commits from January like this one -
https://github.com/Homebrew/homebrew-core/commit/43f2e19d93f04eddf24ec19187b3648f594b80e2

Can see that they've removed support for the option, and now print a caveat on install
instructing the user to edit bashrc if they want the programs to run from their original
names (i.e. sed instead of gsed).

I'm not sure that's particularly an improvement in user experience, but it is what it is.

Need to update the README to detail the new process. Did toy with the idea of including a
test in the script to see if gsed exists (and use that if it does), but I don't really
like the idea of having hacks in there for the sake of one (albeit popular) OS.

In Github #30 (where this was identified), user has provided the output from running `brew
install grep`

-- BEGIN SNIPPET --

brew install grep
==> Downloading https://homebrew.bintray.com/bottles/grep-3.3.mojave.bottle.2.tar.gz
######################################################################## 100.0%
==> Pouring grep-3.3.mojave.bottle.2.tar.gz
==> Caveats
All commands have been installed with the prefix "g".
If you need to use these commands with their normal names, you
can add a "gnubin" directory to your PATH from your bashrc like:
PATH="/usr/local/opt/grep/libexec/gnubin:$PATH"

 -- END SNIPPET --

That path can differ depending on flags passed in, but that should be the default.

Also need to find out whether this post
(https://scriptingosx.com/2017/04/about-bash_profile-and-bashrc-on-macos/) about Macs
being a bit stupid when it comes to bashrc and bash_profile is still accurate.

-----------------------------------------------------------------------------------------

Issue Relations
================
	
	- Github #30 (https://github.com/bentasker/HLS-Stream-Creator/issues/30)



-----------------------------------------------------------------------------------------

Activity
==========

	
-----------------------------------------------------------------------------------------
2019-03-26 17:21:21              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --


Repo: HLS-Stream-Creator
Commit: 5807524b54017b429e77a357c757576d283baff5
Author: B Tasker <github@<Domain Hidden>>

Date: Tue Mar 26 17:18:23 2019 +0000
Commit Message: HLS-33 Test for presence of ggrep and gsed and use those if they exist

This is due to a change in `brew` that means you can no longer use `--with-default-names`,
the replacement method (adjusting PATH in bashrc) seems to have inconsistent results.



Modified (-)(+)
-------
HLS-Stream-Creator.sh



-- END QUOTE --

*Webhook User-Agent*


-- BEGIN SNIPPET --

GitHub-Hookshot/903858c
 -- END SNIPPET --

https://github.com/bentasker/HLS-Stream-Creator/commit/5807524b54017b429e77a357c757576d283baff5


	
-----------------------------------------------------------------------------------------
2019-03-26 17:25:21              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --


Repo: HLS-Stream-Creator
Commit: 4edc45061c8ffda73147d4a01a85be19492357df
Author: B Tasker <github@<Domain Hidden>>

Date: Tue Mar 26 17:24:25 2019 +0000
Commit Message: HLS-33

Not convinced I could have screwed that last commit up much more.

* Reverse the logic (override the var if the command completes successfully)
* Insert missing semicolons

In my defence, the commit *was* done to deploy onto my test system



Modified (-)(+)
-------
HLS-Stream-Creator.sh



-- END QUOTE --

*Webhook User-Agent*


-- BEGIN SNIPPET --

GitHub-Hookshot/903858c
 -- END SNIPPET --

https://github.com/bentasker/HLS-Stream-Creator/commit/4edc45061c8ffda73147d4a01a85be19492357df


	
-----------------------------------------------------------------------------------------
2019-03-26 17:29:45              btasker
-----------------------------------------------------------------------------------------

Feedback in the Github ticket, along with various resources online, suggest that relying
on users to adjust bashrc is going to give fairly inconsistent results (depending on which
terminal program they use), so decided to bite the bullet and write a check into the
script instead.

There's a test encode running at the moment (seemingly successfully, but time will tell).

I'm going to update the README to remove the brew steps (so that they don't cause
confusion). Recording the original content below for posterities sake

-- BEGIN QUOTE --


Segment encryption won't work out of the box on OS X as it relies on arguments which the
BSD `grep` and `sed` commands don't support. In order to use encryption on OS X you must
first install their GNU counterparts


-- BEGIN SNIPPET --


    brew install gnu-sed --with-default-names
    brew install grep --with-default-names

 -- END SNIPPET --


-- END QUOTE --
	
-----------------------------------------------------------------------------------------
2019-03-26 17:33:21              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --


Repo: HLS-Stream-Creator
Commit: 880b71da2f62b97700f5a5117210f0f8080d5c7c
Author: B Tasker <github@<Domain Hidden>>

Date: Tue Mar 26 17:32:03 2019 +0000
Commit Message: HLS-33 Update README



Modified (-)(+)
-------
README.md



-- END QUOTE --

*Webhook User-Agent*


-- BEGIN SNIPPET --

GitHub-Hookshot/903858c
 -- END SNIPPET --

https://github.com/bentasker/HLS-Stream-Creator/commit/880b71da2f62b97700f5a5117210f0f8080d5c7c


	
-----------------------------------------------------------------------------------------
2019-03-26 17:39:46              btasker
-----------------------------------------------------------------------------------------

Test encode completed successfully, no errors thrown. Segments are encrypted on disk, and
the key line has been successfully written into the manifest.

Have asked the user who originally reported issues to try pulling the latest revision to
run on their Mac. Other than creating aliases I don't have a good way to test how a Mac
will behave
	
-----------------------------------------------------------------------------------------
2019-03-28 16:09:00              btasker
-----------------------------------------------------------------------------------------

Frustratingly it seems not to be working. The failure being reported appears when trying
to insert the EXT-X-KEY directive into the manifest (exactly where we expect BSD-sed to
fail), but the new test _should_ be causing us to use gsed

-- BEGIN SNIPPET --

for manifest in ${OUTPUT_DIRECTORY}/*.m3u8
do
    # Insert the KEY at the 5'th line in the m3u8 file
    $SED -i "5i #EXT-X-KEY:METHOD=AES-128,URI="${KEY_PREFIX}${KEY_NAME}.key "$manifest"
done

 -- END SNIPPET --

The user has provided output which shows the test _should_ work (in the sense that command
functions as expected on a Mac)

-- BEGIN SNIPPET --

$ command -v gsed
/usr/local/bin/gsed

$ command -v sed
/usr/bin/sed

 -- END SNIPPET --

I wondered if perhaps it doesn't like the ternary operator used on that line in the
script, so have asked for the output of the following

-- BEGIN SNIPPET --

SED='sed'
command -v gsed >/dev/null 2>&1
if [ "$?" == "0" ]
then
SED='gsed'
fi
echo $SED

command -v foobar >/dev/null 2>&1
echo $?

command -v command >/dev/null 2>&1
echo $?

 -- END SNIPPET --
If exit statuses are set correctly then it'd be worth adjusting to use those. If not,
might need to revisit the original decision to include logic in the script to handle it.
Sodding Macs.
	
-----------------------------------------------------------------------------------------
2019-03-28 17:52:21              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --


Repo: HLS-Stream-Creator
Commit: 765e5c26a8f423b5e6e5c3740cebe884ad1fcdda
Author: B Tasker <github@<Domain Hidden>>

Date: Thu Mar 28 17:49:18 2019 +0000
Commit Message: HLS-33 Macs appear not to like the ternary operator (see #30)

Swapping to a less concise syntax. Either I've missed something more fundamental, or this
should work



Modified (-)(+)
-------
HLS-Stream-Creator.sh



-- END QUOTE --

*Webhook User-Agent*


-- BEGIN SNIPPET --

GitHub-Hookshot/903858c
 -- END SNIPPET --

https://github.com/bentasker/HLS-Stream-Creator/commit/765e5c26a8f423b5e6e5c3740cebe884ad1fcdda


	
-----------------------------------------------------------------------------------------
2019-03-29 09:02:02              btasker
-----------------------------------------------------------------------------------------

User has provided a video of them running HLS-Stream-Creator. Was originally uploaded to
youtube, so I've put a copy here (
https://videos.bentasker.co.uk/Player.html?url=2018/03/201803_hls_33_sed_error/hls_33_rec_master.m3u8
) so that can refer back if needed.
	
-----------------------------------------------------------------------------------------
2019-03-29 09:54:36              btasker
-----------------------------------------------------------------------------------------

Watching the video, this is actually a change in behaviour.

The key line isn't being written into the manifest, but there's also no error being thrown
to stderr. With BSD sed we'd normally see something like

-- BEGIN SNIPPET --

sed: 1: "./output/dil.mp4.m3u8": invalid command code .

 -- END SNIPPET --

and that's not present here. Suggests gsed is being called but something else is going
wrong. Need to try and lay hands on a Mac later to tinker with this some more
	
-----------------------------------------------------------------------------------------
2019-03-29 11:12:21              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --


Repo: HLS-Stream-Creator
Commit: 2114885fc19d3dd80db9cb89c40fbe1b0f4c756f
Author: B Tasker <github@<Domain Hidden>>

Date: Fri Mar 29 11:09:59 2019 +0000
Commit Message: Fix variable name for HLS-33



Modified (-)(+)
-------
HLS-Stream-Creator.sh



-- END QUOTE --

*Webhook User-Agent*


-- BEGIN SNIPPET --

GitHub-Hookshot/903858c
 -- END SNIPPET --

https://github.com/bentasker/HLS-Stream-Creator/commit/2114885fc19d3dd80db9cb89c40fbe1b0f4c756f


	
-----------------------------------------------------------------------------------------
2019-03-29 11:13:12              btasker
-----------------------------------------------------------------------------------------

Have laid hands on a Mac running Mojave and have now set it up with homebrew and ffmpeg
(bit of a rigmarole, why do people like these things so much?).

First run yielded some errors not visible in the video - I accidentally set the wrong
variable for the ggrep detection, so we were setting SED='ggrep'. Which would be why the
key wasn't getting inserted.

Test run following the commit to fix that resulted in the key being inserted correctly.
	
-----------------------------------------------------------------------------------------
2019-03-29 11:39:32              btasker
-----------------------------------------------------------------------------------------

The variable name was also incorrect in the original commit implementing the check -
https://github.com/bentasker/HLS-Stream-Creator/commit/5807524b54017b429e77a357c757576d283baff5
	
-----------------------------------------------------------------------------------------
2019-03-29 11:49:39              btasker
-----------------------------------------------------------------------------------------

I've updated the title of this issue to no longer refer to updating the README, as work
has been done within the script too. Readme is up to date.
	
-----------------------------------------------------------------------------------------
2019-03-29 11:56:21              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --


Repo: HLS-Stream-Creator
Commit: 74ed3d5e5a1ce8d43c9e7d68c436131f3509b78d
Author: B Tasker <github@<Domain Hidden>>

Date: Fri Mar 29 11:53:47 2019 +0000
Commit Message: Add setup steps on Mac OS X, based on having run through them in HLS-33



Modified (-)(+)
-------
README.md



-- END QUOTE --

*Webhook User-Agent*


-- BEGIN SNIPPET --

GitHub-Hookshot/903858c
 -- END SNIPPET --

https://github.com/bentasker/HLS-Stream-Creator/commit/74ed3d5e5a1ce8d43c9e7d68c436131f3509b78d


	
-----------------------------------------------------------------------------------------
2019-03-29 13:09:13              
-----------------------------------------------------------------------------------------

btasker changed status from 'Open' to 'Resolved'
	
-----------------------------------------------------------------------------------------
2019-03-29 13:09:13              
-----------------------------------------------------------------------------------------

btasker added 'Fixed' to resolution
	
-----------------------------------------------------------------------------------------
2019-03-29 13:09:17              
-----------------------------------------------------------------------------------------

btasker changed status from 'Resolved' to 'Closed'