##########################################################################################

       LOC-6: Periodic message clearance

##########################################################################################


Issue Type: New Feature 
-----------------------------------------------------------------------------------------

Issue Information
====================

Priority: Major					Status:      Closed
Resolution:  Done (2018-05-19 12:39:15)
Project: LocalChat (LOC)


Reported By: btasker					
Assigned To: btasker

Components: 
	- Server 


Targeted for fix in version: 
	- V0.0.2


Time Estimate: 0 minutes
Time Logged:   0 minutes


-----------------------------------------------------------------------------------------

Issue Description
==================

One of the core principles of the design is that the server should only store the message
payloads for a short time.

So need to have a thread (or something) running to periodically flush out older messages
(I originally said 15 minutes, but 5 sounds better). 

-----------------------------------------------------------------------------------------

Activity
==========

	
-----------------------------------------------------------------------------------------
2018-05-12 16:21:40              btasker
-----------------------------------------------------------------------------------------

This needs doing, but also need to work out the best way to go about it.

The easiest (tm) way of doing it would be to add a new Flask routed path (say /flush)
which would trigger the flush. Then just add a cronjob.

But it means an extra step in the deployment.

I'm wondering if it might be better to spin up an additional thread which could
periodically place the request (just as a cronjob would) to avoid the install procedure
involving editing cronttab's (as if they're forgotten, or, deleted it could lead to
leakage/compromise).
	
-----------------------------------------------------------------------------------------
2018-05-14 08:54:52              btasker
-----------------------------------------------------------------------------------------

So the way I'm leaning on this, is it's probably best to spin up a thread and create an
API endpoint.

So that thread would periodically (via urllib2) place a request to the API to trigger the
purge. As this is potentially an expensive operation we should protect it with a
passphrase to stop arbitrary users from triggering it and blocking the API.

At startup, the server app should generate a passphrase, which can then be passed into
both the API request handler and the 'cron' thread. 
	
-----------------------------------------------------------------------------------------
2018-05-19 12:06:37              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit 74c2eb491a5bede03dd926d20afd73ffe105a3b7
Author: B Tasker <github@<Domain Hidden>>
Date:   Sat May 19 12:04:00 2018 +0100

Commit Message: LOC-6 Initial (and very rough) periodic message clearance
    
    At server startup we spin up a seperate scheduler thread, which places a request to
the main app every 60 seconds, in order to purge any messages older than the threshold in
the system.
    
    It's currently very, very rough, but seems to work.
    
    There's still a bit to be done here though
    
    	- Need to validate the submitted password
    	- Need to actually generate a password rather than using the temporarily hardcoded
1234
    
    And probably quite a lot else too

 server/LocalChat.py |   75 ++++++++++++++++++++++++++++++++++++++++++++++++---
 1 files changed, 71 insertions(+), 4 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/74c2eb491a5bede03dd926d20afd73ffe105a3b7
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/74c2eb491a5bede03dd926d20afd73ffe105a3b7

	
-----------------------------------------------------------------------------------------
2018-05-19 12:24:37              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit 60da935cdf68ed97f73f3cdd0012bf008b44f572
Author: B Tasker <github@<Domain Hidden>>
Date:   Sat May 19 12:20:47 2018 +0100

Commit Message: LOC-6 Use autogenerated password for scheduler requests
    
    At startup we generate a 64 char password for the scheduler to identify itself with.
    
    Also, move the scheduler thread launch to within the app processing thread. We do this
because otherwise we end up with two scheduler threads (because of the way Flask handles
launching the 'main' thread). Also means it'll refresh if the processing thread is changed
for some reason.

 server/LocalChat.py |   40 ++++++++++++++++++++++++++--------------
 1 files changed, 26 insertions(+), 14 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/60da935cdf68ed97f73f3cdd0012bf008b44f572
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/60da935cdf68ed97f73f3cdd0012bf008b44f572

	
-----------------------------------------------------------------------------------------
2018-05-19 12:36:37              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit 6aad0d3551218f84dc186b6333323d78d49574f5
Author: B Tasker <github@<Domain Hidden>>
Date:   Sat May 19 12:31:58 2018 +0100

Commit Message: LOC-6 Switch to shorter values and insert print for testing
    
    Configures to run the scheduler every minute and expire messages after a minute.
    
    Created a room, joined it, posted some messages and then left.
    
    Scheduler dumped the following to stdout
    
-- BEGIN SNIPPET --


        1526729206.144380 Request start
        {u'action': u'schedulerTrigger', u'pass':
u'GRu5yYY8ngPGtBfbl6wYTdVA7R5CtYPuTjkG4Xh2SfcClhdEaMQmx9JCWhqQDCtg'}
        Tidying
        Start 8 messages
        End 8 messages
        127.0.0.1 - - [19/May/2018 12:26:46] "POST / HTTP/1.1" 200 -
        1526729266.210174 Request start
        {u'action': u'schedulerTrigger', u'pass':
u'GRu5yYY8ngPGtBfbl6wYTdVA7R5CtYPuTjkG4Xh2SfcClhdEaMQmx9JCWhqQDCtg'}
        Tidying
        Start 8 messages
        End 0 messages
        127.0.0.1 - - [19/May/2018 12:27:46] "POST / HTTP/1.1" 200 -
        1526729326.238788 Request start
        {u'action': u'schedulerTrigger', u'pass':
u'GRu5yYY8ngPGtBfbl6wYTdVA7R5CtYPuTjkG4Xh2SfcClhdEaMQmx9JCWhqQDCtg'}
        Tidying
        Start 0 messages
        End 0 messages
        
 -- END SNIPPET --
    As the auto-purge is an important function, should probably make the period
configurable so that we can write automated tests to verify the behaviour.

 server/LocalChat.py |   12 +++++++++++-
 1 files changed, 11 insertions(+), 1 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/6aad0d3551218f84dc186b6333323d78d49574f5
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/6aad0d3551218f84dc186b6333323d78d49574f5

	
-----------------------------------------------------------------------------------------
2018-05-19 12:38:36              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit ec523c9d6a648bfc037d4d6e0eff25e17016b987
Author: B Tasker <github@<Domain Hidden>>
Date:   Sat May 19 12:37:02 2018 +0100

Commit Message: LOC-6 Remove test lines and make purge interval configurable
    
    It's currently hardcoded at 10 minutes (600 seconds)

 server/LocalChat.py |   23 ++++++-----------------
 1 files changed, 6 insertions(+), 17 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/ec523c9d6a648bfc037d4d6e0eff25e17016b987
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/ec523c9d6a648bfc037d4d6e0eff25e17016b987

	
-----------------------------------------------------------------------------------------
2018-05-19 12:39:08              btasker
-----------------------------------------------------------------------------------------

This is now implemented, and based on testing, seems to be working.

Marking this issue as complete (and we've now unblocked LOC-11 woohoo)
	
-----------------------------------------------------------------------------------------
2018-05-19 12:39:15              
-----------------------------------------------------------------------------------------

btasker changed status from 'Open' to 'Resolved'
	
-----------------------------------------------------------------------------------------
2018-05-19 12:39:15              
-----------------------------------------------------------------------------------------

btasker added 'Done' to resolution
	
-----------------------------------------------------------------------------------------
2018-05-19 12:39:18              
-----------------------------------------------------------------------------------------

btasker changed status from 'Resolved' to 'Closed'