##########################################################################################

       LOC-24: Build Tests

##########################################################################################


Issue Type: Task 
-----------------------------------------------------------------------------------------

Issue Information
====================

Priority: Major					Status:      Closed
Resolution:  Done (2018-06-08 18:31:10)
Project: LocalChat (LOC)


Reported By: btasker					
Assigned To: btasker



Targeted for fix in version: 
	- V0.0.2


Time Estimate: 0 minutes
Time Logged:   0 minutes


-----------------------------------------------------------------------------------------

Issue Description
==================

Need to build a test wrapper so that we can test that important functionality in the
server is working before release.

Also means we can test for regressions etc more easily in future - will just need to add a
test case for whatever bugs get fixed.

The wrapper should import the client and use it's methods rather than re-implementing (at
least where possible) so that client breakage also affects the tests.

-----------------------------------------------------------------------------------------

Issue Relations
================
	
	- is blocked by LOC-15: Testing Mode


-----------------------------------------------------------------------------------------

Activity
==========

	
-----------------------------------------------------------------------------------------
2018-05-20 10:46:36              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit 77224eae90c2f3d8ce0ff1c1a8412c7872abd687
Author: B Tasker <github@<Domain Hidden>>
Date:   Sun May 20 10:44:42 2018 +0100

Commit Message: LOC-24 Start building test harness
    
    Creating the test harness - it can currently spin up a background instance of the
server (and will kill it once done).
    
    Doesn't currently contain or run any tests, and doesn't currently do anything with the
client.

 tests/run_tests.py |   78 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 78 insertions(+), 0 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/77224eae90c2f3d8ce0ff1c1a8412c7872abd687
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/77224eae90c2f3d8ce0ff1c1a8412c7872abd687

	
-----------------------------------------------------------------------------------------
2018-05-20 22:22:36              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit c2d0159f3d7352f4d2a970c0a0bb1af8b3ed108c
Author: B Tasker <github@<Domain Hidden>>
Date:   Sun May 20 22:18:33 2018 +0100

Commit Message: LOC-24 Build some basic tests
    
    As much to start getting the structure as anything - it could be better/tidier but
it's functional.
    
    Uses the client as a module in order to run 3 basic tests
    
    - test_one Create a room
    - test_two Attempt to join with invalid credentials
    - test_three Join with valid credentials
    
    Test functions simply return a list of the following structure
    
    	
-- BEGIN SNIPPET --


    	[
    	{'Test' : 'Join the room','Result' : 'FAIL', 'Notes': '' },
    	isFatal
    	]
    	
 -- END SNIPPET --
    
    Where isFatal dictates whether a failure should abort the test run or not (i.e. if
other tests depend on something it does, just give up)

 tests/run_tests.py |  225 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 222 insertions(+), 3 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/c2d0159f3d7352f4d2a970c0a0bb1af8b3ed108c
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/c2d0159f3d7352f4d2a970c0a0bb1af8b3ed108c

	
-----------------------------------------------------------------------------------------
2018-05-20 22:24:36              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit 0e3dcf4dfe636578e88305d28247ce75bce145dc
Author: B Tasker <github@<Domain Hidden>>
Date:   Sun May 20 22:22:51 2018 +0100

Commit Message: LOC-24 Add numbers to tests for easier reference when failures happen

 tests/run_tests.py |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/0e3dcf4dfe636578e88305d28247ce75bce145dc
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/0e3dcf4dfe636578e88305d28247ce75bce145dc

	
-----------------------------------------------------------------------------------------
2018-05-20 22:38:36              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit 6bb4b1f7db36dbc3b0840710eea3c71d5d4eb53c
Author: B Tasker <github@<Domain Hidden>>
Date:   Sun May 20 22:35:59 2018 +0100

Commit Message: LOC-24 Add test to ensure SYSTEM is using E2E encryption
    
    Test steps
    
    - Check that a message was pushed to the room
    - Check it isn't plain text
    - Check we can decrypt it with the syskey we got when joining the room
    - Check the plaintext is valid JSON
    - Check it's a correctly formatted message payload

 tests/run_tests.py |   47 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 46 insertions(+), 1 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/6bb4b1f7db36dbc3b0840710eea3c71d5d4eb53c
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/6bb4b1f7db36dbc3b0840710eea3c71d5d4eb53c

	
-----------------------------------------------------------------------------------------
2018-05-20 22:52:36              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit 7d46d53ee2e22d1b1e8d9cdf0a3c8a48f17a2ceb
Author: B Tasker <github@<Domain Hidden>>
Date:   Sun May 20 22:49:45 2018 +0100

Commit Message: LOC-24 Close and re-open DB connection, and implement E2E test
    
    These are part of one commit because implementing the test revealed a potential race.
    
    API requests can fail because of lock contention with the test script, leading to a
failed test (and often then an exception when we try to query the database).
    
    Now, we open a new connection to the database just before querying and then close
straight after to ensure locks do not become an issue.
    
    This wouldn't be an issue with a production deployment as the DB is in memory and only
ever being accessed by the one thread.

 tests/run_tests.py |   77 +++++++++++++++++++++++++++++++++++++++++++++++----
 1 files changed, 71 insertions(+), 6 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/7d46d53ee2e22d1b1e8d9cdf0a3c8a48f17a2ceb
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/7d46d53ee2e22d1b1e8d9cdf0a3c8a48f17a2ceb

	
-----------------------------------------------------------------------------------------
2018-05-20 23:04:37              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit f6539e6c016cb6a51fa92aeb5f0e1366184dd23b
Author: B Tasker <github@<Domain Hidden>>
Date:   Sun May 20 22:57:57 2018 +0100

Commit Message: LOC-24 Added a test for user invitation

 tests/run_tests.py |   31 ++++++++++++++++++++++++++++++-
 1 files changed, 30 insertions(+), 1 deletions(-)
-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/f6539e6c016cb6a51fa92aeb5f0e1366184dd23b
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/f6539e6c016cb6a51fa92aeb5f0e1366184dd23b

	
-----------------------------------------------------------------------------------------
2018-05-20 23:04:37              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit 4e9cdea7c681f114bbbd7db12c27322a69d8ae3d
Author: B Tasker <github@<Domain Hidden>>
Date:   Sun May 20 23:02:34 2018 +0100

Commit Message: LOC-24 Invited user now joins under a new client instance
    
    This is partly to make things available for later tests, but also to ensure that the
invite functionality is actually working in the background

 tests/run_tests.py |   60 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 58 insertions(+), 2 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/4e9cdea7c681f114bbbd7db12c27322a69d8ae3d
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/4e9cdea7c681f114bbbd7db12c27322a69d8ae3d

	
-----------------------------------------------------------------------------------------
2018-05-20 23:40:36              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit bf92471864dcf9c22ae9a6fe3431586746c915fd
Author: B Tasker <github@<Domain Hidden>>
Date:   Sun May 20 23:22:09 2018 +0100

Commit Message: LOC-24 Added "Send a message" test
    
    Test does the following
    
    - Poll for messages as admin user to flush the queue
    - Send a message as testuser
    - Poll again as admin user to (hopefully) fetch the message
    - Poll as testuser to flush queue (so it's tidy for any future tests)
    - Check admin received a message and that it's well formed
    - Check that the reported payload is a match

 tests/run_tests.py |   45 ++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 44 insertions(+), 1 deletions(-)
-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/bf92471864dcf9c22ae9a6fe3431586746c915fd
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/bf92471864dcf9c22ae9a6fe3431586746c915fd

	
-----------------------------------------------------------------------------------------
2018-05-20 23:58:36              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit e39d36ad48666caae5a9df1cd05b3db7500f786d
Author: B Tasker <github@<Domain Hidden>>
Date:   Sun May 20 23:55:45 2018 +0100

Commit Message: LOC-24 Added tests for kick/ban functionality
    
    Test 11 invites a user, has them join and then kicks them (before verifying in the
database)
    Test 12 has that user rejoin and then bans them (again verifying against the
database)

 tests/run_tests.py |  170 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 169 insertions(+), 1 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/e39d36ad48666caae5a9df1cd05b3db7500f786d
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/e39d36ad48666caae5a9df1cd05b3db7500f786d

	
-----------------------------------------------------------------------------------------
2018-05-21 00:41:38              btasker
-----------------------------------------------------------------------------------------

A variety of tests have now been added, though the script output is simplistic

-- BEGIN SNIPPET --

Running test_one 
Running test_two 
Running test_three 
Running test_four 
Running test_five 
Running test_six 
Running test_seven 
Running test_eight 
Running test_nine 
Running test_ten 
Running test_eleven 
Running test_twelve 
Running test_thirteen 
Running test_fourteen 
| No | Test                             | Result | Notes |
|--------------------------------------------------------|
| 1  | Create a Room                    | Pass   |       |
| 2  | Join the room with invalid creds | Pass   |       |
| 3  | Join the room                    | Pass   |       |
| 4  | SYSTEM uses E2E                  | Pass   |       |
| 5  | Ensure payloads are encrypted    | Pass   |       |
| 6  | Invite a user                    | Pass   |       |
| 7  | Join as Invited User             | Pass   |       |
| 8  | Send a Message                   | Pass   |       |
| 9  | LOC-5 Try to invite SYSTEM       | Pass   |       |
| 10 | Send a Direct Message            | Pass   |       |
| 11 | Invite and kick a user           | Pass   |       |
| 12 | Ban a user                       | Pass   |       |
| 13 | Automated message expiration     | Pass   |       |
| 14 | Automated Room Closure           | Pass   |       |
----------------------------------------------------------

 -- END SNIPPET --

The output at the top is largely just to aid in identifying where an exception happened
(if there is one) as the backtrace won't tell you. It'll be removed later.

Tests 13 & 14 are quite long running as they each have to wait a number of minutes to
allow the server's scheduler to fire.

It could actually do with being moved to using more of a plugin like model (with each test
being it's own plugin) but I was in the mood to write the tests so have done it as one
monolithic file for now.

There's functionality that isn't currently covered too - for example leaveRoom is not
currently tested - so more tests will need to be added, but it's getting quite late so I
figured I'd call it a day.

Also need to add a datestamp to the output so that it can be piped to a file to create a
record of a test. Along those same lines, I'd also like the test script (if possible) to
get the commit reference of the last commit in the repo so that any record of the test
includes the revision that was tested.
	
-----------------------------------------------------------------------------------------
2018-05-21 00:52:36              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit 52b178296b78a821e74de19ac9eab8dd61730f8a
Author: B Tasker <github@<Domain Hidden>>
Date:   Mon May 21 00:50:13 2018 +0100

Commit Message: LOC-24 Add timestamp and revision to test output
    
    The test script will now output a serial number for the test in the format
    
-- BEGIN SNIPPET --


    	[YYYYmmDDHHMMSS]-[Latest commit]
    
 -- END SNIPPET --

 tests/run_tests.py |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/52b178296b78a821e74de19ac9eab8dd61730f8a
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/52b178296b78a821e74de19ac9eab8dd61730f8a

	
-----------------------------------------------------------------------------------------
2018-05-23 11:42:38              git
-----------------------------------------------------------------------------------------


-- BEGIN QUOTE --

Repo: LocalChat
Host:Rimmer

commit e614010c1b267314f313cd5f3fb67e2d0f8a72de
Author: B Tasker <github@<Domain Hidden>>
Date:   Wed May 23 11:40:34 2018 +0100

Commit Message: LOC-24 add test for /leave functionality
    
    Moved it before the earlier tests as otherwise the room it tests against will
auto-close because of the delay

 tests/run_tests.py |   91 ++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 88 insertions(+), 3 deletions(-)

-- END QUOTE --

https://repos.bentasker.co.uk/projects/LocalChat.git/commits/e614010c1b267314f313cd5f3fb67e2d0f8a72de
|
https://repos.bentasker.co.uk/projects/LocalChat.git/commitdiff/e614010c1b267314f313cd5f3fb67e2d0f8a72de

	
-----------------------------------------------------------------------------------------
2018-06-08 18:31:11              
-----------------------------------------------------------------------------------------

btasker changed status from 'Open' to 'Resolved'
	
-----------------------------------------------------------------------------------------
2018-06-08 18:31:11              
-----------------------------------------------------------------------------------------

btasker added 'Done' to resolution
	
-----------------------------------------------------------------------------------------
2018-06-08 18:31:14              
-----------------------------------------------------------------------------------------

btasker changed status from 'Resolved' to 'Closed'