########################################################################################## MISC-13: Run tests against 480p HLS Stream ########################################################################################## Issue Type: Sub-task ----------------------------------------------------------------------------------------- Issue Information ==================== Priority: Major Status: Closed Resolution: Done (2016-01-10 21:34:14) Project: Miscellaneous (MISC) Reported By: btasker Assigned To: btasker Child of: MISC-12 - Optimising Video Delivery for Tor / Building a Tor based CDN Affected Versions: - TorCDN Targeted for fix in version: - TorCDN Labels: 480p, Delivery, Experiments, Tests, Tor, Video, Time Estimate: 210 minutes Time Logged: 150 minutes ----------------------------------------------------------------------------------------- Issue Description ================== Need to repeat the tests in the parent issue, but against the 480p HLS stream of Big Buck Bunny ----------------------------------------------------------------------------------------- Issue Relations ================ - Test Summary Metrics (Projectsstatic) (http://projectsstatic.bentasker.co.uk/MISC/MISC12/Metrics-480p-10s.txt) ----------------------------------------------------------------------------------------- Activity ========== ----------------------------------------------------------------------------------------- 2015-12-18 16:54:57 ----------------------------------------------------------------------------------------- btasker changed timespent from '0 minutes' to '45 minutes' ----------------------------------------------------------------------------------------- 2015-12-24 13:12:07 ----------------------------------------------------------------------------------------- btasker changed timespent from '45 minutes' to '85 minutes' ----------------------------------------------------------------------------------------- 2015-12-24 14:02:39 ----------------------------------------------------------------------------------------- btasker changed timespent from '85 minutes' to '100 minutes' ----------------------------------------------------------------------------------------- 2015-12-24 14:08:15 btasker ----------------------------------------------------------------------------------------- The single player/single stream tests are largely complete - Test 27 (Direct to Origin) - Test 28 (Direct to midtier, coldcache) - Test 29 (Direct to midtier, warmcache) - Test 30 (Cold edge, warm midtier) - Test 31 (Warm edge, Warm midtier) - Test 32 (Cold edge, cold midtier) - Test 33 (Artificially warmed Edge 1) The result of Test 33 in particular was quite surprising. The edge showed good cache efficiency (4/5 requests were cache_hits) but the player still didn't attempt to play bitrates above 1024. Test 34 will repeat Test 33 to see if the same behaviour is observed ----------------------------------------------------------------------------------------- 2015-12-26 15:29:33 ----------------------------------------------------------------------------------------- btasker changed timespent from '100 minutes' to '150 minutes' ----------------------------------------------------------------------------------------- 2015-12-26 15:31:04 btasker ----------------------------------------------------------------------------------------- The multiplayer single stream tests are complete - Test 34 (Artificially warmed Edge 1 - repeat of Test 33) - Test 35 - Two players, cold edge - Test 36 - Two players, warm edge - Test 37 - Two players, warm edge - Test 38 - Two players, warm edge Unsurprisingly, the main finding is that as the cache warms, the players average choice of bitrate increases. ----------------------------------------------------------------------------------------- 2015-12-26 21:22:01 btasker ----------------------------------------------------------------------------------------- Was in the process of setting up a multi client test, but the midtier is currently unavailable. tor.log shows -- BEGIN SNIPPET -- Dec 26 21:10:10.000 [notice] We'd like to launch a circuit to handle a connection, but we already have 32 general-purpose client circuits pending. Waiting until some finish. -- END SNIPPET -- Can't see any other issues in the log, which suggests it may be the guard. Have moved all cached data out of the way and restarted Tor to try and get a new guard, but no dice. Will have to leave it for 10 minutes and see if the issue resolves ----------------------------------------------------------------------------------------- 2015-12-26 21:53:04 btasker ----------------------------------------------------------------------------------------- It's back, and the test is running. Kinda underlines the need for multiple caches at the midtier in a production environment. Also important that caches use different guards so that an issue in one doesn't take too large a proportion of the infrastructure out. ----------------------------------------------------------------------------------------- 2016-01-10 21:16:01 ----------------------------------------------------------------------------------------- btasker changed Project from 'BenTasker.co.uk' to 'Miscellaneous' ----------------------------------------------------------------------------------------- 2016-01-10 21:16:01 ----------------------------------------------------------------------------------------- btasker changed Key from 'BEN-602' to 'MISC-13' ----------------------------------------------------------------------------------------- 2016-01-10 21:16:40 ----------------------------------------------------------------------------------------- btasker added 'TorCDN' to Version ----------------------------------------------------------------------------------------- 2016-01-10 21:16:46 ----------------------------------------------------------------------------------------- btasker added 'TorCDN' to Fix Version ----------------------------------------------------------------------------------------- 2016-01-10 21:34:14 ----------------------------------------------------------------------------------------- btasker changed status from 'Open' to 'Resolved' ----------------------------------------------------------------------------------------- 2016-01-10 21:34:14 ----------------------------------------------------------------------------------------- btasker added 'Done' to resolution ----------------------------------------------------------------------------------------- 2016-01-10 21:34:41 ----------------------------------------------------------------------------------------- btasker added '480p Delivery Experiments Tests Tor Video' to labels ----------------------------------------------------------------------------------------- 2016-01-10 21:34:45 ----------------------------------------------------------------------------------------- btasker changed status from 'Resolved' to 'Closed' ----------------------------------------------------------------------------------------- Worklog ======== ----------------------------------------------------------------------------------------- 2015-12-18 16:54:57 btasker 45 minutes ----------------------------------------------------------------------------------------- Running first set of tests ----------------------------------------------------------------------------------------- 2015-12-24 13:12:07 btasker 40 minutes ----------------------------------------------------------------------------------------- Running single-player, single stream tests ----------------------------------------------------------------------------------------- 2015-12-24 14:02:39 btasker 15 minutes ----------------------------------------------------------------------------------------- Running final single player test ----------------------------------------------------------------------------------------- 2015-12-26 15:29:33 btasker 50 minutes ----------------------------------------------------------------------------------------- Running remaining tests