MISC-13: Run tests against 480p HLS Stream

Issue Information

Issue Type: Sub-task
Priority: Major
Status: Closed

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: Miscellaneous (MISC)
Resolution: Done (2016-01-10 21:34:14)
Affects Version: TorCDN,
Target version: TorCDN,
Labels: 480p, Delivery, Experiments, Tests, Tor, Video,

Created: 2015-12-18 16:04:13
Time Spent Working
360 minutes
210 minutes
150 minutes
Child of: MISC-12: Optimising Video Delivery for Tor / Building a Tor based CDN

Need to repeat the tests in the parent issue, but against the 480p HLS stream of Big Buck Bunny

Issue Links

Test Summary Metrics (Projectsstatic)
Toggle State Changes


btasker changed timespent from '0 minutes' to '45 minutes'
btasker changed timespent from '45 minutes' to '85 minutes'
btasker changed timespent from '85 minutes' to '100 minutes'
The single player/single stream tests are largely complete

- Test 27 (Direct to Origin)
- Test 28 (Direct to midtier, coldcache)
- Test 29 (Direct to midtier, warmcache)
- Test 30 (Cold edge, warm midtier)
- Test 31 (Warm edge, Warm midtier)
- Test 32 (Cold edge, cold midtier)
- Test 33 (Artificially warmed Edge 1)

The result of Test 33 in particular was quite surprising. The edge showed good cache efficiency (4/5 requests were cache_hits) but the player still didn't attempt to play bitrates above 1024.

Test 34 will repeat Test 33 to see if the same behaviour is observed
btasker changed timespent from '100 minutes' to '150 minutes'
The multiplayer single stream tests are complete

- Test 34 (Artificially warmed Edge 1 - repeat of Test 33)
- Test 35 - Two players, cold edge
- Test 36 - Two players, warm edge
- Test 37 - Two players, warm edge
- Test 38 - Two players, warm edge

Unsurprisingly, the main finding is that as the cache warms, the players average choice of bitrate increases.
Was in the process of setting up a multi client test, but the midtier is currently unavailable.

tor.log shows
Dec 26 21:10:10.000 [notice] We'd like to launch a circuit to handle a connection, but we already have 32 general-purpose client circuits pending. Waiting until some finish.

Can't see any other issues in the log, which suggests it may be the guard. Have moved all cached data out of the way and restarted Tor to try and get a new guard, but no dice.

Will have to leave it for 10 minutes and see if the issue resolves
It's back, and the test is running.

Kinda underlines the need for multiple caches at the midtier in a production environment. Also important that caches use different guards so that an issue in one doesn't take too large a proportion of the infrastructure out.
btasker changed Project from 'BenTasker.co.uk' to 'Miscellaneous'
btasker changed Key from 'BEN-602' to 'MISC-13'
btasker added 'TorCDN' to Version
btasker added 'TorCDN' to Fix Version
btasker changed status from 'Open' to 'Resolved'
btasker added 'Done' to resolution
btasker added '480p Delivery Experiments Tests Tor Video' to labels
btasker changed status from 'Resolved' to 'Closed'

Work log

Ben Tasker
2015-12-18 16:54:57

Time Spent: 45 minutes
Log Entry: Running first set of tests

Ben Tasker
2015-12-24 13:12:07

Time Spent: 40 minutes
Log Entry: Running single-player, single stream tests

Ben Tasker
2015-12-24 14:02:39

Time Spent: 15 minutes
Log Entry: Running final single player test

Ben Tasker
2015-12-26 15:29:33

Time Spent: 50 minutes
Log Entry: Running remaining tests