BHAYSTACKG-3: Analyse desirable traffic behaviour

Issue Information

Issue Type: Task
Priority: Major
Status: Open

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: BASH Haystack Generator (BHAYSTACKG)
Resolution: Unresolved
Affects Version: 0.1,
Target version: 0.1,
Components: Design ,

Created: 2014-12-08 13:49:06
Time Spent Working

Need to re-consider the desirable traffic pattern.

The only considerations currently set in stone are

- Traffic generated by BASH Haystack Generator shouldn't be immediately identifiable (from outside the tunnel) as being from that source.
- There should be an option to generate random traffic in both directions (i.e. upstream and downstream)
- Traffic needs to look sufficiently like genuine use that it's indistinguisable (i.e. completely random is bad)

Issues were encountered (BHAYSTACKG-2) with multiple runs overlapping, leading to exhaustion of File Descriptor limits on the remote server.

Whilst some overlap is probably still desirable, need to re-asses the base assumptions about what the traffic pattern should be

Issue Links

Toggle State Changes


Should go without saying that it's the traffic volumes (upstream and downstream) which need to be considered. The fictional adversary is on the outside of the tunnel, so can only assess the volumes of traffic transiting the tunnel (so can't see destination IP's/Ports etc).

A good starting place is probably to identify the different profiles of 'real' traffic (splitting into seperate comments to improve readability)
Web Browsing

- Very small amount upstream (potentially with an occasional spike if a file is POST'd somewhere), pretty much just request headers
- Small amounts downstream
- Batches of rapid requests (as page requisites such as CSS and JS are retrieved)
Note: You'd also normally see the destination IP's change regularly, but as the adversary is outside the tunnel they'd not see this
Video Streaming

Depends on the type of video being streamed, with a few potential complications if the stream is adaptive (see further down)

Non-adaptive HLS

- Small amount upstream (just web requests)
- Almost identically sized chunks coming downstream, with the occasional small response (the manifest file)
- if a live stream, Manifest file responses may vary in size, but generally not by much
- If a VOD stream, Manifest file size won't change, whether it's re-requested at all would depend on player behaviour

Smooth Streaming

More or less the same assumptions can be made for Microsoft's Smooth Streaming , though as well as requests for the video content, there'd be requests for the Audio.

If a stream was adaptive then the size of the responses may occasionally change (whether increase or decrease) if the player changes bitrate. The changes would be in batches though, so there'd be at least a few requests in a row for each size.

Progressive Download

The profile for a Progressive download, would be different, though it would depend on player behaviour (as some use byte-range requests instead of a single request)

- One request going out
- Sustained (and potentially very large) download

If byte range requests were used by the theoretical player, then the pattern would be closer to that of HLS (although there'd be no manifest requests)
File Sharing

The exact behaviour obviously differs by protocol in use, but assuming we want to vaguely ape bit-torrent traffic

On a per-torrent basis:

- Small initial upstream with some downstream responses (joining the swarm)
- Lots of Small requests, larger downstream: fetching individual pieces - max 1MB each (but always multiples of 16KB as that's the bittorrent block size)
- As the 'download' progresses, the client would then start making retrieved pieces available to other peers, so we'd also get some small downstream requests, with larger response (again, max 1MB each but always a multiple of 16KB)
- A drop-off in traffic as the torrent completes (assuming the user has set their share ratio to 0)

Whilst an outside-the-tunnel adversary shouldn't be able to see the exact specifics, it's relatively important to ensure that the traffic grows in this nature.

It might also be worth looking at the traffic profile of other common uses (VoIP/Skype for example).

The aim isn't to swamp the tunnel with so much traffic that it's impossible to see anything but occasional spikes from normal usage, it's (in theory) far better, if possible, to ape standard usage patterns.
The causes leading to BHAYSTACKG-2 have been identified as a bug in the version of NGinx being used on the remote server, however longer term I think the idea of having more realistic usage profiles is a good idea
The traffic profile of the generator under the current configuration (values documented in BHAYSTACKG-2) can be see here:

Whilst the profile for a short Web-browsing session can be seen here:

The difference between the two is obviously very distinct, with the generator creating much higher levels of traffic than a single browsing session. This isn't necessarily a bad thing (as higher levels help ensure that only a small percentage increase in traffic is observed when 'real' traffic joins the mix).

The level of Granularity for the graphs is 5 minutes, however it'd also be worth generating graphs with higher levels of granularity (perhaps based on PCAPs rather than interface stats) to identify the bigger differences between the two that likely exist at a higher granularity.

Given the large difference in median ingress rates for the two profiles;

- Generator: ~250 KB/s
- Web Browsing: ~80 KB/s

I don't think you could reasonably assume that the traffic created by the generator could be considered by an outside observer to be a sign of multiple users browsing the net.

Similarly, the egress rate of the generator doesn't conform to pattern.

That said, it is reasonable to say that not all users of an Internet connection would be doing the same thing at once, some may be streaming video whilst others browse the net (for example), so the generator's traffic profile isn't necessarily immediately identifiable.
General Notes on Generated Traffic

The ingress (and to some extent, the egress) rate of the tunnel fluctuates quite a lot, sometimes it's quite high and sometimes really quite low.

Whilst a larger sampling is needed, there doesn't appear to be any specific pattern emerging, which if true, means that the current traffic levels (and pattern) should quite effectively mask any modest spikes in usage - some of the time.

The highest ingress rate observed (in the ongoing charting) is around 500 KB/s (note, bytes not bits!) with the media rate somewhere around the 250 KB/s mark.

Therefore, any 'real' traffic which caused the traffic rate to spike above 500 KB/s would likely be immediately identifiable. Although it would be difficult to calculate what proportion of that spike was genuine traffic, subtracting 250 KB/s would give a rough estimate.

That's likely to be an issue in any case, though - any system relying on junk traffic masking the real traffic will only ever be effective to a certain percentage of it's own levels. What might be helpful, though, is a way to make it easier to calculate what this percentage actually is on a per deployment basis.

In the current build (v0.1), it should be possible to calculate roughly, but because of the random nature of the traffic levels it would be incredibly difficult to do so accurately. Creating traffic profiles based on 'real' usage would likely make this a lot easier, especially if the option were available to use multiple profiles per run (so that the generator could simulate streaming video at the same time as someone browses the web for example).
BHAYSTACKG-4 has been raised to look at implementing support for various browsing profiles, though still need to run further metrics in this issue to identify what the desired behaviour should be.

Hardship that it is, that means doing some web-browsing and video-streaming (at a minimum) whilst running PCAPs so that granular traffic patterns can be analysed.
Have captured data to generate a graph with 1 ms of resolution for a web-browsing session (though it also includes any VPN overhead as an outside observer would also see this) -

Had to double check the script I used to generate the chart, as there's a far higher level of egress than I expected. Script looks fine, but I'm going to double check the test methodology (and have a closer look at the PCAP) as it just doesn't look quite right.