PAS-4: HTTPS Referrer Search should only match on the Referrer field

Issue Information

Issue Type: Bug
Priority: Major
Status: Closed

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: PCAP Analysis Script (PAS)
Resolution: Fixed (2015-11-27 13:17:10)
Affects Version: 0.1,
Target version: 0.1,
Components: Processing Logic , HTTP , SSL/TLS ,
Labels: Referrer, SSLPathExtraction,

Created: 2015-11-22 09:46:13
Time Spent Working
90 minutes
80 minutes
10 minutes

The current means of searching for HTTPS sites within HTTP referrers is overly broad, the HTTP CSV is simply grepped.

As a result, if a site has a cookie with a value matching the string we're searching for it will be matched and included in the known SSL paths section.

For example
1447885734.980645000   58834   80   GET     /wp-content/uploads/2015/11/black-box-600x338-e1447374320621.jpg
eaking-research/ Mozilla/5.0 (Linux; Android 4.4.2; en-gb; SAMSUNG GT-I9505 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/1.5 Chrome/28.0.1500.94 Mobile Safari/537.36    original_referrer=;_ ga=GA1.2

Would match a search for despite not actually being related.

Note: the fix for PAS-2 _might make this point moot

Toggle State Changes


btasker added 'Referrer SSLPathExtraction' to labels
The changes made in PAS-2 make this less likely, but do not eliminate it. For example, if there are genuine entries for the search would still pick out the example above.

So additional work still needs to be done to implement this issue
It's a little less efficient, but matching is now performed exclusively against the referrer field. To avoid continually grepping the full HTTP request set, the httpsreferers temp file is still used.

Repo: PCAPAnalyseandReport
Commit: 089c2bfe605f4a4df6f584b74ad0a8553ec3c01b
Author: B Tasker <github@<Domain Hidden>>

Date: Sun Nov 22 13:03:24 2015 +0000
Commit Message: SSL Path extraction is now limited to matches on HTTP Referers. See PAS-4

Modified (-)(+)

Webhook User-Agent


View Commit

btasker changed timespent from '0 minutes' to '10 minutes'
Seems to be working, so closing as fixed.

Clearly I massively overestimated the work required to fix :)
btasker changed status from 'Open' to 'Resolved'
btasker added 'Fixed' to resolution
btasker changed status from 'Resolved' to 'Closed'
Re-opening to assign to a component
btasker removed 'Fixed' from resolution
btasker changed status from 'Closed' to 'Reopened'
btasker changed status from 'Reopened' to 'Resolved'
btasker added 'Fixed' to resolution
btasker changed status from 'Resolved' to 'Closed'

Work log

Ben Tasker
2015-11-22 13:07:25

Time Spent: 10 minutes
Log Entry: Changing and testing