########################################################################################## PAS-19: ssltraffic.txt should be a CSV ########################################################################################## Issue Type: Bug ----------------------------------------------------------------------------------------- Issue Information ==================== Priority: Major Status: Closed Resolution: Fixed (2015-11-27 13:12:57) Project: PCAP Analysis Script (PAS) Reported By: btasker Assigned To: btasker Components: - Reports Affected Versions: - 0.1 Targeted for fix in version: - 0.1 Labels: SSLPathExtraction, Time Estimate: 5 minutes Time Logged: 40 minutes ----------------------------------------------------------------------------------------- Issue Description ================== It wasn't originally designed that way, but really _ssltraffic.txt_ should be a CSV. All the other metadata files are generated on the basis that they can be used as a "quick-read" reference, and additional information can then be extracted from webtraffic.csv (or whichever file is relevant) if needed. The benefit of the approach above is that it's relatively easy to merge files generated from runs against multiple PCAPs. Extracting duplicates is pretty simple. That's not possible with ssltraffic.txt though, and even leaving out that downside, I'm not sure it's doing a good job of providing the information that it needs to. Essentially, it should give us the name of the site in question, and the path that's been identified. Duplicate paths should probably be removed too. That can be achieved in CSV format (leading to the benefit above) by switching to a format of -- BEGIN SNIPPET -- FQDN, Identified path -- END SNIPPET -- Where paths are of interest, they can soon be located within webtraffic.csv using a quick grep. ----------------------------------------------------------------------------------------- Activity ========== ----------------------------------------------------------------------------------------- 2015-11-27 00:57:34 btasker ----------------------------------------------------------------------------------------- I've commented out the old sections, and introduced new so that a CSV called _httpspaths.csv_ is generated using the columns described above. The old code was a bit opaque in terms of what it was doing and why, so I've made sure the new introductions are properly commented. Test run going at the moment ----------------------------------------------------------------------------------------- 2015-11-27 00:57:47 ----------------------------------------------------------------------------------------- btasker changed timespent from '0 minutes' to '10 minutes' ----------------------------------------------------------------------------------------- 2015-11-27 01:44:13 btasker ----------------------------------------------------------------------------------------- Obviously getting tired, that took a little longer to manually verify than it should have. All seems to be working, and documentation has been updated. ----------------------------------------------------------------------------------------- 2015-11-27 01:44:25 ----------------------------------------------------------------------------------------- btasker added 'SSLPathExtraction' to labels ----------------------------------------------------------------------------------------- 2015-11-27 01:44:34 ----------------------------------------------------------------------------------------- btasker changed status from 'Open' to 'Resolved' ----------------------------------------------------------------------------------------- 2015-11-27 01:44:34 ----------------------------------------------------------------------------------------- btasker added 'Fixed' to resolution ----------------------------------------------------------------------------------------- 2015-11-27 01:44:51 ----------------------------------------------------------------------------------------- btasker changed timespent from '10 minutes' to '40 minutes' ----------------------------------------------------------------------------------------- 2015-11-27 01:44:57 ----------------------------------------------------------------------------------------- btasker changed status from 'Resolved' to 'Closed' ----------------------------------------------------------------------------------------- 2015-11-27 01:45:46 git ----------------------------------------------------------------------------------------- -- BEGIN QUOTE -- Repo: PCAPAnalyseandReport Commit: 3e20cd1668c6ab9bdcd6b17fe22b7bafe8ce458d Author: Ben Tasker