########################################################################################## PAS-26: Generate list of observed unresolvable FQDNs ########################################################################################## Issue Type: New Feature ----------------------------------------------------------------------------------------- Issue Information ==================== Priority: Major Status: Open Resolution: Unresolved Project: PCAP Analysis Script (PAS) Reported By: btasker Assigned To: btasker Components: - SSL/TLS Affected Versions: - 0.1 Targeted for fix in version: - 0.1 Time Estimate: 4 minutes Time Logged: 86 minutes ----------------------------------------------------------------------------------------- Issue Description ================== I want the script to extract a list of FQDN's from SNI, as well as any returned certificate Common Names (and SANs) and then attempt to resolve each of them. For any which are unresolvable, the FQDN, the associated IP and port should be recorded ----------------------------------------------------------------------------------------- Issue Relations ================ - relates to PAS-28: Detect likely Tor Handshakes ----------------------------------------------------------------------------------------- Activity ========== ----------------------------------------------------------------------------------------- 2016-02-03 11:22:25 btasker ----------------------------------------------------------------------------------------- There are two reasons why this information may be interesting: *Unadvertised Services* Seeing (for example) a HTTPS connection go out with foo.bar.invalid as the SNI FQDN suggests that the destination server has a service responding to that FQDN. That it's not being advertised in DNS means it's potentially interesting as an apparent attempt has been made to keep it hidden from public view *Identifying Tor Usage* Depending on the version of the tor client the user is using, the value of the SNI and the certificate Common Name during a SSL handshake can help identify connections to a Tor Entry Guard. The SNI FQDN tends to be a random (at least in appearance) string (e.g. www.3avkpvvrqtgkdk.com) with the guard then returning a different string as the cert Common Name (e.g. www.52iaby6bzurz7c4gugy.net) So within tshark we'd be looking at the following - ssl.handshake.extensions\_server\_name - x509sat.printableString The presence of two non-resolvable FQDN's in one handshake is a reasonable starting indicator that the connection may be Tor related. For further confirmation, any IP's meeting that criteria could then be cross-compared against the publicly available list of Tor nodes. Whilst this method is more reliable than relying on destination port numbers (as a node's ORPort is configurable) it is contingent on _tshark_ recognising that the SSL dissector should be applied to the stream (which won't be the case for some ports) so may need to look into sane ways of forcing that. To begin with, though, simply grabbing the low-hanging fruit is a reasonable starting point ----------------------------------------------------------------------------------------- 2016-02-03 11:33:04 btasker ----------------------------------------------------------------------------------------- Note that PAS-17 implemented a configuration option to only allow passive checks. Given this feature will be contingent on attempting to resolve any FQDN observed in the PCAP this falls firmly in the active check category, so will need to honour the configuration option _PASSIVE\_ONLY_ ----------------------------------------------------------------------------------------- 2016-02-03 11:42:49 btasker ----------------------------------------------------------------------------------------- Need to double check that it won't break anything, but should be able to drop extraction of CN/SAN's in here https://github.com/bentasker/PCAPAnalyseandReport/blob/c157a45136b1fa11f9a2c09b67e1779530d24c01/PCAP_Analysis.sh#L412 Would then need to adjust processing of that temp file (https://github.com/bentasker/PCAPAnalyseandReport/blob/c157a45136b1fa11f9a2c09b67e1779530d24c01/PCAP_Analysis.sh#L499) to pull them out If we add an additional printf into that loop we can simply write the IP, Port, SNI Name, CN and SANs out to a seperate report file for processing later. In effect, it'll be very similar to the existing _visitedsites.csv_ report - except that that report doesn't include certificate CN/SANs, so an alternative might simply be to update that report to include the source for a FQDN (i.e. Host Header, SNI Hostname, Cert CN, Cert SAN) and then use that as an input to avoid duplication No part of that is active, so we don't need to honour _PASSIVE\_ONLY_ at this point. ----------------------------------------------------------------------------------------- 2016-02-03 11:45:03 btasker ----------------------------------------------------------------------------------------- That looks like a reasonable way of approaching it actually, _visitedsites.csv_ is currently generated as follows -- BEGIN SNIPPET -- printf '\tBuilding FQDN list\n' cat ${TMPDIR}/httprequests.txt ${TMPDIR}/sslrequests.txt | awk -F ' ' '{print $8}' | sort | uniq > "${REPORTDIR}/visitedsites.csv" -- END SNIPPET -- So splitting that into multiple parts so we can include a source identifier makes sense. Can then look at adjusting the generation of _sslrequests.txt_ so that there's a column for CN,SANs etc (or if needed, dump them out to a seperate temp file) ----------------------------------------------------------------------------------------- 2016-02-03 11:51:27 git ----------------------------------------------------------------------------------------- -- BEGIN QUOTE -- Repo: PCAPAnalyseandReport Commit: 291593330f0a39ca07fc53d04a3b0a8943fb0028 Author: Ben Tasker