It'd be helpful to be able to generate a report sourced from multiple PCAPs.
Where a large PCAP is supplied, if the host running the processing doesn't have sufficient RAM to handle it, errors like the following will be seen
ben@milleniumfalcon:/tmp$ ~/Documents/src.old/System_Audit_utilities/Traffic/PCAP_Analysis.sh TEST-PCAP-sample-7.pcap
Analysing Port 80 Traffic
(process:9236): GLib-ERROR **: /build/buildd/glib2.0-2.40.2/./glib/gmem.c:103: failed to allocate 41495301 bytes
/home/ben/Documents/src.old/System_Audit_utilities/Traffic/PCAP_Analysis.sh: line 18: 9236 Trace/breakpoint trap (core dumped) tshark -q -r "$PCAP" -Y "http.host" -T fields -e frame.time_epoch -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport -e http.host -e http.request.method -e http.request.uri -e http.referer -e http.user_agent -e http.cookie > "${TMPDIR}/httprequests.txt"
Analysing HTTPS traffic
(process:9362): GLib-ERROR **: /build/buildd/glib2.0-2.40.2/./glib/gmem.c:103: failed to allocate 41495301 bytes
/home/ben/Documents/src.old/System_Audit_utilities/Traffic/PCAP_Analysis.sh: line 26: 9362 Trace/breakpoint trap (core dumped) tshark -q -r "$PCAP" -Y "ssl.handshake" -T fields -e frame.time_epoch -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport -e ssl.handshake.extensions_server_name -e ssl.handshake.ciphersuite > "${TMPDIR}/sslrequests.txt"
Identifying HTTPS pages from HTTP Referrers
Looking for XMPP traffic
(process:10156): GLib-ERROR **: /build/buildd/glib2.0-2.40.2/./glib/gmem.c:103: failed to allocate 41495301 bytes
/home/ben/Documents/src.old/System_Audit_utilities/Traffic/PCAP_Analysis.sh: line 53: 10156 Trace/breakpoint trap (core dumped) tshark -q -r "$PCAP" -Y "tcp.dstport == 5222" -T fields -e frame.time_epoch -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport > "${TMPDIR}/xmpprequests.txt"
Building reports
Done
If the PCAP is broken down into smaller chunks, recombining most of the current output is fairly straightforward
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/associatedhosts.csv | sort | uniq | less
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/observedcookies.csv | sort | uniq | less
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/observedhttpuseragents.csv | sort | uniq | less
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/visitedsites.csv | sort | uniq | less
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/webtraffic.csv | sort | uniq | less
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/xmpppeers.csv | sort | uniq | less
However, there's no easy way to recombine the
ssltraffic.txt files
What'd be ideal is if it were possible to tell the script to read in multiple PCAPs and then generate a single set of reports based on the content
Activity
2015-11-25 15:53:14
Using something like mergecap isn't an option given the reason the files were split in the first place is the system doing the processing has insufficient resources to process as a single PCAP.
I guess the simplest way will be to wrap each of the tshark calls in a for loop and cycle through any input files that have been specified