##########################################################################################

       PAS-6: Reading of multiple PCAPs

##########################################################################################


Issue Type: New Feature 
-----------------------------------------------------------------------------------------

Issue Information
====================

Priority: Minor					Status:      Open
Resolution:  Unresolved
Project: PCAP Analysis Script (PAS)


Reported By: btasker					
Assigned To: btasker


Affected Versions:		
	- 0.1  			


Targeted for fix in version: 
	- 0.1

Labels: Input, PCAP, Report, 

Time Estimate: 120 minutes
Time Logged:   0 minutes


-----------------------------------------------------------------------------------------

Issue Description
==================

It'd be helpful to be able to generate a report sourced from multiple PCAPs.

Where a large PCAP is supplied, if the host running the processing doesn't have sufficient
RAM to handle it, errors like the following will be seen

-- BEGIN SNIPPET --

ben@milleniumfalcon:/tmp$
~/Documents/src.old/System_Audit_utilities/Traffic/PCAP_Analysis.sh
TEST-PCAP-sample-7.pcap 
Analysing Port 80 Traffic

(process:9236): GLib-ERROR **: /build/buildd/glib2.0-2.40.2/./glib/gmem.c:103: failed to
allocate 41495301 bytes
/home/ben/Documents/src.old/System_Audit_utilities/Traffic/PCAP_Analysis.sh: line 18: 
9236 Trace/breakpoint trap   (core dumped) tshark -q -r "$PCAP" -Y "http.host" -T fields
-e frame.time_epoch -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport -e http.host -e
http.request.method -e http.request.uri -e http.referer -e http.user_agent -e http.cookie
> "${TMPDIR}/httprequests.txt"
Analysing HTTPS traffic

(process:9362): GLib-ERROR **: /build/buildd/glib2.0-2.40.2/./glib/gmem.c:103: failed to
allocate 41495301 bytes
/home/ben/Documents/src.old/System_Audit_utilities/Traffic/PCAP_Analysis.sh: line 26: 
9362 Trace/breakpoint trap   (core dumped) tshark -q -r "$PCAP" -Y "ssl.handshake" -T
fields -e frame.time_epoch -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport -e
ssl.handshake.extensions_server_name -e ssl.handshake.ciphersuite >
"${TMPDIR}/sslrequests.txt"
Identifying HTTPS pages from HTTP Referrers
Looking for XMPP traffic

(process:10156): GLib-ERROR **: /build/buildd/glib2.0-2.40.2/./glib/gmem.c:103: failed to
allocate 41495301 bytes
/home/ben/Documents/src.old/System_Audit_utilities/Traffic/PCAP_Analysis.sh: line 53:
10156 Trace/breakpoint trap   (core dumped) tshark -q -r "$PCAP" -Y "tcp.dstport == 5222"
-T fields -e frame.time_epoch -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport >
"${TMPDIR}/xmpprequests.txt"
Building reports
Done

 -- END SNIPPET --

If the PCAP is broken down into smaller chunks, recombining most of the current output is
fairly straightforward

-- BEGIN SNIPPET --

ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/associatedhosts.csv | sort | uniq | less
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/observedcookies.csv | sort | uniq | less
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/observedhttpuseragents.csv | sort | uniq
| less
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/visitedsites.csv | sort | uniq | less
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/webtraffic.csv | sort | uniq | less
ben@milleniumfalcon:/tmp$ cat report.TEST-PCAP-*/xmpppeers.csv | sort | uniq | less

 -- END SNIPPET --
However, there's no easy way to recombine the _ssltraffic.txt_ files

What'd be ideal is if it were possible to tell the script to read in multiple PCAPs and
then generate a single set of reports based on the content

-----------------------------------------------------------------------------------------

Activity
==========

	
-----------------------------------------------------------------------------------------
2015-11-25 15:53:14              btasker
-----------------------------------------------------------------------------------------

It will definitely require multiple runs of tshark, as it doesn't support reading from
multiple files at once -
https://www.wireshark.org/lists/wireshark-users/200808/msg00105.html

Using something like _mergecap_ isn't an option given the reason the files were split in
the first place is the system doing the processing has insufficient resources to process
as a single PCAP.

I guess the simplest way will be to wrap each of the tshark calls in a for loop and cycle
through any input files that have been specified