PAS-22: TCP Transaction Log

Issue Information

Issue Type: New Feature
Priority: Major
Status: Closed

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: PCAP Analysis Script (PAS)
Resolution: Done (2015-11-29 00:03:42)
Affects Version: 0.1,
Target version: 0.1,
Labels: TCP,

Created: 2015-11-28 17:23:55
Time Spent Working
90 minutes
65 minutes
25 minutes

I haven't thought of a specifically "evil" use for this yet, but it'd be handy to have a timestamped log of TCP flags observed.

It'd add far too much noise to webtraffic.csv (especially as it'll include traffic not included in the file), but if structured carefully it should be fairly easy for the user to merge in if needed (although it'd mean making certain fields multipurpose to increase readability).

We probably don't want to log every time a segment is sent, so much as specific connection events, so


There might be instances where we want to log either PSH or ACKs (the second could be used as an indicator of the first anyway) though.

For example, for a HTTP Keep-alive connection, it'd show us the idle time between the last transmission and the connection being torn down (so we can deduce the client or server idle time depending which side sends the FIN). Once tuned, the same technique could potentially be used to deduce settings on HTTPS connections.

The standard fields should obviously be used, and I think the final report should just contain a human readable version of the flags, so whilst we might extract as
$STANDARD_FIELDS  Ack  Push  Reset  Syn  FIN
$STANDARD_FIELDS  1  0  0  1 0

We probably want the final report to be more like

If the resulting report were combined with webtraffic.csv by doing something like
 cat webtraffic.csv tcpreport.csv | sort > mergedreport.csv

The flags would show up in the FQDN column, which at least makes the CSV easy for a human to scan over, and the limited set of non-fqdn's should make it relatively easy for the file to be parsed by a custom script if needed.

Toggle State Changes


I guess, though, if we're proceeding on the assumption that one use of the report might be to create a hybrid of the tcptransactions and webtraffic files, it might be worth looking at including ACK's and PSH's as well. They can soon be stripped out when combining if needed.

Will implement something that logs them all, and then see what the savings are in terms of filesizes/processing time by disabling them. If nothing else, it could potentially become a config option.
Thinking about it, one thing this can definitely be used for is identifying port-knocking (and the sequence to be used)

Repo: PCAPAnalyseandReport
Commit: 2fb7048350bde276a65049e6c72059ecbd05f5f0
Author: Ben Tasker <github@<Domain Hidden>>

Date: Sat Nov 28 21:18:54 2015 +0000
Commit Message: Implemented basic tcp flag logging for PAS-22

Modified (-)(+)

Webhook User-Agent


View Commit

Basic implementation is in place, and combining with webtraffic.csv has been tested.

Logging straight PSH and ACKs adds very little overhead once the reports have been generated, though for a busy long-live connection it can extend the processing a bit.

A bigger saving, though, can probably be achieved by adjusting the known IP/Ports run ( so that it instead uses the data the new run has collected. At the moment we're doing two sweeps of the PCAP un-necessarily when it'll almost certainly be faster to grep the TCP transaction log for "SYN" and parse out the fields we need.
btasker changed timespent from '0 minutes' to '15 minutes'
So, to take Port-Knocking as an example use.

Assuming the server had the following KnockD config
        sequence    = 7000,8000,9000
        seq_timeout = 10
        tcpflags    = syn
        command     = /usr/sbin/iptables -A INPUT -s %IP% -p tcp --dport 22 -j ACCEPT

We might observe something like the following in tcptraffic.csv
1448627578.620575000                    41365   7000      "SYN"
1448627578.620830000                    19862   8000      "SYN"
1448627578.620863000                    23462   9000      "SYN"
1448627578.620865000                    55242   22        "SYN"
1448627578.620865000                    22      55242     "SYN/ACK"

It's probably worth coming up with a good way to script identifying that sort of pattern, especially as there might well be other traffic mixed in at the same time (a browser tab reloading somewhere, or multiple users behind NAT). Also worth bearing in mind that KnockD allows you to configure the flag to watch for, so it doesn't have to be a SYN. It also doesn't have to be TCP, but we're not going to pick up on UDP knocks in tcptraffic.csv anyway

As a general set of rules though
- Source/Dest IP will be the same for the Knock and the eventual connection
- Knock traffic will be the same before every connection to that service (e.g. Port 22 for SSH)
- There will (probably) be a close knock using a different set of ports

With observation of multiple connections over time, it should be possible to draw a pattern out, at least where one time knocks aren't in use

Repo: PCAPAnalyseandReport
Commit: c5e92d476ead1998f96b414f64b13ef8f10b8663
Author: Ben Tasker <github@<Domain Hidden>>

Date: Sat Nov 28 22:47:46 2015 +0000
Commit Message: Removed now duplicate tshark run when hunting for SYNs. See PAS-22

Modified (-)(+)

Webhook User-Agent


View Commit

btasker changed timespent from '15 minutes' to '25 minutes'
Tests have all gone OK, marking as resolved
btasker changed status from 'Open' to 'Resolved'
btasker added 'Done' to resolution
btasker changed status from 'Resolved' to 'Closed'

Work log

Ben Tasker
2015-11-28 21:22:17

Time Spent: 15 minutes
Log Entry: Implementing and testing

Ben Tasker
2015-11-28 22:54:30

Time Spent: 10 minutes
Log Entry: Removing tshark run for dest IP/ports and testing