Why and How to Extract Network Connection Timestamps for DFIR Investigations

For as long as I have been doing forensics, or more specifically, live response, there has been a lot of value in reviewing a Windows system’s network connections during an investigation--in fact this is recognized as standard practice. There are many reasons to do so, however, this work is essentially done to find an anomaly, something suspicious.

In this blog I introduce the importance of extracting an unfamiliar artifact - the timestamp of each network connection-- and a method for doing so.

 

The Standard Practice

As mentioned in The Art of Memory Forensics, “almost all malware has some sort of networking capability, whether the purpose is to contact a command-and-control server, spread to other machines, or create a backdoor on the system.” Of course, this is also true for non-automated attacks, where real, live attackers do their malicious activity manually.

SANS Institute’s poster “Know Abnormal… Find Evil” lists all kinds of suspicious network activities that an investigator should look for when he or she reviews the machine’s connections:

Examples of suspicious network activities (SANS Institute)

Pick any best practice guide for doing incident response, and there’s a big chance you’ll see a recommendation to collect open network connections. See, for example:

• NIST’s Guide to Integrate Forensics Techniques Into Incident Response
• Corey Harrell’s Journey into IR Methodology.

What you do not see in these standard practices is the importance of the timestamp. By knowing not only what connections were made, but when, you add the valuable ability to construct a timeline of connections and potentially correlate the network connections with other indicators, and thereby create better context for comprehensive analysis of attacker activity and streamline the incident response process.

As part of the research, I went through the most common tools that allow you to investigate volatile data, whether it’s through a Memory Forensics analysis or during live response. Indeed, all of the tools include some sort of network connections evidence collection.

It looks like all of the available tools (or at least the ones I reviewed) try to generate an output similar to Windows’ netstat.exe, and include the following information about each connection: local IP address and port, remote IP address and port, state and owning process ID.

Example output of netstat.exe with -abno flags

Note: If you are already familiar with open network connection evidence collection and how it is used, feel free to jump right into the new findings part.

 

Existing Tools for Network Connection Extraction

Before I dive into the technical review and demonstrate our premise, I want to first explore the existing tools and their features.

Live Response

Every tool in this category, whether it be agent-based (osquery & GRR) or agentless (Kansa), has the ability to collect network connections evidence, typically by parsing the output of Windows built in utility netstat.exe, or by leveraging Windows API functions and structures.

None of the above tools currently extract the connection’s timestamp.

Memory Forensics

Since memory forensics is all about volatile data, frameworks like Volatility and Rekall also include network-related evidence as part of their plugins list.

Although the above-mentioned tools use a memory dump or a kernel driver to extract data, the information is retrieved from the same operating system resources used by netstat.exe, as explained in Michael’s (@iMHLv2) blog about the Volatility’s netscan plugin.

Volatility’s netscan plugin outputs the same information you could get from netstat, but with a bonus: the creation timestamp of a bound socket or an established connection.

Example output of Volatility’s netscan, taken from Volatility Command Reference

When I first ran the network-related plugins of Volatility, it occured to me that a timestamp for open connections could be a great piece of evidence for live response as well.

My assumption was that since Volatility was extracting the timestamps from the kernel pool, it would not be possible to get the information from user-land, so I never investigated it further.

Additionally, I thought that if netstat, and all the other user mode tools, weren’t showing a timestamp, it's probably impossible. Turns out I was completely wrong! It is possible to extract a network connection timestamp from user mode.

 

Extracting Network Connections Timestamps in User Mode

The best things happen unexpectedly.

It all started when we asked our development team to modify the current output of netstat in our forensics collection tool to an easier-to-consume output. When dealing with parsing a netstat output, most tool developers either write their own parser for netstat output stream (like Kansa), or they call the Windows API (like psutils).

It didn’t take too long for one of our development team leaders (@Liad Gareh) to reach out to us with the following conclusion: When using Windows API to extract networks connections, one can also get the timestamp of each created connection.

For example, in order to get the timestamps of all TCP network connections, these are the required steps:

1. The GetExtendedTcpTable function retrieves a table that contains a list of TCP endpoints (IP & port combination) available to the application.

   GetExtendedTcpTable, taken from MSDN

2. TCP_TABLE_CLASS enum parameter defines the type of the TCP table structure to retrieve.
3. Passing TCP_TABLE_OWNER_MODULE_ALL as the table class parameter of GetExtendedTcpTable will result in a TCP table of type MIB_TCPTABLE_OWNER_MODULE.
4. Every entry (row) in the resulted TCP table is of type MIB_TCPROW_OWNER_MODULE, which according to MSDN, “contains information that describes an IPv4 TCP connection with ownership data, IPv4 addresses, ports used by the TCP connection, and the specific process ID (PID) associated with connection”.

   MIB_TCPROW_OWNER_MODULE, taken from MSDN

5. Pay close attention to the struct above and the addition of the member liCreateTimestamp, documented as A SYSTEMTIME structure that indicates when the context bind operation that created this TCP link occurred.

 

Get-NetworkConnection with Timestamp

At first I was very skeptical... Why has no tool already been using this priceless piece of information? Why is this not part of netstat, tcpvcon, psutils? I went over all the code in GitHub that has the above-mentioned structures to see if there’s any tool that uses this evidence.

I found that many tools do in fact use the API and relevant structs, yet I was unable to find one that extracts the timestamp member. The closest thing that was found was a PowerShell code written by Lee Christensen (@tifkin_), called Get-NetworkConnection which extracts current TCP and UDP connections using the above mentioned Windows API calls, but without the timestamp evidence.

Rather than reinventing the wheel, I reached out to Lee, who was awesome, and gave me permission to use his code (thanks Lee!). In order to extract the timestamp I was seeking, I simply modified Get-NetworkConnection to include the following operations:

1. Extract the value of the CreateTimestamp member.
2. Include the Timestamp in the output.

Run it, it works!

Example of Get-NetworkConnection, with Timestamp added

After I validated that it actually works, and that one can extract the timestamp of TCP and UDP connections using this simple Windows API call, I asked myself the following questions:

1. Is it really a valuable piece of forensic evidence?
2. Can this be added to a forensics timeline?

The best way to answer these questions is to test it. I created some scenarios and tried to see if the new piece of evidence I have can come in handy.

 

Incorporating Evidence into a Timeline

I tested the value of the connection timestamp using two different scenarios. In both scenarios an attacker or malware was simulated. Our testing proved the tool to be helpful.

Scenario A: Malicious Processes

Started with a general scenario; a malicious process is spawned on an endpoint, and immediately opens a connection. This scenario can take one of two forms:

1. Backdoor installed on a system and waited for an attacker to connect.
2. Malware is dropped on a system and immediately established a connection to its C&C.

For our purposes, I went with option 1, using netcat to simulate the backdoor example, executed our forensics collection tool and threw the logs into a timeline.

Netcat listener’s connection in a timeline, alongside process execution evidence

It is mentioned in The Art of Memory Forensics that the CreateTimestamp field is populated immediately after the socket is bound, so the timestamp evidence doesn’t bring a lot of value in this case. The reason is that other evidence is created around the time of the connection, i.e. process execution evidence (Windows Event ID 4688, Prefetch files, UserAssist, etc), however these artifacts are not always available. In any case, the timestamp of the connection gives us some extra context regarding the incident.

Scenario B: Injected Code

Next, I tested a scenario in which a malware or an attacker injects code into an already running process, and only then initiates a connection. For example, if we inspect this suspicious connection associated with the process with PID 1856, we can see that it has an established outgoing connection to 10.0.2.15 over TCP port 1234.

The PID 1856 belongs to explorer.exe:

Thus far, the only timestamp we could extract would be one of the following:

1. The process’ start time, if it was still running.
2. Any evidence of execution which includes a timestamp.

Lacking the evidence of a connection timestamp, the only focal point of inspection is the timestamp associated with the creation/execution of explorer.exe. A further analysis of the timeframe, minutes before and after explorer.exe was created, shows very little information; just a regular booting procedure after a user logs in.

Execution of explorer.exe, after a user (“boss”) logged in, as seen in a timeline view

Incorporating our new evidence into the timeline above, we reveal the connection timestamp for explorer.exe. Let’s have a look at our timeline in the timeframe of the connection:

TCP Connection made by explorer.exe in the timeline reveals other malicious activity

We can see that just before the connection was created, the Injector.exe process was executed. This could possibly be the point in time where explorer.exe was injected with malicious code.

Known Limitations

As useful as this tool may be, there are some caveats.

1. Since this information is retrieved using Windows API, there is an obvious downside that rootkits can hide a connection by hooking the relevant functions.
2. Network connection information is volatile; therefore, connections can be closed by the time you collect the information.
3. The tests focused on Windows 10. I have encountered some non-deterministic results while extracting the timestamps in Windows 7 (In some cases, the connection timestamp was 1/1/1601 2:00:00 AM). Further research is required to fully understand this behavior.

 

Summary

Does this piece of evidence really change anything? Well, if you follow Harlan Carvey’s blog (you should), you will often read his take on timelines and how they can be used to “increase relative confidence in data, as well as provide increased context to the data.”

I feel that extracting the timestamp of a network connection does exactly that. The tested scenarios showed that this evidence can be used during an investigation as pivot points to help the investigator to gain greater confidence in the direction of his or her analysis. You can also use the network connection timestamps when you expand the investigation to a bigger scope, and use it for example to correlate with data in network firewall or IDS logs.

Many IR tools already use the Windows API calls to get all the network connections, but they don’t extract the timestamp of each connection. It would be really easy to include it in the results of these tools, and I highly recommend to do so.

The enhanced version of Get-NetworkConnection that includes timestamps extraction is available in our GitHub repository.