Previous Page Next Page

4.8. Logging

The Honeyd framework supports several ways of logging network activity. It can create connection logs that report attempted and completed connections for all protocols. To get more detailed information, services can log arbitrary information to Honeyd via stderr. The framework also uses syslog for communicating warnings or system-level errors. In most situations, we expect that Honeyd runs in conjunction with a network intrusion detection system (NIDS) or a set of custom scripts to parse and analyze the log files.

4.8.1. Packet-Level Logging

Packet-level logs can be enabled via the -l command line option. It takes a single filename as an argument. The directory in which the file is being created needs to be writeable by the user that Honeyd is running as — usually nobody. Analyzing packet logs is the easiest way to get an overview of what kind of traffic your honeypots receive. The log file contains information about the source and destination IP addresses, and which protocols and ports were being used. If a connection gets established, the log file also contains information about when the connection started, when it ended, and how many bytes were transmitted. Figure 4.14 contains an example from a log file in table format.

Figure 4.14. Example output from Honeyd's packet-level log file. It shows connection establishment, connection termination, and probe packets.
DateProtoTSourceDestinationInfoComment
IPPortIPPort
2005-04-02-15:35:15tcp(6)S10.3.6.139182710.1.2.1243128 [Windows XP SP1]
2005-04-02-15:35:56tcp(6)-10.3.6.139437810.1.2.848080:48 S[Windows XP SP1]
2005-04-02-15:36:11tcp(6)-10.4.7.196267110.1.2.1752380:40 RA[FreeBSD 5.0-5.1]
2005-04-02-15:39:47tcp(6)E10.3.6.139182710.1.2.1233128:9950 240 
2005-04-02-15:40:18icmp(1)-10.3.5.182 10.1.3.99: 11(0): 56 


The Date column contains a timestamp of when the packet was received by Honeyd. The next column contains information about the Internet protocol, usually TCP, UDP, or ICMP. However, when receiving rare network probes, it could also be any other Internet protocol. The third column labeled T contains the connection type: S stands for connection start, E stands for connection end, and - indicates that the packet does not belong to any connection. The next four columns show information about the source IP address, source port, destination IP address, and destination port. For some protocols, like ICMP, the port columns are empty because these protocols do not use ports. The Info column contains information associated with a connection or packet. When a connection ends, it contains the number of bytes received and sent by Honeyd, respectively. For a probe packet, it contains additional protocol information:

The Info column contains additional human readable information. In many cases, it at least contains a guess on the remote operating system based on passive fingerprinting.

For protocols that support connections like TCP or UDP, Honeyd does not log all packets but instead logs the start of connection and the corresponding end in a fashion similar to Netflow. The main benefit is reduced clutter in the logs. For example, if somebody were to download a large file from a honeypot, there is no benefit in logging each individual packet in the download. Honeyd uses an S to indicate the start of a connection and summarizes the amount of information exchanged when the connection ends using the code E.

The packet logs are very useful for data mining. A simple Python script can be used to calculate the number of different IP addresses that probe our honeypots per day, a distribution of operating systems or a list of the most popular ports. The example Python script in Figure 4.15 computes the number of unique IP addresses that contact our honeypots per day. Measuring the number of IP addresses will give you a good idea of the scanning activity your honeypots are exposed to. This measure is likely to grow over time.

Figure 4.15. A Python script to compute the number of source IP addresses per day from Honeyd's packet log files.

import sys
old_day = ''
ips = {} # Dictionary containing each unique IP once
for line in sys.stdin:
   (date, _, _, srcip, _) = line.split(' ', 4) # Extract date and source IP
   day = '-'.join(date.split('-')[0:3])
   if day != old_day:
      if old_day:
         print old_day, len(ips)
      old_day = day
      ips = {}
   ips[srcip] = 1
print day, len(ips)

These log files can grow very large over time, depending on how much traffic your honeypots receive. It is good practice to rotate these log files so your filesystem does not overflow. Honeyd supports log rotation via the USR1 signal. When Honeyd receives this signal, all current log files are being closed and new ones are opened. To manually rotate a log file, use the script shown in Figure 4.16.

Figure 4.16. Rotating log files by sending SIGUSR1 to Honeyd. We assume that Honeyd is configured to log packet logs to logfile and service level logs to logfile.srv.

mv logfile logfile.0
mv logfile.srv logfile.srv
kill -USR1 $(cat /var/run/honeyd.pid)

4.8.2. Service-Level Logging

Service-level logs can be enabled via the -s command line option. This flag takes a single filename as an argument. The directory in which the file is being created needs to be writeable by the user that Honeyd is running as — usually nobody. While packet logs give us an overview of the overall traffic, service logs give us very detailed information about the ongoing traffic. Each service script can ask Honeyd to write information into this log file by printing information to stderr. That also entails that the precise format of this log files can differ from service to service. If you write your own service emulation scripts, it is up to you to choose a format that is easy to analyze.

The example in Figure 4.17 shows a remote IP address falling for our fake proxy and SMTP servers. The IP address tried to anonymously send e-mail by connecting to a mail server via an open proxy. To the remote mail server, it seems that the e-mail originates from the IP address of the proxy. Other interesting examples gleaned from these logs show attempts to break into the secure web servers of oil companies in Russia or the login servers of instant messaging companies.

Figure 4.17. Example output from Honeyd's service log file. The format of the data section depends on each individual service emulation script. This e-mail shows a fake proxy server and a fake open mail relay.
DateProtoSourceDestinationData
IPPortIPPort
2005-04-10-00:56:48tcp(6)10.3.23.14325910.1.3.2223128:CONNECT 10.4.228.113:25 HTTP/1.0
2005-04-10-00:59:12tcp(6)10.3.23.14334310.1.3.1248000:CONNECT 10.5.167.5:25 HTTP/1.0
2005-04-10-01:04:20tcp(6)10.3.23.14411610.1.3.2093128:some@net.em rightwards double arrow schan@net.em


Previous Page Next Page