The cryptographic assassin

Is she still alive? That was the first question asked.

But she was not. She was clearly dead. That was undeniable. It was so clear by simply glancing over the crime scene, that the question was offensive for the policeman who first inspected the body of the young lady, now died on the floor.

The rest of the questions: who? what? why? how? were all still floating on the air like those flies trying to suck blood from the dead body.

The police inspector could only answer one question: the how.
He was over 50, and had been serving in the police department for more than twenty years, hence he had already seen too many dead bodies.  Somebody had strangulated her from behind, using a barbed wired cable – was his verdict.

Brutal, disgusting… All that was left behind was a memory stick and a beheaded body.

Call her! – the inspector shouted in anger.

Call who? – Replied the policeman

Call her!!! Call that bastard, we need her here now!!!

Hmmm, you mean the hacker? – replied the policeman knowing that his question could irritate even more the inspector.

Yes, I mean that punk lady that knows everything about computers.

And… in the meanwhile, and this is a direct order, no one is going to insert that memory stick in any computer  -continued the police inspector.

Is it that sufficiently clear to everyone? It is strictly forbidden to touch that thing, no one, not even the fingerprint team can touch it!

Only God knows what kind of atrocity it does contains.

Let’s leave her to puzzle out what all this shit is about – mumbled the inspector.

Side Projects are good

Unfortunately, when we are working, we normally do focus on some specific tasks that need to be accomplished in the shortest period of time, and with the maximum efficiency. That makes us highly productive at work, which is great, but on overall reduces our knowledge limiting it to what we use on a regular basis at work.

Hence, having a side project is important, as it is a terrific way of taking us out of our comfort zone. When you start a side project, nobody is going to answer the questions. There is no one you can directly ask something. As you are flying solo, you really need to evolve yourself to walk the path.
Do you want to learn web programming? Start a side project to create that website you always thought about.

Do you want to finish that book? Keep writing!

Do you want to create your first Iphone app? Start today better than yesterday to learn the required skills.

I know, is so hard to crash against the white paper.

But, there is no other way, and it never was.

TCP/IP Troubleshooting: Troubleshooting @ the transport layer

Just a reminder: TCP/IP is both a protocol suite (a set of communications protocols used on the Internet and other networks alike), and a couple of specific protocols: TCP and IP.

TCP/IP as a suite of protocols take its name from the most widely used protocols on the suite: TCP as a transport protocol and IP as a network protocol. Those two protocols became so famous that they are used to name the rest of the set. TCP as a transport protocol and IP as a network protocol are the rock stars of the TCP/IP suite, but the suite contains much more protocols other than TCP and IP. For example: ICMP, SSH, TELNET,LDAP,UDP... You get the idea.

We are focusing here on the TCP as a transport protocol.

As the TCP/IP protocol suite uses a layered model where each layer serves the purpose of the layer above, one good way of detecting issues in a network, is to focus on the transport layer. If there is a problem at the transport layer, you do not have to further investigate the layers above, which most of the times require to know detailed specifications of the protocol like knowing the specification details of CIFS, NFS or others.

So, the transport layer  is a good place to start seeking for issues. If nothing is wrong at the transport layer level, then we can start troubleshooting upper layers. If something is wrong at the transport layer level, then we need to troubleshoot problems at the trasport layer or below.

So far, so good, let´s start.

What are the basics concepts to know for troubleshooting  TCP?

We need to know how a TCP segment looks like.
Thanks to the Wikipedia, we have this picture of the header of a TCP segment:

The goal of the TCP protocol is to provide a connection oriented communication ensuring reliable, ordered and error-checked delivery of streams of bytes. Therefore, each of the fields in the TCP header is there for a reason.
Lets explain the basic fields.

Source port and destination port: source port and destination port allows us to identify the service the data must be sent to (destination port) and where the data is going to be sent from (source port). Because a single host offers various services such as http, ftp, telnet, etc.,  all clients connecting to it must use a destination port number to choose which particular service they would like to use. The services listening in each port number are register in  the IANA. Check here

The sequence number is important for in-order, reliable delivery.
Sequence numbers are implemented as a 32 bits number. A TCP communication can be seen as two communication streams, one from source to destination and the other one from destination to source. Source and destination maintain his own sequence numbers, each one from his side of the communication, and use sequence numbers and acknowledge numbers to keep track of the conversation, and to advance or retro cede the conversation if required.

ACK number is a 32 bits number that acknowledges the reception of  all prior bytes of information. It tells the sender until which byte, it has been received properly, so the flow of information can happen smoothly. TCP is a cumulative acknowledgment system, which can only use a single number to acknowledge data, the number of the last contiguous byte in the stream successfully received.

TCP FLAGS are 1 bit containing crucial information.

The most important are:

———————————————————————————————-

ACK flag: If the ACK bit flag is set, it means that the ACK sequence number field is relevant and contains ACK information.

SYN flag: It is used during the session setup to agree on initial sequence numbers. TCP is connection oriented, so session setup is required to agree on the sequence numbers, which are random to avoid easy attacks and spoofing.

FYN flag: It is used during a graceful session close to show that the sender has no more data to send.

RST flag: Reset is an instantaneous abort (normally and abnormal session disconnection)

PSH flag: If this flag is set, then it pushes (forces) the data delivery without waiting for buffers to fill. The data will also be delivered to the application on the receiving end without buffering.

———————————————————————————————–

Enough for the TCP flags, what else do we have?

Window Size: is a very important 2 bytes field in the TCP header. It can be read also as receive window. Is the amount of bytes in a point of time that the sender can process without exceeding its capabilities. So it is the maximum amount of data that can be received by the sender in a particular moment of time.

Hopefully we have now the basics concepts more clear, is time then to start seeing what are the situations in the transport protocol that can be causing problems to the overall communication.

Zero Window:

Zero Window means that the receiver cannot cope with the amount of data the sender is transmitting and that the available reception buffer is 0. If this happen, there is a delay, as the sender needs to wait until some space is available at the destination to continue sending information.

The filter that can be used in Wireshark  to detect a Zero Window condition is

tcp.analysis.zero_window

The presence of high number of zero window, is a clear indication that they receiver cannot keep the rhythm of the sender. Increasing the tcp buffers on the receiver is likely required in those scenarios, or reducing the amount of data being sent from the sender to the receiver.

Duplicated ACKs, Missing packets and Retransmissions

Duplicated ACKs are normally a symptom of missing packets.
When a receiver receives a packet with a sequence number higher than the expected one, it proceeds as if some data was dropped. To help make the sender aware of the apparently dropped data as quickly as possible, the receiver immediately sends an acknowledgment (ACK), with the ACK number set to the sequence number that seems to be missing. That is a dup ack;  is the receiver detecting a gap in the sequence number and basically saying to the sender: “you are sending me a segment that contains a sequence number higher than the one I am expecting, could you please send me the one I am expecting?

So the receiver detects a gap in the sequence numbers, and generate a duplicate ACK (an ACK with an ACK sequence number equal to the sequence number it is expected, which is lower that the latest sequence number received from the sender). The receiver keep sending dup acks for each subsequent segment it receives on that connection, until the missing segment is successfully received because it finally arrives by natural means or because it is retransmitted.

If the sender has fast retransmissions enabled, once it receives 3 dup acks, it will resent the missing segment.

Great number of dup acks can be a clear indication of dropped/missing packets or out of order packets (in this case packets are out of order because they have a higher than expected sequence number).

The way to determine if the dup acks end up with a retransmission is noticing the number of the dup ack:  dup ack #1, dup ack #2, and so on.
If fast retransmissions are enabled, then after 3 dup ACKS  are received by the sender, TCP performs a retransmission of that segment without waiting for the expiry of the retransmission timer.
This is known as fast retransmit because it happens before the retransmission timer expired naturally.

You can filter duplicate ACKs in Wireshark using the filter:

tcp.analysis.duplicate_ack

Bear in mind that duplicated acks can be normal as they also identify packets that arrive out of order (specifically with a higher sequence number than the expected),  so not all the duplicated acks end with a retransmission or indicate an issue. Sometime after one or two dup ack, the right segment is received, it just arrives with a delay, but there is no packet loss and no need to re-transmit, as TCP is prepared for reordering.

The way to detect retransmissions in Wireshark is with the filters:

tcp.analysis.fast_retransmission –> Retransmission triggered by the reception of 2 dup ack. Fast retransmissions.

tcp.analysis.retransmission –> Retransmission triggered by the expiration of the retransmission timer.

Out of order packets can also occur because a sequence number at the destination is lower than expected

The TCP protocol was designed to deal with out of order packets, but as TCP only passes the data up to the application when all the received bytes are in order, if there are many out of order packets,  or if they arrive with a delay, there would be a degradation in performance.

There is a  filter in Wireshark to identify out of order packets:

tcp.analysis.out_of_order

Wireshark marks a packet as out of order based on the fact that it (a) contains data, (b) does not advance the sequence number value (meaning that it is a packet that has arrived after another one that had a higher sequence number), and (c) arrives within 3 ms of the highest sequence number seen.

If reordering is excessive or applications are overly sensitive to it the performance will be degraded significantly.

On average around a 3% of the packets are out of order, if it is greater than that it can cause a performance issue, and requires further investigation.

Some references:

https://tools.ietf.org/html/rfc793
http://blogs.technet.com/b/networking/archive/2011/05/16/tcp-dupacks-and-tcp-fast-retransmits.aspx

https://wiki.wireshark.org/TCP_Analyze_Sequence_Numbers
https://support.microsoft.com/en-us/kb/224829
http://www.tcpipguide.com/free/index.htm
http://www.man.poznan.pl/cmst/2005/v_11_2/07-Przybylski.pdf
http://referaat.cs.utwente.nl/conference/6/paper/6794/the-occurrence-of-tcp-packet-reordering.pdf

Processing network captures in pcap format

I would like to build my own tool for analyzing network captures.

Most of the time my job involves to spot why a network protocol is performing slowly: using Wireshark for that task is nice, but I want something more automatic, just able to detect the issues I normally deal with, in an automatic way.

That is the reason I have decided to do some homework and analyze which libraries are available for processing pcap files.

The original pcap library in C can be found @ tcpdump.org

Information about the file format can be found @ Libpcap information

There is a port to python of the pcap library Python port of the pcap library

Also in Python, there is a tool that seems to be much more ambitious but worthy: Scapy

This is just a first sketch. Lot of job needs to be done to accomplish something useful.