explanation of nestat -an command results 1

[RobJordan]

Hi, I am writing a script
that monitors connections to webserver ports
and I need a better understanding
of the different connection states reported
in plain English.

The following are the different states I can see:
CLOSE_WAIT
ESTABLISHED
FIN_WAIT_2
LISTEN
TIME_WAIT

ESTABLISHED and LISTEN are pretty self explanatory
so I am looking for an explanation of the following:
CLOSE_WAIT
FIN_WAIT_2
TIME_WAIT

Any assistance is appreciated.

Thank you,

Robert

Robert G. Jordan
Unix Sys Admin
Sleepy Hollow, Illinois U.S.A.

http://www.InYourMind.com

FREE Unix Scripts

 

[DSMARWAY]

Have a read of this , got this info from somewhere on the net can't remember which site , see if it helps

CLOSE_WAIT

It is sometimes the case that the main program that created or accepted the socket has terminated but a child is still
referencing the socket.
you can use the freeware utility lsof ( need to download depending on OS ) to determine which process(s) is still referencing the socket.

Once this is determined, the customer (or the application provider) will need to determine if the application is supposed to still have the socket open and, if not, what the application problem is. It may be the case that the application has yet to finish reading data from the socket. If you "kill" the socket, the data is "killed" as well. Not a good thing to do unless you know the application.


FIN_WAIT

This state is actually the "other end" of a connection that is in CLOSE_WAIT (no they are not "the same"!) In the case of a FIN_WAIT* state, the local application has done a "final close" on the local socket, that is, all processes that may have referenced the socket are now gone. In this case, the local TCP has sent a TCP packet to the remote with the FIN flag send (often called a "FIN packet&quot. The remote may have acknowledged the FIN flag (FIN_WAIT_2) or not (FIN_WAIT_1). In either case, the other end has not yet
indicated that it is "done" with the connection.

This is actually a very unique state(s), in that a socket may be in this state and will have no application referencing it. In this case, and this case alone, lsof will NOT show an application that has a file descriptor pointing to the socket. The local TCP is now just waiting for the other end of the connection to tell us it has read all the data and is done with the connection as well. It is never appropriate to shutdown such a connection without knowing what is going on with the other end of the connection.
There have been issues where the application on the other end of the connection took over 30 minutes to do what it needed to do and close the other end of the connection. In one particular case where this was true, the local socket (FIN_WAIT_2) was "shut down" after 10 minutes and the remote aborted. The job had been running for almost a week the whole job had to be restarted from scratch.

This brings up the point of, why do you "need" to "shutdown" the socket? Usually this is for one of two reasons. A real problem is when trying to start or restart an application (typically a server process) we get an error from the bind(2) system call indicating that the address is "in use". This usually indicates that we are trying to bind a TCP port number to the socket that is already being used.

This may be a time when you will need to "shutdown" a socket that is in FIN_WAIT*.( I think there may be a network tunable paramater to do this no -a to list it and set a time interval , if the socket is in this state for so long it will kill it , can't remember the parameter) HOWEVER... it also indicates that the server process is not really written in the manner it "should be". A server that binds a specific port number to a socket (most do) should let the port number be "reused". This is the way most well written server apps work and then it does not matter if there is an "old" socket still around in a state such a FIN_WAIT*. The server will still be able to (re)start.


TIME_WAIT

TIME_WAIT state cannot move to the CLOSED state until it has waited for two times the maximum amount of time an IP datagram might live in the Inter net. The reason for this is that while the local side of the connection has sent an ACK in response to the other side's FIN segment, it does not know that the ACK was successfully delivered. As a consequence this other side might re transmit its FIN segment, and this second FIN segment might be delayed in the network. If the connection were allowed to move directly to the CLOSED state, then another pair of application processes might come along and open the same connection, and the delayed FIN segment from the earlier incarnation of the connection would immediately initiate the termination of the later incarnation of that connection.

The TIME-WAIT state is a process waiting for enough time to pass to be sure the remote Transmission Control Protocol (TCP) received the acknowledgement of its connection termination request.)

There is also a tunable parameter to check how long the connections are kept in timewait state before they are killed.

HTH

 

[RobJordan]

DSMARWAY, thank you
for the helpful information.

Here's is the script I am using
which monitors connections to Iplanet webservers.

Robert

!/usr/bin/ksh
# webserver status
# 7/22/2002
# Version 1.0
# Author: Robert G. Jordan x2335
# monitor iplanet webservers
# requires lsof utility
clear
LSOF='/usr/local/bin/lsof -V'
EXIT=false
TMP=/tmp/TMP.file.$$
HOST_NAME=`uname -n`
IP=`/usr/bin/nslookup $HOST_NAME|grep "Address:"|tail -1|awk '{print $2}'`

clear
echo "Working..."

$LSOF|grep TCP|grep httpd|grep LISTEN|awk '{print $2,$3,$9}'|sort -u > $TMP.ws_info

until [ "$EXIT" = "true" ]
do
TIME=`/usr/bin/date`
echo " $TIME" >> $TMP.output
echo " $HOST_NAME" >> $TMP.output
echo " $IP" >> $TMP.output
cat $TMP.ws_info|while read WS_INFO
do
WS_PID=`echo $WS_INFO|awk '{print $1}'`
WS_OWNER=`echo $WS_INFO|awk '{print $2}'`
WS_PORT=`echo $WS_INFO|awk '{print $3}'|awk -F : '{print $2}'`
if [ "$WS_PORT" = "http" ]
then
WS_PORT=80
fi
WS_NAME=`ps -ef|grep httpd|grep " $WS_PID "|awk -F https- '{print $2}'|awk -F "/config" '{print $1}'`
netstat -an|grep "$WS_PORT "|egrep "CLOSE_WAIT|ESTABLISHED|FIN_WAIT_1|FIN_WAIT_2|TIME_WAIT" > $TMP.netstat
ESTABLISHED=`cat $TMP.netstat|grep ESTABLISHED|wc -l`
TIME_WAIT=`cat $TMP.netstat|grep TIME_WAIT|wc -l`
FIN_WAIT_1=`cat $TMP.netstat|grep FIN_WAIT_1|wc -l`
FIN_WAIT_2=`cat $TMP.netstat|grep FIN_WAIT_2|wc -l`
CLOSE_WAIT=`cat $TMP.netstat|grep CLOSE_WAIT|wc -l`
echo " ----------------------------------------" >> $TMP.output
echo " Webserver:\t\t$WS_NAME" >> $TMP.output
echo " Port:\t\t\t$WS_PORT" >> $TMP.output
echo " PID:\t\t\t$WS_PID" >> $TMP.output
echo " Owner:\t\t\t$WS_OWNER" >> $TMP.output
echo " CONNECTIONS" >> $TMP.output
echo " Established:\t\t$ESTABLISHED" >> $TMP.output
echo " Time_Wait:\t\t$TIME_WAIT" >> $TMP.output
echo " Fin_Wait_1:\t\t$FIN_WAIT_1" >> $TMP.output
echo " Fin_Wait_2:\t\t$FIN_WAIT_2" >> $TMP.output
echo " Close_Wait:\t\t$CLOSE_WAIT" >> $TMP.output
done
clear
cat $TMP.output
> $TMP.output
echo;echo " press Ctrl-c to exit"
sleep 2
done


----------------------------------------
Webserver: prods
Port: 443
PID: 10360
Owner: intranet
CONNECTIONS
Established: 1
Time_Wait: 23
Fin_Wait_1: 0
Fin_Wait_2: 0
Close_Wait: 0
----------------------------------------
Webserver: intranet
Port: 80
PID: 2005
Owner: intranet
CONNECTIONS
Established: 1
Time_Wait: 0
Fin_Wait_1: 0
Fin_Wait_2: 0
Close_Wait: 0
----------------------------------------
Webserver: admserv
Port: 8888
PID: 2205
Owner: root
CONNECTIONS
Established: 0
Time_Wait: 0
Fin_Wait_1: 0
Fin_Wait_2: 3
Close_Wait: 0
----------------------------------------
Webserver: usanet
Port: 445
PID: 22315
Owner: intranet
CONNECTIONS
Established: 0
Time_Wait: 0
Fin_Wait_1: 0
Fin_Wait_2: 0
Close_Wait: 0
Robert G. Jordan
Unix Sys Admin
Sleepy Hollow, Illinois U.S.A.

http://www.InYourMind.com

FREE Unix Scripts