multi-threaded simple socket server... 1

[shadedecho]

I've written a single-threaded (though not intentionally) socket server in C++ on a linux box. It binds to a specific port, listens indefinitely for incoming connections (like via telnet), and sends/receives some data on that connection, and then calls "close()" on that connection when a certain command is issued.

it's structured right now as an infinite outer loop (while(true)) that sets up an accept for a connection, and then an "infinite" inner loop which reads/writes data until that connection is closed.

It works fine in a single-threaded situation... meaning if i telnet to it, i get the interaction i expect. However, if i have one telnet session open, and then from another machine try to do another, the second one "waits" until I exit the first, hence I assume this is a single-threaded setup.

I don't need a socket library or anything else like that, obviously I have the socket part figured out fine. I just need someone to help me understand how to adjust this to work in a multi-threaded set up, so that it is able to handle any number of connections.

I think what i need to do is have a parent program which is basically just the outer loop, which acts as the daemon, and each time a new connection comes in, it forks a new process to handle just the connection (written as a separate program, i guess).

Can someone help me understand how to do this straightforwardly in c++ on a linux system?

 

[shadedecho]

for reference, here's basically what my main program code looks like:

std::string strtolower(std::string s) {
  for (int i=0; i<s.size(); i++) {
    s[i] = tolower(s[i]);
  }
  return s;
}

int main ( int argc, int argv[] )
{
  cout << "running....\n";

  try
    {
      // Create the socket
      ServerSocket server ( 26 );

      while ( true )
        {
          ServerSocket new_sock;
          server.accept ( new_sock );

          std::string welcomemsg = "Hello. Type some stuff, see what happens.\r\n";
          std::string donemsg = "Closing Connection. Good bye.\r\n";
          new_sock << welcomemsg;

          try
            {
              while ( true )
                {
                  char c = '\0';
                  std::string data = "";
                  while (c != '\n') {
                    new_sock >> c;
                    data += c;
                  }
                  if (strtolower(data).substr(0,4) == "quit") {
                    new_sock << donemsg;
                    new_sock.disconnect();
                  }
                  else {
                    new_sock << data;
                  }
                }
            }
          catch ( SocketException& ) {}

        }
    }
  catch ( SocketException& e )
    {
      cerr << "Exception was caught:" << e.description() << "\nExiting.\n";
    }

  return 0;
}

As you can see, this is just a simple echo server, meaning it will echo back whatever you send to it, unless you send the "quit" command, in which case it will terminate the connection.

So, basically, how do i make that above program multi-threaded?

 

[CodingNovice]

A separate process to handle each connection sounds like overkill to me. I reckon a seperate thread per connection would be better.Something like the main thread listening for connection requests and on recieving one spin a thread to handle the connection.Boost.org have a threading library or you can use pthreads.

 

[Clairvoyant1332]

Depending on what it's being used for, it's definitely easier to fork off a new process. If you switch to a multithread architecture, you have to make sure all your existing code is thread safe. The bigger the codebase, the more you'll likely have to do. Of course, threading is more efficient, so you have to weigh that against the additional coding required.

For what you're doing (having a parent process listen and spawn a child for each connection), this is a classic example of when to fork.

 

[shadedecho]

Clair..."-
The "codebase" you refer to is literally almost as simple as the above code posted. It's simply listening on a socket for an incoming connection, accepting that connection, listening for certain sets of input, and giving responses. It requires no other complicated logic or anything.

Basically, I am trying to create a daemon that acts a lot like an SMTP server, except that it doesn't actually do anything, it just accepts commands and then responds with certain output.

It sounds like threading would be the more appropriate path, given that similar things like Sendmail or Apache are also multi-threaded instead of forking. Plus, I'd like to learn how to do that.

I guess the only concern is making sure the socket code ("library") itself is thread-safe... I wouldn't even know how to go about evaluating that.

Also, I guess I need an adjustment in my mindset about how multi-threading works... do I still create an endless running program (like the above one) which has while(true) type loops? Does each iteration of the loop only occur when a connection comes in? How would one structure this?

 

[Clairvoyant1332]

You main program would still do an endless loop accepting connection. But now, you spawn off a thread to do the work on the connected socket instead of doing it right there in the loop. You would use the pthread_create() function to start a new thread by passing it the address of a function which handles the accepted socket.

I'm not too familiar with the classes you're using, as I'm more of a C guy as opposed to C++, but I would guess the socket classes are essentially wrappers around the regular socket API, which is thread safe, so you should be OK there. To be sure, check the man pages. They'll tell you whether or not a particular function is thread safe. For example, the strtok() function is not thread safe because is uses a static variable. You would instead use strtok_r() which is thread safe.

Another aspect of being thread safe has to do with your code being careful not to have multiple threads access the same variables or resources at the same time. Some examples of this are a global state variable that multiple threads can modify or a log file that all threads access through the same file descriptor. You would need to use a mutex to lock out other threads one one writes to such a resource.

 

[shadedecho]

in my above code, you can see that each iteration actually creates a new connection and accepts/listens on it. As I think I am piecing together, one has the choice when creating a socket to make it blocking or non-blocking. I think this refers to whether or not it will WAIT for that connection to come in before going through another iteration of the outer loop... am I on the right track with that?

If that loop runs continuously without somehow waiting for each connection, it seems like it would be creating infinite amounts of sockets. So that is what led me to the conclusion that what it's doing is creating a "blocking" connection on the first iteration of the outer loop, but it doesn't finish the "accept()" function until one actually comes in?

As part of my testing and trial-and-error to understand this code, I tried setting it to non-blocking, and then my telnet connections to my program stopped working (they would connect and then immediately disconnect). I guessed that maybe this was because setting it to non-blocking caused the loop to just iterate over and over and over again, without waiting for a connection to come, constantly spawning listening sockets to the same port, which overwrote/reset somehow whatever connection had already just come in?

Basically, will that continuous outer loop do one iteration for each thread that it starts, WAITING on a single iteration of the loop until another connection comes in (blocking, i think), or will it run continuously (spawning one non-blocking socket listener at a time), somehow testing with an IF to see if there is a new connection pending and THEN start the thread and hand it that socket?

 

[Clairvoyant1332]

If you have a socket marked as non-blocking and you attempt to either accept() or read() when nothing is available, the function will return -1 with errno set to EWOULDBLOCK (or EAGAIN). If the socket is marked as blocking, then accept() will wait until a connection is established, and read() will wait until data becomes available. So yes, you are correct in your understanding of how that works.

By default sockets are blocking. In your example, the process/thread that is doing the accept is not doing anything else, so blocking in that situation is fine.

 

[shadedecho]

Thanks so much, Clair... that is very helpful! star for you!