TIdHTTPServer, OnGet, and Threads

[ajbufort]

Hello All,

I am writing an HTTP server (TIdHTTPServer) program that services GET requests for images. This 'OnGet' procedure, a procedure of my main form 'Form1', calls other procedures of Form1 to do various tasks, and then finally gets back to the client via TIdHTTPResponseInfo.

I have come to understand that TIdHTTPServer is, by default, multithreaded - that is, whenever someone makes a GET request, and the OnGet procedure is called, a new thread is created just to deal with that one request. However, when I run multiple clients against the server at the same time, the response time significantly slows down. Of course, this could either be the result of what I am doing in those methods (graphics processing), or it could be the result of bad thread considerations on my part. I want to make sure it's not the latter.

So my question is: even though there are multiple threads being spawned to deal with clients, are the procedures that OnGet is calling being executed serially, and thus slowing things down? Do I have to create other threads or do something special to those procedures to make sure multiple occurrences of them can run in parallel, so to speak? Or is there something else I should be looking at here?

If anything needs clarification, please do not hesitate to ask. I need to make my server handle hundreds of clients, and I am seeing significant response-time slowdown after only 3+ connections!

Thanks,

-Tony

 

[Griffyn]

The VCL runs in a single thread. Additional threads can use it, but only one thread at a time. That is, if you have 100 threads all wanting to read a TEdit on your form, they all do it one after the other.

Rather than call a method contained in a TForm, write all your code attached to one of the TidHTTPServer events, like OnExecute (if it has one).

 

[buho]

I agree with you the component is calling the OnCommandGet event in the context of a thread created by itself (at least that is what TIdHTTPGetEvent help entry appears to say, but better check it with the debugger if you are not 300% sure).

Being so, the TForm methodes you are calling from the event are running in the context of the TIdPeerThread and not in the VCL thread context. From this point of view I can't see any serialization issues.

Synchronization issues are another beast. If you are blocking your code in a lock and the process is long, then yes: you are introducing serialization in the server requests.

HTH.
buho (A).

 

[ajbufort]

Hi Again, Guys!

OK, here's the deal - I was referencing a status label on the form, but I commented that reference out, just to be safe. So now when my code runs, I am not accessing anything on the form whatsoever during those GET requests. The procedures I had originally attached to the form I now have as procedures of a Thread. This thread pretty much encompasses all of the activity I need to execute the GET response. So, as far as I can tell, everything execuutes within threads now. When I try this out, however, I still get crappy performance. As soon as a second client starts making requests, the response time drops down.

I have tried many variations of things to get the performance to improve. This is the latest code I have for the GET method:

procedure TPresenterForm.OnGet(AThread: TIdPeerThread;
  ARequestInfo: TIdHTTPRequestInfo; AResponseInfo: TIdHTTPResponseInfo);
var
  CaptureThreadT : CaptureThread;
begin
  if ( ARequestInfo.Params.Values['stop'] = 'true') then
  begin
    Application.Terminate;
  end;

  numOfRequests := numOfRequests + 1;
  // StatusBar.Caption := 'Processing request #' + IntToStr(numOfRequests);

  CaptureThreadT := CaptureThread.Create(true);
  CaptureThreadT.FreeOnTerminate := true;

  AResponseInfo.ContentStream := CaptureThreadT.Execute;
  AResponseInfo.WriteContent;
end;

You'll notice that I am handling the Execute method of the thread as a FUNCTION, not a procedure.

function CaptureThread.Execute: TMemoryStream;
var
  JpegStream : TMemoryStream;
  pic, pic2 : TBitmap;
  CurrentScreen: TMemoryStream;
  
begin
  pic := TBitmap.Create;
  JpegStream := TMemoryStream.Create;
  ScreenShot(pic);
  // pic.LoadFromFile('screen.bmp');
  DrawCursor(pic);
  pic2 := TBitmap.Create;

  pic2.Width := 830;
  pic2.Height := 580;

  SmoothResize( pic, pic2 );
  BMPtoJPGStream( pic2, JpegStream );

  CurrentScreen := TMemoryStream.Create;
  CurrentScreen.CopyFrom( JpegStream, 0 );

  // PresenterForm.StatusBar.Caption := IntToStr(CurrentScreen.Size);

  JpegStream.Free;
  pic.FreeImage;
  pic2.FreeImage;
  FreeAndNil(pic);
  FreeAndNil(pic2);

  result := CurrentScreen;
end;

I am doing this because, in a previous test, I had the thread being executed by a timer every so many milliseconds, and populating a variable (CurrentScreen)which was assigned to the content stream. This resulted in even crappier performance. Also, I had to do the content stream assignment within a check to see if the timer was disabled first - otherwise, I had access issues with regard to CurrentScreen. This is what I was doing in TimerTimer:

procedure TPresenterForm.TimerTimer(Sender: TObject);
var
  CaptureThreadT : CaptureThread;
begin
  Timer.Enabled := False;

  CaptureThreadT := CaptureThread.Create(true);
  CaptureThreadT.FreeOnTerminate := true;
  CaptureThreadT.Resume;

  Timer.Enabled := True;
end;

And by the way, this is how I have defined my CaptureThread:

CaptureThread = class(TThread)
  protected
    function Execute: TMemoryStream;
    procedure ScreenShot( ScreenShotBitmap : TBitmap );
    procedure DrawCursor( ScreenShotBitmap : TBitmap );
    procedure SmoothResize(Src, Dst: TBitmap);
    procedure BMPtoJPGStream(const Bitmap : TBitmap; var AStream: TMemoryStream);
  end;

I know that I am doing some graphics-intensive stuff here. Is the idea of repeated screenshots on demand via HTTP feasible, or do sockets HAVE to be used to get proper multi-client performance. I get the feeling i am missing something very subtle and basic to get all of this to work.

-Tony

 

[whosrdaddy]

Tony,
what you're doing here makes absolute NO sense at all.

a) the onget event is fired from a thread (not the main VCL) thread
b) EACH session has it own thread

What I want to say is the following, your capture thread is not needed here since you are already in a thread. even worse, having a lot of threads can create serious overhead for the windows scheduler.

here's what you must do :

function TForm1.ScreenToMemoryStream : TMemoryStream;

var JpegStream : TMemoryStream;
    pic : TBitmap;
begin
 pic := TBitmap.Create;
 JpegStream := TMemoryStream.Create;
 ScreenShot(0,0,Screen.Width,Screen.Height,pic);
 BMPtoJPGStream(pic, JpegStream);
 Result:=JPegStream;
 pic.FreeImage;
 FreeAndNil(pic);
end;

procedure TForm1.IdHTTPServer1CommandGet(AThread: TIdPeerThread; ARequestInfo: TIdHTTPRequestInfo; AResponseInfo: TIdHTTPResponseInfo);

begin
 AResponseInfo.ContentStream := ScreenToMemoryStream;
 AResponseInfo.WriteContent;
end;

nothing more, nothing less. I tested this out and it gave good performance
One small question though, are your clients webbrowsers or programs?

--------------------------------------
What You See Is What You Get

 

[ajbufort]

whosrdaddy,

I appreciate your comments. As I went through trying different variations on my code, I felt like I might have been getting off course. However, what you have essentially proposed is what I started out with, and its performance with multiple clients was not impressive (these clients, by the way, are web browsers). I basically had a version of the GET method that made all of the method calls in your ScreenToMemoryStream. As each client (browser) would contact the server and make requests simultaneously, performance degraded significantly. Hence, my desperation in trying all sorts of things.

-Tony

 

[whosrdaddy]

Tony,

Making a screenshot and send this one over to a client is reasonably cpu intensive, so I can only imagine that performance will degrade with the number of connections. How do you see that performance degrades (I mean, what's your testmethod).

Cheers

--------------------------------------
What You See Is What You Get

 

[ajbufort]

whosrdaddy,

Unfortunately, the performance is so obviously degraded, that a mere visual inspection of several client browser windows open simultaneously is enough to see it. Open up one browser window pointing to the server, and the screenshots come through at a nice pace. Open up another, and things take a second or two longer. Open up a third, and that seems to almost double, open a fourth, and the system begins to be so delayed in getting the next screenshot as to be annoying.

I had my doubts about this 'screenshot on demand' system, because, as you say, the stuff I am doing is very CPU intensive. There has to be a way to do this somehow, though, right?

-Tony

 

[whosrdaddy]

Open up one browser window pointing to the server, and the screenshots come through at a nice pace

you say screenshots, do you mean they come automatically or are you doing a refresh in the browser??

--------------------------------------
What You See Is What You Get

 

[ajbufort]

whosrdaddy,

I have JavaScript in an HTML page which constantly hits the server via a url like '

http://www.server.com/shot.jpg'.

As soon as the screen loads, the JavaScript replaces the last image displayed. So we are talking about a lot of constant hits here. What I have been trying to determine is whether there was something I was doing to gum up the works, as I know that web servers should be able to handle a lot more load. It's the time the damned screenshots are taking. If I were serving up a static picture, there would be no problem. So I am desperate for a solution which can accommodate all of these screen captures on demand somehow.

-Tony

 

[buho]

Daddy:
Actually he IS running in the thread created by the server object.

His CaptureThread is not started, it is created with delayed execution, Resume is never called and the code is not run from inside the thread Execute procedure, but from that confusing Execute function.

ajbufort:
The only code running in the thread context is the code called from inside the Execute procedure; any thread object method invoked from outside Execute is not running in the thread context but in the caller context.

When calling the Execute function (which is not the Execute procedure needed to run in the thread context, but a simple function), all the code is run in the context of the thread calling the OnGet.

Furthermore, the thread object is never freed, as FreeOnTerminate have no meaning for a thread which never started.

Your CaptureThread is actually not a thread but an unnecesary object which is never freed.

buho (A).

 

[ajbufort]

Okay, buho, thanks for that clarification! I thought I might be screwing things up by making Execute into a function. It's great to have that confirmed.

Now to get back to the code I USED to have - the code which called all of the procedures from within the GET method. You stated earlier that you did not see any serialization issues, but that "Synchronization issues are another beast. If you are blocking your code in a lock and the process is long, then yes: you are introducing serialization in the server requests." I am not doing any such blocking, so I am at a loss to explain why I get such slowdown after so few requests.

-Tony

 

[buho]

My best advise is you throwing your code away, taking a day to clean your head and starting again from scratch.

1) Use an architectural approach like Daddy's code.

2) Do not access visual objects (like status bars, edits, labels and so on) from the OnGet procedure. If you can not avoid doing that, use the due locks and be aware you are introducing serialization.

3) Keep your code at the bare minimum. Get it working efficiently first and add the bells and whistles last.

4) Start transmitting BMPs, so you don't have the JPG conversion overhead. Make the BMPs little (not the whole screen but part of it) so you can have a good data transmission rate. Optimize the code. Only after having a good "get" rate add the JPG conversion code. Optimize the code again. After that, start getting the whole screen. Last at all add the eye candy. This way you can have a good idea about where your code is bottlenecking.

buho (A).

 

[ajbufort]

Hi Guys!

Well, I finally figured out what was going on. Didn't have to alter my code much at all, except for getting rid of the useless thread and putting the procedure calls back in the GET method like I had them originally. Apparently, the problem was this one line (part of my FormCreate procedure, which I never posted):

   SetPriorityClass(GetCurrentProcess, HIGH_PRIORITY_CLASS);

Having the priority of the app set that high apparently messed things up. Putting it back down to NORMAL resulted in behavior much more along the lines of what I was hoping for. Opened up 21 browser clients connected to the server, and there is a tolerable delay between when the first client receives picture and when the final one does. I would still like things to be faster, but I am now at the point where tweaking the app can be done, rather than needing a complete overhaul.

Thank goodness!!

-Tony