INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

Weird intermittent failure with TRTTIContext.FindType

Weird intermittent failure with TRTTIContext.FindType

(OP)
Code snippet:

CODE

uses
  RTTI;

var
  ctx: TRTTIContext;
  rct: TRTTIType;
  cn: String;
begin
  // Lookup value and assign to cn;
  // ... snip ...
  rct := ctx.FindType(cn);
  Assert(Assigned(rct));
end; 

The code above is run via TIdHTTPServer.OnCommandGet, so is running in a separate thread. It works most of the time, but during testing, I noticed these 404 errors come up that I traced back to rct=nil in the code above. Maybe once every 15 times. The thread is launched via a bunch of almost identical ajax calls (in my tests, the code above is being called 3 times almost at once), so I'm thinking that it's some multi-threading issue with the RTTI unit. But my understanding is that it's thread-safe.

Anyone else had trouble like this?

RE: Weird intermittent failure with TRTTIContext.FindType

(OP)
I should have tested further prior to posting. I can confirm that by using a TCriticalSection to prevent multi-threading during FindType, the issue goes away.

RE: Weird intermittent failure with TRTTIContext.FindType

RTTI is known to be threadsafe.
Is that all your code you are showing?

/Daddy

-----------------------------------------------------
Helping people is my job...

RE: Weird intermittent failure with TRTTIContext.FindType

(OP)
There's obviously more around it, but the only change I made, and the code I have now is

CODE

// ... snip ...
FBlock.Acquire;    // TCriticalSection
try
  rct := ctx.FindType(cn);
finally
  FBlock.Release;
end;
// ... snip ... 

And it hasn't faulted during testing. Using Delphi XE6.

RE: Weird intermittent failure with TRTTIContext.FindType

Hi Griffyn,

I see, but you are not answering my question,
RTTI is known to be threadsafe, so ctx.FindType() always works.
That being said, the part where you lookup cn could not be threadsafe and this can lead to a wrong value in cn so that FindType fails.
That's why I asked you to provide all possible details ;)

/Daddy

-----------------------------------------------------
Helping people is my job...

RE: Weird intermittent failure with TRTTIContext.FindType

(OP)
cn is a local variable. There are no class fields being used, all variables are local. In my tests, I added lots of logging, and could see that cn would always have the correct value, but rct would occasionally be nil after FindType.

Full method:

CODE

procedure TNexusMaintenanceLogic.HTTPCatchAllRequestEvent(AIP: String;
  ARequestInfo: TIdHTTPRequestInfo; AResponseInfo: TIdHTTPResponseInfo;
  var AErrorNo: Integer);
var
  up : TUserProfile;
  cn : String;
  fid : Integer;
  ctx : TRTTIContext;
  rct : TRTTIType;
  fpc : TFunctionPackageClass;
begin
  up := TUserProfile.Create;
  try
    with Jet.CreateDBQuery(
      'SELECT F.id, U.validate, F.module_id, F.class ' +
      'FROM   urls U, functions F ' +
      'WHERE  U.function_id=F.id AND (U.url=:URL) AND (f.active=True)') do
      try
        Parameters.ParamByName('URL').Value := ARequestInfo.Document;
        Open;
        First;
        if EOF then
          exit;
        if FieldByName('validate').AsBoolean
          and (not _IsValid(up, FieldByName('module_id').AsInteger, FieldByName('id').AsInteger)) then
        begin
          AErrorNo := 403;
          exit;
        end;
        cn := FieldByName('class').AsString;
        fid := FieldByName('id').AsInteger;
      finally
        Free;
      end;
    FRTTIBlock.Acquire;
    try
      rct := ctx.FindType(cn);      // Removing the TCriticalSection around this line causes issues
    finally
      FRTTIBlock.Release;
    end;
    if (rct <> nil) and (rct is TRTTIInstanceType) then
    begin
      fpc := TFunctionPackageClass(TRTTIInstanceType(rct).MetaclassType);
      with fpc.Create(Jet, Nexus, Ragpip, BluecentralShared, FWeb.FileRoot, fid) do
        try
          UserProfile := up;
          ProcessURL(ARequestInfo, AResponseInfo);
          AErrorNo := AResponseInfo.ResponseNo;
        finally
          Free;
        end;
    end;
  finally
    up.Free;
  end;
end; 

RE: Weird intermittent failure with TRTTIContext.FindType

Well the only reason that FindType would return nil is that the line

CODE -->

cn := FieldByName('class').AsString; 

returns an empty or invalid string.
Adding the CS apparently alleviates the problem but you don't want that because now you are serializing all your HTTP requests.

is Jet.CreateDBQuery() creating a new db connection? if not, you are violating COM marshalling rules.

/Daddy

-----------------------------------------------------
Helping people is my job...

RE: Weird intermittent failure with TRTTIContext.FindType

(OP)
cn is definitely never empty or invalid - logging shows that.

Jet.CreateDBQuery creates a new TADOQuery object and assigns to an existing and connected TADOConnection object. I only ever use one TADOConnection object throughout my service apps, and it remains connected all the time. I do this because it's thread-safe, and I haven't experienced any issues so far. I'm not familiar with COM marshalling rules.

I've created an isolation test you can run yourself. Create a new VCL project, drop a TMemo and a TButton, replace unit1 with below, and assign the Form1.OnCreate, Form1.OnDestroy and Button1.OnClick events. Adding in the Memo1 complicated the logging, so there's a few critical sections to ensure thread safety. The key CS is the GRTTIBlock in TTestThread.Execute. Currently disabled, I get between 3 and 5 failures when I run with 200 threads. Enabling the GRTTIBlock CS removes the failures.

CODE

unit Unit1;

interface

uses
  Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics,
  Vcl.Controls, Vcl.Forms, Vcl.Dialogs, Vcl.StdCtrls, SyncObjs, Contnrs, RTTI;

type
  TTestThread = class(TThread)
  private
    FFailed: Boolean;
    FRan: Boolean;
    FId: Integer;
  protected
    procedure Execute; override;
  public
    property Failed: Boolean read FFailed;
    property Ran: Boolean read FRan;
    property Id: Integer read FId write FId;
  end;

  TForm1 = class(TForm)
    Memo1: TMemo;
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
    procedure FormCreate(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
  private
    FLogBlock: TCriticalSection;
    FThreadBlock: TCriticalSection;
    FMaxThreadCount: Integer;
    FThreadCount: Integer;
    FRanCount: Integer;
    FFailureCount: Integer;
    procedure Log(AStr: String);
    procedure ThreadFinished(Sender: TObject);
    procedure LaunchThreads;
  end;

var
  Form1: TForm1;

implementation

var
  GRTTIBlock: TCriticalSection;

{$R *.dfm}

{ TTestThread }

procedure TTestThread.Execute;
var
  ctx : TRTTIContext;
begin
//  GRTTIBlock.Acquire;
  try
    FFailed := not Assigned(ctx.FindType('Unit1.TForm1'));
    FRan := True;
  finally
//    GRTTIBlock.Release;
  end;
end;

{ TForm1 }

procedure TForm1.Button1Click(Sender: TObject);
begin
  Randomize;
  LaunchThreads;
  Log(Format('Threads: %d, Ran: %d, Failures: %d',
    [FMaxThreadCount, FRanCount, FFailureCount]));
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
  FLogBlock := TCriticalSection.Create;
  FThreadBlock := TCriticalSection.Create;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
  FThreadBlock.Free;
  FLogBlock.Free;
end;

procedure TForm1.Log(AStr: String);
begin
  FLogBlock.Acquire;
  try
    Memo1.Lines.Add(AStr);
  finally
    FLogBlock.Release;
  end;
end;

procedure TForm1.ThreadFinished(Sender: TObject);
var
  tt : TTestThread;
begin
  tt := TTestThread(Sender);
  Log(Format('Thread %d finished', [tt.Id]));
  FThreadBlock.Acquire;
  try
    Dec(FThreadCount);
    if tt.Failed then
      Inc(FFailureCount);
    if tt.Ran then
      Inc(FRanCount);
  finally
    FThreadBlock.Release;
  end;
end;

procedure TForm1.LaunchThreads;
var
  c : Integer;
  ol : TObjectList;
  t : TTestThread;
begin
  FRanCount := 0;
  FFailureCount := 0;
  FMaxThreadCount := 200;
  ol := TObjectList.Create(False);
  try
    // get all the thread objects created and ready
    for c := 1 to FMaxThreadCount do
    begin
      t := TTestThread.Create(True);
      t.FreeOnTerminate := True;
      t.OnTerminate := ThreadFinished;
      t.Id := c;
      ol.Add(t);
    end;
    FThreadCount := FMaxThreadCount;
    // start them all up
    for c := 0 to ol.Count - 1 do
    begin
      TTestThread(ol[c]).Start;
      Log(Format('Thread %d started', [TTestThread(ol[c]).Id]));
    end;
    repeat
      Application.ProcessMessages;
      FThreadBlock.Acquire;
      try
        if FThreadCount <= 0 then
          Break;
      finally
        FThreadBlock.Release;
      end;
    until False;
  finally
    ol.Free;
  end;
end;

initialization
  GRTTIBlock := TCriticalSection.Create;

finalization
  GRTTIBlock.Free;

end. 

RE: Weird intermittent failure with TRTTIContext.FindType

Ok thank you very much for adding an MCVE, will check that out.
Just to inform you about the TADOConnection.
If the Query is not in the same thread as the connection, this will lead to issues and you need to follow COM Marshalling rules across threads.
The solution is simple, keep the connection and the query on the same thread, not abiding this rule will get you into trouble in multithreadingland.

-----------------------------------------------------
Helping people is my job...

RE: Weird intermittent failure with TRTTIContext.FindType

Griffyn,

I ran your test project several times, but it never fails on my PC.
I must admit that I only have Delphi XE, I suppose you have a higher Delphi version?

Small nitpicks on the testproject:

- The FLogBlock CS is not needed since the OnTerminate procedure is synchronized with the main thread (and there is no logging code in the TThread.Execute method)
- You can use a Semaphore in conjunction with WaitForSingleObject for thread bookkeeping.
I will post an example how to this later.

Cheers,
Daddy

-----------------------------------------------------
Helping people is my job...

RE: Weird intermittent failure with TRTTIContext.FindType

(OP)
Thanks for taking a look. I use XE6, so perhaps it's an issue introduced since XE.

As you can tell, my understanding of threads is basic. Particularly waiting for them to finish tasks.

Thanks for the information about queries/connections across threads. I'll investigate that.

RE: Weird intermittent failure with TRTTIContext.FindType

Giffyn,

Do you have an account on Stack Overflow?
I would pose your question over there, Maybe you hit a bug in XE6.
I searched Quality Central entries but found nothing related to your problem.

/Daddy

-----------------------------------------------------
Helping people is my job...

RE: Weird intermittent failure with TRTTIContext.FindType

Sure no problem,
we're here to help each other out :)

/Daddy

-----------------------------------------------------
Helping people is my job...

RE: Weird intermittent failure with TRTTIContext.FindType

(OP)
Looks like the bug has been confirmed by others, and a QC has been raised with Embarcadero. A comment indicated it affects D2010 onwards, so not sure why you didn't experience any failure whosrdaddy.

Did you run my test project as is (with superfluous TCriticalSections), or did you clean it up first? I wonder if the speed and/or number of cores in your PC affected anything.

RE: Weird intermittent failure with TRTTIContext.FindType

Yes,
I have the problem too once I removed all logging, for some reason the CS of the logging block prevented the problem from happening.
I rely heavily on RTTI in a bunch of projects, so I was quite surprised about this bug.
On the bright side, David Heffernan's fix is quite straightforward, so already modified my code base (though I never encountered this problem)

P.S: I have the same username on SO :)

/Daddy

-----------------------------------------------------
Helping people is my job...

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close