Threads - a little confused on this one....

[golcarlad]

Hi folks...

I am a bit new to the concept of threads.

I have a block of code that does various tasks and takes about 15 hours to complete - I need it to run 3 times - so total 45 hours to do whole thing - is there not a way to use 3 threads to run this block of code simultaneously?, bearing in mind the server the code will run on has 4 processors - it would be so cool to be able to allocate each processor each thread to run - thefore my total running time would be cut down dramatically...if you get my meaning.

However (having said all this) I suspect there is a problem if 2 or more threads are accessing the same variables code, am I right?, if so - I dont get the point of using threads?? Can anyone shed some light on this for me?

Cheers guys...

 

[jwarmuth]

Read up on Lock, Monitor, and Mutex. Those are all methods of protecting data from alteration from other threads during execution.

Jeff W.
MCSE, CNE

 

[golcarlad]

ok, thanks for that, but from what I have said do you think its possible to have 3 threads running concurrently, and for them to be allocated a processor each or something by the OS?

 

[chiph]

It depends on what the threads need to do.
If they're all accessing the same information, then you'll have locking issues. Use the lock keyword, as well as Mutex and WaitHandles.

If any of the three processes depend on information produced by a previous process, then threading would not solve your problem.

Threading is best described in terms of throughput, and not performance.

Chip H.


____________________________________________________________________
If you want to get the best response to a question, please read FAQ222-2244 first

 

[golcarlad]

None of the 3 processes depend on each other, they are quite seperate - the fact is object is really a big method that iterates through a loop, thousands of times and does stuff, its end result is to dump a CSV file.

I was looking at MethodImplAttribute or Lock to lock the method - but I dont see how another thread can access this method (because its locked) and kick off its process - starting to wonder whether it would be better to use Process.???

 

[chrissie1]

If you run one process does it consume all the processortime of all the processors?

What OS are you using? Can and will it use all the processors independently?

Best advice: TRY IT.



Christiaan Baes
Belgium

"My new site" - Me

 

[golcarlad]

I think, from what I have read - threads can be used to execute 2 methods at roughly the same time. But its not possile with just 1 method.

The program on a single processor PC - uses 100%, on a 4 processor server it uses 25% of 1 processor - so there is much power not even touched. I need to know how to harness this dormant power.

I am using Win2000 in test environment, and Win2k3 Server on the server (live environment).

 

[chrissie1]

I think you will need to test it on the server to see if it helps. Since the threads on the 1 processor machine are't really executed at the same time. It just gives the control to one thread at a time and then switches it does this several times a second so that you get the impression it is mutlitasking but really it isn't. On a 4 processor system the OS could divide the work load over the 4 processors and do all the threads at once with a proessor to spare. But it depends on the OS not your program (except perhaps with some lower level programming languages). I think win2003 server should be able to do this.

Christiaan Baes
Belgium

"My new site" - Me

 

[golcarlad]

Would it be best to have 2 threads doing the same code block - and use the locking technique on it?

 

[chrissie1]

Depends on the code you want to execute. It is diffcult to form an opinion about something you haven't seen. And optimization is all in the details and testing.

Christiaan Baes
Belgium

"My new site" - Me

 

[golcarlad]

OK - here is the code that I want to put in a thread - I know its really long, but the content is not so important, its just to give an idea of some of the things I am doing, bear in mind that most of it calls 3rd party libraries.

public class MainOps
	{
		public CGridInfo oGrid;
		public DataManagement oDataMan;
		public Layers oLayers;
		public string sInitWorkingDir = ConfigurationSettings.AppSettings["initialWorkingDir"];
		public string sInputDir = ConfigurationSettings.AppSettings["inputDir"];
		public string sClutterPtsShpFileName = ConfigurationSettings.AppSettings["clutterPoints"];


		public MainOps(CGridInfo objGrid, DataManagement objDataMan, Layers objLayers)
		{
			oGrid = objGrid;
			oDataMan = objDataMan;
			oLayers = objLayers;
		}
			
		
		public void InitialProcess()
		{
			IFeatureClass pRasToPolyFeatClass;
			IFeatureClass pMeanPointFeatClass;
			IFeatureClass pClutterFeatClass;
			Tables2 oTables2 = new Tables2();
			OutputInfo winOutputInfo = new OutputInfo();
			SpatialOps oSpatialOps = new SpatialOps();
			IRaster pGrid;
			int[] iaUniqueCellIds;
			IFeatureCursor pPolyFeatCursor;
			ArrayList alPointFeatures = new ArrayList();
			IPoint pWeightedPoint = null;
			SpatialOps.Threshold pThesh = new CellCentroid.SpatialOps.Threshold();
			IPoint pWGS84point = new PointClass();
			string sCellName = null;
			string sRNC = null;
			string sBSC = null;
				
			try
			{	
				//Copy from network a copy of the latest UMTS raster into working directory
				oLayers.CopyRaster(oGrid.CurrentVersionLocation, oGrid.CurrentVersionName, sInitWorkingDir,
					oGrid.CurrentVersionName);

				winOutputInfo.lblStartTime.Text = DateTime.Now.ToShortTimeString();
				winOutputInfo.lblBearerName.Text = oGrid.CurrentVersionName;
				winOutputInfo.Show();
				winOutputInfo.Refresh();
				pGrid = oLayers.OpenRaster(sInitWorkingDir, oGrid.CurrentVersionName);
			
				//Convert raster to shapefile
				oLayers.DeleteExistingShapefile(sInitWorkingDir, "RasToPoly");
				oDataMan.ConvertRasterToPolygon2(sInitWorkingDir + @"\" + oGrid.CurrentVersionName,
					sInitWorkingDir + @"\rastopoly");
	
				//Get unique values from the shapefile - which are the cell_id's
				pRasToPolyFeatClass = oLayers.OpenFeatureClassShapeFileWithNameObjects(sInitWorkingDir, "RasToPoly");
				oLayers.ApplySpatialReference(pRasToPolyFeatClass, 27700);
				iaUniqueCellIds = oTables2.GetUniqueValuesFromShapefile(pRasToPolyFeatClass, "GRIDCODE");
				

				//Create Meanpoint shapefile (empty) for storage of mean points	
				pMeanPointFeatClass = oLayers.CreateShapefile(sInitWorkingDir, oGrid.MeanPointShapfileName, 27700,
					esriGeometryType.esriGeometryPoint);
				oDataMan.CreateMeanPointShapefile(pMeanPointFeatClass);
				int iEastingIndex = pMeanPointFeatClass.Fields.FindField("EASTING");
				int iNorthingIndex = pMeanPointFeatClass.Fields.FindField("NORTHING");
				int i70Index = pMeanPointFeatClass.Fields.FindField("70");
				int i80Index = pMeanPointFeatClass.Fields.FindField("80");
				int i90Index = pMeanPointFeatClass.Fields.FindField("90");
				int iLatIndex = pMeanPointFeatClass.Fields.FindField("LAT");
				int iLongIndex = pMeanPointFeatClass.Fields.FindField("LONG");
				int iCellNameIndex = pMeanPointFeatClass.Fields.FindField("CELLNAME");
				int iCellIdIndex = pMeanPointFeatClass.Fields.FindField("CELLID");
				int iBscRncIndex = pMeanPointFeatClass.Fields.FindField("BSC_RNC");


				//Set the spatial reference for projecting each new point
				ISpatialReferenceFactory2 pSpRefFactory = new SpatialReferenceEnvironmentClass();
				ISpatialReference pSpRef = pSpRefFactory.CreateSpatialReference(27700);


				pClutterFeatClass = oLayers.OpenFeatureClassShapeFileWithNameObjects(sInputDir, sClutterPtsShpFileName);
				//Get raster for query
				IRaster pQueryRaster = oLayers.OpenRaster(sInitWorkingDir, oGrid.CurrentVersionName);

				//----------------START OF MAIN LOOP------------------//
				for (int i = 0; i < iaUniqueCellIds.Length; i++)
				{
					DateTime oCurrDateTime;
					TimeSpan runlen;
					oCurrDateTime = DateTime.Now;


					//Get polygon set per unique ID
					pPolyFeatCursor = oSpatialOps.MakeSelectionFromShp(pRasToPolyFeatClass, "GRIDCODE = " + iaUniqueCellIds[i]);
					
					//Get point array per polygon set
					alPointFeatures = oSpatialOps.ClipPoints(pPolyFeatCursor, pClutterFeatClass);
					if (alPointFeatures.Count == 0) continue;
					
					//Get average wieghted point from point array
					pWeightedPoint = oSpatialOps.MakeWeightedMeanPoint(alPointFeatures);

					//Make space for point and store in shapefile
					IFeature pWeightedFeature = pMeanPointFeatClass.CreateFeature();
					pWeightedFeature.Shape = pWeightedPoint;
					pWeightedPoint.Project(pSpRef);

					//Store rest of attributes
					pWeightedFeature.set_Value(iEastingIndex, pWeightedPoint.X);
					pWeightedFeature.set_Value(iNorthingIndex, pWeightedPoint.Y);

					//Calculate thresholds
					pThesh = oSpatialOps.CalculateThresholds(pWeightedPoint, alPointFeatures);

					//Create array to store 70, 80, 90 thresholds relate to 0,1,2 in array respectively
					pWeightedFeature.set_Value(pMeanPointFeatClass.Fields.FindField("70"), pThesh.SeventyPercent);
					pWeightedFeature.set_Value(pMeanPointFeatClass.Fields.FindField("80"), pThesh.EightyPercent);
					pWeightedFeature.set_Value(pMeanPointFeatClass.Fields.FindField("90"), pThesh.NinetyPercent);
					
					//Make long and lat values in table
					pWGS84point.PutCoords(pWeightedPoint.X, pWeightedPoint.Y);
					pWGS84point = MainGeoTasks.ConvertPointToWGS84(pWGS84point);
					pWeightedFeature.set_Value(iLongIndex, pWGS84point.X);
					pWeightedFeature.set_Value(iLatIndex, pWGS84point.Y);

					//Insert Cell_old value in meanpoint.shp
					sCellName = RasterOps.QueryRasterBySQL(pQueryRaster, "Cell_Old", "cell_id = " + Convert.ToString(iaUniqueCellIds[i]));
					if (sCellName != "-9999") pWeightedFeature.set_Value(iCellNameIndex, sCellName);
				
					//Insert CellId value in meanpoint.shp
					pWeightedFeature.set_Value(iCellIdIndex, Convert.ToString(iaUniqueCellIds[i]));

					//Need to switch query for either non-3G (BSC), or 3G (RNC)
					if (oGrid.MeanPointShapfileName == "meanpointUMTS")
					{
						sRNC = RasterOps.QueryRasterBySQL(pQueryRaster, "Rnc", "cell_id = " + Convert.ToString(iaUniqueCellIds[i]));
						if (sRNC != "-9999") pWeightedFeature.set_Value(iBscRncIndex, sRNC);
					}
					else
					{
						sBSC = RasterOps.QueryRasterBySQL(pQueryRaster, "Bsc", "cell_id = " + Convert.ToString(iaUniqueCellIds[i]));
						if (sBSC != "-9999") pWeightedFeature.set_Value(iBscRncIndex, sBSC);
					}
					
					//Commit the point in the shapefile
					pWeightedFeature.Store();
					GC.Collect();

					runlen = DateTime.Now - oCurrDateTime;					  
					winOutputInfo.lblOutput.Text = i + @"/" + iaUniqueCellIds.Length;
					winOutputInfo.txtOutput.Text = "Finished Cell ID " + iaUniqueCellIds[i] +
						" in " + runlen.TotalSeconds.ToString() + " seconds";
					winOutputInfo.Refresh();
				}
			}
			catch (Exception e)
			{
				MessageBox.Show(e.Message + "\n" + e.InnerException + "\n" + e.StackTrace);
				winOutputInfo.Close();
				Environment.Exit(0);
			}
			winOutputInfo.Close();

		}

 

[chiph]

//Copy from network a copy of the latest UMTS raster into working directory
                oLayers.CopyRaster(oGrid.CurrentVersionLocation, oGrid.CurrentVersionName, sInitWorkingDir,
                    oGrid.CurrentVersionName);

OK, here it appears that you're doing a file operation over the network. This is inherently thread-unsafe, so you'll want to surround this code with a lock (as well as a try-catch block, which may already be in your CopyRaster method)

You might want to pre-load this info if possible.

Chip H.


____________________________________________________________________
If you want to get the best response to a question, please read FAQ222-2244 first

 

[golcarlad]

Hmmm, well if that is thread unsafe, Im betting that alot of the other file ops that I am doing will also be unsafe, I think in the long term it will be easier to not implement threads - and just run each process in sequence. But thanks to everyone for taking their time out to help me with this (I think a bit tricky) concept. I have learned some things about threads that I didnt know before, so that can only be a good thing.

 

[chrissie1]

it will be easier to not implement threads

Always.

Christiaan Baes
Belgium

"My new site" - Me

 

[chiph]

Yeah, you generally have to design for threads, and it's not easy to add them in after the fact.

Chip H.


____________________________________________________________________
If you want to get the best response to a question, please read FAQ222-2244 first

 

[golcarlad]

Handy knowing - I definately did not design that code with threads in mind, but I can see you how you would definately have to, it requres a different design structure.

By the way do think that code is too long to be in Main, or should it be broken down even more, sometimes I dont know whether its worth it because your code becomes destructured.

Anyway cheers guys....

 

[chrissie1]

Just one comment. Putting it in one gigantic try catch block is not considered good practice.

and doing this

winOutputInfo.lblOutput.Text = i + @"/" + iaUniqueCellIds.Length;

is also considered thread unsafe since the form will be runnning on another thread.

Christiaan Baes
Belgium

"My new site" - Me

 

[chiph]

By the way do think that code is too long to be in Main, or should it be broken down even more, sometimes I dont know whether its worth it because your code becomes destructured.

Sometimes it helps to put your "maintainer" hat on and look at your code from someone's viewpoint that is having to go in and make changes sometime in the future. Since 80% of software cost goes to code maintenance and not the initial development, anything you can do to make it more maintainable is a win.

Chip H.


____________________________________________________________________
If you want to get the best response to a question, please read FAQ222-2244 first