Really strange Out of Memory error

Kalisto · Sep 15, 2003

I am playing about trying to catalogue the files on a hard disk (It just seemed a reasonable project to do, it will make me think on recursion, file access, and using some MFC features)

Below is a small fragment of code. The last line of the code is the point at which my system crashes, and throws an out of memory exception.

BUT

Through a form, the user can specify at which point (folder) that the cataloguing starts from). For some folders, the new statement throws, for others it doesn't.

The path to get to the folder doesn't seem to be at issue.
The exception is always thrown for these folders. If the loop had begun processing I could maybe understand why I was getting an out of error message. But it hasn't, all it has done is try to allocate some memory for a pointer to a structure (the structure is 248 bytes in fact)

Can anyone explain to me why this is, or even just give me some more ideas as to where I can look to begin to identify what is causing this ?

Thanks

K

Code:

//Find the First Entry in this Folder, be it a Directory or a File
   HANDLE hFind = FindFirstFile(lpFind,&lpData);

   //Check Handle to see if an Error
   if (hFind == INVALID_HANDLE_VALUE) 
   { 
	return false;
   } 

//Find First Removes the '.' directory
//This FindNext removes the '..' directory
BOOL blNextFile = FindNextFile(hFind, &lpData);
	
//Look at the First Valid File in the Directory
blNextFile = FindNextFile(hFind, &lpData);
	
//If 0 then an Error - Throw the relevant Exception
if (!blNextFile)
	DWORD dwError = GetLastError();

while(blNextFile)
{
	//Create a File Header Object in Memory
	sFileRecord* sFR = new sFileRecord;

PerFnurt · Sep 15, 2003

>Can anyone explain to me why this is, or even just give me some more ideas as to where I can look to begin to identify what is causing this ?

1) Is blNextFile ever assigned false?

2) Can you guarantee that the first 2 files are . and ..?
(just to be sure I would make thet check inside the while)

/Per
_{if (typos) cout << "My fingers are faster than my brain. Sorry for the typos.";}

Cagliostro · Sep 15, 2003

you should read only at most two levels of directory nesting. If you want more, do it only when you need.

Ion Filipski

ICQ: 95034075
AIM: IonFilipski
filipski@excite.com

Kalisto · Sep 15, 2003

They are things worth checking, but that doesn't explain why sometimes I have mo memory when I get to the new statement, unless a '.' or '..' directory handle eats memory ?

K

Cagliostro · Sep 15, 2003

in addition to '..' and '.' is '/' or '\'

Ion Filipski

ICQ: 95034075
AIM: IonFilipski
filipski@excite.com

adholioshake · Sep 15, 2003

I think PerFnurt is right. From the code given, blNextFile is never false, which would lead to a definate memory allocation problem. Secondly, it is best to check for the '.' and '..' directories by name as I think they are not always be the first two directories. If you have started any recursion, this might be where the problem lies. For example, if there is a '.' or '..' directory in the recusion unchecked it could lead to an infinate loop of recursions, again wasting a lot of memory.

I have wrote a program (in just plain C) which does exactly what you need and will post the code if you like.

It might be useful to you.

Kalisto · Sep 15, 2003

I have been through it with the debugger, and it dies, only on certain folders, on the first time it tries to allocate space for a sFileRecord.

Everything you have mentioned is definitely something I need to consider.

Ion, why should I only ever read 2 levels of nesting ?

And the blNextFile does go false, I just omitted the rest of the while loop as the code doesn't get far enough to be affected by that.

Thanks so far

Cagliostro · Sep 15, 2003

because
1. there could be any nesting levels, and you do not know how many they are. If you try to read for example the c:\windows or c:\winnt by you will not nave enough memory at all, however your computer could eb very powerful.
2. usualy you will use recursive algorithms, which are stack expensive.
3. You do not need usualy to see all the levels. You will not be able just to see all levels however you will be able to show all of them. Your visual capacity is limited.
4. If you want to see some more information than two levels, you will do it only for one of nested directories. In this case, you will start your algorithm again for that directory. For example when you click on a tree item which represents a directory, you will begin to read directory and add tree subitems.
5. The best approach is to read once nested level. All the erst of nesting you will read dinamically then needed.

Ion Filipski

ICQ: 95034075
AIM: IonFilipski
filipski@excite.com

Cagliostro · Sep 15, 2003

In addition, maybe you know that:
. means current directory
.. means parent directory
/ or \ means root directory

Ion Filipski

ICQ: 95034075
AIM: IonFilipski
filipski@excite.com

Kalisto · Sep 16, 2003

OK, that again is worth bearing in mind later on, at the present moment the code doesn't walk the directory tree, it just creates a vector of sFileHeaders. One of the folders that is causing the crash contains 3 files, so I can't see where the memory is going.

It looks like I need to add some code to check the amount of memory thats available and dump it to my logfile just before I call the new command.

Kalisto · Sep 17, 2003

OK, I have identified what is causing my code to go wrong, but I don't understand why. If my final directory that I am investigating is > 6 characters in length then the system doesn't like it.

I don't understand this, as I can have a long path to the last directory and it is happy with that, it just seems that it is the last folder that is important.

Having identified that, I can work round it until I can find a solution, which has caused me a seperate error. Now I get an unhandled exception in Malloc. I have put a section of the code below, to see if any of you can spot what I am doing wrong (or if you can spot any improvements, I am always open to suggestions)

Code:

bool PackFile::GenerateHFT()
{
	//generate Random Number for the Key Value and for a + or -
	int scratch; 
	int nMod=0;
	int nValue;
	LPCSTR lpFind;
	WIN32_FIND_DATA lpData;
	scratch = rand();
	nMod = (float)(scratch/RAND_MAX) *2;
	scratch=rand();
	nValue = ((float)scratch/RAND_MAX) * 64;

	//Create a wide string for some of the output
	std::wstring strOut;
	//Access the Log File Singleton
	CLogger* pLF = CLogger::Instance();

	//Destroy the Data Vector
	FileRecords.clear();

	//Generate a Unique 10 char array
	for (int i = 0;i<10;i++)
	{
		scratch=rand();
		sHeader.szUniqueID[i]=_T((48+((float)scratch/RAND_MAX) * 74));
	}
	//Update the Header Structure Properly
	sHeader.bCipherAddition = nMod;
	sHeader.iCipherValue = nValue;
	strOut = _T(sHeader.szUniqueID);
	*pLF << L&quot;Unique ID = &quot; << strOut << L&quot;\n&quot;;
	if (sHeader.bCipherAddition)
		*pLF << L&quot;Additive Key - &quot;;
	else
		*pLF << L&quot;Subtractive Key - &quot;;
	
	short temp = sHeader.iCipherValue; 
	*pLF  << temp << L&quot;\n&quot;;

	//Parse through the Directory 
	// as *.* is used, this code will also get directories.
	wchar_t* szFindPath = new wchar_t;
	wcscpy(szFindPath,szFolderPath);
	wcscat(szFindPath,L&quot;\\*.*&quot;);
	
	//Convert Char_W Array into an LPSTR for find access
	lpFind = CW2A(szFindPath);

	//Find the First Entry in this Folder
	HANDLE hFind = FindFirstFile(lpFind,&lpData);

	//Check Handle to see if an Error
	if (hFind == INVALID_HANDLE_VALUE) 
	{ 
		*pLF << &quot;Invalid Folder\n&quot;;
		pLF = NULL;
		delete pLF;
		return false;
	} 

	strOut = L&quot;First Entry = &quot;;
	std::string strOut2 = lpData.cFileName;
        *pLF << strOut;	//This Point Here causes Unhandled Exception in Malloc
        *pLF << strOut2 << &quot;\n&quot;;

Thanks for all your help so far,

K

Kalisto · Sep 17, 2003

Damn, the fault is in this funciton here, where I am opening my File Stream.

OpenLogFile Opens and sets the exceptions for a wofstream.

Code:

CLogger& CLogger::operator<<(const std::wstring T)
{

std::wofstream* fs = new std::wofstream;
if (!fs->is_open())
	OpenLogFile(fs);
try
{
	*fs << T;
	fs->flush();
}
catch(std::ios::failure)
{
	printf (&quot;Exception Thrown.\nCannot Find File %s\n&quot;,strFname.c_str());
}
return *_CLogger;
}

Cagliostro · Sep 17, 2003

Is

1.
std::string strOut2 = lpData.cFileName;
*pLF << strOut; //This Point Here causes
CLogger& CLogger:

perator<<(const std::wstring T)
//do you pass a std::string where is required std::wstring?
{

2.
std::wofstream* fs = new std::wofstream;
if (!fs->is_open())//what do you open there?
OpenLogFile(fs);

Ion Filipski

ICQ: 95034075
AIM: IonFilipski
filipski@excite.com

Kalisto · Sep 17, 2003

1. Sorry, no, I definitely pass a wstring. (I have 2 operator<<, 1 takes a std::string, and 1 takes a std::wstring. I have been thru the debugger to see which is called.

2.
void CLogger::OpenLogFile(std::wofstream* fs)
{
try
{
fs->open(strFname.c_str),std::ios:

ut|std::ios::app);
}
catch(std::ios::failure)
{
printf ("Exception Thrown.\nCannot Find File %s\n",strFname.c_str());
}

}

This is so frustrating, and now I know that I am starting to hack at my code to get it working, and so if I am not careful I'll lose all my structure.

K

Kalisto · Sep 17, 2003

OK, this thread is getting too long, and the problem has eveolved, I'll start a new one with a more specific title

Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

Really strange Out of Memory error

Kalisto

Programmer

PerFnurt

Programmer

Cagliostro

Programmer

Kalisto

Programmer

Cagliostro

Programmer

adholioshake

Programmer

Kalisto

Programmer

Cagliostro

Programmer

Cagliostro

Programmer

Kalisto

Programmer

Kalisto

Programmer

Kalisto

Programmer

Cagliostro

Programmer

Kalisto

Programmer

Kalisto

Programmer

Similar threads

Part and Inventory Search

Sponsor