reading large text files

[Slomo8]

I am using the FileSystemObject to read from a text file and write to another text file. I am using the read method of the Textstream object for optimal performance. The program works for files up to 4 megabytes but does not work for files larger than this. How can I overcome this? thanks in advance. the code:


Sub Prgm_Parse8

const ForReading = 1
const TristateFalse = 0
dim sName
dim outputFile
dim fso
dim objFile
dim objTS
set fso = CreateObject("Scripting.FileSystemObject&quot
sName = "W:\Copy of 2L14_9C016_B.wrl"
set objFile = fso.GetFile(sName)
set objTS = objFile.OpenAsTextStream(ForReading, TristateFalse)

'***********-----
contents = objTS.Read(objFile.Size)

dim fFile, contents, aFile, iLine, iLines, sLine, sWord

'loading file into a string
set fFile = fspenTextFile( sName, 1 ) '1 = ForReading
'fFile.Close

'splitting string into an array
aFile = Split( contents, vbCrLf )

'scanning lines
iLines = UBound( aFile ) + 1
iLine = 0
do while iLine < iLines
sLine = aFile( iLine )

dim c
c=Left(sLine,1)
While c=Chr(9) or c=&quot; &quot;
sLine=Mid(sLine,2)
c=Left(sLine,1)
Wend
sWord=Left(sLine,3)

if ( sWord = &quot;DEF&quot; ) then

str1 = left(sLine, 5)
str2 = mid(sLine, 7)
str3 = UCase(Mid(sLine, 6, 1))

dim check_for_num
check_for_num = IsNumeric(str3)

if (check_for_num = true) then
sLine = str1 + &quot;X&quot; + str2
else
sLine = str1 + str3 + str2
end if

dim position

position = instr(sLine, (&quot;Separator&quot)
position = (position - 5)
sLine = mid(sLine, 5, position)
sLine = CleanString( sLine, Chr(34) & &quot;/,.*-&:&quot; & &quot; &quot
str1 = left(str1, 4)
sLine = str1 + sLine + &quot; Separator {&quot;
end if

if ( sWord = &quot;USE&quot; ) then

str1 = left(sLine, 5)
str2 = mid(sLine, 7)
str3 = UCase(Mid(sLine, 6, 1))

'dim check_for_num
check_for_num = IsNumeric(str3)

if (check_for_num = true) then
sLine = str1 + &quot;X&quot; str2

else
sLine = str1 + str3 + str2

end if

sLine = mid(sLine, 5)
sLine = CleanString( sLine, Chr(34) & &quot;/,.*-&:&quot; & &quot; &quot
str1 = left(str1, 4)
sLine = str1 + sLine
end if

'- put it back in array with
aFile( iLine ) = sLine

iLine = iLine + 1
loop

'rejoining array items into a string, writing to output file
contents = Join( aFile, vbCrLf )

set fFile = fso.CreateTextFile( sName, True )
fFile.Write contents

fFile.Close

end sub

function CleanString( sInput, sSymbols )

dim sClean
sClean = &quot;&quot;

dim l, i, c
l = len( sInput )
i = 1

'- while more input string
do while i <= l

'- snip out a character
c = mid( sInput, i, 1 )

'- if not a symbol, add to cleaned string
if InStr( sSymbols, c ) = 0 then
sClean = sClean & c
end if

i = i + 1
loop

CleanString = sClean
end function

 

[ClayG]

Since you are splitting the file into lines anyway, would it really be slower to read it line-by-line in the first place ?

 

[Slomo8]

how can I scan the first word of each line without splitting it into lines?

 

[mrmovie]

strLine = Trim(fFile.ReadLine)
'then split you line into words using a &quot; &quot; as the delimiter

 

[Slomo8]

what are the limits on the number of characters an array and string can hold?