Search and replace text in file

[robsuttonjr]

I am using the following to search and replace text. Well it is actually creating a new file but it does what I need. Anyone know of a better way to say search and replace within a file without having to create a new file and not using memo's but directly editing the file?

The following is in a prg which I pass 4 parameters.


LPARAMETERS lcPath, lcFileName, lcSearchFor, lcReplaceWith

oFSO = CreateObject("Scripting.FileSystemObject")

oInFile = oFSO.OpenTextFile(lcPath+lcFileName)
oOutFile = oFSO.OpenTextFile(lcPath+"tmp"+lcFileName,2,.T.)

Do WHILE !oInFile.AtEndOfStream
oOutFile.WriteLine(STRTRAN(oInFile.Readline,lcSearchFor,lcReplaceWith))
ENDDO

oInFile.Close
oOutFile.Close

Forums rule, pass it on!!!

Rob

 

[mspratt]

Hi Rob,

WOW!

Why not STRTOFILE() AND FILETOSTR() combined with STRTRAN()?

Regards,

Mike

 

[robsuttonjr]

The file is 100 MEG so we need the fastest choice. The best choice is to edit the file without creating a new one.

Forums rule, pass it on!!!

Rob

 

[rgbean]

Because you aren't sure that you start and stop with the same number of characters in the file, then you are going to have to work with two files.

How you tried low-level file IO? It's probably faster than WSH, due to lower overhead.

Rick

 

[robsuttonjr]

Let me back up a bit. The reason we are having to edit the .CSV file is that sometimes extra " are put in the file:
"test",""someone@aol.com"",""

In the above example the email address has 2 double quotes instead of 1. I am working on a custom code that will import .CSV file and not pass the " into the cursor.

Forums rule, pass it on!!!

Rob

 

[TamarGranor]

I think you'll be surprised how quickly VFP can do text processing on a large file. You should at least experiment with FileToString() and StrTran().

Tamar

 

[robsuttonjr]

what would really be nice is if when doing in append from a file I could strtran() characters when doing the append. So lets say I am appending from .CSV and the file has these extra "" I could append from DELIMITED WITH ',' and somehow filter out all of the ". This is what I am working on but its tough. Using filetostring then strtran takes too much memory and if very slow when working with 100 MEG files.

Forums rule, pass it on!!!

Rob

 

[mspratt]

Hi Rob,

What you are doing with WSH uses two files.
Ho long does this take?

lcMyString = FILETOSTR("MyFile.csv")
STRTRAN(lcMyString, ',""', ',"') && Note that I use ' to be
STRTRAN(lcMyString, '"",', '",') && able to use " in the string
STRTOFILE(lcMyString, "MyFile.csv")

You may also want to increase the computer's RAM.

Regards,

Mike

 

[robsuttonjr]

Yes I have tried this idea also. What would be nice is to figure out how to use the same methods that are used in the FoxPro command APPEND FROM. Has anyone figured out how to view the code behind this command?

Forums rule, pass it on!!!

Rob

 

[mspratt]

Hi Rob,

How long did it take?

Try APPEND FROM myFile.csv DELIMITED WITH ",". To include quotes (double or single) usually requires the quotes be surrounded with the other type quotes. Perhaps it will ignore the extra double quotes.

Regards,

Mike

 

[robsuttonjr]

The filetostr and strtran version takes much longer. The quickest method thus far is to use FGETS and FPUTS. Basically to copy and paste into new file. Using this method it takes around 14 seconds. Other methods are 30-60 seconds or more.

Forums rule, pass it on!!!

Rob

 

[mspratt]

Hi Rob,

How about if you lose the double STRAN()

WAIT WINDOW "LOADING" NOWAIT
lcMyString = FILETOSTR("MyFile.csv")
WAIT WINDOW "TRAMSLATING" NOWAIT
STRTRAN(lcMyString, '""', '"')
WAIT WINDOW "SAVING" NOWAIT
STRTOFILE(lcMyString, "MyFile.csv")

Trouble with VFP idenifing Character fields?

Regards,

Mike