Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations strongm on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

.readline() crashes while processing text file

Status
Not open for further replies.

phadobas

Technical User
Jul 30, 2005
607
0
16
US
I have systems that generate log-files daily and it's full of useless data. I wrote a python script that weeds out what I need. Based on user input, it goes through files, examines the file line by line (readline()) and if it matches certain criteria, the given line is then written into a separate text file.
However, there appears to be a certain type of character that crashes the program.
The error message I get is

UnicodeDecodeError: 'utf16' codec can't decode bytes in position 70-71: unexpected end of data

When I open the file with Notepad ++, I see the attached.

When I use the readline() with Try:, then the program doesn't crash, it actually bypasses some of these errors, but most of the time it hangs.

Currently the only way I can use this program is to run it, see where it crashes, manually remove the pictured characters, and run the program again, and when it crashes, manually... etc. etc, etc. Very time-consuming...

Any idea?
 
 http://files.engineering.com/getfile.aspx?folder=cfa91d20-cb5d-4380-afcb-e9285031f558&file=Text.JPG
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top