Need to delete blank lines from text files

[sirnose1]

I have as txt file that has blank lines. I need to remove the blank lines the function I made:

Function DeleteBlankLines()

Dim blkLines() As String
Dim fileName As String = Me.txtLoadFile.Text

Try
Dim sd As New StreamReader(fileName)
Dim fileText As String = sd.ReadToEnd()
sd.Close()
blkLines = Split(fileText, vbCrLf)
'Return
'this path is for testing
Dim fs As New FileStream("c:\test1.txt", FileMode.Create)
Dim sw As New StreamWriter(fs)
Dim foundBlank As Boolean
Dim blkLineCtr As Integer = 0

For Each line As String In blkLines
If line.Length > 0 Then
sw.WriteLine(line)
' Reset blank line flag
foundBlank = False
Else
If Not foundBlank Then
' Blank line: write first one
sw.WriteLine(line)
' Set flag to indicate that blank line was found
foundBlank = True
blkLineCtr = blkLineCtr + 1
End If
End If
Next
If blkLineCtr > 0 Then
File.Delete("c:\test1.txt")
DeleteBlankLines()
End If
blkLineCtr = 0
sw.Close()
fs.Close()

Catch ex As Exception
MessageBox.Show(ex.Message)
End Try

End Function

Here is the code that calls that function:
Private Sub btnProcess_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnProcess.Click
Dim EID, EID_2 As String
Dim GID, GID_2 As String
Dim IsParent, IsParent_2 As String
Dim MD5, MD5_2 As String
Dim lineString As String = ""
Dim lineString2 As String = ""
Dim dlmtr As String = ","
Dim fields() As String
Dim fields2() As String
Dim sr As StreamReader
Dim rdr As StreamReader
Dim MD5_Log As StreamWriter
Dim error_log As StreamWriter
Dim loadFile As String = Me.txtLoadFile.Text
Dim folder As String = Me.txtOutputFldr.Text

'Dim temp As String


If Me.txtLoadFile.Text <> "" And Me.txtOutputFldr.Text <> "" Then
Try
DeleteBlankLines()

sr = New StreamReader(loadFile)
MD5_Log = New StreamWriter(folder & "\" & "MD5_log.txt", False)
error_log = New StreamWriter(folder & "\" & "error_log.txt", False)
MD5_Log.AutoFlush = True
error_log.AutoFlush = True
Me.StatusStrip1.Text = "Processing File...................."
While sr.Peek() <> -1
lineString = sr.ReadLine()
If lineString = "" Then
' deal with empty line
Else
fields = lineString.Split(dlmtr)
If fields.Length <> 4 Then
error_log.WriteLine(lineString & "|" & "Error:Field Length <> 4")
End If

EID = fields(0)
GID = fields(1)
IsParent = fields(2)
MD5 = fields(3)

If IsParent <> "YES" Or IsParent = "" Then
error_log.WriteLine("Help")
Else
MD5_Log.WriteLine(EID & "," & GID & "," & IsParent & "," & MD5)
End If

rdr = New StreamReader(loadFile)
While rdr.Peek() <> -1
lineString2 = rdr.ReadLine()
If lineString2 <> "" Then
fields2 = lineString2.Split(dlmtr)
If fields2.Length <> 4 Then
error_log.WriteLine(lineString & "|" & "Error:Field Length <> 4")
GoTo errorskip
End If
lineString2 = rdr.ReadLine()
fields2 = lineString2.Split(dlmtr)
EID_2 = fields2(0)
GID_2 = fields2(1)
IsParent_2 = fields2(2)
MD5_2 = fields2(3)
Else
error_log.WriteLine("Empty Field")
GoTo errorskip
End If

If (IsParent_2 <> "YES") And (GID_2 = GID) And (MD5_2 <> MD5) Then
MD5_Log.WriteLine(EID_2 & "," & GID_2 & "," & IsParent_2 & "," & MD5)
End If
errorskip:
End While
rdr.Close()
rdr = Nothing
End If
End While
'errorskip:
Catch ex As Exception
'System.Diagnostics.Debug.WriteLine(ex.Message)
MessageBox.Show(ex.Message)
Finally
Me.StatusStrip1.Text = "Ready"
MessageBox.Show("Complete")
End Try
Else
If Me.txtLoadFile.Text = "" Then
MessageBox.Show("Please enter the load file")
Me.btnLoadFile.Focus()
ElseIf Me.txtOutputFldr.Text = "" Then
MessageBox.Show("Please select an output directory")
Me.btnOutputFldr.Focus()
End If
End If
End Sub

Is supposed to remove the blank lines. Unfortunatey sometimes there may be as many as 5 blank lines. I have tried everything. Any suggestions or help?

 

[petermeachem]

Seems complicated. Can you replace carriage return, line feed pairs before you split the file? You would need to do this multiple times. Or could you replace line feed / carriage returns pairs? This should remove all of them in one go and just leave the first carriage return and last line feed in mutiples and ignore single instances

 

[RiverGuy]

I'm confused on the ' Blank line: write first one portion of your code. What does that mean? Anyways, this routine will take in a file_name and remove all of the blank lines. You should be able to adapt it to what you need:

    Private Sub DeleteBlankLines(ByVal file_name As String)
        Dim sr As New System.IO.StreamReader(file_name)
        Dim sw As New System.IO.StreamWriter(file_name + "_temp", False)
        Dim s As String
        Dim FoundBlankLines As Boolean = False
        While sr.Peek >= 0
            s = sr.ReadLine
            If s.Trim <> "" Then
                sw.WriteLine(s)
            Else
                FoundBlankLines = True
            End If
        End While
        sr.Close()
        sw.Close()
        If FoundBlankLines = True Then
            System.IO.File.Delete(file_name)
            System.IO.File.Move(file_name + "_temp", file_name)
        Else
            System.IO.File.Delete(file_name + "_temp")
        End If
    End Sub

 

[sirnose1]

I was just informed that I can skip over the blank lines and not make a separate file to read from. I cant find any info on how to skip over a blank line. How would that be accomplished?

 

[Sorwen]

You've got a lot going on to really give you good help. RiverGuy's answer will work for you delete line. If you don't want to bother with a temp file really need to have a better idea on what you want to do. Assuming you still want to read in a text file what do you want to do with the data after that?

Here is what I commonly do. I read in a text file and put it into a collection. I've added to skip blank lines which I normally don't do.

        Dim sr As StreamReader
        Dim CurrentLine As String
        Dim LineList As New Collection

        If File.Exists(FileLocation) = True Then
            sr = New StreamReader(FileLocation)

            Do While Not sr.EndOfStream
                CurrentLine = sr.ReadLine()
                If CurrentLine <> "" Then
                    LineList.Add(CurrentLine)
                End If
            Loop
            sr.Close()
        End If

-I hate Microsoft!
-Forever and always forward.

 

[petermeachem]

or

Dim fs As New FileStream("c:\temp\b.txt", FileMode.Create)
Dim sw As New StreamWriter(fs)
Dim sd As New StreamReader("c:\temp\a.txt")
Dim fileText As String = sd.ReadToEnd()
fileText = Replace(fileText, Chr(10) & Chr(13), "", 1)
sd.Close()
sw.Write(fileText)
sw.Close()

With a bit of fiddling round with the declares I don't see why this shouldn't write back to the same file. It does read the whole file, so huge files are going to be a problem.

 

[sirnose1]

Thanks for all your help. However I have one last issue with this program. (this is really complicated). I convinced the powers that be to not skip over the lines but to delete them. What my problem is is that some of my values are being repeated.

example of some of the output:

DET-01-002737,DET-01-002737,YES,64239764b32fefc915a593f41bdb5730
DET-01-002738,DET-01-002737,NO,64239764b32fefc915a593f41bdb5730
DET-01-002739,DET-01-002737,NO,64239764b32fefc915a593f41bdb5730
DET-01-002740,DET-01-002737,NO,64239764b32fefc915a593f41bdb5730
DET-01-002741,DET-01-002737,NO,64239764b32fefc915a593f41bdb5730
DET-01-002742,DET-01-002737,NO,64239764b32fefc915a593f41bdb5730
DET-01-002743,DET-01-002737,NO,64239764b32fefc915a593f41bdb5730
DET-01-002744,DET-01-002737,NO,64239764b32fefc915a593f41bdb5730
DET-01-002745,DET-01-002737,NO,64239764b32fefc915a593f41bdb5730
DET-01-002746,DET-01-002737,NO,64239764b32fefc915a593f41bdb5730
DET-01-002747,DET-01-002737,NO,64239764b32fefc915a593f41bdb5730
DET-01-002739,DET-01-002737,NO,BE29D4AC2C809C4C3703E7B57272AA4B
DET-01-002740,DET-01-002737,NO,BE29D4AC2C809C4C3703E7B57272AA4B
DET-01-002741,DET-01-002737,NO,BE29D4AC2C809C4C3703E7B57272AA4B
DET-01-002742,DET-01-002737,NO,BE29D4AC2C809C4C3703E7B57272AA4B

if the field(2) is "NO" it gets appended under "YES" if field(1) is = field(2)

Here is the code:

Dim EID, EID_2 As String
Dim GID, GID_2 As String
Dim IsParent, IsParent_2 As String
Dim MD5, MD5_2 As String
Dim lineString As String = ""
Dim lineString2 As String = ""
Dim dlmtr As String = ","
Dim fields() As String
Dim fields_2() As String
Dim sr As StreamReader
Dim rdr As StreamReader
Dim MD5_Log As StreamWriter
Dim error_log As StreamWriter
Dim error_log_2 As StreamWriter
Dim loadFile As String = Me.txtLoadFile.Text
Dim folder As String = Me.txtOutputFldr.Text


If Me.txtLoadFile.Text <> "" And Me.txtOutputFldr.Text <> "" Then
Try
DeleteBlankLines()

sr = New StreamReader(folder & "\" & "MD5_clean.txt")
MD5_Log = New StreamWriter(folder & "\" & "MD5_log.txt", False)
error_log = New StreamWriter(folder & "\" & "error_log.txt", False)
error_log_2 = New StreamWriter(folder & "\" & "Child_error_log.txt", False)
error_log_2.AutoFlush = True
MD5_Log.AutoFlush = True
error_log.AutoFlush = True
Me.StatusBar1.Text = "Processing File...................."


While sr.Peek() <> -1
lineString = sr.ReadLine()
If Not lineString = "" Then
fields = lineString.Split(dlmtr)
If fields.Length <> 4 Then
error_log.WriteLine(lineString & "|" & "Error:Field Length <> 4")
'GoTo errorskip
End If
Else
error_log.WriteLine(sr.Peek & "Test")
End If
EID = fields(0)
GID = fields(1)
IsParent = fields(2)
MD5 = fields(3)

If IsParent <> "YES" And IsParent <> "NO" Then
error_log.WriteLine("No IsParent Identified")

ElseIf IsParent = "YES" Then
'write the parent and its children
MD5_Log.WriteLine(EID & "," & GID & "," & IsParent & "," & MD5)
End If

rdr = New StreamReader(folder & "\MD5_clean.txt")

While rdr.Peek() <> -1
lineString2 = rdr.ReadLine()
fields_2 = lineString2.Split(dlmtr)

If fields_2.Length <> 4 Then
error_log_2.WriteLine(lineString2 & "|" & "Error:Field length <> 4")
'GoTo errorskip
Else
EID_2 = fields_2(0)
GID_2 = fields_2(1)
IsParent_2 = fields_2(2)
MD5_2 = fields_2(3)


If IsParent_2 <> "YES" And IsParent_2 <> "NO" Then
error_log.WriteLine(lineString & "|" & "IsParent Field not Yes or No")
Else
If EID_2 = "" Or GID_2 = "" Or IsParent <> "YES" Or MD5_2 = "" Then
error_log.WriteLine(lineString & "|" & "Error:Value missing")
End If
If (IsParent_2 <> "YES") And (GID_2 = GID) And (MD5_2 <> MD5) Then
MD5_Log.WriteLine(EID_2 & "," & GID_2 & "," & IsParent_2 & "," & MD5)
End If
End If
End If
End While
End While

Catch ex As Exception
'System.Diagnostics.Debug.WriteLine(ex.Message)
MessageBox.Show(ex.Message)
Finally
rdr.Close()
rdr = Nothing
Me.StatusBar1.Text = "Ready"
MessageBox.Show("Complete")
End Try
Else
If Me.txtLoadFile.Text = "" Then
MessageBox.Show("Please enter the load file")
Me.btnLoadFile.Focus()
ElseIf Me.txtOutputFldr.Text = "" Then
MessageBox.Show("Please select an output directory")
Me.btnOutputFldr.Focus()
End If
End If

Cant for the life of me figure out what is wrong with my looping. Is my errorskip in the wrong place?

 

[mmilan]

Consider using regular expressions - getting shut of your blank lines is a lot easier (and safer) that way.

Martin