vb.net在大型.log文件中搜索特定的行

Question

我正在读取输出到文本视图的1-5mb日志文件，并搜索输出到另一个文本视图的特定行。目前，一个1mb的文件大约需要一分钟的时间。除了我使用的方法之外，有人知道其他更快的搜索行或字符串的方法吗？

Imports EnterpriseDT.Net.Ftp
Public Class Form1
Private Sub SettingsToolStripMenuItem_Click(sender As System.Object, e As System.EventArgs) Handles SettingsToolStripMenuItem.Click
End Sub
Private Sub Button2_Click(sender As System.Object, e As System.EventArgs) Handles Button2.Click
    Dim sw As New Stopwatch
    Dim FullLine As String = ""
    Dim ScriptLine As String = ""
    sw.Start()
    Dim ll As New Queue(Of String)
    Dim i As String = ""
    Using TestFile As New IO.StreamReader("c:test.txt", System.Text.Encoding.Default, False, 4096)
        Using OutFile As New IO.StreamWriter("c:SBOutFile.txt", False, System.Text.Encoding.Default, 4096)
            While TestFile.EndOfStream = False
                i = TestFile.ReadLine
                If i.Contains(".sqf") And i.Contains("handleGear.sqf") = False Then
                    ScriptLine = ScriptLine & i & vbNewLine & vbNewLine
                    FullLine = FullLine & i & vbNewLine & vbNewLine
                Else
                    FullLine = FullLine & i & vbNewLine & vbNewLine
                End If
            End While
        End Using
    End Using
    sw.Stop()
    TextBox1.Text = FullLine
    TextBox2.Text = ScriptLine
    RichTextBox1.AppendText(String.Format("Run_Queue took {0} Milliseconds." & Environment.NewLine, sw.ElapsedMilliseconds))
End Sub
Private Sub Button1_Click_1(sender As System.Object, e As System.EventArgs) Handles Button1.Click
    Try
        'connect to ftp server
        Dim ftp As New FTPConnection
        ftp.ServerAddress = "-"
        ftp.ServerPort = "-"
        ftp.UserName = "-"
        ftp.Password = "-"
        ftp.Connect()
        ftp.ChangeWorkingDirectory("-")
        ftp.TransferType = FTPTransferType.BINARY
        'download a file
        ftp.DownloadFile("c:test.txt", "scripts.log")
        'ftp.RenameFile("scripts.log", "scripts_test.log")
        'close the connection
        ftp.Close()
    Catch ex As Exception
        MessageBox.Show(ex.Message.ToString())
    End Try
End Sub
Private Sub Button3_Click(sender As System.Object, e As System.EventArgs) Handles Button3.Click
End Sub

最终类

Answer 1

考虑到你在阅读时所做的大量阅读和串联，我怀疑这是你的时间/性能问题的一部分。我可能会考虑将ScriptLine和FullLine的声明从String类型更改为StringBuilder，因为从技术上讲，String是不可变的。这意味着每个串联实际上都是对前一个对象的分解，并在其位置创建一个新对象。StringBuilders是专门为大量串联场景设计的。循环完成后，可以将其转换回字符串。

此外，编译后的正则表达式可能比String.Contains搜索得更快。正则表达式字符串可能类似于"（？！handleGear）.sqf"，意思是"在字符串".sqf"前面查找除"handleGear"之外的任何零个或多个字符的序列。

我还没有机会测试这个表达式，所以它有这个警告。如果我有机会做一个测试，我很乐意修改并告诉你。

祝你好运！

Answer 2

我只是想发布我最终的想法。这对我来说是一个很大的进步。

'Read file
    Dim sw As New Stopwatch
    Dim FullLine As String = ""
    Dim ScriptLine As String = ""
    sw.Start()
    Dim ll As New Queue(Of String)
    Dim i As String = ""
    Dim builder As New StringBuilder
    Using TestFile As New IO.StreamReader("c:test.txt", System.Text.Encoding.Default, False, 4096)
        builder.AppendLine("Started at: " & DateTime.Now.ToLongTimeString().ToString)
        RichTextBox1.AppendText(Now.ToShortTimeString & " Reading Log File Started" & vbNewLine)
        RichTextBox1.SelectionStart = RichTextBox1.TextLength
        Dim rowCount As Integer = 0
        Do Until TestFile.EndOfStream
            ScriptLine = TestFile.ReadLine
            ScriptLine = LCase(ScriptLine)
            If InStr(ScriptLine, ".sqf") > 0 And InStr(ScriptLine, "handlegear.sqf") < 1 Then 'And InStr(ScriptLine, "createmarkerlocal.sqf") < 1 And InStr(ScriptLine, "setmarkerposlocal.sqf") < 1 
                builder.AppendLine(ScriptLine)
                builder.AppendLine()
            End If
            rowCount = rowCount + 1
        Loop
        builder.AppendLine(Now.ToShortTimeString & "==== Searched " & rowCount & " rows" & vbNewLine)
        builder.AppendLine(Now.ToShortTimeString & " Finished" & vbNewLine)
    End Using
    sw.Stop()
    RichTextBox2.AppendText(builder.ToString & vbNewLine)
    RichTextBox1.AppendText(Now.ToShortTimeString & String.Format(" Run_Queue took {0} Milliseconds." & Environment.NewLine, sw.ElapsedMilliseconds))