Re: Problems with TextFile parsing
From: eshipman (eshipman_at_yahoo!!!.com)
Date: 12/11/03
- Next message: Sven Pran: "Re: When random isn't random"
- Previous message: Dennis: "Re: Does _FILE_ exist in Delphi as in C?"
- In reply to: Eric Hill: "Re: Problems with TextFile parsing"
- Next in thread: Harley Pebley: "Re: Problems with TextFile parsing"
- Reply: Harley Pebley: "Re: Problems with TextFile parsing"
- Reply: Eric Hill: "Re: Problems with TextFile parsing"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Thu, 11 Dec 2003 16:12:47 -0600
In article <3fd8e401@newsgroups.borland.com>, eric@ijack.net says...
> > Now, here's the catch. If I do ANYTHING else while this is running, such
> > as move the application's window, activate another application, etc, the
> > import slows down to a crawl and it takes just under one minute to
> > import 10 LINES...
>
> Odd, just moving the window should just queue up a few messages and that's
> it. Maybe you're running out of message space and the whole system is
> swapping... A thread is really what you want.
>
Well, It's happening. I had previously added an IEProgressDialog to
monitor the process but if I moved it, the process would slow down.
Now it slows down if I do anything.
> > Anyone know what may be going on here? What could be the bottleneck?
> > Should I place this code in another thread? If so, how do I do that,
> > I've never done any multi-threading.
>
> A thread should be pretty easy to create. Take all your code that actually
> /does/ the processing, as well as any database components and the like, and
> put them all into a TThread descendent. In the Execute method of your
> TThread, start the process.
>
Not sure how...
> You can do all sorts of other things for monitoring and the like if
> desired.
>
> > I am willing to post pertinent source code, if it is needed.
>
> For a more detailed answer, please do.
>
Here is the essential code to parse the file, kinda long...
(procedure ImportAttackList)
bDone := False;
AssignFile(t, FileName);
Reset(t);
sl := TStringList.Create;
try
// skip the first three lines in the file
ReadLn(t, s);
ReadLn(t, s);
ReadLn(t, s);
while (not Eof(t)) and (not bDone) do
begin
Application.ProcessMessages;
ReadLn(t, s);
s := Trim(s);
sl.Clear;
if Length(s) > 0 then
begin
// if the first two chars of the line are ##, replace them
s := StringReplace(s, '##', '##, ', [rfReplaceAll]);
// Update progressbar and label
if (j1 mod 10 = 0) then
begin
PBar.Progress := Round((j1/giTotRecords)*100);
if (j1 mod 200 = 0) then
begin
// function to get the estimated completion time
sEst := GetEstimatedCompletion(giTotRecords,
j1, gdtBegtime);
lblEstCompl.Caption := 'Estimated Completion time: ' + sEst;
end;
Application.ProcessMessages;
end;
// See function below
Split(s, ',', sl);
// replace the first record based upon
// the third record's content
if Trim(sl[0]) = '##' then
begin
if Pos('started', sl[3]) > 0 then
sl[0] := '1';
if Pos('stopped', sl[3]) > 0 then
sl[0] := '7';
end;
// sl[0] can not be > 10
if sl[0] = '11' then
sl[0] := '1';
if sl[0] = '40' then
sl[0] := '4'
// see function below
LoadLine(sl);
end;
end;
finally
sl.Free;
CloseFile(t);
PBar.Visible := False;
end;
Here is the LoadLine procedure, it adds a record to the table,
parses the stringlist into the table's fields and then posts the
record.
procedure TTest.LoadLine(ALines: TStringList);
var
i: Integer;
begin
ZeosQuery.Append;
for i := 0 to 13 do
begin
case i of
0: ZeosQuery.FieldByName('Severity').AsInteger := StrToIntDef(Trim
(Alines[i]),0);
// DateStringToDateTime is function to parse yy-mm-dd hh:mm:ss date
// format into DateTime.
1: ZeosQuery.FieldByName('GMTTimestamp').AsString := DateTimeToStr
(DateStringToDateTime('yyyy-mm-dd', 'hh:mm:ss', Trim(ALines[i])));
2: ZeosQuery.FieldByName('IssueID').AsString := Trim(Alines[i]);
3: ZeosQuery.FieldByName('IssueName').AsString := Trim(Alines[i]);
4: ZeosQuery.FieldByName('IntruderIP').AsString := Trim(Alines[i]);
5: ZeosQuery.FieldByName('IntruderName').AsString := Trim(Alines
[i]);
6: ZeosQuery.FieldByName('VictimIP').AsString := Trim(Alines[i]);
7: ZeosQuery.FieldByName('VictimName').AsString := Trim(Alines[i]);
8: ZeosQuery.FieldByName('Parameters_').AsString := Trim(Alines
[i]);
9: ZeosQuery.FieldByName('ResponseLevel').AsString := Trim(Alines
[i]);
10: ZeosQuery.FieldByName('Count_').AsInteger := StrToIntDef(Trim
(Alines[i]),0);
11: ZeosQuery.FieldByName('IntruderPort').AsInteger := StrToIntDef
(Trim(Alines[i]),0);
12: ZeosQuery.FieldByName('VictimPort').AsInteger := StrToIntDef
(Trim(Alines[i]),0);
13: ZeosQuery.FieldByName('Packet_Flags').AsString := Trim(Alines
[i]);
end;
end;
Inc(j1);
ZeosQuery.Post;
end;
I use this function to split the line into a stringlist:
procedure Split(S, Delimiter: string; Strings:TStrings);
var
P, OldP: integer;
Token: string;
begin
if (Strings = nil) or (Length(S) = 0) or (Length(Delimiter) = 0) then
Exit;
P := Pos(Delimiter, S);
OldP := 1;
while P > 0 do
begin
Token := Copy(S, OldP, P-OldP);
Strings.Add(Token);
OldP := P + 1;
P := PosEx(Delimiter, S, OldP);
end;
if P = 0 then
Strings.Add(Copy(S, OldP, Length(S)));
end;
- Next message: Sven Pran: "Re: When random isn't random"
- Previous message: Dennis: "Re: Does _FILE_ exist in Delphi as in C?"
- In reply to: Eric Hill: "Re: Problems with TextFile parsing"
- Next in thread: Harley Pebley: "Re: Problems with TextFile parsing"
- Reply: Harley Pebley: "Re: Problems with TextFile parsing"
- Reply: Eric Hill: "Re: Problems with TextFile parsing"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
