Re: Load from file using asterisk as field delimiter

Dan,

> There are by the way a total of 247,938 records that I am importing
SMImport loads the 70-90 000 records (15 non-numeric fields and 15 fields
with transformations like currency, date etc) per minute in "dataset" mode
on client (to insert the record into TDataset, ActionKind = akDatasetApply).
In "sql-generator" mode (to generate the sql-script for update, ActionKind =
akGenerateSQL) the file will be processed much faster, of course.

I'm sure that you do something incorrectly (maybe record separator or field
delimiter defined incorrectly) or your dataset engine (connection to
Advantage database) is very slow for some reason (custom code in dataset
event or extended field validation, for example).

A few questions:
1. do you use the ADO to connect to Advantage? If yes, do you use the ODBC
driver in connection string or direct driver for Advantage?
2. did you try to insert the 250 000 dummy records into dataset? How many
time takes this process? Just execute this code:
var
i, j: Integer;
begin
yourDataset.DisableControls;
for i := 0 to 250000 do
begin
yourDataset.Append;
for j := 0 to yourDataset.FieldCount-1 do
yourDataset.Fields[i].AsString := IntToStr(j);
yourDataset.Post;
end;
yourDataset.EnableControls;
end;
3. if this code takes a lot of time, you must "turn" the yourDataset
settings. The main time takes the inserts into dataset, not the file
parsing..
--
With best regards, Mike Shkolnik
EMail: mshkolnik@xxxxxxxxxxxxx
http://www.scalabium.com


.