Re: data passing

iu2 <israelu@xxxxxxxxxxx> writes:

Hi all,

I need your professional opinion about this (the project is C++ but it
doesn't realy matter):

We handle big structs of data. We also write them to files for use by
other teams.
Here it goes - we use these files also as a means of passing the data
to other stages in our own program.
Why we do so? Because some central modules were developed as stand
alone modules, using input files. Then they were joined to the
program.
As it turned out, passing data via files proved contagious, for now
it's our main system for passing data.

There is something appealing with files, it feels that they loosen up
the need of having strict interfaces among modules.


I would like to here what you say about that. Pros and cons.

It's OK. If the processing times of each (or some) phases are big,
it's even a good idea to save the data to files, it's kind of a
snapshoot, and if a later phase crashes, you can restart from the
latest files.

On the other hand, if you have shorter phases, one thing that could be
worthwhile is to have them read stdin and write stdout, instead of
random files (you can perhaps pass to the commands /dev/stdin
/dev/stdout (on linux)), so you can use pipes and keep the data in RAM
while the phases run in parallel.

--
__Pascal Bourguignon__ http://www.informatimago.com/

COMPONENT EQUIVALENCY NOTICE: The subatomic particles (electrons,
protons, etc.) comprising this product are exactly the same in every
measurable respect as those used in the products of other
manufacturers, and no claim to the contrary may legitimately be
expressed or implied.
.

Relevant Pages

  • data passing
    ... I need your professional opinion about this (the project is C++ but it ... doesn't realy matter): ... We handle big structs of data. ... As it turned out, passing data via files proved contagious, for now ...
    (comp.programming)
  • marshal string data from unmanaged to managed
    ... I found a lot of information on passing data from C# to a C++ dll ... What I cannot find is a way to return C++ structs of TCHAR string data back ... short snOperParams; ...
    (microsoft.public.dotnet.languages.csharp)