Re: Read binary data file

~kurt wrote:

I understand Hunter's comments, and and while I don't know much about
ASN.1 encoding, what I am pointing out is that binary files are usually
*not* intended to be used across sytems.
>
I think its use is quite industry-dependent: I've never seen it used in financial messaging (that's more likely to use SWIFT formats, which are tagged text) but its common in the telecommunications industry.

Telcos (both fixed line and mobile) use a lot of binary data for control and accounting purposes, mainly because this minimizes message size and there's a LOT of stuff flying around controlling the network in real time and accounting for its use. Switches from large vendors, e.g. Erickson, tend to use proprietary, flat message formats but if the data will be exchanged between different types of kit (e.g. roaming billing data) they tend to use ASN.1: CCITT likes it.

ASN.1 has a lot in common with XML in that its a tagged field protocol, allows nesting, and uses a tag dictionary to associate meanings with tags. Compared with XML its a LOT more compact (tags are one byte, fixed length fields don't have terminators, variable length fields are preceded by a one or two byte length) and it has a number of predefined field types as well as arrays. If you have the dictionary its easy to interpret on the fly though, like XML, you can also use the dictionary to generate code to encode and decode ASN.1 records.

Every binary data file I have
ever worked with was intended to be used either by the program that wrote
it, or separate applications that used the same utility libraries as the
application which wrote the data.
>
There's also a lot of binary data in large commercial systems. Formerly it was in large serial files, then flat indexed files, now its probably in a database. A really good reason for using an RDBMS is that it not only hides implementation details (like endian conventions) from the application, but the interfaces (SQL, JDBC, ODBC, etc) typically provide field conversion facilities.

There is nothing wrong with simply writing
the C structure to a file, and reading it in the same way.
>
I'd probably use a CSV format any place where a database would be obvious overkill, but ymmv.

Using CSV rather than binary makes debugging easier and (said with his *NIX hat on) it allows the data to be handled by common scripted utilities like awk, perl and even shell scripts. Oh yeah, Java too :-)


--
martin@ | Martin Gregorie
gregorie. | Essex, UK
org |
.

Relevant Pages

  • Re: Getting NASM from C
    ... > format we were going to edit it in... ... all XML is? ... HTML is a specific "subset" of XML for displaying ... it's a text file with "tags" inside ...
    (alt.lang.asm)
  • Re: Microsoft Word as competition for LaTeX
    ... LaTeX with larger workflows for scholarly publishing in a ... XML formats will be increasingly more ... the TeX scene one way or another (actually it's already here: ...
    (comp.text.tex)
  • Re: Gotta start somewhere ... how many of us are really out there?
    ... if you are thinking of using XML ... pure shell commands ... ... or whatever XML tags you want to use using the acutal ... accompish all this in hindsight,as all the commands ...
    (freebsd-questions)
  • Re: Storing data structure
    ... like trees and graphs to be stored in files and retrieved later ... also common is implementing a format based on line-oriented text files. ... in this way, we can use fprintf, fgets, and sscanf, in order to serialize and reload the data. ... this approach tends to lead to formats that are both fairly dense, and have a good tradeoff between speed and flexibility. ...
    (comp.lang.c)
  • Re: REZKONV und Apple OS X
    ... only MasterCook XML export format). ... (name of recipe) designates the Name element, ... These tags work with plain-text files or 3rd party XML files. ... There should be only one ingredient per line. ...
    (de.rec.mampf)