Re: IO with Java Objects: slow!

From: Collin VanDyck (thehoppocket_at_yahoo.com)
Date: 03/02/04


Date: Tue, 02 Mar 2004 22:52:19 GMT


"Charles Packer" <mailbox@cpacker.org> wrote in message
news:f88bc493.0403021441.1013f167@posting.google.com...
> I have a Vector of 120,000 Vectors, each of which is about 60 bytes long.
> Writing it out to the disk in an ObjectOutputStream as a single Object
> takes 20 seconds on our system, as does writing out the 120K Vectors
> individually as Objects. But if I change each Vector into a byte array and
> write them to the disk in a BufferedOutputStream, it takes only 6 seconds
> when I write them as individual byte arrays. If I build a byte buffer of
> the whole dataset first, writing it takes less than one second.
>
> The implication is that where speed of file access is critical, it's
better
> to bypass the convenience of dealing with Objects and use plain vanilla
> formats, the tradeoff being some extra parsing.

Yes, you'll definitely find a tradeoff between Arrays (which are Objects
too!) and the Collections classes. Note though that you chose a
slow-by-nature Collection -- the Vector. See what your time is using an
ArrayList, which is faster.

-CV



Relevant Pages

  • Fastt600 Performance
    ... I created a similar configuration of 2 7 disk arrays each on ... also make sure half are owned by one controller and half by the other, ... Yard, John wrote: ...
    (AIX-L)
  • Temporary lock-up under heavy write, MegaRAID RAID-5
    ... LSI Logic MegaRAID 320-1, 64MB cache ... when writing very large files. ... disk write access, actually). ... The RAID controller is currently set to "write-through". ...
    (Debian-User)
  • Re: X-CD Roast question again please
    ... Using generic SCSI-3/mmc CD-R driver. ... Disk sub type: Medium Type A, ... ATIP start of lead in: -11318 ... writing 152 KB of pad data. ...
    (RedHat)
  • Re: OT - Free Sun servers
    ... talking about hobbyist uses of Sun servers and other gear, ... an Ultra10 workstation, a few generic SCSI disk racks, and several SENA ... storage arrays ... Making sure other people play nice and dont kill each other ...
    (rec.crafts.metalworking)
  • Re: efficacy of Linux w/o swap
    ... | Caching is supposed to happen, and this does not affect performance. ... that the impact starts to degrade the writing process itself. ... | Writing directly to disk will substantially decrease performance. ... Performance for the first 3/4 of an IDE disk ...
    (comp.os.linux.development.system)