Achieving very smooth scrolling of large data sets?

From: John Doe (spammeallyoucan_at_hotmail.com)
Date: 06/24/04


Date: 24 Jun 2004 11:51:13 -0700

Hello all,

I'm looking to smoothly scroll large data sets. I'm already using
an AbstractTableModel that caches blocks of a few hundred records
at a time (plenty to fit a few displays full of data).

My getValueAt looks as follows:

--
    public Object getValueAt(int rowIndex, int columnIndex) {
        if (rowIndex>this.maxrow) {
            load_block(rowIndex);  // causes 0.3 second delay
        } else {
            if (rowIndex<this.minrow) {
                load_block(rowIndex); // 0.3 second delay here too
            }
        }
        int relrow=rowIndex-this.minrow;
        Vector record=(Vector)this.blockdata.get(relrow);
        return record.get(columnIndex);
    }
--
The load_block does decent caching; it only loads records that 
are not currently in memory and keeps memory usage modest. 
For line by line scrolling this works smooth enough (scrolling 10
lines per second) however when dragging the scrollbar or using PgUp
and PgDn, the system won't keep up because load_block is called more
often than the database can return the data. As a result things feel
slightly unresponsive- the scrollbar thumb lags behind the mouse.
How would I be able to optimize this so that the system will respond
faster (or simply feel more responsive)?
I'm thinking of:
- Only doing a database lookup when the user stops dragging the mouse,
  however this does not protect from PgUp/PgDn;
- skipping database lookups altogether if more scroll actions are
still
  pending, is it possible to see this somehow? This would also save
  bandwidth when the database is on a network;
- Doing the lookups in the background, any thoughts on this?
Opinions and suggestions will be much appreciated. Extra bonus points
for snippets of helpful code!
Regards,
John
(Disclaimer: I'm a quite experienced programmer but relatively 
new to Java.)