newbie question: processing large database in chunks?


Hi,

When I
select * from XX
I'm geting an OutOfMemoryError. I'm using JDBC with Mysql,
and calls like:

ResultSet rs = dbc.executeQuery("select * from thetable");

My beginner's question: what's the right way to read and process the
table in chunks, to avoid running out of memory?

Is using LIMIT n1,n2 inside the query the right thing to do, giving
e.g. LIMIT 1,100 followed by another query using LIMIT 101,200, etc?
Is there a more efficient way?

I guess I'm surprised that it's even necessary. I would have thought
that the data would stay on the server until pulled to the client
by resultset.next() and a resultset.getXX(). But evidently
the whole table is being sent to the client??
.

Relevant Pages