Any chances to perform parallel (multithreaded) INSERTs?
- From: au@xxxxxxx
- Date: Wed, 3 Dec 2008 12:22:44 +0100 (CET)
Hi all,
with my Perl script I have to upload large amounts of data into various
tables of a SQL Server database. The actual upload process takes some (too
much) time although I am already using prepared statements. I am using
ithreads in my script so it would be easy to speed up the upload process
by parallelizing the INSERTs. Unfortunately this doesn't work: DBI
remembered me of some issues:
Thread 12 terminated abnormally: DBD::ODBC::db prepare
failed: handle 2 is owned by thread 100103370 not current
thread 108773770 (handles can't be shared between threads
and your driver may need a CLONE method added)
The explanation in the DBI docs finally put me over the edge that there is
an easy way of parallelizing:
Threads and Thread Safety
[...]
However, the internal pointer data within the handles will
refer to the DBI and drivers in the original interpreter.
Using those handles in the new interpreter thread is not
safe, so the DBI detects this and croaks on any method call
using handles that don't belong to the current thread
(except for DESTROY).
Now I am wondering if there are still any chances to workaround this and
to parallelize the INSERTs. My SQL statements are covered by a transaction
so I cannot let my threads create a database handle individually.
The Unix (Solaris) part of the ecosystem consists of Perl 5.8.8, DBI,
DBD::ODBC, unixODBC, and FreeTDS. The database is a Microsoft SQL Server
2005 database.
Thanks for your help!
Stephan
.
- Prev by Date: Re: Protecting a mod perl 1.3 site from slow MySql processes
- Next by Date: Fwd: Protecting a mod perl 1.3 site from slow MySql processes
- Previous by thread: DBI + ithreads performances
- Next by thread: Quoting in SQL-Statements?
- Index(es):
Relevant Pages
|
Loading
