Too many open files
- From: AMD <amdescombes@xxxxxxxxx>
- Date: Mon, 04 Feb 2008 13:57:39 +0100
Hello,
I need to split a very big file (10 gigabytes) into several thousand smaller files according to a hash algorithm, I do this one line at a time. The problem I have is that opening a file using append, writing the line and closing the file is very time consuming. I'd rather have the files all open for the duration, do all writes and then close them all at the end.
The problem I have under windows is that as soon as I get to 500 files I get the Too many open files message. I tried the same thing in Delphi and I can get to 3000 files. How can I increase the number of open files in Python?
Thanks in advance for any answers!
Andre M. Descombes
.
- Follow-Ups:
- Re: Too many open files
- From: AMD
- Re: Too many open files
- From: Gary Herron
- Re: Too many open files
- From: Larry Bates
- Re: Too many open files
- From: Steven D'Aprano
- Re: Too many open files
- From: Jeff
- Re: Too many open files
- Prev by Date: polling for output from a subprocess module
- Next by Date: Re: Python GUI toolkit
- Previous by thread: polling for output from a subprocess module
- Next by thread: Re: Too many open files
- Index(es):