Re: email module windows and suse



On Apr 13, 3:55 pm, Tim Roberts <t...@xxxxxxxxx> wrote:
Lev Elbert <elbert...@xxxxxxxxxxx> wrote:

I have to make a custom email module, based on the standard one. The
custom module has to be able to work with extremely large mails (1GB
+), having memory "footprint" much smaller.

Then you have a design problem right from the start.  It is extremely rare
to find a mail server today that will transmit email messages larger than a
few dozen megabytes.  Even on a 100 megabit network, it's takes a minute
and a half for a 1GB message to go from the server to the user's
workstation.

What are you really trying to do here?  In most cases, you would be better
off storing your attachments on a web server and transmitting links in the
email.

The modified program has to work in SUSE environment, while the
development is done under Windows.  I'm not too good with linux and do
not know if speedup in Windows translates one-to-one into speedup in
SUSE. For example, if the bottleneck is IO, in windows I can spawn a
separate thread or 2 to do "read-ahead".

We would need more information on your processing to advise you on this.
Disk I/O is slow, network I/O is slower.  You can't go any faster than your
slowest link.

Are threads available and as effective in SUSE as they are in Windows?

Threads are available in Linux.  There is considerable debate over the
relative performace improvement.
--
Tim Roberts, t...@xxxxxxxxx
Providenza & Boekelheide, Inc.

Thank you.

I have a 100mb mail file. I just made a very simple expiremnt

the message_from_file method boils down to a loop:
1 while True:
2 data = fp.read(block_size)
3 if not data:
4 break
5 feedparser.feed(data)
6
Total time is 21 seconds (lines 1-6), while processing (non IO) lines
3-5 is 20 seconds. This means, that no IO optimization would help.
This also explains the following fact: changing the block_size from
8K to 1M has almost no processing time impact. Also multithreading
wouldn't help.

I beleive I have to change Message class (more exactly: derive another
class, which would store pieces on a disk.
.



Relevant Pages

  • Re: email module windows and suse
    ... I have to make a custom email module, ... and a half for a 1GB message to go from the server to the user's ... development is done under Windows. ... Disk I/O is slow, network I/O is slower. ...
    (comp.lang.python)
  • Re: WIN+SQL over sual Xeon (sorry)
    ... Moving to Windows 2003 should see better performance as disk I/O is better ... > Sql2000 over w2000 adv server ...
    (microsoft.public.sqlserver.setup)
  • SecurityFocus Microsoft Newsletter #154
    ... MICROSOFT VULNERABILITY SUMMARY ... ISS RealSecure Server Sensor SSL Denial Of Service Vulnerabi... ... Roger Wilco Remote Server Side Buffer Overrun Vulnerability ... available for Microsoft Windows operating systems. ...
    (Focus-Microsoft)
  • SecurityFocus Microsoft Newsletter #49
    ... Subject: SecurityFocus Microsoft Newsletter #49 ... Microsoft Windows NNTP Denial of Service Vulnerability ... Microsoft IIS SSI Buffer Overrun Privelege Elevation Vulnerability ... Microsoft ISA Server H.323 Memory Leak Denial of Service... ...
    (Focus-Microsoft)
  • ~~~~~~~~~~~~~~ CANNOT FIND ~~~~~~~~~~~~~~
    ... acrobat cannot find external windows handler ... activesync cannot find exchange server ... aol internet explorer cannot find server ... brother network scanner cannot find pc ...
    (sci.geo.fluids)