Re: Overhead of individual python apps

"Qopit" <russandheather@xxxxxxxxx> writes:

> When running in Windows, launching each application generates a
> process, and each of those processes ends up taking up > 4MB of system
> memory. This memory usage is as reported by the Windows Task manager
> for the python.exe image name.

The first step is to clarify what's being reported. If WTM is
reporting the total memory usage for each process, then it's over
estimating the total usage by a considerable amount. In particular,
all the Python executable code should be shared by all the
processes. Unless you load compiled extensions, anyway - those will
only be shared by the ones that use them. You'll need something that
will give you the stack, heap and code segment sizes separately to
work out what the memory usage really is.

> My Question: Is there any way to reduce this per-process overhead? eg:
> can you set it somehow so that one python.exe instance handles multiple
> processes?

The OS should do that for you by default.

> One possibility considered is to run them as threads of a single
> process rather than multiple processes, but this has other drawbacks
> for my application and I'd rather not,

That shouldn't help memory usage - the data that isn't across
processes would need to be thread-private in any case.

The reason for the uncertainty is that I'm not positive that Windows
behaves sanely in this area. It may be that Windows doesn't have
shared executables. In this case, one solution is to move to a modern
OS - like v6 Unix :-).

<mike
--
Mike Meyer <mwm@xxxxxxxxx> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
.

Relevant Pages

  • Re: Growing memory usage in D4 app with SerialNG
    ... it must run on a Windows platform, usually a laptop, and the hardware that might use the RTS and CTS has a limited amount of memory for a buffer, and it would certainly overrun if it were asked to hold off transmission for any longer than a hundred mSec or so. ... But while it was easy to have an almost-realtime program run on DOS, one will have all kind of problems when the same is attempted on Windows. ... The initial memory usage shown when I minimize the application is 1100 kB, which is exactly what I would expect for those figures, but I don't know why it should keep climbing. ...
    (comp.lang.pascal.delphi.misc)
  • RE: Virtual memory
    ... You can configure virtual memory in windows xp and thereby improve the ... How to set performance options in Windows XP ... if you have background programs such as printing or disk ... It is also known as the paging file. ...
    (microsoft.public.windowsxp.perform_maintain)
  • [NT] NNTP Service in Windows Contains Memory Leak
    ... NNTP Service in Windows Contains Memory Leak ... An affected server could be restored to normal service by ...
    (Securiteam)
  • Re: Physical Memory
    ... These Windows services are started: ... Memory optimizers/defragers are nothing more than snake ... These optimizers work by making demands on the Windows Memory Manager ... The funny thing about all of this nonsense is that the snake oil memory ...
    (microsoft.public.windowsxp.general)
  • Re: How hard is socket programming?
    ... finds the page which still happens to be in memory and ... paged out on Windows 7 with 8.0 GB RAM. ... As far as I can tell FastCGI is good here ... one that has been established by long practice (the ...
    (microsoft.public.vc.mfc)