creating garbage collectable objects (caching objects)

Hi.

I started playing with PIL.

I'm performing operations on multiple images and would like compromise
between speed and memory requirement.

The fast approach would load all images upfront and create then multiple
result files. The problem is, that I do not have enough memory to load
all files.

The slow approach is to load each potential source file only when it is
needed and to release it immediately after (leaving it up to the gc to
free memory when needed)



The question, that I have is whether there is any way to tell python,
that certain objects could be garbage collected if needed and ask python
at a later time whether the object has been collected so far (image has
to be reloaded) or not (image would not have to be reloaded)


# Fastest approach:
imgs = {}
for fname in all_image_files:
imgs[fname] = Image.open(fname)
for creation_rule in all_creation_rules():
img = Image.new(...)
for img_file in creation_rule.input_files():
img = do_somethingwith(img,imgs[img_file])
img.save()


# Slowest approach:
for creation_rule in all_creation_rules():
img = Image.new(...)
for img_file in creation_rule.input_files():
src_img = Image.open(img_file)
img = do_somethingwith(img,src_img)
img.save()



# What I'd like to do is something like:
imgs = GarbageCollectable_dict()
for creation_rule in all_creation_rules():
img = Image.new(...)
for img_file in creation_rule.input_files():
if src_img in imgs: # if 'm lucke the object is still there
src_img = imgs[img_file]
else:
src_img = Image.open(img_file)
img = do_somethingwith(img,src_img)
img.save()



Is this possible?

Thaks in advance for an answer or any other ideas of
how I could do smart caching without hogging all the system's
memory





.

Relevant Pages

  • Re: Rapid gfx display Qs
    ... I think with Autoredraw False you are effectively timing the total time both of loading the picture from disk into memory and also the time taken for the OS to dump that picture to the screen. ... In contrast, with Autoredraw True, you are timing only the time taken to load the picture from disk into memory because the VB Autoredraw system itself will dump the property to the actual display just after your code has finished. ... I get a time of about 79 msecs just to load the picture into a StdPicture object, even though the file size of the jpg is very much smaller, only about one tenth of the file size of the equivalent bmp. ... Much of course depends on the nature of your images, and it is hard to comment any further really without knowing what sort of images they are. ...
    (microsoft.public.vb.general.discussion)
  • need to limit or disable files being retained in memory or cache
    ... The viewing program loads in a browser window ... and gives the user an option to view images. ... The images take a fairly long time to load 1 - 3 ... That tells me that they are in memory somewhere. ...
    (microsoft.public.windowsxp.perform_maintain)
  • Re: creating garbage collectable objects (caching objects)
    ... The fast approach would load all images upfront and create then multiple ... that I do not have enough memory to load ...     img.save ...
    (comp.lang.python)
  • Re: need to limit or disable files being retained in memory or cache
    ... The viewing program loads in a browser window ... > and gives the user an option to view images. ... In my testing I noticed that if I load the ... That tells me that they are in memory somewhere. ...
    (microsoft.public.windowsxp.perform_maintain)
  • Re: need to limit or disable files being retained in memory or cache
    ... The viewing program loads in a browser window ... > and gives the user an option to view images. ... In my testing I noticed that if I load the ... That tells me that they are in memory somewhere. ...
    (microsoft.public.windowsxp.perform_maintain)