Re: Race condition when generating .pyc files

yogamatt1970@xxxxxxxxx wrote:
I have a large body of Python code which runs on many different (Unix)
machines concurrently. Part of the code lives in one place, but most
of it lives in directories which I find at runtime. I only have one
copy of each Python source file and I think I'm hitting a race
condition where two hosts attempt to import the same module at the
same time. My import fails on one of the machines and the following
exception is thrown:
EOFError: EOF read where object expected
My hypothesis is that there's contention between the two (or more)
hosts when the module's .pyc file is generated.

Possible solutions I see:
1) Running a cron job before my code executes which compiles all the
python source first.
2) Making separate copies of all the .py files for each host running
the code - I'd rather not do this, it seems like a big pain.
3) Inhibiting the generation of .pyc files altogether if that's even
possible - I saw a PEP for a possible -R flag (http://www.python.org/
dev/peps/pep-0304/) but I don't think it has been added to a released
version of Python yet plus I am stuck with Python 2.4 for the time
being.

Any and all help appreciated.

Are you using the same version of Python on all the hosts that are using
the code? If not, the different versions will be forever arguing about
whether the .pyc files need recompiling.

Just a thought ...

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

.

Relevant Pages

  • Re: Race condition when generating .pyc files
    ... Part of the code lives in one place, ... copy of each Python source file and I think I'm hitting a race ... hosts when the module's .pyc file is generated. ... If you don't want pyc files to be created, you could set Unix permissions such that Python cannot write to the directory. ...
    (comp.lang.python)
  • Race condition when generating .pyc files
    ... I have a large body of Python code which runs on many different ... Part of the code lives in one place, ... hosts when the module's .pyc file is generated. ...
    (comp.lang.python)
  • Re: Python bytecode compatibility between interpreter versions
    ... Note that I specifically mentioned .pyc files vs. java class files, ... not Java the language/environment/platform vs. Python. ... > your platform. ... Perhaps one day .pyc files can work in such a manner. ...
    (comp.lang.python)
  • Re: Any way to not create .pyc files?
    ... As long as you zipped up the package will all .pyc and .pyo files removed, Python will have no choice but to compile the files every time they are imported - unless I'm grossly mistaken Python won't put the pyc files into the zip archive, or modify any that were there already. ... Instead of the users starting the application directly, they could start a starter application that checks with the server to determine if local files need to be updated, and if so grab them and then start the main app. ...
    (comp.lang.python)
  • Re: bytecode non-backcompatibility
    ... > So there are currently 7 implementations or variations of the Python ... >The same can't be said for .pyc files ... One of the principles of OOP is separation of interface from ... Python language is interface. ...
    (comp.lang.python)