Re: Parallel Python

Paul Boddie wrote:
parallelpyt...@xxxxxxxxx wrote:
The main difference between MPI python solutions and ppsmp is that with
MPI you have to organize both computations
{MPI_Comm_rank(MPI_COMM_WORLD, &id); if id==1 then ... else ....} and
data distribution (MPI_Send / MPI_Recv) by yourself. While with ppsmp
you just submit a function with arguments to the execution server and
retrieve the results later.

Couldn't you just provide similar conveniences on top of MPI? Searching
for "Python MPI" yields a lot of existing work (as does "Python PVM"),
so perhaps someone has already done so. Also, what about various grid
toolkits?

[...]

Overall ppsmp is still work in progress and there are other interesting
features which I would like to implement. This is the main reason why I
do not open the source of ppsmp - to have better control of its future
development, as advised here: http://en.wikipedia.org/wiki/Freeware :-)

Despite various probable reactions from people who will claim that
they're comfortable with binary-only products from a single vendor, I
think more people would be inclined to look at your software if you did
distribute the source code, even if they then disregarded what you've
done. My own experience with regard to releasing software is that even
with an open source licence, most people are likely to ignore your
projects than to suddenly jump on board and take control, and even if
your project somehow struck a chord and attracted a lot of interested
developers, would it really be such a bad thing? Many developers have
different experiences and insights which can only make your project
better, anyway.

Related to your work, I've released a parallel execution solution
called parallel/pprocess [1] under the LGPL and haven't really heard
about anyone really doing anything with it, let alone forking it and
showing my original efforts in a bad light. Perhaps most of the
downloaders believe me to be barking up the wrong tree (or just
barking) with the approach I've taken, but I think the best thing is to
abandon any fears of not doing things the best possible way and just be
open to improvements and suggestions.

Paul

[1] http://www.python.org/pypi/parallel

I'd be interested in an overview.
For ease of use a major criterion for me would be a pure python solution, which also does the job of starting and controlling the other process(es) automatically right (by default) on common platforms.
Which of the existing (RPC) solutions are that nice?


Robert
.

Relevant Pages

  • Re: NET framework and hyperthreading/MMX/SSE/....
    ... >> that it is the developers responsibility to deal with threading within ... Anyone who would touch taht particular compiler would likely do so strictly ... > will get scared because they have no control. ... > know if ILAsm contains MMX and the SSEx-like instructions? ...
    (microsoft.public.dotnet.framework)
  • Re: Development environment best practices?
    ... > do developers typically revision control Windows CE? ... Confusion on the part of the BSP - I've always thought of a BSP as the ... > dev board and include proprietary drivers in the BSP? ...
    (microsoft.public.windowsce.platbuilder)
  • Re: Procedural ["bare ASP.NET"] versus OOPy ASP.NET
    ... involved with using a control, ... > design a reusable component one must know beforehand all the ways it ... > (OOP, procedural, and functional) over many years. ... > developers in general. ...
    (microsoft.public.dotnet.framework.aspnet)
  • Re: Lets think who will like to say delphi is dying?
    ... developers don't feel like they have enough control over the JavaScript. ... Working with HTML and graphics designers on large projects.. ... Amazing RAD features, while still producing a hi-performing executable. ...
    (borland.public.delphi.non-technical)
Loading