Re: cffi hell
- From: Richard M Kreuter <kreuter@xxxxxxxxx>
- Date: Sun, 15 Jul 2007 01:00:18 -0400
Matthew Swank <akopa_is_very_much_like_my_email@xxxxx> writes:
Is it my imagination, or is cffi the single hardest library to
provide a single working version across a large range of dependent
libraries. I seem to end up with ghettoized projects where I have
to be careful to not include certain wrapped libraries together.
Quite a few projects follow cffi with the new datatype additions,
but newer cffi's don't seem terribly backward compatible.
Though providing compatibility or consistency would be nice, it's
just wallpaper over what seems to be a more serious problem: there
seems to be a fundamental breakage in the way asdf and the package
system usually provides name-space management.
That is to say it would be nice to provide versioned libraries in a
way that allows me effectively provide a version of a package to the
code that consumes it. Control over what package is imported for the
version of a library seems doable, but hard coded fully qualified
symbols (eg cffi:blah) seems more problematic.
Actually, the package-qualified-symbol problem is moderately easy to
solve, sort of: modify PERFORM on (COMPILE-OP, LOAD-OP, or
LOAD-SOURCE-OP) and CL-SOURCE-FILE) so to rearrange the package
namespace before and after calls to CL:COMPILE-FILE or CL:LOAD. The
easiest way to do this is with an :AROUND method on PERFORM
specializing on those operation classes and CL-SOURCE-FILE. (It would
be better if all inter-package name usage were isolated into
DEFPACKAGE-like forms, but that's not how people do things.)
It's also fairly easy to track which packages get created by which
systems, by comparing the package namespace before and after compiling
or loading a CL-SOURCE-FILE. With this information, it's easy enough
to rename packages created by a system, say, by appending the system's
version to the name of each package created as a side-effect of
compiling or loading the system. An :AFTER method on PERFORM
specializing on one of the default operation classes and
CL-SOURCE-FILE should do the trick here.
So then all you need to do is attach versioned dependency information
to systems that need specific versions of their dependencies. You
might do this without subclassing anything by using the PROPERTIES
slot available via the COMPONENT-PROPERTY function.
The problems are several:
* ASDF's mechanism for finding systems is customizable, but not
generic: FIND-SYSTEM first converts its argument to a string, and
then invokes a customizable sequence of functions on the string
until one returns a pathname. So it's effectively necessary to
encode version information in system names in order to provide
multiple system versions in the search space. This means renaming
.asd files and rewriting :depends-on information in defsystem forms.
* A few ASDF systems extend add some subclasses of ASDF's classes, by
having code in the .asd file. Some extra work is necessary to
ensure that the contents of those .asd files don't conflict with
each other. Usually, changing some package names in the .asd files
suffices, but a few systems contain components that hard-code the
names of symbols in the package used in the .asd file. So it's
necessary, in general, to include the package used in the .asd file
as part of the package namespace rearranging. It turns out to be
impossible to automatically register the package used in the .asd
file as belonging to the system, however, so that means more
metadata has to go into the defsystem form.
* In principle, any program might assume that the run-time package
namespace is the same as the compile-time or load-time package
namespace. For example, a program might do
(with-standard-io-syntax
(let* ((*package* (find-package "P"))
(thing (read-from-string string)))
(if (eql thing 'p:foo)
...)))
This code will lose if the package that was named "P" when the code
was loaded has been renamed by the time the code is run. This has
nothing to do with ASDF per se; the program should say something
like
(find-symbol "FOO" "P")
but there's some code in the wild that does the wrong thing
analogous to the above.
* Some features of SLIME don't work correctly when the run-time
package namespace has been rearranged, for reasons like the ones
above.
* Various systems in the wild don't have version numbers, at least not
in the .asd file. This is a minor nit, since you can always make up
some numbers.
* Finally, it's probably a violation of encapsulation to add methods
to ASDF's protocol functions that specialize on its default classes,
but there's no other way to customize ASDF's site-wide behavior.
Since you have to edit .asd files that you want to be part of this
pseudo-versioning system in order for ASDF to find them, you might
subclass CL-SOURCE-FILE and add the methods described above as
specializing on your subclass, but then you'll have to do extra work
for those systems whose .asd files actually extend ASDF.
So what you're talking about is technically possible without much hard
work, though it's clearly not supported, and by inspection it's not
worth the effort. I implemented the above about a year ago as an
experiment, only to find that the annoyances involved in maintaining a
mess of locally-tweaks all over the place outweighed the benefits of
having multiple versions of a system in one image.
Anyway, kind of a rant, but it seems like a problem with no easy
answer.
Agreed.
I wonder what people did to support multiple versions of a system on
the LispM. Were packages versioned there?
--
RmK
.
- Follow-Ups:
- Re: cffi hell
- From: Rainer Joswig
- Re: cffi hell
- References:
- cffi hell
- From: Matthew Swank
- cffi hell
- Prev by Date: Re: rewrite comparison Qi, Lisp OCaml - results now on www
- Next by Date: script solution in common lisp
- Previous by thread: Re: cffi hell
- Next by thread: Re: cffi hell
- Index(es):
Relevant Pages
|
|
