Re: Wrapping a C++ library using ?FFI



Sohail Somani wrote:
Sohail Somani wrote:

I'm part of the way through writing some lisp gibberish that wraps a C++ library.

Ok, I've done most tricky things except for inheritance which I will tackle next. The general idea is to take the headers and generate the glue C, C++ and CFFI code. FWIW, this is how my brain works and I found it easiest to work this way.

I've attached a brain dump. It's kinda long but I would appreciate comments if you have them.

I suspect that for inheritance, the generated code will need to be more dynamic in that the cast_to_base function will need to do some sort of table lookups.

If that floats your boat, then have at it. Below, I've written a quick brain dump of why I don't think its a good long-term solution. But sometimes "worse is better".

As a quick note, ECL's "embedded C" strikes me as being cleaner than the more portable "write a C interface and a CFFI binding to the C interface". [Invoke the C++ compiler through some variant of (require :cmp) (setf compiler::*cc* "g++") (compile-file "c++binding.lisp")]



Portability gains from using the C layer are lost by the need to keep the build environment compatible with that of the C++ library. This includes getting all the include paths and #defines and other stuff 100% compatible with what the real build system uses. A little -m64 or -m32 can have far-reaching effects.

IIRC, boost::python does the best here since it is easily injected into the regular build system. Systems like SWIG require the user to scan command lines and try copying over all relevant details. And C++ template instantiation has enough bugs in real compilers that I don't trust any binding tool (other than the gccxml kludge) to get this completely bug-compatible with any

Another minus for the C layer -- its an abstraction which gains a little implementation simplicity at the cost of a little runtime performance and increased end-user pain. The cost of a C trampoline is easily calculated. But the real cost is in requiring end-users to install (the right version of) a C++ lib and then build a C interface lib (with the right flags). People may rightly say "this sounds brittle" and easily broken whenever the C++ lib is upgraded.


Taking the CFFI approach and implementing the ABI in Lisp isn't particularly brittle (C++ ABIs are already brittle); it just takes more work to set up (esp. for some compilers). As mentioned before before, the Itanium ABI [1] has been standardized for use by all x86_64 compilers (and with modifications for other 64-bit platforms). This (and what GCC/linux traditionally uses) makes the C++ calling convention exactly the same as the C convention, just with a hidden "this" parameter being passed, implicit pointer arithmetic, constructors/dtors being called, etc. Other legacy (esp MS) ABIs may require more work; [2] is an excellent reference.

[1] http://www.codesourcery.com/cxx-abi/abi.html
[2] Manual 5 on http://www.agner.org/optimize/

Later,
Daniel
.



Relevant Pages

  • Re: Wrapping a C++ library using ?FFI
    ... the more portable "write a C interface and a CFFI binding to the C interface". ... People may rightly say "this sounds brittle" and easily broken whenever the C++ lib is upgraded. ... implementation of libstdc++ has some ABI issues between versions. ... for use by all x86_64 compilers. ...
    (comp.lang.lisp)
  • Re: Wrapping a C++ library using ?FFI
    ... implementation of libstdc++ has some ABI issues between versions. ... I don't know exactly what you are talking about but inlined code cannot have any hope of binary compatibility. ... particularly brittle; it just takes more work to set up (esp. ... for some compilers). ...
    (comp.lang.lisp)
  • Re: Trying to understand "disassemble"
    ... Normally a C compiler comes with an ABI standard - it's basically a specification of how you call your function, how you provide the arguments, etc. ... But granted, this is what gives freedom to Lisp Compilers to optimize in areas, which are even not possible in standard C language - for example ... As you can see SBCL assumes that first argument is in EDX, second in EDI, and result is in EDX - also it restores the caller state ...
    (comp.lang.lisp)
  • Re: C# Plugin system - same interface in two different assemblies...
    ... easy to see why the .NET designers went with the tried and true approach of just checking a unique identifier for each type. ... This is essentially what you achieve with COM by saying "yes, I implement this interface with this UID too", but then you want it by structural compatibility, not an agreed-upon UID. ... I'm not sure what you mean, as GCC, Intel, and Microsoft's compilers all produce compatible plugins for several products in our product suite. ... As long as you don't get subtle with multiple inheritance, method pointers or exceptions, those compilers will all produce the same low-level binary stuff to make sure it'll work with the other compilers. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Returning structs
    ... The C standard requires that it be supported, ... ABI to make that happen. ... implemented consistently between compilers. ... struct values, but they may use different techniques to do it. ...
    (comp.lang.c)