I had something I wanted to try in Python running against Yahoo! Messenger. The obvious choice of library for talking to Yahoo! Messenger was libyahoo2. I could not find a Python binding for it, so I started sketching one together with SWIG. The first step is creating a bunch of empty callbacks. The libyahoo2 in Ubuntu’s package is compiled without USE_CALLBACK_STRUCT, so libyahoo2 expects to find a bunch of extern functions defined to interact with the host. I made empty callbacks in a C file and started reading more about the API.
It rapidly became clear that I was going to want a layer on top of the library to make interacting with it from Python more palatable. I switched to Pyrex, since I wanted to write that wrapper in Python (or something Python-like) rather than building a straight-C wrapper so using SWIG would continue to make sense. SWIG’s big benefit in my mind over Pyrex is easy support for more languages and better tools for defining straight wrappers. I wasn’t going to get "free" use in other languages and it wasn’t going to be a straight wrapper now but rather a module that exposed the functionality of libyahoo2 to Python.
I kept the callbacks.c I had defined but started migrating the definitions to the Pyrex file as I implemented. This way my library would continue to link without complaint about functions I didn’t have yet.
Following the usual pattern for Python bindings to libraries that need wrappers to be more Pythonic, I planned to have a ‘yahoo2′ module in Python and a ‘_yahoo2.so’ extension module. The _yahoo2 module is written in Pyrex.
libyahoo2 appears not to adhere to the documentation it defines. It’ll call ext_yahoo_remove_handler with a tag that was never returned by ext_yahoo_add_handler… (0). It looks like an undocumented (that I found) part of the charter of ext_yahoo_add_handler is not to add a given handler more than once.
This also made defining the callbacks the way libyahoo2 expected easier.
D’oh, I got it far enough along to get this:
libyahoo2.c:620: debug: Key: 4 Value: Yahoo_Messenger
libyahoo2.c:620: debug: Key: 5 Value: hodorbot
libyahoo2.c:620: debug: Key: 14 Value: This version of Messenger expired on April 2, 2008. Please upgrade now to the latest supported version: http://messenger.yahoo.com Learn more: http://messenger.yahoo.com/eol
libyahoo2.c:620: debug: Key: 15 Value: 1225167367
libyahoo2.c:620: debug: Key: 97 Value: 1
I’ve learned what I wanted to, so instead of seeing if a more recent libyahoo2 than what’s in Ubuntu works (> 0.7.5+dfsg-3), I’m just going to call it here. I’ll post it in case someone can learn something useful from it.
You can pull a working repository with a command like:
git clone http://notes.xythian.net/media/2008/10/pythonlibyahoo2.git/ mydirectory
There’s three files that do anything:
- yahoo2.py – wraps some of the lower level details from the Pyrex layer, including exposing the IO bits as an asyncore dispatcher
- _yahoo2.pyx – is the binding
- callbacks.c – exists to have empty functions defined to satisfy the linker until those have implementations in the Python binding
Not much works, really; there’s implementations of connect and async_connect, but only async_connect is called by libyahoo2 before it gets far enough along to not log in with the error message above. There’s a wrapper for setting the log level (which was key to discovering the above fact…).
This is a typical definition of one of the library callbacks:
cdef public int ext_yahoo_connect_async(int id, char *host, int port, \
yahoo_connect_callback callback, void *callback_data):
cdef ConnectionHandle handle
handle = ConnectionHandle(id, host, port)
handle.connect_callback = callback
handle.connect_data = callback_data
handle.async_connect(callback, callback_data)
HANDLER_MAP[id].connections.append(handle)
MANAGER.add(handle)
return handle.fileno()
ConnectionHandle is an extension class which wraps a Python socket object and can make the callbacks the libyahoo2 IO layer expect. MANAGER is the connection manager, which is implemented in the higher level layer (yahoo2.py) so it could be replaced with something that interacted with a GUI event loop.
I’m probably done with this for the forseeable future and hope publishing it along with a git repoistory may allow someone to build on or learn from what I’ve done. I stumbled upon Cython while working on this, but didn’t want to derail any progress to something working by playing with it.
So what did I learn?
- Write toy bots against open protocols with existing libraries in your language of choice, such as XMPP, lest your project get hijacked working on a binding rather than the toy.
- Pyrex is pretty nice for building wrappers that have more meat than a typical SWIG binding.
- Cython is probably worth checking out.
- git is worth playing with more