Goals of Project Phoenix
The high level goals and driving forces of this project are:
- Generate as much as possible of the bindings code (or rather the input to the bindings generator) from the "interface headers" used for the wx Doxygen documentation. (Or more likely just use the XML output that doxygen can produce.)
- The intent here is to drastically reduce the work-load required to maintain wxPython.
- Currently there is much by-hand work needed to add new classes/methods to wxPython or to adapt to other changes in the C++ code.
- Starting from the wx interface headers will remove most of the burden of that work from Robin and evenly distribute it across all who contribute to the C++ code because those developers will be (or should be) updating the interface headers for the documentation anyway.
- When the system is fully implemented then many of the additions or changes made to the C++ code (and accurately represented in the interface headers) will require no additional work for wxPython other than possibly adding just a few lines of code and then triggering a rebuild.
Allow for the overriding and tweaking of the info coming from the Doxygen XML as needed -- after the XML had been loaded and processed -- for implementing wxPython-specific customizations.
- By making the Python-specific customizations a separate step that is allowed to override and augment what is produced from the interface headers then it reduces or eliminates the need for special cases and messy confusing code that might effect the production of the C++ docs or the willingness of the C++ developers to continue to maintain it.
- Allow for having more than one code generator that operates on the tweaked data structures coming from the Doxygen XML information. This will enable easier experimentation with different backends, and also could allow for other language bindings to use the same front-end tools.
- Another major goal for this project is the automatic generation of a set of wxPython-specific docs using the C++ docs (or the Doxygen XML again) as a base, allowing the overriding of C++ documentation snippets as needed to make them more Python specific, and the same design and implementation patterns used for the wrappers will be used here.
- Sphinx is the current favorite for the doc generator, so the tools developed for Phoenix will generate reference docs using sphinx's restructured text input format.
- Think real hard about dumping SWIG and using something else for generating the bindings wrapper code.
SIP looks very, very nice
- Many of the kludges and work-arounds I did with SWIG are already implemented and much less kludgy.
- SIP produces Python extension types rather than just a collection of functions that are glued together into classes later in Python code, so no "shadow" classes (and the overhead they impose) are necessary. One downside to this is that static source analyzers such as IDEs won't have anything to analyze but there are probably some workarounds for that which could be done. Another downside is that I've become very used to inserting snippets of Python code into the code generated by SWIG in order to enhance the API or to work around problems.
- By not supporting a dozen or so target languages SIP is able to put much more effort into perfecting the code generated for Python wrappers and making it very efficient.
- Generated wrapper code is clean and readable.
- Python3 is supported.
- The SIP project has already accepted and applied a patch I've contributed (keyword argument support) and Phil has added, or is adding, other features that I've requested.
- Pyrex (or Cython) would be cool, but probably only if we intended to still be doing lots of by-hand work, so the first goal above eliminates this as a possibility. Also there still seems to be some big holes or complexities with wrapping C++ classes with these tools, such as allowing virtual method calls to be reflected into overrides implemented in a derived class in Python.
- Anthony Baxter recommended generating a wxC library and then using ctypes for the Python integration as being the easiest path to supporting both Python 2.6+ and Python 3. Although this would also be beneficial for other wx language bindings, it just doesn't feel right to me. Also, it would likely have even more runtime overhead than what we have now.
- We'll have enough information to skip 3rd party code generators and do it all ourselves if desired, probably not worth the effort however. If we've already got a good enough wheel then there is very little need to reinvent it. (Maybe just add a few more spokes.)
- If we stay with SWIG then at the very least we should try to remove all need for the patched SWIG and move back to using the stock SWIG tool and the newest version.
Another SWIG option is to write a new code generation module for it that builds Python extension types like SIP does. Could use PWIG (http://pwig.sourceforge.net/) to write that code generator in Python instead of C. This would be another reinventing the wheel solution, but the wheel could be exactly what we need it to be.
- Define a set of standards for the naming, coding and documenting of the modules in wx.lib and recruit somebody (or multiple somebodies) to migrate those modules to the new coding standards, and also adapt to the new wxPython APIs as needed.
- Like Python 3, eliminate all the old cruft and things that I've regretted for a long time. Try to maintain most backwards compatibility, but don't insist on it. For example, here are some possibilities:
- Remove most or all of the aliases that I've added over the years that seemed like a good idea at the time but that really just add clutter.
Don't wrap wxDateTime at all, just provide typemaps that convert to/from Python datetime objects. (Python's datetime class didn't exist at the time wx.DateTime was added to wxPython.)
- Do away with widget IDs? (Actually hide them in the wxPython API so the programmers never need to see them or deal with them, unless they want to.) Provide something for stock objects created from stock IDs. This will be welcomed by many, but may be too big of a compatibility break...
- Be able to more cleanly handle overridable virtual methods without so much maintenance work. (SIP does this well, and SWIG has improved its directors feature.)
- Can we avoid the need to rename overloaded methods without losing keyword args ability? (We can with SIP, but SWIG 2.0.1 still has this limitation.)
- Type safety for C++ enums from the Python code? (the "Pythonic" evangelists will not like this)
- Propagate Python exceptions up the stack through the C++ layers
- Handle the GIL better. (Don't release it unless the wrapped API will block or may result in a callback.)
- Use unicode only, no auto converts of string objects. Or maybe just assume that any string objects are in utf-8 and only auto convert using that encoding.
- Think about getting rid of wxversion and multi-version installs, use eggs instead?
- Make better use of the Python buffer interface for typemaps that are dealing with "data" rather than "strings" in wxImage, DnD, etc. Use the new buffer protocol for allowing lists of points, etc. to come from buffer objects such as numpy arrays.
- Phase out the wx.gizmos and any other modules/classes that are not well supported on the C++ side.
- Clean out old and unmaintained modules in wx.lib.
- Blend wxLog and Python logging so the programmers only need to deal with Python logging
- Should the getters and setters used for properties be hidden (or renamed with a leading '_') so only the property itself will be part of the public API? (I like the idea but this might be too much of a compatibility break...)
- And so on...