Saturday, April 16, 2011

Introduction

PyJS, which is part of Pyjamas package, is some very under-appreciated tool in Python world, and generally is not recognized as 'python implementation' and often not considered as separate from widget set.
This is partially due to lack of documentation and clear API for use-cases other than Pyjamas itself. Translator class has rigid templates, limiting its applications, and Linker/Builder are very pyjamas-centric, pulling in DOM/Widget libraries by default right now.

Moreover, right now translator lacks ability to compile itself, and do compilation in run-time independent of CPython (or other base implementation), making it look far less capable, than it actually is.

As I'm one of those who are interested in PyJS itself rather than whole Pyjamas suite, I felt that GSoC would be a great opportunity to fix these issues and bring some public attention to PyJS as stand-alone tool, capable of running arbitrary python on javascript VMs like Google V8 and SpiderMonkey, having potential to be even faster than CPython thanks to advanced JIT of these platforms.

While this may sound as big undertaking, most of needed infrastructure is already there - we have pure-python python parser, translator itself is pure python as well, and road to making it independent of CPython is straight.

Improving API and supporting statement/expression-level compilation is harder, as currently translator lacks globals()/locals() support, and implementing it isn't trivial, but definitely possible.

Another step would be to update and improve CLI utilities we ship, and include typical 'python' utility to make it familiar for non-pyjamas users. I've already worked on this part, but API update for translator/linker should come first. While I'm at this, I would also need to add tests for CLI tools themself, as breaking one of them and not noticing is too common right now.

Aside from confidence increase and improvement of public image, there would be enough direct benefits for Pyjamas as well:
  • Ability to do compilation in run-time could let us send raw python code to the browsers, significantly reducing total javascript size (up to 10 times).
  • Improvement and simplification of widgets development, bringing full power of python introspection and interactivity to browsers environment.
Another goal is to evaluate ability to support python 3 syntax for parser and translator, and ability to do initial compilation with python 3 interpreters.
While its too early to talk about full python 3 support for Pyjamas, as that would take too much time and would be too hard to maintain afterwards, parser and translator themselves are much more feasible target for GSoC.
Since we are translating python into not-so-pythonic javascript, its possible to translate most of python 3 into same javascript with (mostly) same pyjslib, with some prior AST transformations.
3to2 project achieved a lot in figuring what can be translated and what not, so first step here would be to study their findings and see what else could be resolved by python-to-javascript translation.
Most likely we could not support all of python 3 at once, but bottom line is to document all changes needed for full support, and keep python 3 support in mind while re-designing translator, so we don't end up dead-locked with python 2 later.

5 comments:

  1. Well, PyJS is not a python interpreter because it misses important python semantics. That way it'll never be considered that. Am I missing something?

    ReplyDelete
  2. @Maciej Fijalkowski,
    There are several different compilation modes and 'strict' mode is intended to be fully compliant to python specification.
    There are still some things missing right now (including run-time compilation), but I would not say 'never', reaching full compliance is one of project goals.

    ReplyDelete
  3. How far is 'strict' mode from being complete? does it support sys.exc_info? Will it ever support? New style classes with all the complicated pieces of descriptor protocol?

    Is the goal for 'strict' mode to be fast, or the goal is for some other mode to be fast and strict to be strict?

    Feel free to direct me to mailing list dicussion, docs or whatever other sources

    ReplyDelete
  4. > does it support sys.exc_info?

    Yes, http://pyjs.org/pygit/#file=pyjs/src/pyjs/lib/sys.py&id=99d68fe50c1dace8a0b0f3cd53d77a34deca69fb&mimetype=text-x-python

    > descriptor protocol?

    To some extent. Code is there in pyjslib, but I'm not sure how compliant it is. We do support @property, but AFAIK do not support slots.

    > Is the goal for 'strict' mode to be fast, or the goal is for some other mode to be fast and strict to be strict?

    There is -O which is performance-oriented, and --strict is intended to be compliant at the cost of performance.

    As I've mentioned in the post, docs on translator are very sparse, you have to look at the code to check details, but there is overview of features, http://pyjs.org/wiki/migrationguide/
    Although that one is a bit old, describing version 0.6 and we are at version 0.8 now.

    ReplyDelete
  5. oh wow, pyjamas moved a lot since I looked last time.

    ReplyDelete