Planet London Python

April 16, 2014

Ian Ozsvald

2nd Early Release of High Performance Python (we added a chapter)

Here’s a quick book update – we just released a second Early Release of High Performance Python which adds a chapter on lists, tuples, dictionaries and sets. This is available to anyone who has bought it already (login into O’Reilly to get the update). Shortly we’ll follow with chapters on Matrices and the Multiprocessing module.

One bit of feedback we’ve had is that the images needed to be clearer for small-screen devices – we’ve increased the font sizes and removed the grey backgrounds, the updates will follow soon. If you’re curious about how much paper is involved in writing a book, here’s a clue:

We announce each updates along with requests for feedback via our mailing list.

I’m also planning on running some private training in London later in the year, please contact me if this is interesting? Both High Performance and Data Science are possible.

In related news – the PyDataLondon conference videos have just been released and you can see me talking on the High Performance Python landscape here.


Ian applies Data Science as an AI/Data Scientist for companies in Mor Consulting, founded the image and text annotation API Annotate.io, co-authored SocialTies, programs Python, authored The Screencasting Handbook, lives in London and is a consumer of fine coffees.

by Ian at April 16, 2014 08:11 PM

April 13, 2014

Sylvain Hellegouarch

Having fun with WebSocket and Canvas

Recently, I was advised that WebFaction had added support for WebSocket in their custom applications by enabling the according nginx module in their frontend. Obviously, I had to try it out with my own websocket library: ws4py.

Setting up your WebFaction application

Well, as usual with WebFaction, setting up the application is dead simple. Only a few clicks from their control panel.

Create a Custom application and select websocket. This will provide you with a port that your backend will be bound to. And voilà.

Now, your application is created but you won’t yet be able to connect a websocket client. Indeed, you must associate a domain or subdomain with that application.

It is likely your application will be used from a javascript connector in living in a browser, which means, you will be bound by the browser same-origin security model. I would therefore advise you to carefully consider your sub-domain and URL strategies. Probably something along:

  • http://yourhost/ : for the webapp
  • http://yourhost/ws : as the base url for all websocket endpoints

This is just a suggestion of course but this will make it easier for your deployment to follow a simple strategy like this one.

In the WebFaction control panel, create a website which associates your web application with the domain (your webapp can be anything you need to). Associate then your custom websocket application with the same domain but a different path segment. Again, by sharing the same domain, you’ll avoid troubles regarding working around the same-origin security model. I would probably advise as well that you enable SSL but it’s up to you to make that decision.

Once done, you will have now a configured endpoint for your websocket application.

The Custome websocket application will forward all requests to you so that you can run your web and websocket apps from that single port. This is what the demo below allows itself doing. I would recommend that you run two different application processes, one for your webapp and another one for your websocket endpoint. Be antifragile.

Drawing stuff collaboratively

I setup a small demo (sorry self-signed certificate) to demonstrate how you can use HTML5 canvas and websocket features to perform collaborative tasks across various connected clients.

That demo runs a small webapp that also enables a websocket endpoint. When you are on the drawing board, everything you draw is sent to other connected clients so that their view reflects what’s happening on yours. Obviously this goes in any way frm any client to any other clients.

The demo is implemented using ws4py hosted within a CherryPy application. Drawing events are serialized into a json structure and sent over to the server which dispatches them to all participants of that board, and only that board (note, this demo doesn’t validate the content before dispatching back so please conservative with whom you share your board with).

Open the link found in the demo page and share it on as many browsers as you can (including your mobile device). Starting drawing from one device will make all other devices been drawn onto simultaneously and synchronously. Note that the board stays available for up to 5mn only and will disconnect all participants then.

The source code for the demo is located here.

Some feedback…

Let me share a bit of feedback about the whole process.

  • WebSockets are finally a reality. Unless you’re running an old browser or old mobile platform, RFC6455 is available to you.  This means, you can really leverage the power of push from the server. Mind you, you might want to look at Server-Side Event  as well.
  • There isn’t yet a clear understanding on how to properly configure your server resources. In my demo, the whole webapp also hosts the websocket app but this is probably not a good idea if you have a large amount of connected clients or intensive work done server side. Even though the initial connection is initiated from a HTTP request, I would suggest the websocket server is disconnected from the HTTP server process.
  • Security wise, I would suggest you follow the usual principles of validating that any data coming through before you process or dispatch them back.
  • WebFaction supports for websocket is dead easy to setup and fast (at least, since my demo is hosted in Europe and I live in France, I almost cannot see any delay). I would consider their performances good enough to support some really funky real-time applications.
  • jCanvas is really useful to unify your canvas operations.  For this demo, it’s been a blessing.
  • Device motion events are low level and you need to do a lot of leg work to actually make sense of them. This demo is probably not making a really good use of them.
  • There seems to be no universal way to detect that you are running on a mobile device. Go figure.

Next, I wouldn’t mind adding websocket to that fun demo from the mozilla developer network.

by Sylvain Hellegouarch at April 13, 2014 02:57 PM

April 12, 2014

Steve Holden

Intermediate Python: An Open Source Documentation Project

There is a huge demand for Python training materials, and there are many people who just don't have the spare cash to buy books or videos. That's one reason why, in conjunction with a new Intermediate Python video series I have just recorded for O'Reilly Media I am launching a new, open source, documentation project.

My intention in recording the videos was to produce a broad set of materials (the linked O'Reilly page contains a good summary of the contents). I figure that most Python programmers will receive excellent value for money even if they already know 75% of the content. Those to whom more is new will see a correspondingly greater benefit. But I was also keenly aware that many Python learners, particularly those in less-developed economies, would find the price of the videos prohibitive.

With O'Reilly's contractual approval the code that I used in the video modules, in IPython Notebooks, is going up on Github under a Creative Commons license. Some of it already contains markdown annotations among the code, other notebooks have little or no commentary. My intention is that ultimately the content will become more comprehensive than the videos, since I am using the video scripts as a starting point.

I hope that both learner programmers and experienced hands will help me turn it into a resource that groups and individuals all over the world can use to learn more about Python with no fees required. The current repository has to be brought up to date after a rapid spate of editing during the three-day recording session. It should go without saying that viewer input will be very welcome, since the most valuable opinions and information comes from those who have actually tried to use the videos to help them learn.

I hope this will also be a project that sees contributions from documentation professionals (and beginners they can help train), so I will be asking the WriteTheDocs NA team how we can lure some of those bright minds in.

Sadly it's unlikely I will be able to see their talented array of speakers as I will still be recovering from surgery. But a small party one evening or a brunch at the office might be possible. Knowing them it will likely involve sponsorship or beer. Or both. We shall see.

I think it's a worthwhile goal to have free intermediate-level Python sample code available, and I can't think of a better way for a relative beginner to get into an open source project. I also like the idea that two communities can come together over it and learn from each other. Suffice it to say, if there are enough people with a hundred bucks* in their pocket for a six-hour video training I am happy to use part of my share in the profits to support this project to some degree.

[DISCLOSURE: The author will receive a proportion of any profit from the O'Reilly Intermediate Python video series]

* This figure was plucked from the air before publication, and is still a good guideline, though as PyCon opened (Apr 11) a special deal was available on a package of both Jessica McKellar's Introduction to Python and my Intermediate Python.

by Steve (noreply@blogger.com) at April 12, 2014 03:22 AM

A Rap @hyatt Customer Service Request

It's 2am and the wireless is down
@Hyatt ... #pycon
That's why my face is wearing a frown
Even though I'm at ... #pycon

I love all these Canadians
And Montreal is cool
But don't you know how not to run a network
fool?

If I were a rapper
Then you'd have to call me Milton
Because frankly I get much better service
@Hilton

I'm a businessman myself
And I know we're hard to please
So kindly please allow me
To put you at your ease

Your people are delightful
And as helpful as the best
I want to help, not diss you
I'm not angry like the rest

The food is amazing
And the bar could be geek heaven
If only you weren't calling
For last orders at eleven

We're virtual and sleepless
So we need your help to live
And most of us are more than glad
To pay for what you give

But imagine you're away from home
And want to call your Mom
The Internet's our family
So you've just dropped a bomb

I've had my ups and downs with Hyatt
Over many years
But never felt before
That it should fall on other's ears

I run conferences, for Pete's sake
And I want to spend my money
If only I could reach someone
And I'm NOT being funny

PyCon is my baby
So I cherish it somewhat
But this has harshed my mellow
And just not helped a lot

We're bunch of simple geeks
Who get together every year
We aren't demanding, I don't think
Our simple needs are clear

I don't believe that I could run
Your enterprise right here
It's difficult, and operations
Aren't my thing I fear

So please, don't take this badly
But you've really disappointed
Which is why a kindly soul like me
Has made remarks so pointed

We will help you if we can
We know you pay a lot for bits
But I have to know if web sites
Are receiving any hits

You've cut me off, I'm blind
And so I hope there's nothing funky
Happening to my servers
While I'm sat here getting skunky

Enough, I've made my point
So I must stop before I'm rude
The Internet's my meat and drink
You've left me without food.

trying-to-help-while-disappointed-ly yr's  - steve

by Steve (noreply@blogger.com) at April 12, 2014 02:59 AM

Peter Bengtsson

COPYFILE_DISABLE and python distutils in python 2.6

My friend and colleague Jannis (aka jezdez) Leidel saved my bacon today where I had gotten completely stuck.

So, I have this python2.6 virtualenv and whenever I ran python setup.py sdist upload it would upload a really nasty tarball to PyPI. What would happen is that when people do pip install premailer it would file horribly and look something like this:

...
IOError: [Errno 2] No such file or directory: '/path/to/virtual-env/build/premailer/setup.py'

What?!?! If you download the tarball and unpack it you'll see that there definitely is a setup.py file in there.

Anyway. What happens, which I didn't realize was that within the .tar.gz file there were these strange copies of files. For example for every file.py there was a ._file.py etc.

Here's what the file looked like after a tarball had been created:

(premailer26)peterbe@mpb:~/dev/PYTHON/premailer (master)$ tar -zvtf dist/premailer-2.0.2.tar.gz
-rwxr-xr-x  0 peterbe staff     311 Apr 11 15:51 ./._premailer-2.0.2
drwxr-xr-x  0 peterbe staff       0 Apr 11 15:51 premailer-2.0.2/
-rw-r--r--  0 peterbe staff     280 Mar 28 10:13 premailer-2.0.2/._LICENSE
-rw-r--r--  0 peterbe staff    1517 Mar 28 10:13 premailer-2.0.2/LICENSE
-rw-r--r--  0 peterbe staff     280 Apr  9 21:10 premailer-2.0.2/._MANIFEST.in
-rw-r--r--  0 peterbe staff      34 Apr  9 21:10 premailer-2.0.2/MANIFEST.in
-rw-r--r--  0 peterbe staff     280 Apr 11 15:51 premailer-2.0.2/._PKG-INFO
-rw-r--r--  0 peterbe staff    7226 Apr 11 15:51 premailer-2.0.2/PKG-INFO
-rwxr-xr-x  0 peterbe staff     311 Apr 11 15:51 premailer-2.0.2/._premailer
drwxr-xr-x  0 peterbe staff       0 Apr 11 15:51 premailer-2.0.2/premailer/
-rwxr-xr-x  0 peterbe staff     311 Apr 11 15:51 premailer-2.0.2/._premailer.egg-info
drwxr-xr-x  0 peterbe staff       0 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/
-rw-r--r--  0 peterbe staff     280 Mar 28 10:13 premailer-2.0.2/._README.md
-rw-r--r--  0 peterbe staff    5185 Mar 28 10:13 premailer-2.0.2/README.md
-rw-r--r--  0 peterbe staff     280 Apr 11 15:51 premailer-2.0.2/._setup.cfg
-rw-r--r--  0 peterbe staff      59 Apr 11 15:51 premailer-2.0.2/setup.cfg
-rw-r--r--  0 peterbe staff     280 Apr  9 21:09 premailer-2.0.2/._setup.py
-rw-r--r--  0 peterbe staff    2079 Apr  9 21:09 premailer-2.0.2/setup.py
-rw-r--r--  0 peterbe staff     280 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/._dependency_links.txt
-rw-r--r--  0 peterbe staff       1 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/dependency_links.txt
-rw-r--r--  0 peterbe staff     280 Apr  9 21:04 premailer-2.0.2/premailer.egg-info/._not-zip-safe
-rw-r--r--  0 peterbe staff       1 Apr  9 21:04 premailer-2.0.2/premailer.egg-info/not-zip-safe
-rw-r--r--  0 peterbe staff     280 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/._PKG-INFO
-rw-r--r--  0 peterbe staff    7226 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/PKG-INFO
-rw-r--r--  0 peterbe staff     280 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/._requires.txt
-rw-r--r--  0 peterbe staff      23 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/requires.txt
-rw-r--r--  0 peterbe staff     280 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/._SOURCES.txt
-rw-r--r--  0 peterbe staff     329 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/SOURCES.txt
-rw-r--r--  0 peterbe staff     280 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/._top_level.txt
-rw-r--r--  0 peterbe staff      10 Apr 11 15:51 premailer-2.0.2/premailer.egg-info/top_level.txt
-rw-r--r--  0 peterbe staff     280 Apr  9 21:21 premailer-2.0.2/premailer/.___init__.py
-rw-r--r--  0 peterbe staff      66 Apr  9 21:21 premailer-2.0.2/premailer/__init__.py
-rw-r--r--  0 peterbe staff     280 Apr  9 09:23 premailer-2.0.2/premailer/.___main__.py
-rw-r--r--  0 peterbe staff    3315 Apr  9 09:23 premailer-2.0.2/premailer/__main__.py
-rw-r--r--  0 peterbe staff     280 Apr  8 16:22 premailer-2.0.2/premailer/._premailer.py
-rw-r--r--  0 peterbe staff   15368 Apr  8 16:22 premailer-2.0.2/premailer/premailer.py
-rw-r--r--  0 peterbe staff     280 Apr  8 16:22 premailer-2.0.2/premailer/._test_premailer.py
-rw-r--r--  0 peterbe staff   37184 Apr  8 16:22 premailer-2.0.2/premailer/test_premailer.py

Strangly, this only happened in a Python 2.6 environment. The problem went away when I created a brand new Python 2.7 enviroment with the latest setuptools.

So basically, the fault lies with OSX and a strange interaction between OSX and tar.
This superuser.com answer does a much better job explaining this "flaw".

So, the solution to the problem is to create the distribution like this instead:

$ COPYFILE_DISABLE=true python setup.py sdist

If you do that, you get a healthy lookin tarball that actually works to pip install. Thanks jezdez for pointing that out!

April 12, 2014 12:39 AM