Planet London Python

April 21, 2014

Steve Holden

Neat Notebook Trick

I'm still trying to digest the little I saw of PyCon. Sadly I was pretty wiped out by the three-day recording session for Intermediate Python in California and then the intensive editing work that followed to help the amazing O'Reilly team get the whole thing ready a day before PyCon officially opened.

I also made the tactical (and, as it turned out, strategic) mistake of choosing to stay at the Hyatt in Montreal. This meant a considerable walk (for a gimpy old geezer such as myself) to the conference site, when the Palais des Congres is already intimidatingly large.

So the combination of exhaustion and knee pain meant I hardly got to see any talks (not totally unheard of) but that I also got very little time in the hallway track either. Probably the most upsetting absence was missing the presentation of Raymond Hettinger's Lifetime Achievement Award. As a PSF director I instituted the Community Service Awards, but these have never really been entirely appropriate for developers. This award makes it much clearer just how significant Raymond's contributions have been.

Because of the video releases I did spend some time of the O'Reilly stand, and signed away 25 free copies of the videos. I was also collecting names and addresses to distribute free copied of the Python Pocket Reference. If you filled out a form, you should receive your book within the next three weeks. We'll mail you with a more exact delivery date shortly.

But the real reason for this post is that I had the pleasure of meeting Fernando Perez, one of the leaders of the IPython project. He was excited to hear that the Intermediate Python notebooks are already available on Github, and when he realized the notebooks were all held in the same directory he showed me that if I dropped that URL into the Notebook Viewer site I would get a web page with links to viewable versions of the notebook. [Please note: they aren't currently optimally configured for reading, so it's still best to run the notebooks interactively, but in the absence of a local notebook server this will be a lot better than nothing. It will get better over time].

He also mentioned a couple of other wrinkles I hadn't picked up on, and we briefly discussed some of the interesting aspects of Notebooks being data structures.

The conversation was interesting enough that I plan to visit Berkeley soon to try and infiltrate my way into the documentation team and see if we can't make the whole system even easier to use and understand. One way or another, open source seems to be in my bloodstream.

by Steve ( at April 21, 2014 02:39 AM

Intermediate Python: An Open Source Documentation Project

There is a huge demand for Python training materials, and there are many people who just don't have the spare cash to buy books or videos. That's one reason why, in conjunction with a new Intermediate Python video series I have just recorded for O'Reilly Media I am launching a new, open source, documentation project.

My intention in recording the videos was to produce a broad set of materials (the linked O'Reilly page contains a good summary of the contents). I figure that most Python programmers will receive excellent value for money even if they already know 75% of the content. Those to whom more is new will see a correspondingly greater benefit. But I was also keenly aware that many Python learners, particularly those in less-developed economies, would find the price of the videos prohibitive.

With O'Reilly's contractual approval the code that I used in the video modules, in IPython Notebooks, is going up on Github under a Creative Commons license [EDIT: The initial repository is now available and I very much look forward to hearing from readers and potential contributors - it's perfectly OK if you just want to read the notebooks, but any comments yuu have about your experiences will be read and responded to as time is available]. Some of it already contains markdown annotations among the code, other notebooks have little or no commentary. My intention is that ultimately the content will become more comprehensive than the videos, since I am using the video scripts as a starting point.

I hope that both learner programmers and experienced hands will help me turn it into a resource that groups and individuals all over the world can use to learn more about Python with no fees required. The current repository has to be brought up to date after a rapid spate of editing during the three-day recording session. It should go without saying that viewer input will be very welcome, since the most valuable opinions and information comes from those who have actually tried to use the videos to help them learn.

I hope this will also be a project that sees contributions from documentation professionals (and beginners they can help train), so I will be asking the WriteTheDocs NA team how we can lure some of those bright minds in.

Sadly it's unlikely I will be able to see their talented array of speakers as I will still be recovering from surgery. But a small party one evening or a brunch at the office might be possible. Knowing them it will likely involve sponsorship or beer. Or both. We shall see.

I think it's a worthwhile goal to have free intermediate-level Python sample code available, and I can't think of a better way for a relative beginner to get into an open source project. I also like the idea that two communities can come together over it and learn from each other. Suffice it to say, if there are enough people with a hundred bucks* in their pocket for a six-hour video training I am happy to use part of my share in the profits to support this project to some degree.

[DISCLOSURE: The author will receive a proportion of any profit from the O'Reilly Intermediate Python video series]

* This figure was plucked from the air before publication, and is still a good guideline, though as PyCon opened (Apr 11) a special deal was available on a package of both Jessica McKellar's Introduction to Python and my Intermediate Python.

by Steve ( at April 21, 2014 02:06 AM

April 18, 2014

Peter Bengtsson

Grymt - because I didn't invent Grunt here

grymt is a python tool that takes a directory full of .html, .css and .js and prepares the html for optimial production use.

For a teaser:

  1. Look at the "input"

  2. Look at the "output" (Note! You have to right-click and view source)

So why did I write my own tool and not use Grunt?!

Glad you asked! The reason is simple: I couldn't get Grunt to work.

Grunt is a framework. It's a place where you say which "recipes" to execute and how. It's effectively a common config framework. Like make.
However, I tried to set up a bunch of recipes in my Gruntfile.js and most of them worked well individually but it was a hellish nightmare to get it all to work together just the way I want it.

For example, the grunt-contrib-uglify is fine for doing the minification but it doesn't work with concatenation and it doesn't deal with taking one input file and outputting to a different file.
Basically, I spent two evenings getting things to work but I could never get exactly what I wanted. So I wrote my own and because I'm quite familiar with this kind of stuff, I did it in Python. Not because it's better than Node but just because I had it near by and was able to quicker build something.

So what sweet features do you get out of grymt?

  1. You can easily make an output file have a hash in the filename. E.g. vendor-$hash.min.js becomes vendor-64f7425.min.js and thus the filename is always unique but doesn't change in between deployments unless you change the files.

  2. It automatically notices which files already have been minified. E.g. no need to minify somelib.min.js but do minify otherlib.js.

  3. You can put $git_revision anywhere in your HTML and this gets expanded automatically. For example, view the source of and look at the first 20 lines.

  4. Images inside CSS get rewritten to have unique names (based on files' modified time) so they can be far-future cached aggresively too.

  5. You never have to write down any lists of file names in soome Gruntfile.js equivalent file

  6. It copies ALL files from a source directory. This is important in case you have something like this inside your javascript code: $('<img>').attr('src', 'picture.jpg') for example.

  7. You can chose to inline all the minified and concatenated CSS or javascript. Inlining CSS is neat for single page apps where you have a majority of primed cache hits. Instead of one .html and one .css you get just one .html and the amount of bytes is the same. Not having to do another HTTP request can save a lot of time on web performance.

  8. The generated (aka. "dist" directory) contains everything you need. It does not refer back to the source directory in any way. This means you can set up your apache/nginx to point directly at the root of your "dist" directory.

So what's the catch?

  1. It's not Grunt. It's not a framework. It does only what it does and if you want it to do more you have to work on grymt itself.

  2. The files you want to analyze, process and output all have to be in a sub directory.
    Look at how I've laid out the files here in this project for example. ALL files that you need is all in one sub-directory called app. So, to run grymt I simply run: grymt app.

  3. The HTML files you throw into it have to be plain HTML files. No templates for server-side code.

How do you use it?

pip install grymt

Then you need a directory it can process, e.g ./client/ (assumed to contain a .html file(s)).

grymt ./client

For more options, check out

grymt --help

What's in the future of grymt?

If people like it and want to add features, I'm more than happy to accept pull requests. Some future potential feature work:

  • I haven't needed it immediately, yet, myself, but it would be nice to add things like coffeescript, less, sass etc into pre-processing hooks.

  • It would be easy to automatically generate and insert a reference to a appcache manifest. Since every file used and mentioned is noticed, we could very accurately generate an appcache file that is less prone to human error.

  • Spitting out some stats about number bytes saved and number of files reduced.

April 18, 2014 09:58 PM

April 16, 2014

Ian Ozsvald

2nd Early Release of High Performance Python (we added a chapter)

Here’s a quick book update – we just released a second Early Release of High Performance Python which adds a chapter on lists, tuples, dictionaries and sets. This is available to anyone who has bought it already (login into O’Reilly to get the update). Shortly we’ll follow with chapters on Matrices and the Multiprocessing module.

One bit of feedback we’ve had is that the images needed to be clearer for small-screen devices – we’ve increased the font sizes and removed the grey backgrounds, the updates will follow soon. If you’re curious about how much paper is involved in writing a book, here’s a clue:

We announce each updates along with requests for feedback via our mailing list.

I’m also planning on running some private training in London later in the year, please contact me if this is interesting? Both High Performance and Data Science are possible.

In related news – the PyDataLondon conference videos have just been released and you can see me talking on the High Performance Python landscape here.

Ian applies Data Science as an AI/Data Scientist for companies in Mor Consulting, founded the image and text annotation API, co-authored SocialTies, programs Python, authored The Screencasting Handbook, lives in London and is a consumer of fine coffees.

by Ian at April 16, 2014 08:11 PM