Planet London Python

From time to time, the idea of a standard Python “Enum” object is raised on the Python lists. You know the kind of thing: a lightweight means of mapping numbers to labels so you can do set_colour(Colours.red) without having a long module of manifest constants or magic numbers lying around all over your codebase.

It all sounds very straightforward, and Barry Warsaw had an existing module which seemed like a fairly good starting point, so PEP 435 was started and it all looked like it was just a formality.

Now, literally *hundreds* of mailing list posts and endless, endless threads later, GvR has just pronounced his approval of the PEP and it’s good to go.

If you — like me — thought “this one won’t be controversial”, then just point your search engine of choice at mail.python.org/pipermail/python-dev and look for “enum” or “435″, or just look at the archive for May alone (which only represents the final few days of details being thrashed out) to realise just how much discussion and work is involved in what appears to be quite a simple thing.

Of course, part of the problem is precisely the fact that the idea is so simple. I’m sure most people have rolled their own version of something like this. I know I have. You can get up and running with a simple “bunch” class, possibly throw in a few convenience methods to map values to names and then just get on with life. But when something’s got to go into the stdlib then it all becomes a lot more difficult, because everyone has slightly (or very) different needs; and everyone has slightly (or very) different ideas about what constitutes the most convenient interface.

And there’s always the danger of the “bikeshed” effect. If a PEP is proposing something perhaps quite fundamental but outside most people’s experience, then only people with sufficient interest and knowledge are likely to contribute. (Or argue). But an enum: everyone’s done that, and everyone’s got an interest, and an idea about how it should be done.

But, bikesheds aside, I’m glad that the Python community is prepared to refine its ideas to get the best possible solution into the standard library. As a developer, one naturally feels that one’s own ideas and needs represent everyone else’s. It’s only when you expose your ideas to the sometimes harsh winds of the community critique that you discover just how many different angles there are to something you thought was simple.

Thankfully, we have a BDFL (or, sometimes, a PEP Czar) to make the final decision. And, ultimately, that means that some people won’t see their needs being served in the way they want. But I think that that’s far preferable to a design-by-committee solution which tries to please everybody and ends up being cluttered.

Having remained silent since PyCon, Adria Richards has now published a statement reflecting on those events, though sensibly without going into specifics. I think it's important that her retrospective view gets some air, so I am reproducing it here with my own comments.

Those who know me well in the the developer and tech community recognize that I have always tried to conduct myself in a way that builds bridges for everyone. My central aim is to do everything I can to help create new, inclusive inroads for all, no matter who they are, where they come from or what they believe. Development is about innovation, creativity, and in a grand sense, the betterment of human society through technology. So, it stands to reason that everyone should have a seat at the table, and everyone involved in this vital community should feel welcome, safe and respected. In essence, the worldwide community of developers can and should function as a reflection of what our wider society strives to be.

I don't know Adria, so I take her opening statement at face value. Those who disagree may do so elsewhere. Garann Means recently made a fairly good case that the denizens of the technology world don't act in a way that inspires respect from other communities. I think it's fair to say (and some of the comments in forums like Hacker News amply demonstrate) that there is plenty of room for improvement in what's considered acceptable conduct in the technology world.

I cannot comment at this time on the specifics of what occurred at PyCon on March 17, and the subsequent events of the following days, but I can offer some general thoughts. I don’t think anyone who was part of what happened at PyCon that day could possibly have imagined how this issue would have exploded into the public consciousness the way it has. I certainly did not, and now that the severest of consequences have manifested, all I wish to do is find the good in what has been one of the most challenging weeks of my life.

This positive attitude is to be admired. I have been through similarly shitty situations in my own life (as most of us have at one time or another) and while it's good to move on I like to extract lessons that will help me try to avoid making the same mistakes. If I can find those I can feel more positive about the experience.

And I do believe there is good to be found in this situation. Debate and recrimination can and must give way to dialog that explores the root causes of these issues in the tech industry. As developers and members of the startup community, we can welcome newcomers, women and people of color who, as of now, are under-represented in our ranks. And, all of us can learn a great deal from those who are well-established in the field. We can solidify the values of our workplaces (yes, conference spaces are workplaces!), and set new, positive and inclusive examples for other professional disciplines.

This is a good idea. PyCon was started to try and provide an inclusive space where everyone with interests in Python can come together for the betterment of all. I would be suspicious of those who don't support the above as an ideal. We surely have to accept that there's still along way to go to get there. So dialog is a good idea.

What happened at PyCon has cast a spotlight on a range of deep issues and problems in the developer world. As ugly as this situation has become, all of these issues have reasonable, and, I think, easily reached solutions that will help us cast conflict aside and construct a more cohesive and welcoming professional environment based on respect, trust and open communication. I do not, at this time, wish to concentrate on the fallout of the last several days. Instead, I want to be an integral part of a diverse, core group of individuals that comes together in a spirit of healing and openness to devise answers to the many questions that have arisen in the last week. Together, we can work to make the tech world a better place to work for everyone, and in doing so, we make the wider world a better place for all.

It's interesting that this episode happened at PyCon. When we started to talk seriously about Codes of Conduct in the Python world there was a lot of kick-back, and people said things like "but this is the Python community" (implying that we are such nice people that we won't have the same issues) and "but that's just normal behavior" (not even implying, but specifically stating, that anyone who needed a Code of Conduct to condition their behavior shouldn't be going to conferences anyway).

We pressed ahead none the less, refining the code to improve its expression of our ideals of openness and inclusivity. When issues had to be addressed (and you can read the reports in the PyCon blog if you need to refresh your memory) the code of conduct was applied, and to my mind it worked. The fact that two people lost their jobs as a result of the fall-out was nothing to do with the code of conduct, and yet there were many who felt that the blame should be laid at PyCon's door. I shudder to think what the result would have been at an event with no explicit code.

I hope that Adria can come back to PyCon without being subjected to any hostility. She expresses a great ideal, one that we should all be striving for. Now we just have to work out how to achieve it. But that's an implementation detail, right? I wish!

As I've mentioned in previous blog articles, my wife and I have been working on driving an analog VU meter based on the sound going out the Raspberry Pi's audio outputs. This now works!

Here's a video demonstrating the result:

The music [1] is playing from a Raspberry Pi, with software running on the Pi digitally sampling the peak output audio level and writing that out to an 8-bit digital-to-analog converter (DAC). The DAC output is then used to drive the analog meter. If you're interesting in knowing how all this hangs together, keep reading.

Read more...

Having returned from Chile last year, settled in to consulting in London, got married and now on honeymoon I’m planning on a change for June.

I’m taking the month off from clients to work on my own project, an open sourced brand disambiguator for social media. As an example this will detect that the following tweet mentions Apple-the-brand:
“I love my apple, though leopard can be a pain”
and that this tweet does not:
“Really enjoying this apple, very tasty”

I’ve used AlchemyAPI, OpenCalais, DBPedia Spotlight and others for client projects and it turns out that these APIs expect long-form text (e.g. Reuters articles) written with good English.

Tweets are short-form, messy, use colloquialisms, can be compressed (e.g. using contractions) and rely on local context (both local in time and social group). Linguistically a lot is expressed in 140 characters and it doesn’t look like”good English”.

A second problem with existing APIs is that they cannot be trained and often don’t know about European brands, products, people and places. I plan to build a classifier that learns whatever you need to classify.

Examples for disambiguation will include Apple vs apple (brand vs e.g. fruit/drink/pie), Seat vs seat (brand vs furniture), cold vs cold (illness vs temperature), ba (when used as an abbreviation for British Airways).

The goal of the June project will be to out-perform existing Named Entity Recognition APIs for well-specified brands on Tweets, developed openly with a liberal licence. The aim will be to solve new client problems that can’t be solved with existing APIs.

I’ll be using Python, NLTK, scikit-learn and Tweet data. I’m speaking on progress at BrightonPy and DataScienceLondon in June.

Probably for now I should focus on having no computer on my honeymoon…