I love Python, I really do. It has clear syntax, a nice library support and I feel very productive using it.
But Python is an interpreted language and it’s slow. You don’t feel it until you do something stupid like going through all the pixels in a bitmap and converting them into gray scale. I tried doing that once to create grayed out versions of my button images in a wxPython app I was building. I couldn’t belive how slow that was. Then again, the slowness might have been due to the calls to the wxWidgets C++ layer and back.
Anyway..
I was completely unaware of the fact that the guys at YouTube, the mega site that serves millions of users daily, use Python. And they don’t just use it, their whole damn server side runs on it.
Here’s a lecture by Cuong Do Cuong, the engineering manager at YouTube. Apparently, he was there from the very start and he tells an interesting story of dealing with exponential growth and handling the scalability issues as they arised.
[Update: The lecture is a bit boring, but the real interesting stuff is during the Q&A at the last 10 minutes or so. You should watch them.]
The amazing thing is that the guys that did all kinds of insanely amazing things to acommodate their growth rate, including hacking the lighttpd source code to improve the way it does multithreading, still use Python for their server side code. According to Coung, it never was the bottleneck and the speed problems they did have were easily solved by throwing in several more servers. The speed of development in Python on the other hand, helped them respond quickly to the changes and implement new ideas almost on the spot.
When I chose Python as the main language for Tuzig about two years ago, I knew very little about it. I knew only that everything I tried thus far (C++, Java and .NET) wasn’t going get us quickly enough to where we were going and that Python looked very promising. I haven’t looked back since. I usually try to choose the best tool for the job, but Python seems to excel in many areas : from small one-off scripts to complex GUI applications (thanks to wxPython) to web applications. I can even write Python stored procedures for PostgreSQL.
And speedwise, if it’s good enough for YouTube, it should be good enough for me.
If you don’t know Python, you should take a look at DiveIntoPython. It’s a great book by Mark Pilgrim that is written for experienced programmers and focuses on the things unique to Python instead of explaining, like many other books, what a for loop is. It’s also freely available on the net so can dive right in.
