Programming Massively Parallel Processors

The next step for a CUDA newbie after CUDA by Example would naturally be the book Programming Massively Parallel Processors by David Kirk and Wen-mei Hwu. They taught the first course on CUDA at UIUC for a few semesters and this book is based on its lecture notes. I had used their lecture notes and videos when I first learnt CUDA.

To write any CUDA application that moulds the problem optimally to the CUDA architecture requires the programmer to think very differently from programming on a CPU. Using a matrix multiplication example, the authors walk the student through many levels of improvement. The authors introduce the different facets of the architecture and end up improving the performance of the solution by as much as two orders of magnitude in the end.

All the concepts of the CUDA architecture are covered: the thread-block-grid hierarchy, the global-shared-local memories and barrier synchronization. Details of the warps and the warp scheduler are explained. Since most CUDA applications are scientific, there is an entire chapter on the floating point format. This chapter gives a practitioner’s perspective that I found to be more useful than the popular but obscure What every computer scientist should know about floating-point arithmetic. There are two chapters on application case studies, which are mostly useless since one cannot understand the application intimately enough to draw any lessons from it.

CUDA runs only on NVIDIA devices. OpenCL is its twin that is designed to be used on all kinds of CPU and GPU processors. The authors have thrown in a chapter on OpenCL for folks who need to transition to it. OpenCL is exactly like CUDA, except that it does not have an equivalent of the CUDA Runtime API. So, the programmer ends up spending some time building the scaffolding required to run his kernels.

Programming Massively Parallel Processors is a easy book to study from. It should be accessible to any intermediate-to-expert programmer. Newbies can check out CUDA by Example before studying this book. I do wish this book covered some information on cache configuration, launch bounds, profiling, compiler options and other intimate details which one ends up using to squeeze out the last bit of performance. Currently, I do need to fall back onto the CUDA Programming Guide for such information. The book is also a wee bit outdated since the Fermi architecture is not well covered and the new Kepler architecture has already been released.



A few years ago, Joel Spolsky decided to show the world how software is created. He invited a documentary crew to film the summer interns at his company Fog Creek create what was code-named Project Aardvark. It would be later branded as the product Copilot. This documentary was named Aardvark’d: 12 Weeks with Geeks and sold on DVD. The entire movie is now available on Youtube here.

When there is some problem on their computer, a lot of people ask their family or friends to help fix it. Most of the time the helpers are physically far away. So, Project Aardvark set about creating a website which could connect the seeker and the helper, the helper could view and interact with the seeker’s desktop and hopefully fix the problem.

Aardvark’d follows the four interns: three programmers and one marketer as they go about learning and creating this software. I have nothing more to say because this Aardvark’d is super-boring, watching paint dry would be more exciting than this. Neither do we get to see the daily hustle-bustle life of a Fog Creek employee nor are we given the technical insight into Project Aardvark. Instead we are fed shots of the interns sitting at their computer working, some stupid contests and some talking points from Joel Spolsky and Paul Graham.

Aardvark’d is a mega-bore of a documentary and I highly recommend avoiding spending the one hour or so of your life on it. Just watch the trailer, believe me when I say that there is absolutely nothing more to see in the movie than what is shown in the 2 minutes of that trailer.

Le Gamin au vélo (The Kid with a Bike)

Childhood is a time when we try to adapt to the world around us and it is a place from where we draw most of our memories. Le Gamin au vélo (The Kid with a Bike) follows the childhood of a boy for a few days as he tries to navigate and live with the world he is cast in. When the movie begins, Cyril is at a foster home where his single dad left him due to financial problems. Cyril is a bright kid and tries to track down where his father has disappeared to. His search brings him across Samantha, a woman who helps get his old bicycle back. She allows him to stay at her home during weekends, while he continues his search. When he finally finds his dad, his heart breaks when his dad asks him to never see him again since he wants to move on with life. Distraught Cyril falls prey to the local teenage gangster, who uses him to get one of his jobs done. The repercussions of his con job end up affecting him and his relationship with Samantha.

Set in summery Belgium, Le Gamin au vélo is a light and breezy experience. Reminding me very much of my carefree summer holidays of childhood, the cinematography feels very natural and personal. Many facets of the human experience emerge through the prism of Cyril’s adventures: search for a father figure, discovering good and bad, learning that actions have reactions, and finally a woman discovering that she likes to have a son. All of this comes through without any of the affectations that are typical baggage in a Hollywood movie.

Thomas Doret, who plays Cyril, is a gem of a find. He brings across the precociousness of his character with a performance that feels very real and devoid of mannerisms. How the directors managed to extract such a raw performance out of him, I cannot even imagine. Cyril is never seen being concerned about his mother. Leaving the mother out of the story leads the viewer naturally to use Samantha to fill that role. Cécile de France plays this role quite effortlessly, as she discovers slowly that she quite likes having a son around. Directed by the Dardenne brothers, a Belgian director duo, Le Gamin au vélo is a sunny and warm look at a tiny slice of childhood. I found the movie to be a delight and might check out more movies by the Dardenne duo in the future.

Lifeu Ishtene (ಲೈಫು ಇಷ್ಟೇನೆ)

After penning the scripts for many of Yograj Bhat’s movies, Pawan Kumar made his directorial debut with Lifeu Ishtene (ಲೈಫು ಇಷ್ಟೇನೆ) last year. Taking a philosophical look at love, this movie is dark, much like his earlier Manasaare. It follows the life of the hero (Diganth) through his many trysts with love from school to college to the workplace. The hero ends up realizing that he (and everyone else) is on a futile and endless pursuit of love that is caught in an Escher-esque infinity.

Pawan is a guy with a million ideas, anyone can see that with his earlier movies. However, Lifeu Ishtene suffers from his amateur directing skills. The idea is great, but the implementation is not. The movie is unnecessarily long, dealing with too many of the hero’s attempts at love. I tend to enjoy short movies that deal with a slice of life, not the baggage of the entire life of a person. The songs here are okay, but none stands out or is worth putting on a playlist. If Pawan is serious, he should stop putting such songs (the typical practice in Indian movies) which break the flow of the plot. The only song which helps the story along and the one I like is Kanasina Hosaputa. It is a nice introspection by the hero after he has been stiffed, shot very much like Tanhayee in Dil Chahta Hai. Lifeu Ishtene is a decent attempt that would have been watchable, if it was shorter and took itself a bit less seriously.

De vrais mensonges (Beautiful Lies)

It is hard to resist watching a romantic-comedy set in an European summer, that too along the Mediterranean coast. In De vrais mensonges (Beautiful Lies), popular French actress Audrey Tautou is a hairdresser who is watching her separated mom slip into depression. Sami Bouajila, the help at her salon is besotted with Audrey and sends her an anonymous love letter. She redirects that letter to her mom through Sami in a bid to cheer her up. A love triangle takes shape when her mom pursues Sami, believing him to be her secret lover, while Audrey starts to have feelings for him.

Petite Audrey is sickly thin in De vrais mensonges, I hope she is keeping well. She is pretty unremarkable here, with Nathalie Baye (her mom) grabbing all the sunshine. The fairly predictable plot has a few nice surprises, but is a bit too long. This is a fun movie to watch once, if not for the story then just for enjoying sun dresses in Sun-soaked Southern France. 🙂

Algorithms (4th Edition)

One of the biggest roadblocks in studying Computer Science is the popular tome Introduction to Algorithms by CLRS. It is the most common textbook for introductory algorithms courses at universities. While it is a great book for learning analysis of algorithms and as a reference, it is a horrible book to learn algorithms. I am pretty sure a great many number of students have been turned off from and have fear of algorithms since this book was their first look at algorithms. Thankfully, there is an excellent cure for this problem.

I recently re-discovered the wonderful world of microprocessors with the book Inside the Machine. It made me realize that there must be books out there on every topic that are more intuitive, stimulating and illustrative than the status quo. I started to look for such a book on algorithms. After checking out almost every textbook on this topic out there, I discovered Algorithms (4th Edition) by Robert Sedgewick and Kevin Wayne. And, I fell in love! 😀

This is an introductory textbook for algorithms. The book covers all the most important algorithms on sorting, searching, graphs and strings. Real-world problems and historical perspectives are provided at the beginning of every chapter to motivate the algorithm. After that a skeleton of a Java class is sketched out with the API that is demanded by the problem. The authors then show why other algorithms studied earlier in the book cannot solve this problem. Using these failings as a key, the user is introduced to a new concept that can solve the problem. This is then advanced to the final algorithm with full working Java code to fill out the skeleton API. Finally, many propositions about the time and space complexity of the algorithm are put forward with their proofs.

I never thought that I would find an Algorithms book so addictive that I would study it cover to cover! What is so special about this book? There are a few reasons for that. First are the illustrations. There are more illustrations in this book than there are pages (yes, unbelievable!). The authors have taken great pains to tear apart the difficulty in understanding how and why an algorithm works. And then the code: this is the simplest, most elegant real code ever written for algorithms. Do red-black trees haunt your dreams? This book banishes that fear by elegantly using 2-3 trees and morphing that understanding to red-black trees. And finally the proofs. Every proposition and its intuitive proof is such a joy to work through and finally understand that it gives one a high!

In my humble opinion, Algorithms (4th Edition) by Sedgewick and Wayne is the best introduction to algorithms. There is quite simply no other book out there that will make you fall in love with algorithms like this one. My only advise is that you make sure you understand every illustration and concept in the book. That goes a long way in giving you an intuitive feel of the algorithm. If you are picking up this book, do remember to check out the book’s website. It contains a lot of supporting content, including code and errata. (I am honored to have been able to contribute quite a few errata there 😉 ). The final icing on the cake is that Sedgewick and Wayne will soon be teaching a two-part Algorithms course based off this book at Coursera soon!

How to get up without an alarm clock

(Disclaimer: I keep an alarm clock as backup. But most days I can wake up before it rings.)

Sleep is one of my obsessions. For most of my life, I had trouble waking up on time for school or work. A few years ago I discovered that I could wake up on time without an alarm clock (or more accurately, wake up before it rings).

How to get up without an alarm clock:

1. Find out how much sleep you need for an active day (say 8 hours of sleep)
2. Find out the time you need to wake up every day (say 0700)
3. Subtract time (1) from time (2) to get your bedtime (0700 – 8 hours = 2300)
4. Start sleeping regularly at your bedtime (2300) and keep an alarm for your wakeup time (0700).

Is it not elementary, my dear Watson? Do this for a week or two and soon you will find yourself awake before the alarm clock rings.

Both you and I know very well that there are devils hidden in this simple method. One of them is the bedtime. A consistent bedtime is required for this to work 😉