Book review: Advanced R

I would like to call this the best second book on R, except that I wouldn’t know what the first one would be. I learned R from classes and tutorials about 10 years ago, used it on my PhD and four articles, and use it today on a daily basis at work; yet only now, after reading this book, do I feel like I could possibly be called an R programmer rather than just a user.

The book deals with a variety of topics that are seldom discussed in the R tutorials you are likely to find freely available. Some are perhaps unnecessary in a book like this (Ch. 5 Style Guide), some could easily deserve an entire book (Ch. 7 OO field guide), but the chapters on Functions, Environments, the three chapters in Part II (Functional Programming) and the chapter on Non-standard evaluation are easily reasons enough to buy this book.

How many time indeed have you spent hours, frustrated, trying to write a function that would act as a wrapper around, say, lattice plotting functions that use non-standard evaluation? Or try to call subset() from another function, only to see cryptic error messages? Now, for the first time, the solution is not only clear to me; I feel like I could also explain to a colleague why things work the way they do.

R is incredibly powerful and dynamic and will, most of the time, do just what you expect it to do. But if you want to really understand what is going on under the hood, or if you want to write your own packages (whether for self-, internal-, or widespread use), you owe it to yourself to read this book.

Review: Growing Object-Oriented Software, Guided by Tests

Growing Object-Oriented Software, Guided by TestsGrowing Object-Oriented Software, Guided by Tests by Steve Freeman

I didn’t know what to expect when I picked up this book. In spite of its excellent reviews I feared it was going to be another redundant addition to the mountain of books harping on the virtues of Test-Driven Development (TDD), without adding anything significant to the standard sermon.

Nothing could be further from the truth.

I read a fair share of technical books, but this book is the only one in years that I immediately began to re-read again after finishing. It is easily one of the most important books on software engineering out there, and is likely to remain so for some time to come.

The authors present what is now known as the London school of TDD, where the correctness of an object is defined by its interactions with its collaborators, not necessarily by its state. Although I had seen mocking frameworks in action before, never had I seen one being used throughout the development of a software project.

Another fascinating idea is the notion of writing an end-to-end test first, before even starting to write unit tests. We have been so thoroughly drilled on the virtues of fast tests, that it doesn’t occur to us anymore that it’s even possible—even preferable—to exercice the whole system, perhaps in a separate test suite.

But the best part of the book is the sample project used to illuminate these concepts. It consists in writing a desktop application with which a user can automate the process of bidding in online auctions. The graphical part is done with the Swing framework in Java, and the application talks via XMPP to the auction house. The first chapter in the case study is about setting up literally an end-to-end test, i.e. a test (written with JUnit) that will verify if the graphical display matches the XMPP communications.

From there on, the case study proceeds with the implementation of feature after feature, always following the same pattern: write the end-to-end test first, implement the feature with TDD, refactor, repeat.

No book is worth reading if it doesn’t change your approach to your existing projects. This one showed me immediately where our current project (an embedded system for energy management) was lacking in terms of testing.

Go read this book, and send me flowers and chocolates.

Review: RESTful Web Services

RESTful Web Services RESTful Web Services by Leonard Richardson My rating: 5 of 5 stars

I began reading “Restful web services” while researching technical solutions for Neurobat Online, the web service version of our intelligent heating controller. Prior to this, most (all?) web service projects I had been involved in were based on SOAP.

REST is a heavily overloaded term in our industry, and can mean different things to different people. The author avoids that controversy by coining the term “Resource-Oriented Architecture”, and shows different examples of web services that can be built using this approach: a social bookmarking service inspired by Delicious, and a mapping service. Both examples are RESTful but the author does an excellent job at showing why being RESTful is not enough. To fully leverage the existing web architecture (including the full HTTP protocol) you need, he argues, to do more than merely being RESTful, and he shows how.

The author never says so explicitly, but after reading this book I found that RESTful web services have at least two significant advantages over what the author coyly calls “Big Web Service” (aka SOAP):

  1. Testability: do not underestimate the advantage of exposing a service that your testing team can test with cURL instead of having to setup a tool such as SOAPUI.

  2. Discoverability: everything is a resource, and all resources will respond to a limited number of HTTP verbs. You don’t have to worry whether adding a bookmark is done through addBookmark() or appendBookmark(); if your service is RESTful, you know that you need to send a POST to some URI.

The biggest takeaway from this book for me was to realise that it’s possible to design an application where “everything is a resource”, and that all resources respond to the same set of methods. Think of it for a moment. Will this not change the way you design non-web services too? Are not all your objects resources? Imagine, for a minute, if all your classes were restricted to expose not more than 5 public methods, and if these methods had the same names. It may sound crazy, but it’s quite possible that you’d end up with a cleaner design built out of many small classes with small interfaces. Is this not an easy way to clean code?

What makes this book great and not merely good is that the author doesn’t simply explain what RESTful web services are about. He is also clearly opinionated about it; however he is never patronising, never condescending. He takes very occasional jabs at systems he calls “Big Web Services” but never belittles them. His message comes across as entirely believable and convincing. By example after example he shows how popular services can be designed as collections of resources, and it is up to the intelligence of the reader to judge whether which kind of design is better.

Review: Linkers and Loaders

Linkers and Loaders Linkers and Loaders by John R. Levine

My rating: 4 of 5 stars

You may have written hundreds, maybe thousands of programs, but if you are like most programmers then everything that happens after the compilation is kind of mysterious. Why does the compiler have to create object files? What are they? What is this so-called linker who combines those files into a library, or an executable? What’s its purpose? John Levine’s book answers those questions, and more.

Item 53 in 97 Things Every Programmer Should Know: Collective Wisdom from the Experts is “The Linker Is not a Magical Program”, and this book goes a long way towards taking that magic away. It carefully explains step by step what happens from the moment the code is compiled until it actually runs on the machine; and what’s more important, it makes it very clear why things are as they are today.

I was recommended this book in a reply to a Stackoverflow question, and I am not disappointed. The book goes occasionally perhaps a little bit too much into technical details, which I felt could be safely skipped. Perhaps a case study, i.e. going through every single step towards running a complete program, would have been useful, instead of exposing how different systems solve the different steps one by one.

Until I read this book I simply did not understand how a program actually ran on my computer. A few details are still a bit fuzzy, but now I feel much better equipped for dealing with obscure linker errors or custom linker scripts. Highly recommended for any programmer who wants to get to the bottom of things.

View all my reviews

Book review: Agile Project Management with Scrum

I began reading Ken Schwaber’s ‘Agile Project Management with Scrum’ for two reasons: 1) it’s a book about Scrum, and 2) it’s from Ken Schwaber, one of the fathers of Scrum. Having now read it, I think these are the only reasons I don’t entirely regret reading it.

The book is a series of case studies, bases on real-world experiences Schwaber has had managing projects. Each case study shows how a particular aspect of Scrum was applied, adapted, or tweaked on a real project. Schwaber devotes each chapter to a distinct aspect of Scrum, e.g. the ScrumMaster’s role, the project backlog, the sprint planning, etc.

As such, the book is clearly intended for experienced ScrumMasters, which I am not. But I think an experienced ScrumMasters reading this book will find it lacking in depth. There is almost too much material in this book, covered too shallowly. Each chapter in this book might easily provide enough material for a separate book, or a workshop, or a detailed whitepaper. Even with my limited knowledge and experience of Scrum I felt frustrated by the lack of detail Schwaber gave in the book. For instance, he never shows us a real-world example of a product backlog. Instead of showing us a snapshot of a real sprint backlog taped to a team’s room, the book shows us a nicely formatted table with very obviously watered-down entries.

In short, I think an experienced ScrumMaster will find this book lacking in detail, whereas the Scrum student will find it hard to relate to the case studies. As such I find it hard to recommend this book, and I think anyone interested in Scrum should rather consult Schwaber’s earlier book, ‘Agile Software Development with Scrum’, or Henrik Kniberg’s excellent ‘Scrum and XP from the Trenches’, also available for free from here.

Book review: “Hot, Flat and Crowded”

I’ve read Thomas Friedman’s “Hot, Flat and Crowded”, and firmly believe this book belongs on the shelf of anyone involved in making buildings more energy-effective.

Mr Friedman’s previous bestseller, “The World is Flat”, discussed the changes to our world that enabled more and more people to participate in a global economy. “Hot, Flat and Crowded” is sort of a sequel to that book, although there is no need to have read it first (I haven’t).

Mr Friedman’s thesis is that as more and more people participate in the global economy (“flat”), the standards of living are increased everywhere and world population grows (“crowded”), while more and more people aspire to a western style of life, with its damning consequences to the environment (“hot”).

According to Mr Friedman, the small habit changes that we are all asked to pick up (changing incadescent lights to more efficient lights, turning off our TVs instead of leaving it on standby, etc), while commendable, are simply insufficient. There is nothing we can do invidually to prevent atmospheric CO2 levels from reaching dangerously high levels in this century. Instead, he proposes a series of measures nations should take, such as imposing carbon taxes, cap-and-trade schemes, and several regulatory laws on energy efficiency for vehicules and buildings. Only such drastic measures, he argues, will make a significant difference.

An entire chapter, in particular, is dedicated to what he calls an “energy internet”, in which he envisions how a network of appliances and utilities could cooperate to dramatically reduce the energy demand for buildings. Buildings, incidentally, represent about 40% of any nation’s energy demand, and have this annoying property of being long lived. Once a building is built, it will consume energy and water for the next 30-40 years, which makes it all the more important to build them right from the beginning.

Contrary to many “green” books, though, Mr Friedman gives more arguments in favour of “going green” than the obvious environmental ones. There are at least two other, less obvious reasons why we should work on reducing our energy demand and developing renewable energy sources:

  • Current demand for oil finances petro-dictatorship all over the world, preventing many countries from achieving freedom and democracy.
  • Many poor countries lack ready access to cheap and clean energy, preventing their further development.

The arguments Mr Friedman develops for these aspects of the problem are worth the price of the book alone, in my opinion. But the real eye-opener is the extensive discussion on the way utilities have traditionally worked and how they should work in the future. He doesn’t say that our homes and buildings should simply become more efficient; he says that by wiring together buildings, appliances and utilities together, we can escape from local optimums in energy efficiency and aim for much more important savings.

And that, of course, is one of the aims of this blog: to bring together information on how to achieve that vision. We definitely have the technology (we have had it for the past 30 years or so), we only need the economic incentives to work on this problem. And Mr Friedman provides several suggestions that governments the world over could and should adopt, suggestions that will, if implemented, leave us no choice but to build the infrastructure he describes.

Presentation checklist

This week I started reading Garr Reynolds’ Presentation Zen, the well-known book on presentation design and guidelines. (I’ve borrowed it from our library after waiting for something like 9 months. There were about 10 reservations before mine, so I guess it must be popular.)

On page 61 there’s a nice list of questions one should always ask oneself before preparing a talk. Drilling through that list is definitely a good idea, and I’m writing it down here for further reference.

  • How much time do I have?
  • What’s the venue like?
  • What time of the day?
  • Who is the audience?
  • What’s their background?
  • What do they expect of me?
  • Why was I asked to speak?
  • What do I want them to do?
  • What visual medium is most appropriate for this particular
    situation and audience?
  • What is the fundamental purpose of my talk?
  • What’s the story here?
  • What is my absolutely central point?

Go through these questions the next time you give a talk and you can’t go wrong.

Review: Decompiling Java

I just finished reading through Decompiling Java by Godfrey Nolan. You might ask, why would I mention a book on reverse-engineering bytecode in a weblog dedicated to smart buildings? What is the relationship between the intellectual property of companies writing Java code and home automation? The answer, essentially, is OSGi.

An OSGi framework, in a nutshell, is a Java program for managing and running other Java programs. When you start it you get a shell prompt that allows you to download, install, start, stop, upgrade and remove other Java programs delivered as bundles, i.e. compressed sets of files.

Since recent years it has become economically possible to have embedded controllers that run on stripped-down versions of Linux with specially designed Java Virtual Machines (JVM). In other words, it has become possible to run Java on these constrained targets. And Java is a good language for writing home automation systems. Indeed, according to Peter Kriens, the author of OSGi’s official weblog, OSGi was initially designed in 1998 for the home automation market.

I work together with Adhoco AG, whose flagship product is a home control system running an OSGi framework on an embedded controller. The author of the JVM we use has pointed out to us the relative ease with which compiled code can be decompiled, which is why I picked up this book after seeing it being reviewed in the ACM Computing Reviews.

Most of the book is about the process of decompilation itself, i.e. extracting the maximum amount of information from a compiled bytecode. While technically interesting, the meat of the book was in chapter 4, “Protecting Your Source: Strategies for Defeating Decompilers”.

That chapter will, I think, be the most interesting one for businesses and it is neatly summarized in a table at the end of the chapter. Protection strategies range from naive obfuscation (cheap to do, but provides weak protection) to the use of native methods for sensitive parts of the code (best protection in my opinion, but breaks portability).

I would not recommend the purchase of this book if all your business wanted to do was to protect yourself against decompilation. After all, about 90% of the book is concerned with the writing of a Java decompiler. You might be best served by borrowing this book from a library and reading chapter 4 instead. But if you want to know the internal workings of a Java classfile, then I am not sure where I would find a book with as much information as this one.