Tuesday, June 23, 2009

Innovation and evolution: winning the battle, losing the war

Life itself is a chancy thing, but like love, the pursuit of innovation, is among the most fickle and heartbreaking means to spend one's time on earth. And just as nobody ought to fall in love for any reason but to love, it is wise never to consider innovating unless the process of innovation is itself intrinsically appealing and satisfying to the heart. Whatever comes out of the process ought to be treated as a bonus.


Millions of organisms, many, beautiful to behold have become extinct over the eons, and thousands more face extinction everyday. They have reached an evolutionary dead-end, not because of any intrinsic shortcoming, but because they no longer satisfy the criterion of fitness with their environments. It is Richard Dawkins' Blind Watchmaker at work, dispassionately removing those organisms that no longer fit into the grand scheme of things, regardless of their intrinsic merit.

The same sort of phenomenon is observed with respect to innovations, except that the ecosystem in which they emerge (or in which they face extinction) is human society and not the natural world. There is a certain tragic quality to this state of affairs: On the one hand, it is in human nature to try to perfect any artifact that emerges out of the imagination, and in the case of complex artifacts such as advanced technologies, such perfection requires intense, repetitive effort, and many iterations and years before any level of perfection is achieved. On the other hand, the ecosystem is an unfeeling context which cares little for human aspirations regarding the perfection of innovations. At any moment, an artifact or technology can be rendered unfit (in an evolutionary sense) because of changes in the environment, especially the emergence of competing technologies; and however wondrous and intricate it may be, further development ceases abruptly, and it is left to be mourned only by its sometimes resentful inventors, but is cruelly forgotten, or even ridiculed, by the masses. Not long after, it begins to appear quaint, archaic, obsolete, something that would never have had a chance to survive, anyway.

Charles Babbage's complex Difference Engine, a mechanical calculator made up of thousands of gears that he designed in 1822, is one such example. Machines constructed from his designs are beautiful to behold and demand considerable, precision effort. But they are far more expensive to build and far less capable and accurate than any modern, digital, hand-held calculator. The mechanical computers were great innovations, but in retrospect would never have been able to scale up. A very similar example is that of beautiful, expensive, high precision mechanical watches, and their far less expensive, much more mundane, but nevertheless far more accurate digital successors. Mechanical winding watches have been relegated (or perhaps, elevated) to the status of expensive fashion accessories, and even jewellery whose primary function is that of an adornment and an object of wonder rather than a device that tells the time.

Over and over again, Mother Nature seems to have been right, but her correctness is perceivable only in retrospect, and often only after an interminably long period.

I came across two other instance of Cruel But Always Correct Nature: the American Apollo Space Program and Blu-Ray optical disks. I was surprised to see the first described as some sort of a failure -- an evolutionary dead end -- and to see the latter as obsolete already; but upon reading through the articles, I understand and agree with the sentiments expressed. The US space program launched in right earnest, thus:
America had been inspired by President Kennedy's wish, announced in 1961, of "achieving the goal, before this decade is out, of landing a man on the moon and returning him safely to Earth." After his assassination in 1963, the idea became a homage to him, a way of showing the world what the United States would have achieved had he lived. Within days of the Apollo 11 astronauts' safe return to Earth, someone put a message on Kennedy's grave: "Mr President, the Eagle has landed." Job done, in other words.
By 1969, the task was done, and the last moon mission occurred in 1972 - Apollo 17; the next three missions were cancelled, and that was that. The space mission was launched not so much to land a man on the moon as to tell the world that the US could beat the USSR at its own game. And it did -- at immense financial and personal cost:
It was also an extraordinarily expensive project, it should be noted. The entire Apollo programme cost $24bn in 1960s money - around $1 trillion in today's - and for several years was swallowing up almost 5 per cent of the US federal budget. In addition, there was also a considerable emotional cost to the missions, a point stressed by Christopher Riley, co-producer of the 2007 documentary In the Shadow of the Moon. "A great many Americans suffered premature heart attacks and strokes from their efforts in making the Apollo project succeed. More than 400,000 workers were employed by private contractors to build capsules, rocket engines, space suits, and computers for Apollo and the vast majority worked flat out, over weekends and holidays, much of the time for free, for several years to make sure the programme succeeded."

For example, at the Grumman factory in New Jersey, where the lunar module was built, staff would clock off at 5pm, leave by the front door, walk round to the back and work for free until midnight. Similarly, employees at the International Latex Corporation - which made the suits worn by the Apollo astronauts - worked with equally obsessive intensity. In a recent documentary, the company's senior seamstress, Eleanor Foraker, recalled working 80-hour weeks without days off or holidays for three continuous years, suffering two nervous breakdowns in the process. "I would leave the plant at five o'clock in the morning and be back by seven. But it was worth it, it really was."
Looking back, it appears that the Apollo mission was destined to be an evolutionary dead-end. It achieved its principal purpose, and then it became clear it was fit for little else:
In the end, the real problem for Nasa is that it did the hardest thing first. Kennedy's pledge to fly to the moon within a decade was made when its astronauts had clocked up exactly 20 minutes' experience of manned spaceflight. "We wondered what the heck he was talking about," recalls Nasa flight director Gene Kranz. To get there before the Russians the agency was obliged to design craft that were highly specific to the task. Hence the Saturn V, the Apollo capsule and the lunar module. Unfortunately, these vehicles were fairly useless at anything else in space - such as building a space station - and Nasa, having nearly broken the bank with Apollo, had to start again on budgets that dwindled dramatically as the decades passed.
The article concludes:
The conclusion is therefore inescapable. Kennedy's great vision and Armstrong's lunar footsteps killed off deep-space manned missions for 40 years - and probably for many decades to come. As DeGroot says: "Hubris took America to the moon, a barren, soulless place where humans do not belong and cannot flourish. If the voyage has had any positive benefit at all, it has reminded us that everything good resides here on Earth."
The other example here is the Blu-Ray optical disc which was created by an industry consortium as a successor to the ubiquitous DVD disc. As compared to the DVD's 4.5 GB capacity, the Blu-Ray has a far higher capacity of 25 and 50 GB. The Blu-Ray competed for market space with Toshiba's HD-DVD and won the format wars. But not for long, it would appear. While scientists were focused on a successor to the DVD (which in turn succeeded the Compact Disc), the ecosystem changed around them and direct digital downloads over the internet are growing in popularity along with increasing bandwidth connections. The Blu-Ray has won the battle, but it may eventually lose the war. It's not clear if Blu-Ray's developers will ever recoup their developmental expenses. Blu-Ray players are priced high to recoup the cost, but this very situation militates against their rapid market penetration, especially during a recession. From the article:
Blu-ray will doubtlessly continue to grow in popularity as more of us buy large HD-capable flat screen televisions. In the same vein, it’ll continue to make inroads in computing and video gaming markets. But it’s a case of too little, too late, as long-term trends point to a slower uptake than DVDs ever had. When we can simply download a good-enough copy of a movie from iTunes and save it to a USB drive or mobile device for viewing pretty much anywhere, why would we even bother with a power-hungry, noisy, expensive and frankly inconvenient disc in the first place?

By the time most consumers have asked themselves this question, the answer will already be in: Optical discs are a fading technology, and investing in them now could be a shorter-term move than you might have initially anticipated.
Your typically management tome might ask 'managers' to perform some sort of 'strategic analysis of the market/technology landscape' by employing a sainted analytical tool which gives cute names to different cells in a matrix. The reality is that no management theory can beat large-scale paradigm shifts, or even demands of the moment. Oftentimes, you do what you are able to do, and hope the ecosystem will continue to support you, even while placing a few bets on the side on other, alternatives whose prospects currently seem remote.

No comments: