-
"It's like a Swiss Army Knife for propellor-heads."
— review of the NeXT Cube (1990)
This blog entry is written from the point of view of a software guy, so I apologize in advance for the specialized examples, but I believe the lessons are universal for any technology.
One of the fun things about being a techie in the Internet Era is watching the accelerating pace of fads as they sweep across the meme-space. In my field of software development I've seen the following come (and often go):
- ALGOL
- Backus-Naur Form (BNF)
- PL/1
- structured programming
- Nassi-Shneiderman Diagrams
- waterfall development
- Ratfor
- UCSD PASCAL
- LISP
- SQL
- UNIX
- object-oriented programming
- SMALLTALK
- C++
- TCP/IP
- workstations
- client-server
- Solaris
- multi-tiered
- CORBA
- PCs
- patterns
- Visual Basic
- Java
- Javascript
- Java Beans
- Objective C
- C#
- LINUX
- Tcl
- Perl
- Python
- AJAX
- agile development
- open source
- XML
- PHP
- Ruby on Rails
- SaaS
- Go
- Node.js
The Devil You Know and the Devil You Don't Know
The big question we face when we evaluate new technology is "Too Soon" vs. "Too Late." If you are too early an adopter you can suffer from bugs (especially "infant mortality" type), an undersized installed base and user community, and skepticism from many quarters including managers, clients and investors. There is also the danger that a proprietary solution from an unprofitable startup may simply go away, making you a technical orphan. Of course popular open-source projects can also lose favor and fade away. But on the other side a mature technology can fade away as well, if it gets too much "bit rot" to be fixed or is not adapted to its changing environment. Old software is well-known to frequently suffer from the effect that fixing one bug causes two more.
But timing, of course, is everything. The earliest of early adopters almost always suffer for their innovations on the "bleeding edge." As they like to say, "A pioneer ends up with an arrow in the back." But there is a "sweet spot" — for example, Apple's entry into the music player business with the iPod — that maximizes "lock in" and other early benefits. And on the other side, there are technologies like FORTRAN, COBOL, DOS, and XP that seemingly just won't die and are remarkably stable, if not feature-rich.
I have seen extreme examples of the failure modes of both "horns" of this technical dilemma.
I worked for a midwestern startup that had probably adopted a technology too early. They had a mapping suite of software written in Smalltalk, an early object-oriented language, that depended on an "object-oriented database" (OODB) to store its data, and that database had some operational bugs as well as design flaws that made the software pretty much impossible to deploy. One of the operational bugs was that entities in the database could not be completely deleted; their names remained as "ghost objects" forever and could not be re-used.` One of the design flaws was that only one program could use the database, unlike every "real" database I've ever used. These and other problems caused the product, and the company, to fail.
On the other extreme I once worked on an accounting application (in this century I should add) based on an obsolete database from the early 1980s that was so old it didn't use the standard Structured Query Language (SQL) of all modern databases. The vendor was no longer supporting the database for any other customers, and gave the bare minimum of support to my employers. This company was limping along in a fragile and vulnerable state, performing accounting services for scores of clients in a mode that could fail dramatically at any moment, with no short-term disaster plan and no long-term road map for migrating off of this ticking time bomb. They are still in business but I'm glad I'm not one of their investors.
Criteria
So how do you decide when it's too early or too late?
My hierarchy of criteria goes like this:
- stability
- deployability
- portability
- maintainability
- longevity
- security
- features
On the other extreme I began learning PHP a few years ago and was dismayed by its lack of maintainability and difficulty of deployment. I have written precisely one PHP app, which you can see on-line at:
Deployment was easy because the web hosting company already has PHP installed. If I suddenly had to port this app to run on someone's local computer, I would allocate at least an afternoon to install and test PHP and the Apache web server it needs. I would probably end up using this manual:
Since going through this process I have learned that PHP has some security issues as well. I certainly don't need much security for my little "page-a-day" calendar app, but most uses I'm aware of are for e-commerce, where it really matters. This is why I currently take a "wait and see" attitude with PHP, and only use it when I am given code by others in the language.
Another point to consider when selecting a technology is how to deal with worst-case scenarios. I've noticed that both automobiles and software libraries tend to break down more often after 5 PM on Friday before a long holiday weekend. I have found I would rather tell my boss or client "We're going to work all weekend to find the bug in our code and meet the Tuesday deadline," rather than "The problem is in a library, we don't have time to write a replacement, so we'll have to wait until Tuesday to call tech support. There's no way we can make our deadline."
I know what you're thinking: what about open source software? Well, I have found it is both a blessing and a curse. Its a blessing if you want source code to everything and have time to find and fix other people's bugs. It's a curse when you pile on too many open source pieces under a tight deadline and then have trouble integrating and debugging them. I have learned to approach every such decision with risk management foremost in my mind.
Strategies
There are two main strategies I like to employ to mitigate risk in dealing with new technologies: pilot projects and modularity.
- pilot projects
In classic treatise on software development "The Mythical Man-Month" (1975)
- ( 0201835959 )
Frederick P. Brooks Jr. says: "Throw one away — you will anyway." He is referring mainly to prototypes, which test technology, architecture, and even spec all at once, and result in something you can set in front of the customer or user and ask, "Is this close to what you need?" Once these issues are resolved by testing and evaluation, real development can begin, often by starting from scratch with revised plans.
In contrast, a pilot project is a complete project, from design through development to deployment, mainly designed to test technology. A key factor is that it is a non-critical project. If you can afford to utterly fail on this one the team will be less tempted to lie to themselves or their bosses. (The idea that something "has to" work is so toxic to projects.) The solution if it does fail is to reject the technology and go back to an older, more reliable one and re-implement the project, but only if it is really needed at all.
- modularity
Simply put, if you break functionality into interlocking pieces and implement the pieces separately, it makes it easier to test, debug, maintain and if necessary replace or totally overhaul each piece. I have found the advice in the book "The Pragmatic Programmer" (1999) by Andrew Hun and David Thomas
- ( 020161622X )
— as well as other books in the series — to be helpful in this architecting for modularity. It is a lesson I learned originally from the classic software engineering text "Software Tools" (1976) by Brian W. Kernighan and P. J. Plauger.
- ( 020103669X )
Thought their examples re in FORTRAN and PL/1, I find their design principles to be timeless. I use them all the time in my work.
In addition I've found the following abstraction useful: break things into pieces horizontally and vertically. In a famous tutorial on the UNIX shells we are shown how to print out a file listing in order of size: largest to smallest, starting at the current folder and moving away from the root. The command is:
du -k -a | sort -nr | more
In contrast, a vertical partition involves breaking things into layers, a concept which is a little more subtle. An example will probably help. I'm involved in a software project currently in which we are still debating how to have programs communicate: using shared memory, or sockets, or "RESTful" web services, or even dropping communications files in a designated folder. I suggested we add an "abstraction layer" by writing a set of communication functions/methods, and have the discipline to always use them. Then the underlying technology can be "swapped out" seamlessly.
One of the most powerful tools in layering software is emulation in which one technology emulates another. I'm old enough to have used a VT-100 terminal from DEC before they became obsolete in the 1980s as we switched from minicomputers to PCs, But there are still mainstream programs which emulate that old terminal, including the Linux console.
A pervasive example of software layering is what is sometimes called the "network stack," though nowadays it's usually called the Open Systems Interconnection (OSI) model.
The technology that makes the Internet work is broken into seven layers. Numbered from "bottom" to "top" they are:
- Physical Layer
- Data Link Layer
- Network Layer
- Transport Layer
- Session Layer
- Presentation Layer
- Application Layer
The Network Layer is where the Internet Protocol (IP) is found, and the Transport Layer has the Transaction Control Protocol (TCP). Together they are called TCP/IP. Ever heard of that one? Part of the genius of the internet's design is the breaking of these functionalities into layers, which can each be improved or replaced independently.
Non-Technical Example
a railroad date nail from 1918
If this discussion of the innards of the internet has been too geeky for you, my last example is from a century ago: an obscure industrial artifact known as the railroad "date nail."
My friend Bob and I discovered this delightful collectible while visiting the Western America Rail Museum (W. A. R. M.) in Barstow, CA.
They have a whole room of the things. They're small and sturdy enough to make great collectibles, and have rich histories that can usually be documented. They are nails with dates or other codes stamped into the head that are nailed into lumber used as railroad ties and trestle beams, so their age can be determined when they must eventually be replaced. The types of chemical treatments applied to the lumber were sometimes noted along with a date.
What do we learn from this practice? A surprising number of lessons can be extracted:
- plan for change
Understand that you can't keep using techniques forever; each will have to someday be replaced.
- everything is an experiment
You can never test things enough in engineering, so every field deployment is really an experiment.
- document results
You can't manage what you can't measure, and you should plan ahead to make measurement and records keeping easier.
- be prepared to change course
Every change can prove disastrous, and you need to be able to back out and return to a previous method.
- try multiple approaches
Test technologies against each other. Make sure you have choices informed by performance measures.
Disclaimer
I receive a commission on everything you purchase from Amazon.com after following one of my links, which helps to support my research. It does not affect the price you pay.
This free service is brought to you by the book:
travelingtechie.com
