Paradox of Automation


Crash: how computers are setting us up for disaster

by Tim Harford at

The paradox of automation, then, has three strands to it. First, automatic systems accommodate incompetence by being easy to operate and by automatically correcting mistakes. Because of this, an inexpert operator can function for a long time before his lack of skill becomes apparent his incompetence is a hidden weakness that can persist almost indefinitely. Second, even if operators are expert, automatic systems erode their skills by removing the need for practice. Third, automatic systems tend to fail either in unusual situations or in ways that produce unusual situations, requiring a particularly skilful response. A more capable and reliable automatic system makes the situation worse.

ibm-704-1954_thumb1Back in the late 1970s I was responsible for supporting an then state-of-the-art communications system (think airline reservation system) with the latest hardware and the most elegant coding. However, at that time the console for the system was a Model 33 Teletypewriter (unless, as we occasionally were forced to do, you input instructions through the front-panel toggle switches).

Because of the hierarchical structure of the corporation, the computer console operators were fairly low on the responsibility (and pay) scale. But my experience in this situation is closely aligned with Tim Harford’s observations (especially the third strand).

Things are far more sophisticated today but the script of the computer console back then was complicated enough. One observation I should add to the Harford paradox is that training and education will never be sufficient to resolve unusual situations.

We had several console operators throughout the day and for my money the best was a young man who was generally accepted as the slowest operator and possibly not the brightest either. However, when something unusual occurred on the console, he immediately went to the supervisor for instructions. We also had a few operators who were pretty smart, but they tried to substitute their smartness for training and that got them into trouble. Too often they tried to solve the problems themselves and screwed things up even more.

imgres.jpgThere’s a lot to think about in the Harford article coupled with my own experiences, but I possibly can focus the problem by pointing out two common situations that have plagued the computer world for decades: first, attempts to sectionalize the computer rooms with special door-codes often result in the doors being propped open with a handy trashcan, and second, complicated or obscure passwords are inevitably written down or even scribbled on the face of the keyboard.

For some people security is a nuisance, for others it is a challenge.



What are your thoughts on this?

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s