Kearneysville, WV. Recently I had my driver’s license renewed. After I sat for my new picture, the lady behind the counter asked if I would like to have my fingerprint included in the information kept on file. I told her no. After all, I can’t think of any compelling reason they would need my print. Nevertheless, I was told that “the computer says you gave your fingerprint last time and it can’t proceed unless you give it your print this time.” Without much thought I sighed, stepped forward, and let the electronic gadget take an image of my finger.
But even as I did so, something seemed odd. The lady didn’t use her authority to compel me to comply with a specific rule or law. She didn’t speak as if the power of the state was standing behind her words. She merely told me that “the computer says” that I needed to behave in a certain way, and I did.
I am bothered by my quick and unquestioning compliance with the authority of a machine, but if this was an isolated incidence, it wouldn’t be worth mentioning. However, consider how often we behave in a certain way because a machine, often sporting a pleasantly benign human voice, instructs us to do so.
It is today a somewhat rare event to actually speak with a human being when calling for help or information. Instead of a person we get a computer impersonating a person and as a result the possibilities of this interaction are dramatically narrowed. “If you are looking for our operating hours, say ‘hours.’” And we obey, perhaps grudgingly, but the alternative is a dead end, for though the voice is that of a friendly woman who seems almost too eager to please, the strictures on the “conversation” are limited. If you fail to speak clearly, “she” will say—still in that perfectly pleasant voice—“I’m sorry, I didn’t get that. Could you say it again?” Speak clearly and “she” moves on through “her” narrow range of options. “If you want to go back say ‘go back.’” And you can backtrack to the fork in the “conversation” that might yield more fruitful results. I fear for my sanity the day I tell “her” thank-you at the end of our “conversation.”
Or consider the helpless office worker, perhaps at a bank, who sadly shakes her head and tells you she can’t do what you are asking because “the computers are down.” She can’t take my money, can’t run my credit card, can’t release a needed document. We have become so dependent on this system that we are forced to put our lives on hold when the computer breaks. Is that a healthy level of dependence?
But as my experience at the DMV indicates, the problems are not only when computers go down. They can be programed to insist on a particular series of actions and if those actions are not performed, the computer will “refuse” to comply with the user’s request. Again, the range of options is quite narrow and our ability to negotiate is forfeited since we are dealing not with a person but with a machine.
Perhaps some level of automation is necessary in our complex age, however, the potential downside should not be ignored. I would like to suggest at least three implications of our current situation. First, we are learning to obey anonymous authorities. When the clerk shrugs his shoulders and says he can’t do what we ask because “the computer won’t let him,” he is showing his powerlessness in the face of the authority of the computer, and we comply knowing that there is no one to whom we can appeal. The habit of complying with the anonymous authority of a machine is little different from complying with the anonymous bureaucrat who insists that he can’t do things in a certain way because “the rules” forbid it. In either case, obedience becomes habitual and unthinking.
Second, we are growing accustomed to taking orders from irrational devices. This obvious fact is easily forgotten when the irrational device is given a pleasant human voice and even thanks us when we obey. But is habitual obedience to an irrational voice a healthy thing?
Finally, because these irrational devices cannot be argued with, we learn to meekly obey. Irrational devices cannot change their minds (for they have no minds to change). They cannot negotiate. They are not creative in their interactions. They cannot come up with a mutually satisfying compromise. There is, therefore, no use in debating. One must obey or be stymied.
But the habit of questioning, of civil debate, of negotiating, and compromise are precisely the habits necessary for a thriving democracy. They are the mental habits required of a free people. To the extent that our machines teach us to comply without question and to thoughtlessly obey anonymous and irrational authorities, we are sowing the seeds of complacency and servitude. I suppose I should have refused to give my fingerprint at the DMV, but what could I do? The computer insisted.
Have you read Asimov’s iRobot? This article reminds me of the last short story in that book.
It does not make sense to call a machine irrational. The software on a computer is doing what it is designed to do — outside of bugs.
Logic, when it operates with incorrect or inadequate input, may seem insane or irrational from the outside. But from the computer’s point of view, it is making the only “rational” choices.
Our problem is not with the computers, or even with the programmers, it lies with the nameless, faceless government and corporate bureaucrats who limit the choices available within a system for the sake of efficiency. Or, sometimes, simply out of stupidity.
@mushroom: “Rational” means capable of reasoning. “Irrational” does not necessarily mean erratic or unpredictable. It simply means not possessing reason. Thus, computers are irrational, unless they have recently made the jump and are now in possession of rational faculties.
I agree. My pencil is doing what it was designed to do. It is not thereby rational.
Which is why I put rational in quotes.
I’m merely a software engineer so I’m sure you know than I do about the English language. However, the connotations of irrational are much more inclusive than simply not “capable of reasoning”. If you look it up in a dictionary, it can mean “illogical” or “something that does not make sense”.
A computer, like a hammer, is a tool, and nothing more. People would look at you rather funny, I think, if you said your hammer was irrational.
Mark,
I see/experience the syndrome you describe frequently. To resurrect an old thread of discussion on FPR, I think Matthew Crawford says some interesting things about the computer making me do it, or not letting me do it, or allowing me to do it even though I don’t I don’t know why. “Shop Class as Soulcraft”
I am not anti-computer–in fact I sometimes laugh at those in this venue who use a series of computers and incredible technology to rail against the machines. (BTW, I don’t think your article is in that category.) The problem is not when–as in your case–an anomoly occurs. The problem is the way computers are part of the philosophy of work that is moulding our culture.
The idea behind your experience at the DMV is that really only one person needs to understand the process. That person (usually unknowingly) assuming a measure of omniscience, programs the computers so that, theoretically, anyone who can operate a computer can perform a service. Understanding the process is unnecessary, perhaps even undesirable. The proliferation of medical technicians who understand nothing about healthy, or sick for that matter, people, but who are “competent” to do one or two procedures, are another expression of this syndrome.
My thoughts go on, but I’ll finish with a “one-better” on your bank experience. One Thursday morning I went to our local McDonald’s for my weekly meeting with some other preacher-types. Ronald was out of business; his computers were down. Apparently, flipping pancakes and making change for $2.39 out of $5 couldn’t be done without silicon-based help/direction. The sad thing is, after further reflection, I figured the Mc-management was right.
So fingerprints are a requirement of getting a drivers license now ehh? Well this is something entirely new for us to be proud about for being Americans.
Not that I’m precious about my prints mind you. The City of New York already catalogs mine. Seven of the best hours of my life were spent in a Gotham Precinct house, thrown there for not wearing a seat belt and as a result, meeting a Cuban ex pat film stunt man hauled in with me. The arresting officer did not know what to do with us and near the end of our time in the hoosegow, she appeared at the bars and directed our attention to the little camera recording our stay, quipping ;” see that camera up there, the boys in the back room are watching you and they don’t like it when people have a good time in jail”.
No, we are supposed to be glum and nervous. Good humor is a threat to the State.
Computers are not “irrational”. They are in fact perfectly rational, and that’s the problem: perfect rationality is a poor fit for a world that is all too often not rational.
This would be OK if the machines were not allowed to have last the world so that some human could simply override them when they demaded something that makes sense in Computerworld but not in Humanworld, but it’s fairly uncommon that such overrides are included.
“but it’s fairly uncommon that such overrides are included.”
And often that lack of override is deliberate. I understand that Kodak has functioning UNIX drivers that are fully compatible with Linux. They will not give them to their customers, because what if they could use them to avoid paying the Ink prices? So they only give out the Microsoft Windows and OS X drivers.
I wouldn’t draw a sharp line between ‘all human’ and ‘all dependent on the machine’. Back when I was doing clerk work, we had electromechanical registers that were fully as complicated and rulebound as the modern computer-based type. We also had purely mechanical credit-card stamping machines, and plain old paper ledgers to summarize the day’s results. When the power or phone lines went out, all the machines and pens could still be used. You could hand-crank the register.
The worst problem is not our dependence on machines, but dependence on totally centralized machines that are programmed from International Corporate Headquarters and return all data to the ‘cloud’.
When everything is centralized, everything is wide open to hacking, spying and sabotage. There may be less chance for local cheating, but there’s also less chance for local detection of central cheating, which can be vastly larger.
“This obvious fact is easily forgotten when the irrational device is given a pleasant human voice and even thanks us when we obey. But is habitual obedience to an irrational voice a healthy thing?” No necessarily.
Comments are closed.