I am a child of Apollo; one of my earliest memories is of the launch of Apollo 15, which my family drove to Florida to witness. I was five years old. I came to love science, especially but not only astronomy and space. Like many others, I started down the path to a STEM career simply because I was fascinated by the science, by the beauty of the ideas. Many of us, including me, start out as techno-utopians. We naively assume that the things that we discover or build will be useful or make the world better in some way. But as an engineer, I have come to recognize that technology does not exist in a vacuum. The things we are building need to be placed in their proper context in society.
And as an educator, it is my responsibility to see that our students at least begin the process of understanding their role through an ethical lens, whether they become scientists, engineers, business people, policy makers, or activists, and in all cases in their roles as citizens. Rather than a full understanding of theory, this should be applied ethics and society.
We all hear a lot these days about technology-driven anthropogenic global warming (including, recently, the astounding power consumption of cryptocurrency mining in proof-of-work systems), autonomous weapon systems, machine learning systems learning (from human behavior and data) to be racist, technological disruption of labor markets, and online spying, bullying, and scams. While it might seem too early to be talking about the ethical implications of something as embryonic as quantum computing and communications, in fact the first major burst of funding for the field was driven by the discovery of Shor's factoring algorithm. Its implications for public key cryptography have immediate consequences for national security, and so the spooks and others scrambled to be involved. Although most quantum researchers trace the birth of the ideas to the theoretical, abstract work of Benioff, Feynman, Deutsch, Bennett and Brassard in the 1980s (or to Wiesner in the 70s, Bell in the 60s, or even Einstein-Podolsky-Rosen in the 30s), it's fair to say that as a field in its own right quantum engineering was born with ethical concerns. Moreover, the students we are educating today will lead the field for the next four or five decades. Now is the time to begin the conversation.
To that end, recently, I posted this short message on Facebook:
Okay, people, I'm thinking curricular thoughts: how do we raise ethical #QuantumNative engineers? What are your book-level reading recommendations? I think I want four books, one each on the following topics:
1. How technology makes the world a better place.
2. The limitations of technological attempts to make the world a better place.
3. The consequences of technological failure.
4. How technology can be employed in antisocial/anti-democratic ways.
Recommendations? (I am also willing to be persuaded that I don't yet have the right set of topics.)
Asking these questions actually has three purposes. The first is to augment a blog posting of mine on a quantum native engineers bookshelf, the second is thinking about revising our own undergraduate curriculum here for all our students, and the third is my involvement in a Japanese government funded effort to create an undergraduate quantum engineering curriculum.
I received some nice responses from my friends, and even more from the friends of Mike Nelson. Rather than replay the conversation, in this posting I am going to attempt to synthesize the collective knowledge. With apologies to those who contributed, I am certain that almost everyone will disagree with some element of this synthesis, but I hope it represents roughly some sort of centroid among those who spend a lot of their waking hours worrying about such matters. I will list the names of those who were kind enough to contribute, and did not ask to be anonymous, at the end.
Most of the contributors took the idea quite seriously, not just as an adjunct to an engineering curriculum but as an important topic in its own right, so some of the suggestions drill deeper than is achievable within the confines of a four-year engineering program, but could be incorporated into a master's program.
It's worth noting that many of the books listed here are only roughly categorized. Many of them take a broader look at the topic and so cover more than one of these questions, or focus on one technology or historical period and look at it from multiple points of view. Indeed, the more I look at the set of books the more it seems my original questions are naïve or oversimplified in categorization. So, rather than attempting to read these books as if they answer a single question, perhaps the better approach is to read any of these books with that set of questions in mind.
Indeed, as I am editing this, I am coming to reconsider the structure. This is also the "long list", with nothing yet culled; eventually I will have to make a (Japanese) semester-sized syllabus as well as a shorter list for my bookshelf.
Somewhat unexpectedly, I got several fiction recommendations. After some thought, I kept them with the appropriate categories.Though as discussed below they run a larger risk of misinterpretation than nonfiction, they can make a deeper, emotional connection and explore the impact of the what-ifs on people in a different way. The fiction books are marked.
Where I have been able to find them, I have used links to publishers' and authors' sites, rather than to Amazon. After all, even as much I use Amazon, I think we all agree it is a prime example of too much concentration of market power.
-1. Quantum
Before we start the list, I suppose it's worth a comment or two about what is unique about quantum computing and communication, with respect to other engineering disciplines. The short answer is, nothing.
The long answer is twofold:
- As an interdisciplinary field, comprised of physics, mathematics, computer science, and computer engineering, as well as application fields such as chemistry, AI, operations research, and cryptography, the people involved come from very different viewpoints and backgrounds, and use very different vocabularies. While WWII was known as "the physicists' war", most present-day researchers spend little time thinking about the implications of their work. Finding common ground is crucial.
- Quantum is the biggest change in our computational and communication capabilities since the transition from analog to digital. I expect this Second Quantum Revolution to run for fully half a century yet; there is still some time before the actual capabilities catch up to the vision. Nevertheless, the changes will be so profound and far-reaching that it is hard to foresee any problems that will develop.
One could argue that the problems quantum will raise are less fraught than the work already underway on brain-machine interfaces and even current machine learning. Perhaps. In educational terms, I do think the foundations are the same, hence quantum programs can cooperate seamlessly with other engineering programs, so perhaps rather than creating my own list here my time would be better spent finding an existing program I admire enough to emulate. But I'll bet in a couple of decades there is an independent sub-field of quantum ethics.
To be clear, as of this writing (March 2022), I see the challenges as primarily educational rather than research.
There is a small community of people who are already publishing on the topic, but the things I have seen to date demonstrate only a rudimentary understanding of the technology, and so have only a little to add to the conversation so far. Thus, besides educating our own engineers, it is incumbent on us to work with ethics researchers so that the learning is mutual.
An approach taken by at least one researcher is to tie quantum to AI, and hence to the profound issues that ethicists are wrestling with there. The researcher can hardly be blamed for making the connection, since quite a number of quantum researchers are themselves pushing it. Personally, I am skeptical that quantum will transform AI, though if the critical problem of operating on large datasets can be solved then quantum may accelerate performance. I do believe that a well-rounded #QuantumNative engineer will have a grasp of the important AI ethical issues.
0. Foundations
Ahmed Amer emphasized that students need to begin not just with my questions above but with a framework for thinking about ethical issues, so let's begin there. Call it question zero. A couple of these books are specific to AI and hence perhaps should be categorized a little lower in this list, but most are more general.
- Thomas Kuhn, The Structure of Scientific Revolutions: Reportedly, Al Gore's favorite book. I've read it a couple of times, and find it not an easy read but an important one, although I have heard that modern thinking has moved past the particular structure that Kuhn proposed.
- Herb Simon, The Sciences of the Artificial: We are, after all, engineers. It's important to understand how the things we build fit into the taxonomy of the universe.
- Abraham Flexner, The Usefulness of Useless Knowledge: Another book on why we do science, a philosophy rather than ethics book, but about the role of science in society. Get the edition with modern commentary.
- Shannon Vallor, Technology & the Virtues: Ahmed's first recommendation, this should provide the basic framework and vocabulary for thinking about ethics. (I have not yet read this book, but it is very high on my list.)
- Peter L. Berger and Thomas Luckmann, The Social Construction of Knowledge: Cited 70,000 times, which would catch the eye of any metric-watcher. You can find (possibly copyright-violating) PDFs if you look. (I have not yet read this book.)
- Karl Mannheim, Ideology and Utopia: Written in 1936, perhaps the oldest book on this list. (I have not yet read this book.)
- Lin, Abney, Jenkins, eds., Robot Ethics 2.0: From Autonomous Cars to Artificial Intelligence: (I have not yet read this book, but it looks like a good overview, so I listed it here.)
- Joseph Weizenbaum, Computer Power and Human Reason: An early (1976) nonfiction book on what we would now call AI ethics by an important, early AI researcher. I'm not certain if it is still in print, but I think you can find (possibly copyright violating) PDFs on the web if you look. (I have not yet read this book.)
- Robert E. McGinn, Science, Technology, and Society: I haven't read this, and don't really know much about it, but apparently is/has been used in college classes. It is apparently out of print (1990).
- Naomi Oreskes, Merchants of Doubt: More about how scientists can go wrong than about how others misapply our work, so I put this in foundations. (I have not yet read this book.)
- Marshall McLuhan and Quentin Fiore, The Medium is the Massage: A super-famous notion, but I admit I have not read the book.
- Woodrow Hartzog, Privacy's Blueprint: I haven't read this book, but I have heard the author speak. This book can serve as our look into the relationship between engineering and law. (You might want to substitute his even newer book coming out right now, but I gather this one is a little more focused on that legal connection.)
- Brian Green, Space Ethics: I am told that the first part of this presents a useful, broad framework.
1. Benefits
On question one, we should see some examples so that we know what success looks like.
- S. Pinker, Enlightenment Now: (I have not yet read this book.)
- D.A. Henderson, Smallpox: The Death of a Disease: (I have not yet read this book.)
- Kevin Ashton, How to Fly a Horse: Apparently not strictly related to ethics, but more about how innovation happens. (I have not yet read this book.)
- Eric D. Beinhocker, The Origin of Wealth: Evolution, Complexity, and the Radical Remaking of Economics: (I have not yet read this book.)
- Tom Standage, The Victorian Internet: This comes strongly recommended. (I have not yet read this book, but Jun Murai is fond of citing a late 19th century pamphlet(?) by Yukichi Fukuzawa that includes an image of people connected via telegraph.)
- Lee McKnight and Audrey N. Selian, The Role of Technology in the History of Well-Being: A book chapter rather than a book, but at 40 pages has enough room to establish depth. (I have not yet read this.)
- Paul Kriwaczek, Babylon: Mesopotamia and the Birth of Civilization: (I have not yet read this book.)
- (fiction) David Brin, EARTH: I've read this and consider it an extended thought experiment. I'm not sure I would categorize it here, but that was the recommendation.
2. Limitations
I think it's also important to understand that factors other than sheer technical success influence how large an effect on society that a new technology can have. Engineers must work with others who are working to make society better, and must approach the the task with profound humility. In short, things never work out the way you expect, and engineers shouldn't even try to go it alone.
- Justin Reich, Failure to Disrupt: Perhaps the best cautionary tale I have read on computers and society, showing that technology alone is not enough. This is something of a specific case study rather than a more complete textbook. It might feel off topic, but I think everyone should be aware of the limitations of our ability as engineers alone to remake the world. Fundamentally, it shows how important the social structures that support the deployment of technology are.
- Daniel Sarewitz, Frontiers of Illusion: Science, Technology, and the Politics of Progress: (I have not yet read this book.)
- Sherry Turkle, Alone Together: I have not yet read this book and am uncertain about its categorization, but this seems like a good start.
- (fiction) Karl Schroeder, Stealing Worlds: (I have not yet read this book.)
3. Failures
In this context, I mean technical failure: the systems we design and build do not always operate the way we intend. These failures have consequences in the real world, sometimes including the death of humans or damage to the environment. It is critical for an engineer to understand these consequences and some of the common causes and to take to heart the importance of, and equally the impossibility of, technical perfection.
Beyond outright crashes (of programs or vehicles), this category includes failure to operate properly in a broad set of legitimate circumstances due to insufficiently broad or rigorous requirements and/or testing, such as the now-famous failure of some automated sinks that use infrared sensors to correctly detect the hands of dark-skinned people.
- Peter Neumann, Computer-Related Risks: To the best of my knowledge, there's still nothing else like it. Peter is still the chair of the ACM RISKS Forum. If you don't have a healthy respect for what can go wrong with, and as a result of, computing technology, then you are Icarus incarnate. (The book is 1995, and availability is apparently limited; is there a more modern equivalent?)
- Henry Petroski, To Engineer is Human: another book from the 1990s on failure in engineering and design. (I have not yet read this book.)
- Charles Perrow, Normal Accidents: This book received multiple recommendations. (I have not yet read this book.)
- (fiction) Mary Shelley, Frankenstein: The recommender suggested this meets all four questions, but I think I would place it here.
4. Misapplication
As engineers, we carry in our heads an image of how we expect our products to be used. That image may unintentionally exclude some groups, such as women or people of color or differently abled people. Thus, if we become aware of the issues, we can do better. Worse, some may deliberately use digital (and ultimately quantum) tools to effect repression, destabilize democracy or even prosecute a war, goals which hopefully few quantum engineers would actively endorse.
This category is the most vague and hence has the most entries. It's certainly not necessary to read all of these, but it is a good idea to get a broad view.
- Edwin Black, IBM and the Holocaust: I read some of the articles about this when the book came out, but have not read the book itself. Seems like Exhibit #1 for this category.
- Steven Feldstein, The Rise of Digital Repression: perhaps the clearest, most direct book on this list about how technology enables political authoritarianism, well written with a strong focus on how existing authoritarian organizational structure influences choices in the digital arena, as well.
- Your Computer is on Fire: I found this book uneven, and I don't think it provides a well-rounded, comprehensive survey, but it's still probably the best book-length thing I've read on how technology influences and is influenced by human biases and problems. If you want to replace or augment this with something else on the ethics of computing technology, including race and AI, the surveillance society, etc., I am okay with that. Reading the news every day and thinking about its implications is necessary but not sufficient, IMO.
- Sasha Costanza-Chock, Design Justice: Like the above, a book that very much comes from the point of view that "artifacts have politics". (I have not yet read this complete book, only excerpts, but the full text is open access.)
- James Williams, Stand Out of our Light: (I have not yet read this complete book, only excerpts, but the full text is open access.)
- Cathy O'Neil, Weapons of Math Destruction: One of the more famous books on this list, but I haven't yet read it.
- Kate Crawford, Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence: (I have not yet read this book.)
- Tim Wu, The Master Switch: The Rise and Fall of Information Empires: (I have not yet read this book.)
- Barbara Ehrenreich and Deirdre English, Witches, Midwives, and Nurses: A History of Women Healers: I have not yet read this book, and of all the suggestions made this is the only one that seems likely to result in significant disagreement as it is listed under "New Age" and apparently rather substantially disses modern medical science; Amazon reviews are mostly positive but a few set up significant alarm bells for me. This is apparently a reprint of a 1970s feminist tract with commentary from 2010.
- Ruha Benjamin, Race After Technology: (I have not yet read this book.)
- Caroline Criado Perez, Invisible Women: (I have not yet read this book.)
- Brad Smith, Carol Ann Browne, foreword by Bill Gates, Tools & Weapons: the Promise & the Peril of the Digital Age: (I have not yet read this book.)
- Jonathan Zittrain, The Future of the Internet and How to Stop It: I have not yet read this book, but it is well known. Published in 2009, it is worth reading while keeping an eye on how the Internet actually has evolved in the intervening decade-plus. Perhaps it can help head off the worst of the ideas in the current "web3" brouhaha. (I have not yet read this book.)
- David Brin, The Transparent Society: One of my own favorite thought experiments. Basic thesis: privacy is dead, get over it. Governments and corporations have all the incentive in the world to collect data on us; as individuals, our only recourse is to, in turn, have equal transparency into those organizations and what they are doing with the data.
- Neil Postman, Technopoly: (I have not yet read this book.)
- Chris Ategeka, The Unintended Consequences of Technology: (I have not yet read this book.)
- Paul A. Offit, Pandora's Lab: (I have not yet read this book.)
- Siegfried Kracauer, From Caligari to Hitler: An intriguing recommendation, with the suggestion that it shows how cinema was used by the Third Reich to manipulate public opinion (although the blurb says it is more about the Weimar Republic). (I have not yet read this book.)
- (fiction) George Orwell, 1984: this was recommended by a friend, and of course it's one of the most important novels of the 20th century, but I don't think of it so much as being about the impact of technology on society. Moreover, one commenter pointed out that not everyone takes the same message away from reading the novel; fiction even more than nonfiction appears very different depending on the mindset you bring to reading it. I think it's a book that everyone should read, but I am a little reluctant to include it in a curriculum like this.
- (fiction) Aldous Huxley, Brave New World: Of course someone suggested this, as well.
- (fiction) John Brunner, Shockwave Rider: (I have not yet read this book.)
- (fiction) John Brunner, Stand on Zanzibar: (I have not yet read this book.)
Closing Thoughts
As I write this, Russia has just invaded Ukraine, and simultaneously apparently attacked a number of Ukraine government websites. The COVID-19 pandemic continues, where advanced research empowered the vaccines and the Internet has been a source of both indispensable research communication and public outreach, and appalling misinformation. Examples of the importance of ethics in daily life and our profession flow through the news daily.
Ethics in AI has become an important topic, perhaps belatedly given its broad implications. Quantum may (or may not) wind up contributing to the advance of AI, so it is particularly urgent for quantum folks to be aware of the discussions. (See a few links below.)
One topic that is not yet addressed here is work in the defense industry. Many graduating engineers in all fields will go to work in the defense industry; quantum will be no different. How should they decide what work is ethical? I believe this would be an entirely separate, and very long, discussion.
Inevitably, even a list of this length only scratches the surface. This is a veritable book factory subject, with dozens and dozens of books by people from all backgrounds, especially in the loosely-defined fourth topic above. It does seem to be more a topic covered in books and white papers rather than journals, but perhaps I'm looking in the wrong places. People devote their careers to this topic; I would be thrilled if our own graduates did so. Hopefully, this will embed an ember deep in the brains of today's students that will glow and eventually flare into a flame that sheds new light on the world.
Sources and Resources
I am not by any means the first to think about incorporating ethics into STEM teaching; inarguably, I am late to the game. From my point of view, this lets me take advantage of the learning and work done by others.
I am grateful to the following contributors: Michael Alan Aisenberg, Suzanne Aldrich, Scott Alexander, Ahmed Amer, Lara A. Ballard, Richard Bennett, Maya Bernstein, David Brin, Patrick Coin, Margaret Cullen, Bay Fang, Castor Fu, Mike Godwin, Steve Gómez, Margret Hjalmarson, Annalie Killian, Kelly Knox, Charlie Marcus, Lee Warren McKnight, Ashley Merryman, Sue Moon, Mike Nelson, Craig Patridge, Karla Peterson, Alejandro Pisanty, Stuart Ray, Susan N Erik Read, April Rinne, Kavé Salamatian, Avery Sen, Jun Takei (竹井淳).
Note that even the centuries-old technology of the English alphabet, and our practice with respect to organizing lists of names, is not entirely value neutral; it puts people with family names starting with the letter A in a position of priority.
n.b.: Mike Nelson added a discussion about writing when he reposted my original. While I 110% agree about its importance, I think it's a separate discussion from raising ethical, thoughtful #QuantumNative engineers.
Other web resources:
https://oecd.ai/en/wonk
https://standards.ieee.org/industry-connections/ec/autonomous-systems/
https://ethicsinaction.ieee.org/
https://genderedinnovations.stanford.edu/index.html
https://www.scu.edu/ethics/focus-areas/technology-ethics/resources/embedding-ethics-into-computing-curricula-resources-and-suggestions/
Revision History
- 2022/2/4: first message posted to Facebook
- 2022/3/5: first full draft sent to a handful of people for review
