Nobel Prize in Physics to...Hopfield and Hinton for Artificial Neural Networks?!?

 You have probably heard by now, but about twelve hours ago the Nobel Prize in Physics was awarded to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks”. To say that this prize is surprising is an epic understatement. It's also causing some consternation and even anger in some quarters. I don't normally feel qualified to comment on the Nobels, but in this case let me record just a few thoughts that are enhanced by conversations with and social media postings by a few friends.

Hopfield was faculty at Caltech when I was there, but was oriented more toward biology at the time, and I wasn't aware enough to take the really important and interesting classes. He taught a class on neural networks early during my time, which a few of my friends took but I didn't. In 1981-83, he, Carver Mead, and Richard Feynman taught a triple-listed Bi/CS/Ph course that was reportedly wild.  I'm sorry I missed that! (I did take Feynman's class later, in 1985-86, that was a direct descendant of that class. We learned a little about neural networks, and in fact a couple of friends and I implemented one in C, though it wasn't very good. That class changed how I view the process of computation, and indirectly led me into quantum computing several decades later.)

One of my closest friends took a class Hinton co-taught back in the early 80s at CMU. She said it was fascinating, all the way back then.

On the prize itself, at least one of my physicist friends is angry. Paraphrasing, it's not physics and why should we be celebrating things that detract from human accomplishment? I don't agree with the latter, but the former will be the basis of bar arguments for a long time to come.

My opinion? Hmm...

It's not even entirely clear to me, after watching the press conference, whether the prize is being awarded for neural networks being physics, or changing the way people do physics. The former figured prominently in the press conference, as they talked about Boltzmann machines.  Yes, Hopfield and Hinton both used ideas from their background in physics, but to me this seems to be a bit of a stretch. The committee also talked about the latter, about the use of neural nets in doing physics. In that case, why not the inventor of the supercomputer, or the laptop?  The inventors of the Intel 4004 microprocessor (Faggin, Hoff, Mazor and Shima), most often cited as the world's first microprocessor, are all still alive. The invention of extreme ultraviolet photolithography is also another good candidate.

I've heard funny takes on this prize, including that the tech bro billionaires bribed the committee.  My own is that it was ranked voting in the committee and everybody refused to budge on their first choice but somehow they all ranked AI high enough that in the end Hopfield and Hinton had the most points and everyone on the committee went, "wait, what???" But they couldn't undo the decision.

That, or they were all angry that Hopfield was left off of Hinton's 2018 Turing Award, shared with Bengio and LeCun. (Speaking of which, I said that Hinton was the first to receive both the Turing Award and a Nobel, but that's not true -- Herb Simon did it first! Interestingly, both Simon and Hinton were on the faculty at CMU.)

I do think it's okay that the committee is stretching the definition of "physics"; they have done that before, with prizes for information technology. But with only a single prize awarded each year in physics, there are many, many discoveries, and discoverers, out there waiting for the public recognition they most definitely deserve. There are other prizes for computing, of course, notably the Turing Award. So while a broad look at how physics has changed society is a good thing, but I think it would be okay to say, "Nah, they already have a prize in their main area, that should be enough."

But in the end, it's recognition of the importance of machine learning, neural networks and ultimately artificial intelligence in our lives already, a fact that will continue to grow. And if it gives Hinton (and others) more of a platform for and recognition of his (and their) reservations about where we are headed with AI, all of that's a good thing. It's a necessary conversation we need to be having, now.

Finally, the score so far for the science prizes this year:

• White men from elite institutions: 4
• Everyone else: 0

We'll see when the Chemistry prize is announced this evening if the rest of the world gets skunked again!

Strength

A friend of mine on Facebook said they think the U.S. needs a strong leader, and that Donald Trump is that person.

I disagree. Trump is not strong.

Calling your opponent "mentally retarded" is not strong.

Stoking fear and hatred of immigrants instead of asking "what brings people to our shores, and how can we help them get the same opportunities we have had?" is not strong.

Bragging about not paying overtime is not strong.

Failing to pay the arenas where you have held rallies is not strong.

Refusing to admit losing an election is not strong.

Being obsessed with and afraid of sharks is neither particularly strong nor particularly weak, but failing to recognize their importance to the ecosystem is definitely not strong.

Nuking hurricanes is not strong.

Threatening to jail your political opponents and reporters who write stories you don't like is not strong.

Speaking of reporters, mocking a talented reporter who happens to have a physical impairment is not strong.

Mocking those with different accents, dress, speech patterns, body types or other differences is not strong; very likely those people speak multiple languages, have a broader experience of the world, and still struggle every day in a culture not their childhood one -- those people, they are strong.

Allowing yourself to simply take as your own opinion the opinion of whoever talked to you last is not strong.

Torpedoing a bipartisan deal on the southern border simply to allow the situation to fester long enough to become a campaign issue is not strong.

Failing to stand up to extremists in your own party is not strong.

Giving Putin whatever he wants is not strong.

Insulting women, assaulting women and taking away their constitutional rights most definitely is not strong.

I could go on.

But finally, attempting to overthrow the U.S. government and install yourself as some sort of petty tyrant because you are not strong enough to lose with grace is weak and cowardly.

Donald Trump is the opposite of strong in every way.

Spelunking CACM, vol. 21 (1978): Cray-1, RSA, CSP, and more, more, more!

WOW, the January issue is a must-read!

 It's chock full of papers on some of the most important architectures, both contemporary and historical, in a special issue, though at a glance I don't see an introduction. (But I don't see the front matter in the Digital Library...hmmm?)

• The Manchester Mark I and Atlas. 'Nuff said. Know your history, kids. Interesting that these two bracket the transistor computer, first developed at Manchester.
• MU5: This one, I wasn't familiar with, but it's from Manchester, one of the most important places in computer architecture history. Manchester's accomplishments include building the world's first transistor computer. (Wait, didn't I just say that?)
• Sperry-UNIVAC 1100: a still-active family of 36-bit mainframes (36 bits???), the first of which were vacuum tubes, later transistorized ones with SSI and MSI (I presume). Although the names in the series followed the 11xx convention, the later ones weren't compatible with the earlier ones. Interestingly, this architecture used ones-complement integer arithmetic instead of twos-complement, so it's theoretically possible to have both +0 and -0 values. Like the DECsystem-10 below, it supported unlimited indirect addressing.
• DECsystem 10: TOPS-20 systems, the next generation of this system, were workhorses at USC/ISI when I first worked in the computer center there. This architecture has a couple of clever, elegant features: essentially a single addressing mode, where the registers are the lowest handful of addresses, and indirection, where any pointer you follow has a bit that says, "Treat this one as pointer instead of the data you're looking for, and follow it." Yes, that could recurse. The OS was innovative, too. Oh, and did I mention that this marvelous has a word length of...36 bits?!? (Didn't I just say that?)  And that a byte can one of several sizes, with 7 and 9 bits being the most common?
• The Cray-1: the most important supercomputer in history, perhaps, and a beautiful, elegant piece of work. Check out that gorgeous image at the top of this posting. Freon liquid cooling, chained 64-bit floating point vector operations working from vector registers, bipolar memory chips, dual-rail encoding to reduce fluctuation in power and signal noise, and the totally 1970s naugahyde love seat around its set-of-wedges circular design. 138 sustained megaFLOPS, what performance! Fastest machine in the world.

I'm tempted to just stop here and declare victory after January, but I suppose I should at least glance through the other issues...let's see, another report on status of women and minorities in CS faculty...

Ahem. Ah. Let's see, this might be important:

The RSA (Rivest-Shamir-Adelman) public key cryptosystem. Yeah, I'd say that's worth mentioning. Oh, and then Merkle, too. And at the end of the year Needham-Shroeder. A good year for cryptography.

Backus's Turing Award lecture. Want to think about programming languages? Read this first.

It was also an incredible year for the theory of distributed systems. Lamport on Time, clocks, and the ordering of events in distributed systems. One of the classic, classic papers in distributed systems. I still have my students read this, almost half a century later. Tony Hoare's Communicating Sequential Processes. Another of the classic, classic papers. And Per Brinch Hansen's own model for distributed programming. His affiliation is listed as USC, but I never met him; I have no idea when he left USC.

My sister's dog likes to lie on her back and just wallow.  I'm tempted to do that while reading all of these fantastic papers!

Spelunking CACM's Second Decade

 As you know, I am working my way through Communications of the ACM, beginning to end, stopping to read a handful of articles from each year. These aren't necessarily the papers with the highest citations, but instead things that catch my eye from the point of view of the 2020s.  A full two years ago I finished the first decade of CACM; now I have finished the second, so let's index them. (Hmm, I've fallen out of the habit of giving them descriptive names, I should go back to that.)

• Spelunking CACM, vol. 11 (1968): Operating Systems Bonanza and Ethics, but not the ARPAnet!
• Spelunking CACM, Vol. 12 (1969): CAD, encouraging(?) minority participation in computing, and discrete event simulation
• Spelunking CACM, Vol. 13 (1970): Nucleus, Bloom filters and magnetic tape
• Spelunking CACM, Vol. 14 (1971): Flynn's taxonomy
• Spelunking CACM, Vol. 15 (1972): Hough transforms, Dijkstra on correctness
• Spelunking CACM: Some Thoughts on Operating Systems (a "special issue" posting)
• Spelunking CACM, Vol. 16 (1973): threaded code and too many PhDs
• Spelunking CACM, Vol. 17 (1974): Clipping Polygons, Parallelism, and UNIX
• Spelunking CACM (1975): Phong Shading
• Spelunking CACM (1976): PhD production, data flow
• Spelunking CACM, Vol. 20 (1977): Certifying information flow, an early network routing protocol

Spelunking CACM, Vol. 20 (1977)

I have finally reached my twentieth article in this series! It's probably time for another summary, but first let's do the things that caught my eye in 1977. This was the year that "Star Wars" (with the simple title) came out. I was eleven. I probably wasn't yet particularly aware of computers and computing, but I was already a fan of "Star Trek" and of the space program (which was in the Apollo-Soyuz/space shuttle interlude at the time).

When I first skimmed the contents for 1977, I flagged eighteen articles for further investigation. For a systems person like me, there was a lot to catch the eye. I've winnowed it down, but if you look there are still more good papers than just these.

If you're interested in data about demographics and numbers of students in CS, there is a report following on from the previous year.  What's depressing is not how bad it was in 1976, but how bad our diversity remains today.

As long as we're on the topic of education, it was already possible in 1977 to put together a survey of 200 publications on CS ed. ACM's first curriculum committee issued its report in 1968, though, and SIGCSE was founded in 1970, so it should be no surprise. The CS GRE was also introduced recently and analyzed in CACM.

Dorothy Denning and Peter Denning give an elegant description of how to certify information flow in programs for security purposes. Consider, for example, the simple statement

If the variables y and x are in separate security classes, then there is a potentially unauthorized information flow between the classes. After the execution of this statement, the value of y will equal the value of x, whether we considered that acceptable or not. The Dennings go on to discuss a complete method for static, compile-time analysis of this information flow.

Morgan and Levin provided a mathematical means of assigning files to network nodes to optimize overall system performance. The eye-catching hypercube with cut planes at the top of this blog posting is taken from the article. They even consider multiple copies of the files in the networks, and divide traffic into "query" and "update" and consider them separately. They recognize that assigning the files to the nodes is an exponentially complex problem, and provide some heuristics for avoiding searching the entire space.

Speaking of networks, Tajibnapis developed a routing protocol for the MERIT network in Michigan. His article on TIP was originally submitted in 1974, and finally published in 1977. To me, the NETCHANGE protocol specifying the message contents for routing protocols sounds a lot like RIP, which was developed quite a bit later. However, TIP doesn't seem to have had a lot of impact; I'm not sure why. Of course, even by 1974 Dijkstra's shortest path first algorithm was fifteen years old, and there was quite a flowering of work on routing in the mid-1970s, so it's likely that there was other, similar work. MERIT would go on to be a pivotal place for networking research for decades to come.

In operating systems work, Lamport wrote about readers and writers, which I still teach about in my OS class today. Fundamental stuff, though this article is a little...rococo in its notation, I think.

Parallel computing was in its earliest phase of trendiness. While I associate a lot of theory of correctness in parallel programming to K. Mani Chandy, there is a paper here on how to prove parallel programs correct, using Susan Owicki's techniques and an implementation of Dijkstra's on-the-fly garbage collector as the example. As author David Gries says,

Building a program with little regard to correctness and then debugging it to find errors is even more folly for parallel programs than it is for sequential programs.

Word.

And finally, if you want some history to go with your history, there are Rabin's and Scott's Turing Award lectures, given for their work on nondeterministic automata some years prior.

Lynn Conway

I was in a Starbucks when I saw Dave Farber's message on his IP mailing list saying that Lynn Conway had passed away, and I said out loud, "Oh, no!" and started crying. She was a hero and an icon, and stands high in our technical pantheon.

Of course, every person has the fundamental human right to live as they are, as they understand themselves to be, and the rest of us get no say in who they are. Saying, "It's okay that Freddie Mercury was gay, he was an amazing singer and artist," fundamentally misunderstands this. The vast majority of LGBTQ+ people are perfectly ordinary people, and that is perfectly fine. Margo Selzer said, "It is not the job of the underrepresented to solve underrepresentation," and the same is true for other aspects of life as a minority. It's the job of the majority to change ourselves to be accepting; no minority should be required to step up and be a hero. So, Lynn "owed" no one her work as an activist; it was a role she chose late in life, and we should be grateful for it.

FWIW, it took IBM 52 years to get around to apologizing for firing her (falling just short of the 55 years it took the UK government to apologize for chemically castrating Alan Turing for being gay).
https://www.nytimes.com/2020/11/21/business/lynn-conway-ibm-transgender.html

As it happens, I was talking to a couple of students yesterday about citation counts for researchers in computer science and electrical engineering, and we found a website where the top researcher has half a million citations. You won't find Lynn's name on a list like that, and yet I would put her contribution far above almost everyone on such a list. She received dozens of awards, but far fewer than she deserved, IMO. There would BE no "chip industry" without her.  Pretty much everything else in our research field and our entire industry...is secondary.

Wikipedia tells me that IEEE Solid-State Circuits Magazine published a special issue on her career in 2012. I didn't know that, but it was well deserved. Her own reminiscences are worth reading -- every sentence.
https://ieeexplore.ieee.org/document/6392995
https://ai.eecs.umich.edu/people/conway/Memoirs/VLSI/Lynn_Conway_VLSI_Reminiscences.pdf

We all owe her a tremendous debt. Write her name in the history books, and then go and pay it forward. I'll tell my Computer Architecture class and my quantum computing research group about her tomorrow. I didn't know her in person, but I probably won't be able to keep my eyes dry.

[written right after hearing about her passing, posted a couple of weeks later.]

[edit: obituaries:

• New York Times
  
• Washington Post
  
• IEEE Spectrum
  
• University of Michigan
  

]

Gordon Bell

 Gordon Bell has passed away.

Gordon was one of the most important computer architects of all time. He designed, co-designed or was executive lead in charge of most of Digital Equipment Corporation's key machines in its heyday, from the initial PDP-1 to the VAX-11, in two stints working for the company. Although he wrote the seminal description of the PDP-11, perhaps the most influential minicomputer of all time, I don't think he had much to do with that design or with the PDP-10, and by the time of the Alpha he had already left DEC for good. But still, much of the company's design sensibilities grew from him.

I learned a great deal from the book he coauthored on computer engineering, using all of the DEC machines as examples. Most of the chapters were coauthored by Gordon and other members of the various technical teams. (Wow, I paid ten bucks for my copy at a DECUS in 1986, but even the Kindle version now goes for a hundred bucks?!?)

He also established the Gordon Bell Prize for accomplishments in parallel computing. One of the recent prizes was for simulation of quantum mechanics, though nothing to do with quantum computing.

RIP Gordon, thanks for the machines and for being such an important advocate of parallel computing.