Spelunking CACM, vol. 5 (1962)

For 1962, I considered choices such as an early Knuth paper, on tricks for making evaluation of polynomials more efficient, an early (but not the first) paper on theorem proving machines, a description of an event for high schoolers that points out that there were already 8,000 computers and 30,000 professionals in the country (sadly, the article has no demographic info on attendees), and especially an early paper on multiprogramming from NASA (what's not to love about a paper that says, "Some educationally valuable mistakes were made"? It's instructive that it refers to "the interrupt feature", indicating its newness, but the modern term "interrupt service routine" was already in use.). In the end I settled on a notice about ACM's policy toward standardization. CACM already had a section on "Standards", edited by S. Gorn, but this notice is otherwise unsigned.

Early, the paper makes three binary divisions: users v. "professional computer people", industrial v. theoretical (interestingly, not academic), and hardware v. software. This divides those with an interest into eight categories.

It points out the risks of too-early standardization. As a vendor, if you tie yourself to a standard too early, an innovative competitor can introduce something new, and your hands are tied.

I found this table intriguing. Likely you're vaguely aware of some of these organizations, but you may not realize how early and dynamic they were.  Keep in mind that this is a mere 17 years after the end of World War II, and yet Germany, Italy, the Netherlands, France, and Japan, who collectively suffered some of the worst devastation, are represented. (Russia, China, Poland and Belgium are listed, too, but don't seem to have entries, so I'm baffled as to why they are included.) (In the most breathtaking post-WWII recovery, just two years later Tokyo would host the Olympics and the first shinkansen line would open.)

It's also interesting that the table focuses on language; today, the UN's list of official languages notwithstanding, almost all international standardization work takes place in English, with a vestige still of French. I admit to being lucky to have been born in an English-speaking family at a time when it is the de facto language of science, technology and international commerce. Not so long ago, the choices would have been French (esp. for diplomacy) and German (science and technology).

Quoting from the paper:

The policy of ACM toward standardization is therefore the following:

1. It is extremely conservative as far as the development and promulgation of standards is concerned.

2. It is resistant toward precipitate standardization, specially in any area in which not enough is known to make such standardization theoretically sensible or stable.

3. It tends to be neutral in those areas where standardization is a matter of arbitrary selection, in spite of its recognition of the usefulness of such selection. That part of its membership which is vitally interested in such arbitrary selection is already represented in the industrial side of the activity.

On the positive side, the society is vitally interested in maintaining wide open channels of communications.

4. Thus it takes a positive interest in the stabilization of terminology, whether by reporting common usage or by declaring preferred usage (the normative function).

5. It is interested in the development of appropriate fundamental concepts, the establishment of the relationships among them, and in the quick dissemination of such developments.

6. Finally, it is interested in the development of standard methods of specification of processors, whether they be computers, programs or systems, of languages for such processors and of translation processors for such languages. Included in the methods of specification are methods of documentation for each type of audience or interest in the computer area.

Overall, the policy expresses some interest in standardization of systems, esp. programming languages, it seems, but little else. Almost sixty years later, we can see that indeed ACM, despite its importance in the computing ecosystem, has largely remained aloof from the issues of standardization, leaving that to ANSI, IEEE, FIPS, ISO, IETF, NBS/NIST et al.

Building a Raspberry Pi 4 MPI Cluster in 2021

 

We are using Ubuntu on Raspberry Pi 4 boards with 8GB RAM, 64GB flash drives, coupled using a cheap gigabit Ethernet switch. I wanted to use PoE (power over Ethernet), but that requires a "hat", an additional daughter board to extract the power, and it's moderately expensive compared to the cost of ordinary power supplies. Moreover, PoE-capable switches are more expensive and a shade harder to get ahold of. (Apologies for the mess in the photo, we should straighten that out. We also don't yet have a permanent location for this. Another group in our lab 3-D printed holders and a 19" rack mount frame for theirs, but we haven't gotten that far yet.)

Getting it all running was rather a pain. Here were our pain points and some advice:

• We accidentally installed ARM7 Ubuntu on some machines and ARM64 on others. This problem won't become apparent until you compile and run your own code on multiple nodes via MPI, at which point it will tell you "Exec format error," and you'll have to go back and make all the nodes agree on architecture & chipset support. This was the last major problem we had to solve, but I mention it first since it's one you want to get right up front. All things being equal, unless you're creating a mixed cluster with older hardware, you probably want the 64-bit installation.
• My first mistake was mixing installs of MPICH and OpenMPI. They are two separate implementations of MPI. Either is apparently fine, but don't mix them. If you just do
  sudo apt install mpi
  you will get OpenMPI. It doesn't include headers and the development tools, so you won't be able to compile. You also need the package mpi-default-dev.
• You need openssh-server, but that's usually included a default Ubuntu setup. Likely, you'll also need to install gcc, make, git and gdb.
• We're still tinkering with the best way to share setup info, including username databases and SSH keys for students and the like, but what we've settled on for the moment is Ansible, a popular networked systems management tool.
• We set things up to share the executable via NFS. (We're not doing data-intensive stuff, just introductory programming exercises for now, so we're not sharing some major data farm.) Getting permissions right here took a little bit of work.
• Our biggest pain point, which took the longest to solve, was getting the firewall settings right. Even though ompi_info tells me it's not compiled for IPv6, in fact the basic ssh that is used to initiate communications apparently runs over IPv6 anyway, if v6 is configured on our systems. Took us a couple of hours to figure this one out. Even when we briefly turned off the firewall entirely for debugging purposes, we were getting timeouts that baffled us. (ss was a big help here in figuring out what connections are trying to happen, but it takes a little greping to sort the wheat from the chaff.) (And random, 35-year-long rant: what is it with UNIX folks and short commands/tool names? "ss"? What is that?!? At least "netstat" has some mnemonic relationship to what it does.)
  Also, the default setting for Ubuntu firewall is "all outbound traffic allowed, no inbound traffic allowed," so even if you think you have the firewall entirely off, that might not mean what you think it means!

When your setup is close to working right, 

mpirun -np 2 --host raspi1,raspi2 hostname 

should print out the names of your hosts.  (Replace raspi1 and raspi2 with DNS names or IP addresses for your machines.) That just executes the command hostname on the remote host, showing that your communication is working. Since each machine has that command on it, it won't reveal the first problem above, the ARM7/64 issue.

 

That's just some quick notes in case you're running into similar problems. I'll try to flesh this out later.