Modern-Day Optical Network Physical Signal Encoding

(PAM-3 eye pattern; image from Wikipedia)

This blog posting is still in editing, and is posted just so I could talk to some students about elements of the contents.

Recently, I posted about how SDH optical networks encode bits at the physical level. My interest in the topic stems from a desire to know how to multiplex classical and quantum information on different channels/wavelengths, and part of that involves a basic understanding of the classical signals on the fiber. In both the demonstration network we are building and in the longer term as experimental specifications develop into standards, we may choose to put classical synchronization signals and the like for the quantum signals into the same fiber, or we may decide to carry full-on classical data traffic.

Of course, SDH is rather old now, going back to the 1990s, and optical networking has advanced considerably, especially for data center and local area networks. The most obvious place to look for newer developments is Ethernet, so here we are. First, let's look at almost-but-not-quite leading edge networks, where the technological decisions are more settled than in, say, Ultra Ethernet, which is still under development. (I do hope to come back to Ultra soon, but the draft specs are currently closed to the public.)

[tl;dr: PAM-4, with four distinct signal amplitudes, is common. Development of Ethernet using 16QAM was suspended after a draft specification was developed but not approved. As far as I can tell there is no standardized use of quadrature amplitude modulation in the optical regime, though it's common in RF.]

Many of Ethernet specifications are available for free from IEEE, including 802.3db-2022 - IEEE Standard for Ethernet - Amendment 3: Physical Layer Specifications and Management Parameters for 100 Gb/s, 200 Gb/s, and 400 Gb/s Operation over Optical Fiber using 100 Gb/s Signaling, and 802.3df-2024 - IEEE Standard for Ethernet Amendment 9: Media Access Control Parameters for 800 Gb/s and Physical Layers and Management Parameters for 400 Gb/s and 800 Gb/s Operation which mention PAM but not QAM.

400Gbps Ethernet has 11 separate physical layers that run over fiber (one still in development), two twisted-pair copper and one backplane form.  Let's focus on the fiber variants, since our interest here is photons in fibers (sometimes many of them, sometimes only one). Five of the variants listed at Wikipedia use PAM-4 (sometimes written PAM4 in the page), one is listed as 16QAM (but more below), and the rest don't say; perhaps they are simply on/off NRZ keying, like in earlier optical networks like SONET/SDH.  (It's nice that this information was much easier to find than the original SONET/SDH stuff!  Partly because I better understand what I'm looking for this time, I suppose.)

...So what are PAM-4 and 16QAM?

PAM, or Pulse Amplitude Modulation, is pretty straightforward: instead of using signal ON at full power and OFF to represent a single bit, if you use several voltages (or RF or optical signal strengths), then each symbol can represent more than one bit.   A related term is Amplitude Shift Keying; I'm not sure exactly why the Ethernet folks stick to PAM instead of ASK. The picture at the top of this posting is PAM-3: the level at the bottom, the level in the middle, and the level at the top. The sloping lines are transitions from one level to another; the cleaner those lines are, the bigger the "eye" is, indicating that your circuit is very stable. (If you don't know how to read an eye diagram, you should learn.)

PAM-4 uses four signal levels, carrying two bits per symbol. Prolabs and Samtec have nice explanations of PAM-4, including eye diagrams. I mentioned above that there are many different physical layers for Ethernet; PAM-4 is used in the 400GBASE-DR4, 400GBASE-FR8, 400GBASE-LR8, 400GBASE-FR4, and 400GBASE-LR4-6 variants, all running over four or eight single-mode optical fibers working together. (More on that multi-fiber concept some other time.)

That was easy. Whew! ...but what about 16QAM? Gotta mind your Ps and Qs...same thing? Nope! QAM is a lot more complicated, involving a lot of signal processing theory we're not going to get into in this blog, but let's take a quick look anyway.

QAM is Quadrature Amplitude Modulation. In QAM, we modulate the signal using two separate waves, one sine term and one cosine term. When using on-off keying or PAM on optical fiber, our carrier signal is just the laser light amplitude, and the system is relatively insensitive to the phase of the light. With QAM, however, the phase of the carrier is critical.

One of the most basic types of QAM is 16QAM.  When first getting oriented, I found this web page to be helpful, but keep in mind that it's talking about the use of 16QAM for radio signals, not optical. In 16QAM, we use two signals, I and Q, that are both sinusoids, 90 degrees out of phase. Each of those two signals is modulated to carry two bits, then the two are combined, so that I+Q is the signal transmitted. The modulation involves two choices of amplitude, and two choices of phase -- either 0 or 180 degrees added to the already 0 or 90. Since there are four possible choices for each of I and Q, we have a total of 16 possible waveforms, hence 16QAM. The 16 waveforms are shown below.

The wild thing is that you can recover the four bits cleanly from that.  I won't go into the math here, but you can check Wikipedia.

Those sixteen waveforms are arranged in a particular way, corresponding in order to what is called a constellation diagram. The constellation diagram helps you understand the noise in the system and figure out whether or not your hardware can reliably reconstruct the original set of bits.

A helpful, if not super-high quality, video:
https://www.youtube.com/watch?v=6BIqEWEe5-I

The 400Gbps Ethernet variant 400GBASE-ZR was supposed to use 16QAM. BUT:

The 802.3cw website says, "The work of the IEEE P802.3cw 400 Gb/s over DWDM Systems Task Force concluded with the withdrawal of IEEE P802.3cw PAR on 22 May 2024." Apparently, they published P802.3cw/D3.0, Dec 2023 - IEEE Draft Standard for Ethernet Amendment: Physical Layers and Management Parameters for 400 Gb/s Operation over DWDM (dense wavelength division multiplexing) systems as "Active - Draft" in Dec. 2023.  It's not available in the freely accessible documents yet (which only include approved standards at least six months old, as I understand it), and even my university account seems to be unable to reach it.  Too bad, that's definitely where I wanted to be looking!  https://www.ieee802.org/3/dj/public/24_03/motions_3cwdj_2403.pdf says the PAR (Project Authorization Request) was withdrawn by unanimous consent on 13 March 2024. This was foreshadowed a month earlier in an email from John D' Ambrosia, chair of the TF.

According to the Wikipedia page on terabit Ethernet, it was proposed in 802.3cw to use dual-polarization 16QAM, which might add an extra bit but sounds even more complicated to me.

I don't know yet where the carrier for reconstructing the signal comes from...if it's just the laser itself, that's about 200THz for 1.5um light, and we need some reference to find the right phase for the carrier. One research paper on carrier recovery:
https://opg.optica.org/jlt/abstract.cfm?uri=jlt-27-15-3042
No idea if that's what's used...

And another survey paper, heavily cited in papers and in (at least) 20 patents:
https://ieeexplore.ieee.org/abstract/document/5464309

As a companion to this posting, I am developing a Jupyter notebook on 16QAM that made the plot above.

Julia Parsons, "Code Girl", 1921-2025

(Image taken from Seattle Times, where it is credited the World War II Foundation.)

 The New York Times reported last week that Julia Parsons passed away. (The Seattle Times has a copy not behind a paywall.) She was probably the last living member of the WWII Naval Communications Annex team responsible for deciphering Enigma messages sent to and from German U-boats. She joined the WAVES in 1942, right after graduating from Carnegie Tech (now Carnegie Mellon University), and was assigned to work in the unit from 1943 through the end of the war.

I don't recall if she was mentioned by name in Liza Mundy's Code Girls, but she was definitely part of that crew.  If you haven't read that book, you really should.

As one of the youngest members of the group, her initial task was to work directly on the deciphering of the messages from the U-boats. She worked with the US Navy Bombe, feeding it possible plaintext and ciphertext. The Bombe would then produce a "menu" of possible Enigma wheel settings that had to be checked to determine which (if any) of them would correctly decrypt the message. The Wikipedia article has an excellent description of the workflow.

Because the work she did was classified, she didn't talk to anyone about it until 1997, when she discovered it had been declassified in the 1970s. We probably lost a lot of history that way, as even by the 1970s many of the senior people involved had doubtless passed away.

Thank you, Ms. Parsons, for what you did for democracy and freedom. I know it came with a cost.

Basic Signal Modulation in SDH Optical Networks

 Last night, I had an extremely basic question on how signals get onto an optical fiber, so I started looking for information on it, including the standards themselves.  Despite starting from what I think of as a reasonable base of knowledge, it took me over an hour to find the answer, so I figured I'd write it down here, for both myself and posterity.

[tl;dr: At least for the set of specs I looked at, it's simple on-off keying with NRZ (non-return to zero) encoding, though RZ is also possible; that is, the light being on is a 1, and light being off is a 0. The ones I looked at don't use exotic things like phase-shift keying or the like.]

I already knew that I wanted to look at the standards for SONET or SDH (Synchronous Digital Hierarchy), which are essentially the same, and I knew that ITU-T is the organization that handles SDH. ITU-T is a standards organization, part of the ITU, which is an agency of the UN (although ITU itself is much older than the UN, go figure).  It handles a lot of aspects of communication, but the part of interest here is Series G: Transmission systems and media, digital systems and networks.  (Simply determining that this is the set of specs to look at took me a long time.  Google doesn't really index into the ITU-T pages very well, and I wasn't familiar with the "Series" structure of the ITU-T standards.  Moreover, when Google does give you a direct link to the file, it's often an older version instead of the correct, up-to-date one.) (It's also worth noting here that these are published as "Recommendations", and don't have the force of law unless adopted into a law by some country.)

It was pretty easy to find information on frame formats for STM-1, which is the key transmission format for SDH.  Getting from there down to the physical layer encoding was the next step, where I stalled.  Here's what I eventually found, most of them in the G.95x Digital line systems series:

• G.691 : Optical interfaces for single channel STM-64 and other SDH systems with optical amplifiers 
  This spec isn't part of the G.95x series but it has more on physical layer, eye mask, rise times, power levels, dispersion accommodation, etc.
• G.955 : Digital line systems based on the 1544 kbit/s and the 2048 kbit/s hierarchy on optical fibre cables
  This is actually pretty old (1996), but hasn't been withdrawn.  There are lots of values for allowable attenuation, etc. that were simply listed as "under study".
• G.957 : Optical interfaces for equipments and systems relating to the synchronous digital hierarchy
  One of the most important, but be careful; it has been updated, so there are multiple versions floating around and the older one doesn't say "superseded".  You want the 200603 version, dated 03/2006. This one also talks about dispersion, if that's your gig.
• G.959.1 : Optical transport network physical layer interfaces
  Ah, finally, here's the money!!!  A few simple lines in Sec. 8.2.2.13: "The convention adopted for optical logic levels is:
• − emission of light for a logical '1';
• − no emission for a logical '0'."

G.959.1 also refers to "optical tributary signal class NRZ 2.5G" and several similar terms.  Knowing to Google for those may help in the future.  Also, this is where I got the eye mask figure (a "mask" of acceptable values for the eye diagram) at the top of this posting.

This still leaves me with some questions:

1. How is clock recovery done in the NRZ system?
2. How are frames demarcated?
3. How do you turn a laser diode on and off that fast in an electrical circuit?
4. Definitely need to study up more on filtering, DWDM (especially how close channels are allowed to be), and add/drop devices.

There are also newer optical networks, including the most advanced forms of today's Ethernet, and I'd like to look at the same questions about them. I'll come back to all of these another time.

Oh, and for what it's worth, check out the Y series, where you'll find Recommendations on Quantum Key Distribution, Internet of Things, and other leading-edge topics.

[Edit: The book Optical Networking Standards: A Comprehensive Guide for Professionals was published back in 2006. I found it after writing this blog post, but I have found it useful. My university has access to the PDF, yours may, too.]

Spelunking CACM, Vol. 23 (1980): Pilot and Medusa

The structure of the magazine is changing, but not the covers, which are still primarily black and blue, with some abstract design. The articles are longer and more in-depth, and each issue has only a few articles. Some of the older types of notices and articles, such as short algorithm descriptions, have been retired. And then rather suddenly around August or perhaps a little earlier, the article format itself changed and became more modern, with a nicer header and a three-column, large-magazine format.

Apparently there was a thing called the IBM Programmer Aptitude Test, which I have never heard of.  A quick check didn't find a copy of it online, but some discussion seems to indicate that it was a lot of math word problems. A couple of researchers tested it rigorously, and found, surprise, very weak correlation between the test result and grades in an introductory FORTRAN programming course. They also found an even weaker correlation between gender and performance.

Harold Abelson and Peter Andreae wrote about tradeoffs in VLSI design. Interestingly, this is the first time I recall seeing the term "VLSI", though maybe it just didn't catch my eye before. The term itself should have been only about three years old at the time (according to Lynn Conway's reminiscences), and yet the article doesn't bother to expand the acronym.

Not only was there a chess tournament featuring a dozen programs at a 1979 ACM conference, there was also an early attempt at a man-machine team, what we might now call "centaur" or "advanced" chess, playing against (in this case) a lone human. The article authors were relieved that the lone human won.

Xerox Business Systems (not PARC?) authors published about Pilot (the figure below), an OS for personal computers complete with a single 32-bit virtual address space.  (In fact, we might call it a 33-bit address space today, since addresses were to 16-bit words.) Pilot used a flat (non-hierarchical) namespace for files, each of which had a 64-bit unique identifier they refer to as a capability. The capability is supposed to be unique in space and time, across all machines. Frustratingly, the article doesn't contain much on the hardware required to run Pilot, but it's implemented in Mesa and very closely tied to that language.  Inter-process communication can be either via shared memory or the communication libraries provided, which primarily focused on the PUP protocol suite, though the describe similarities to the ARPANET protocol suite. Like TCP/IP, PUP includes internetworking concepts in it.

As long as we're talking operating systems, maybe the more interesting one technically is Medusa, which ran on the Cm* multiprocessor   (the figure at the top of this posting), developed at Carnegie Mellon University. The article by Ousterhout et al. describes an extremely sophisticated OS running on a distributed shared memory system. The hardware, like a NUMA system today, can directly access local or remote memory, with up to about a 10x latency penalty. The processors are LSI-11s, a version of the workhorse PDP-11 that was used for so many things over two decades and several hardware and OS iterations.

A task force includes activities, with the former roughly resembling a modern process and the latter corresponding to threads, where each activity has a specific role in the overall program/utility -- except that each activity is bound to a processor, but a task force can apparently consist of activities on many processors. One approach, shown below, is to have the many utilities (daemons, in modern terms) of the OS each running on a separate processor.

Arguably Medusa echoes some aspects of Farber's DCS and in turn influences things like VAXclusters, though neither of those systems had direct hardware access to remote memory, as far as I know/recall.

Guy Steele and Gerald Sussman described a LISP microprocessor.  Interestingly, despite the fame of that pair, this article hasn't been cited much; perhaps the commercial LISP machines of just a few years later don't really owe much to it?

Also, I hadn't realized that there were formal attempts to verify security in an OS that far back -- and using capabilities, to boot.

Enough for now. Once again, this is turning into a catalog rather than a dive into one or two pleasing papers, but it's intriguing to see so much on distributed OSes showing up. What a time it was, and I was still too young to participate at all, even though some of this major work was taking place driving distance from my parents' house, in Pittsburgh. If I had known then about that work, and that I wanted to do computing systems, I might very well have gone to CMU instead of Caltech, and how different my life would have been then. Although my life later converged with many good people from CMU!

Cross-validating Quantum Network Simulators

 New paper on the arXiv. I'll be presenting this one at an INFOCOM workshop in London next month.

During this cross-validation process, we not only fixed bugs in both simulators, but we gained a deeper understanding of the performance differences caused by protocol design differences.

Cross-Validating Quantum Network Simulators

Joaquin Chung, Michal Hajdušek, Naphan Benchasattabuse, Alexander Kolar, Ansh Singal, Kento Samuel Soon, Kentaro Teramoto, Allen Zang, Raj Kettimuthu, Rodney Van Meter

We present a first cross-validation of two open-source quantum network simulators, QuISP and SeQUeNCe, focusing on basic networking tasks to ensure consistency and accuracy in simulation outputs. Despite very similar design objectives of both simulators, their differing underlying assumptions can lead to variations in simulation results. We highlight the discrepancies in how the two simulators handle connections, internal network node processing time, and classical communication, resulting in significant differences in the time required to perform basic network tasks such as elementary link generation and entanglement swapping. We devise common ground scenarios to compare both the time to complete resource distribution and the fidelity of the distributed resources. Our findings indicate that while the simulators differ in the time required to complete network tasks, a constant factor difference attributable to their respective connection models, they agree on the fidelity of the distributed resources under identical error parameters. This work demonstrates a crucial first step towards enhancing the reliability and reproducibility of quantum network simulations, as well as leading to full protocol development. Furthermore, our benchmarking methodology establishes a foundational set of tasks for the cross-validation of simulators to study future quantum networks.

https://arxiv.org/abs/2504.01290

Open Faculty Positions at Keio's Shonan Fujisawa Campus

 We have five, count 'em, five, open faculty positions in Keio's Faculty of Environment and Information Studies. If you are a researcher looking for a tenure-track position, please consider applying.

This call is very open; people of all stripes are encouraged to apply. As chair of the Cyber-Informatics Program, of course, I am hoping we will have the opportunity to hire some first-rate people in computing (defined very, very broadly).

Applications are due the end of March, I believe.

https://www.sfc.keio.ac.jp/en/employment/

Africa and Foreign Aid Today

A week or so ago, as the scale of the USAID cuts was becoming clear, an acquaintance on Facebook gleefully posted a meme about Africa. At best it callously suggested that U.S. aid to Africa is having no effect. A more critical interpretation of the intent was the racist message that Africans have always lived in grass huts and always will, that it's a basket case not worth caring about.

I'm no expert on Africa; I have never set foot on the continent (unlike a number of friends here, some from the continent itself). I know only what I learned in college four decades ago (where I studied under the brilliant Ned Munger and the equally brilliant Thayer Scudder), what I read, and what I hear from working with students, postdocs and collaborators from Egypt, Eritrea, South Africa and Senegal. But if I don't speak up, who will?

First off, of course, Africa is not just one thing. Those four countries I just named are probably at least as different as Finland, Romania, Greece and the U.K. But Africa is often roughly divided into sub-Saharan Africa and North Africa (often lumped in with the Middle Eastern Islamic countries), and the links I include below follow that division.

Africa today is a dynamic and growing place. Over the last three decades, GDP across the continent has tripled, outpacing the growth of the U.S. economy, which doubled over the same period. An increasing number of countries have reached World Bank middle income or upper middle income status; check the links at the bottom of this post.

Africa is urbanizing rapidly. Currently around 45% of the continent lives in cities. This comes with its own problems of sanitation, water, electricity, pollution, transportation and general governance; no one would pretend it's perfect. But the idea of Africans all living in huts is...well, you know.

Life expectancy is also growing rapidly in many countries. Some countries have levels that rival those of some European countries.

These metrics vary dramatically across the continent and correlate strongly with the quality of governance. This is an area where European and American countries are particularly responsible, thanks to centuries of colonial rule and suppression of local voices, and the devastation to West African societies caused by the slave trade (yes, that ended long ago, and the effects persist today). Colonialism has only ended within our lifetimes. The colonies were set up by Europeans without regard to existing cultural, political and economic structures, and some African countries that were amalgams of various cultures were left unprepared to deal with the issues of governance.

Africa is full of ambitious, smart, creative, hardworking people. To get a glimpse, check out the CNN programs Inside Africa and African Voices Changemakers. (I wish CNN had programs as good on Asia!)

How much of all of this progress is due to U.S. foreign aid, I can't say. But PEPFAR alone has saved the lives of millions of Africans, and work done by NGOs on topics such as guinea worm eradication have improved quality of life for all.

We should all celebrate when any country or region grows and improves on these metrics, and worry and offer a hand when they don't.

We are all in this together. If you don't know how to care about other human beings, I don't know how to explain it to you.

https://data.worldbank.org/indicator/NY.GDP.MKTP.CD...

https://www.worldbank.org/en/country/mic/overview

Research is Winning the War Against Cancer

 Do I have any friends whose lives have not been touched by cancer?

If your life has been touched by cancer -- your own, a family member's, a friend's -- then you are probably aware of how difficult a disease it is, but you may or may not know this:

"The country’s cancer death rate has declined 33 percent since 1991, thanks largely to cancer research that has led to new treatments, gains in early cancer detection and, most significantly, a sharp decline in tobacco use, according to a new American Cancer Society (ACS) report."

In addition, five-year survival rates for many types of cancer have improved, and the treatments themselves have fewer side effects.

The research that leads to these improvements is a global effort, but the biggest funder of research (across all fields, from astronomy to zoology) in the world is the United States National Institutes of Health.

This kind of work takes DECADES. It is not work that can be started and stopped on the whims of individuals; an unplanned-for pause can literally destroy decades of work. It takes dedicated researchers, clinicians, and hospital staff; drug development science and chemical engineering; complicated distribution and testing systems; work to understand and mitigate tradeoffs in the effects on the body, work to sort out differences in effectiveness for people with different body types, current health and genetic backgrounds; and above all a system that is built to create AND SUSTAIN the workforce we need, including universities.

When we work for the common good, without thought of profit or politics, the U.S. makes the world a better place.

https://www.cancercenter.com/community/blog/2023/01/cancer-survival-rates-are-improving