http://www.geonius.com/writing/other/voyager.html

      +-----------+                                  +--------+
      || Writing || - GEONius.com ---- 18-May-2024 - | E-mail |
      +-----------+                                  +--------+

                         Forget Crazy Earth,
                         Look to the Future:
                        Voyager and Fortran 5

                                2024

The Voyager spacecraft, Voyager 1 and Voyager 2, were not programmed
in Fortran 5. Or in any Fortran for that matter. This widely accepted
misbelief can be traced back to a widely misunderstood statement in a
single magazine article.

1.  Introduction

2.  How It Started

3.  What is Fortran 5?

4.  Fortran V?

5.  So What Language was Used?

6.  Why Assembly?

Simulators & Other Random Thoughts

AACS, HYPACE, and HYBIC

Miscellaneous Information

Bibliography

This is a personal project of mine. Everywhere I turned the past few
years, someone was saying that the Voyager spacecraft were programmed
in Fortran 5. I didn't believe it. I gathered a few links here and
there, but, in early 2024, I sat down and did some serious online
research. Well, kind of serious and not particularly systematic. This
is what I found. It's wordy, it rambles, and the writing is somewhat
uneven, but I hope you'll find the information useful or at least
follow the links in your own explorations. There are many fascinating
aspects of the Voyager project that one normally might not think of
and only learn of by accident. I think you'll find that even casual
browsing can yield rich rewards!

You'll find phrases like "apparently", "perhaps", "it seems", and "it
suggests" throughout the text. They mean I can't definitively assert
what the sentence is about to say. For example, a great deal of my
discussion of sequencing and simulation is based on McEvoy's Viking
Orbiter paper. There is no comparable Voyager paper that provides his
level of detail, so I can't with certainty say that some thing done
on Viking is done in the same way on Voyager--it just seems likely!
These uncertainties do not affect the fact -- and central thrust of
this piece -- that the Voyager onboard computers were not programmed
in Fortran.

My background: I'm a retired software developer. In the 1980s, I
worked on the image processing ground system for NASA's LANDSAT 4 and
5 remote sensing satellites; we used VAX/VMS Fortran-77. In the late
1980s and early 1990s, I worked on a configurable, Unix
workstation-based, control center for NASA's Goddard Space Flight
Center, mostly written in C. In the late 1990s through the mid-2000s,
I worked on a similar system in C++ for commercial satellite fleets.
(The latter was originally based on the former thanks to NASA's
generosity in sharing technology with industry.) My brother got
interested in ham radio when we were in junior high school and I
picked up a rudimentary knowledge of electronics and radio through
osmosis.

The title of this piece is constructed from the titles of two songs
by Fortran 5 (Wikipedia) released on their 1991 album, Blues: "Crazy
Earth" (YouTube) and "Look To The Future" (YouTube)!

1.  Introduction

The Voyager program took advantage of a rare planetary alignment to
send two spacecraft on a tour of the solar system's gas giants:
Jupiter, Saturn, Uranus, and Neptune. In addition to the cameras and
scientific instruments, each Voyager has 3 computers (plus their
backups, for a total of 6):

  * Computer Command Subsystem (CCS) - is the basically the master
    controller of the spacecraft. As such, it "provides sequencing
    and control functions. The CCS contains fixed routines such as
    command decoding and fault detection and corrective routines,
    antenna pointing information, and spacecraft sequencing
    information." (Voyager: The Spacecraft, JPL) The CCS CPU is the
    same as used on the Mars Viking Orbiters.

  * Attitude and Articulation Control Subsystem (AACS) - "controls
    spacecraft orientation, maintains the pointing of the high gain
    antenna towards Earth, controls attitude maneuvers, and positions
    the scan platform." The cameras and a couple of other instruments
    are mounted on the scan platform, hence adjusting their
    orientation must be done in conjunction with the spacecraft's
    orientation. (Voyager: The Spacecraft, JPL) The AACS computer is
    not the at-the-time experimental HYPACE computer. This turned out
    to be an unexpected rabbit hole, so see "AACS, HYPACE, and HYBIC"
    below for more details.

  * Flight Data Subsystem (FDS) - "controls the science instruments
    and formats all science and engineering data for telemetering to
    Earth." (pre-launch Voyager Press Kit, August 4, 1977, p. 22,
    PDF) The FDS used a custom-designed CPU.

Dr. James E. Tomayko's Computers in Spaceflight: The NASA Experience
(1988) is an oft-cited, authoritative (and eminently readable) source
of information about computing systems in space up until the date of
the report. Chapter Six, Section 2 gives the history of and details
about the three Voyager computers. (Since the CCS computer was
borrowed from Viking, also see the more detailed description of the
Viking Orbiter CCS in Chapter Five, Section 6.)

The links above take you to an archived copy of the HTML version of
the study. Sometime in December 2023 or January 2024, NASA removed
the HTML version and redirected all links to this download page for a
PDF version. At first, it was a 500-MB PDF! After a query from me,
they replaced it with an optimized 36-MB PDF. The text is not as
crisp, but it's readable. (The higher-quality 500-MB PDF is still
available for online reading or download at the Internet Archive.)

And, expanding our vocabulary:

  * Flight Computers - are the computers onboard a spacecraft, like
    the three above.

  * Flight Software - is the collection of programs and routines that
    run on the flight computers. Like any computer program, the
    flight software is composed of low-level CPU instructions such as
    "Load CPU register A with the value at memory address 1234".
    Flight software engineers write and maintain these programs.

  * Spacecraft Commands - are binary commands that perform operations
    on the spacecraft. Uplinked to the spacecraft by mission control,
    they are not CPU instructions. In a spacecraft without computers,
    the commands are fed to a hardware decoder that effects the
    desired operations. In a spacecraft with computers, the software
    interprets the commands and runs the sometimes complex code
    needed to perform the operations. (Think of the Unix bash shell,
    where the "flight software" is the bash executable and the
    "spacecraft commands" are the command-line commands: cd, ls,
    etc.)

  * Command Sequences - are sequences of spacecraft commands uplinked
    to the spacecraft as a singular bundle. A given sequence may be
    executed immediately, executed at a scheduled time, or executed
    only when the spacecraft's hardware or software detects some
    condition. Sequence engineers design and develop command
    sequences. (A simplified definition of sequences; see Sequences
    and Simulators below for more details.)

In the telemetry data radioed back to Earth:

  * Engineering Data - has to do with the health and operation of the
    spacecraft; e.g., temperatures, power levels, spacecraft
    attitude, and readouts of flight computer memory for debug
    purposes!

  * Science Data - is that returned from the scientific instruments.
    The prime example is the image data from the cameras.

                 Voyager 1 Enters Interstellar Space

  "This artist's concept shows plasma flows around NASA's Voyager 1
          spacecraft as it approaches interstellar space."
 (Source: "Voyager 1 Enters Final Region of Our Solar System Before
       Interstellar Space", Mike Killian, AmericaSpace, 2013.)

A brief Voyager timeline (the fly-by dates are actually years of
closest encounters):

    1977 - Voyager 2 and then Voyager 1 launched.
    1979 - Both Voyagers fly by Jupiter.
    1980 - Voyager 1 flies by Saturn.
    1981 - Voyager 2 flies by Saturn.
    1986 - Voyager 2 flies by Uranus.
    1989 - Voyager 2 flies by Neptune.
    2004 - Voyager 1 crosses the termination shock.
    2007 - Voyager 2 crosses the termination shock.
    2012 - Voyager 1 enters interstellar space.
    2018 - Voyager 2 enters interstellar space.

After the 1980 encounter with Saturn, Voyager 1 angled north from the
ecliptic (planetary) plane at about 35deg. After the 1989 encounter
with Neptune, Voyager 2 angled south of the ecliptic plane at about
-48deg. (Voyager Fact Sheet, JPL)

Entering interstellar space means the Voyagers have emerged from the
heliosphere, the region in which solar winds predominate. The
spacecraft have not yet escaped the Solar System, as the Sun's
gravity still holds sway. That will take another 30,000 years until
the spacecraft cross the outer edge of the hypothetical Oort Cloud!

For a real treat, see the real-time Voyager Mission Status! I noticed
that Voyager 1's distance from Earth was rapidly decreasing (!) while
Voyager 2's was slowly increasing. At the same time, both
spacecraft's distances from the Sun were increasing at a constant
rate as expected, both at about 10 miles every second. I eventually
figured this out. (Completely missing, like an idiot, the "Distance
from Earth" link that pops up an explanation!) Earth is traveling
around the Sun at nearly twice the speed of the Voyager spacecraft.
For half of its orbit, Earth will be moving towards, for example,
Voyager 1 and is thus "catching up" to the spacecraft. (That's a
rough, simplified explanation. Given the Earth's and Voyager 1's full
3-D velocities, you must separate out the components along the
changing Earth-Voyager 1 axis. When the Earth's speed along that axis
exceeds Voyager 1's, the distance will decrease. So the distance will
decrease on only a portion of that half-year half-orbit.)

And the DSN Now page shows the real-time status of the Deep Space
Network antennas and which spacecraft they are talking to, VGR1 and
VGR2 being the Voyager spacecraft.


                            Suzanne Dodd

    "Voyager Project Manager Suzy Dodd shows off her spacecraft".
                       (Source: Bill_D@Flickr)

2.  How It Started

Wired

In 2013, Wired published an article based on an interview with
Suzanne Dodd, Voyager project manager at the time and, currently,
also Director of the Jet Propulsion's Interplanetary Network
Directorate. In the article, the author, Adam Mann, wrote (his words,
not a direct quote from Dodd):

    The spacecrafts' original control and analysis software was
    written in Fortran 5 (later ported to Fortran 77). Some of the
    software is still in Fortran, though other pieces have now been
    ported to the somewhat more modern C.

    --Adam Mann, "Interstellar 8-Track: How Voyager's Vintage Tech
    Keeps Running", Wired, September 2013.

My impression upon first reading this was that the ground-system
mission control and science data analysis software was written in
Fortran and parts were later ported to Fortran 77 and C. (I now think
Dodd was referring to mission control-related analysis instead of
science data analysis.) I'd never heard of "Fortran 5", but the rest
was unremarkable and tracked with my own experience. In the early
1980s, I worked on the image processing side of the LANDSAT-4/5
ground system, programming in Fortran 77 on VAX/VMS minicomputers.
The mission-control developers on the other half of our cubicle farm
at NASA's Goddard Spaceflight Center programmed in Fortran on
DEC-2060 mainframe computers. And, in the late 1980s, I was part of a
small team developing a generic, UNIX workstation-based ground
system, TPOCC, for Goddard -- programmed in C, of course!

So, aside from the "Fortran 5" oddity, Mann's article was spot-on and
provided an excellent update for me on the Voyager project. Others,
however, inferred more from the quote and, shortened to one sentence
with the "control and analysis" phrase dropped, a misleading form of
the quote spread in the popular press and in numerous hardware and
software forums. The inference or "meme", easily identified by the
"Fortran 5", is that the Voyagers' onboard computers were programmed
in Fortran, etc., etc., as shown here:

    The initial software program was Fortran 5, then they were
    reprogrammed during flight to Fortran 77, and finally C.

    --South Australian Doctor Who Fan Club Inc. (SFSA), "Voyagers
    disco party!", The Wall of Lies, No. 169, Nov-Dec 2017, p. 2. (
    4-page newsletter, PDF)

(I'm not dumping on the fan club. For my example, the juxtaposition
of Doctor Who and disco practically screamed, "Choose me! Choose me!
")

Popular Mechanics

I originally dismissed the following 2015 article in Popular
Mechanics as just another knock-off of the Wired article. However, I
was wrong and, upon rereading it more closely, I have to say it is a
superb article. Based on the author's own phone interview with
Suzanne Dodd, the article is about the Voyager project seeking a new
software developer to replace the soon-to-retire Larry Zottarelli,
"the last original Voyager [software] engineer". From other
information I gleaned from the web, it appears he was one of the
original flight software engineers, joining the project pre-launch in
the early-to-mid 1970s. Don't judge the article by this isolated
quote (again, the author's words, not Dodd's):

    Know Cobol? Can you breeze through Fortran? Remember your Algol?
    Those fancy new languages from the late 1950s? Then you might be
    the person for the job.

    --John Wenz, "Why NASA Needs a Programmer Fluent in 60-Year-Old
    Languages", Popular Mechanics, October 29, 2015.

Between Wired and Popular Mechanics, I'm starting to wish Suzanne
Dodd had been a bit clearer in these interviews about what the
programming languages were used for. On the other hand, in her
defense, Dodd has spent decades having to and trying to convey
scientific and technical information in a meaningful way to the
public and non-tech-oriented journalists (in general; I don't mean
Wenz).

In this particular case, I expect that knowledge of Fortran is needed
for some of the support software, but not for actually programming
the onboard computers. (Keep in mind that Fortran is not hieroglyphs!
Moderately experienced programmers should be able to pick it up
fairly easily.)

Aside: Algol? Really?! I have long taken some small measure of pride
in the fact that I am one of the few people in the U.S. who have
actually programmed in Algol! When I first caught the computer bug in
1977, I read one of Daniel McCracken's Fortran books and his Algol
book ... before I even took my first computer class and got access to
a computer. In two classes where the other students used Fortran, I
used Algol because I wanted to try it out -- I liked it. This was the
Norwegian University (NU) Algol 60 compiler on a Univac
1100-something mainframe.

Unrelated aside: In the Popular Mechanics article, Wenz contrasts the
Voyagers' programmability with that of other spacecraft with
fixed-hardware operation sequencers, such as ISEE-3. The generic,
UNIX workstation-based, misson control system I worked on at Goddard,
TPOCC (mentioned above), was used to replace ISEE-3's Xerox
Sigma-based (if I remember correctly) control system in the early
1990s! Launched in 1978, ISEE-3 was repurposed in 1982 as the
International Cometary Explorer (ICE) and underwent a complex set of
maneuvers leading to a rendevouz with Comet Giacobini-Zinner in 1985.
(See this cool graphic of the maneuvers.) (The mission control system
was called the ICE/IMP control center, so it was controlling two
spacecraft, the second probably being the Interplanetary Monitoring
Platform, IMP-8.)

3.  What is Fortran 5?

A distinct detail in every replication of this meme is "Fortran 5". I
haven't used Fortran since the 1980s, but the unknown-to-me Fortran 5
was an immediate red flag. In all the online discussions of this
subject that I've seen, no one questions "Fortran 5". Even in a
Fortran Discourse thread where the users are very knowledgeable about
Fortran. It can't just be me ... please!

On Wikipedia's Fortran page, Fortran 5 is called an obsolete variant
(i.e., non-standard version) of Fortran:

    Fortran 5 was marketed by Data General Corporation from the early
    1970s to the early 1980s, for the Nova, Eclipse, and MV line of
    computers. It had an optimizing compiler that was quite good for
    minicomputers of its time. The language most closely resembles
    FORTRAN 66.

Data General's Nova 840 minicomputer was introduced in 1973; a
contemporary DG brochure, "840: The Loaded Nova", touts the Fortran 5
compiler in its feature list:

    Fortran 5, an extremely thorough, multipass compiler that
    produces globally optimized code that's nearly as efficient as
    assembly language code

And Data General advertised Fortran 5 with "It's a Real Pig": "Pigs
are the Smartest Animals in the Barnyard"! The Fortran 5 compiler was
slow because it was taking the time to produce smaller and faster
executables. (Flickr image)

Data General's 1978 Fortran 5 Reference Manual (PDF) lists 1972 as
the first copyright date, incidentally the year the Voyager project
officially began.


                       PC or Mac? No, UNIVAC!

        "PC or MAC? UNIVAC.", The Piranha Club by Bud Grace.
 (Source: James McDonald Stewart's Response to Fankuchen Award 2001)

4.  Fortran V?

I've seen no mention of Data General computers in connection with the
Voyager project in my research, so the use of Fortran 5 on the
project seems highly unlikely.

My guess is that, when speaking to Wired's Adam Mann, Suzanne Dodd
said "Fortran Five", meaning "Fortran Roman Numeral Five", or Fortran
V. This is consistent with Fortran II being known as "Fortran Two"
and Fortran IV as "Fortran Four". (Fortran III, in case you're
interested, was an unreleased, internal IBM compiler dating to 1958.)

Fortran V, like Fortran 5, is one of Wikipedia's non-standard,
obsolete variants of Fortran:

    [In addition to Control Data Corporation,] Univac also offered a
    compiler for the 1100 series known as FORTRAN V. A spinoff of
    Univac Fortran V was Athena FORTRAN.

Now things fall into place. Remember that Suzanne Dodd spoke of
"control and analysis" software. She joined the Voyager team in 1984
as a sequence engineer in the lead-up to Voyager 2's January 1986
fly-by of Uranus. These engineers designed/developed sequences of
spacecraft commands for upload to the Voyagers ... work done on one
or more Univac 1100-series mainframe computers. From an interview:

    DODD: When I first started, I started on a-- I don't even remember
    the name of it, but I do recall it had an eight-inch floppy
    drive. That was our command medium, an 8-inch floppy drive. Not a
    memory stick, not even a CD. When we did these designs and plots
    that showed--there was a program that you could design an
    observation, like you want to make a mosaic over Uranus, and you
    could lay out the observation, and then it would tell you what
    the commands you need to do it are. That was done on a UNIVAC
    computer, so kind of more of a mainframe refrigerator-size
    computer.

    David Zierler, "Suzy Dodd (BS '84), Engineer and Deep Space
    Pioneer" Caltech Heritage Project, June 9, 2023.

I found more specific information about JPL's mission control
computing facilities in a pre-launch press kit (Voyager Press Kit,
August 4, 1977, pp. 106-107, PDF):

  * Mission Control and Computing Facility (MCCF) - three IBM 360-75
    mainframes used for the day-to-day operations of the spacecraft,
    including command uploads and tracking.

  * General Purpose Computing Facility (GPCF) - three Univac 1108
    mainframes used for "navigation and mission sequence systems ...
    [and] prediction programs and detailed spacecraft engineering
    performance analysis". These are the systems that Suzanne Dodd
    would have used. (Assuming JPL hadn't upgraded the 1108s to newer
    models in the intervening 7 years.)

  * Mission Test and Computing Facility (MTCF) - "three strings of
    UNIVAC and Modcomp computers to receive, record, process and
    display" the engineering and science data downloaded from the
    spacecraft. (The Univac computers here were minicomputers such as
    the UNIVAC 1230. The Modcomp minicomputers included the MODCOMP
    II and MODCOMP IV; for some reason, Tomayko [Chapter 8, Section
    3, p. 265] calls them the ModComp 2 and 4, respectively, though
    everyone else on the web uses the Roman numerals!)

Sun Kang Matsumoto is a CCS flight software engineer who joined the
Voyager program in 1985, also, like Dodd, in the ramp-up for the
Uranus fly-by. In a 2016 paper about the Voyager Interstellar Mission
(VIM), she gives some more details about the evolution of Dodd's
"control and analysis" software from Univac to Unix (and thus from
Fortran to C, presumably):

    During the prime mission and early VIM, Voyager had been using
    the JPL-developed software programs called SEQTRAN (to generate
    sequences) and COMSIM (to simulate sequences and CCS FSW
    changes). They ran on now-antiquated UNIVAC mainframe computers.
    Shortly after the start of VIM [in 1990], these programs were
    converted over to more modern UNIX-based SEQTRAN and High Speed
    Simulator (HSSIM). They were rewritten to maintain the same
    functionality of the old SEQTRAN and COMSIM, and tailored for
    VIM. Rewriting and testing required significant effort from the
    developers and the project personnel; however, the end result is
    much improved speed and efficiency.

    Sun Kang Matsumoto, "Voyager Interstellar Mission: Challenges of
    Flying a Very Old Spacecraft on a Very Long Mission" (PDF) p. 4,
    2016 SpaceOps Conference, Daejeon, Korea.

SEQTRAN and COMSIM/HSSSIM are programs Dodd would have used as a
sequence engineer. As a connoisseur of good program names, I can't
fail to mention Matsumoto's mention of two new programs: VAMPIRE
(Voyager Alarm Monitor Processor Including Remote Examination) and
MARVEL (Monitor/Analyzer of Real-time Voyager Engineering Link)!
Matsumoto's paper is well-worth reading not only for the technical
information, but because she provides a valuable portrayal of how the
Voyager team has managed to overcome, by hook and by software crook,
the very real problems of aging hardware at a great distance. And
have done so despite steep technological and budgetary challenges.

I think (or hope) it is clear by now that, in the 2013 Wired article,
Suzanne Dodd was referring to the ground system software and not the
software running on the flight computers in space.

Okay, Univac and Fortran. But what about Fortran V? I could only find
one instance of "Fortran V" being explicitly mentioned in connection
with a Voyager mission operations program. (I do not include academic
papers whose authors used Fortran V programs to analyze the Voyagers'
science data at institutions outside of JPL; e.g., universities.)

The scarcity of "Fortran V" and "Voyager" on the web is not
surprising. Most programming languages maintain backwards
compatibility between versions, so, for example, a Fortran IV program
usually can be recompiled with a Fortran V compiler without error and
with identical functionality. Consequently, programmers do not in
general specify versions when speaking about languages. A C
programmer doesn't say, "I wrote this program in C89 and later ported
it to C99." Instead, it's just an unadorned "C program". (This is not
a slight on the Dodd-Mann remark.)

Also, Univac's Fortran V compiler was available as early as 1966, so
it probably would have been the default Fortran compiler on Univac
mainframes throughout the 1970s. Many Fortran users may therefore
have been unaware that the specific version they were using was
Fortran V. (The documentation that I could find online showed that a
generic @FOR invoked the Fortran V compiler and @FTN invoked the
ASCII Fortran compiler discussed in the second aside below.)

Aside: I anticipate eventually having to eat my pontificating words
on programming language versions! In the meantime ... I couldn't find
an online copy of Univac's Fortran V manual, but the 1970 UNIVAC 1108
System Description (Section 10, p. 7) had this to say: "FORTRAN V,
being an outgrowth of the earlier FORTRAN languages (in particular,
UNIVAC 1107 FORTRAN IV and IBM FORTRAN IV as announced in IBM form
C-28-6274-1), accepts these languages as compatible although the
reverse is not necessarily true."

Aside: The transition to Fortran 77 on a Univac mainframe would have
been non-trivial. Univac's Fortran 77 compiler, ASCII FORTRAN,
largely accepted Fortran V source code with a few, documented, mostly
Univac-specific exceptions. The big problem, I think, would have been
character sets. Univac's 36-bit mainframes used 6-bit FIELDATA
characters stored 6 per 36-bit word. The ASCII FORTRAN compiler used
the incompatible 7-bit ASCII character set. (On the Univac 1108
computer I worked on c. 1980, ASCII characters were stored as 9-bit
quantities, 4 per word.) Vanilla character strings in the source code
would just be converted to ASCII without complaint by the compiler,
but programs that depended on specific characteristics of FIELDATA
characters would require changes. That includes programs that must
read or write FIELDATA-compatible tapes. (I have no experience with
ASCII FORTRAN, so these are just my thoughts from perusing the ASCII
FORTRAN Reference Manual; Appendix A addresses the differences
between FORTRAN V and ASCII FORTRAN.)

Unrelated aside: Circa 1980, I overheard a graduate student remark
that the shell pipe, |, was Unix's gift to mankind and @ADD was
Univac's gift to mankind. He was right!

The Voyager ground-system software I found that was explicitly stated
to have been written in Fortran V is the Orbit Determinaton Program
(ODP), which was also used on other missions. The mathematics in the
following 1983 paper is way beyond me, but it establishes the ODP's
host system:

    To give an idea of the computational burden that is involved,
    consider a typical radiometric SRIF/SRIS solution with 67 state
    variables (Table I). This model contains only 4 process noise
    states (line-of-sight acceleration and 3 camera pointing errors);
    there are 3500 data points and 132 time propagation steps. The
    problem run on a UNIVAC 1110, in double precision, used 275 CPU s
    [econds] for filtering; smoothed solutions and covariance
    computation used 265 CPU s[econds]. The entire run scenario
    including trajectory variational equation integration, observable
    partials generation, solution mapping, and generation of smoothed
    residuals used 4320 CPU s[econds].

    ...

    We note in closing this factorization algorithm discussion that
    the SRIF and U-D algorithms that were used in this application
    have been refined and generalized, and are commercially available
    in the form of portable Fortran subroutines.

    --James K. Campbell, Stephen P. Synnott, and Gerald J. Bierman; "
    Voyager Orbit Determination at Jupiter" (PDF), IEEE Transactions
    on Automatic Control, Vol. AC-28, No. 3, March 1983, pp. 259-261.

For those not mathematically inclined, the paper is still worth
skimming just for the enumeration of some of the esoteric details of
the Voyager spacecraft that they had to account for in the
calculations. We learn what language was used in a 2008 presentation
by Lincoln J. Wood of JPL:

    The mainframe computers used include the IBM 7090, IBM 7094,
    UNIVAC 1108, and UNIVAC 1100/81. During the 1980s the ODP was
    transported to minicomputers, with the software maintained in
    both mainframe and minicomputer operating environments to fulfill
    the desires of various flight projects. The ODP consisted of
    300,000 lines of code as of 1979, with FORTRAN V being the
    primary language.

    --Lincoln J. Wood, The Evolution of Deep Space Navigation:
    1962-1989" (PDF), p. 6, 31^st Annual AAS Guidance and Control
    Conference, 2008, Breckenridge, Colorado.

Shortly after writing the above, I came across this entry in NASA's
long-gone, COSMIC software catalog:

    Calculating Trajectories and Orbits
    Two programs calculate the motions of spacecraft and landers.

    The Double Precision Trajectory Analysis Program, DPTRAJ, and the
    Orbit Determination Program, ODP, have been developed and
    improved over the years to provide the NASA Jet Propulsion
    Laboratory with a highly reliable and accurate navigation
    capability for their deep space missions like the Voyager. DPTRAJ
    and ODP are each collections of programs that work together to
    provide the desired computational results. DPTRAJ, ODP, and their
    supporting utility programs are capable of handling the massive
    amounts of data and performing the various numerical calculations
    required for solving the navigation problems associated with
    planetary fly-by and lander missions. They were used extensively
    in support of NASA's Voyager project.

    ...

    DPTRAJ-ODP is available in two machine versions. The UNIVAC
    version (NPO-15586) is written in FORTRAN V, SFTRAN, and
    ASSEMBLER. (A processor is supplied for SFTRAN, a structured
    FORTRAN.) DPTRAJ and ODP have been implemented on a UNIVAC
    1100-series computer. The VAX/VMS version (NPO-17201) is written
    in FORTRAN V, SFTRAN, PL/1 and ASSEMBLER. It was developed to run
    on all models of DEC VAX computers under VMS and has a
    central-memory requirement of 3.4 Mb. The UNIVAC version was last
    updated in 1980. The VAX/VMS version was developed in 1987.

    --NASA Tech Briefs, September 1989, p. 33.

A whopping 3.4 MiB of memory required for a lousy 300,000-line
program? Software hogs like that were the reason why we couldn't have
nice things back then ...

                             Voyager Art

         Adam's Voyager Spacecraft for Kerbal Space Program.

5.  So What Language was Used?

Let's ask an actual Voyager flight software engineer what language
was and is used to program the onboard computers. Perhaps the Voyager
Fault Protection and CCS Flight Software Systems Engineer who's
worked on Voyager since 1985:

    Both the AACS and FDS use assembly language. The CCS uses
    assembly language and Voyager-unique pseudo code (interpreter).
    As a result, it is difficult to attract younger programmers to
    join the project.

    Sun Kang Matsumoto, "Voyager Interstellar Mission: Challenges of
    Flying a Very Old Spacecraft on a Very Long Mission" (PDF) p. 6,
    2016 SpaceOps Conference, Daejeon, Korea.

In the acknowledgements at the end of the paper, Matsumoto thanks
fellow flight software engineer Larry Zottarelli, now retired, and
Suzanne Dodd, among others, for reviewing the paper. I assume
Zottarelli, Dodd, or one of the others would have spoken up if they
took issue with the above paragraph.

That would be two or three assembly languages: one for the CCS CPU,
possibly a variant for AACS's modified CCS CPU, and definitely
another for the completely different FDS CPU.

The phrase "pseudo code (interpreter)" makes me think of an embedded
scripting language, but Matsumoto is probably referring to executing
uplinked spacecraft commands and sequences. The CCS also must process
internally generated power codes received from the AACS computer:

    A "power code" is a 6 bit message sent to the CCS computer, which
    may be only informational or may cause a power command to switch
    power to an AACS component. Such power switching commands are
    usually the means by which redundant elements are exchanged.
    These power codes are an important part of the fault protection
    logic, allowing the CCS computer to issue commands in response to
    a fault condition. These commands may be a simple power command
    (A gyro on) or a command sequence which will turn the spacecraft
    in a pattern designed to re-oriented the spacecraft towards the
    sun from an entirely random attitude. Some serious faults result
    in an OMEN power code, which causes CCS to save the next three
    power codes (normally lost) for later analysis.

    Assessment of Autonomous Options for the DSCS III Satellite
    System Prepared for the U.S. Air Force by JPL personnel (Donna L.
    S. Pivirotto and Michael Marcucci?), "Volume III: Options for
    Increasing the Autonomy of the DSCS III Satellite" (PDF), p. 179,
    Aug. 6, 1981.

In her paper, Matsumoto discusses a problem with the "power decoder
relay matrix":

    The power decoder relay matrix problem that first manifested in
    1998 makes commanding of the spacecraft extremely difficult.
    Basically, the faulty decoder may cause an issuance of extraneous
    power commands in addition to the intended command.

    Sun Kang Matsumoto, "Voyager Interstellar Mission: Challenges of
    Flying a Very Old Spacecraft on a Very Long Mission" (PDF) p. 8,
    2016 SpaceOps Conference, Daejeon, Korea.

I wonder if this is the interface used to convey the power codes from
the AACS to the CCS. Later, she describes a patch for this error made
to AACSIN, the CCS routine that she notes receives and processes
power codes, but the specific description of the patch again uses the
term, "power commands".

                                                           * The CPUs
                                                 * Memory Constraints
                                            * Performance Constraints

6.  Why Assembly?

Since so many commenters in discussion forums had no problem with
"Fortran 5", it might be valuable to examine some possible reasons
why the flight software was written in assembly language.

The CPUs

There is not much information available specifically about the
Voyager CCS computer, borrowed from the Viking Orbiter. Fortunately,
Tomayko does provide details about the Viking CCS:

    In general, the design of the processor was exceedingly simple,
    yet fairly powerful, as indicated by the use of direct
    addressing, a minimal set of registers, and a reasonably rich set
    of 64 instructions. The key is that the design placed relatively
    light demands on spacecraft resources while replacing both the
    programmable sequencer and the command decoder used in the
    Mariners. The fact that the processor was later adopted by the
    Voyager project as its Command Computer and modified for use as
    the attitude control computer is not only a statement of JPL's
    frugality but also a testament to the versatility of the design.
    (p. 157)

    ...

    [On the Viking Orbiter, the] Command Computer's central processor
    contained the registers, data path control and instruction
    interpreter. The machine was serial in operation, thus reducing
    complexity, weight, and power requirements. It had 18-bit words
    and used the least significant 6 bits for operation codes and the
    most significant 12 for addresses, as numbered from right to
    left. This permitted 64 instructions and 4K of direct addressing,
    both of which were fully utilized. Data were stored in signed
    two's complement form, yielding an integer range from -131,072 to
    +131,071. Average instruction cycle time came to 88 microseconds.
    Thirteen registers were in the Command Computer, mostly obvious
    types such as an 18-bit accumulator, 12-bit program counter,
    12-bit link register that pointed to the next address to be read,
    and a 4-bit condition code register that stored the overflow,
    minus, odd parity, and nonzero flags. (Box 5-3, p. 159)

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Five, Section 6.

Note the 12-bit addresses, giving an addressing range of 0..4095. And
note, in the following, that the Viking CCS flight software engineers
masked most interrupts in routines, thus simplifying code (which
reduced memory usage) and increasing performance:

    Viking Orbiter software had to be written in an assembler, which
    fortunately had relocatable addresses, simplifying the
    maintenance task. The 64 instructions were mostly common to other
    computers, but there was no multiply or divide. There were two
    sets of loads, stores, increments, and subroutine calls: one used
    during independent operation and one aimed at dual operation, so
    that the two memories could be kept equivalent. Even though many
    interrupts were available, most routines as coded had all but the
    internal error and counting interrupts disabled. Many routines
    were free to run out without being interrupted, in contrast to
    the highly interrupted Apollo and shuttle software. Programmers
    avoided the memory and processing time overhead required to
    preserve the current accumulator and register contents during an
    interrupt.

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Five, Section 6, Box 5-4, p. 162.

The Voyager AACS computer uses a modified CCS CPU (e.g., with the
added index register), so most of the CCS details above apply as well
to the AACS. (In Box 6-1, p. 179, Tomayko uses "HYPACE" to refer to
the AACS despite pointing out two paragraphs earlier that the term is
a misnomer.)

The FDS CPU was developed especially to meet the high-rate data
handling requirements of the Voyager spacecraft. Designed by Jack L.
Wooddell, the CPU, in the form of a breadboard prototype, evolved in
a collaboration between Wooddell and flight software engineer Richard
J. Rice. For some reason, I picture Wooddell seated in front of a
really large breadboard with a soldering iron in one hand and a
wirewrap tool in the other. Rice is standing beside him with an open
laptop in the crook of his left arm, his right hand is typing on the
keyboard, and the changing flight software is downloaded via wi-fi to
the breadboard computer for testing. Wrong era, I realize, and Rice
may well have been left-handed!

    Voyager's data computer is different from most small
    general-purpose computers in several ways. Its special registers
    are kept in memory, permitting a large number (128) of them.
    Wooddell also wrote more powerful shift and rotate instructions
    because of data-handling requirements. Despite its I/O rate, the
    arithmetic rate is quite slow, mostly due to byte-serial
    operation. This meant 4-bit bytes are operated on in sequence.
    Since the word size of the machine is 16 bits, it takes six
    cycles to do an add, including housekeeping cycles. If all the
    arithmetic, logic and shifting were not done in the general
    registers, the machine would have been even slower. Reflecting
    its role, in addition to the usual ADD, SUB, AND, OR, and XOR
    instructions found on most computers, the data computer has many
    incrementing, decrementing, and branching instructions among the
    36 defined for the flight version of the machine.

    Overall, the Flight Data System requires 14 watts of power and
    weighs 16.3 kilograms. Its computer needs just one third of a
    watt and 10 volts, less than the power required for a temperature
    sensor! At first the estimated throughput required was 20,000
    16-bit words per second. By flight time, the instruction
    execution rate was 80,000 per second, with data rates of 115,000
    bits per second, much higher than previous Flight Data Systems.

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Six, Section 2, Box 6-2, p. 184.

Note that Tomayko uses the term byte-serial for non-8-bit "bytes", in
this case, when describing how the FDS CPU's arithmetic logic unit
processes 16-bit operands 4 bits at a time. The last two sentences
about throughput, while not wrong, seem to ask the reader to compare
apples and oranges.

The breadboard prototype began with the same 4K-by-18 plated wire
memory as the CCS computer. I don't know if this means the CPU was
originally designed as a 18-bit processor, but, in any case, it ended
up as a 16-bit processor with 8K-by-16 CMOS memory. From Tomayko's
description on p. 183 of Chapter 6, Section 2, my impression is that
the CPU used the same 12-bit addresses as the CCS, but two,
independent-of-each-other address lines were added that effectively
provided a 13-bit (8K) address range for instruction fetches and a
13-bit (8K) address range for data accesses. I don't know if these
extra address bits were incorporated into the CPU's instruction set
or if they had to be explicitly set/reset in a separate operation by
the programmer.

Don't be dismayed by the CCS and AACS CPUs' lack of multiply and
divide instructions and the FDS CPU's slow arithmetic operations.
Spacecraft computers don't crunch numbers. (Out of necessity, in my
ground-software experience up to the mid-2000s. More modern flight
computers with more powerful processors and abundant RAM may have
changed this dynamic.) Sensors and detectors return unit-less raw
counts. If these readings are needed in real-time for autonomous
spacecraft operations, it is faster and more memory-efficient for
command sequences or flight software to work with these counts
directly. Otherwise, the ground system can apply its sizeable CPU,
RAM, and disk resources to perform what are called engineering unit
conversions; e.g., the conversion of a hypothetical temperature
sensor's raw 0-255 count to degrees Celsius. The FDS computer is
focused on assembling and formatting science and engineering data for
transmission back to Earth, so fast bit and byte manipulation is more
important than fast arithmetic operations.

Memory Constraints

Wise words:

    The Voyager flight software design was very heavily impacted by
    the limited memory space. Extensive effort and ingenuity was
    required to perform the necessary functions in the available
    space. The flight software was written in assembly code.

    Assessment of Autonomous Options for the DSCS III Satellite
    System Prepared for the U.S. Air Force by JPL personnel (Donna L.
    S. Pivirotto and Michael Marcucci?), "Volume III: Options for
    Increasing the Autonomy of the DSCS III Satellite" (PDF), p. 179,
    Aug. 6, 1981.

Words that annoy me:

    The computers aboard the Voyager probes each have 69.63 kilobytes
    of memory, total.

    --Adam Mann, "Interstellar 8-Track: How Voyager's Vintage Tech
    Keeps Running", Wired, September 2013.

An otherwise good article, but ... First, I'm old-school, so the use
of "kilobytes" in the modern 1,000 sense annoys me. Second, the
pointless (and truncated) precision of 69.63 is annoying. Third, the
sentence is wrong because of the confusing wording. The computers
don't each have that much memory. The sum total of memory for all 3
computers and their 3 backups is 69,632 bytes:

     CCS - 4,096 18-bit words =  73,728 bits =  9,216 bytes
    AACS - 4,096 18-bit words =  73,728 bits =  9,216 bytes
     FDS - 8,192 16-bit words = 131,072 bits = 16,384 bytes
                                               ------
                                               34,816 bytes
                                              x     2 for the backup computers
                                               ------
                   Total for all 6 computers:  69,632 bytes

When examining the memory limitations from a programming standpoint,
expressing the memory size in bytes can be misleading. The CCS
computer, for example, has 9,216 bytes of RAM. Readers may mistakenly
infer from that figure that there are 9,216 addressable code/data
units when, in fact, there are only 4,096 addressable code/data
units, a significantly lower number that more accurately reflects the
limits in which the programmers had to work.

For example, if you want a variable that counts from 1 to 10, you
can't use a memory-efficient single byte and must instead use 21/4
bytes (18 bits). Yes, you can pack multiple data items into an 18-bit
word and I'm sure the Voyager assembly language programmers did just
this, but doing so incurs added memory/performance costs for packing
and unpacking the individual data items. And, yes, clever coding can
reduce those costs and or even avoid the need for packing/unpacking
altogether. Which makes a strong case for using assembly language
rather than a higher-level language like Fortran 5!

A given computer's memory holds the flight software code (CPU
instructions), the software's dynamic variables, and uplinked data
(tables, command sequences, and such). Regarding the AACS, Dr.
Tomayko wrote (emphasis added by me):

    The programmers must have done an outstanding job, considering
    the slow processor and limited memory. At launch, only two words
    of free space remained in the 4K of plated wire. Tight memory is
    now a problem because the scan platform actuators on Voyager 2
    are nearly worn out, and software has to compensate for this
    during Uranus and Neptune encounter periods.

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Six, Section 2, p. 178.

(Interestingly, Tomayko's 1988 report was published after the Uranus
encounter and before the Neptune encounter! He was writing not a
history of the distant past, but an account of a still very much
active and prominent project.)

A big chunk of Voyager 2's CCS memory is consumed by the Backup
Memory Load (BML), identified by name in Matsumoto's paper and
described by Tomayko as follows:

    As pioneered on Mariner X, a disaster backup sequence was stored
    in the Voyager 2 CCS memory for the Uranus encounter, and later
    for the Neptune encounter. Required because of the loss of
    redundancy after the primary radio receiver developed an internal
    short, the backup sequence will execute minimum experiment
    sequences and transmit data to earth; it occupies 20% of the 4K
    memory.

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Six, Section 2, p. 176.

In other words, if Voyager 2 loses the ability to receive commands
from the ground, the BML ensures that, on its own, the spacecraft
will continue to perform science operations and to downlink science
and engineering data.

Fault protection is very important in a remote unmanned spacecraft
and is, incidentally, one of Sun Kang Matsumoto's areas of expertise.
In the 1981 Assessment of Autonomous Options for the DSCS III
Satellite System, Volume III, we learn that 8 CCS fault routines take
up 26.5% of the CCS memory (Table B-3, p. 176) and 9-plus AACS fault
routines take up 19.4% of the AACS memory (Table B-4, p. 180). This
suggests that nearly half of Voyager 2's CCS memory was devoted to
the fault routines (26.5%) and the BML (20%), leaving only a little
more than half of the memory, about 2,200 words, for other CCS
functions such as command sequence storage and processing:

            Fault S/W     BML        Everything Else!     

(That's a graph, not a diagram of the memory layout.) This seems hard
to believe and the sizes could well have changed up or down over the
decades since this 1981 study and Tomayko's 1988 report.

A couple of weeks after I wrote the above, I came across a 1987 paper
by the Voyager Flight Operations Manager, published between the
Uranus and Neptune encounters. This paper seems to indicate that, on
Voyager 2, there was even less memory available for sequence storage
and processing, about 1,250 words:

    Out of the total CCS memory, the fault recovery routines, other
    fixed routines, and flight reserve leave only about 2,500 words
    available for sequencing of preplanned activities. These
    activities involve spacecraft maneuvers, scan platform movements,
    data mode changes, data recording or playback, scientific
    observations, etc.

    ... skip one paragraph ...

    Since the failure of one of the Voyager 2 command receivers in
    1978, that spacecraft has had a back-up sequence stored in its
    CCS that occupies approximately one-half of the available
    sequencing space. In the event of a failure of the remaining
    receiver, this back-up sequence will execute until late in 1989,
    allowing for limited mission return including data-taking during
    the Neptune encounter.

    Terrence P. Adamski, "Command and Control of the Voyager
    Spacecraft" (abstract) p. 3, American Institute of Aeronautics
    and Astronautics (AIAA), 25th AIAA Aerospace Sciences Meeting,
    March 24-26, 1987, Reno, Nevada.

(And a couple of more weeks later, I found the exact number: 1,239
words per CCS computer! See Morris's 1986 paper, p. 173.)

The "fixed routines" would probably have included the core software
such as the executive discussed below, interrupt handlers,
communications with the AACS and FDS computers, ground
communications, etc. At the beginning of the Voyager Interstellar
Mission in 1990, a BML was installed on Voyager 1 too (see
Matsumoto's paper, p. 3). It and probably an updated Voyager 2 BML
would have been tailored to the post-planetary-encounter,
interstellar mission. (I suspect that Voyager 2's original BML,
intended for the Uranus and Neptune encounters, could have been
reduced in scope and size afterwards.)

Performance Constraints

I couldn't find much detailed information about the actual flight
software. Both Tomayko and the Autonomous study describe the cyclic
executive used on the AACS computer to schedule routines (functions)
in regularly-spaced time slots:

    A flow chart of the AACS flight software is shown in Figure 4-14
    in the main body of this report. Normal program execution occurs
    in three different rate groups having periods of 10 ms, 60 ms,
    and 240 ms. The fourth rate group shown (20 ms) was used only for
    the Propulsion Module operation. Functions requiring high rates
    such as thruster activation and scan platform stepper motor
    operations are performed by the 10 ms logic. The bulk of the
    attitude control functions, such as attitude sensor 'reads' and
    control law algorithms, are accomplished by the 60 ms logic. The
    240 ms logic performs a variety of tasks that do not require the
    higher execution rates, such as decoding CCS commands from the
    input buffer, fault monitor and correction, and "power code"
    processing.

    Assessment of Autonomous Options for the DSCS III Satellite
    System Prepared for the U.S. Air Force by JPL personnel (Donna L.
    S. Pivirotto and Michael Marcucci?), "Volume III: Options for
    Increasing the Autonomy of the DSCS III Satellite" (PDF), p. 179,
    Aug. 6, 1981.

In a more visual form, here are the 10-ms time slots and the 60- and
240-ms intervals:

     .-- 10 ms
     v
    |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
    |   <- 60 ms ->   |                 |                 |                 |
    |                              <- 240 ms ->                             |

Difficult-to-update, hand-drawn flowcharts are a thankfully retired
relic of the 1970s software industry and I provide here a pseudocode
version of the scheduling algorithm on the AACS. It is not apparent
from the Autonomous study's flowchart, but I do make an assumption
that the 60-ms routines, for example, are not all run in a single
10-ms time slot every 60 ms. Instead, the per-slot load is lessened
by distributing them over the six 10-ms time slots in the interval.
So, given 60-ms routines ABC and DEF, then ABC might be run in the
10-ms slots 2, 8, 14, 20, 26, ... and DEF might be run in slots 5,
11, 17, 23, 29, etc.

    Offset60 = 0
    Offset240 = 0
    EVERY 10 ms DO {
        Run all of the 10-ms routines.
        Run any 60-ms routines scheduled for time slot Offset60.
        Increment Offset60.  If 6 then reset Offset60 to 0.
        Run any 240-ms routines scheduled for time slot Offset240.
        Increment Offset240.  If 24 then reset Offset240 to 0.
    }

Within a time slot, all the scheduled routines must be sure to finish
before the start of the next time slot. The programmers must quantify
beforehand the worst-case execution times for each of the routines
and ensure that the sum does not exceed 10 ms. The programmers
probably are counting instruction cycles for the assembly statements
in each routine. (I use the present tense because this would have
been done for the routines prior to the 1977 launch and still needs
to be done for routines updated/uploaded nearly 50 years later.) If
the time exceeds 10 ms and offloading some 60-ms and 240-ms routines
to other time slots is not a possibility, the programmers must speed
up the routines. This is most easily done working directly with the
assembly language. It would be awkward to do this in a higher-level
language because the quanitification still has to be done at the
assembly language level.

Both Tomayko and Autonomous also mention the FDS computer's P periods
, with Tomayko providing a little more detail. With 2.5-ms time
slots, the execution constraints were even more stringent than those
on the AACS. There is no indication of whether or not the FDS had
multiple levels of intervals like the AACS. And apparently the CCS
software was also built on a cyclic executive?

    Like the command computer, the data computer has a simple
    executive. Time is divided into twenty-four 2.5-millisecond
    intervals, called "P periods." Each 24 P periods represent one
    imaging system scan line. Eight hundred of those lines is a
    frame. At the beginning of each P period, the software
    automatically returns to memory location 0000, where it executes
    a routine that determines what functions to perform during that P
    period. Care is taken that the software completes all pending
    processes in the 2.5-millisecond period, a job made easier by the
    standardization of execution times once the direct memory access
    cycle was added.

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Six, Section 2, Box 6-3, p. 185.

Unrelated to performance concerns, the use of an executive proves the
truth in Richard Rice's wry quip:

    Rice characterized the unique nature of the data computer
    software this way: "We didn't worry about top-down or structured;
    we just defined functions."

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Six, Section 2, pp. 183-184.

The executive provided the top-level structure, so the programmers
really were just writing functions to be called when their time slot
came up!

                      Voyager Team Celebrating

The Voyager team celebrates the successful resumption of intelligible
 data transmissions from Voyager 1! In November 2023, the spacecraft
 began sending jumbled data. The problem was eventually traced to a
failed section of FDS memory. The flight software team relocated the
 code in the failed section elsewhere and were relieved and excited
  when Voyager 1 resumed sending good data on April 20, 2024. I can
 identify three people from the It's Quieter in the Twilight movie:
Todd Barber, Suzanne Dodd, and Sun Matsumoto. The man with both arms
 upraised might be Enrique Medina and the woman to his left, at the
 table, might be Lu Yang -- I'm not sure. (Source: "NASA's Voyager 1
  Resumes Sending Engineering Updates to Earth", April 2024, JPL.)

---------------------------------------------------------------------
                                                                     
                                       * Shoestring Space Exploration
                                           * Sequences and Simulators

Simulators & Other Random Thoughts

    I'm sure you'd just love the reputation of being the guy who lost
    2 deep space probes that had gone the furthest of any manmade
    object and had been doing just fine for decades. Through your
    bright idea for how Things Could Be Done Better.

    --Mike Pellatt, comment on The Register forum, October 31, 2015,
    for "Think Fortran, assembly language programming is boring and
    useless? Tell that to the NASA Voyager team" (based on John
    Wenz's Popular Mechanics article).

In the comments on an Ars Technica article, Humanity's Most Distant
Space Probe Jeopardized by Computer Glitch" (February 6, 2024), there
was an interesting subthread about the lack of simulators for the
flight computers. (Actually just for the AACS and FDS computers?
Matsumoto mentioned the COMSIM/HSSIM simulator for command "sequences
and CCS FSW changes".) I wanted to address some of the issues raised
in this section, but then I decided to get into some of the more
general issues; hence the "& Other Random Thoughts" in the title.

In every online discussion of anything touching on the Voyager
computers, there are always a number of questions such as "Why didn't
they do this?" and "Why don't they do that?" The answer to the first
question is more than likely, "Well, you had to have been there." The
second question is ably answered by Mike Pellatt above.

                 ----------------------------------

Shoestring Space Exploration

The Voyager spacecraft had more than fulfilled their nominal missions
after Voyager 2 flew by Neptune, so they've been living on borrowed
time ever since. However, NASA had long been keeping a close eye on
budget concerns after being raked over the coals by Congress in 1974
for Viking cost overruns. (See Viking project hearings in the
bibliography.)

Suzanne Dodd spoke of a bathtub effect in personnel levels between
the Saturn and Uranus flybys:

    DODD: Now, Voyager went through Jupiter and Saturn, and then it
    had a bathtub, because it had five years to get out to Uranus. So
    a lot of people left. Because Jupiter and Saturn were like 18--
    maybe two years apart, 18 months to two years apart in
    encounters. Then there was this lull, so they had to cut-- a lot
    of people left, and/or they just needed to trim staffing. A
    bathtub in staffing, because there was such a long duration.

    David Zierler, "Suzy Dodd (BS '84), Engineer and Deep Space
    Pioneer" Caltech Heritage Project, June 9, 2023.

(McLaughlin and Wolff also refer to the bathtub effect and describe
how JPL tried to reduce the effect between Uranus and Neptune by
spreading the expected work out more evenly in the interval.)

The "bathtub" after Neptune would be endless and the Voyager team was
very small in the succeeding decades. For those of us interested in
the software side of things, keep in mind that programming would only
have been one of the areas of engineering expertise needed on the
team. And the 10-12 full-time employee equivalency means that some,
if not many, of the team were and are working only part-time on
Voyager. As knowledgeable people retired, the remaining members by
necessity assumed multiple roles.

The Voyager program was almost cut off completely in the 2000s:

    NASA officials said the possibility of cutting Voyager and
    several other long-running missions in the Earth-Sun Exploration
    Division arose in February, when the Bush administration proposed
    slashing the division's 2006 budget by nearly one-third -- from
    $75 million to $53 million.

    --Guy Gugliotta, "Historic Voyager Mission May Lose Its Funding",
    The Washington Post, April 4, 2005.

(Voyager's share of the $75 million was $4.2 million.) Thankfully,
this didn't happen, but program leaders like Dodd (who returned to
Voyager in 2010) have probably had to fight for funding every year.

Between the lack of personnel, the part-time nature of the work, and
the sword-of-Damocles-like budget cut-off hanging over the project,
the time and money resources available for writing new simulators,
rewriting the flight software in C++ or Python, etc. were scarce. And
both would run up against an insurmountable obstacle: "If it ain't
broke, don't fix it!" We've got a very tenuous connection with two
spacecraft, one 15 billion miles away and the other 12 billion, and
you want to do what?! Paging Mike Pellatt ...

                                * * *

Finally, a few more words about spacecraft operations. The
constraints on flight operations range from the mundane to
out-of-this-world. Examples of the mundane are staffing levels.
Adamski (p. 4) notes that during the long, hopefully uneventful,
cruise intervals between planets, operations were limited to daytime
hours since the control center was not manned around the clock; this
was a contraint taken into consideration in the design of a sequence.

Skipping in-this-world concerns such as earthquakes (see Kohlhase, p.
120) and going straight to out-of-this-world constraints, everyone
seems to know that the Voyager spacecraft are very far away and that
it takes about 40 hours to send a command and receive a response
back. (This is called the Round Trip Light Time, or RTLT.) However, I
sometimes wonder if people also understand that the Voyager team is
not in constant contact with the two spacecraft. You can't just make
a change to the flight software, recompile it, and uplink it to the
spacecraft.

I use 40 hours as a nice round number for RTLT. More specific values
in 2024 are about 45 hours for Voyager 1 and about 38 hours for
Voyager 2. If you need even more accuracy, the real-time Voyager
Mission Status page has the hours, minutes, and seconds of the
one-way light times. As I noted in the "Introduction", the distance
of each spacecraft goes up and down depending on whether the Earth in
its orbit is falling behind or catching up with the spacecraft. As
the distance changes, so does the transmission time.

Communications with the Voyager spacecraft are handled by NASA's Deep
Space Network (DSN), which consists of three tracking stations spaced
about 120deg of longitude apart in Goldstone (California), Madrid
(Spain), and Canberra (Australia). Each station has a 70-meter dish
antenna, multiple 34-meter antennas, and others. According to
Matsumoto's predictions (p. 6), by now (2024), a 70-m antenna is
required to send commands to either Voyager spacecraft and a 70-m
antenna or two 34-m antennas are required to receive engineering and
science data.

Aside: A 70-meter dish antenna is 230 feet in diameter, 3/4 of the
length of an American football field, as you can see in this fanciful
picture of the Goldstone 70-m antenna placed in Pasadena's Rose Bowl
stadium! (Pasadena is also home to JPL. Source: The Voyager Neptune
Travel Guide, p. 28.)

                     70-m antenna in Rose Bowl!

The Voyagers are not the only spacecraft using the DSN, so the
Voyager team must request and compete for transmitting and receiving
time on the DSN. (At DSN Now, you can see in real-time which antennas
are talking to which spacecraft.) When scheduling time, the team must
take into account the 40-hour round trip light time. Depending on a
variety of factors, it is possible that they might have to settle for
time slots more than 40 hours apart, in which case the team must
ensure they can still verify the results of an operation. Matsumoto
gives an example:

    Command Loss (CMDLOS) timer, the timer that triggers CMDLOS FPA
    entry when it reaches to the pre-set value, is set for six weeks.
    It is still desirable to send the command to reset the timer
    every week but it is a best effort approach based on the DSN
    resources, usually without downlink coverage for command receipt
    verification. (The receipt of a command is verified indirectly by
    the CCS hourly status data when the telemetry is available.)

    Sun Kang Matsumoto, "Voyager Interstellar Mission: Challenges of
    Flying a Very Old Spacecraft on a Very Long Mission" (PDF) p. 5,
    2016 SpaceOps Conference, Daejeon, Korea.

After passing Saturn, Voyager 1 angled north of the ecliptic
(planetary) plane as it headed out into space. After passing Neptune,
Voyager 2 angled south. Consequently, communications with Voyager 2
are only possible via the Canberra DSN station. This became a problem
in 2020 when Canberra's 70-m dish was taken offline for 11 months for
upgrades. Voyager 2 continued transmitting data to the smaller dish
antennas, but commands could not be sent to the spacecraft. Contact
was successfully reestablished in early 2021 when the 70-m dish came
back online. (The prior preparation and the actual downtime are
featured in the 2022 film, It's Quieter in the Twilight, about the
Voyager team.)

The DSN stations send commands to the Voyagers at a radio frequence
of about 2.1 gigahertz. Because of the Earth's rotation (as well as
the revolution of the Earth around the Sun and other factors), a dish
antenna as a whole moves around in space. As a result, the relative
motions of the antenna and the spacecraft (which is also moving)
cause a continuously varying Doppler shift in the radio signal's
frequency as seen by the spacecraft. (Kobele, p. 13, says that the
Doppler shift can be up to 3 kilohertz during a DSN pass.) The
Voyager's receivers were designed with this in mind and use a
phase-lock loop (PLL) circuit to detect, lock onto, and maintain the
lock on a varying frequency signal within a 100-kilohertz range.

In 1978, less than a year after launch, ground control forgot to
reset Voyager 2's Command Loss timer and the aforementioned CMDLOS
function was tested for real when the timer expired. The CCS computer
thought the primary receiver was no longer working, so it switched to
the backup receiver. Unfortunately, the backup receiver's PLL circuit
had failed partially prior to this switchover and the receiver could
not lock onto the command signal. After 12 hours, the CMDLOS function
switched back to the primary receiver and contact was reestablished.
"Phew!", as Wordle likes to say. A half-hour later, the primary
receiver failed completely. Back again, a week later, to the backup
receiver that couldn't receive. (The CMDLOS timeout value was 1 week
for the primary receiver and 12 hours for the backup receiver.)

(Note that human error triggered these two problems, it didn't cause
them. The partial failure of the backup receiver's PLL circuit had
already happened and the complete failure of the primary receiver was
just waiting to happen. The project was actually fortunate that this
occurred early on and not later, for example, in the middle of flying
by Jupiter.)

I don't know how they did it, but somehow the engineers figured out
that the backup's PLL circuit was limited to a single, nominal
frequency with a tolerance of only 100 Hz -- that's 100
cycles-per-second in a 2.1 billion cycles-per-second radio frequency.
Wow! Since Voyager 2's receiver could no longer compensate for the
Doppler shift (up to 3 kHz during a pass), the DSN transmitter had to
compensate by sending a varying frequency signal that canceled out
the pre-calculated, expected Doppler shifts, thus delivering a
constant-frequency signal to the spacecraft.

Note: The 100 kHz and 100 Hz figures are from Ludwig and Taylor, p.
39-40; other sources have slightly different values.

Would that things were so simple! The frequency at which the backup
receiver is listening for commands is called the best-lock frequency
(BLF). The BLF is extremely sensitive to the temperature of the
receiver and small changes in temperature cause shifts in BLF
exceeding the 100-Hz range. Many activities on the spacecraft cause
these temperature changes. To cope with this added difficulty, the
ground system must (1) limit uplinks to no-activity periods when the
receiver's temperature is stable and (2) regularly recalibrate the
BLF. The former is accomplished by following a period of activity
with an uplink-free, 24- to 72-hour command moratorium that gives the
temperature time to stabilize. The latter is done with a scatter-shot
approach, sending something (probably a test command of some kind) at
multiple frequencies and seeing which one actually gets through. (
McLaughlin and Wolff, p. 5, describe using the scatter-shot approach
to uplink emergency or time-sensitive commands during active periods:
send the same command at multiple frequencies in hopes that one will
get through. This technique may have been used in planetary fly-bys
to upload onboard sequence updates.)

Amazingly, all this was made to work and Voyager 2 successfully flew
by Jupiter, Saturn, Uranus, and Neptune and returned valuable images
and science data under these operational constraints. And continues
to do so. (The 2020 MAG-roll-gone-awry in It's Quieter in the
Twilight was probably followed by a command moratorium.)

Addendum: Various sources attribute the crippling of the backup
receiver to the failure of a tracking-loop capacitor in the PLL.
Kobele, 1989, p. 14, was more specific: "a resistive short in the
tracking loop capacitor due to particle migration through the
dielectric material". This necessitated a periodic maintenance
operation to prevent complete failure: "It has been demonstrated that
particles which have not yet completed their migration can be
destroyed when the capacitor charge is maximized by periodically
pulling the uplink signal to the extremes of the loop bandwidth." (I
have not seen this mentioned elsewhere.)

                 ----------------------------------

Sequences and Simulators

Regarding simulators, you'll find comments such as, "Give me the CPU
details and I'll whip up a simulator over the weekend." These
comments and the earlier questions are not bad and I admit to often
having similar thoughts myself. However, as software developers, we
of all people should have -- but don't have -- an instinctual reaction
that nothing is ever as simple as it first seems.

In this case, simulating the CPU may be the least complicated part of
a flight computer simulator, itself part of a larger system. It turns
out that, much to my amazement, the engineers back in the 1970s --
without JavaScript and Rust, without gigabytes of RAM, and without
55-MB web pages -- somehow managed to write sophisticated and complex
programs!

The following discussion is largely based on the papers by McEvoy
(1975) and Adamski (1987) about the Viking and Voyager simulators,
respectively. The descriptions here are my understanding of the
systems, which may likely be imperfect. The intended takeaway is that
the systems were complex and I think that will be made clear in spite
of any errors. (Linick and Weld's 1992 paper examines the
streamlining of the prior sequencing process, as described in
Adamski, for the Voyager Interstellar Mission, an effort that
successfully met two goals, among others, of moving off the Univac
mainframes and operating with greatly reduced personnel in the long
term. The paper provides a somewhat managerial evaluation of the
changes and doesn't provide the new hardware and software details I
would have liked.)

Note something that I completely missed in my initial readings. The
Viking Orbiter had only one computer (and its backup), the CCS. Its
Attitude Control Subsystem (ACS) and its Flight Data Subsystem (FDS)
did not have their own computers. The Viking FDS did have a
1K-by-8-bit plated-wire memory:

    The FDS memories were required to perform a number of essential
    tasks and were additionally used to accomplish a variety of FDS
    housekeeping functions, including reading engineering identifiers
    for the flexible formats; buffering MAWDS, VIS, and IRTMS (A/PW)
    science data; storing and updating VO time; storing PN sequences
    for the VIS and science formats; executing memory-alteration
    commands from CCS; performing most of the science format
    multiplexing; and counting TV pictures taken and controlling
    related FDS logic.

    --Neil A. Holmberg, Robert P. Faust, and H. Milton Holt, Viking
    '75 Spacecraft Design and Test Summary, Volume II: Orbiter Design
    (NASA Reference Publication 1027), p. 61, November 1980.

Because of signal transmission delays, the Viking and Voyager
spacecraft were designed to function autonomously, a form of
operation that depends heavily on stored command sequences as opposed
to single, separate, "real-time" commands. Consequently, simulation
is especially geared towards the testing of command sequences.

In the introduction, I defined a command sequence as a sequence of
commands, but they are much more than that. A sequence is composed of
blocks, each of which consists of:

  * Events - are actions (?) that may or may not be associated with
    spacecraft commands. In McEvoy's sample Viking Orbiter block
    (Figure 9, p. 414), events with CCS spacecraft commands are
    actions targeted to the external subsystems such as the ACS and
    FDS. In one entry, the CCS instructs the ACS: "Inertial mode;
    stop roll turn; stop yaw turn; negative turn direction". Events
    without CCS commands are actions taken internally by the CCS, as
    in this CCS event: "Disable the ACS accelerometer calibration
    within the CCS".

  * Data - any data required by the block; e.g., tables.

  * Flight software - is the non-core software needed to support the
    commands/events in the block.

In developing a sequence, engineers such as Suzanne Dodd could draw
on previously defined blocks in the block dictionary for subtasks,
much like programmers use functions from libraries. Blocks used in
such a manner are customized via block options and pull in their own
required flight software. A Viking block's flight software was stored
as OSTRAN macros (McEvoy, p. 415); OSTRAN was the Viking counterpart
to Voyager's SEQTRAN. The output of OSTRAN is assembly source code
that is passed on to an assembler and linker, ultimately resulting in
the memory load (containing sequence, data, and flight software) that
will be uplinked to the spacecraft. It appears that the flight
software resident in memory consists of (i) the core software that is
always present (executive, sequence processing, fault protection,
etc.) and (ii) only the non-core software needed to support the
currently loaded sequences.

Throughout the development and simulation of sequences, there are
automated and human checks for physical and operational constraint
violations. When a violation is detected, engineers determine how
serious the violation is and, if necessary, go back to the drawing
board to remedy the problem. The engineers also need to verify that a
sequence actually does what it is supposed to do; e.g., take a
picture of Neptune. Consequently, the design-develop-test cycle for
sequences is an iterative process.

A block diagram for the Voyager simulator (COMSIM) would look similar
to that of the Viking Orbiter simulator (OCOMSM) below. (I think -- I
haven't been able to find detailed information about Voyager's
simulator. Incidentally, the Viking Lander simulator was called
LCOMSM.) Note that the nested CCS and FDS simulators are subsystem
simulators. I'm puzzled by the absence of an ACS simulator. The
Viking Data Storage Subsystem, DSS, handled the Digital Tape Recorder
(DTR), a function taken over by Voyager's FDS.

                      Viking Orbiter Simulator

  Viking Orbiter Simulator Functional Block Diagram (click image to
                enlarge) (McEvoy, Figure 11, p. 417)

The simulator outputs on the right side of the diagram are very
important and provide:

  * Visibility into the internal sequence processing to the sequence
    and flight software engineers.

  * A step-by-step timeline for the flight controllers to follow
    (after the fact because of the time delay!) as the sequences are
    actually executed on the spacecraft.

The step-by-step timeline is called the Sequence of Events (SOE) on
Voyager:

    In addition to the blocks of load commands, the sequence
    generation process provides other products that are used in the
    command and control of the spacecraft. A Sequence of Events (SOE)
    is produced that is a printed, time-ordered listing of the
    activities contained in the CCS sequence. Performance predictions
    are generated for power usage, scan platform pointing, and other
    spacecraft functions. Additional uplink commands may also be
    generated along with the main sequence. These commands are held
    for subsequent transmission at pre-identified points in the
    sequence in order to perform discrete functions or enable
    checkpointed events.

    Terrence P. Adamski, "Command and Control of the Voyager
    Spacecraft" (abstract) p. 5, American Institute of Aeronautics
    and Astronautics (AIAA), 25th AIAA Aerospace Sciences Meeting,
    March 24-26, 1987, Reno, Nevada.

A spacecraft is more than just its flight software and simulators
must also simulate the hardware. McEvoy suggested APL for both
describing and simulating hardware, although he didn't come right out
and say they used APL. I wasn't aware of APL's role in the hardware
realm, but it's a real thing. (I do question the widely asserted
notion that APL was originally a hardware description language. That
doesn't seem to agree with Kenneth Iverson's contemporary writings
and later recountings of APL's history, but it is a Fortran 5-like
rabbit hole that I'll leave to someone else to explore!)

    One difficulty in modeling digital hardware is the different
    languages used in the hardware and software disciplines. Hardware
    description languages that describe hardware at the logic-gate
    bit-time register-transfer level provide a commonly
    understandable detailed definition that can bridge the gap. The
    APL language itself provides an excellent basis for describing
    digital hardware. Power-on-reset states and hardware
    idiosyncracies, that could affect a sequence, must be considered
    in the modeling stage. Any of these that could affect operation
    of the hardware, software, timing, or other subsystems should be
    modeled.

    Maurice B. McEvoy, "Viking Orbiter Uplink Command Generation and
    Validation via Simulation" (PDF) p. 417, The Institute for
    Operations Research and the Management Sciences (INFORMS) Winter
    Simulation Conference 1975 Conference Proceedings.

Voyager's CCS subsystem simulator would also have looked similar to
Viking's:

                        Viking CCS Simulator

Viking Orbiter CCS Simulator (click image to enlarge) (McEvoy, Figure
                             12, p. 418)

Each of the CCS, AACS, and FDS subsystem simulators would have had to
simulate:

  * the two CPUs,
  * the internal elements of the subsystem (e.g., memory banks),
  * interfaces with elements external to the subsystem (including
    interfaces with the other computer subsystems),
  * cross-strapping (switching) between primary/redundant elements
    both internal and external,
  * and the timing of everything.

The two CCS CPUs could also work independently of each other and not
in a strict primary/backup relationship. Despite my listing of the
simulation "requirements" above, there is a lot I don't know about
the simulators:

  * Were CPUs simulated in software or were actual hardware spares
    used?

  * For CPUs simulated in software, was the simulation at the
    instruction-set level or was it a simulation of the CPUs' actual
    electronic circuits (e.g., using APL)?

  * And similar questions about interfaces with the various non-CPU
    elements: Was an interface simulated at a logical level or as an
    electronic circuit? Or was a real interface with a hardware spare
    used?

  * What elements could be cross-strapped? And was cross-strapping
    simulated?

In other words, my ideas about the simulators are largely guesswork.
McEvoy does provide a few more details about the CCS simulator,
although some of my questions are still unanswered:

    The Computer Command Subsystem (CCS) simulator (Fig. 12)
    accurately models the Block redundant CCS processors, output
    units, and memories at the instruction and register transfer
    level. The accuracy with which software programs (in the
    memories) are simulated is dependent primarily on the accuracy
    with which the bit-time clocks can be calibrated during flight
    and timing accuracies of the 32 priority interrupts and 24 level
    indicators. Output simulation consisting of discrete commands
    (relay closures), coded commands, and telemetry data can be
    routed through either or both output units. A hardware and
    software self-test must be satisfied before access can be
    obtained to an output unit. (p. 418)

    ...

    The large amount of time used to simulate uplink command decoding
    is due primarily to the CCS hardware and software self-test
    activity, which in turn is affected by the FDS engineering
    telemetry rate and format. (p. 419)

    Maurice B. McEvoy, "Viking Orbiter Uplink Command Generation and
    Validation via Simulation" (PDF) The Institute for Operations
    Research and the Management Sciences (INFORMS) Winter Simulation
    Conference 1975 Conference Proceedings.

(I don't understand the purpose of the "hardware and software
self-test" in the last sentence of the first paragraph. Was this done
on the actual spacecraft? The later sentence seems to indicate "Yes".
I can understand self-tests performed repeatedly in the background or
prior to a set of operations, but I am perhaps overestimating the
frequency of the self-tests.)

So, spacecraft simulators are complex systems. What did the Voyager
simulator look like?


    The validation end of the software development process was
    handled by the Capability Demonstration Laboratory (CDL).
    Completed after the initial software was produced, it was a
    collection of either breadboard or flight surplus computer and
    science hardware, and its interfaces interconnected in the same
    way as those on the actual spacecraft. Its function is identical
    to that of the Shuttle Avionics Integration Laboratory (SAIL), in
    which both software and hardware changes could be tested to see
    if they functioned successfully.

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Six, Section 2, p. 175.

And the CDL lapsed into disrepair:

    In addition, the [Voyager flight team] also has to work around a
    lack of a hardware test bed, the limited memory of on-board
    computers, and antiquated programming languages.

    The Capability Demonstration Lab (CDL), the testbed used during
    the prime mission, could not be maintained and had to be
    abandoned at the start of VIM. The failures of the testbed were
    too often, even in pre-VIM, due to aging hardware and
    disappearing repair expertise. The project had to move to a new
    location in early VIM and the CDL did not survive the move. There
    are no simulators for the AACS or FDS and only the CCS has a
    simulator, i.e., HSSIM. As a result, any FSW changes other than
    something very simple have to be done in the CCS.

    Sun Kang Matsumoto, "Voyager Interstellar Mission: Challenges of
    Flying a Very Old Spacecraft on a Very Long Mission" (PDF) p. 6,
    2016 SpaceOps Conference, Daejeon, Korea.

The last sentence above suggests to me that a non-trivial change to
the AACS or FDS flight software is considered risky and that, if
possible, it is better to leave the existing AACS/FDS software as-is
and have the CCS take over responsibility for the changed
functionality. That's my impression at least; given the lack of
rigorous simulation capabilities, I think it's a sensible decision.

Oddly, searching on Google for JPL's "Capability Demonstration Lab",
mentioned by both Tomayko and Matsumoto, only brings up a couple of
references to Matsumoto's paper; on Bing, you get a single result ...
for JPL's Cyber Defense Laboratory! By a strange coincidence, a
couple of hours after reading the previous sentence, I was reading
McLauglin's & Wolff's paper about the preparations for Voyager 2's
Uranus fly-by and I came across the most detailed description of the
CDL yet. Which is not saying much, but the passage below does give a
good picture of a flight software engineer's life!

The physical actions and movements in the Digital Tape Recorder (DTR)
produce "equal and opposite reactions" in the spacecraft and thus
affect the spacecraft's attitude. After the Saturn encounter, a
routine, DSSCAN, was added to simultaneously fire attitude control
thrusters when performing high-rate tape operations, thereby greatly
reducing the time it takes for the spacecraft attitude's
rate-of-change to slow enough to begin the next science operation. (A
savings of over 3 minutes in the case of the end-of-track turnaround,
valuable time during the all too brief hours of closest approach.)
That's my understanding; it's not clear which computer DSSCAN was
added to, AACS or CCS.

Note how Voyager 1, its mission already completed, was used as a test
bed for the Voyager 2 FSW modifications. Note also that the AACS and
CCS programs were not reassembled, but instead patched, a practice
that had been ongoing since launch. What a nightmare the onboard
software must have become! I don't know how long the practice
continued, but it makes the VIM team's reluctance to modify the AACS
software, per Matsumoto, even more understandable.

    The DTR is mounted on the spacecraft such that its angular
    momentum is introduced into the yaw and pitch axes of the
    spacecraft with almost none going into the roll axis. DSSCAN was
    first programmed to introduce cancelling momentum in the yaw axis
    only. The modification to the AACS and CCS software took place in
    an environment of a scarcity of available memory so that, from a
    programming point of view, it had to be carefully fit in. The
    "patch" was carefully tested in the Voyager Capability
    Demonstration Laboratory (CDL) before loading onboard Voyager 1.
    (The AACS and CCS programs were modified without being
    reassembled as is the case with all AACS and CCS changes since
    launch.) The CDL is a digital/analog simulation of many of the
    spacecraft capabilities. Modifications or tests of any degree of
    complexity are done first, whenever possible, on Voyager 1 before
    implementation on Voyager 2, a reflection of the fact that
    Voyager 2 still has two planetary encounters scheduled while
    Voyager 1 has none.

    Once onboard the spacecraft, DSSCAN was tested during periods of
    DTR operation to determine empirically how many thruster pulses
    should be used to counteract a DTR event. For example, tests on
    Voyager 1 in March of 1984 showed that five thruster pulses were
    the optimum number. These pulses each have an
    electronic-activation length of 10 ms, and a single pulse will
    change the drift rate about the spacecraft yaw axis by
    approximately 11 ur/s. See Fig. 12.

    Now that the effect of DSSCAN on yaw-axis momentum introduced by
    DTR events is known, the subroutine is being modified to
    incorporate momentum (near) cancellation in the pitch axis.
    Although the modification is, in principle, straightforward in
    view of the yaw axis experience, in practice this is not the
    case. The relative complexity arises from the tight programming
    that must be done because of the shortage of AACS and CCS words
    and existing programs that have already been heavily modified;
    with resultant complexity of code. Add to this the potentially
    serious consequences that could arise from any mistake in
    modifying the control law of the spacecraft, and one appreciates
    the fact that even seemingly simple changes can require extensive
    time for testing and experienced personnel for safe
    implementation. (pp. 11-12)

    ...

    During implementation of the sequence, certain critical or
    complex subsequences may be checked for validity by running them
    through a sophisticated analog/digital simulation of extensive
    portions of the spacecraft: the Voyager Capability Demonstration
    Laboratory (CDL). The CDL contains, for example, duplicate
    versions of CCS, AACS, FDS, scan platform actuators, and the DTR.
    (p. 17)

    --W.I. McLaughlin and D.M. Wolff, "Voyager Flight Engineering:
    Preparing for Uranus" (abstract), pp. 11-12, American Institute
    of Aeronautics and Astronautics (AIAA), 23rd Aerospace Sciences
    Meeting, January 1985, Reno, NV.

Aside: I'm not sure how to square the "patching-only" of the CCS code
(or if it even needs to be squared) with McEvoy's description of
assembly source code being pulled from the sequencing block
dictionary, code that would be assembled and placed in a sequence's
uplink memory load.

The High Speed Simulator (HSSIM) is a flexible spacecraft simulator
developed at JPL in the 1990s and written in C++. Spacecraft
components are represented by C++ objects and a component/object may
have one or more interfaces. Interfaces are strongly typed and
connections between components can only be made over compatible
interfaces. For example, you can connect a CPU's data bus to a memory
bank's data bus, but not to the memory bank's address bus. The link,
at the C++ level, between two interfaces is called a splice. Each
component has its own Tcl interpreter that allows scriptable behavior
in addition to other functions:

    C++ was chosen as the implementation language for simulation
    components, because of its run time efficiency and its support
    for object-oriented programming. A processor component, for
    example, may emulate a hardware instruction set and the code
    which implements this must be sufficiently fast to meet the
    performance objectives of the simulator.

    Most components contain a[n] embedded interpreter which
    recognizes the language Tcl. Tcl is a freely available embeddable
    language and concrete interpreter implementation developed at the
    University of California, Berkeley. Tcl was augmented to make it
    multi-thread ("MT") safe and to add additional features, in
    particular, a C++ class wrapper and a fast "remote procedure
    call" that can execute commands remotely on named interpreters
    (many of the components within HSS contain named interpreters,
    which are used to manipulate the component) and return the result
    of the execution to the local interpreter. The HSS Tcl subsystem
    is used for many things, including model state examination and
    alteration, establishment of model interconnections, and model
    state monitoring and display (using Tk, a toolkit based on Tcl
    which can quickly create Graphical User Interfaces). As a general
    rule, Tcl is used unless other performance or robustness concerns
    dictate the use of splices.

    --K. Patel, W. Reinholtz, and W. Robison, "High Speed Simulator --
    A Simulator for All Seasons" (PDF), p. 751, SpaceOps 1996,
    Proceedings of the Fourth International Symposium, Munich,
    Germany.

HSSIM supports multiple implementations of a given spacecraft
component/object, with the choice of which implementation to use
being made at the time of a simulator run. Depending on their need,
an engineer can choose between, for example, a slower, bit-level
(logic-gate) simulation of a component or a faster, functional-level
simulation. HSSIM was successfully used in various roles for the
Jupiter orbiter, Galileo, beginning some time after the spacecraft's
high-gain antenna failed to fully deploy in 1991. According to the
paper, versions for the Saturn orbiter, Cassini (to be launched in
1997), and Voyager were to be delivered "later this year" (1996?).

As a software developer, I admire and respect the design of HSSIM.
And my knowledge of the capabilities and implementation of the
earlier COMSIM and CDL is almost entirely inferred from McEvoy's
description of the Viking simulator; i.e., I could be way off-base.
Keeping this in mind, I suspect that the HSSIM delivered to and used
on Voyager is a bare-bones CCS simulator. Building a software-based
simulator for a specific spacecraft is a labor-intensive task, with
or without HSSIM. You need people knowledgeable about the spacecraft
and buy-in from a management willing to invest in the effort. The
nearly 20-year-old Voyager spacecraft had already fulfilled their
primary missions and the program lacked some key requirements for
building a simulator:

  * Many people with intimate knowledge of the spacecraft hardware
    and flight software had moved on or retired.
  * JPL as an organization was probably losing interest in the
    program as newer -- much newer -- programs vied for attention.
  * Machine-readable documentation (e.g., hardware specifications)
    was probably non-existent.
  * HSSIM developers themselves were probably none too eager to work
    on technology from the ancient past!

Knowing C++ and Tcl would have been helpful. The HSSIM paper did
mention automatic code generation using an expert system and, under
"Future Direction", proposed other means of automatically building
parts of the simulator. Those would lessen the need for C++/Tcl
software expertise, but there's still the laborious task of
assembling and manually entering all the information needed as inputs
to the expert system and to the other proposed mechanisms.

So, no AACS or FDS simulators. Only a bare-bones CCS simulator.
(Again, that is my impression from my readings and I hope I am
wrong.)

---------------------------------------------------------------------

AACS, HYPACE, and HYBIC

NOTE that the AACS computer is not the at-the-time experimental
HYPACE (Hybrid Programmable Attitude Control Electronics) computer.
For budgetary reasons, Voyager instead used a CCS computer running at
a higher clock rate and with an added index register found useful on
HYPACE. Dr. Tomayko wrote:

    Documentation for the system still refers to the attitude control
    computer as HYPACE, even though its heart was the [CCS] command
    computer.

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Six, Section 2, p. 178.

(General Electric made both the CCS and HYPACE computers.) With
respect to HYPACE, Dr. Tomayko wrote:

    Index registering meant that the same block of code could be used
    for all three axes, reducing memory requirements.

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Six, Section 2, p. 177.

This sounds like the typical use of an index register: add the value
in the index register to addresses when accessing memory. By changing
the value in the index register, a program can switch between each
axis's set of variables prior to calling a common subroutine.

I suspected a simple way of implementing an index register so as to
avoid changing the CPU's instruction set would have been to place a
memory-addressable chip or electronic circuit between the CPU and
memory to hold the index value. The following description of the AACS
computer found elsewhere gives a name to a circuit that may or may
not be such an implementation:

    The Attitude and Articulation Control Subsystem used an augmented
    version of the CCS computer that inserted a unit (the Hybrid
    Buffer Interface Circuit (HYBIC)) between the CPU and RAM, which
    intercepted instructions to add indexed addressing capability (at
    the expense of other instructions), and accelerated instructions
    that used idle cycles.

    --John Culver, "The CPUs of Spacecraft Computers in Space", The
    CPU Shack.

Unlike other (incorrect) descriptions, Culver's doesn't conflate the
AACS with HYPACE, so I think he knows what he's talking about. His
description does seem to indicate that changes were made to the CCS
CPU and its instruction set for the AACS. Strangely, Tomayko's report
doesn't mention HYBIC at all.

In her 2016 paper, Matsumoto describes a problem in a decidedly more
complex HYBIC:

    The Hybrid Buffer Interface Circuits (HYBIC) on V1 was switched
    to the backup unit in 2002 due to a failing component inside the
    HYBIC Analog to Digital (A/D) converter. The star tracker on the
    original HYBIC was performing well at the time of the switch but
    each HYBIC has a unit (A/D converter, sun sensor, and star
    tracker) dedicated to the HYBIC. The star tracker on the current
    HYBIC is degrading rather rapidly and closely monitored; it is
    quite likely that a HYBIC switch back to the original is needed
    before the end of the mission

    Sun Kang Matsumoto, "Voyager Interstellar Mission: Challenges of
    Flying a Very Old Spacecraft on a Very Long Mission" (PDF) p. 3,
    2016 SpaceOps Conference, Daejeon, Korea.

Okay, HYBIC is a lot more than just an index register or registers.
(See the picture of the HYBIC Logic Subassembly at the Smithsonian.)
A 2006 JPL presentation about the Voyager 2 HYBIC problems included a
functional block diagram of the AACS (click image to enlarge):

                         AACS Block Diagram

The diagram doesn't show an interface between the HYBIC and its
associated memory bank. However, a 1981 U.S. Air Force study,
Assessment of Autonomous Options for the DSCS III Satellite System,
has a similar diagram (Figure B-3, p. 178) that adds a 6th arrow
between the HYBIC and RAM. And, interestingly, has a dashed box
labeled "HYPACE" around the upper portion of the diagram!

The role of HYBIC in the AACS is up in the air for me. Online
information about HYBIC is extremely scarce and what little I could
find is basically a rehash in one form or another of the information
above.

---------------------------------------------------------------------

Miscellaneous Information

                        Voyager Team Relaxing

  The Voyager team takes a break from worrying about bit flips and
   decreasing RTG power! Left to right: Lu Yang, Todd Barber, Sun
 Matsumoto, Enrique Medina, and Jefferson Hall. The picture is from
  the movie, It's Quieter in the Twilight, about 45-50 minutes in,
right after the camera panned away from Chris Jones and Roger Ludwig
on the right side of the table. (Source: JPL's internal newsletter, 
                      Universe, June 2023, PDF)


Ben-Hur Rides Again!

Suzanne Doddd: Early Voyager Years

Larry Zottarelli

Real Programmers Don't Use Pascal

Clash of Viking Mainframes

Ben-Hur Rides Again!

Eat your heart out, Lew Wallace, author of Ben-Hur! I don't recall
ever seeing the whole movie, but I did read the novel. My memory is
that the chariot scene was even more exciting in the book than in the
movie, but neither hold a candle to Voyager 2's launch troubles!

Based on Tomayko's telling (Box 6-1, p. 179-180, in Computers in
Spaceflight), my understanding is that the Voyager 2 spacecraft had
separated from the launch vehicle and was in the process of extending
the booms while at the same time trying to correct its whopperjawed
attitude. Things began to go wrong, but I'll let Rasmussen and Litty
tell the story:

    The first launch provided more excitement with little delay. An
    unbelievable combination of events shortly after injection
    initiated a chain reaction that ultimately triggered nearly every
    single fault routine in the AACS repertoire. Briefly, the
    sequence happended as follows.

    The spark to the fuse was a partially latched boom, resulting in
    high tip-off rates at propulsion module separation. Meanwhile,
    reconfiguration activities and a precautionary restoration of
    isolation value states had temporarily disabled attitude control.
    While thrusters were waiting to fire, the tip-off effects were
    accumulating, so that by the time the isovalve power requests
    reached the top of the queue, a large angular offset existed. By
    coincidence, thruster control was re-enabled just before sequence
    activation of thruster fault detection. This quickly responded to
    the large error by again disabling thruster firing in preparation
    for a switch to backup thrusters. More isovalve activity ensued
    causing further delays, three more repetitions of the error
    reponse, and finally a self-inflicted heartbeat failure, all
    within sixteen seconds of separation!

    This last action was pivotal in resolving the situation. The new
    system, with no memory of past difficulties, including the large
    error, quickly brought the rates under control. Finding the sun
    nowhere in sight, this system then proceeded automatically to
    search for and acquire celestial references, achieving success
    less than three hours after launch. Ground operations spent the
    next several days recovering.

    --R.D. Rasmussen and E.C. Litty, "A Voyager Attitude Control
    Perspective on Fault Tolerant Systems" (abstract), p. 246,
    American Institute of Aeronautics and Astronautics (AIAA)
    Guidance and Control Conference, August 19-21, 1981, Albuquerque,
    New Mexico.

That last exquisite sentence is priceless!

                 ----------------------------------

Suzanne Dodd: Early Voyager Years

A couple of additional paragraphs from the Caltech Heritage Project
article:

    DODD: It was quite a bit different. But, you come in, and it was
    a fairly routine thing. You would be working on a sequence, and
    it would take six weeks to develop that sequence, and then there
    was different steps, and different software programs, and
    different reviews. That process then repeats itself every six
    weeks. From a work standpoint, you're doing a fair amount of the
    same thing, although you sometimes had special events. You
    sometimes worked with a particular science team on a special
    calibration they wanted to do. What I do remember is the Voyager
    team, the sequence area had a lot of pranksters in it. They would
    like fill a person's drawer with water and put fish in there, and
    so when they came in the next day, the water would splash and
    they'd have these goldfish swimming around in their desk.
    [laughs] It's that kind of a team camaraderie. Most people would
    eat lunch together, too. You just really were cohesive as a team.

    ZIERLER: What areas did you have real responsibility, even in the
    early years?

    DODD: I hired in when four other people hired in. Now, Voyager
    went through Jupiter and Saturn, and then it had a bathtub,
    because it had five years to get out to Uranus. So a lot of
    people left. Because Jupiter and Saturn were like 18-- maybe two
    years apart, 18 months to two years apart in encounters. Then
    there was this lull, so they had to cut-- a lot of people left,
    and/or they just needed to trim staffing. A bathtub in staffing,
    because there was such a long duration. So, I was part of the
    crew that came back in, new group of crew that came back in
    during Uranus. There were four of us that hired in all at the
    same time, to do basically the same job, which was sequencing. It
    was basically a very junior job on Voyager, sequencing engineer,
    but a super exciting mission. Really gets you going for liking
    this type of work. Then, when it came to Neptune, I got to be the
    lead sequence engineer. So, of the four of us that were doing it,
    I got the closest approach sequence.

    David Zierler, "Suzy Dodd (BS '84), Engineer and Deep Space
    Pioneer" Caltech Heritage Project, June 9, 2023.

                 ----------------------------------

Larry Zottarelli

In an article about astrophysicists debating whether or not Voyager 1
actually had entered interstellar space, the following encapsulation
of a CNN report on Larry Zottarelli appeared:

    [Determining if Voyager 1 has really entered interstellar space]
    will also be just a little bit harder than it was last week,
    because the last original member of the Voyager team has retired.
    Larry Zottarelli, aged 80, left NASA's employ this week after 55
    years on the job. Zottarelli helped to develop Voyager's on-board
    computers and has worked on the mission since 1975. CNN reports
    that he was sent on his way with a handshake from actress
    Nichelle Nicholls, Star Trek's Lt. Uhura. NASA is reportedly
    seeking a replacement fluent in FORTRAN, Algol and assembly
    language for the Voyagers' 250 KHz General Electric 18-bit TTL
    CPUs, complete with single register accumulator and bit-serial
    access to 4096-word plated-wire RAM.

    --Simon Sharwood, "Has Voyager 1 escaped the Sun yet? Yes, but
    also no, say boffins", The Register, October 30, 2015.

Boffin is British slang for a scientist or engineer! And the original
CNN report has pictures, especially of Zottarelli with Nichelle
Nichols AKA Lt. Uhura! (The Register gets it wrong, however. The
picture with Nichols is from the Voyager 30th anniversary celebration
in 2007, not Zottarelli's retirement.)

The following is from an (unpublished?) article about the Voyager
team:

    The quiet voice, that's Larry Zottarelli, the programmer and
    "computer whisperer", understanding his machines like no one
    better does.

    --Joachim J. Kehr, Editor SpaceOps News of the Journal of Space
    Operations & Communicator; April 2016, "Voyager -- The Right Staff
    " (PDF), p. 4.

I wasn't fortunate enough, but I think I would have liked working
with him! (The Kehr article includes an interview with Suzanne Dodd.)

I don't quote it here, but this widely cited, 2017 article in the New
York Times leads with a picture of and story about Larry Zottarelli:
"The Loyal Engineers Steering NASA's Voyager Probes Across the
Universe", by Kim Tingley, New York Times, Aug. 3, 2017.

                 ----------------------------------

Real Programmers Don't Use Pascal

    Some of the most awesome Real Programmers work at the Jet
    Propulsion Laboratory in California. Many of them know the entire
    operating system of the Pioneer and Voyager spacecraft by heart.
    With a combination of large ground-based FORTRAN programs and
    small spacecraft-based assembly language programs, they can do
    incredible feats of navigation and improvisation, such as hitting
    10-kilometer wide windows at Saturn after six years in space, and
    repairing or bypassing damaged sensor platforms, radios, and
    batteries. Allegedly, one Real Programmer managed to tuck a
    pattern-matching program into a few hundred bytes of unused
    memory in a Voyager spacecraft that searched for, located, and
    photographed a new moon of Jupiter.

    --Ed Post, "Real Programmers Don't Use Pascal", Datamation, July
    1983, "READERS' FORUM", pp. 263-265. (HTML version; the full
    magazine issue is available as a large PDF.)

                 ----------------------------------

Clash of Viking Mainframes

This section is not related to Voyager. While searching for
information about the Voyager flight computers, I landed on Herb
Johnson's web page, "COSMAC 1802: history of microprocessors in space
", and his accompanying notes about the Viking flight computers. The
notes rely mostly on Tomayko's report and a Martin Marietta
Corporation report, Viking Software Data. (The links to the report in
Johnson's notes don't work anymore.)

Preceding the Voyager program by several years, the Viking mission to
Mars began in 1968. Two identical pairs of spacecraft were launched
in 1975 and arrived at Mars in 1976. Each pair consisted of (i) an
Orbiter which went into orbit around Mars and (ii) a Lander which
landed on Mars.

JPL was responsible for the Orbiter. Martin Marietta designed and
developed the Viking Lander and its flight software. They also
developed the ground-based, mission operations (mission control)
software for the Lander. Afterwards, Martin Marietta wrote an
evaluation of the software development process for the U.S. Air
Force, the document used by Herb Johnson above.

Viking Software Data turned out to be very interesting reading. After
the introductory material, the document consists of individual
evaluations of different techniques used in the development and
testing process. Each evaluation is structured as follows: name,
summary, application considerations, recommendation, history,
description, qualitative results, and quantitative impact.

I was motivated to write "Clash of Viking Mainframes" upon being
taken aback by the description of the second technique, "DIFFERENT
DEVELOPMENT/INTEGRATION SITES". Martin Marietta developed and tested
the mission operations software on Control Data Corporation (CDC)
mainframes at their Denver, Colorado facility (denoted MMC) and then
converted the source code to be compiled and tested again on the
target Univac and IBM mainframes at JPL's Pasadena, California Viking
mission control center! This is somewhat akin to developing a native
Windows application under Linux because you only had a Linux PC;
though possible, it would be unthinkable today. However, as you'll
see, there were some very good reasons for this approach -- some known
beforehand and some uncovered during the process -- and I can't fault
Martin Marietta or JPL for taking this approach.

NASA management feared JPL overextending itself, so it assigned the
Orbiter to JPL and the Lander to Martin Marietta. However, the Lander
integration and test and the actual operations would still take place
at JPL. So:

    Martin-Marietta Corporation, the Viking Lander contractors, had
    to do some dangerously unique software development when NASA
    decided to move control of the Lander from Langley to JPL. Since
    Orbiter software development and giving support to other missions
    tied up JPL's computers, Martin took the chance of developing the
    Lander software in a "minimal higher order language,"
    specifically a hopefully transportable subset of FORTRAN.
    Martin's solution reflected its recent migration to IBM 370
    series and Control Data 6500 series computers at its Denver
    plant. These were technologically more advanced than the JPL
    computers and could not be trusted to produce directly
    transportable software. The idea worked, but Martin admitted that
    the requirement for delivering mission support software 10 months
    before the flight provided strong motivation.

    James E. Tomayko, Computers in Spaceflight: The NASA Experience,
    1988 Chapter Eight, Section 3 (Unmanned mission control
    computers), p. 267.

And getting into the "DIFFERENT DEVELOPMENT/INTEGRATION SITES":

    APPLICATION CONSIDERATIONS: The forecasts for the loads on the
    computer sets at JPL indicated that the MMC portions of the
    Viking operational software system could not be developed on
    them. The MMC facility contained CDC 6400, CDC 6500 and IBM 360/
    75 computer sets, whereas the JPL facility contained UNIVAC 1108
    and IBM 360/75 computer sets. The operating systems of the IBM
    computers at the two facilities were different. No equipment
    similar to the JPL UNIVAC 1108 system was available in the Denver
    area. Pathfinder studies indicated software could be developed on
    non-target computers without creating any serious conversion
    problems. (p. 30)

    HISTORY: One of the problems which faced the management of the
    Viking mission operations lander software development was that
    the development computers differed from the operational
    computers. The Martin Marietta facility consisted of CDC 6400 and
    CDC 6500 and IBM 360/75 whereas the JPL computer facility
    consisted of IBM 360/75s and UNIVAC 1108s. Thus the software
    conversion task became a management concern very early in the
    software development activities. (p. 31)

    Martin Marietta Corporation, Viking Software Data: Final
    Technical Report (15MB PDF) May 1977, RADC-TR-77-168.

Keep in mind that lacking the target Univac mainframe could not be
remedied by running down to the local computer store, picking a
computer off the rack, and then shoving it in a spare closet with an
ethernet cord back at the office. A mainframe computer and its
peripherals were very expensive and you had to commit to sizeable
initial and long-term investments in real estate (new or existing
housing for the installation), utilities (power, etc.), supplies
(drums, disks, tapes, punch cards, etc., etc.), and especially all
the support personnel needed to manage and run the whole thing.

I should note that my only experience with mainframes was with a
Univac 1100-series mainframe in the University of Maryland's Computer
Science department in the late 1970s. (They had two 1100s, but I only
ever used one of them.) Supporting hundreds if not thousands of
students, faculty, and staff meant that the department's mainframe
installation fit my perhaps overbroad characterization above, but I
realize that smaller groups or companies could have much leaner
installations.

The CDC 6500 had two 6400 CPUs; the separate 6400 computer was
apparently not used for Viking development. I don't know why the
report says here that Denver had an IBM 360/75 when elsewhere it
speaks of a 370, specifically a 370/155 on p. 260. The computers from
the three companies had widely differing CPU architectures:

+-------------------------------------------------------------------+
|  Computer  |Words |   Integers   |  Floats  |     Characters      |
|------------+------+--------------+----------+---------------------|
|IBM 360 &   |32    |Two's         |32/64 bits|8-bit EBCDIC         |
|370         |bits  |complement    |          |                     |
|------------+------+--------------+----------+---------------------|
|UNIVAC 1108 |36    |One's         |36/72 bits|6-bit FIELDATA       |
|            |bits  |complement    |          |                     |
|------------+------+--------------+----------+---------------------|
|CDC 6500    |60    |One's         |60/108    |6-bit CDC display    |
|            |bits  |complement    |bits      |codes                |
+-------------------------------------------------------------------+

Pathfinder studies showed the feasibility of developing the software
on the CDC mainframes and converting it for the IBM and Univac
mainframes, as well as providing guidance and recommendations on
practices and techniques. I think two particular practices
contributed to the ultimate success of the development process: (i)
determining a minimal subset of Fortran that, if adhered to, would
significantly reduce the level of manual effort in the conversion
process and (ii) educating the programmers in the whys of the minimal
subset, in the 3 different architectures, and in the importance of
writing portable code.

The conversion process was successful, but not without a number of
serious problems. The minimal subset of Fortran couldn't encompass
some of the different but needed operating-system-specific
capabilities. Assembly language code was required in some cases for
performance reasons or to stay within memory limitations. IBM
assembly code could be assembled and tested to some extent on MMC's
IBM 370, but the Univac assembly code could only be developed and
tested on JPL's 1108. Fortran programs tested and changed at JPL had
to have the changes back-ported to the CDC source code, an annoying
and error-prone task.

I was very surprised at how difficult the IBM computers made
everything, and not just in the conversion process. JPL had an
unhappy history with IBM 360s according to Tomayko (Chapter 8,
Section 3, p. 265). In late 1969 and early 1970, JPL received two
hand-me-down 360s from two other NASA centers; JPL later purchased a
third, brand-new 360. The one previously used on the Apollo program
had a bespoke, real-time operating system from IBM. At the time of
Mariner 9's launch in May 1971, the ground software failed every 5
hours. By the time Mariner 9 reached Mars in November, the failure
rate had thankfully dropped to only every 20 hours! (I am reminded of
a colleague in the 1980s who went to work at an IBM facility in
Maryland for a low-earth-orbiting satellite. Per his telling, the
ground system would usually crash once during a 20-minute satellite
pass, but the system rebooted fast enough to keep the pass from being
a total waste. Again, this was in the 1980s and I don't remember if
the system used an IBM or non-IBM minicomputer.)

Some mistakes were self-inflicted, although I would not count among
those good-faith decisions that just didn't turn out as planned. All
the logic and reason in the world can't foresee, for example, a
faulty IBM Fortran compiler on JPL's 360. However, Martin Marietta
should have foreseen the major hit the Viking effort would take from
switching the CDC 6500's operating systems from CDC's unofficial MACE
OS to its official SCOPE OS in the middle of development. Viking
would not have been the only project supported by the CDC mainframe
and perhaps MMC calculated that the benefits to other projects which
needed SCOPE would outweigh the cost to Viking and other ongoing
projects.

Despite MMC not having a Univac 1108, the development, conversion,
and testing of the 1108 software was much smoother than for the IBM
software. Being able to compile and test the Univac programs at JPL
anytime day or night was probably a significant factor in this.

    QUALITATIVE RESULTS: ... This occurred in the IBM 360 conversion
    effort. Due to the operational procedures at JPL, in order to
    receive the necessary turnaround time to do program conversion
    and testing, blocks of computer time had to be scheduled. These
    blocks of time were usually 4 to 6 hours in duration during the
    week starting at 9:00 PM to 4:00 AM, and up to 48 hours duration
    on the weekends. The pressures of trying to make as many runs as
    possible and meet delivery schedules forced many extra errors
    into the software. Instead of doing a detailed analysis of the
    code and the dump of the program the programmer or engineer would
    shot-gun many runs to try to fix errors. During longer block-time
    stretches many engineers and programmers would work until they
    introduced new errors due to physical and mental exhaustion. This
    altering of normal work habits did not occur on the 1108
    conversion effort. The turnaround was excellent and the machine
    was available for use during normal working hours. (p. 34)

    ... The JPL version of OS was a real-time system with standard OS
    features but was a different release than the MMC's IBM OS. This
    caused problems in the conversion effort. In one instance the
    difference in FORTRAN compilers between MMC IBM and JPL IBM
    caused schedule slippages due to errors in the target compiler
    that were not discovered until the conversion process began. This
    problem happened early in the conversion process and two steps
    were taken to help solve the problems: 1) the release of the
    compiler JPL had was installed at MMC for use by the software
    still in development and 2) the release of the MMC compiler was
    installed at JPL for use when needed. (pp. 34-35)

    ... The operating system [on MMC's CDC computers] was changed
    from MACE to SCOPE during the period in which the software was
    being developed. This change caused many schedule impacts in the
    Viking software development. Much time was spent by the
    development programmers changing their job control cards, their
    file naming conventions, their file structures, and in learning
    how the system worked. This coupled with extra down time and
    running of two operating systems caused much confusion. (p. 35)

    QUANTITATIVE IMPACT: ... The conversion process represented
    approximately five percent of the development effort for 1108
    programs, and approximately ten percent of the development effort
    for 360 programs. (p. 36)

    Martin Marietta Corporation, Viking Software Data: Final
    Technical Report (15MB PDF) May 1977, RADC-TR-77-168.

I haven't read Martin Marietta's report from cover to cover, but I
have read substantial amounts in the operations software and flight
software sections. The prose is not dry and Martin Marietta was
honest about what techniques and practices worked and which didn't
and why. Again, another sort-of-Voyager-related-but-not-really
document you can dive into on any page and enjoy.

I was unable to find an online copy of 1973's "Characteristics of
FORTRAN" (download page for brief-description PDF) by W.R. Garner of
Martin Marietta, which detailed the differences between Fortran on
the IBM 360, Univac 1108, and CDC 6000s. Nor was I able to find a
copy of Martin Marietta's Viking Flight Operations Programmer Guide
(mentioned in the Viking Software Data report), which described the
minimal subset of Fortran to be used at MMC.

However, I did came across another, slightly later instance of Martin
Marietta converting a Fortran program developed on the CDC 6500 to
run on a Univac 1108. In this case, the program implemented a
contamination model for the ESA Spacelab flown on multiple Space
Shuttle missions. The target 1108 was located at NASA's Marshall
Space Flight Center in Huntsville, AL. This 1976 requirements study
detailed the differences between FORTRAN IV on the CDC 6500 and
FORTRAN V on the Univac 1108:

    This study was performed in order to determine the modifications
    necessary to convert the Martin Marietta Aerospace (MMA) Spacelab
    Contamination Computer Model, written for a CDC 6500 computer,
    for use on a Marshall Space Flight Center (MSFC) computer ...

    Major differences between the CDC 6500 and UNIVAC 1108 are memory
    capacity, word length, and various specific functional
    capabilities. The CDC 6500 uses Fortran IV whereas, the UNIVAC
    1108 uses Fortran V control language. None of the differences are
    such as to create any great difficulties in program conversion
    and a good programmer, familiar with the contamination program,
    should be able to process the necessary changes with minimum
    difficulty, knowing the differences.

    --L.E. Bareiss, V.W. Hooper, E.B. Ress, and D.A. Strange, "Payload
    /Orbiter Contamination Control Requirement Study - Computer
    Interface Study", Final Report, p. ii, September 30, 1976.

---------------------------------------------------------------------
                                                                     
                                                  * Essential Reading
                                * Command Sequences & Flight Software
                                                      * Other Sources
                                                        * Multimedia!

Bibliography

Essential Reading

I recommend reading Tomayko's Computers in Spaceflight, Chapter 6-2
first as knowledge of the Voyager flight computing hardware will help
when reading the other texts. Then, Matsumoto's paper provides an
excellent overview and details about the current (as of 2016) state
of the spacecraft and the full range of efforts needed to extend the
life of the spacecraft as long as possible.

Following those two, I recommend McEvoy's paper for its detailed
description of spacecraft command sequencing and CCS simulation for
the Viking Orbiter. Adamski's paper, under "Other Sources", covers
similar territory for Voyager, with a slightly differing focus.
McEvoy's paper gets down into some of the software details, but
they're both very interesting, so read both!

Tomayko, James E. Computers in Spaceflight: The NASA Experience (36MB
PDF; higher quality 500MB PDF), 1988. The archived,
easier-to-navigate HTML version:

  * Title Page
  * Table of Contents
  * Chapter Five, Section 6, "Viking computer systems" (details the
    Orbiter CCS computer used for Voyager's CCS and AACS computers)
  * Chapter Six, Section 2, "Voyager - The flying computer center"
  * Chapter Eight, Section 3, "Unmanned mission control computers"

Matsumoto, Sun Kang. "Voyager Interstellar Mission: Challenges of
Flying a Very Old Spacecraft on a Very Long Mission" (PDF), 2016
SpaceOps Conference, Daejeon, Korea.

    Matsumoto's paper was a pleasure to read the first time, but it
    wasn't until the second read for the purposes of this page that I
    realized how packed it is with information about the Voyager
    hardware, software, and operations. Aside from the treasure trove
    of information, the paper also gives you a good idea of the kinds
    of real problems the Voyager team has had to deal with over the
    decades and the many constraints under which they diagnose and
    solve problems. Some other papers I discovered later provide
    similar insights into the Voyager environment in the pre-VIM era
    (1970s and 1980s): Brooks; McLaughlin and Wolff; and Morris. But
    Matsumoto remains essential reading. (Matsumoto is the Voyager
    Fault Protection and CCS Flight Software Systems Engineer.)

McEvoy, Maurice B. "Viking Orbiter Uplink Command Generation and
Validation via Simulation" (PDF), The Institute for Operations
Research and the Management Sciences (INFORMS) Winter Simulation
Conference 1975 Conference Proceedings.

    McEvoy's paper was in an important source for me as it supplied
    much needed details about sequencing that I hadn't found in
    earlier reading. Written and published before the Viking
    launches, the paper presents a refined, sophisticated, mature
    sequence development and simulation process, or so it seemed to
    me. I assumed that the Viking experience and many of the
    associated software tools could have been reused on the Voyager
    project. After writing most of this web page, I discovered Brooks
    's paper in which he described the somewhat chaotic, rough and
    tumble evolution of Voyager sequencing, both before and
    continuing after launch. I began to wonder if McEvoy was perhaps
    giving us a distinctly rose-colored view of Viking sequencing. Be
    that as it may, the details about sequences and simulation are no
    less useful.

Command Sequences & Flight Software

In addition to Matsumoto's and McEvoy's papers immediately above ...

The full texts of papers annotated as "(abstract)" can be found
online with some effort.

Adamski, Terrence P. "Command and Control of the Voyager Spacecraft"
(abstract), American Institute of Aeronautics and Astronautics
(AIAA), 25th AIAA Aerospace Sciences Meeting, March 24-26, 1987,
Reno, Nevada.

    Adamski's paper is similar to McEvoy's above. Whereas McEvoy gets
    down into some of the computing details of sequencing (e.g.,
    SEQGEN and OSTRAN), Adamski provides a fuller overview of the
    entire sequencing process from initial science proposals to the
    final review and uplink approval of a sequence. This bureaucracy,
    vitally important for the planetary encounters, was streamlined
    and downsized for the interstellar mission, as described in
    Linick and Weld below. (Adamski was the Voyager Flight Operations
    Manager.)

Brooks, Robert N., Jr. "The Evolution of the Voyager Mission Sequence
Software and Trends for Future Mission Sequence Software Systems"
(abstract), American Institute of Aeronautics and Astronautics
(AIAA), 26th Aerospace Sciences Meeting, January 1988, Reno, NV.

    This was a late addition to my reading, unfortunately, tracked
    down via a citation in another paper. As the title suggests,
    Brooks recounts the pre- and post-launch evolution of the Voyager
    sequencing process. The process presented here is very rough
    around the edges, in contrast to McEvoy's smoothly functioning,
    Viking Orbiter sequencing process. Brooks does go into
    considerable detail about the Univac-based sequencing software. I
    was delighted to discover that the sequence engineers, like any
    good engineers, wrote unofficial programs to assist in their
    analysis and design of sequences -- in BASIC! Being interpreted,
    these programs were convenient to use and modify and, equally
    importantly, they were invisible to the POISON program! POISON
    scanned the computer, looking for unofficial programs in an
    attempt to ensure that only officially sanctioned software was
    used to produce uplink loads. The penultimate section in the
    paper discusses JPL's research into a role for AI in sequence
    design and development, something that would not be out of place
    in the 2020s! (And was not out of place in the 1980s either.)

Linick, Susan H. and Weld, Kathryn R. "Voyager Uplink Planning in the
Interstellar Mission Era", SpaceOps 1992: Proceedings of the Second
International Symposium on Ground Data Systems for Space Mission
Operations, March 1, 1993. (PDF)

    Although over 30 years old now, this paper provides a more modern
    look at Voyager sequencing, especially the move off of the Univac
    mainframes onto PCs and, because of the reduced workforce, the
    streamlining of the sequencing process. Linick and Weld do get
    into some of the details of the technical challenges and
    constraints, but the paper strikes me as mostly a management view
    of the redesign that, understandably because it's not the focus
    of the paper, lacks the hardware and software details I would
    have liked.

McLaughlin, W.I. and Wolff, D.M. "Voyager Flight Engineering:
Preparing for Uranus" (abstract), American Institute of Aeronautics
and Astronautics (AIAA), 23rd Aerospace Sciences Meeting, January
1985, Reno, NV.

    This paper explores some of the flight software and sequencing
    changes planned for Voyager 2's 1986 Uranus fly-by. Other papers
    describe sequencing in fairly general terms, but McLaughlin and
    Wolff examine actual, concrete problems and proposed solutions in
    detail. The paper covers a number of areas, but a few stood out
    for me: (1) working around the reliance on Voyager 2's partially
    failed backup radio receiver; (2) offloading image compression
    duties to the backup FDS computer; (3) figuring out how usable
    the scan platform's crippled actuator was; and (4) preventing
    image smear during the long exposure times required for Uranus
    and Neptune. Another item of interest to me: flight software was
    written to alleviate the CCS computer's limited memory by storing
    not-yet-needed sequences in the FDS memory. Since this paper was
    written pre-encounter, I can't tell if this capability was ever
    uploaded and used.

Morris, Ray B. "Sequencing Voyager II for the Uranus Encounter"
(abstract), American Institute of Aeronautics and Astronautics
(AIAA), Astrodynamics Conference, August 1986, Williamsburg, VA.

    Like the others, Morris's paper also describes the sequencing and
    simulation process. Where Morris really comes into his own is
    when he delves into the operational limitations of sequences and
    how the Voyager team worked around them. A very important example
    is that the pre-planned, planetary encounter sequences must be
    uplinked before the flyby, but must be able to be updated as
    observations closer to the planet provide more accurate
    information and reveal needed changes. This requirement was met
    by the introduction of movable blocks for the observations and
    late stored updates. I'm not sure I understand them completely,
    but you get the idea. (Morris was the Voyager Flight Engineering
    Office Sequence Team Chief.)

Other Sources

I encountered a lot of articles online that were reworkings either of
Mann's 2013 Wired article or of Wenz's 2015 Popular Mechanics
article. I haven't included them here unless I specifically reference
them in this piece.

The full texts of papers annotated as "(abstract)" can be found
online with some effort.

Assessment of Autonomous Options for the DSCS III Satellite System,
Prepared for the U.S. Air Force by JPL personnel (Donna L. S.
Pivirotto and Michael Marcucci?), "Volume III: Options for Increasing
the Autonomy of the DSCS III Satellite" (8MB PDF), August 6, 1981.

Bareiss, L.E.; Hooper, V.W.; Ress, E.B.; and Strange, D.A. "Payload/
Orbiter Contamination Control Requirement Study - Computer Interface
Study", Final Report, September 30, 1976.

    Not Voyager-related; quoted in Clash of Viking Mainframes.

Campbell, James K.; Synnott, Stephen P.; and Bierman, Gerald J. "
Voyager Orbit Determination at Jupiter" (PDF), IEEE Transactions on
Automatic Control, Vol. AC-28, No. 3, March 1983.

Culver, John. "The CPUs of Spacecraft Computers in Space", The CPU
Shack.

Data General Fortran 5 Documentation, including the Fortran 5
Reference Manual (10MB PDF).

Eickhoff, Jens. Onboard Computers, Onboard Software and Satellite
Operations, 2012. Chapter 3.2.2, "Transistor based OBCs with CMOS
Memory: The Voyager 1/2 Missions", pp. 35-37.

Gugliotta, Guy. "Historic Voyager Mission May Lose Its Funding", The
Washington Post, April 3, 2005.

Heacock, Raymond L. "The Voyager Spacecraft" (abstract), Proceedings
of the Institution of Mechanical Engineers, Vol. 194, June 1980.

    Heacock's paper is widely cited because it provides a concise,
    readable, detailed description of the components on the
    spacecraft, excluding the science instruments. He includes some
    history of the mission and the reasoning behind some of the
    component/configuration designs.

Holmberg, Neil A.; Faust, Robert P.; and Holt, H. Milton. Viking '75
Spacecraft Design and Test Summary, Volume II: Orbiter Design (NASA
Reference Publication 1027), November 1980. (Download page for 13MB
PDF)

Kehr, Joachim J. "Voyager -- The Right Staff" (PDF), April 2016. Kehr
was the editor of SpaceOps News, of the Journal of Space Operations &
Communicator; this article was apparently not published.

Kobele, P. "Maintainability of Unmanned Planetary Spacecraft: A JPL
Perspective" (abstract), American Institute of Aeronautics and
Astronautics (AIAA)/NASA Maintainability of Aerospace Systems
Symposium, July 1989, Anaheim, CA.

    Kobele examines how anomalous conditions are/were handled in the
    Mars Viking Orbiter, Voyager, and the Jupiter Galileo Orbiter.
    (Galileo hadn't been launched yet at the time of publication.)
    Whereas the continuation of science observations has been a
    priority for Voyager when an anomaly occurs, the philosophy on
    Galileo was to put the spacecraft in a stable configuration
    ("safe hold") and await instructions from the ground. Since
    Galileo was an orbiter, it made sense to postpone science
    observations to later orbits and instead protect the health of
    the spacecraft.

    Kobele describes notable Viking and Voyager anomalies, but,
    again, Galileo had yet to be launched and anomalies like the
    failed deployment of the high-gain antenna were still in the
    future. The paper finishes up with an in-depth look at "lessons
    learned", which is quite good.

Kohlhase, Charles. The Voyager Neptune Travel Guide, 1989. (Download
page for 116MB PDF)

    A late addition ... This is a beautiful 290-page book, even in
    its black-and-white PDF form, written in an easy-going, engaging
    style. The purpose of the book is to get the reader excited about
    (i) the wonders of space and (ii) the wonders of the technology
    used to explore space. It's aimed at the educated layperson. Some
    familiarity with space and spacecraft probably helps, but I think
    a neophyte could just accept what they don't understand and still
    revel in the wonder of it all. I've only read the first 40 pages
    and I learned a lot; for example, one of the reasons the scan
    platform is mounted on a boom is so that, post-encounter, the
    cameras can look back at a receding planet without their view
    being blocked by the high-gain antenna. Fascinating!

    The hard-copy version of the book features a flip-book animation
    of the Neptune/Triton encounter. Someone with patience could snip
    the approximately 130 graphics from the PDF and produce an
    animated GIF. (Kohlhase was Manager of Voyager Mission Analysis
    and Engineering.)

Laeser, Richard P. "Voyager -- Uranus at Our Doorstep" (abstract),
Acta Astronautica, Volume 14, 1986.

    This paper turned out to not be pertinent to my Fortran 5
    inquiry, but it is nevertheless a very interesting read. Laeser
    goes into good detail about two serious Voyager 2 problems with
    the spacecraft's radio receivers and the scan platform,
    describing how the engineers figured out what the problems were
    and how to work around them. (Laeser also wrote a similar,
    somewhat complementary paper, "Engineering the Voyager Uranus
    Mission".)

Ludwig, Roger and Taylor, Jim. "Voyager Telecommunications" (3MB
PDF), Article 4, March 2002, of JPL DESCANSO Design & Performance
Summary Series.

    A number of other sources also describe Voyager 2's receiver
    problem, but this paper about Voyager radio communications
    provides more radio- and electronics-focused details (pp. 39-40).
    Other sources also give slightly different numbers for the
    100-KHz and 100-Hz PLL ranges, but they all round off to Ludwig's
    and Taylor's values, so I use the latter in the text of my web
    page. Telecommunications-specific actions in the Backup Mission
    Load (BML) are detailed on p. 41.

Magnin, Vincent. "A patch was sent to Voyager 2 yesterday", Fortran
Discourse, October 2023.

    Throughout this discussion on a Fortran forum, Magnin provided
    useful commentary and links to sources, some in common with mine
    and others new to me. Aside: Browsing the Fortran Discourse forum
    was very enlightening for me. I had read about Fortran 90 when
    its standard was released over 30 years ago, but I had no
    occasion to use it and I haven't kept up with subsequent
    versions. It's changed! My hat's off to modern-day Fortran
    developers. (Although we were no slouches back in the heady days
    of VAX/VMS Fortran-77.)

Mann, Adam. "Interstellar 8-Track: How Voyager's Vintage Tech Keeps
Running", Wired, September 2013.

Martin Marietta Corporation. Viking Software Data: Final Technical
Report (15MB PDF), May 1977, RADC-TR-77-168. Of interest: "Mission
Operations Software Techniques" (ground system), pp. 4-101 and
"Flight Software Techniques" (Lander only), pp. 102-159.

    Not Voyager-related; inspiration for Clash of Viking Mainframes.

NASA. "Calculating Trajectories and Orbits", NASA Tech Briefs,
September 1989, p. 33. (COSMIC catalog entry for the Orbit
Determination Program.)

NASA. Voyager Press Kit (6MB PDF), August 4, 1977. (Pre-launch)

Patel, K.; Reinholtz, W.; and Robison, W. "High Speed Simulator -- A
Simulator for All Seasons" (PDF), SpaceOps 1996, Proceedings of the
Fourth International Symposium, Munich, Germany.

Post, Ed. "Real Programmers Don't Use Pascal", Datamation, July 1983,
"READERS' FORUM", pp. 263-265. (HTML version; the full magazine issue
is available as a 51MB PDF.)

Rasmussen, R.D. and E.C. Litty. "A Voyager Attitude Control
Perspective on Fault Tolerant Systems" (abstract), American Institute
of Aeronautics and Astronautics (AIAA) Guidance and Control
Conference, August 19-21, 1981, Albuquerque, New Mexico.

    The above is a very interesting paper about fault detection,
    isolation, and management on the Voyager spacecraft, with a
    special focus on the AACS. It does get into some of the details
    about cross-strapping, about which I posed questions in "
    Sequences and Simulators" above.

Sharwood, Simon. "Has Voyager 1 escaped the Sun yet? Yes, but also
no, say boffins", The Register, October 30, 2015.

South Australian Doctor Who Fan Club Inc. (SFSA), "Voyagers disco
party!", The Wall of Lies, No. 169, Nov-Dec 2017, p. 2. (4-page
newsletter, PDF)

Tingley, Kim. "The Loyal Engineers Steering NASA's Voyager Probes
Across the Universe", New York Times, August 3, 2017.

UNIVAC 1108 System Description (14MB PDF), 1970. (FORTRAN V
description in Section 10, pp. 7-12.)

UNIVAC 1108 FORTRAN V Programmer's Reference Manual, 1966, Artifact
Details (not the actual manual) at the Computer History Museum.

Univac ASCII FORTRAN Reference Manual (download page for 65MB PDF).

Viking project hearings before the Subcommittee on Space Science and
Applications "of the Committee on Science and Astronautics, U.S.
House of Representatives, Ninety-third Congress, second session,
November 21, 22, 1974 [i.e. 1975]", "Flight Software Technical
Problems".

    The link above takes you to the beginning of a set of slides
    about the flight software, which is how I discovered this
    hearing. Don't limit yourself to that brief presentation -- random
    browsing in the document will land you on interesting testimony
    by NASA personnel and exchanges with congressmen.

Wenz, John. "Why NASA Needs a Programmer Fluent in 60-Year-Old
Languages", Popular Mechanics, October 29, 2015.

Wood, Lincoln J. The Evolution of Deep Space Navigation: 1962-1989"
(PDF), 31^st Annual AAS Guidance and Control Conference, 2008,
Breckenridge, Colorado.

Zierler, David. "Suzy Dodd (BS '84), Engineer and Deep Space Pioneer
", Caltech Heritage Project, June 9, 2023.

Wikipedia (abridged list). There is actually a whole forest of
Wikipedia pages I used, following links here and there, but these are
a few of the main ones:

  * Data General and DG Nova
  * Fortran
  * Viking program
  * Voyager program
  * Voyager 1
  * Voyager 2

  * Control Data Corporation, CDC 6000 series, and CDC display code
  * IBM, 360, 370, and EBCDIC
  * UNIVAC, 1100/2200 series, and FIELDATA

Multimedia!

I believe Aaron Cummings's YouTube video showed up in search results
when I was looking up information about the Voyager computers. I
learned about the two movies (and lots of other information) from the
Ars Technica community.

Cummings, Aaron. "Uptime 15,364 days - The Computers of Voyager"
(YouTube), Strange Loop Conference, September 14, 2019.

    A video of a 30-minute presentation followed by a 10-minute Q&A
    session. A great speaker and my only complaint is that the
    presentation wasn't longer. I didn't really find out anything new
    about the computers since Cummings drew from many of the same
    sources as me, but there was a lot of information about the
    mission history, hardware, and communications that I found very
    interesting. He noted the lack of information about the flight
    software. Also, people had told him the flight software was in
    Fortran, but he didn't buy it (at 17:21 in the videos). His
    reference list has additional links to a variety of information
    about the spacecraft and mission.

The Farthest (IMDB), 2017. (Watch on PBS)

    A 90-minute documentary film about the Voyager program. In
    contrast to the present-day focus of It's Quieter in the
    Twilight, this movie documents the early years of the program and
    the four planetary encounters: Jupiter, Saturn, Uranus, and
    Neptune. (The film does take a brief look at the interstellar
    mission towards the end.) There's a lot of contemporary news and
    science footage and a lot of looking-back commentary from people
    who'd been with the project since the beginning. This movie
    documents the science and technology of Voyager, whereas It's
    Quieter in the Twilight is a lower-key recording of the everyday
    life of the current Voyager team. The former gives the big
    picture and the latter provides some insight into the unglamorous
    world of maintaining two spacecraft more than 10 billion miles
    away. Watch both!

It's Quieter in the Twilight (IMDB), 2022.

    An 80-minute documentary film about the Voyager team prior to and
    into the 2020 COVID-19 pandemic. They worked in a garden-office
    complex next to a McDonald's, about a mile from the JPL campus.
    (And had been doing so for a couple of decades?) During the
    pandemic, their offices were moved back to JPL.

    After the Neptune encounter, Voyager 2 angled south of the
    ecliptic (planetary) plane as it headed out into space. As a
    result, the spacecraft can only be "seen" from the Australian
    Deep Space Network (DSN) station and commands can only be sent to
    Voyager 2 using that station's 70-meter dish antenna. An
    11-month-long, major upgrade to the aging antenna was scheduled
    to begin in February 2020. During the upgrade, commands could not
    be sent to Voyager 2, but smaller antennas were available for
    receiving engineering and science data from the spacecraft. A
    good part of the film revolves around the preparations of Voyager
    2 for this downtime and then the downtime itself. Some unexpected
    and unwanted drama occurred right before the upgrade when a
    routine "MAG roll" went awry. NASA admirably delayed the upgrade
    until March so the Voyager team could figure out what had
    happened and successfully finish preparing Voyager 2 for 11
    months of command-less travel. A test transmission or transaction
    of some kind was made in October 2020 prior to the 70-m dish
    returning to full operational status in February 2021.

---------------------------------------------------------------------

Alex Measday  /  E-mail