Saving Voyager 1! - Bruce Waggoner at !!Con 2024

2 weeks ago, Bruce Wagoner from the Voyager program gave a talk at !!Con about how they recovered from the CMOS memory issue that they had a year ago. It’s basically blind debugging with a latency of 45 hours. The talk is amazing and goes through the computer architecture of the spacecraft as well as the challenges of dealing with something that is so old, with so documentation that has conflicting information or unreadable etc. ▶ this video

When Voyager failed last year with a CMOS memory error, one of the big problems was that a bunch of low level information was gone or conflicting. For example, they sometimes had to guess assembler instructions because the code printouts were low quality photocopied pages. Or because there were handwritten comments or comments scratched out with pencil without any clue about why it was done. One saving grace was the fact that the architecture and the code space was simple enough so that they could reason through the symptoms and actions to take, something that would have been much harder with modern spacecraft. Check out this amazing talk: ▶ this video

You might like this and or find it inspiring, in case you haven't seen it already: Saving Voyager 1! - Bruce Waggoner at !!Con 2024 - ▶ this video Edit: Also, please do share when it's playable :)

Check out this talk: ▶ this video The source code isn't hiding in a repo somewhere for security reasons — it's spread around on various pieces of paper and computers over the last 50 years. There isn't a single source of truth. Adds a whole other level of wizardry to keeping the thing running.

I had the extreme pleasure of seeing Bruce Waggoner of the Voyager team give a keynote at !!Con just last month. The recording landed on YouTube just a couple days ago, so this is great timing: ▶ this video

> Regarding "that almost nobody on Earth fully understands anymore", I claim this is nonsense, and definitely not an obstacle. It seems like you aren't very familiar with the history of the Voyager or the computer systems it uses. The Voyager has three onboard computers, each of which has a custom ISA. Two of the three computers were only used on Voyagers. After the main Voyager mission ended in the late 1980s and they were repurposed to collect data on interstellar space, the probes were reprogrammed to only require limited commands on one of the computers, and no intervention whatsoever on the other two. They also got rid of the testbench for testing code on the ground, so the only working Voyager systems are billions of miles away from Earth. Since then, Voyager has been operated by a skeleton crew that has been shrinking over time as people retire. The result of all of this is that they legitimately do not have a full understanding of how the hardware and software operates. Here's one example from a talk about how the crew fixed an issue with the Flight Data Subsystem, one of the computers that hadn't needed any significant software changes since the interstellar mission began ( ▶ this video ): > So our top priority was to figure out what the FDS was and how it worked. Because unfortunately the person who was the real expert had retired decades ago and the person who was their fill-in, their backup, had retired two years ago. So it was the worst place it could have hit us. The