RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing by Joe Duffy
Confreaks2 HN recommenders · recommended in 2 months, 2017–2025
talk · intermediate · evergreen · 2017–2025 on HN
Joe Duffy's RustConf 2017 closing keynote on his experience at Microsoft building Midori, an operating system written in a Rust-like safe systems language. He covers the technical and cultural challenges of applying those lessons to Windows.
HN repeatedly cites it for Midori lessons: compatibility matters for adoption, and the Windows team stayed sceptical even with Midori running real workloads.
Up next
11 mo on HNPrinciples for Dealing with the Changing World Order by Ray Dalio
Principles by Ray Dalio
10 HN recommender · 2022–2025
3 mo on HNJohn Carmack Tech Talk with UMKC-SCE
UMKC School of Science and Engineering
3 HN recommender · 2017–2023
4 mo on HN"Data Driven UIs, Incrementally" by Yaron Minsky
Strange Loop Conference
4 HN recommender · 2018–2022
11 mo on HNMaybe Not - Rich Hickey
ClojureTV
14 HN recommender · 2018–2025
6 mo on HNThe Clean Architecture in Python
Next Day Video
5 HN recommender · 2017–2021
3 mo on HNGopherCon 2016: The Design of the Go Assembler - Rob Pike
Gopher Academy
3 HN recommender · 2017–2025
4 mo on HNDtrace Review
GoogleTalksArchive
4 HN recommender · 2013–2026
11 mo on HN"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
Strange Loop Conference
12 HN recommender · 2014–2024
11 mo on HNCppCon 2015: Andrei Alexandrescu “std::allocator...”
CppCon
7 HN recommender · 2017–2025
12 mo on HNLexical Scanning in Go - Rob Pike
GoogleFOSSSydney
13 HN recommender · 2012–2025
10 mo on HNCppCon 2017: Matt Kulukundis “Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step”
CppCon
9 HN recommender · 2018–2026
9 mo on HNRustConf 2018 - Closing Keynote - Using Rust For Game Development by Catherine West
Rust Programming Language
9 HN recommender · 2018–2023
12 comments from Hacker News
See also "Safe Systems Software and the Future of Computing" by Joe Duffy, the closing keynote at RustConf 2017: ▶ this video Midori and Rust have several striking similarities, and Microsoft's recent uptick in interest in Rust (and Rust-like languages) bodes well for improved software quality.
Related: "Safe Systems Software and the Future of Computing by Joe Duffy" at RustConf 2017. ▶ this video I summarized this excellent talk here [1], but one of the main points is that compatibility with existing systems is important for adoption. (They learned that the hard way -- by having their entire project cancelled and almost everything thrown out.) He advocates unit-by-unit rewrites rather than big-bang rewrites, just like Kell does in this conference article. And compatibility with C in Windows should be easier than it is in the Unix world, because the whole OS is architected around a binary protocol AFAIK -- COM. My sense is that Rust may not have thought enough about compatibility early in its life. Only later when they ran into adoption problems did they start talking more about compatibility. Also, it seems Rust competes more with C++ than C, and there seems to be very little attempt to be compatible with C++ (although perhaps that problem is intractable.) Personally I don't think Rust will be a successful C replacement. It will have some adoption, but the Linux kernel will still be running on bajillions of devices 10 years from now, written in C. And in 20 years, something else will come along to replace either C or Linux, but that thing won't involve Rust. [1] link
Mostly because of internal politics, it was used in production for Bing. > While never reaching commercial release, at one time Midori powered all of Microsoft’s natural language search service for the West Coast and Asia. From link Enjoy this recording of an internal presentation, while it is still available on YouTube, link Joe Duffy also did a few presentations, on one of them (too lazy to search for the exact moment), he mentions that even with Midori running in front of them, the Windows team was very sceptical of it, "RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing by Joe Duffy" ▶ this video "Safe Systems Programming in C# and .NET" link
I agree with your point of view, this was the second coming of Longhorn, WinRT was after all pushed by the same folks that killed the Longhorn effort, those on Windows team that sided with Steven Sinofsky, using COM in Vista as future foundation for .NET ideas in Longhorn. The Hilo tutorial for Windows 7 remarks regarding C++ are quite telling of their point of view, on which programming language one should be using, link "The rich user experience of Windows 7 is best accessed through a powerful, flexible language, and that means C++: by using C++ you can access the raw power of the APIs for Windows 7." From link Joe Duffy has some remarks on how even with Midori running Asia Bing infrastructure, and proving workloads in front of the Windows team, the reception to it was rather cold. ▶ this video Eventually he left MSR and created Pulumin on top of Go. Meanwhile Apple/NeXT and Google, decided to push languages like Objective-C, Swift, Java, Kotlin, over classical C and C++, proving the point that change is possible if management is willing to support the team, even if that is a very long run. See Apple's Metal Swift bindings, versus Microsoft's Agility SDK for DirectX.
Project Oberon, link The evolution of Oberon based OSes, Ethos, link Active Oberon which is the Oberon variant I prefer, link link link Some screenshots at my article, take it while the site still exists, link SPIN, done in Modula-3 link Singularity, link link Midori, although no source code, the blog posts, existing talks and internal session at Microsof do provide some nice overview, link "The Midori Operating System Overview" link "Safe Systems Programming in C# and .NET" link "Safe Systems Software and the Future of Computing" ▶ this video Xerox PARC Mesa, used on the Xerox Star OS link link link Xerox PARC Cedar, used on Dorado platforms link link link link link Xerox PARC Smalltalk, original documentation can be taken from link - Smalltalk-80, Bits of History, Words of Advice - Smalltalk-80: The Language and its Implementation - Smalltalk-80, The Interactive Programming Environment Squeak and Pharos linage, link link Xerox PARC Interlisp-D, with Medley link link House, written in Haskell link MirageOS, written in OCaml, partially used by Docker (TCP/IP stack), and Xen Hypervisor link link link AmigaOS, link link link link Solo in Concurrent Pascal, The solo operating system: A concurrent pascal program link Lillith in Modula-2 link link Now go out and explore, UNIX is not the be all, end all of how an OS is supposed to be, neither is C the ultimate systems programming language.
See also from him, Safe Systems Programming in C# and .NET link RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing ▶ this video The latter has some remarks on how the WinDev disregarded Midori efforts even with the system proving its value in workloads like Bing[0]. [0] - link
Regarding Midori, besides the blog posts, Joe Duffy did two talks about the subject, "RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing by Joe Duffy" ▶ this video "Safe Systems Programming in C# and .NET" link In one of them, he mentions that even with Midori proving its value to the Windows team, they were quite dimissive of it. It appears to have also been yet another victim of the usual DevDiv vs WinDev politics.
The usual politics, from .NET vs C++ at Microsoft. Here Joe Duffy mentions towards the end that even with Midori running in front of them, the Windows team was sceptical of it. ▶ this video Since .NET's introduction, Microsoft seems to lack the same kind of culture that Apple and Google have towards into steering their platforms into safer languages (e.g. how constrained NDK happens to be, or first class bindings to all OS APIs in Swift). It appears that every attempt to do so ends up being sabotaged in some way to assure C++'s reign at Microsoft and Windows subsystems. Note that Windows is the only desktop/mobile OS where the GUI stack is still fully C++ aware, and they even make a point out of it. link > WinUI is powered by a highly optimized C++ core that delivers blistering performance, long battery life, and responsive interactivity that professional developers demand. Its lower system utilization allows it to run on a wider range of hardware, ensuring your sophisticated workloads run with ease.
To fully get where I am coming from, you have to go back to when .NET was released. .NET was supposed to be the great reunification of VB, C++ and COM runtimes, then also got a Java touch into the mix and .NET happenend (initially was known as Ext-VOS). link Hence why CLR is just like WASM + GC if you prefer a modern comparisasion. If you go back into web archives, when Visual Studio.NET was released, it was going to be .NET everywhere, across the whole stack. However a big management mistake happened, .NET was part of DevTools business unit, while C++ was kept under WinDev, up until Satya started to change the culture, it has been pretty much WinDev vs DevTools. So Managed DirectX comes, eventually gets killed, XNA and Silverlight take over Windows Phone 7, get killed by WinRT and DirectXTK and so on. Going back to the originally statement, if you Google for why Longhorn did not work out, you will find many .NET blaming. link Yet Android, ChromeOS, Midori are examples of what happens when actually everyone works into the same direction bringing an OS into production. Joe Duffy does some remarks on his two talks, where he hints at why it was a failure to fight Windows culture "Systems Programming in C# " - link "Safe Systems Software and the Future of Computing" - ▶ this video Note that for some time the Asian Bing nodes were actually running on top of Midori as production test. A big decision of Vista, was to replicate the .NET design using COM instead (hello WinDev), hence
At RustConf 2017 keynote presented by Joe Duffy regarding Midori at a certain moment, he mentions that even with Midori running in front of them, WinDev guys were still not open to the idea of such kind of system being possible. "Safe Systems Software and the Future of Computing" ▶ this video Sorry a bit lazy to track down the exact moment.
First of all, there is no such language as C/C++. C++ although tainted by copy-paste compatibility with C89, does offer the language features for security conscious developers to make use of, and C++17 is quite a pleasure to use. C on the other hand could be nuked for what I care, it has been clear since 1979 that security and C would never go together. Now regarding your assertions. z/OS was initially written in a mix of Assembly and PL/I. C and C++ came later into the picture as the system got a POSIX personality. The way to write libraries exposed to all languages available on z/OS is via the z/OS Language Environment. Chapter 9.12 of z/OS Basics, link .NET has hardly any pure C code, rather C++. Even so, the C# and VB.NET compilers have been bootstrapped thanks to Rosylin and now C++ is gone from the compiler side. F# was bootstrapped from the early days. Since Roslyn came into production, the .NET team started planning moving parts of the runtime from C++ to C#. "So, in my view, the primary trend is moving our existing C++ codebase to C#. It makes us so much more efficient and enables a broader set of .NET developers to reason about the base platform more easily and also contribute." link Also the first version of .NET's GC was actually prototyped in Common Lisp, link Unity introduced HPC# at GDC 2018 and they are now in the process of migrating C++ engine code into HPC#. Both the requirements of HPC# as C# subset for performance critical code, as the experience wi
It has been proven multiple times, at Xerox PARC, ETHZ, DEC Olivetti, Microsoft that garbage collection systems programming languages are doable. However technology adoption always needs to fight against non-believers. Joe Duffy has stated on his Rustconf 2017 presentation [0] that even Midori running on front for their eyes, with some use cases where System C# was even shoulder-with-shoulder with C++ [1], that Windows devteam was not getting it. Nowadays he is an happy Go coder instead of trying to convince them otherwise. D has a big problem with their GC implementation, which provides good arguments for the anti-GC crowd in systems programming. However they have been improving it quite a bit during the latest releases, as well as, making the runtime library more @nogc friendly. As for reaching mainstream, I share your thoughts. It is not only Rust and Go. There is also Swift, C++17 and the ongoing improvements to make Java and C# more machine friendly (memory management and AOT compilation). Still, even if just a small group gets to use it, it is already a victory. Most languages never achieve it. [0] - ▶ this video [1] - link