Stanford Seminar: Beyond Floating Point: Next Generation Computer Arithmetic

For anyone interested in this sort of thing you'd probably also be interested in John Gustafson's work on unums/posits. Here's a nice introductory presentation ▶ this video

You may like: ▶ this video It turns out the math is slightly harder but it's faster than IEEE FP in hardware, probably because of less conditionals in the spec. There is better performance in terms of numerical accuracy at the cost of substantially more difficult error analysis. Disclaimer: I'm in the video.

Have you seen "Stanford Seminar: Beyond Floating Point: Next Generation Computer Arithmetic"? ▶ this video

Well unums in particular (especially the new type 3 "posits" ones) are very interesting and practical for hardware implementation. John Gustafson, the creator of unums, gave the first talk on type 3 unums at Stanford earlier this month, and there are already multiple hardware implementation efforts (one of which I am indirectly overseeing). While many people have recently hyped up low precision floating point for things like machine learning, Posits provide greater dynamic range AND greater accuracy for very few bits (as low as four, but with good ML starting 8 to 10 bits). On top of all of that, it costs less area and power in silicon than IEEE hardware. Check out the talk: ▶ this video

I'm very surprised nobody has brought up John Gustafson's Unum work. His latest iteration is the posit, which he presented at Stanford earlier this month[0][1] EDIT: apologies to CalChris[2]. What I should have said is discussed Gustafson's Unum work. [0] ▶ this video (relevant question at 1h22m37s) [1] link [2] link

2003, was working in a science lab, lab tech came in and said he (or maybe it was his brother) had just bought a house, really cheap, in hawaii. He said it was in lanikai estates. Family is from hawaii and I'm a bit of a geography geek, so I hopped onto mapquest to see where it was... Noticed it was due east of pu'u o'o, and I know the volcano is moving east, so I told him, look you have 10-15 years to sell that house before it winds up under a volcano. Fast forward 15 years and volcano! The volcano just snuck in under my time window, haha. To be fair, I do not know for sure if HIS house went under; there are houses in the neighborhood that were spared. I didn't bother to ask because when I stalked him it said he was an employee of Theranos, and I decided it was probably not a good idea to poke a hornet's nest. Speaking of Theranos, back when I was in my 2nd postdoc (2013) we were talking about how theranos was a fraud and it was going to come crumbling down, Some other crazy short term things I predicted: Sarah Palin VP candidate, Feb 2008, Simone Giertz becoming famous (ca winter 2015). I also predicted the failure of the first starliner launch (documented here in HN). I also have managed to pick some ascendant "obscure" PLs just before hockey sticking. Used Julia professionally in 2015 (proof of use in 2017: ▶ this video ), used Elixir in 2017, been using Zig since 2019. And I haven't really used any other non-mainstream PLs, so I have a pretty good hit record I think. And a co

There was no paper, sorry, we talk about some of the research here ▶ this video big julia demo 40 minutes in. But the video doesn't feature the results I talk about (that happened a few months after the video) the specific results are somewhere in a DARPA presentation (but I don't have those slides, and the data are like 5 laptops back for me). The parallelism was, well, Gustafson parallelization, I pushed a randomly-generated 10k x 10k matrix solve to each of the nodes with a custom soft floating point to measure the relative numerical accuracy. Each node did all of one run per core, it was done very fast , but it would have taken months on a pc. As it was a gustafson scaling at its finest, there was zero cross-node communication during the computation. The containerization I used was singularity. I don't really love them much anymore, for various non-technical reasons, but it got the job done.

I should probably post evidence; this was a live demo I did of the floating point stuff at Stanford (demo begins ~53 minutes in) ▶ this video

Most likely related: John L. Gustafson presenting "Beyond Floating Point: Next Generation Computer Arithmetic" ▶ this video With papers here: - UNUMS: link - Posits: link

Unums are unlikely to gain much usage. Posits[1][2], also by Gustafson, are a more reasonable alternative to IEEE-754 floating point (but will still have a difficult time displacing IEEE-754, if they can at all). [1] link [2] ▶ this video

so I'm the author of one of the first usable unum implementations. We're actually moving to something called sigmoid numbers which are even better. The lecture doesn't cover "valid mode" but that's more like the "unum". At the end of the video, there's a demonstration where I show some very interesting results concerning machine learning. ▶ this video You can try out sigmoid numbers in julia here: link Also, I am working on some verilog implementations, so we may have this in hardware rather soon. A few things about sigmoid numbers: 1) they have fixed sizes, so you don't have to worry about keeping track of things (which was a pain). 2) they have a property I call "isomorphic structure" - which means that zero-padding does not affect the value, and truncation gives you the nearest value at a lower bit resolution. 3) they really do perform better than floating points in several applications. There's a higher bit-per-bit entropy for 'most things you want to do', so the information content is richer. The distribution of the numbers is "more human" - that is to say at a certain point you stop caring about the details around the number, and more care that it's 'really big' or 'really small'; and the distribution reflects that.

Are these the Neuromorphic chips [1] Jeff Hawkins of Numenta [2] has been talking about? [1] Neuromorphic Chips link [2] Numenta papers/videos link It would also be good to see a major chip manufacturer or cloud provider that makes its own chips (Google/IBM) get serious about graph processing chips [3,4] and moving beyond floating point [5]. [3] Novel Graph Processor Architecture link [4] Novel Graph Processor Architecture, Prototype System, and Results link [5] Stanford Seminar: Beyond Floating Point: Next Generation Computer Arithmetic ▶ this video