Tesla Autonomy Day

I heard from a tesla engineer that they do use lidar to train their models, but that they don’t see the value in deploying it to the fleet. It’s worth watching the tesla event from about a year ago if you haven’t seen it for their full position: ▶ this video The quick notes are that our roads are set up for human vision and a truly capable level 5 system will require human level interpretation of visual data (at which point lidar becomes redundant). He thinks lidar with non human level visual interpretation won’t get us to full autonomy. They see lidar as a short term half measure, a local maximum that’s a distraction from what has to get done anyway.

Watch the presentation from 6 months ago, where they explain the decision to build their own hardware for inferring : ▶ this video It's not surprising that they also build the hardware for training. Correct me if I'm wrong, but Google use the same TPUs for training and inference, because the underlying operations are the same : multiply then add numbers. Once Tesla built the hardware for inferring, the design of the hardware for training is probably similar. Unlike Google's TPUs, Tesla have a specific use case for the hardware (computer vision for automotive), and maybe than means they can further optimize the computation pipeline with their own specialized hardware.

After the recent "Tesla Autonomy Day" video ▶ this video (almost 4h long, but worth a watch), I do rather feel like the incumbent services are about to be in a spot of bother. I think there's a bubble in this sector, and it's going to pop.

Oh hey I get to copy paste a list of times elon said FSD would come out next year![1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16] [1] link [2] link [3] link ... [4] link ... [5] link [6] link ... [7] link [8] link [9] link ... [10] link [11] ▶ this video [12] link ... [13] link [14] link ... [15] link ... [16] link (Or simply, link )

Sources: 1 link 2 link 3 link 4 link 5 link 6 link 7 link 8 link 9 link 10 link 11 link 12 link 13 link 14 link 15 link 16 SXSW 2018 17 link 18 link 19 link 20 link 21 ▶ this video 22 link 23 link 24 link 25 link 26 link 27 link 28 link 29 link 30 link 31 link 32 link 33 link 34 link 35 link 36 link 37 link 38 link 39 link 40 link 41 link

You don't understand. Next year, for sure, Tesla will have over 1 million rob taxis on the road. [1] [1] ▶ this video

Yes they have. Tesla Autonomy Investor Day 2019 ▶ this video At 3:13:41. Robotaxis is clearly written on that slide. Elon Musk is talking about robotaxis for minutes as well during the official presentation to investors.

Tesla Autonomy Day 2019-04-22: "By the middle of next year [2020] we'll have over a million Tesla cars on the road with FSD hardware feature complete at a reliably level that we would consider that no one needs to pay attention. Meaning you could go to sleep. From our standpoint, if you fast-forward a year, maybe a year and 3 months, but next year for sure we will have over a million Robotaxis on the road. The fleet wakes up with an over-the-air update. That's all it takes." Source: ▶ this video

Thanks for the question and the papers. Like some of those companies, we are believers in the data centric approach to ML. But labeling is not our focus (unlike Snorkel, HumanLoop, Prodigy or Cord.Tech). We focus on model diagnosis and mining the best data to improve them. So there are more similarities with Aquarium or Lightly. There is a great talk from Tesla [1] on what they call the "Data Engine" (that probably inspired some of us :)). One of the things we took from it was that in order to truly close the loop on the ML data flywheel, we needed to turn production into a reliable datasource. It had to become accessible, understandable and minable. To achieve this we took the approach of combining ML observability with Active Learning mining frameworks. Combining both is important in our view because Observability tells how the model behaves in the real world and Active Learning finds the right samples to fix / improve the model on real world data. They go hand in hand. Technically, it means that we integrate with serving and labeling platforms. We ingest data both in streaming and batch. We can mine on production streams including on device (for iot use-cases where accessing data is a challenge). We have an extensive set of metrics to understand model behavior in the wild and solve use cases like data drift (detecting it, triggering mining and sending the data for labeling/retraining). And we are geared toward automation. Regarding which data sampling works well and which

I don't believe it's a critical component of Autopilot. From the last public video release in 2019 [1], it seems like they use the stereo cameras to derive depth information, and use radar as a kind of ground-truth in informing the model. I would think the source-of-truth for what the Autopilot system thinks is the position of objects is the stereo camera, i.e. the radar is not part of the online flow, but is used offline to correct the model. See specifically at 2:20:22 [2]: > We use that radar to annotate what the vision is seeing, the bounding boxes that come out of the neural networks, so instead of human annotators telling you that this car, in this bounding box, is roughly 25 meters away, you can annotate that data much better using sensors. I take that to mean that the radar is used as part of supervised learning for updating the model offline, whereas just the cameras are used as input to the model. [1] ▶ this video [2] ▶ this video

> It's verbatim from Elon Musk on Autonomy Day 2019 No, that was NOT a verbatim quote. Here is the exact quote with me adding emphasis on the critical (but oft omitted) qualifier: "All Teslas being produced right now have this computer. We switched over from the Nvidia solution for S and X about a month ago, and we switched over the Model 3 about ten days ago. All cars being produced have the all the hardware necessary, compute and otherwise, for full self-driving. I'll say that again. All Tesla cars being produced right now have everything necessary for full self-driving. All you need to do is improve the software." ▶ this video > This is the same event where he promised a million Tesla robotaxis on the road by 2020. That is, in my opinion, not a contextually correct reading of what Elon said. Within the context of the speech it's clear (to me) that for him, "robotaxi" is used as shorthand for "vehicles with the hardware capable of being a robotaxi." Obviously I cannot prove this, and I recognise that responsibility ultimately lies with Elon to be clear. I do agree Elon did say multiple sentences that are absolutely misleading when read in isolation, but his actual words in context (and with some recognition that this is a forward looking statement made with optimism) were far less egregiously wrong than the snipped quote implies on its own. You can watch him say it in context and make up your own mind: ▶ this video

Same thing with Tesla cars: they are already remotely asking their AI in their cars to send back recordings that they are interested in by giving example photos. It's used for improving their self-driving and they say it's "anonymized", but it's a little scary thinking about how it could be used in the wrong hands (if there is a hack, or some agency forces them). For example asking for faces or to detect certain behaviors. It's not just stored data, but it's data ready for querying with powerful distributed AI in a self-driving remotely controllable car. ▶ this video