Using Python to Code by Voice

> The second thing we need to figure out is how we can compress voice input to make it faster to transmit. What’s the voice equivalent of a thumbs-up or a keyboard shortcut? Can I prompt Claude faster with simple sounds and whistles? This reminds me of the amazing 2013 video of Travis Rudd coding python by voice: ▶ this video The number of times in the last few years I've wanted that level of "verbal hotkeys"... The latencies of many coding llms are still a little bit too low to allow for my ideal level of flow (though admittedly I haven't tried one's hosted on services like groq), but I can clearly envision a time when I'm issuing tight commands to a coder model that's chatting with me and watching my program evolve on screen in real time. On a somewhat related note to conversational interfaces, the other day I wanted to study some first aid stuff - used Gemini to read the whole textbook and generate Anki flash cards, then copied and pasted the flashcards directly into chat GPT voice mode and had it quiz me. That was probably the most miraculous experience of voice interface I've had in a long time - I could do chores while being constantly quizzed on what I wanted to learn, and anytime I had a question or comment I could just ask it to explain or expound on a term or tangent.

Maybe lisp could shine here? While writing lisp code in classic editors, you already shouldn't be editing or navigating code with classic commands, instead you will be using structured editing. Of course it isn't limited to lisp, but languages with C-like syntax are harder to use for structured editing. I highly recommend watching this video, where the speaker uses voice commands to write both in emacs-lisp and in python. ▶ this video

I'm surprised that there seem to be no examples anywhere of anyone seriously using chorded typing for programming. After cheap chorded keyboards entered the market and the associated software became free, I assumed it would only be a matter of time before something like Travis Rudd's incredible demo of voice coding [1] came out. Perhaps that it hasn't speaks to how difficult it is to get real speed improvements with a chorded setup? [1] ▶ this video

A similar project using Emacs and really clever use of Python and Dragon Naturally Speaking SDK was my favorite presentation at PyCon this past year. ▶ this video

Are you thinking perhaps of this [1] PyCon talk by Tavis Rudd, who had RSI and a rotator cuff injury? He demos a voice command system that's all short nonsense syllables, including with a few humorous video clips interspersed, and then explains it after the demo. [1] ▶ this video (demo starts 9 minutes in)

Tavis Rudd's Using Python to Code by Voice [0] is also interesting. Looking up that URL, I came across the more recent Coding by Voice with Dragonfly [1]. Haven't seen that one yet, but just quickly skimmed through. Unfortunately the live demos are silent (you can't hear speaker Boudewijn Aasman talk to the computer), but it seems worth checking out if you're interested in setting up some recent voice recognition software with your own custom voice command syntax. [0] ▶ this video [1] link

Your comment reminded me of an interesting video I saw a while back. The video is a talk [0] by a programmer explaining and demonstrating how he developed and used a voice recognition system to program in emacs, after a significant RSI injury. The commands and syntax he uses to interact with the system might be a decent approximation of the custom English syntax you suggest. [0]: ▶ this video

See this: ▶ this video > In a fast-paced live demo, I will create a small system using Python, plus a few other languages for good measure, and deploy it without touching the keyboard. The demo gods will make a scheduled appearance. I hope to convince you that voice recognition is no longer a crutch for the disabled or limited to plain prose. It's now a highly effective tool that could benefit all programmers. Turns out it's doable, you just need to invent your own language.

IMHO I don't think it would fly. The most difficult things of programming are too abstract for that. You may be interested in the guy that did all his coding with voice for a while because he couldn't use his hands, and still uses it to this day about half of the time. ▶ this video

[This talk by Tavis Rudd]( ▶ this video ) about a system he used when his hands were afflicted with RSI is pretty interesting. Not entirely a no-hands approach, but I can't seem to remember the developer who made his first app in the hospital by typing with one/two fingers. He wrote a blogpost about his process that was really inspiring.

There was an interesting talk at PyCon 2013 about programming using only your voice: ▶ this video Might be worth looking into for your case. It's worth mentioning that the guy did this for himself due to severe RSI in the hands, but there was another talk on using a stenographic keyboard to program and I remember this guy stood up and said he wished he'd known about it earlier as it's another approach for people who suffer from RSI since there's less typing.

You might be interested in watching this talk: ▶ this video Although, if I recall correctly, he uses Dragon Naturally Speaking, which is a commercial product.