Applied Sciences Group: High Performance Touch

I'm taking issue with latency in general and it's dismissal. There are reams of ergonomic targets, data, and guides on user interface latency that are just completely ignored these days. A terminal is close to the simplest possible interface whose job is to provide input and output to a user with as little disruption as possible. 30 ms is in fact noticeable. Here is a short video from Microsoft (about 10 years old now) on the experience of using a touchscreen with 100ms of delay vs 1ms showing the steps in between. The difference is extraordinary for the user even if the end result is the same. ▶ this video

It can be very detectable, I'd say. Try watching this: ▶ this video It shows the difference between from >100ms to ~1ms. It's mind baffling.

Yes there is. Humans are normally not very responsive to the time between when an action is started and when it is carried out on the screen, but this changes for actions like drawing and dragging. This is the reason for the new Pro Motion behavior on the recently released iPad Pro models. For more information, this video [1] is a good place to start. [1] ▶ this video

The F1166 standard permits up to 250ms latency for touch screen response. That is ridiculously high. It practically guarantees people will think it's not working and tap again, undoing any toggle function they might have activated. They won't necessarily notice their mistake. I once accidentally ran a washing machine at 90C temperature because of high-latency touch controls. I instinctively double-pressed the touch surface to reduce temperature because I thought the first touch hadn't registered, causing it to wrap around back to maximum. It did some minor damage to the clothes before I noticed. And that was with more like 200ms latency, not 250ms. Latency on touchscreens is more important than with physical controls because there's no tactile feedback. Even 10ms is obviously imperfect for drag controls. Microsoft Research published a demonstration video: ▶ this video

I would like to try a 1 kHz display. I can definitely tell the difference between 120 and 240 Hz for mouse pointer motion. The motion blur tests mentioned in the article make it obvious as well. And Microsoft research has shown that when using direct touch interaction people can perceive latency all the way down to 1 ms. ▶ this video Also an underrated feature of high frame rate is that it's a universal fix for poorly written software with multiple frames of latency. An application with 10 frames of latency will be faster on a 1 kHz display than a perfectly coded application on a 60 Hz display.

The ability to use touch and drag gestures without any visible delay [1]. This dramatically improves usability as a drawing and painting tool for artists. [1] ▶ this video

> I seem to remember that a good pencil has lag on the order of 1ms (how long it takes the tip to settle out after high speed motion), but I don't remember where this number came from, so it's untrustworthy. Not sure what lag there is on actual pencil, but there's this somewhat famous video[0] from Microsoft Research, where they test various values of input lag on a touch screen, and demonstrate that even 10ms is noticeable - but 1ms is about enough for it to feel instant. With 1ms lag, drawing on a tablet seems to feel just like drawing on paper (I personally haven't had an opportunity to test a tablet with 1ms lag, so I can't tell). -- [0] - ▶ this video

You can easily observe how significant latency is in videos like this: ▶ this video (Microsoft Research presenting its ultra low latency displays for touch interactions). Many mobile games have you drag and drop things, so it's not like it's just first person shooters that suffer from latency. You're a lay person, you couldn't have known this, you're using words with very specific meaning to streaming (like latency) and you're comparing it to human reaction times, which are measuring something else entirely. You kind of reasoned about from a first principle in a very Paulgrahamarian way, and it led you deeply astray. That happens. And you're not the only person doing this, this is a comment section full of people who play games and parrot stuff they seen in YouTube, and don't have a concrete grasp of what it is they're even talking about, so it's understandable when it's laypeople shouting at laypeople that it's just a bunch of blah. One of the reasons I hate HN and write in throwaways nowadays is that the comments section is a better example of Knoll's law than actual journalism.

Oh thank goodness, so it's not just me who is driven insane by the mouse lag in browser input. And while I understand your cynicism, I don't think it's deliberate as much as it's complacency, given that it's latency issues all the way down when it comes to modern user interfaces. A literal decade ago Microsoft Research uploaded a video highlighting how much of an effect minimizing latency for touchscreens had on the user experience[0]. The vast majority of tablets still utterly suck in this regard a decade later, and I wonder if any manage to get below the 10ms barrier. And of course there is Dan Luu's famous 2017 article on computer latency in general[1]. [0] ▶ this video [1] link

I think you meant ▶ this video . It was indeed not too easy to find, due to SEO tactics polluting YT search with gaming-oriented stuff from recent times (where one may actually reach the frame rates necessary for such low latencies in a frame-by-frame display technology).

May be Speed isn't the right word, Latency would be better. We can look at Input Lag [1], and Microsoft Research's work [2] on Touch Input. Apple's ProMotion being part of that as well. For the past 20 years we have make 10 - 100x improvement in bandwidth at the expense of Latency. Now we need to do more work on it . Especially if we want VR or AR which are extremely latency sensitive. John Carmack [3] used to talk a lot about it when he was still working on Oculus. How it was faster sending something hundreds of miles away than showing it on your computer screen. [1] link [2] ▶ this video [3] link

You are probably thinking ▶ this video Bottleneck is in the input layer. While using a desktop computer look at the mouse cursor, now move the mouse - hardware accelerated 2D graphics, imperceptible latency. Low latency thru orthogonal multiplexing link