Leap 3D Motion Control System

[Insectoid]

http://www.theverge.com/2012/5/21/3033634/leap-3d-motion-control-system-video

I found this on reddit. Looks neat. I want one.
The general consensus seems to be that there are 2 major issues with this:

-Developer adoption
-Arms getting tired

Most experimental touch/spacial interfaces suffer these faults, so it's reasonable to assume Leap will fail the same way. However, I do not believe this to be the case.

Concerning developer adoption:

This interface has the convenient benefit of near zero need for developer adoption. Why? Because it isn't a new interface, it's just a new shape for the mouse. Unless the founder is an idiot, this thing is gonna ship with a mouse driver. Plug it in, install, bam. Instant support for every mouse-based application. And since you only use the mouse with one hand, the other is free to use the keyboard. Older interface 'solutions' ditched the keyboard, but Leap complements it. Likely Leap's driver will contain a comprehensive gesture library so the end user can map different gestures to different keys, buttons & macros. With Kinect, we saw the issues with adoption. There were initially very few games and no good ones, so a lot of players waited to buy it. Because there were so few players, developers didn't bother making many games. In the case of Leap, nearly every game you own is already compatible. Developers don't even have to be aware that Leap exists, because their programs are compatible weather they intended it or not. Devs will, if anything, be happy to have more 'input bandwidth' than a standard 2-button mouse & keyboard, and to have the option to include Leap-specific support. I imagine menu interfaces could be heavily optimized for Leap, the same way they're pessimizing console menus.

As for arms getting tired, this is also a non-factor.

The article mentions adjustable sensitivity. Wave your arms around all you want, but I'm gonna rest my wrist on my desk and lazily twitch my fingers. Wouldn't surprise me to find people discussing and sharing their gesture bindings to optimize the ergonomics.
This project has huge potential. It's cool, so people are interested in it. it doesn't require a massive OS re-design. It isn't prohibitively expensive. It doesn't take anything away from the user. It's already compatible with all keyboard/mouse applications for Windows and OS X. 10/GUI, on the other hand, was destined to fail. It required expensive hardware. It required a completely new interface paradigm totally incompatible with existing systems. It took users out of their comfort zones. Leap however is wondrously designed to succeed.

What do you guys think? Plausible? Garbage? The way of the future?

[mirhagk]

The physical gestures are the weakest part of kinect in my (and many people I know) opinion(s). The coolest part about kinect is the fact that it can filter out just your voice anywhere in a room (which you will realize is amazing if you've ever used a desktop mic), and it can respond to voice commands (quite easily).

Other than the occasional running around being an idiot with family playing Kinect sports, I wouldn't use the motion gestures for games, no matter how precise they were. A physical keyboard/mouse or controller is always more precise, and provides tactile feedback of whether it worked or not. (It's like comparing a touchscreen keyboard to a mechanical keyboard). The thing I use in games is the voice commands (fus ro DAH!) which can actually save so much time because of the number of commands that are possible (for instance in skyrim you can say "equip" and then a great number of types of items and it will equip it for you).

And in the non-game areas the gestures get even more pointless. If I'm too lazy to reach over for the controller, I'm too lazy to wave to the screen, then do a preset motion. The "xbox pause" is something I actually use because it requires very little time and effort (although it still takes a second to happen, which is an area they could improve).

What I think of LEAP is that they took the worst part of the Kinect, and improved that. It's like improving the touchscreen keyboard for a laptop. No matter how good it gets, I'm only going to use it as long as it's new and exciting (ie five seconds).

[Insectoid]

It's the convenience of not reaching for your mouse. Of not having to move the curser to the button, 'cause you can just point at it. The mouse is slow and even you agree with that. It's our big bottleneck and it needs to be replaced. This is a strong contender.

I can rest my wrist on my laptop case and point anywhere on the screen without more than my finger. I don't even have to take my hand off the keyboard (which I do to reach my mouse).

Imagine using this a lot for web browsers. Scroll with a flick of the thumb would be far, far faster than a mouse wheel or trackpad. Switch tabs with a wave of your hand. The entire keyboard argument is moot, because this does not replace the keyboard (it could, but that would be dumb). Because my wrist is on the laptop case, a lightly vibrating chip built into the computer could provide tactile feedback (the Macbook layout has lots of wristroom). If anything, this would remove the need for a trackpad to free up a ton of space.


You absolutely would not use this like kinect. This is for computer use. It has a 4 square foot 'stage'. Your fingers would be mere inches from the screen. In FPS-style games, I imagine using joystick-like gestures (as demonstrated here) as opposed to pointing at what you're aiming at.

I disagree about voice recognition, but only applied to Leap. It's cool on Kinect when you can't reach the controller but otherwise, speaking is slow and solves no problems and has no place in Leap.

Quote:

I'm too lazy to wave to the screen, then do a preset motion.

Again, this isn't Kinect. There's no waving. Ideally, it would be as natural as Apple's trackpad gestures.


Continuing your Skyrim example, I think it would be badass to actually shout the...well, shouts but I doubt I'd often use it.
Binding each menu to a gesture would put every menu in instant access (the default is awful and it takes like 5 clicks to switch items). For example, you could define a 2-finger point gesture at the bottom left corner of the screen that opens the favorites menu. You'd have to take your fingers off the WASD keys, but by Skyrim's own design, opening the favorites menu pauses the game so it doesn't matter. Put your fingers back, the menu closes, and the game resumes (btw, I got to level 20 before I even knew this menu existed- that's how bad Skyrim's UI is).

You could swing a melee weapon by mimicking a sword with your index finger, or you could just tap your thumb on your hand. You could draw a bow by pinching your thumb and forefinger and 'drawing it back'...like a bow. Cast spells by mimicking the in-game animation. You could make it as involved or basic as you want.

Dual-wielding is harder to solve I admit, but that's a skyrim-specific problem (how many games dual-wield where right&left clicks control left&right arms?).


Kinect's gestures are really imprecise, which limits the number of available gestures. Leap is extremely accurate, so there's theoretically as many possible gestures as you can reliably and distinctively reproduce with your hand.

At any rate, for $70 USD, I'd gladly buy one, if only so I can experiment and write applications for it. There's no risk in the buy, like there was with the HP Touchpad. Hell, it's the same price as Apple's mouse, and I know which I'd rather have.

[mirhagk]

The remark about having to wave to the screen still stands. How does the system know when I am moving my fingers to control it, or moving my fingers because I'm fidgeting or reaching for a paper or something. The reason kinect requires a hand wave is to tell the device to listen. I don't see a good solution to this problem in LEAP, and without an action that you are unlikely to do by accident, you will accidentally do all sorts of things you didn't mean to.

One important thing to note is that without any solid devices for input, you might not know where exactly is ground 0 for things. For instance with the thumbsticks, where exactly should your thumbs be to not move? Granted gestures don't need a ground 0, but non-gesture input needs a reference location (and imagine having to keep moving you thumb forward to just walk forward, it would be hugely annoying)

[Insectoid]

Moving your thumbs forward? I was talking about a joystick, not a thumb nub. Ground zero doesn't matter. It's the orientation and movement of your hand that defines the gesture, not the location. The stage is easily big enough to not accidentally leave it. There's certainly no 'ground zero' on my trackpad (I know some have a scroll bar on them). You get a 2D space for 2D gestures. This gives you a 3D space for 3D gestures.

This thing tracks your whole hand. It can create a wireframe model of your hand in real time. You can use very specific gestures that you accurately recreate every time. If you're worried about fidgeting, just make sure none of your gestures are fidget-ish.

I suppose I'll wait until I have one to make final judgement, but I'm optimistic.

[mirhagk]

What is the proposed system for walking? Because games take the route of either holding down a key, or holding a thumbstick (or joystick) in a direction. Neither of those are gestures, and as I said a gesture based system for something constant like walking would suck.

By the way your trackpad does have a reference point, wherever you first placed your finger. Trackpads, mice and scrollbars all have the advantage of removing your finger from the device to stop recording your finger's location. Basically the gestures would have to be distinct from moving your hand back to a comfortable position, which might be hard (not impossible) to do for movement actions in 3D space (even 2D).

You also have to remember that the sensor can be as precise as you want, doesn't mean the user will be (especially without something physical to know their position). The sensor could easily tell whether the user's finger is pointed in the direction of the close button or the maximize button, but the user probably can't. This is beneficial in some applications (for instance a game where fine movement control is the ) but horrible in others. That means for specialized applications this is awesome, but is not the best for general use.

[jr5000pwp]

Personally, I'd rather have a mouse/keyboard/gamepad because of the ease of using them, the amount of relaxation you can have, and the minimal amount of muscular movements they require. That being said, something such as the leap is a neat device, and is amazing, until your arm gets sore.

I would buy these new products such as leap, but I am not going to replace my trusty mouse and keyboard any time in the near future.

[Insectoid]

Quote:

What is the proposed system for walking?

Hold down W? Like we've done for decades? I've said (more than once I think) that the keyboard isn't replaced.

Quote:

Basically the gestures would have to be distinct from moving your hand back to a comfortable position

Hence 2-finger waves, and motions you simply never would do otherwise.

Quote:

You also have to remember that the sensor can be as precise as you want, doesn't mean the user will be (especially without something physical to know their position).

Trivialized with a crosshair, then the user will get used to it. Do you think we were good with mice naturally? Hell no, we've just used it so long we're well practised. I imagine mastering common gestures in a few hours at worst.

[mirhagk]

I've noticed from using gesture technology (gesture touch pads [apple or otherwise], android's swype, kinect's gestures) that people's performance with it gets worse, not better. This obviously isn't a tested fact, and you can't go off of ad hoc observations, but it makes sense that once the "coolnes" of the technology wears off, you become less focused, and more lazy.

LEAP could be cool, but it might at most replace the mouse, and for $60 instead of $5 (unless your dumb and buy apple products, or a crazy gamer and buy gamer mice, which would be much more precise in gaming scenarios than LEAP) it isn't likely to be a revolution.

It's neat technology, just not something new, and not revolutionary.

[Insectoid]

Quote:

It's neat technology, just not something new, and not revolutionary.

Well, it's the cheapest and most accurate consumer motion-control system ever created. That's revolutionary if I do say so myself.

Quote:

I've noticed from using gesture technology (gesture touch pads [apple or otherwise], android's swype, kinect's gestures) that people's performance with it gets worse, not better.

Bullshit. Makes sense for xbox, where the gestures really are a gimmick because they suck. Everyone I know who owns a modern mac uses gestures all the time. Every. Single. Person. Maybe not all of them (there's a lot), but the common ones. Scrolling, zooming, rotating, window switching, etc.

Quote:

LEAP could be cool, but it might at most replace the mouse, and for $60 instead of $5 (unless your dumb and buy apple products, or a crazy gamer and buy gamer mice, which would be much more precise in gaming scenarios than LEAP) it isn't likely to be a revolution.

Do you think the first, crappy commercial trackball mouse costed $5? The first optical mouse? Hell no. And have you seen the magic mouse? It's a trackpad and mouse in one. You can use Apple gestures on it. I'm not personally a fan of it, but it's certainly worth the cost if a trackpad/mouse is what you want.

This is the first-gen edition. First-gen is (almost) always expensive. It's always got a few issues. The price will certainly drop in a few years, and like I said, $60 isn't prohibitively expensive. Enough people can afford it that it's viable *right now*.

[mirhagk]

You have a very accepting definition of revolutionary. Making something cheaper and better is simply natural progression, and something that will happen eventually with any decent technology. Whether there would've been a kinect 2.0 is debatable because the success of Kinect wasn't that awesome, and the interest in motion controls is dying off because it's no longer new and cool (although it wasn't when kinect came out either, it was just new to most people because previous attempts didn't advertise well enough).

And your right insectoid, this is the first gen, but whether there will actually be a 2nd gen, or just another company to jump on board with a better version is the question (history with motion controls has shown us the latter is more likely). LEAP is a great new competitor, but they aren't exactly breaking new ground here, just kinda digging deeper in a hole that other companies have abandoned (Sony abandoned it years ago, Microsoft's interest is starting to die off, although it's still strong).

[Insectoid]

You seem to be of the opinion that motion control will never be good enough to use day-to-day, in which case debating this is pointless.

'Revolutionary' literally means 'to cause a revolution'. The mouse has been relatively unchanged since it was first created- if this replaced it on a large scale, that would be revolutionary. Might wanna pick up a dictionary before you tell me I don't know the words I'm using.

[mirhagk]

lol insectoid, I was accepting your definition of revolution, just pointing out that your accepting a lot as revolutionary. If your really don't understand that revolution can be entirely subjective look at the definition of revolution: "a sudden, complete or marked change in something". Marked change? Sudden? both of those are subjective words, in the vast expanse of time, anything that happens within a lifetime is sudden. With technology however sudden usually tends to be a much quicker time frame (maybe a year or two), also marked change is subjective. Technically every piece of technology causes a change, so it's subjective whether or not something actually causes a significant change.

now that we're done this useless argument about what the word revolution means, and have pulled the obligatory dictionaries out (always a sign the discussion has been reduced to primitive insults, name-calling and trolling), can we get back to actual discussions?

[Insectoid]

The mouse has been unanimously used in every corner of the world as the pointing device on computers since it was created. To date, nothing has significantly altered the design of the mouse, since its invention 50 years ago.

I argue that Leap has the potential to replace the mouse in a span of years. If the device is hyped up before release and has a low enough price tag, it will see massive adoption within weeks. Even 2 years is a very quick time to replace a 50-year monopoly on curser control, let alone weeks.

The scope of the change is also significant. This could affect every person that currently uses a mouse.

This has the potential to end a 50-year global monopoly in weeks or months. If you don't consider that revolutionary, could you give an example of what is?

Quote:

now that we're done this useless argument about what the word revolution means, and have pulled the obligatory dictionaries out...

You 'pulled the dictionaries out' by challenging my definition of 'revolutionary'. I summarily explained why I was correct in my usage.

Quote:

...always a sign the discussion has been reduced to primitive insults, name-calling and trolling

I don't see any primitive insults or name-calling anywhere in this thread by any party involved. Only proper debate. However, you don't seem to be reading any answers, because a lot of your arguments are repeated as if I'd never answered in the first place, so if I seem hostile, that's why.

[mirhagk]

Insectoid @ Thu May 24, 2012 12:20 pm wrote:

If the device is hyped up before release and has a low enough price tag, it will see massive adoption within weeks.

Wow that is quite optimistic considering they are about 6 months from the release date and the majority of even tech oriented people have yet to even hear about it. It is also quite optimistic considering the fact that most people typically haven't jumped at the seams to grab new technology. The only company able to push out an entirely new concept to widespread adoption even with years has been Apple (which have taken ideas, improved them a bit, marketed like crazy, and got widespread adoption. Now the only marketing they need is making it white, and adding an apple logo, but I digress).

1 or 2 years to get semi-widespread adoption (as in most people know what it is, and everyone knows someone who uses it) is a realistic goal if this kicks off.

Insectoid @ Thu May 24, 2012 12:20 pm wrote:

However, you don't seem to be reading any answers, because a lot of your arguments are repeated as if I'd never answered in the first place, so if I seem hostile, that's why.

My points have been similiar, just as your counters have been similar. The fact is that I am not optimistic about a technology that is already in use, and has been in use for years and years to suddenly spark everyone's interest just because they have made it more accurate and cheaper. Yes this will be successful, but I doubt it will be much more successful than previous attempts (Do you know how many PDA's and tablets were released before they became widespread? Do you know how many voice control systems were released before they became known to most people (and most people still don't even use them, despite the years of work in making them as good as they can, because it's not perfect, and it takes just as long to type it out as it does to say it for most people).

Don't get me wrong, this company might make some good money, and might even get a great product out there, but it's only a matter of time until someone else takes the idea, builds it into their computer more tightly, colours it white, and puts a picture of an apple on it. (<-- just a joke, but realistically another company will likely either buy it up, or produce their own within a year)