Another Leap!

We have the opportunity of trialing an Installation of Great Trails at a local Tourist Information Centre (TIC), this is a great chance to gain user feedback on the Installation Concept. With the Covid-19 situation still a major concern, the TIC was keen to work with a touchless interface which led us to revisit the work we did with the Leap Motion hand tracking late last year (see blog post here).

This early work showed the potential for the Leap Motion to provide touchless interaction but was very much a proof of concept, being somewhat tricky to use, and did not cover the full extent of the application, being really restricted to the Overview level. Thus we have been very busy over the last month or so working on refining the concept.

Fortunately, the increasing demand for touchless interfaces has led to some development of the technology and there is now a simple way of using the hand tracking to mimic a mouse, or touch screen press. This means that it is fairly simple to adapt a conventional interface to work with the Leap Motion.

Essentially the leap motion recognises an invisible plane in front of the screen. When the users hand is in range of the device, but in front of the plane, a solid circle appears on the screen (analogous to a mouse pointer). When the hand approaches the plane another circle appears around the solid circle which shrinks as the plane is approached. When the hand passes through the plane the solid circle shrinks to a dot to indicate that a mouse press event has been recognised, at no point does the screen need to be touched. Both mouse clicks and click - drag events can be registered, this is important to use drop down menus and the joysticks used in the Explore and Experience levels.

Leap Motion conversion of hand tracking to Mouse Event

With some fairly basic modifications to the interfaces for the various levels, we have arrived at a functional solution. For the Overview level we have used a combination of the Desktop and Mobile interfaces, repositioning the buttons and also making them larger. The 43” monitor we are using is a little large for the device and we have moved the buttons in from the edge of the screen to improve usability.

For the Explore and Experience levels we have used a modification of the Touch Interface. As multi-touch is not currently implemented we have removed the jump button and made the sprint button simply on/off rather than something that must be held. This has simplified the operation and seems to work well. The interface is presented automatically when the level loads and disappears after 30 seconds of inactivity, to allow the view to be enjoyed without the joysticks in view, reappearing again when a press gesture is detected.

Touchless Interfaces for Overview and Explore Levels

We have also been thinking about the design of the stand which will support the monitor and the Leap Motion in the TIC. The positioning of the monitor (height and angle) and the position of the Leap Motion below and in front of the monitor is quite critical. In order to test options we built a test rig which allowed a range of settings to be assessed these have now been fed into the concept design of the stand.

Test rig and stand design.

We look forward to progressing this with the TIC and gaining user feedback on the concept and usability.