[nfbcs] FW: [program-l] physically interactive augmentation project

[Rasmussen, Lloyd]

Another interesting project from Yuma Decaux and company.

Lloyd Rasmussen, Senior Staff Engineer
National Library Service for the Blind and Physically Handicapped, Library of Congress
Washington, DC 20542   202-707-0535
http://www.loc.gov/nls/
The preceding opinions are my own and do not necessarily reflect those of the Library of Congress, NLS.

From: program-l-bounce at freelists.org [mailto:program-l-bounce at freelists.org] On Behalf Of Yuma Decaux
Sent: Saturday, July 29, 2017 9:03 AM
To: Blind programmer mailing list
Subject: [program-l] physically interactive augmentation project

Hi All,

I'd like to call your attention to a project we have started which allows us blind users to interact physically with objects in a confined 50 by 50 by 50 cm cube volume.

It uses the leap motion controller to identify fingers and hands in its line of sight, and the camera is used to retrace the objects which are within those bounds to make a mirroring between the physical space and the virtual world.

To put it briefly, imagine you have a map, with buildings, roads, parcs etc, printed in 3D. The same objects exist within the application and wherever your finger touches an object, the application knows which one it is, and you can pull up extra information, interact via sound, navigate through the environment as if you were an eye and pair of ears in the sky, and get more information about what you are touching. This can be useful for the following cases:
-maps
-matrices
-diagrams
-chess board or other game board
-interactive action/adventure/RTS game
-chemistry or physics diagrams

The list goes on.

Here is the blog entry, with the repository and a video giving a quick and dirty of the project.

https://www.oseyeris.com/post/blind-math-alfaeditor


Our next step is to print a 3D topology, or game arena, and place soldiers in the 3D virtual world, where you can hear them at rest, camping etc, and you tell them to move into another area of the 3D printed map. You just have to use your finger around the volume to know where your soldiers are going, and will hear swords clashing etc when you have encountered an ennemy squad, or you can spy on them by moving your finger around the map to explore and discover new things.


As this is open source, we would love it if you can add some features we may not have thought of, and I'm available for any clarifications on some of the classes, as I have not documented the project yet, which I will do sooner rather than later.

Obviously, one needs a leap motion to itneract with the space, but at 50 bucks, it's well worth the fun and testing.

Have a great weekend