6th Sense Project by Pranav Mistry

1. SIXTH SENSE DEVICE
codename : WUV~Wear Ur World
”integrating information with the real world”

submitted by :

1

2. About…
The sixth sense is a wearable gestural interface
device that augments the physical world with
digital information and lets people use natural
hand gestures to interact with that information.
It was developed by Pranav Mistry,a PhD
student in the fluid interfaces group at the MIT
media lab.
2

3. 3

4. The basics…
Sixth Sense allows people to use the internet without a
screen or a keyboard.
It can turn any surface into a touch screen.
It is made with store-bought components.
“The software program processes the video stream data
captured by the camera and tracks the locations of the
colored markers [that you wear on your fingers] using
simple computer-vision techniques”.
According to a number of sources, graduate students at
MIT have been working on this for the past four months
“day and night”.
4

5. Just a few things it can do…
“The device recognizes items in stores, and then gathers
and projects information about products to the user. It
also is able to provide quick signals to let users know
what the best products are based on their needs.”
“Another amazing feature is the device’s ability to look at
an airplane ticket and let the user know whether the flight
is delayed or on time. It is also able to recognize books
(based on an image or RFID) in a book store and then
project reader reviews or author information from the
Internet onto blank pages.”
“The device can also recognize articles in newspapers,
retrieve the latest related stories or video from the
Internet and then display them on pages for the user. Say
goodbye to your dictionary and encyclopedia.”
5

6. Components & Principles
The sixth sense prototype is composed of a
pocket projector,a mirror and a camera.The hardware
components are coupled in a pendant like mobile
wearable device.Both the projector and the camera are
connected to the mobile computing device in the user’s
pocket working such as :
• The projector: projects visual information, enabling
surfaces, walls and physical objects around the wearer to
be used as interfaces;
•The camera and hands: recognizes and tracks the user's
hand gestures and physical objects using computer-
vision based techniques.
6

7. Components
7

8. Components & Principles
The software program: processes the video
stream data captured by the camera and tracks
the locations of the colored markers (visual
tracking of colored fingers) at the tip of the user’s
fingers using simple computer-vision techniques.
The movements and arrangements of these
fiducials are interpreted into gestures that act as
interaction instructions for the projected
application interfaces.
The maximum number of tracked fingers is only
constrained by the number of unique fiducials,
thus SixthSense also supports multi-touch and
multi-user interaction.
8

9. Interest
The SixthSense prototype implements several
applications that demonstrate the usefulness,
viability and flexibility of the system :
• The map application lets the user navigate a
map displayed on a nearby surface using hand
gestures, similar to gestures supported by multi-
touch based systems, letting the user zoom in,
zoom out or pan using intuitive hand movements.
• The drawing application lets the user draw on
any surface by tracking the fingertip movements
of the user’s index finger.
9

10. Interest
SixthSense also recognizes user’s freehand
gestures (postures). For example, it implements
a gestural camera that takes photos of the scene
the user is looking at by detecting the ‘framing’
gesture. The user can stop by any surface or wall
and flick through the photos he/she has taken.
SixthSense also lets the user draw icons or
symbols in the air using the movement of the
index finger and recognizes those symbols as
interaction instructions. For example, drawing a
magnifying glass symbol takes the user to the
map application or drawing an ‘@’ symbol lets
the user check his mail.
10

11. Interest
The SixthSense system also augments physical
objects the user is interacting with by projecting
more information about these objects projected
on them. For example, a newspaper can show
live video news or dynamic information can be
provided on a regular piece of paper.
The gesture of drawing a circle on the user’s
wrist projects an analog watch.
11

12. Here the video shows some of the
applications of this device.

12

13. Educational Use
Engagement
Research
Collaboration
Cost-Effectiveness
13

14. Engagement
Teachers may find that Sixth Sense, because of
its unique ability to work with many of Gardner’s
multiple intelligences, will be a catalyst for
students who might otherwise remain unengaged
in learning.
Sixth Sense uses the visual, kinesthetic,
intrapersonal and interpersonal among others.
If this technology does nothing else, the idea that
students can simply use their hands to draw
images on walls and take pictures with their
hands will be (at least initially) quite motivating
14

15. Research
Sixth Sense will give new meaning to
“looking something up.”
Since we are essentially talking about the
internet, students will be given the
opportunity to portably research the world.
They can all go on a virtual field trip and
project the images on their own desks, for
example.
15

16. Collaboration
Students will be given an opportunity through
this technology to feel more connected to others.
Although there does seem to be the sense that
individuals could walk through life wired and
lonely, I like to think the classroom would serve
as the environment in which students and
teachers using Sixth Sense would want to share.
Not only could the sharing take place in the
classroom, I suppose since the users are
connected online, sharing could take place
around the world.
16

17. Cost-effectiveness
As mentioned earlier, this product currently costs
only $350. From an educator’s stand point, this is
minimum given the amount of money spent on
textbooks, computer labs and LCD projectors.
For educator’s, Sixth Sense would be a way to
put technology (literally) into each student’s
hands, because of its size and its cost.
17

18. Advantages of sixth sense
Portable
Inexpensive
“Cool”
Multi-sensory
Connectedness between the world and
information
Critical thinking skills would be somewhat vital
here…allowing teachers a fantastic opportunity
for interdisciplinary study with critical thinking
components
18

19. 19

20. Anticipated Time Frame
According to a number of sources, the MIT
students already have a patent on Sixth Sense.
Still it is unclear as to when this device might
actually become available for general
purchasing.
Microsoft Surface does currently exist (which is
similar in theory). But Microsoft’s product is not
portable like Sixth Sense
Even if it came out next week, as educators we
may need to wait some time in order to learn of
the dangers that are sure to exist with the
product.
20

21. Cost of sixth sense
• $350 dollars
•The interesting thing here is that Sixth Sense can
never really be more expensive than its individual
components .
• Another interesting thing is the inventor of sixth
sense,Mr. Pranav Mistry has declared to make it
open source.
21