Panoptic is an immersive augmented reality and an artificial intelligence project in which you see and hear your current city while you are in another location through an immersive helmet.

In the Panoptic System, we aimed to enhance human cognition and perception using artificial intelligence through an immersive augmented reality device. Another goal is to augment human sight and hearing by knowing and understanding our surroundings. We contributed two forms, a model for “diminished” reality and another for “augmented” reality. The multimodal experience includes augmented sounds and images that are generated by a semantic segmentation of your visual field. The segmentation produces overlaid sonic textures and superimposed visuals, modeling a newly perceived reality. Moreover, the new reality is created by altering your visual field by modified the colors that they might unconsciously suggest to another color, for example, if you see a gray sky, the Panoptic System can transform it in real-time to blue.

The Panoptic System aims to enhance human cognition and perception by using artificial intelligence through an immersive augmented reality device. Another goal is to augment your vision by knowing and understanding your surroundings. By using a Deep Learning model, the user’s view is a “diminished” or an “augmented” reality in which it is possible to manipulate and control in real-time reality. In addition, the purpose is to offer a helpful, immersive, subjective — rather than a detached, observational – experience.

In Panoptic, we propose an immersive AR experience, where it is possible to imagine that you were in another city—in this case, Mexico City. The Panoptic system uses the fact that perceived colors and sounds can convey an emotional stimulus and can unconsciously change one’s behavior. Moreover, in the system, we implemented an artificial intelligence (AI) model that can semantic segment different parts of the user’s field-of-view. The segmentation is used to alter the completely or partially the perceived user’s field-of-view.

The immersive helmet includes the following components:

• 4k Camera
• VR Headset
• 2x Ominidirectional Microphones
• 4 Capacitive touch zones

Renders of different possible Panoptic Helmets

Reality based on audio layers.