UH Mānoa’s Ambient Lab

Shaping the Future of Interactive Computing

As today’s fast-paced, high-tech world demands even more, technology is being pushed towards the seamless integration of computers into the surrounding environment through an approach known as Ambient Computing.

There was a time when computers took up an entire room, ran inefficiently and were not very user-friendly. Since then, computers have evolved into small, powerful, cost-efficient and easy to use tools that have changed our understanding of the world around us and the universe beyond. The computing power provided by the current generation of servers, laptops, tablets, smartphones and connected devices has transformed society into one that has become quite dependent on technology and the convenience it provides.

However, this convenience still requires a certain level of input from human operators to perform operations and complete tasks. Whether searching for a nearby restaurant on Google or adjusting a Nest thermostat to further cool the room, these actions must still be keyed in or programmed manually on a smartphone or smartwatch. As today’s fast-paced, high-tech world demands even more, technology is being pushed towards the seamless integration of computers into the surrounding environment through an approach known as Ambient Computing.

Ambient Computing technologies often perform computational tasks without a direct command from the user, resulting in interactions that can occur without the user’s knowledge. Amazon’s Alexa and Google Assistant are current examples of Ambient Computing devices that turn on lights, play songs or provide the weather report in response to spoken commands issued to the environment itself. As Ambient Computing progresses, technologies like motion tracking, gestures, voice recognition, artificial intelligence and others will be incorporated to achieve an increasingly seamless interplay of devices and services.

Ambient Computer Lab Projects

Web-controllable robotics platform with integrated media streaming for telepresence applications
Interactive Augmented Reality simulation sharing for medical training (JABSOM collaboration)
Interactive proximity data visualization for analyzing and tracking hospital acquired infections
Control system for robotic arms based on Brain Computer Interface and microcontrollers
Articulating projector lens mirror system with network-based control for advanced projection mapping
Augmented Reality platform for dynamic eLearning content (HoloLens and Mobile AR)

The Ambient Computing Laboratory at the University of Hawai’i at Mānoa’s College of Engineering develops advanced cyber-physical systems that combine interactive multimedia with the unique research possibilities offered by the billions of connected devices that are increasingly connected to the Internet. This evolution of the Internet, known as the Internet of Things (IoT), is well underway, with the current 20 billion connected devices projected to surpass 75 billion devices by the year 2025. 

Assistant Professor Darren Carlson, PhD

“Our students develop novel algorithms, frameworks and approaches for building IoT ‘Smart Spaces’ that will power next-generation computing environments,” said Electrical Engineering Assistant Professor Darren Carlson, the cyber-physical systems expert who founded and directs the Ambient Lab. “These environments are physical locations infused with sensors, embedded computers and intelligently coordinated network services that provide users with digitally augmented experiences that are responsive, adaptive and personalized.”

The Ambient Lab’s research focuses on several Hawai’i-specific needs, such as remote collaboration, distance education and connected healthcare. Together with UH System’s Academy for Creative Media (ACM System), the lab also helps foster UH’s emerging Computational Media study program, which trains students and working professionals in the areas of video game development, virtual and augmented reality, and digital storytelling and filmmaking.

“These new forms of digital content are causing seismic shifts in media consumption habits and consumer behavior worldwide,” said Carlson. “Incredibly, the video game and interactive media industries are estimated to surpass $300 billion in global revenue by 2025—dwarfing mainstream entertainment sectors, such as television, music, and film.”

Recognizing the strategic value of this ongoing digital transformation, ACM System’s Founder and Director, Chris Lee, has spent over fifteen years developing Hawai’i as a center of excellence in media. Lee, who previously served as president of production for both TriStar Pictures and Columbia Pictures, established ACM System in 2003 to promote integrated multimedia, arts, and technology education in Hawai’i as a catalyst for creating 21st century knowledge-based jobs across the islands. Lee has been instrumental raising the public and private capital needed to fund its activities, building initiatives, and attracting visionary researchers—like Carlson to UH in 2016.

“With the current industry shift toward interactive media, one of my goals is to fuse computer engineering with new forms of digital design and interactive entertainment,” said Lee. “Darren’s globally recognized expertise in cyber-physical systems coupled with his award-winning research track on interactive environments was an essential piece of the puzzle to help us achieve that goal.”  

Carlson’s appointment complements ACM System’s ongoing support of UH Professor Jason Leigh’s Laboratory for Advanced Visualization & Applications (LAVA) at UH Mānoa. Since 2014, LAVA has served as a dynamic maker space and innovation hub at UH, where researchers develop a wide range of big data visualization techniques for science, engineering and training. LAVA also forms the core of the Hawai’i Data Science Institute, which serves the data intensive science, engineering and training needs of UH System’s 10 campuses.

Carlson initially designed two innovative courses that incorporate engineering, computer science and design. His first course teaches students how to build smart, connected products using IoT technology and Cloud Computing. During the semester, students form “mini start-ups” and create prototype IoT systems that are presented during a simulated venture capital pitch event on the last day of class. His second course covers the advanced programming techniques used to build video games, virtual reality systems and interactive installations. This course supports UH degree programs and is also part of the UH’s upcoming Creative Computational Media Certificate, which brings together students and faculty from the Ambient Lab, LAVA, UHM Animation and even UH Mānoa’s Department of Theatre and Dance.

“The Ambient Lab’s core research area addresses the critical fragmentation challenges that threaten the IoT’s existence as an open ecosystem,” said Carlson. “Although connected devices and network connectivity are becoming ubiquitous, a historic lack of open protocol standards and fierce competition for market dominance is rapidly fragmenting the IoT at the application layer—creating a patchwork of isolated walled gardens with limited reach.”

Joseph McConnell demonstrates his team’s IoT “life remote” project

To unify wide-area IoT interactions, the Ambient Lab is developing an adaptive mobile computing framework, called Ambient Dynamix, which enables diverse connected devices and network services to work together across IoT ecosystem boundaries. Dynamix provides an open plug-and-play architecture that allows for the dynamic integration of new algorithms, connected device support and third-party software at runtime.

After installing Dynamix, mobile users can simply “walk up and use” almost any nearby connected device using their smartphone or tablet—regardless of its network, vendor or IoT ecosystem. As users change locations, Dynamix automatically discovers the available IoT devices and services nearby and then deploys the plug-ins needed to communicate with them over-the-air at runtime. As users interact with new physical environments, Dynamix automatically models the encountered networks, services and connected devices at scale, by transforming the growing community of Dynamix-enabled mobile clients into a distributed IoT crowd-sensing platform—aggregating the scanned results into the lab’s IoT search engine, which supports context-aware device discovery, plug-in based adaptation and service orchestration.

Building on these techniques, the Ambient Lab is also creating an advanced Smart Space framework that can be used to build a variety of digitally augmented physical environments using next-generation Edge Computing.  “Edge Computing refers to a paradigm in which the resources of a small data center are strategically placed at the edge of the Internet,” said Carlson. “By offloading processor and memory intensive tasks to powerful servers situated at the network edge, connected device applications enjoy fast response times, stable bandwidth characteristics and a lower risk of connection loss.”

This new framework, called Ambient Edge, enhances traditional Edge Computing techniques with proactive service staging, plug-and-play device integration, real-time messaging, automatic task offloading and seamless handover between Smart Spaces. The Ambient Lab is also building a visual programming extension for Ambient Edge, which will enable designers without extensive programming knowledge to build a variety of immersive systems, including interactive installations, enhanced learning spaces, digitally augmented architecture and projection mapping that turns any surface into an interactive video display.

Learn more about the Ambient Lab at: http://ambientlabs.org.