Augmented Human Intelligence: 9 Ways Machines Can Support Us

It has also made us highly dependent on digital devices. MIT Media Lab strives to improve human interaction with tech.

“We live in a world where humans and technology cannot be separated.”

In the opening statement of her speech at Burning Minds 2022, Pattie Maes, MIT Media Lab professor, AI scientist, and pioneer in human-computer interactions, argued that the digital devices we take for granted today do not necessarily work to our advantage.

“They are far from perfect. Their use is disruptive and inefficient,”

Maes stated. “Study after study is showing that devices actually decrease our memory capability, our thinking ability, our creativity, social connectivity, and many more.”

There is also a privacy concern. AI-powered systems rely on collecting a lot of personal data that, if it gets into the wrong hands, can be used against us.

With so many limitations, should we continue the chase after more advanced technology?

Pattie Maes, during Burning Minds 2022, speaking about the future of tech and digital devices for reaching humanity's potential.

Making the human-computer relationship work

The future doesn’t have to look so bleak. Since the very beginning of her journey in the AI space, Maes has been dedicated to seeking ways that technology could help with decision-making, communication, and augmenting our memories. As she shared in the story featured in the Hidden Heroes series, “I was always much more interested in helping people.”

This attitude served her and her team as a North Star in the last few decades while they have been working on human-computer interaction systems at the MIT Media Lab. “We try to envision the future potential where these systems could really help people,” she said in her speech.

She believes

“we really have an unprecedented opportunity to change our relationship with digital devices to enhance our human cognition.”

By using sensor data, in devices like smartwatches and all sorts of wearables, as well as AI techniques, we can help humans be more creative, remain attentive, improve memory and motivation, communicate clearly and empathetically, and generally improve our wellbeing.

Her team have developed countless devices that, even though they look like futuristic tools from sci-fi movies, have already been proven to bring tangible benefits to many.

Memory 2.0.

The NeverMind system, invented by an MIT research group, was designed to help people memorize things by connecting our factual and spatial memories. The team created augmented reality glasses that encode facts and link them with familiar places.

A person wearing the glasses would go on a virtual stroll where certain things to memorize were prompted together with some visual reference in the alternative reality.

The research group performed an experiment in which participants were asked to memorize the winners of the Super Bowl year by year. While they were virtually walking from the subway station to their office, they would encounter objects that would somehow relate to the football team members. This simple trick, based on associations, proved useful in improving memory.

The control group spent the same amount of time studying the same set of winners on paper. Results? The group’s recall accuracy the next day was 50% lower than the NeverMind users.

Oscar Rosello, Marc Exposito, and Pattie Maes. NeverMind: Using Augmented Reality for Memorization. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST ’16)

My fake AI teacher

Despite a lot of bad press that deepfakes have been getting in recent years, they can also play a positive role. “We decided to use synthetic AI and create teachers that a student looks up to to improve their motivation to learn,” Maes shared during the presentation.

In the experiment, an Elon Musk deepfake and another non-existing character were programmed to teach a class about vaccines. “We actually told people these were deepfakes and they would give the exact same lecture,” Maes said. They even used the same voice. The results showed that people who admired Musk actually did better and were much more eager to learn.

Pataranutaporn*, Leong*, Danry et al., 2021

This finding can really make an impact in education by helping students who don't have access to quality educators. “We can create teachers that people look up to, thereby helping them with motivation and attitude towards learning,” Maes believes.

AI critic for critical thinking

Daniel Kahneman, the Nobel Prize winner and author of “Thinking, Fast and Slow” theory, claims we have two systems of thinking:

1. System one, is fast, unconscious, and automatic, but prone to errors and very biased.
2. System two, is slow, more deliberate and conscious, and takes effort.

Pattie Maes and her team built on this theory to empower people to make more conscious decisions, especially in times of information overload and fake news.

Daniel Kahneman: Thinking, Fast and Slow

They designed AR glasses that, when turned on, register the same things you hear and give you real-time advice on whether and why information might be right or wrong. The device strives to verify if there is enough evidence to a statement, and checks for other reliability factors.

The study showed that people using the set would further develop their critical thinking skills and be more willing to challenge statements before accepting them automatically using the “default mode” of thinking.

Fighting attention span loss

Information overload has made a huge impact on our attention spans. A recent survey of the UK public by the Policy Institute and Center for Attention Studies at King’s College London revealed that 50% of people report shorter attention spans caused by the modern information environment.

Because the media we consume are largely designed for shorter formats – think about Twitter with its character limit, or Instagram’s 15-second limit per individual story – we tend to quickly lose interest in what we’re watching or reading.

One of Maes’ postdocs, Nataliya Kosmyna, designed AR glasses with built-in EEG (electroencephalography) that examine the wearer’s brain waves and EOG (electrooculogram) that follows their eye movements. Based on information from the sensors, the glasses provide real-time feedback when the wearer’s attention drops.

Kosmyna, Morris, Sarawgi, Nguen, and Maes, IEEE BSN’2019, CHI’Adj 2019

Experiments with this device during lectures showed that people receiving this biofeedback when their attention drifted away would do better in tests of the presented material than people who got random reminders to be attentive or people who got no reminders at all.

Using such smart AR glasses in some environments obviously may raise concerns. We wouldn’t want our boss to be tracking our attention and giving us a nudge every time we drift away. However, imagine the device supporting drivers and reminding them to take a break when they experience fatigue – sound like a good use case?

Creativity boost

Joanne Leong, another student of Pattie Maes, built a system that uses Snapchat filters to turn a person into a younger version of themselves, and designed an experiment in which people would engage in creativity tasks as they were looking at three different versions of themselves: their true self, a younger variant, and an inventor one.

The study showed that seeing yourself as a child or as an inventor actually enhances our creative capabilities.

Instant solution to a problem

Another student, Adam Horowitz, designed an experiment inspired by a method practiced by scientists, inventors, and artists in the past, in which one would take a nap while holding some heavy object in hand and force themselves to think about their problem as drifting off to sleep.

Once they would fall asleep, the hand would relax and drop the ball, waking them up. In that short dream, a solution would come to mind – a eureka moment.

Horowitz experimented with the method, but instead of holding a ball, a person would have a sensor on the hand (or a simple app on the phone) that would detect when they start falling asleep and wake them up in the middle of the dream to stimulate creativity.

Voice for the speech impaired

Maes’ team has also been working with people who have lost their ability to speak. They can still move their face, but their vocal cords produce very little sound or no sound at all. The team designed a system called Alter Ego that picks up minuscule muscle signals on the face, neck, and chin as people try to speak quietly to a device. Then, using AI algorithms, they can classify the signals and recognize what the person is saying.

Empathetic communication

Maes’ team have been trying not only to support people with impaired speech, but also to empower everyone to communicate more emphatically.

Camilo Rojas, who works closely with Pattie Maes on “the new generation of empathetic technology,” developed a device that uses machine learning to analyze subtle signals, such as tone of voice, choice of words, facial expressions, electrodermal activity, heart rate, etc. during conversations, and feeds the emotional content back to speakers in real time.

“By amplifying these social signals, people end up talking in a much more empathetic way,” Maes pointed out, referring to key insights from the experiment. It turns out they would ask more questions and be more tuned in with the other person.

Superpower through meditation

The fast-paced and overstimulated reality triggers a lot of stress in our society. There are plenty of tools available out there to bring more peace into one’s life, but the device designed by the MIT team takes practicing mindfulness into another dimension.

The virtual reality system uses EEG sensors to track brain activity. Once we relax, we can use “superpowers” like levitating or telekinesis in the presented virtual world. And every time our focus drops, we lose the superpowers. The brainwave sensors track our stress levels in real time and teach us how to calm ourselves down and stay focused.

A perfect balance in the human-computer interaction

Digital systems have the potential to enhance our human cognition, but we always have to be mindful of possible negative consequences. As Marshall McLuhan said,

“Every extension is an amputation.”

We must remember that while we shape the new technologies, those technologies also shape us and our lives. Pattie Maes and her team at the MIT Media Lab are committed to constantly working on our relationship with digital devices so they serve us in the best possible way.

As she said at the end of her presentation, “we want to provide people with better tools to change themselves than force anyone to use them,” and that’s a North Star that should guide all of the scientists and engineers in building modern-day technology.