TED 2022: Entering the TED Metaverse with Particle Ink
Transmedia storytelling company Kaleidoco’s immersive augmented reality introduction
Welcoming in-person attendees for the first time since 2019, the idea-rich and action-oriented TED conference commenced its main stage programming this weekend with an unexpected request: Whitney Pennington Rodgers, the current affairs curator at TED, asked guests to lift up provided iPads and point them toward the stage. As the venue lights dimmed, a joyful spectacle burst forth that blended projection mapping, live performance and—as seen through the iPad screen—augmented reality. During the display human dancers engaged with digital characters, all enhanced by imaginative depictions of natural phenomena. This powerful hybrid experience was called Particle Ink, and is the work of entertainment technology company Kaleidoco.
Kaleidoco’s founders, Jennifer Tuft and Cassandra Rosenthal, are on a two-fold mission. “First, we are creating purposeful proprietary technology,” Tuft tells COOL HUNTING. “Second, we are building an original content metaverse.” Rosenthal adds that what drives them is the desire “to understand how media and technology are conversing and to create digital empathy by virtue of this proprietary technology.” TED and its curious community made an ideal, eager audience for all of this.
Kaleidoco’s origins are rooted in another beloved annual happening. “Cassandra and I were both at Sundance in 2017 and we saw the beginnings of what is now Particle Ink, brought on by the LightPoets, at the New Frontier program,” Tuft says. “It was a small immersive experience, around 11 minutes, and we both—coming from the background of storytellers and producers of transmedia properties—were totally blown away by what we saw. We realized this is what we had to do. It didn’t take long for both of us to stop doing what we had been doing and commit full-time to building a company around what we saw that day.”
Immediately their approach was different, as Rosenthal and Tuft strove for the perfect balance between reality and augmentation in their work. “The combination of live performance, augmented reality and projection mapping essentially opens up our ability to play with the characters of our imagination,” Tuft says. “The idea that we in our physical world can play with a virtual character in real time has boundless potential.”
A further benefit to their mixed-reality balance is the ability to keep people grounded in a communal experience (as was the case at TED), but at the same time granting them access to the benefits of a virtual world. “We take projection mapping and live performance, the combination of that, and bring it to the next level as a completely immersive experience in a proscenium stage, by using multi-viewer AR technology which we developed with our partners,” Tuft explains, “so that everybody in that room at TED or everybody at home is experiencing both the virtual and the physical world converging live.”
“We’ve broken that fourth wall in the world of entertainment,” Rosenthal adds. “We’re already all living through our devices. But, in many ways, this mixed-reality blending brings you into this world. Rather than watching it from afar, you are more of an active participant in an empathetic, sensitive kind of way. With augmented reality, you feel like you become a part of the story. When you are starting to watch those performer, with those types of technologies integrated, it draws you in closer. You want to walk inside of it all. That’s what it enables.”
“For us, when we build technology in augmented reality, we are thinking very specifically on more than just what makes something cool in a novel way, where people’s minds might be blown but no one is sure what the purpose is and what it is supposed to achieve,” Tuft says. Thus, their work is anchored in something more. For instance, Kaleidoco is attempting to give consciousness to augmented reality characters. “We are enabling them or imbuing them with intelligence so that they can be reactionary to us, so that we can have a real-time reactionary relationship with the augmented world,” Tuft says.
“Jen is describing our proprietary technology called augmented unification, which enables characters to have contextual awareness—so that they are understanding the properties and attributes of an environment and are able to react in real time,” Rosenthal explains. “You become a part of that interaction. It’s taking the story to the next level. In time, these AI characters can also become more a part of your daily life, but in a positive way. For us, its important to create positive feedback loops and create joyful and inspiring experiences that help people play.”
As for how all of these components aligned for the TED presentation, Tuft says they looked toward this year’s theme. “TED 2022 is ‘A New Era,’ so we asked ourselves ‘what is the new era of transmedia storytelling?’ We are integrating older versions of storytelling, then enhancing it. In our day-to-day lives, where we are basically tethered to the digital world. We want to demonstrate how we can bring this to life in a way where it’s not controlling us, but where we are willingly becoming a participant—where it is enhancing our lives.”
Both Tuft and Rosenthal make frequent reference to the “2.5th dimension,” which they consider a term that describes the Particle Ink metaverse. On a metaphoric level, Tuft says “It’s the in-between place—between waking and sleeping or where the rabbit waits before being pulled out of the hat.” “Literally,” she adds, “it’s the place that exists in-between the 2D—our pen to paper or paintbrush to canvas creation—and us as 3D creators.” Rosenthal adds that “It’s what we bring to life in that in-between, as well.”
“The 2.5th dimension is alive and it’s all around us, everywhere,” she continues. You have to find the cracks into that dimension. Those are what we call portals.” For Kaleidoco, the first portal into the 2.5th dimension opened atop the TED main stage. The second portal will open this Saturday, 16 April, to a broader audience in the Las Vegas Downtown Arts District. “Particle Ink: Speed of Dark, at The LightHouse, is a more permanent mixed reality experience. People can come and explore and wander and journey through the 2.5th dimension. There’s a lot of interactivity.”
“We are only at the beginning,” Rosenthal concludes. “I don’t think people realize how much they’re already using augmented reality in their own daily lives. Still, there is a lot of opportunity to create more purposeful and meaningful interactions through augmented reality. That’s not just through the elements of entertainment storytelling but also through education, health and retail. There are a lot of ways augmented reality can come alive.”
All images: Particle Ink performs at SESSION 1 at TED2022: A New Era. 10-14 April, 2022, Vancouver, BC, Canada. Photo: Gilberto Tadday / TED
