A luminous storytelling

While looking at an exhibit, have you ever thought if there was someone to narrate the stories related to it? Well, Lumen: Mixed Reality Storytelling, which has won the Playable Museum Award instituted by the Museo Marino Marini (Florence), intends to do just that. A project by Kochi-based Arvind Sanjeev, it aims at allowing people to immerse themselves in alternate realities in their natural space through machine learning and projection mapping technologies. “I have done this project as part of my Masters programme,” says Arvind, who works in Sweden now. “The idea is to create human-centric designs instead of the current technology and profit-oriented ones,” he adds. 

The two-way interaction enables a dialogue between the person and the environment. Through his project, Arvind tries to connect people. “These days everyone uses headsets. Imagine, one person sitting in living room with her/his family, wearing a headset. Listening through the headset gives a unique experience, but that person is isolated. Through Lumen, I try to give that same experience without being isolated,” explains Arvind, who developed this method after a thorough research.

“It was created through a people-centred design process by talking, co-creating and collaborating with different people. Starting off with several sacrificial concepts that were used for talking with experts in the realm of AR (Augmented Reality) and MR(Mixed Reality) and to ground them on the same thinking, key interactions were then identified from these story sketches. Moreover, co-creation and prototyping sessions with a LARP (Live Action Role Playing) storymaster, Troels Andersen, were really helpful for understanding the elements and experiences that create immersive stories,” he explains. 

As part of the research, escape rooms and museums were also studied for understanding the immersion they are capable of. “For exploring the interactions, several different concepts such as storytelling with kids, storytelling in art galleries, storytelling in homes, etc. were tested with people through functional prototypes. And through these sessions, it was understood that the potential for this kind of tool lies in it being a platform, a storytelling platform that lets people tell their stories through magical interactions with their environment. And that’s how Lumen as a platform was born.” 

Lumen uses the yolo darknet machine learning platform to classify objects that gradually generates stories on top of the objects. It has adopted the concept of flash light to execute it. Suppose, when people visit a museum, they are given a Lumen in the form of a flashlight. “As soon as they turn on this flashlight, they notice that it actually tries to tell the person or the group of people carrying it a story through its enchanted light. It projects arrows and pointers and helps the person navigate to particular parts of the museum. On reaching each waypoint, Lumen then tells the story of the particular exhibit in that location. For example, if it’s a sculpture, Lumen projects a face onto the sculpture that captures the essence of what the artist wanted that sculpture to be. It can then tell the story of how that sculpture came to be and what was going through the artist's mind while creating it. This is one example flow of how I imagine Lumen would live inside a museum,” he elucidates. The concept is to make the viewer feel like walking through history. 

Arvind, who developed an educational initiative called DIY Hacking (now known as Maker.Pro) and a product prototype called ‘Ride Smart’, a car automation or control-Smartphone application, will be awarded a grant of 10,000 euros and the opportunity to be mentored by the judges and project curator, Fabio Viola. The official presentation ceremony will take place when the museum reopens in October.