At last month’s TedX in Sydney conference Karen Palmer spoke of her experiences combining neurogaming, film, and parkour to create dynamic immersive experiences that may be the level beyond VR, AR or MR.
The 41 year old Palmer is an interactive filmmaker and part of a growing creative movement that seeks to empower game players through radical new approaches to technology and gaming. Palmer a veteran film maker in London focuses on neurogaming, where neuroscience meets gameplay. Her work and the work of others in the field is to have 360 immersive experiences that are driven by brain wave activities and not hand controls of consoles.
Previously, her award winning short ‘Mental Block’ was shortlisted for The Jerwood Moving Image Awards, screened at Cannes, broadcast on the Youtube Homepage.
Palmer’s film making experience encompasses short films, documentary and music video. She directed a short film as a trailer for the legendary writer Stephen King’s new book Full Dark, No Stars and was commissioned by Channel 4 to direct the documentary Pirate.
Her projects include the interactive installation ‘The Way’, showcased in 2010 at a solo exhibition in conjunction with The Mayor’s Children’s Art Day, at Westminster Arts Library; ’Evolution’. Her project Syncself 1 involved Parkour runner Shirley Darlington Rowat and was nominated for an HBO Award at the Filmteractive Festival in Poland 2013.
Syncself 1 was a mobile app “which interacted with a video screen to simulate a physical connection – a sort of gamification between your smartphone and your tv,” she explains. For her following project she wanted to close the loop much more and find a way to really connect with how someone felt while immersing themselves in a film. While this is possible via CGI, Palmer wanted to use new live action narrative approaches.
Her next project was SYNCSELF 2 and it recreates the process of transcending fear through parkour with runner Gogoly Yao. The project was showcased at the iconic V&A Museum in London. This interactive parkour experience is controlled by the user’s mental focus, monitored by EEG biosensors.
Karen Palmer combines neurogaming, film, wearable technology and parkour to create dynamic immersive video installations to inspire and empower the user. Her practise involves creating a sense of mindfulness and “being in the moment” to simulate a physiological and narrative journey through storytelling and technology.
The technology was created in conjunction with The Department of Electronic and Computer Engineering at Brunel University London and with the skill of Parkour Generations, (one of the leading parkour organisations in the world).
The system uses the Mindwave Neurosky headset. Palmer explored a lot of options before deciding on the Neurosky. The EEG code was programmed in Javascript at Brunel. The game has a three minute introduction with a real time feedback barometer and a short test simulation. After the explanatory introduction, the user is presented with three game levels. The film branches multiple times or on different narrative paths that are determined by the user’s concentration. The first three minutes not only serves as an instruction but it lets people get past the novelty and get into the game, the system also needs this baseline from which to work.
The EEG headset detects electromagnetic waves generated by your brain that determine your state of mind, whether that’s focused or calm etc, from the dominant frequencies. The headset computes it into a numerical value and then it converts this to a pathway choice in the ‘film’.
From the viewer’s point of view the experience or story just flows, but the path a user takes is completely decided by the headset. If the numerical value is above the say 51% mark, which is picked as being ‘focused’, then it will show the narrative of the parkour making the jump, and if you fall below 50% then the runner falls also. The numerical ‘success’ rate can be set by Palmer, to make the game more or less difficult.
The players wear a headset and your own brain responses decide your path through the game. Faced with a challenge in the game you must focus to succeed. For example, if your character is running towards an obviously hard parkour jump and you glance at or think about something else, your character will not make the jump. When instructing people on how to control the game,
Palmer tells them to concentrate and focus, but what that means is different to every user. Some need to think about the stories camera angles to focus while others the motivation by thinking about what the runners are heading to or from. ‘Focus’ changes from person to person. One woman started actually performing Tai-chi while in the game as her method to focus – and it worked!
The game is designed for adults but there appears no limits based on age with players ranging from kids to senior citizens enjoying the experience.
When Syncself 2 first appeared it was installed in a inflatable igloo at the iconic V&A Museum in London. Syncself 2 consists of a widescreen TV and a sofa. The EEG headset is fitted and then the user is given their warm up session.
Palmer added an actual audio narration as an extra dimension to the game. At one point, a voice tells the user that they can’t make the jump. For some people this stimulates the desire to succeed, for others it defeats their ability to hold concentration. For example, for one woman who Palmer later discovered suffered from depression, she simply could not ‘get past’ the voice of defeat and repeatedly failed this jump in the game. Individuals respond differently, but it always seems to provoke a response, based on the EEG readings.
What makes neurogaming interesting is that to succeed requires one to totally concentrate on the game and immerse yourself in the experience. It is the antithesis of second screening while watching TV.
This approach of bio-feedback has clear implications outside gaming. People with a range of mental health issues have used the system and their may be strong mental health benefits from using neurogaming as a form of mental exercise for people with things such as Attention Deficient Disorder.
Karen’s next project is Riot. It is similar in concept, but you are dropped into the start of an urban riot breaking out.. your one goal is to stay focused enough to get home alive. Riot is very ambitious. The branching game is looking at an expanded set of biomechanical and neurological input devices from EEG to face /emotion tracking, motion sensors, heart rate monitoring. “It will be not just your mind but your whole body experience in game play – but still driving a live action film” explains Palmer. One of the interesting aspect is that the game will be designed to draw out responses that people may not expect. Rather than a first person shooter, the game will test the users resolve, explore their fears and require them to judge in a way they themselves might not expect. Riot is in pre-production currently.
Finally…
As a bonus – if you needed any proof that parkour runner Gogoly Yao was given a superior body by the good Lord, watch this raw footage of his ‘running’…
