Electromagnetic piano can be performed hands-free, guided only by brain waves (developed by DXARTS)

DXARTS develops electromagnetic musical instruments that can be performed hands-free, guided only by your brain waves.

DXARTS is home to several faculty laboratories, classrooms, and auditoriums. The media production and instrument laboratory is where a Yamaha grand piano rests, but this particular piano is unlike any other though: It is electronically communicating to a nearby computer through a MIDI interface and a home-brewed computer program.

It was here that Richard Karpen, UW School of Music director and founder of the Center for Digital Arts and Experimental Media (DXARTS), along with Marcin Pączkowski, a recent Ph.D. graduate turned full-time researcher, had displayed what the four-person faculty of DXARTS has been working on: an instrument played not with hands, but brainwaves.

Their research led to the April 6 production of “Music of Today,” a concert centered in performing music via electroencephalogram (EEG) and electromyography (EMG) devices, which detect brain and muscle neuron activity.

For the piano to work, Karpen, Pączkowski, associate professor of music composition Juan Pampin, and a dedicated team of researchers undertook a collaborative effort which involved composing, programming, and performing to open up the potential for an entirely new experimental style of music.

“We’re not trying to make music that already exists. If Beethoven had the modern orchestra, he wouldn’t have composed all the music he did compose,” said Karpen.

As Karpen speaks, he demonstrates the EEG and EMG in action, which allows him to hold a conversation facing away from a piano that plays seemingly on its own. The technology behind the piano is highly interwoven. EMG armbands sense muscle neurons firing in the left and right forearms, the former controlling the range of the notes played and the latter playing the actual notes, either scales or modalities. To play the piano, no perceptible movement is required — just small twitches below the skin.

The EEG, or what was visible on the head of the performers, is 3D-printed and built on open-source technology. It communicates with the computer via a dedicated Wi-Fi transmitter built by Pączkowski, making it more reliable than traditional bluetooth connections.

“It was a relief finding a way to combine the multiple threads of development,” Pączkowski said. “I was looking to develop more into the digital domain and [DXARTS] enabled me through the resources [to] fuse the creative and technological parts.”

A high-resolution and traditional EEG, rather than the low-budget one DXARTS uses, would siphon any research budget at a breakneck pace, so the researchers at DXARTS are continuing to work on obtaining and developing more accurate technology. The hope is to one day be able to “songify” brain activity in a way that doesn’t require the intermediary of an EMG.

DXARTS is far-reaching in its interdisciplinary efforts. Karpen, Pampin and their colleagues have brought together at various times mechanical engineers, a handful of neuroscientists, and especially recently Thomas Deuel, an EEG clinician at Swedish hospital and a doctor in neuroscience. Deuel works especially in emergent situations and with stroke victims.

Pampin and Deuel are working directly with some of Deuel’s patients to potentially restore lost musical ability, allowing stroke patients to play instruments using the sensory technology where their physical abilities have faltered. To accomplish that, Pampin and Deuel will be working in collaboration with the Swedish Neuroscience Institute thanks to an $80,000 grant awarded from the National Endowment for the Arts late last year.

In the future, Pampin plans to work with a string quartet using the EEG and EMG technologies.

Different instruments have different limitations, especially with what they can physically play, but for the DXARTS team, that’s half the fun. The experimentation is methodical and by working with highly trained musicians in the future, people who generally have precisely the sort of brain activity they would like to track, they hope to hone down the techniques.

“When artists are not experimenting, I think they’re doing something different [than art],” Karpen said.“To me, classical art and classical music are experimental disciplines, and always have been.”

Source: dailyuw