Direct mind-computer interface

Template:NeuropsychologyA brain-computer interface (BCI), sometimes called a direct neural interface or a brain-machine interface, is a direct communication pathway between a human or animal brain (or brain cell culture) and an external device. In one-way BCIs, computers either accept commands from the brain or send signals to it (for example, to restore vision) but not both. Two-way BCIs would allow brains and external devices to exchange information in both directions but have yet to be successfully implanted in animals or humans.

In this definition, the word brain means the brain or nervous system of an organic life form rather than the mind. Computer means any processing or computational device, from simple circuits to silicon chips (including hypothetical future technologies such as quantum computing).

Research on BCIs began in the 1970s, but it wasn't until the mid-1990s that the first working experimental implants in humans appeared. Following years of animal experimentation, early working implants in humans now exist, designed to restore damaged hearing, sight and movement. The common thread throughout the research is the remarkable cortical plasticity of the brain, which often adapts to BCIs, treating prostheses controlled by implants as natural limbs. With recent advances in technology and knowledge, pioneering researchers could now conceivably attempt to produce BCIs that augment human functions rather than simply restoring them, previously only the realm of science fiction.

BCI versus Neuroprosthetics

Main article: neuroprosthetics

Neuroprosthetics is an area of neuroscience concerned with neural prostheses — using artificial devices to replace the function of impaired nervous systems or sensory organs. The most widely used neuroprosthetic device is the cochlear implant, which was implanted in approximately 100,000 people worldwide as of 2006. There are also several neuroprosthetic devices that aim to restore vision, including retinal implants, although this article only discusses implants directly into the brain.

The differences between BCIs and neuroprosthetics are mostly in the ways the terms are used: neuroprosthetics typically connect the nervous system, to a device, whereas the term "BCIs" usually connect the brain (or nervous system) with a computer system. Practical neuroprosthetics can be linked to any part of the nervous system, for example peripheral nerves, while the term "BCI" usually designates a narrower class of systems which interface with the central nervous system.

The terms are sometimes used interchangeably and for good reason. Neuroprosthetics and BCI seek to achieve the same aims, such as restoring sight, hearing, movement, ability to communicate, and even cognitive function. Both use similar experimental methods and surgical techniques.

Animal BCI research

File:MouseBCI.jpg

Rats implanted with BCIs in Theodore Berger's experiments

Several laboratories have managed to record signals from monkey and rat cerebral cortexes in order to operate BCIs to carry out movement. Monkeys have navigated computer cursors on screen and commanded robotic arms to perform simple tasks simply by thinking about the task and without any motor output. Other research on cats has decoded visual signals.

Early work

Studies that developed algorithms to reconstruct movements from motor cortex neurons, which control movement, date back to the 1970s. Work by groups led by Schmidt, Fetz and Baker in the 1970s established that monkeys could quickly learn to voluntarily control the firing rate of individual neurons in the primary motor cortex via closed-loop operant conditioning, a training method using punishment and rewards.

In the 1980s, Apostolos Georgopoulos at Johns Hopkins University found a mathematical relationship between the electrical responses of single motor-cortex neurons in rhesus macaque monkeys and the direction that monkeys moved their arms (based on a cosine function). He also found that dispersed groups of neurons in different areas of the brain collectively controlled motor commands but was only able to record the firings of neurons in one area at a time because of technical limitations imposed by his equipment.

There has been rapid development in BCIs since the mid-1990s. Several groups have been able to capture complex brain motor centre signals using recordings from neural ensembles (groups of neurons) and use these to control external devices, including research groups led by Richard Andersen, John Donoghue, Phillip Kennedy, Miguel Nicolelis, and Andrew Schwartz.

Prominent research successes

Phillip Kennedy and colleagues built the first intracortical brain-computer interface by implanting neurotrophic-cone electrodes into monkeys.

File:LGN Cat Vison Recording.jpg

Yang Dan and colleagues' recordings of cat vision using a BCI implanted in the lateral geniculate nucleus (top row: original image; bottom row: recording)

In 1999, researchers led by Yang Dan at University of California, Berkeley decoded neuronal firings to reproduce images seen by cats. The team used an array of electrodes embedded in the thalamus (which integrates all of the brain’s sensory input) of sharp-eyed cats. Researchers targeted 177 brain cells in the thalamus lateral geniculate nucleus area, which decodes signals from the retina. The cats were shown eight short movies, and their neuron firings were recorded. Using mathematical filters, the researchers decoded the signals to generate movies of what the cats saw and were able to reconstruct recognisable scenes and moving objects.

Miguel Nicolelis has been a prominent proponent of using multiple electrodes spread over a greater area of the brain to obtain neuronal signals to drive a BCI. Such neural ensembles are said to reduce the variability in output produced by single electrodes, which could make it difficult to operate a BCI.

After conducting initial studies in rats during the 1990s, Nicolelis and his colleagues developed BCIs that decoded brain activity in owl monkeys and used the devices to reproduce monkey movements in robotic arms. Monkeys have advanced reaching and grasping abilities and good hand manipulation skills, making them ideal test subjects for this kind of work.

By 2000, the group succeeded in building a BCI that reproduced owl monkey movements while the monkey operated a joystick or reached for food. The BCI operated in real time and could also control a separate robot remotely over Internet protocol. But the monkeys could not see the arm moving and did not receive any feedback, a so-called open-loop BCI.

File:BCI.jpg

Diagram of the BCI developed by Miguel Nicolelis and collegues for use on Rhesus monkeys

Later experiments by Nicolelis using rhesus monkeys, succeeded in closing the feedback loop and reproduced monkey reaching and grasping movements in a robot arm. With their deeply cleft and furrowed brains, rhesus monkeys are considered to be better models for human neurophysiology than owl monkeys. The monkeys were trained to reach and grasp objects on a computer screen by manipulating a joystick while corresponding movements by a robot arm were hidden. The monkeys were later shown the robot directly and learned to control it by viewing its movements. The BCI used velocity predictions to control reaching movements and simultaneously predicted hand gripping force.

Other labs that develop BCIs and algorithms that decode neuron signals include John Donoghue from Brown University, Andrew Schwartz from the University of Pittsburgh and Richard Andersen from Caltech. These researchers were able to produce working BCIs even though they recorded signals from far fewer neurons than Nicolelis (15–30 neurons versus 50–200 neurons).

Donoghue's group reported training rhesus monkeys to use a BCI to track visual targets on a computer screen with or without assistance of a joystick (closed-loop BCI). Schwartz's group created a BCI for three-dimensional tracking in virtual reality and also reproduced BCI control in a robotic arm. The group created headlines when they demonstrated that a monkey could feed itself pieces of zucchini using a robotic arm controlled by the animal's own brain signals.

Andersen's group used recordings of premovement activity from the posterior parietal cortex in their BCI, including signals created when experimental animals anticipated receiving a reward.

In addition to predicting kinematic and kinetic parameters of limb movements, BCIs that predict electromyographic or electrical activity of muscles are being developed. Such BCIs could be used to restore mobility in paralysed limbs by electrically stimulating muscles.

Human BCI research

Invasive BCIs

Invasive BCI research has targeted repairing damaged sight and providing new functionality to paralysed people. Invasive BCIs are implanted directly into the grey matter of the brain during neurosurgery. As they rest in the grey matter, invasive devices produce the highest quality signals of BCI devices but are prone to scar-tissue build-up, causing the signal to become weaker or even lost as the body reacts to a foreign object in the brain.

File:BCI JensNaumann.png

Jens Naumann, a man with acquired blindness, being interviewed about his vision BCI on CBS's The Early Show

In vision science, direct brain implants have been used to treat non-congenital (acquired) blindness. One of the first scientists to come up with a working brain interface to restore sight was private researcher William Dobelle.

Dobelle's first prototype was implanted into "Jerry," a man blinded in adulthood, in 1978. A single-array BCI containing 68 electrodes was implanted onto Jerry’s visual cortex and succeeded in producing phosphenes, the sensation of seeing light. The system included cameras mounted on glasses to send signals to the implant. Initially, the implant allowed Jerry to see shades of grey in a limited field of vision at a low frame-rate. This also required him to be hooked up to a two-ton mainframe, but shrinking electronics and faster computers made his artificial eye more portable and now enable him to perform simple tasks unassisted.

File:BrainGate.jpg

Dummy unit illustrating the design of a BrainGate interface

In 2002, Jens Naumann, also blinded in adulthood, became the first in a series of 16 paying patients to receive Dobelle’s second generation implant, marking one of the earliest commercial uses of BCIs. The second generation device used a more sophisticated implant enabling better mapping of phosphenes into coherent vision. Phosphenes are spread out across the visual field in what researchers call the starry-night effect. Immediately after his implant, Jens was able to use his imperfectly restored vision to drive slowly around the parking area of the research institute.

BCIs focusing on motor neuroprosthetics aim to either restore movement in paralysed individuals or provide devices to assist them, such as interfaces with computers or robot arms.

Researchers at Emory University in Atlanta led by Philip Kennedy and Roy Bakay were first to install a brain implant in a human that produced signals of high enough quality to simulate movement. Their patient, Johnny Ray, suffered from ‘locked-in syndrome’ after suffering a brain-stem stroke. Ray’s implant was installed in 1998 and he lived long enough to start working with the implant, eventually learning to control a computer cursor.

Tetraplegic Matt Nagle became the first person to control an artificial hand using a BCI in 2005 as part of the first nine-month human trial of Cyberkinetics Neurotechnology’s BrainGate chip-implant. Implanted in Nagle’s right precentral gyrus (area of the motor cortex for arm movement), the 96-electrode BrainGate implant allowed Nagle to control a robotic arm by thinking about moving his hand as well as a computer cursor, lights and TV.

Partially-invasive BCIs

Partially invasive BCI devices are implanted inside the skull but rest outside the brain rather than amidst the grey matter. They produce better resolution signals than non-invasive BCIs where the bone tissue of the cranium deflects and deforms signals and have a lower risk of forming scar-tissue in the brain than fully-invasive BCIs.

Electrocorticography (ECoG) measures the electrical activity of the brain taken from beneath the skull in a similar way to non-invasive electroencephalography (see below), but the electrodes are embedded in a thin plastic pad that is placed above the cortex, beneath the dura mater. ECoG technologies were first trialed in humans in 2004 by Eric Leuthardt and Daniel Moran from Washington University in St Louis. In a later trial, the researchers enabled a teenage boy to play Space Invaders using his ECoG implant. This research indicates that it is difficult to produce kinematic BCI devices with more than one dimension of control using ECoG.

Light Reactive Imaging BCI devices are still in the realm of theory. These would involve implanting a laser inside the skull. The laser would be trained on a single neuron and the neuron's reflectance measured by a separate sensor. When the neuron fires, the laser light pattern and wavelengths it reflects would change slightly. This would allow researchers to monitor single neurons but require less contact with tissue and reduce the risk of scar-tissue build-up.

This signal can be either subdural or epidural, but is not taken from within the brain parenchyma itself. It has not been studied extensively until recently due to the limited access of subjects. Currently, the only manner to acquire the signal for study is through the use of patients requiring invasive monitoring for localization and resection of an epileptogenic focus.

The experimental approach was developed based on current understanding of sensorimotor rhythms and on the methodology of current EEG-based BCIs that use these rhythms.[28] Sensorimotor rhythms comprise mu (8-12 Hz), beta (18–26 Hz), and gamma (>30 Hz) oscillations[29-31]. As mentioned earlier, the lower frequencies of mu and beta are thought to be produced by thalamocortical circuits and they change in amplitude in association with actual or imagined movements [32-35]. Higher frequencies (> 30 Hz) are thought to be produced by smaller cortical assemblies [36]. BCIs based on EEG oscillations have focused exclusively on mu and beta rhythms because gamma rhythms are inconspicuous at the scalp [37]. In contrast, gamma rhythms as well as mu and beta rhythms are prominent in ECoG during movements [34-37].

Until recently, the signal was assumed to be very similar to that of EEG in regards to the amount and type of information it could convey. This, however, was not true; the signal itself is quite different. The ECoG signal is much more robust compared to EEG signal: its magnitude is typically five times larger (0.05 - 1.0 versus 0.01-0.2 mV for EEG) [38], its spatial resolution as it relates to electrode spacing is much finer (0.125 versus 3.0 cm for EEG)[39, 40], and its frequency bandwidth is significantly higher (0-200 Hz versus 0- 40 Hz for EEG). When analyzed on a functional level many studies have revealed that higher frequency bandwidths, unavailable to EEG methods, carry highly specific and anatomically focal information about cortical processing [41-44]. ECoG’s superior frequency range is attributable to two factors. First, the capacitance of cell membranes of the overlying tissue combined with their intrinsic electrical resistance constitutes a low-pass (RC) filter that largely eliminates higher frequencies from the EEG [40]. Second, higher frequencies tend to be produced by smaller cortical assemblies (38). Thus, they are more prominent at electrodes that are closer to cortex than EEG electrodes and thereby achieve higher spatial resolution [40].

Recent studies have cogently demonstrated its effectiveness as a signal in BCI application. Leuthardt et al. in 2004 revealed the first use of ECoG in closed loop control. Over brief training periods of 3-24 min, four patients mastered control and achieved success rates of 74-100% in one-dimensional tasks. In additional experiments, the same group found that ECoG signals at frequencies up to 180 Hz accurately reflected the direction of two-dimensional joystick movements [45]. Soon after Schalk, et al. in 2004 demonstrates two dimensional online control using independent signals at high frequencies inconspicuous to that appreciable by EEG [46]. Additionally, Leuthardt et al. in 2005 demonstrated that ECoG control using signal from the epidural space was also possible [47]. All these studies combined show the ECoG signal to carry a high level of specific cortical information which can allow the user to gain control very rapidly. Beyond the technical demonstration of ECoG BCI feasibility, there is some pathologic and clinical evidence to support the implant viability of subdural based devices. There is an extensive body of literature investigating the tissue response to intraparenchymal cortical electrodes and their associated signal prohibitive reactive gliotic sheaths [25, 26]. Though more limited, the studies that have been performed investigating non-penetrating subdural placed electrodes, however, have been more encouraging. In a cat and dog models, long term subdural implants showed minimal cortical or leptomeningeal tissue reaction while maintaining prolonged electrophysiologic recording [48-51]. In clinical studies the use of subdural electrodes as implants for motor cortex stimulation have been shown to be stable and effective implants for the treatment of chronic pain [52-54]. Additionally, preliminary work using the implantable Neuropace device for the purpose of long term subdural electrode monitoring for seizure identification and abortion has also been shown to be stable [55].

ECoG is a very promising intermediate BCI modality because it has higher spatial resolution, better signal-to-noise ratio, wider frequency range, and lesser training requirements than scalp-recorded EEG, and at the same time has lower technical difficulty, lower clinical risk, and probably superior long-term stability than intracortical single-neuron recording. This feature profile and recent evidence of the high level of control with minimal training requirements shows potential for real world application for people with motor disabilities.

Non-invasive BCIs

As well as invasive experiments, there have also been experiments in humans using non-invasive neuroimaging technologies as interfaces. Signals recorded in this way have been used to power muscle implants and restore partial movement in an experimental volunteer. Although they are easy to wear, non-invasive implants produce poor signal resolution because the skull dampens signals, dispersing and blurring the electromagnetic waves created by the neurons. Although the waves can still be detected it is more difficult to determine the area of the brain that created them or the actions of individual neurons.

File:ElectroEncephalogram.png

Recordings of brainwaves produced by an electroencephalogram

Electroencephalography (EEG) is the most studied potential non-invasive interface, mainly due to its fine temporal resolution, ease of use, portability and low set-up cost. But as well as the technology's susceptibility to noise, another substantial barrier to using EEG as a brain-computer interface is the extensive training required before users can work the technology. For example, in experiments beginning in the mid-1990s, Niels Birbaumer of the University of Tübingen in Germany used EEG recordings of slow cortical potential to give paralysed patients limited control over a computer cursor. (Birbaumer had earlier trained epileptics to prevent impending fits by controlling this low voltage wave.) The experiment saw ten patients trained to move a computer cursor by controlling their brainwaves. The process was slow, requiring more than an hour for patients to write 100 characters with the cursor, while training often took many months.

Another research parameter is the type of waves measured. Birbaumer's later research with Jonathan Wolpaw at New York State University has focused on developing technology that would allow users to choose the brain signals they found easiest to operate a BCI, including mu and beta waves.

A further parameter is the method of feedback used and this is shown in studies of P300 signals. Patterns of P300 waves are generated involuntarily (stimulus-feedback) when people see something they recognise and may allow BCIs to decode categories of thoughts without training patients first. By contrast, the biofeedback methods described above require learning to control brainwaves so the resulting brain activity can be detected. In 2000, for example, research by Jessica Bayliss at the University of Rochester showed that volunteers wearing virtual reality helmets could control elements in a virtual world using their P300 EEG readings, including turning lights on and off and bringing a mock-up car to a stop.

In 1999, researchers at Case Western Reserve University led by Hunter Peckham, used 64-electrode EEG skullcap to return limited hand movements to quadriplegic Jim Jatich. As Jatich concentrated on simple but opposite concepts like up and down, his beta-rhythm EEG output was analysed using software to identify patterns in the noise. A basic pattern was identified and used to control a switch: Above average activity was set to on, below average off. As well as enabling Jatich to control a computer cursor the signals were also used to drive the nerve controllers embedded in his hands, restoring some movement.

Electronic neural networks have been deployed which shift the learning phase from the user to the computer. Experiments by scientists at the Fraunhofer Society in 2004 using neural networks led to noticeable improvements within 30 minutes of training.

Experiments by Eduardo Miranda aim to use EEG recordings of mental activity associated with music to allow the disabled to express themselves musically through an encephalophone.

Magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) have both been used successfully as non-invasive BCIs. In a widely reported experiment, fMRI allowed two users being scanned to play Pong in real-time by altering their haemodynamic response or brain blood flow through biofeedback techniques. fMRI measurements of haemodynamic responses in real time have also been used to control robot arms with a seven second delay between thought and movement.

Commercialization and companies

John Donoghue and fellow researchers founded Cyberkinetics. Now listed on a US stock exchange and known as Cyberkinetic Neurotechnology Inc, the company markets its electrode arrays under the BrainGate product name and has set the development of practical BCIs for humans as its major goal. The BrainGate is based on the Utah Array developed by Dick Normann.

Philip Kennedy founded Neural Signals in 1987 to develop BCIs that would allow paralysed patients to communicate with the outside world and control external devices. As well as an invasive BCI, the company also sells an implant to restore speech. Neural Signals' Brain Communicator BCI device uses glass cones containing microelectrodes coated with proteins to encourage the electrodes to bind to neurons.

Although 16 paying patients were treated using William Dobelle's vision BCI, new implants ceased within a year of Dobelle's death in 2004. A company controlled by Dobelle, Avery Biomedical Devices, and Stony Brook University are continuing development of the implant, which has not yet received FDA approval for human implantation.

There are three main consumer-devices commercial-competitors in this area (expected launch date mentioned in brackets) which are going to launch such devices primarily for gaming- and PC-users:

• Neural Impulse Actuator (April, 2008)
• Emotiv Systems (Christmas/Holiday Season, 2008)
• NeuroSky (Not Declared)

Cell-culture BCIs

Main article: cultured neuronal networks

Researchers have built devices to interface with neural cells and entire neural networks in cultures outside animals. As well as furthering research on animal implantable devices, experiments on cultured neural tissue have focused on building problem-solving networks, constructing basic computers and manipulating robotic devices. Research into techniques for stimulating and recording from individual neurons grown on semiconductor chips is sometimes referred to as neuroelectronics or neurochips.

File:CaltechNeuroChip.jpg

World first: Neurochip developed by Caltech researchers Jerome Pine and Michael Maher

Development of the first working neurochip was claimed by a Caltech team led by Jerome Pine and Michael Maher in 1997. The Caltech chip had room for 16 neurons.

In 2003, a team led by Theodore Berger at the University of Southern California started work on a neurochip designed to function as an artificial or prosthetic hippocampus. The neurochip was designed to function in rat brains and is intended as a prototype for the eventual development of higher-brain prosthesis. The hippocampus was chosen because it is thought to be the most ordered and structured part of the brain and is the most studied area. Its function is to encode experiences for storage as long-term memories elsewhere in the brain.

Thomas DeMarse at the University of Florida used a culture of 25,000 neurons taken from a rat's brain to fly a F-22 fighter jet aircraft simulator. After collection, the cortical neurons were cultured in a petri dish and rapidly began to reconnect themselves to form a living neural network. The cells were arranged over a grid of 60 electrodes and used to control the pitch and yaw functions of the simulator. The study's focus was on understanding how the human brain performs and learns computational tasks at a cellular level.

Ethical considerations

Discussion about the ethical implications of BCIs has been relatively muted. This may be because the research holds great promise in the fight against disability and BCI researchers have yet to attract the attention of animal rights groups. It may also be because BCIs are being used to acquire signals to control devices rather than the other way around, although vision research is the exception to this.

This ethical debate is likely to intensify as BCIs become more technologically advanced and it becomes apparent that they may not just be used therapeutically but for human enhancement. Today's brain pacemakers, which are already used to treat neurological conditions such as depression could become a type of BCI and be used to modify other behaviours. Neurochips could also develop further, for example the artificial hippocampus, raising issues about what it actually means to be human.

Some of the ethical considerations that BCIs would raise under these circumstances are already being debated in relation to brain implants and the broader area of mind control.

Theme in fiction

The prospect of BCIs and brain implants of all kinds have been important themes in science fiction. See brain implants in fiction and philosophy for a review of this literature.

See also

• Technologies: Brain implant, Biomechatronics, Cyberware, Neuroprosthetics, Neurotechnology, BrainGate, Brainport, Sensory substitution, Intelligence amplification, Isolated brain
• Scientific Phenomena: Cortical plasticity, Neural ensemble, electrocorticography (ECoG)
• Disciplines: Cognitive Science, NBIC, Neural engineering, Neuroscience
• Speculative: Mind transfer, Whole-body transplant, Mind uploading
• Other: Human enhancement, Mind control, Neurohacking, Simulated reality, Transhumanism

External links

Organisations

(for a list of universities see Neural Engineering - Neural Engineering Labs)

• Avery Biomedical, invasive vision restoration BCI
• Cyberkinetics Neurotechnology Inc, human trials of the Braingate BCI
• Neural Signals, Brain Communicator invasive BCI
• US Defense Advanced Research Projects Agency (DARPA):
  • Human-Assisted Neural Devices program to develop a non-invasive neuroprosthetic BCI
  • Bio: Info: Micro Interface program to develop invasive BCIs (terminated, funding withdrawn)

Researchers listed in above text

• Theodore Berger at the University of Southern California
• Niels Birbaumer at the University of Tuebingen
• Thomas DeMarse at the University of Florida
• John Donoghue's lab at Brown University
• Eberhart E. Fetz at the Washington National Primate Research Center
• Apostolos Georgopoulos at the University of Minnesota
• Eduardo Reck Miranda at the University of Plymouth in Plymouth, Devon
• Miguel Nicolelis’ lab at Duke University
• Hunter Peckham at Case Western Reserve University
• Jerome Pine's lab at Caltech
• Andrew Schwartz at the University of Pittsburgh
• Yang Dan at University of California, Berkeley
• Jonathan Wolpaw at the New York State Department of Health

Portals and articles

• BCI-info.org, BCI news and research portal
• The open-source Electroencephalography project and Programmable chip version, Sourceforge open source EEG projects
• BCI Competition, run by Fraunhofer Institute, third and most recent competition closed mid-2005
• EEG database, Team Phypa's public hub for BCI data exchange
• The Next Brainiacs Wired Magazine, August 2001, article on Jim Jatich’s implant
• Controlling robots with the mind, Scientific American, 16 September 2002, article on Miguel Nicolelis
• Vision quest, Wired Magazine, September 2002, article on artificial vision
• Lymnaea stagnalis and the development of neuroelectronic technologies, Journal of Neuroscience Research, 2004, academic paper on a cell-culture BCI
• 'Brain' in a dish flies flight simulator, CNN, 4 November 2004, article on cell-culture BCI
• How to talk when you can't speak, Slate, 10 February 2005, article on using EEG to communicate with minimally conscious patients
• Mind Control, Wired Magazine, March 2005, article on Matt Nagle
• ...a step towards neuron-based functional chips, Biosens Bioelectron, January 2006, academic paper on a cell-culture BCI
• Functional alignment of feedback effects from visual cortex to thalamus Nature Neuroscience 9, 1330-1336 (2006), 17 September 2006, recent advances in decoding LGN visual signals
• The Memory Hacker, PopSci.com, April 2007, article on Theodore Berger, retrieved 10 April 2007