The Brain-Computer Interface (BCI)

A Brain-Computer Interface (BCI) provides a new communication channel between the human brain and the computer. Mental activity leads to changes of electrophysiological signals like the Electroencephalogram (EEG) or Electrocorticogram (ECoG). The BCI system detects such changes and transforms it into a control signal which can, for example, be used as spelling device or to control a cursor on the computer monitor. One of the main goals is to enable completely paralyzed patients (locked-in syndrome) to communicate with their environment.

European ICT Prize

ICT The g.tec BCI research platform wins the European ICT Prize 2007.

Two learning systems

An interesting question for the development of a BCI is how to handle two learning systems: The machine should learn to discriminate between different patterns of brain activity as accurate as possible and the user of the BCI should learn to perform different mental tasks in order to produce distinct brain signals.

BCI research makes high demands on the system and software used. Parameter extraction, pattern recognition and classification as well as the generation of neurofeedback for a successful training of the user has to run in real-time.

g.BCIsys - The complete development and research system

g.tec provides complete MATLAB-based development/research systems including all hard- and software components needed for data acquisition, real-time and off-line data analysis, data set classification and for providing neurofeedback.

The BCI system can be realized with g.MOBIlab+, g.USBamp or g.BSamp. g.MOBIlab+ is available with up to 8 EEG channels and is portable and available with wireless signal transmission. g.USBamp is available for 16-256 EEG channels and transmits the data over USB to the PC or notebook. g.BSamp is availabe for 8, 16 to 80 channels.

The software package High-Speed Online Processing under SIMULINK allows to read the biosignal data directly into SIMULINK. SIMULINK blocks are used to visualize and store the data. The parameter extraction and classification is performed either with standard SIMULINK blocks, with the g.RTanalyze library or with self-written S-functions.

After the EEG data acquisition the data can be analyzed with g.BSanalyze, the EEG and classification toolbox.

Ready-to-use BCI sample applications allow to make state-of-the-art BCI experiments within a few yours.

Highlights

complete BCI research system for EEG and ECoG  
ready to go paradigms for spelling and cursor control  
seamless integration of real-time experiments and off-line analysis  
runs either with g.MOBIlab+ or g.USBamp technology  
open source paradigms allow to make easily adaptations and to develop own applications  
MATLAB/Simulink Rapid Prototyping environment speeds up development times from month to days  
BCI technology proven by hundreds of subjects and labs  

Scientific approaches

During the past decade many groups all over the world intensified their work in the field of BCI-research. The different methods published in international scientific journals display the wide range of possible solutions for the problem. The following link provides an overview on some of the most prominent methods and authors working on this hot topic: State of the Art in BCI research

To support your start into the fascinating world of Brain-Computer Interface research see some literature here: Publications

Some of the most commonly used strategies to realize a BCI are:

Imagery of movements of different limbs cause changes in oscillatory EEG activity over sensorimotor areas of the central cortex. These changes can be classified by weighting spectral parameters of different frequency bands for different electrode positions.

A P300 component is produced if an unlike event occurs. The P300 occurs about 300 ms after the event and has to be detected by specific algorithms. The P300 components are mainly used to create a spelling device for paralyzed patients.

Slow shifts of cortical potentials occur when a subject performs an imagery of expecting an event (like waiting for a traffic light turning to green). The resulting DC-shift can be used for biofeedback to improve the training effects and to generate a control signal for communication.

Also other mental tasks such as mental arithmetic, mental cube rotation or attention versus relaxation are used to produce characteristic changes of EEG patterns. One attempt has also been not to guide the subjects with any strategy but use specific EEG-biofeedback, so that the user attempts to find his/her own strategy for producing the required changes in the EEG.

Another method uses steady-state visually evoked potentials (SSVEP) from flickering light sources. Directing attention to a source with a specific flicker frequency enlarges evoked components in the EEG with the same frequency.

It can be stated that none of all the methods used in BCI research yields perfect results but the performance was significantly improved by new parameter-extraction algorithms and pattern-recognition/classification methods. The usability of a BCI has to be evaluated with respect to the following aspects:
accuracy (classification error, hits vs. false, false positives, ...)
information transfer (decision speed, bit/min, ...)
number of classes (idling vs. activation of 1 class, 2 or more different classes, ...)
operation mode (synchronous: predefined decision intervals, asynchronous: free decision time)
intended application (spelling device, control of orthotic/prosthetic device, environmental control)
g.BCIsys comes with ready-to-use example BCI-paradigms based on changes in oscillatory EEG activity induced by two different types of motor imageries and paradigms based on the P300 component.

Example Paradigm: motor imagery
Step 1 (initial training):
Based on a cue (arrow on the screen pointing to the left or to the right) the subject performs left and right hand movement imageries (duration 3-4 seconds). To train the classifier between 40 and 160 trials are recommended. EEG should be recorded from electrode positions C3 and C4.

 

Step 3 (training with neuro-feedback):
If cross-validation results yield a classification error below approx. 20 %, the classifier can be used to generate neuro-feedback for further training. For this case data are online classified and the result is graphically presented to the subject as a horizontal bar on the screen deflecting to the right if a right hand motor imagery is detected and deflecting to the left if a left hand motor imagery is detected.
 
Step 2 (offline analysis and classifier generation):
The recorded data are processed with g.BSanalyze. Alpha and beta bandpower parameters for both EEG channels are computed to build the feature matrix. Linear discriminant analysis is used for classification and cross-validation shows the usability of the best classifier.		   Step 4 (classifier update):
The continuous feedback should help the subject to train the motor imageries leading to a correct classification. To improve the performance the classifier should be updated after some successful sessions. A new classifier can also be computed from the data of a feedback session. Offline analysis of the recorded data supports feature optimization.

Example Application: Ping Pong

Everybody knows the famous Ping-Pong game that was played in the seventies on TV sets. In this example two persons are connected to the BCI system and each one is controlling the racket with motor imagery. The racket moves upwards by left hand movement imagination and downwards by right hand movement imagination.

The algorithm extracts EEG bandpower in the alpha and beta ranges of two EEG channels per person. Therefore in total 4 EEG channels are analyzed and classified.

Example Paradigm: P300 spelling device

The P300 paradigm presents e.g. 36 letters in a 6 x 6 matrix on the computer monitor. Each letter is flashing up in a random order and the subject has to be concentrated on the letter it wants to write. As soon as the corresponding letter is flashing up a P300 component is produced inside the brain. The algorithms are analyzing the EEG data and select the letter with the highest P300 component. Then this letter is written onto the screen. Normally between 2-20 flashes per letter are required to achieve a high accuracy. The number is dependent on the electrode position used, the training level of the subject and the individual height of the P300 response of the subject.

In Copy Spelling mode first a word or a sentence has to be entered. The task of the subject is to copy exactly each letter as shown in the following picture:

This allows to calculate the error rate of the spelling device and is mainly used for the training of the subject.

In Free Spelling mode the subject is just concentrated on the letter it wants to write. The BCI system searches for the letter with the highest P300 response and writes the letters onto the screen. This mode can be used by patients for communication, to answer questions or to express needs.

For this system no a priori subject specific EEG data acquisition is necessary. The new subject is just connected with EEG electrodes to the BCI system and the algorithms search for the corresponding features. Some subjects can already spell words within a few minutes.

Further information and literature for P300

g.tec brings the first patient-ready BCI on the market. The EEG-based spelling system is called intendiX and enables the user to select keys from a matrix just by thinking.

www.intendiX.com

Example: ECoG recording

g.USBamp is a CF recording device and therefore it can also be used for invasive recordings. The picture below shows an ECoG electrode grid overlaying the brain. The electrode grid is connected to the BCI system for real-time analysis and paradigm presentation.

ECoG xray

ECoG electrode grid photo by Gerwin Schalk (Wadsworth Center, Albany, USA) and Kai Miller, Jeff Ojemann (University of Washington).

g.USBamp has the necessary CE and FDA certification for invasive recordings.
Example Paradigm: Robot Control with SSVEP
(Steady State Visual Evoked Potentials)

In this example the user gets stimulated by four lightsources flickering with different frequencies. These lightsources can be LEDs or flashing fields presented on a computer monitor.
Four fields are adequate for the control of the robot to let it move forwards, backwards, to the left and to the right. Four lightsources in our case are therefore four possible commands. (Theoretically stimulation can be done with more or less than four lightsources if more or less than four commands are required.)
All the user has to do now is to look at one specific flickering light (for example the one which is assigned to the “move forward”-command). The algorithms try to detect the frequency of the flashing field the user has looked at and make a desicion in the form of a control signal. This can be sent for example to the robot and let it move forward. Additionally there is a zero-decision which is made when it is most likely that the user doesn’t look on any flashing field. In this case the robot doesen’t move at all.


Stimulation with a computer monitor (left) and LEDs (right)

The algorithms do not need any information about the user before operation.
In training-mode (for training of the classification algorithms) and in test-mode (for calculating error-rates and statistics) command arrows appear, indicating the flashfield the user must look on. In free-drive-mode the user can move the robot in every direction he/she likes.

This is a section from a video in which a user had to move a robot towards a glass bottle. You can see the four commands given by the user through his brain: right – forward – left – forward. The user took 22 seconds to achive the goal.

Example Paradigm: Smart Home Control
Controlling a House with Thoughts

In this example the P300 based BCI system was connected to a Virtual Reality (VR) system. The virtual 3D representation of the smart home had different control elements (TV, music, windows, heating system, phone,…) and allowed the subjects to move through the apartment. Some tasks could be done, like playing music, watching TV, open doors, moving around... Therefore seven control masks were created: a light mask, a music mask, a phone mask, a temperature mask, a TV mask, a move mask and a go to mask. The controlling mask for the TV is shown in Fig 1.

Fig.1: The TV BCI controlling mask.

Each mask contained 13 to 50 commands. In Fig. 2 the Goto mask is shown. The mask gives a bird’s eye view of the apartment with characters at specific positions. For going e.g. to the couch you have to focus on the "B".
 
Fig.2: The Go to BCI controlling mask and the virtual Appartment

A P300 based BCI system is optimally suited to control smart home applications with high accuracy and high reliability. As the BCI and P300 method works fine with the Powerwall (shown in Fig.3), the system can act as cheap test environment for real smart homes for paralyzed patients.

Fig.3: One Smart Home Subject in front of the Powerwall

Example Paradigm: The Multimodal Brain Orchestra

4 members in the orchestra playing music with the power of their thoughts. They used two different non-invasive Brain-Computer Interface (BCI) technology concepts to control their virtual and multi-modal instruments. Two of them controlled the volume with the so-called Steady-State Visual-Evoked Potentials (SSVEP). The other two members of the orchestra played the music with the so called P300 response.

For more information please visit the Information about the Multimodal Brain Orchestra.

BCI in virtual environments

Controlling virtual environments by thoughts

Tutorials

Brain Computer Interface Manual (PDF 1.6 MByte) - The document describes the steps to classify the brain-computer interface EEG data with the bandpower algorithm and a linear discriminant analysis.

P300 Spelling Device with g.USBamp and Simulink (PDF 993 kByte) - Description of the steps to perform P300 spelling device experiments with g.USBamp. Furthermore the off-line P300 analysis is descripted.

P300 Seplling Device with g.MOBIlab+ and Simulink (PDF 1.3 MByte) - same as above but with g.MOBIlab+ technology

Brain Computer Interface with g.USBamp (PDF 1.05 MByte) - The document describes real-time BCI experiments with motor imagery

Brain Computer Interface with g.MOBIlab+ (PDF 1.07 MByte) - same as above but with g.MOBIlab+ technology


Videos

Locomotion by thoughts (MPG 12,1 MByte). The movie shows an experiment carried out together with University of Technology Graz (Robert Leeb, Gert Pfurtscheller) and University College London (Mel Slater, Doron Friedman) within the EU project PRESENCCIA.

Music generation by brain waves (MPG 2,7 MByte). Project of University of Plymouth (Eduardo Miranda).

The Multimodal Brain Orchestra (MOV 13 MByte) realized by specs.upf.edu

Robot Control with Steady State Visual Evoked Potentials (MPG 12,5 MByte) by g.tec, Austria

Brain-Computer Interface on CNN (MPG 10,2 MByte)

YouTube versions:

Brain Power on 60 Minutes

Spelling with the Brain-Computer Interface

Controlling a smart home with the brain-computer interface.

P300-based BCI for Internet Browsing

Links:

Motor Imagery based Essex online BCI: http://dces.essex.ac.uk/Research/BCIs/AABAC.html

Prerequisite

MATLAB, Simulink and Signal Processing Toolbox (Release 2008b)
The Signal Processing Blockset is a useful extension for doing signal analysis in time and frequency domains.

 

Up