Analysis and classification of mental states of vigilance with evolutionary computation

Published in Projects
 european comission  EUROPEAN COMMISSION
Research Executive Agency
Marie Curie Actions – International Research Staff Exchange
 Seventh Framework Programme logo

Project details

Duration 01/11/2013-31/10/2016
Funding European Comission - 7th Framework Programme for Research, technological Development and Demonstration - Marie Curie Actions - IRSES. FP7-MC-IRSES
Project coordinator Dr. Pierrick Legrand
Objective Our aim was to develop new search and optimization strategies, based on evolutionary computation (EC) and genetic programming (GP) for the automatic induction of efficient and accurate classifiers. EC and GP are search techniques that can reach good solutions in highly multi-modal, non-differentiable and discontinuous spaces; and such is the case for the problem addressed here: the detection of mental states of vigilance. This project combines the expertise of researcher partners from five converging fields: Classification, Neurosciences, Signal Processing, Evolutionary Computation and Parallel Computing, in Europe - France (Inria, University of Bordeaux), Portugal (BioISI) and Spain (UNEX) - and North America - Mexico (ITT and CICESE). The partners complement and enhance their respective disciplines, allowing for a strong multi-disciplinary collaboration and proposal development. The exchange program goals and milestones gave a comprehensive strategy for the strengthening of current scientific relations amongst the partners, and allowed the construction of long-lasting scientific relationships that produced high quality theoretical and applied research.

Bordeaux Segalen University logo smallInria-corporate smallUniversity of Bordeaux and INRIA – France

 uex marcacolor smallUniversidad de Extremadura - Spain

logo inesc smallINESC-ID - Portugal

logo cicese smallCICESE - Mexico

ITT logo smallITT - Mexico


Over the last decade, Human-Computer Interaction (HCI) has grown and matured as a field. Gone are the days when only a mouse and keyboard could be used to interact with a computer. The most ambitious of such interfaces are Brain-Computer Interaction (BCI) systems. BCI's goal is to allow a person to interact with an artificial system using brain activity. A common approach towards BCI is to analyze, categorize and interpret Electroencephalography (EEG) signals in such a way that they alter the state of a computer. ACoBSEC's objective is to study the development of computer systems for the automatic analysis and classification of mental states of vigilance; i.e., a person's state of alertness. Such a task is relevant to diverse domains, where a person is required to be in a particular state. This problem is not a trivial one. In fact, EEG signals are known to be noisy, irregular and tend to vary from person to person, making the development of general techniques a very difficult scientific endeavor. Our aim is to develop new search and optimization strategies, based on evolutionary computation (EC) and genetic programming (GP) for the automatic induction of efficient and accurate classifiers. EC and GP are search techniques that can reach good solutions in multi-modal, non-differentiable and discontinuous spaces; and such is the case for the problem addressed here. This project combines the expertise of research partners from five converging fields:

Classification, Neurosciences (University of Bordeaux), Signal Processing, Evolutionary Computation and Parallel Computing in Europe (France Inria, Portugal FCUL, Spain UNEX) and South America (Mexico ITT, CICESE). The exchange program goals and milestones give a comprehensive strategy for the strengthening of current scientific relations amongst partners, as well as for the construction of long-lasting scientific relationships that produce high quality theoretical and applied research.

Description of work packages

WP1: Classification with Evolutionary Computation

The objective was to develop EC/GP-based classifiers for noisy real world data that exploit the advantages of statistical based approaches while also encouraging the explorative search capabilities of meta-heuristic based search strategies. It is now obvious that in order to solve difficult problems, hybrid approaches such as this could leverage the main strengths of each domain and possibly minimize some of their main weaknesses. Such algorithms have been useful in the particular sense of this project, where biological data and signals are well-known for their high levels of noise, heterogeneous frequency content, chaotic nature and dynamical properties. However, in a broader sense, developing such algorithms facilitated the use of hyper-heuristic based inductive learning in other real-world domain based on the strong cooperation of INRIA, FCUL and ITT.
In this WP, the participants established a common background context and promising lines of research in EC/GP-based classification, the paradigm used to analyze and interpret biological data and signals, the main goals of the proposal. In this sense, it is closely related to WP4 and WP5. In the former case, the conceptual and software-based tools developed in WP1 has been used to solve the classification problems posed in WP4. In the latter case, the EC-based classifiers have been the main subject of study in WP5, where the goal was to derive predictive models of classifier performance, to develop autonomous classification methods that can choose the best search dynamics for the particular problem instance that is addressed in a particular context. Finally, WP1 relates to WP5, since some of the software tools proposed and developed here are actually integrated into the software toolbox that is expected in WP7.

WP2: Acquisition and study of Bio-Signals

From the discovery of the brain electrical activity and the invention of the electroencephalography recording technique (Berger 1929), the oscillatory rhythms of EEG have been correlated to different vigilance states. Those rhythms, defined by their frequency band, are still often used in diagnostics of pathologies. The use of EEG signal is evolving and is more efficient thanks to the joint improvement of the acquisition methods and signal analysis methods. This WP is directly related to the acquisition methods of EEG and to the study of the analysis method of these signals.
First, this WP contributed to a better understanding of the main goal of this project which is depicted in WP4. Secondly, EEG data are hard to record. The recordings are very dependent of the neighborhood (electrical system, lights, etc...). The university of Bordeaux has an EEG recording system (64 electrodes) and was experienced with this kind of recording. The contribution of this WP was to teach to the other partners how to perform a correct recording and how to analyze these signals.
On the other hand, the partners helped the university to develop new analysis tools.
The main objective of this WP was to use the knowledge of an expert of this domain (F. Faïta) to teach to the other partners how the data are obtained and what the usual processing methods are. The goal was also to develop basic tools for EEG signal processing.

WP3: Analysis and feature extraction from EEG recordings of human states of vigilance

In this WP, the goal was to propose, develop and test several different types of feature extraction methods for EEG-based analysis and interpretation of human mental states of vigilance. The proposal was to take a diverse approach, incorporating different types of formal analysis, allowing us to compare results and evaluate the strengths and weaknesses of each method. Moreover, the proposal includes a heuristic approach, automatic feature synthesis through a GP-based search. It is assumed that no single feature can give a global solution to the problem of discriminately describing EEG signals, therefore hybrid features were proposed, composed of the above mentioned features combined in a principled manner using Fuzzy Logic. Finally, the quality of each proposed feature has been validated using a large scale experimental verification exploiting a distributed grid computing model.
The partners collaborating in this WP provided a multidisciplinary view to the problem of feature extraction. First, the problem has been analyzed from the perspective of the signal processing community, by considering wavelet decompositions and regularity based analysis with Hölder exponents. Second, the EEG signals has been analyzed from the perspective of chaos theory. Third, a heuristic approach to automatic feature synthesis has been developed, using EC and GP algorithms. Finally, several features have been combined in order to gain a more general description of EEG data using data fusion techniques from the computational intelligence community, particularly using EC and fuzzy logic.

WP4: Detection and Classification of human states of vigilance through EEG data

Almost all of the partners contributed in this WP, since most of the work developed in other WPs is used and tested here; including EEG recording methods, GP based classifiers and feature extractors.
The general problem addressed in this WP can be posed as follows. The goal was to automatically detect different mental states of vigilance from human test subjects using EEG recordings. It is currently accepted that if one analyses two EEG recording from the same individual it is possible to determine in which case the person was more "relaxed". However, suppose that only a single recording is available. In this case, the problem becomes increasingly more difficult, since a previous general model would be required. However, most researchers agree that EEG signals are not homogenous across different individuals; i.e., each person can exhibit different signal patterns even when they state that they are experiencing a similar mental experience. Therefore, we were interested in developing new tools to address the problem of feature selection; feature construction and data classification to categorize (classify) the mental state of an (unknown) individual.
The proposed solution incorporates three convergent lines research. First, the EEG data obtained in WP2 is the central object of study. The EEG signals were analyzed using the tools proposed and developed in WP3, providing a set of descriptive features. Then, EEG signals were categorized and classified using the algorithmic tools developed in WP1. This WP also provided the real-world test case for the analysis and descriptive tools of problem difficulty developed in WP5. Finally, the developed algorithms are being integrated into the software tools developed in WP7.
This WP solves the main goal of the proposal, the detection and classification of mental states of vigilance. In this sense, all other WPs are related with it. The classifiers used were developed in WP1; the subject of study, EEG data, was interpreted and pre-processed in WP2; descriptive features used to discriminate data and extract useful information were developed in WP3; finally, predictors of expected performance for the black-box methods developed in WP5, were validated with the experimental results of this WP.

WP5: Problem analysis and prediction of expected performance

Genetic Programming (GP) deals with the development of evolutionary algorithms (EA's) for automatic program induction [1], However, as for every EA, GP systems are stochastic search process, with many degrees of freedom and heuristic components. Therefore, as of yet, it is not possible to derive, from first principles, weather GP can solve a particular problem or task. A current goal within the GP community is to estimate how hard a problem instance might be for a specific GP. Such a measure could allow researchers to correctly choose and tune a GP search without the need of actually executing the code, which usually is computationally expensive. If we want to measure the difficulty of a problem in GP, we can consider at least two different frames of reference. The first is the problem domain, which is independent of the method used to solve the problem [3]. The second frame of reference is to consider a perspective directly related to the process used to find a solution; in the case of GP this frame of reference corresponds with the search space and fitness landscape.
In this WP, the goal was to use the problem domain as the frame of reference, and characterize the difficulty of a problem based on the expected performance the GP search, a quantity that is derived from domain specific features of each problem instance. This is a pragmatic approach, the evolutionary search is taken as a black-box process and the performance of GP on a set of training problems is used to build predictors of the expected performance on unseen problems, following a machine learning methodology. We refer to such measures of problem difficulty as Predictors of Expected Performance (PEPs).
Landscapes and problem difficulty has been the subject of a good deal of research regarding EA's. For instance, researchers have developed work on landscape correlation, autocorrelation, epistasis, monotonicity, locality and neutrality. However, the main work on this topic relates to evolvability indicators, such as fitness distance correlation and the negative slope coefficient, two empirical tools that measure the underlying difficulty of a given search process; however, such measures are not necessarily correlated with the difficulty of the problem or the expected performance a given solution strategy might achieve. Therefore, this WP followed a more recent line of research, where the goal is to directly predict the performance of an EC algorithm without actually performing the search.

WP6: Modelling Neurological Behaviors

The goal of this WP was to derive descriptive models of the object of study, the dynamics that govern the evolution of mental states of vigilance over a series of prescribed stimuli. Given the detailed analysis of how to characterize and discriminate among different mental states, a detailed description of the dynamics of state transitions induced by external stimuli can be of great relevance for future study.
In particular, the goal was to derive two types of descriptive models. First, computational models derived automatically, using heuristic methods, particularly GP, fuzzy logic and neural networks. Second, models based on differential equations describing the dynamics of mental states at a coarse level. In the former case, the goal was to replicate the behaviors exhibited by human beings, as their mental states develop over time within a given context, thus allowing us to replicate these features within an artificial brain. In the latter case, it is possible to localize the compact invariant sets of a system and deduce sufficient conditions under which the dynamics of the system can be controlled in a specified manner.

WP7: Tools integration in GPLAB

The goal of this WP was to assemble the tools developed in the other WPs into GPLAB. GPLAB is a Genetic Programming toolbox for MATLAB initially created and currently developed by Sara Silva. Most of its functions are used as "plug and play" devices, making it a versatile and easily extendable tool. The main objective of WP7 is to provide to the community the numerical products of our collaboration. A parallel objective is to extend and improve GPLAB with the new developed functionalities.
The GP tools developed in WPs 1, 3, 4 and 5, are merged in the toolbox GPLAB in such a way that (i) they use and are used by the other functionalities already implemented, following the same "plug and play" philosophy, and (ii) together they implement specialized functionalities for EEG classification, feature extraction and difficulty prediction. Despite the fact that WP7 is the concluding part of this project, the integration of the tools will be performed as they are developed, beginning in the second year of the project.
Some of the current features of GPLAB are: 3 modes of tree initialization (Full, Grow, Ramped Half-and-Half) + 3 variations on these, several pre-made functions and terminals for building trees, dynamic limits on tree depth or size (optional) , resource-limited GP (variable size populations) (optional) , dynamic populations (variable size populations) (optional) , 4 genetic operators (crossover, mutation, swap mutation, shrink mutation), configurable automatic adaptation of operator probabilities (optional) , steady-state + generational + batch modes, with fuzzy frontiers between them , 5 sampling methods (Roulette, SUS, Tournament, Lexicographic Parsimony Pressure Tournament, Double Tournament), 3 modes of calculating the expected number of offspring (absolute + 2 ranking methods), 2 methods for reading input files and for calculating fitness (symbolic regression and parity problems + artificial ant problems) , runtime cross-validation of the best individual of the run (optional) , offline cross-validation or prediction of results by any individual (optional) , 4 levels of elitism, configurable stop conditions, saving of results to files (5 frequency modes, optional) , 3 modes of runtime textual output, runtime graphical output (4 plots, optional) , offline graphical output (5 functions, optional) , runtime measurement of population diversity (2 measures, optional) , runtime measurement of average tree level, number of nodes, number of introns, tree fill rate (optional), 4 demonstration functions (symbolic regression, parity, artificial ant, multiplexer).
Website of the GPLAB toolbox: