نهمین کنگره نقشه برداری مغز

نمایش مقاله

Title:

Decoding Cognitive Language Information from EEG Using Word- and Sentence-Level Representations

Subject:

Brain–Computer Interfaces (BCIs)

Presenter Author:
Name:	Fateme Majidi

Author:
Name 1:	Fateme Majidi - Corresponding - Presenter
Affiliation 1:	Department of Biomedical Engineering Hamedan University of Technology, Hamedan, Iran

Name 2:	Mohammad Bagher Khodabakhsi
Affiliation 2:	Department of Biomedical Engineering Hamedan University of Technology, Hamedan, Iran

Name 3:	Mohammad Reza Rezaeian
Affiliation 3:	Department of Biomedical Engineering Hamedan University of Technology, Hamedan, Iran

Name 4:
Affiliation 4:

Name 5:
Affiliation 5:

Name 6:
Affiliation 6:

Name 7:
Affiliation 7:

Name 8:
Affiliation 8:

Name 9:
Affiliation 9:

Name 10:
Affiliation 10:

Abstract

Original Research

introduction :

Decoding language-related cognitive information directly from non-invasive brain signals remains one of the most difficult problems in cognitive neuroscience and brain–computer interface (BCI) research. Electroencephalography (EEG), with its high temporal resolution, provides direct access to the fast neural dynamics underlying language comprehension. However, translating sparse and noise-contaminated EEG signals—characterised by limited spatial resolution—into coherent linguistic output is still an unresolved challenge. In this study, we introduce a transformer-based architecture for EEG-to-text decoding that combines linguistically aligned word-level and sentence-level neural representations to capture complementary aspects of cognitive language processing.

Methods:

EEG data are obtained from the ZuCo dataset (Versions 1 and 2), recorded while participants read sentences under natural reading conditions. Neural representations are extracted at two levels of linguistic granularity. At the word level, EEG features are computed from signal segments time-locked to individual word onsets, corresponding to neural activity associated with lexical processing. Sentence-level representations are obtained by temporally pooling EEG activity across the full duration of each sentence, capturing higher-order language-related dynamics. Following standard preprocessing procedures, including band-pass filtering, artefact removal, and signal normalisation, these representations are passed to a transformer-based EEG encoder. The resulting latent features are then used to condition a pre-trained BART language model for text generation. Model training is performed in two stages: first, the EEG encoder is trained while the remaining components are kept fixed; subsequently, the entire model is fine-tuned end-to-end. To reduce exposure bias during autoregressive decoding, Scheduled Sampling is applied during training.

Results:

Model performance is evaluated using standard text generation metrics, including BLEU, ROUGE, word error rate (WER), and character error rate (CER). The results indicate that the model is capable of producing sentences from EEG signals with a meaningful degree of semantic coherence. As commonly observed in autoregressive text generation, BLEU scores decrease with increasing n-gram order (e.g., BLEU-1 = 0.4372 and BLEU-4 = 0.0911), reflecting the growing difficulty of modelling long-range linguistic dependencies. In addition to quantitative evaluation, qualitative inspection suggests that combining word-level and sentence-level EEG representations leads to more semantically consistent generated text.

Conclusion:

This work presents a transformer-based framework that integrates multi-level EEG representations to model neural activity underlying language comprehension. Although the current performance does not yet reach that of conventional text generation systems, the findings demonstrate that decoding cognitive language information from non-invasively recorded EEG signals is feasible. Future research may focus on improving cross-subject generalisation, incorporating large-scale self-supervised pre-training, and enhancing the interpretability of neural representations, thereby further narrowing the gap between computational modelling approaches and their application in cognitive neuroscience.

Keywords:

Electroencephalography (EEG),Brain–Computer Interface (BCI), Transformer