International audience; Introduction: A BCI technology can operate in 3 different modalities: online mode which requires analyzingthe new real-time EEG data while acquiring it, offline mode where data is acquired and saved to a file and thenanalyzed afterwards (giving access to the data as a whole) and pseudo-online mode, which is a mix betweenthe previous two modes, where stored acquired data is processed as if in online mode, but with the relaxationof the real time constraint. Currently, many studies concerning Brain Computer Interfaces (BCI) are testedin the offline mode. This thus leads to unrealistic performance compared to real-life online scenarios. TheMOABB framework typically provides tools to evaluate algorithms in this offline mode. Other studiespropose online algorithms evaluation, but often do not disclose the datasets and/or nor the code used for dataanalysis. There are other frameworks for online processing, but they do not focus on the statisticalevaluation over several sessions/subjects as MOABB does.Material, Methods and Results: The objective of this research is to extend the current MOABB framework,which is currently limited to offline mode to allow comparison of different algorithms in a pseudo-online setting.We focus on asynchronous BCI where data is typically analyzed in overlapping sliding windows. This requiresthe addition of an idle state event to the datasets to mark signal pieces not related to an actual BCI task(s).Doing so generates datasets that are usually highly unbalanced in favor of this idle event, generating problemswith some of the standard metrics used in BCI evaluation. We thus use the normalized Matthews CorrelationCoefficient (nMCC) and the Information Transfer Rate (ITR). We applied this pseudo-online frameworkto evaluate the state-of-the-art algorithms over the last 15 years over several Motor Imagery (MI) datasetscomposed by several subjects.Discussion: Usually offline modality set an upper bound to the performances, while a online signal analysisapproaches generally produce results that are less accurate but more representative of a therapeutic applica-tion usage. The pseudo-online implementation can be used as a methodology that best approximates theonline process while still processing the data after complete recording. It still represents an upper bound onperformance (as real time time is not required) but a more realistic one that can be reached with more powerfulcomputing resources.Significance: The possibility of analyzing the performance of different algorithms first offline, followed bysubsequent validation of performance in pseudo-online mode, will be enable more representative reports on theperformance of classification algorithms for the BCI community.