.. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_plot_visualize_bad_epochs.py: =============================== Visualize bad sensors per trial =============================== This example demonstrates how to use :mod:`autoreject` to visualize the bad sensors in each trial .. code-block:: default # Author: Mainak Jas # Denis A. Engemann # License: BSD (3-clause) # sphinx_gallery_thumbnail_number = 2 First, we download the data from OpenfMRI. We will download the tarfile, extract the necessary files and delete the tar from the disk .. code-block:: default import os import tarfile import autoreject from autoreject.utils import fetch_file subject_id = 16 # OpenfMRI format of subject numbering src_url = ('http://openfmri.s3.amazonaws.com/tarballs/' 'ds117_R0.1.1_sub016_raw.tgz') subject = "sub%03d" % subject_id print("processing subject: %s" % subject) base_path = os.path.join( os.path.dirname(autoreject.__file__), '..', 'examples') target = os.path.join(base_path, 'ds117_R0.1.1_sub016_raw.tgz') if not os.path.exists(os.path.join(base_path, 'ds117')): if not os.path.exists(target): fetch_file(src_url, target) tf = tarfile.open(target) print('Extracting files. This may take a while ...') tf.extractall(path=base_path, members=tf.getmembers()[-25:-9:3]) os.remove(target) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none processing subject: sub016 We will create epochs with data starting 200 ms before trigger onset and continuing up to 800 ms after that. The data contains visual stimuli for famous faces, unfamiliar faces, as well as scrambled faces. .. code-block:: default tmin, tmax = -0.2, 0.8 events_id = {'famous/first': 5, 'famous/immediate': 6, 'famous/long': 7} Let us now load all the epochs into memory and concatenate them .. code-block:: default import mne # noqa epochs = list() for run in range(3, 7): run_fname = os.path.join(base_path, 'ds117', 'sub%03d' % subject_id, 'MEG', 'run_%02d_raw.fif' % run) raw = mne.io.read_raw_fif(run_fname, preload=True) raw.pick_types(eeg=True, meg=False, stim=True) # less memory + computation raw.filter(1., 40., l_trans_bandwidth=0.5, n_jobs=1, verbose='INFO') raw.set_channel_types({'EEG061': 'eog', 'EEG062': 'eog', 'EEG063': 'ecg', 'EEG064': 'misc'}) raw.rename_channels({'EEG061': 'EOG061', 'EEG062': 'EOG062', 'EEG063': 'ECG063', 'EEG064': 'MISC'}) events = mne.find_events(raw, stim_channel='STI101', consecutive='increasing', min_duration=0.003, verbose=True) # Read epochs mne.io.set_eeg_reference(raw) epoch = mne.Epochs(raw, events, events_id, tmin, tmax, proj=True, baseline=None, preload=False, reject=None, decim=4) epochs.append(epoch) # Same `dev_head_t` for all runs so that we can concatenate them. epoch.info['dev_head_t'] = epochs[0].info['dev_head_t'] epochs = mne.epochs.concatenate_epochs(epochs) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Opening raw data file /Users/mainak/Documents/github_repos/autoreject/autoreject/../examples/ds117/sub016/MEG/run_03_raw.fif... Read a total of 8 projection items: mag_ssp_upright.fif : PCA-mags-v1 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v2 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v3 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v4 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v5 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v1 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v2 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v3 (1 x 306) idle Range : 60500 ... 598399 = 55.000 ... 543.999 secs Ready. Current compensation grade : 0 Reading 0 ... 537899 = 0.000 ... 488.999 secs... Filtering raw data in 1 contiguous segment Setting up band-pass filter from 1 - 40 Hz FIR filter parameters --------------------- Designing a one-pass, zero-phase, non-causal bandpass filter: - Windowed time-domain design (firwin) method - Hamming window with 0.0194 passband ripple and 53 dB stopband attenuation - Lower passband edge: 1.00 - Lower transition bandwidth: 0.50 Hz (-6 dB cutoff frequency: 0.75 Hz) - Upper passband edge: 40.00 Hz - Upper transition bandwidth: 10.00 Hz (-6 dB cutoff frequency: 45.00 Hz) - Filter length: 7261 samples (6.601 sec) 294 events found Event IDs: [ 5 6 7 13 14 15 17 18 19 256 261 262 263 269 270 273 274 275 4096 4101 4102 4103 4109 4110 4111 4113 4114 4115] EEG channel type selected for re-referencing Applying average reference. Applying a custom EEG reference. 49 matching events found No baseline correction applied Not setting metadata 8 projection items activated Opening raw data file /Users/mainak/Documents/github_repos/autoreject/autoreject/../examples/ds117/sub016/MEG/run_04_raw.fif... Read a total of 8 projection items: mag_ssp_upright.fif : PCA-mags-v1 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v2 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v3 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v4 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v5 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v1 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v2 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v3 (1 x 306) idle Range : 137500 ... 680899 = 125.000 ... 618.999 secs Ready. Current compensation grade : 0 Reading 0 ... 543399 = 0.000 ... 493.999 secs... Filtering raw data in 1 contiguous segment Setting up band-pass filter from 1 - 40 Hz FIR filter parameters --------------------- Designing a one-pass, zero-phase, non-causal bandpass filter: - Windowed time-domain design (firwin) method - Hamming window with 0.0194 passband ripple and 53 dB stopband attenuation - Lower passband edge: 1.00 - Lower transition bandwidth: 0.50 Hz (-6 dB cutoff frequency: 0.75 Hz) - Upper passband edge: 40.00 Hz - Upper transition bandwidth: 10.00 Hz (-6 dB cutoff frequency: 45.00 Hz) - Filter length: 7261 samples (6.601 sec) 296 events found Event IDs: [ 5 6 7 13 14 15 17 18 19 256 261 262 263 269 270 273 274 275 4096 4101 4102 4103 4109 4110 4111 4113] EEG channel type selected for re-referencing Applying average reference. Applying a custom EEG reference. 49 matching events found No baseline correction applied Not setting metadata 8 projection items activated Opening raw data file /Users/mainak/Documents/github_repos/autoreject/autoreject/../examples/ds117/sub016/MEG/run_05_raw.fif... Read a total of 8 projection items: mag_ssp_upright.fif : PCA-mags-v1 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v2 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v3 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v4 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v5 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v1 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v2 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v3 (1 x 306) idle Range : 279400 ... 820599 = 254.000 ... 745.999 secs Ready. Current compensation grade : 0 Reading 0 ... 541199 = 0.000 ... 491.999 secs... Filtering raw data in 1 contiguous segment Setting up band-pass filter from 1 - 40 Hz FIR filter parameters --------------------- Designing a one-pass, zero-phase, non-causal bandpass filter: - Windowed time-domain design (firwin) method - Hamming window with 0.0194 passband ripple and 53 dB stopband attenuation - Lower passband edge: 1.00 - Lower transition bandwidth: 0.50 Hz (-6 dB cutoff frequency: 0.75 Hz) - Upper passband edge: 40.00 Hz - Upper transition bandwidth: 10.00 Hz (-6 dB cutoff frequency: 45.00 Hz) - Filter length: 7261 samples (6.601 sec) 295 events found Event IDs: [ 5 6 7 13 14 15 17 18 19 256 261 262 263 269 270 271 273 274 275 4096 4101 4102 4103 4109 4110 4111 4115 4357 4359] EEG channel type selected for re-referencing Applying average reference. Applying a custom EEG reference. 47 matching events found No baseline correction applied Not setting metadata 8 projection items activated Opening raw data file /Users/mainak/Documents/github_repos/autoreject/autoreject/../examples/ds117/sub016/MEG/run_06_raw.fif... Read a total of 8 projection items: mag_ssp_upright.fif : PCA-mags-v1 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v2 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v3 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v4 (1 x 306) idle mag_ssp_upright.fif : PCA-mags-v5 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v1 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v2 (1 x 306) idle grad_ssp_upright.fif : PCA-grad-v3 (1 x 306) idle Range : 140800 ... 680899 = 128.000 ... 618.999 secs Ready. Current compensation grade : 0 Reading 0 ... 540099 = 0.000 ... 490.999 secs... Filtering raw data in 1 contiguous segment Setting up band-pass filter from 1 - 40 Hz FIR filter parameters --------------------- Designing a one-pass, zero-phase, non-causal bandpass filter: - Windowed time-domain design (firwin) method - Hamming window with 0.0194 passband ripple and 53 dB stopband attenuation - Lower passband edge: 1.00 - Lower transition bandwidth: 0.50 Hz (-6 dB cutoff frequency: 0.75 Hz) - Upper passband edge: 40.00 Hz - Upper transition bandwidth: 10.00 Hz (-6 dB cutoff frequency: 45.00 Hz) - Filter length: 7261 samples (6.601 sec) 296 events found Event IDs: [ 5 6 7 13 14 15 17 18 19 256 261 262 263 269 270 271 273 274 275 4096 4101 4102 4103 4109 4110 4111 4114] EEG channel type selected for re-referencing Applying average reference. Applying a custom EEG reference. 48 matching events found No baseline correction applied Not setting metadata 8 projection items activated Loading data for 49 events and 1101 original time points ... 0 bad epochs dropped Loading data for 49 events and 1101 original time points ... 0 bad epochs dropped Loading data for 47 events and 1101 original time points ... 0 bad epochs dropped Loading data for 48 events and 1101 original time points ... 0 bad epochs dropped 193 matching events found No baseline correction applied Not setting metadata 0 bad epochs dropped Now, we apply autoreject .. code-block:: default from autoreject import AutoReject, compute_thresholds # noqa this_epoch = epochs['famous'] exclude = [] # XXX picks = mne.pick_types(epochs.info, meg=False, eeg=True, stim=False, eog=False, exclude=exclude) Note that :class:`autoreject.AutoReject` by design supports multiple channels. If no picks are passed separate solutions will be computed for each channel type and internally combines. This then readily supports cleaning unseen epochs from the different channel types used during fit. Here we only use a subset of channels to save time. Also note that once the parameters are learned, any data can be repaired that contains channels that were used during fit. This also means that time may be saved by fitting :class:`autoreject.AutoReject` on a representative subsample of the data. .. code-block:: default ar = AutoReject(picks=picks, random_state=42, n_jobs=1, verbose='tqdm') epochs_ar, reject_log = ar.fit_transform(this_epoch, return_log=True) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Running autoreject on ch_type=eeg Creating augmented epochs: 0%| | 0/70 [00:00` .. container:: sphx-glr-download :download:`Download Jupyter notebook: plot_visualize_bad_epochs.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_