Note
Go to the end to download the full example code.
Detect bad sensors using RANSAC#
This example demonstrates how to use RANSAC [1] from the PREP pipeline to
detect bad sensors and repair them. Note that this implementation in
autoreject [2] is an extension of the original implementation and
works for MEG sensors as well.
References#
# Author: Mainak Jas <mainak.jas@telecom-paristech.fr>
# License: BSD-3-Clause
For the purposes of this example, we shall use the MNE sample dataset. Therefore, let us make some MNE related imports.
import mne
from mne import io
from mne import Epochs
from mne.datasets import sample
Let us now read in the raw fif file for MNE sample dataset.
data_path = sample.data_path()
meg_path = data_path / 'MEG' / 'sample'
raw_fname = meg_path / 'sample_audvis_filt-0-40_raw.fif'
raw = io.read_raw_fif(raw_fname, preload=True)
Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/sample_audvis_filt-0-40_raw.fif...
Read a total of 4 projection items:
PCA-v1 (1 x 102) idle
PCA-v2 (1 x 102) idle
PCA-v3 (1 x 102) idle
Average EEG reference (1 x 60) idle
Range : 6450 ... 48149 = 42.956 ... 320.665 secs
Ready.
Reading 0 ... 41699 = 0.000 ... 277.709 secs...
We can then read in the events
event_fname = meg_path / 'sample_audvis_filt-0-40_raw-eve.fif'
event_id = {'Auditory/Left': 1}
tmin, tmax = -0.2, 0.5
events = mne.read_events(event_fname)
And pick MEG channels for repairing. Currently, autoreject can repair
only one channel type at a time.
raw.info['bads'] = []
Now, we can create epochs. The reject params will be set to None
because we do not want epochs to be dropped when instantiating
mne.Epochs.
raw.del_proj() # remove proj, don't proj while interpolating
epochs = Epochs(raw, events, event_id, tmin, tmax,
baseline=(None, 0), reject=None,
verbose=False, detrend=0, preload=True)
picks = mne.pick_types(epochs.info, meg='grad', eeg=False,
stim=False, eog=False,
include=[], exclude=[])
We import Ransac and run the familiar fit_transform method.
0%| | interpolating channels : 0/50 [00:00<?, ?it/s]
2%|▏ | interpolating channels : 1/50 [00:01<01:02, 1.28s/it]
4%|▍ | interpolating channels : 2/50 [00:02<01:01, 1.28s/it]
6%|▌ | interpolating channels : 3/50 [00:03<01:00, 1.29s/it]
8%|▊ | interpolating channels : 4/50 [00:05<00:59, 1.28s/it]
10%|█ | interpolating channels : 5/50 [00:06<00:57, 1.28s/it]
12%|█▏ | interpolating channels : 6/50 [00:07<00:56, 1.28s/it]
14%|█▍ | interpolating channels : 7/50 [00:08<00:55, 1.29s/it]
16%|█▌ | interpolating channels : 8/50 [00:10<00:54, 1.29s/it]
18%|█▊ | interpolating channels : 9/50 [00:11<00:52, 1.29s/it]
20%|██ | interpolating channels : 10/50 [00:12<00:51, 1.29s/it]
22%|██▏ | interpolating channels : 11/50 [00:14<00:50, 1.29s/it]
24%|██▍ | interpolating channels : 12/50 [00:15<00:49, 1.29s/it]
26%|██▌ | interpolating channels : 13/50 [00:16<00:47, 1.29s/it]
28%|██▊ | interpolating channels : 14/50 [00:18<00:46, 1.29s/it]
30%|███ | interpolating channels : 15/50 [00:19<00:45, 1.30s/it]
32%|███▏ | interpolating channels : 16/50 [00:20<00:44, 1.30s/it]
34%|███▍ | interpolating channels : 17/50 [00:22<00:42, 1.30s/it]
36%|███▌ | interpolating channels : 18/50 [00:23<00:41, 1.30s/it]
38%|███▊ | interpolating channels : 19/50 [00:24<00:40, 1.30s/it]
40%|████ | interpolating channels : 20/50 [00:26<00:39, 1.31s/it]
42%|████▏ | interpolating channels : 21/50 [00:27<00:37, 1.31s/it]
44%|████▍ | interpolating channels : 22/50 [00:28<00:36, 1.31s/it]
46%|████▌ | interpolating channels : 23/50 [00:30<00:35, 1.31s/it]
48%|████▊ | interpolating channels : 24/50 [00:31<00:34, 1.31s/it]
50%|█████ | interpolating channels : 25/50 [00:32<00:32, 1.31s/it]
52%|█████▏ | interpolating channels : 26/50 [00:34<00:31, 1.31s/it]
54%|█████▍ | interpolating channels : 27/50 [00:35<00:30, 1.31s/it]
56%|█████▌ | interpolating channels : 28/50 [00:36<00:28, 1.31s/it]
58%|█████▊ | interpolating channels : 29/50 [00:37<00:27, 1.31s/it]
60%|██████ | interpolating channels : 30/50 [00:39<00:26, 1.31s/it]
62%|██████▏ | interpolating channels : 31/50 [00:40<00:25, 1.32s/it]
64%|██████▍ | interpolating channels : 32/50 [00:41<00:23, 1.32s/it]
66%|██████▌ | interpolating channels : 33/50 [00:43<00:22, 1.32s/it]
68%|██████▊ | interpolating channels : 34/50 [00:44<00:21, 1.32s/it]
70%|███████ | interpolating channels : 35/50 [00:45<00:19, 1.32s/it]
72%|███████▏ | interpolating channels : 36/50 [00:47<00:18, 1.32s/it]
74%|███████▍ | interpolating channels : 37/50 [00:48<00:17, 1.32s/it]
76%|███████▌ | interpolating channels : 38/50 [00:49<00:15, 1.32s/it]
78%|███████▊ | interpolating channels : 39/50 [00:51<00:14, 1.32s/it]
80%|████████ | interpolating channels : 40/50 [00:52<00:13, 1.32s/it]
82%|████████▏ | interpolating channels : 41/50 [00:53<00:11, 1.32s/it]
84%|████████▍ | interpolating channels : 42/50 [00:55<00:10, 1.32s/it]
86%|████████▌ | interpolating channels : 43/50 [00:56<00:09, 1.32s/it]
88%|████████▊ | interpolating channels : 44/50 [00:57<00:07, 1.32s/it]
90%|█████████ | interpolating channels : 45/50 [00:59<00:06, 1.32s/it]
92%|█████████▏| interpolating channels : 46/50 [01:00<00:05, 1.31s/it]
94%|█████████▍| interpolating channels : 47/50 [01:01<00:03, 1.32s/it]
96%|█████████▌| interpolating channels : 48/50 [01:03<00:02, 1.32s/it]
98%|█████████▊| interpolating channels : 49/50 [01:04<00:01, 1.32s/it]
100%|██████████| interpolating channels : 50/50 [01:05<00:00, 1.32s/it]
100%|██████████| interpolating channels : 50/50 [01:05<00:00, 1.31s/it]
[Parallel(n_jobs=1)]: Done 1 tasks | elapsed: 1.1min
[Parallel(n_jobs=1)]: Done 1 tasks | elapsed: 1.1min
0%| | Iterating epochs : 0/72 [00:00<?, ?it/s]
1%|▏ | Iterating epochs : 1/72 [00:00<00:05, 13.48it/s]
3%|▎ | Iterating epochs : 2/72 [00:00<00:03, 18.00it/s]
4%|▍ | Iterating epochs : 3/72 [00:00<00:03, 20.63it/s]
6%|▌ | Iterating epochs : 4/72 [00:00<00:03, 22.35it/s]
7%|▋ | Iterating epochs : 5/72 [00:00<00:02, 23.45it/s]
8%|▊ | Iterating epochs : 6/72 [00:00<00:02, 24.28it/s]
10%|▉ | Iterating epochs : 7/72 [00:00<00:02, 24.94it/s]
11%|█ | Iterating epochs : 8/72 [00:00<00:02, 25.46it/s]
12%|█▎ | Iterating epochs : 9/72 [00:00<00:02, 25.87it/s]
14%|█▍ | Iterating epochs : 10/72 [00:00<00:02, 26.19it/s]
15%|█▌ | Iterating epochs : 11/72 [00:00<00:02, 26.44it/s]
17%|█▋ | Iterating epochs : 12/72 [00:00<00:02, 26.66it/s]
18%|█▊ | Iterating epochs : 13/72 [00:00<00:02, 26.84it/s]
19%|█▉ | Iterating epochs : 14/72 [00:00<00:02, 27.02it/s]
21%|██ | Iterating epochs : 15/72 [00:00<00:02, 27.17it/s]
22%|██▏ | Iterating epochs : 16/72 [00:00<00:02, 27.29it/s]
24%|██▎ | Iterating epochs : 17/72 [00:00<00:02, 27.42it/s]
25%|██▌ | Iterating epochs : 18/72 [00:00<00:01, 27.52it/s]
26%|██▋ | Iterating epochs : 19/72 [00:00<00:01, 27.63it/s]
28%|██▊ | Iterating epochs : 20/72 [00:00<00:01, 27.72it/s]
29%|██▉ | Iterating epochs : 21/72 [00:00<00:01, 27.81it/s]
31%|███ | Iterating epochs : 22/72 [00:00<00:01, 27.87it/s]
32%|███▏ | Iterating epochs : 23/72 [00:00<00:01, 27.92it/s]
33%|███▎ | Iterating epochs : 24/72 [00:00<00:01, 27.98it/s]
35%|███▍ | Iterating epochs : 25/72 [00:00<00:01, 28.06it/s]
36%|███▌ | Iterating epochs : 26/72 [00:00<00:01, 28.10it/s]
38%|███▊ | Iterating epochs : 27/72 [00:00<00:01, 28.16it/s]
39%|███▉ | Iterating epochs : 28/72 [00:01<00:01, 28.19it/s]
40%|████ | Iterating epochs : 29/72 [00:01<00:01, 28.19it/s]
42%|████▏ | Iterating epochs : 30/72 [00:01<00:01, 28.18it/s]
43%|████▎ | Iterating epochs : 31/72 [00:01<00:01, 28.21it/s]
44%|████▍ | Iterating epochs : 32/72 [00:01<00:01, 28.20it/s]
46%|████▌ | Iterating epochs : 33/72 [00:01<00:01, 28.20it/s]
47%|████▋ | Iterating epochs : 34/72 [00:01<00:01, 28.22it/s]
49%|████▊ | Iterating epochs : 35/72 [00:01<00:01, 28.25it/s]
50%|█████ | Iterating epochs : 36/72 [00:01<00:01, 28.26it/s]
51%|█████▏ | Iterating epochs : 37/72 [00:01<00:01, 28.29it/s]
53%|█████▎ | Iterating epochs : 38/72 [00:01<00:01, 28.32it/s]
54%|█████▍ | Iterating epochs : 39/72 [00:01<00:01, 28.35it/s]
56%|█████▌ | Iterating epochs : 40/72 [00:01<00:01, 28.35it/s]
57%|█████▋ | Iterating epochs : 41/72 [00:01<00:01, 28.35it/s]
58%|█████▊ | Iterating epochs : 42/72 [00:01<00:01, 28.37it/s]
60%|█████▉ | Iterating epochs : 43/72 [00:01<00:01, 28.39it/s]
61%|██████ | Iterating epochs : 44/72 [00:01<00:00, 28.38it/s]
62%|██████▎ | Iterating epochs : 45/72 [00:01<00:00, 28.40it/s]
64%|██████▍ | Iterating epochs : 46/72 [00:01<00:00, 28.41it/s]
65%|██████▌ | Iterating epochs : 47/72 [00:01<00:00, 28.42it/s]
67%|██████▋ | Iterating epochs : 48/72 [00:01<00:00, 28.44it/s]
68%|██████▊ | Iterating epochs : 49/72 [00:01<00:00, 28.46it/s]
69%|██████▉ | Iterating epochs : 50/72 [00:01<00:00, 28.47it/s]
71%|███████ | Iterating epochs : 51/72 [00:01<00:00, 28.50it/s]
72%|███████▏ | Iterating epochs : 52/72 [00:01<00:00, 28.51it/s]
74%|███████▎ | Iterating epochs : 53/72 [00:01<00:00, 28.52it/s]
75%|███████▌ | Iterating epochs : 54/72 [00:01<00:00, 28.53it/s]
76%|███████▋ | Iterating epochs : 55/72 [00:01<00:00, 28.53it/s]
78%|███████▊ | Iterating epochs : 56/72 [00:01<00:00, 28.51it/s]
79%|███████▉ | Iterating epochs : 57/72 [00:02<00:00, 28.53it/s]
81%|████████ | Iterating epochs : 58/72 [00:02<00:00, 28.53it/s]
82%|████████▏ | Iterating epochs : 59/72 [00:02<00:00, 28.50it/s]
83%|████████▎ | Iterating epochs : 60/72 [00:02<00:00, 28.50it/s]
85%|████████▍ | Iterating epochs : 61/72 [00:02<00:00, 28.51it/s]
86%|████████▌ | Iterating epochs : 62/72 [00:02<00:00, 28.50it/s]
88%|████████▊ | Iterating epochs : 63/72 [00:02<00:00, 28.52it/s]
89%|████████▉ | Iterating epochs : 64/72 [00:02<00:00, 28.53it/s]
90%|█████████ | Iterating epochs : 65/72 [00:02<00:00, 28.55it/s]
92%|█████████▏| Iterating epochs : 66/72 [00:02<00:00, 28.57it/s]
93%|█████████▎| Iterating epochs : 67/72 [00:02<00:00, 28.58it/s]
94%|█████████▍| Iterating epochs : 68/72 [00:02<00:00, 28.59it/s]
96%|█████████▌| Iterating epochs : 69/72 [00:02<00:00, 28.61it/s]
97%|█████████▋| Iterating epochs : 70/72 [00:02<00:00, 28.61it/s]
99%|█████████▊| Iterating epochs : 71/72 [00:02<00:00, 28.63it/s]
100%|██████████| Iterating epochs : 72/72 [00:02<00:00, 28.63it/s]
100%|██████████| Iterating epochs : 72/72 [00:02<00:00, 28.17it/s]
[Parallel(n_jobs=1)]: Done 1 tasks | elapsed: 2.6s
[Parallel(n_jobs=1)]: Done 1 tasks | elapsed: 2.6s
[Done]
Setting channel interpolation method to {'eeg': 'spline', 'meg': 'MNE'}.
Interpolating bad channels.
Automatic origin fit: head of radius 91.0 mm
Computing dot products for 292 MEG channels...
Computing cross products for 292 → 14 MEG channels...
Preparing the mapping matrix...
Truncating at 80/292 components to omit less than 0.0001 (9.4e-05)
We can also get the list of bad channels computed by Ransac.
print('\n'.join(ransac.bad_chs_))
MEG 0722
MEG 0813
MEG 0812
MEG 0822
MEG 0912
MEG 0942
MEG 0943
MEG 1013
MEG 1023
MEG 1022
MEG 1032
MEG 1422
MEG 1633
MEG 2443
Then we compute the evoked before and after interpolation.
We will manually mark the bad channels just for plotting.
evoked.info['bads'] = ['MEG 2443']
evoked_clean.info['bads'] = ['MEG 2443']
Let us plot the results.
from autoreject.utils import set_matplotlib_defaults # noqa
import matplotlib.pyplot as plt # noqa
set_matplotlib_defaults(plt)
fig, axes = plt.subplots(2, 1, figsize=(6, 6))
for ax in axes:
ax.tick_params(axis='x', which='both', bottom='off', top='off')
ax.tick_params(axis='y', which='both', left='off', right='off')
ylim = dict(grad=(-170, 200))
evoked.pick_types(meg='grad', exclude=[])
evoked.plot(exclude=[], axes=axes[0], ylim=ylim, show=False)
axes[0].set_title('Before RANSAC')
evoked_clean.pick_types(meg='grad', exclude=[])
evoked_clean.plot(exclude=[], axes=axes[1], ylim=ylim)
axes[1].set_title('After RANSAC')
fig.tight_layout()
NOTE: pick_types() is a legacy function. New code should use inst.pick(...).
NOTE: pick_types() is a legacy function. New code should use inst.pick(...).
/home/circleci/project/examples/plot_ransac.py:115: UserWarning: This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.
fig.tight_layout()
To top things up, we can also visualize the bad sensors for each trial using a heatmap.
ch_names = [epochs.ch_names[ii] for ii in ransac.picks][7::10]
fig, ax = plt.subplots(1, 1, figsize=(12, 6))
ax.imshow(ransac.bad_log, cmap='Reds',
interpolation='nearest')
ax.grid(False)
ax.set_xlabel('Sensors')
ax.set_ylabel('Trials')
plt.setp(ax, xticks=range(7, len(ransac.picks), 10),
xticklabels=ch_names)
plt.setp(ax.get_yticklabels(), rotation=0)
plt.setp(ax.get_xticklabels(), rotation=90)
ax.tick_params(axis=u'both', which=u'both', length=0)
fig.tight_layout(rect=[None, None, None, 1.1])
plt.show()
Total running time of the script: (1 minutes 14.274 seconds)