Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing and Cognitive Mapping of the Human Brain
AnaLuPinho
60 views
124 slides
Jun 05, 2024
Slide 1 of 124
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
About This Presentation
Functional Magnetic Resonance Imaging (fMRI) provides means to characterize brain activations in response to behavior. However, cognitive neuroscience has been limited to group-level effects referring to the performance of specific tasks. To obtain the functional profile of elementary cognitive mec...
Slide Content
@ALuisaPinho@[email protected]
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Deep Behavioral Phenotyping in Systems Neuroscience
for Functional Atlasing and Cognitive Mapping of the
Human Brain
Ana Lu´ısa Pinho, Ph.D.
BrainsCAN Postdoctoral Fellow
Western University, London Ontario, Canada
30
th
of May, 2024
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Deep Behavioral Phenotyping in Systems Neuroscience
for Functional Atlasing and Cognitive Mapping of the
Human Brain
Ana Lu´ısa Pinho, Ph.D.
BrainsCAN Postdoctoral Fellow
Western University, London Ontario, Canada
30
th
of May, 2024
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Summary
2/44
Summary
2/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Summary
2/44
Summary
2/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Summary
2/44
Summary
2/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Summary
2/44
Summary
2/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Summary
2/44
Summary
2/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Overview
Overview
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Correspondence Problem
In cognitive neuroscience:
Brain systems
⇐⇒
Mental functions
Experiments typically shall:
•tackle one psychological domain
•be specific enough to accurately isolate brain processes
⇓
Very hard to achieve!
Lack of generality.
4/44
The Correspondence Problem
In cognitive neuroscience:
Brain systems
⇐⇒
Mental functions
Experiments typically shall:
•tackle one psychological domain
•be specific enough to accurately isolate brain processes
⇓
Very hard to achieve!
Lack of generality.
4/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Correspondence Problem
In cognitive neuroscience:
Brain systems
⇐⇒
Mental functions
Experiments typically shall:
•tackle one psychological domain
•be specific enough to accurately isolate brain processes
⇓
Very hard to achieve!
Lack of generality.
4/44
The Correspondence Problem
In cognitive neuroscience:
Brain systems
⇐⇒
Mental functions
Experiments typically shall:
•tackle one psychological domain
•be specific enough to accurately isolate brain processes
⇓
Very hard to achieve!
Lack of generality.
4/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Correspondence Problem
In cognitive neuroscience:
Brain systems
⇐⇒
Mental functions
Experiments typically shall:
•tackle one psychological domain
•be specific enough to accurately isolate brain processes
⇓
Very hard to achieve!
Lack of generality.
4/44
The Correspondence Problem
In cognitive neuroscience:
Brain systems
⇐⇒
Mental functions
Experiments typically shall:
•tackle one psychological domain
•be specific enough to accurately isolate brain processes
⇓
Very hard to achieve!
Lack of generality.
4/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Correspondence Problem
In cognitive neuroscience:
Brain systems
⇐⇒
Mental functions
Experiments typically shall:
•tackle one psychological domain
•be specific enough to accurately isolate brain processes
⇓
Very hard to achieve!
Lack of generality.
4/44
The Correspondence Problem
In cognitive neuroscience:
Brain systems
⇐⇒
Mental functions
Experiments typically shall:
•tackle one psychological domain
•be specific enough to accurately isolate brain processes
⇓
Very hard to achieve!
Lack of generality.
4/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Correspondence Problem
In cognitive neuroscience:
Brain systems
⇐⇒
Mental functions
Task-fMRI experiments allow to:
•link brain systems to behavior
•map neural activity at mm-scale
4/44
The Correspondence Problem
In cognitive neuroscience:
Brain systems
⇐⇒
Mental functions
Task-fMRI experiments allow to:
•link brain systems to behavior
•map neural activity at mm-scale
4/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Principles of Functional Organization
No unique task-activation
↘
Functional specialization
↙
Sparse-coding representation of spatial brain function
Two Neuroscience Principles: •Functional Segregation: brain territories are formed of basic functional
components (Tononi, G.et al.1998)
•Functional Degeneracy: a particular function may recruit different networks
across subjects (Noppeney, U.et al.2004)
5/44
Principles of Functional Organization
No unique task-activation
↘
Functional specialization
↙
Sparse-coding representation of spatial brain function
Two Neuroscience Principles: •Functional Segregation: brain territories are formed of basic functional
components (Tononi, G.et al.1998)
•Functional Degeneracy: a particular function may recruit different networks
across subjects (Noppeney, U.et al.2004)
5/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Principles of Functional Organization
No unique task-activation
↘
Functional specialization
↙
Sparse-coding representation of spatial brain function
Two Neuroscience Principles: •Functional Segregation: brain territories are formed of basic functional
components (Tononi, G.et al.1998)
•Functional Degeneracy: a particular function may recruit different networks
across subjects (Noppeney, U.et al.2004)
5/44
Principles of Functional Organization
No unique task-activation
↘
Functional specialization
↙
Sparse-coding representation of spatial brain function
Two Neuroscience Principles: •Functional Segregation: brain territories are formed of basic functional
components (Tononi, G.et al.1998)
•Functional Degeneracy: a particular function may recruit different networks
across subjects (Noppeney, U.et al.2004)
5/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Principles of Functional Organization
No unique task-activation
↘
Functional specialization
↙
Sparse-coding representation of spatial brain function
Two Neuroscience Principles: •Functional Segregation: brain territories are formed of basic functional
components (Tononi, G.et al.1998)
•Functional Degeneracy: a particular function may recruit different networks
across subjects (Noppeney, U.et al.2004)
5/44
Principles of Functional Organization
No unique task-activation
↘
Functional specialization
↙
Sparse-coding representation of spatial brain function
Two Neuroscience Principles: •Functional Segregation: brain territories are formed of basic functional
components (Tononi, G.et al.1998)
•Functional Degeneracy: a particular function may recruit different networks
across subjects (Noppeney, U.et al.2004)
5/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Principles of Functional Organization
No unique task-activation
↘
Functional specialization
↙
Sparse-coding representation of spatial brain function
Two Neuroscience Principles: •Functional Segregation: brain territories are formed of basic functional
components (Tononi, G.et al.1998)
•Functional Degeneracy: a particular function may recruit different networks
across subjects (Noppeney, U.et al.2004)
5/44
Principles of Functional Organization
No unique task-activation
↘
Functional specialization
↙
Sparse-coding representation of spatial brain function
Two Neuroscience Principles: •Functional Segregation: brain territories are formed of basic functional
components (Tononi, G.et al.1998)
•Functional Degeneracy: a particular function may recruit different networks
across subjects (Noppeney, U.et al.2004)
5/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
How can we clearly describe functional specificity of brain regions based on
concurrent activation?
6/44
How can we clearly describe functional specificity of brain regions based on
concurrent activation?
6/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
How can we clearly describe functional specificity of brain regions based on
concurrent activation?
6/44
How can we clearly describe functional specificity of brain regions based on
concurrent activation?
6/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Deep Behavioral Phenotyping (DBP)
Subject1...Subjectn
↓ ↓
6/44
Deep Behavioral Phenotyping (DBP)
Subject1...Subjectn
↓ ↓
6/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
Deep-Imaging Datasets of task-fMRI
•studyforrest →N≈20,>10h/subject
Hanke, M.et al.Sci Data (2014,2016) / Hanke, M.et al.F1000Research (2015)
Sengupta, A.et al.Sci Data (2016a,b) / Sengupta, A.et al.Neuroimage (2017)
•IBC: Individual Brain Charting →N= 12, 75h/subject, 86 tasks
Pinho, A. L.et al.Sci Data (2018, 2020)
Pinho, A. L.et al.(2023) [preprint; accepted in Sci Data]
•MDTB: Multi-Domain Task Battery→N= 31, 4h/subject, 17 tasks
King, M.et al.NatNeurosci (2019)
•Courtois NeuroMod →N= 6, 91h/subject•OMMABA: Open Multimodal Music →N≈44,∼2h/subject, 3 tasks
and Auditory Brain Archive
7/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Functional Fingerprint
•Vector of activation values that define the functional prototype of a certain
region-of-interest
Thirion, B., Thual, A., &Pinho, A. L.Curr Opin Behav Sci (2021)
8/44
Functional Fingerprint
•Vector of activation values that define the functional prototype of a certain
region-of-interest
Thirion, B., Thual, A., &Pinho, A. L.Curr Opin Behav Sci (2021)
8/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Multi-Modal Paradigms
•Paradigms spanning over a large array of cognitive domains
Domain-general battery
ARCHI Standard
Domain-specific batteries
•ARCHI Spatial
•ARCHI Social
•ARCHI Emotional
•fast event-related paradigms
•100 trials of 5-8 conditions in
each 5min run.
Pinel, P.et al.BMC Neurosci (2007)
Pinho, A. L.et al.Sci Data (2018)
Pinel, P.et al.Neuroimage (2019)
9/44
Multi-Modal Paradigms
•Paradigms spanning over a large array of cognitive domains
Domain-general battery
ARCHI Standard
Domain-specific batteries
•ARCHI Spatial
•ARCHI Social
•ARCHI Emotional
•fast event-related paradigms
•100 trials of 5-8 conditions in
each 5min run.
Pinel, P.et al.BMC Neurosci (2007)
Pinho, A. L.et al.Sci Data (2018)
Pinel, P.et al.Neuroimage (2019)
9/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Multi-Modal Paradigms
•Paradigms spanning over a large array of cognitive domains
Domain-general battery
ARCHI Standard
Domain-specific batteries
•ARCHI Spatial
•ARCHI Social
•ARCHI Emotional
•fast event-related paradigms
•100 trials of 5-8 conditions in
each 5min run.
Pinel, P.et al.BMC Neurosci (2007)
Pinho, A. L.et al.Sci Data (2018)
Pinel, P.et al.Neuroimage (2019)
9/44
Multi-Modal Paradigms
•Paradigms spanning over a large array of cognitive domains
Domain-general battery
ARCHI Standard
Domain-specific batteries
•ARCHI Spatial
•ARCHI Social
•ARCHI Emotional
•fast event-related paradigms
•100 trials of 5-8 conditions in
each 5min run.
Pinel, P.et al.BMC Neurosci (2007)
Pinho, A. L.et al.Sci Data (2018)
Pinel, P.et al.Neuroimage (2019)
9/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
Stimuli: Ecological and Rich on Behavioral Features
Capitalize on features extraction
−→Break down complex behavior into smaller parts
Create ecological stimuli−→Ability to define independent variables
With Musical Paradigms, we can: •use metrics of complexity to decompose excerpts of musical performance;
•useMusic Information Retrieval(MIR) techniques to extract features from
music-perceptual tasks;
•link between many cognitive domains
(ex: sensorimotor coordination, timing and time perception, language, reward, flow
and creativity).
10/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Study on Musical Creativity
Can long-term extensive training influence the neural mechanisms responsible
for musical creativity?
Fronto-parietal activity nega-
tively correlated with experience
in improvisational practice asso-
ciated with music.
Pinho, A. L.et al.J Neurosci (2014)
•Right Angular Gyrus
•Right Dorsolateral Prefrontal Cortex
•Right Insula
•Right Inferior Frontal Gyrus
•Results controlled for age and
conventional musical training
•Results controlled for complexity of
musical pieces played
11/44
Study on Musical Creativity
Can long-term extensive training influence the neural mechanisms responsible
for musical creativity?
Fronto-parietal activity nega-
tively correlated with experience
in improvisational practice asso-
ciated with music.
Pinho, A. L.et al.J Neurosci (2014)
•Right Angular Gyrus
•Right Dorsolateral Prefrontal Cortex
•Right Insula
•Right Inferior Frontal Gyrus
•Results controlled for age and
conventional musical training
•Results controlled for complexity of
musical pieces played
11/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Axis 1:
Multi-task neuroimaging framework to investigate
functional specificity of brain regions
Axis 1:
Multi-task neuroimaging framework to investigate
functional specificity of brain regions
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Precision in Cognitive Mapping
Data-pooling analysis
•Meta-analysis:
pooling data derivatives
•Mega-analysis:
pooling raw data
Requisites for cognitive mapping
Minimize variability of Successful interpretation of
spatial location combined results
same processing no loss of info from sparse
routines
peak-coord. representation
same experimental consistency of
settings
(
()
cognitive annotations
low inter-subject variability
sufficient multi-task data
Large-scale repositories:
•OpenNeuro
•NeuroVault
•EBRAINS
Individual analysis:
•Fedorenko, E.et al.(2011)
•Hanke, M.et al.(2014)
•Pinho, A. L.et al.(2021)
Large-scale datasets:
•HCP
•UK Biobank
•CONNECT/Archi
13/44
Precision in Cognitive Mapping
Data-pooling analysis
•Meta-analysis:
pooling data derivatives
•Mega-analysis:
pooling raw data
Requisites for cognitive mapping
Minimize variability of Successful interpretation of
spatial location combined results
same processing no loss of info from sparse
routines
peak-coord. representation
same experimental consistency of
settings
(
()
cognitive annotations
low inter-subject variability
sufficient multi-task data
Large-scale repositories:
•OpenNeuro
•NeuroVault
•EBRAINS
Individual analysis:
•Fedorenko, E.et al.(2011)
•Hanke, M.et al.(2014)
•Pinho, A. L.et al.(2021)
Large-scale datasets:
•HCP
•UK Biobank
•CONNECT/Archi
13/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Precision in Cognitive Mapping
Data-pooling analysis
•Meta-analysis:
pooling data derivatives
•Mega-analysis:
pooling raw data
Requisites for cognitive mapping
Minimize variability of Successful interpretation of
spatial location combined results
same processing no loss of info from sparse
routines
peak-coord. representation
same experimental consistency of
settings
(
()
cognitive annotations
low inter-subject variability
sufficient multi-task data
Large-scale repositories:
•OpenNeuro
•NeuroVault
•EBRAINS
Individual analysis:
•Fedorenko, E.et al.(2011)
•Hanke, M.et al.(2014)
•Pinho, A. L.et al.(2021)
Large-scale datasets:
•HCP
•UK Biobank
•CONNECT/Archi
Deep datasets:
•
• • • •
13/44
Precision in Cognitive Mapping
Data-pooling analysis
•Meta-analysis:
pooling data derivatives
•Mega-analysis:
pooling raw data
Requisites for cognitive mapping
Minimize variability of Successful interpretation of
spatial location combined results
same processing no loss of info from sparse
routines
peak-coord. representation
same experimental consistency of
settings
(
()
cognitive annotations
low inter-subject variability
sufficient multi-task data
Large-scale repositories:
•OpenNeuro
•NeuroVault
•EBRAINS
Individual analysis:
•Fedorenko, E.et al.(2011)
•Hanke, M.et al.(2014)
•Pinho, A. L.et al.(2021)
Large-scale datasets:
•HCP
•UK Biobank
•CONNECT/Archi
Deep datasets:
•
• • • •
13/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Precision in Cognitive Mapping
Data-pooling analysis
•Meta-analysis:
pooling data derivatives
•Mega-analysis:
pooling raw data
Requisites for cognitive mapping
Minimize variability of Successful interpretation of
spatial location combined results
same processing no loss of info from sparse
routines
peak-coord. representation
same experimental consistency of
settings
(
()
cognitive annotations
low inter-subject variability
sufficient multi-task data
Large-scale repositories:
•OpenNeuro
•NeuroVault
•EBRAINS
Individual analysis:
•Fedorenko, E.et al.(2011)
•Hanke, M.et al.(2014)
•Pinho, A. L.et al.(2021)
Large-scale datasets:
•HCP
•UK Biobank
•CONNECT/Archi
Deep datasets:
•
• • • •
Adopt a Multi-Task
Neuroimaging Framework.13/44
Precision in Cognitive Mapping
Data-pooling analysis
•Meta-analysis:
pooling data derivatives
•Mega-analysis:
pooling raw data
Requisites for cognitive mapping
Minimize variability of Successful interpretation of
spatial location combined results
same processing no loss of info from sparse
routines
peak-coord. representation
same experimental consistency of
settings
(
()
cognitive annotations
low inter-subject variability
sufficient multi-task data
Large-scale repositories:
•OpenNeuro
•NeuroVault
•EBRAINS
Individual analysis:
•Fedorenko, E.et al.(2011)
•Hanke, M.et al.(2014)
•Pinho, A. L.et al.(2021)
Large-scale datasets:
•HCP
•UK Biobank
•CONNECT/Archi
Deep datasets:
•
• • • •
Adopt a Multi-Task
Neuroimaging Framework.13/44
SummaryOverviewMulti-task Framework Ontologies and Taxonomies Op en ScienceConclusion and Acknowledgments
DBP on Functional Atlasing
The Principle of Dictionary Learning
Thirion, B., Thual, A., & Pinho, A. L. Curr Opin Behav Sci (2021)
14/44
DBP on Functional Atlasing
The Principle of Dictionary Learning
Thirion, B., Thual, A., & Pinho, A. L. Curr Opin Behav Sci (2021)
14/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Dictionary of Cognitive Components
Multi-subject, sparse dictionary learning:
min
(U
s
)s=1...n,V∈C
n
X
s=1
ı
∥X
s
−U
s
V∥
2
+λ∥U
s
∥1
ȷ
,
withX
s
p×c,U
s
p×k
andVk×c
•Functional correspondence: dictionary
of functional profiles (V) common to
all subjects
•Sparsity:ℓ1−norm penalty and
U
s
≥0,∀s∈[n]
Pinho, A. L.et al.Hum Brain Mapp(2021) n= 13
Components are consistently mapped across subjects.
Variability of topographies linked to individual differences.
Pinho, A. L.et al.Hum Brain Mapp(2021)
15/44
Dictionary of Cognitive Components
Multi-subject, sparse dictionary learning:
min
(U
s
)s=1...n,V∈C
n
X
s=1
ı
∥X
s
−U
s
V∥
2
+λ∥U
s
∥1
ȷ
,
withX
s
p×c,U
s
p×k
andVk×c
•Functional correspondence: dictionary
of functional profiles (V) common to
all subjects
•Sparsity:ℓ1−norm penalty and
U
s
≥0,∀s∈[n]
Pinho, A. L.et al.Hum Brain Mapp(2021) n= 13
Components are consistently mapped across subjects.
Variability of topographies linked to individual differences.
Pinho, A. L.et al.Hum Brain Mapp(2021)
15/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Dictionary of Cognitive Components
Multi-subject, sparse dictionary learning:
min
(U
s
)s=1...n,V∈C
n
X
s=1
ı
∥X
s
−U
s
V∥
2
+λ∥U
s
∥1
ȷ
,
withX
s
p×c,U
s
p×k
andVk×c
•Functional correspondence: dictionary
of functional profiles (V) common to
all subjects
•Sparsity:ℓ1−norm penalty and
U
s
≥0,∀s∈[n]
Pinho, A. L.et al.Hum Brain Mapp(2021) n= 13
Components are consistently mapped across subjects.
Variability of topographies linked to individual differences.
Pinho, A. L.et al.Hum Brain Mapp(2021)
15/44
Dictionary of Cognitive Components
Multi-subject, sparse dictionary learning:
min
(U
s
)s=1...n,V∈C
n
X
s=1
ı
∥X
s
−U
s
V∥
2
+λ∥U
s
∥1
ȷ
,
withX
s
p×c,U
s
p×k
andVk×c
•Functional correspondence: dictionary
of functional profiles (V) common to
all subjects
•Sparsity:ℓ1−norm penalty and
U
s
≥0,∀s∈[n]
Pinho, A. L.et al.Hum Brain Mapp(2021) n= 13
Components are consistently mapped across subjects.
Variability of topographies linked to individual differences.
Pinho, A. L.et al.Hum Brain Mapp(2021)
15/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Dictionary of Cognitive Components
Multi-subject, sparse dictionary learning:
min
(U
s
)s=1...n,V∈C
n
X
s=1
ı
∥X
s
−U
s
V∥
2
+λ∥U
s
∥1
ȷ
,
withX
s
p×c,U
s
p×k
andVk×c
•Functional correspondence: dictionary
of functional profiles (V) common to
all subjects
•Sparsity:ℓ1−norm penalty and
U
s
≥0,∀s∈[n]
Components are consistently mapped across subjects.Pinho, A. L.et al.Hum Brain Mapp(2021) n= 130.25 0.30 0.35 0.40 0.45 0.50 0.55
Intra-subject
correlation
Inter-subject
correlation
Correlations of the dictionary components on split-half data
Variability of topographies linked to individual differences.
Pinho, A. L.et al.Hum Brain Mapp(2021)
15/44
Dictionary of Cognitive Components
Multi-subject, sparse dictionary learning:
min
(U
s
)s=1...n,V∈C
n
X
s=1
ı
∥X
s
−U
s
V∥
2
+λ∥U
s
∥1
ȷ
,
withX
s
p×c,U
s
p×k
andVk×c
•Functional correspondence: dictionary
of functional profiles (V) common to
all subjects
•Sparsity:ℓ1−norm penalty and
U
s
≥0,∀s∈[n]
Components are consistently mapped across subjects.Pinho, A. L.et al.Hum Brain Mapp(2021) n= 130.25 0.30 0.35 0.40 0.45 0.50 0.55
Intra-subject
correlation
Inter-subject
correlation
Correlations of the dictionary components on split-half data
Variability of topographies linked to individual differences.
Pinho, A. L.et al.Hum Brain Mapp(2021)
15/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Dictionary of Cognitive Components
Multi-subject, sparse dictionary learning:
min
(U
s
)s=1...n,V∈C
n
X
s=1
ı
∥X
s
−U
s
V∥
2
+λ∥U
s
∥1
ȷ
,
withX
s
p×c,U
s
p×k
andVk×c
•Functional correspondence: dictionary
of functional profiles (V) common to
all subjects
•Sparsity:ℓ1−norm penalty and
U
s
≥0,∀s∈[n]
Components are consistently mapped across subjects.Variability of topographies linked to individual differences.0.25 0.30 0.35 0.40 0.45 0.50 0.55
Intra-subject
correlation
Inter-subject
correlation
Correlations of the dictionary components on split-half data 0.000.250.50
read sentence vs. listen to sentence
read sentence vs. checkerboard
left hand vs. right hand
horizontal checkerboard vs. vertical checkerboard
mental subtraction vs. sentence
saccade vs. fixation
guess which hand vs. hand palm or back
object grasping vs. mimic orientation
mental motion vs. random motion
false-belief story vs. mechanistic story
false-belief tale vs. mechanistic tale
expression intention vs. expression gender
face trusty vs. face gender
face image vs. shape outline
punishment vs. reward
0.000.250.50
tongue vs. any motion
right foot vs. any motion
left foot vs. any motion
right hand vs. any motion
left hand vs. any motion
tale vs. mental addition
relational processing vs. visual matching
mental motion vs. random motion
tool image vs. any image
place image vs. any image
face image vs. any image
body image vs. any image
2-back vs. 0-back
read pseudowords vs. consonant strings
read words vs. consonant strings
read words vs. read pseudowords
read sentence vs. read jabberwocky
read sentence vs. read words
inter-subject correlation
intra-subject correlation
Intra- and inter- subject correlation of brain maps
Pinho, A. L.et al.Hum Brain Mapp(2021)
15/44
Dictionary of Cognitive Components
Multi-subject, sparse dictionary learning:
min
(U
s
)s=1...n,V∈C
n
X
s=1
ı
∥X
s
−U
s
V∥
2
+λ∥U
s
∥1
ȷ
,
withX
s
p×c,U
s
p×k
andVk×c
•Functional correspondence: dictionary
of functional profiles (V) common to
all subjects
•Sparsity:ℓ1−norm penalty and
U
s
≥0,∀s∈[n]
Components are consistently mapped across subjects.Variability of topographies linked to individual differences.0.25 0.30 0.35 0.40 0.45 0.50 0.55
Intra-subject
correlation
Inter-subject
correlation
Correlations of the dictionary components on split-half data 0.000.250.50
read sentence vs. listen to sentence
read sentence vs. checkerboard
left hand vs. right hand
horizontal checkerboard vs. vertical checkerboard
mental subtraction vs. sentence
saccade vs. fixation
guess which hand vs. hand palm or back
object grasping vs. mimic orientation
mental motion vs. random motion
false-belief story vs. mechanistic story
false-belief tale vs. mechanistic tale
expression intention vs. expression gender
face trusty vs. face gender
face image vs. shape outline
punishment vs. reward
0.000.250.50
tongue vs. any motion
right foot vs. any motion
left foot vs. any motion
right hand vs. any motion
left hand vs. any motion
tale vs. mental addition
relational processing vs. visual matching
mental motion vs. random motion
tool image vs. any image
place image vs. any image
face image vs. any image
body image vs. any image
2-back vs. 0-back
read pseudowords vs. consonant strings
read words vs. consonant strings
read words vs. read pseudowords
read sentence vs. read jabberwocky
read sentence vs. read words
inter-subject correlation
intra-subject correlation
Intra- and inter- subject correlation of brain maps
Pinho, A. L.et al.Hum Brain Mapp(2021)
15/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Dictionary of Cognitive Components
Multi-subject, sparse dictionary learning:
min
(U
s
)s=1...n,V∈C
n
X
s=1
ı
∥X
s
−U
s
V∥
2
+λ∥U
s
∥1
ȷ
,
withX
s
p×c,U
s
p×k
andVk×c
•Functional correspondence: dictionary
of functional profiles (V) common to
all subjects
•Sparsity:ℓ1−norm penalty and
U
s
≥0,∀s∈[n]
Components are consistently mapped across subjects.Variability of topographies linked to individual differences.0.25 0.30 0.35 0.40 0.45 0.50 0.55
Intra-subject
correlation
Inter-subject
correlation
Correlations of the dictionary components on split-half data 0.000.250.50
read sentence vs. listen to sentence
read sentence vs. checkerboard
left hand vs. right hand
horizontal checkerboard vs. vertical checkerboard
mental subtraction vs. sentence
saccade vs. fixation
guess which hand vs. hand palm or back
object grasping vs. mimic orientation
mental motion vs. random motion
false-belief story vs. mechanistic story
false-belief tale vs. mechanistic tale
expression intention vs. expression gender
face trusty vs. face gender
face image vs. shape outline
punishment vs. reward
0.000.250.50
tongue vs. any motion
right foot vs. any motion
left foot vs. any motion
right hand vs. any motion
left hand vs. any motion
tale vs. mental addition
relational processing vs. visual matching
mental motion vs. random motion
tool image vs. any image
place image vs. any image
face image vs. any image
body image vs. any image
2-back vs. 0-back
read pseudowords vs. consonant strings
read words vs. consonant strings
read words vs. read pseudowords
read sentence vs. read jabberwocky
read sentence vs. read words
inter-subject correlation
intra-subject correlation
Intra- and inter- subject correlation of brain maps
Pinho, A. L.et al.Hum Brain Mapp(2021)
Dictionary of components
is a better approach to
capture shared
representations.
15/44
Dictionary of Cognitive Components
Multi-subject, sparse dictionary learning:
min
(U
s
)s=1...n,V∈C
n
X
s=1
ı
∥X
s
−U
s
V∥
2
+λ∥U
s
∥1
ȷ
,
withX
s
p×c,U
s
p×k
andVk×c
•Functional correspondence: dictionary
of functional profiles (V) common to
all subjects
•Sparsity:ℓ1−norm penalty and
U
s
≥0,∀s∈[n]
Components are consistently mapped across subjects.Variability of topographies linked to individual differences.0.25 0.30 0.35 0.40 0.45 0.50 0.55
Intra-subject
correlation
Inter-subject
correlation
Correlations of the dictionary components on split-half data 0.000.250.50
read sentence vs. listen to sentence
read sentence vs. checkerboard
left hand vs. right hand
horizontal checkerboard vs. vertical checkerboard
mental subtraction vs. sentence
saccade vs. fixation
guess which hand vs. hand palm or back
object grasping vs. mimic orientation
mental motion vs. random motion
false-belief story vs. mechanistic story
false-belief tale vs. mechanistic tale
expression intention vs. expression gender
face trusty vs. face gender
face image vs. shape outline
punishment vs. reward
0.000.250.50
tongue vs. any motion
right foot vs. any motion
left foot vs. any motion
right hand vs. any motion
left hand vs. any motion
tale vs. mental addition
relational processing vs. visual matching
mental motion vs. random motion
tool image vs. any image
place image vs. any image
face image vs. any image
body image vs. any image
2-back vs. 0-back
read pseudowords vs. consonant strings
read words vs. consonant strings
read words vs. read pseudowords
read sentence vs. read jabberwocky
read sentence vs. read words
inter-subject correlation
intra-subject correlation
Intra- and inter- subject correlation of brain maps
Pinho, A. L.et al.Hum Brain Mapp(2021)
Dictionary of components
is a better approach to
capture shared
representations.
15/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Reconstruction of Functional Contrasts
Leave-p-out CV (p=3 subjects)
experiment to learn the shared
representations from contrasts of
eleven tasks of the IBC dataset.
(n= 13)
Predict all contrasts from the
remaining task
Train a Ridge-regression model with individual
contrast mapsiof tasks−jto predict taskjon
individual contrast-mapss̸=i:
bw
s,λ,j
= argmin
w∈R
c−1
X
i̸=s
∥X
i
j−X
i
−jw∥
2
+λ∥w∥
2
Prediction output for one contrast of taskjin
subjects:
b
X
s
j=X
s
−j
bw
s,λ,j
.
Cross-validated R-squared for taskjat locationi:
R
2
i(j) = 1−mean
s∈[n]
∥
b
X
s
i,j
−X
s
i,j
∥
2
∥X
s
i,j
∥
2
Pinho, A. L.et al.Hum Brain Mapp(2021)
n= 13
max R
2
Most of the brain regions
are covered by the
predicted functional
signatures.
n= 13
Pinho, A. L.et al.Hum Brain Mapp(2021)
Ridge-Regression model
for the scrambled case:
bw
s,λ,j
= argmin
w∈R
c−1
X
i,k̸=s
∥X
i
j
−X
k
−j
w∥
2
+λ∥w∥
2
Cross-validated R-squared:
R
2
i
(j) = 1−mean
s∈[n]
∥bX
s
i,j
−X
s
′
i,j
∥
2
∥X
s
′
i,j
∥
2
Permutations of subjects
decrease the proportion of
well-predicted voxels in all
tasks, showing that
topographies are driven by
subject-specific variability.
16/44
Reconstruction of Functional Contrasts
Leave-p-out CV (p=3 subjects)
experiment to learn the shared
representations from contrasts of
eleven tasks of the IBC dataset.
(n= 13)
Predict all contrasts from the
remaining task
Train a Ridge-regression model with individual
contrast mapsiof tasks−jto predict taskjon
individual contrast-mapss̸=i:
bw
s,λ,j
= argmin
w∈R
c−1
X
i̸=s
∥X
i
j−X
i
−jw∥
2
+λ∥w∥
2
Prediction output for one contrast of taskjin
subjects:
b
X
s
j=X
s
−j
bw
s,λ,j
.
Cross-validated R-squared for taskjat locationi:
R
2
i(j) = 1−mean
s∈[n]
∥
b
X
s
i,j
−X
s
i,j
∥
2
∥X
s
i,j
∥
2
Pinho, A. L.et al.Hum Brain Mapp(2021)
n= 13
max R
2
Most of the brain regions
are covered by the
predicted functional
signatures.
n= 13
Pinho, A. L.et al.Hum Brain Mapp(2021)
Ridge-Regression model
for the scrambled case:
bw
s,λ,j
= argmin
w∈R
c−1
X
i,k̸=s
∥X
i
j
−X
k
−j
w∥
2
+λ∥w∥
2
Cross-validated R-squared:
R
2
i
(j) = 1−mean
s∈[n]
∥bX
s
i,j
−X
s
′
i,j
∥
2
∥X
s
′
i,j
∥
2
Permutations of subjects
decrease the proportion of
well-predicted voxels in all
tasks, showing that
topographies are driven by
subject-specific variability.
16/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Reconstruction of Functional Contrasts
Leave-p-out CV (p=3 subjects)
experiment to learn the shared
representations from contrasts of
eleven tasks of the IBC dataset.
(n= 13)
Predict all contrasts from the
remaining task
Train a Ridge-regression model with individual
contrast mapsiof tasks−jto predict taskjon
individual contrast-mapss̸=i:
bw
s,λ,j
= argmin
w∈R
c−1
X
i̸=s
∥X
i
j−X
i
−jw∥
2
+λ∥w∥
2
Prediction output for one contrast of taskjin
subjects:
b
X
s
j=X
s
−j
bw
s,λ,j
.
Cross-validated R-squared for taskjat locationi:
R
2
i(j) = 1−mean
s∈[n]
∥
b
X
s
i,j
−X
s
i,j
∥
2
∥X
s
i,j
∥
2
Pinho, A. L.et al.Hum Brain Mapp(2021)
n= 13
max R
2
Most of the brain regions
are covered by the
predicted functional
signatures.
n= 13
Pinho, A. L.et al.Hum Brain Mapp(2021)
Ridge-Regression model
for the scrambled case:
bw
s,λ,j
= argmin
w∈R
c−1
X
i,k̸=s
∥X
i
j
−X
k
−j
w∥
2
+λ∥w∥
2
Cross-validated R-squared:
R
2
i
(j) = 1−mean
s∈[n]
∥bX
s
i,j
−X
s
′
i,j
∥
2
∥X
s
′
i,j
∥
2
Permutations of subjects
decrease the proportion of
well-predicted voxels in all
tasks, showing that
topographies are driven by
subject-specific variability.
16/44
Reconstruction of Functional Contrasts
Leave-p-out CV (p=3 subjects)
experiment to learn the shared
representations from contrasts of
eleven tasks of the IBC dataset.
(n= 13)
Predict all contrasts from the
remaining task
Train a Ridge-regression model with individual
contrast mapsiof tasks−jto predict taskjon
individual contrast-mapss̸=i:
bw
s,λ,j
= argmin
w∈R
c−1
X
i̸=s
∥X
i
j−X
i
−jw∥
2
+λ∥w∥
2
Prediction output for one contrast of taskjin
subjects:
b
X
s
j=X
s
−j
bw
s,λ,j
.
Cross-validated R-squared for taskjat locationi:
R
2
i(j) = 1−mean
s∈[n]
∥
b
X
s
i,j
−X
s
i,j
∥
2
∥X
s
i,j
∥
2
Pinho, A. L.et al.Hum Brain Mapp(2021)
n= 13
max R
2
Most of the brain regions
are covered by the
predicted functional
signatures.
n= 13
Pinho, A. L.et al.Hum Brain Mapp(2021)
Ridge-Regression model
for the scrambled case:
bw
s,λ,j
= argmin
w∈R
c−1
X
i,k̸=s
∥X
i
j
−X
k
−j
w∥
2
+λ∥w∥
2
Cross-validated R-squared:
R
2
i
(j) = 1−mean
s∈[n]
∥bX
s
i,j
−X
s
′
i,j
∥
2
∥X
s
′
i,j
∥
2
Permutations of subjects
decrease the proportion of
well-predicted voxels in all
tasks, showing that
topographies are driven by
subject-specific variability.
16/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Reconstruction of Functional Contrasts
Train a Ridge-regression model with individual
contrast mapsiof tasks−jto predict taskjon
individual contrast-mapss̸=i:
bw
s,λ,j
= argmin
w∈R
c−1
X
i̸=s
∥X
i
j−X
i
−jw∥
2
+λ∥w∥
2
Prediction output for one contrast of taskjin
subjects:
b
X
s
j=X
s
−j
bw
s,λ,j
.
Cross-validated R-squared for taskjat locationi:
R
2
i(j) = 1−mean
s∈[n]
∥
b
X
s
i,j
−X
s
i,j
∥
2
∥X
s
i,j
∥
2
Pinho, A. L.et al.Hum Brain Mapp(2021)
n= 13
max R
2
Most of the brain regions
are covered by the
predicted functional
signatures.
n= 13
Pinho, A. L.et al.Hum Brain Mapp(2021)
Ridge-Regression model
for the scrambled case:
bw
s,λ,j
= argmin
w∈R
c−1
X
i,k̸=s
∥X
i
j
−X
k
−j
w∥
2
+λ∥w∥
2
Cross-validated R-squared:
R
2
i
(j) = 1−mean
s∈[n]
∥bX
s
i,j
−X
s
′
i,j
∥
2
∥X
s
′
i,j
∥
2
Permutations of subjects
decrease the proportion of
well-predicted voxels in all
tasks, showing that
topographies are driven by
subject-specific variability.
16/44
Reconstruction of Functional Contrasts
Train a Ridge-regression model with individual
contrast mapsiof tasks−jto predict taskjon
individual contrast-mapss̸=i:
bw
s,λ,j
= argmin
w∈R
c−1
X
i̸=s
∥X
i
j−X
i
−jw∥
2
+λ∥w∥
2
Prediction output for one contrast of taskjin
subjects:
b
X
s
j=X
s
−j
bw
s,λ,j
.
Cross-validated R-squared for taskjat locationi:
R
2
i(j) = 1−mean
s∈[n]
∥
b
X
s
i,j
−X
s
i,j
∥
2
∥X
s
i,j
∥
2
Pinho, A. L.et al.Hum Brain Mapp(2021)
n= 13
max R
2
Most of the brain regions
are covered by the
predicted functional
signatures.
n= 13
Pinho, A. L.et al.Hum Brain Mapp(2021)
Ridge-Regression model
for the scrambled case:
bw
s,λ,j
= argmin
w∈R
c−1
X
i,k̸=s
∥X
i
j
−X
k
−j
w∥
2
+λ∥w∥
2
Cross-validated R-squared:
R
2
i
(j) = 1−mean
s∈[n]
∥bX
s
i,j
−X
s
′
i,j
∥
2
∥X
s
′
i,j
∥
2
Permutations of subjects
decrease the proportion of
well-predicted voxels in all
tasks, showing that
topographies are driven by
subject-specific variability.
16/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Reconstruction of Functional Contrasts
Train a Ridge-regression model with individual
contrast mapsiof tasks−jto predict taskjon
individual contrast-mapss̸=i:
bw
s,λ,j
= argmin
w∈R
c−1
X
i̸=s
∥X
i
j−X
i
−jw∥
2
+λ∥w∥
2
Prediction output for one contrast of taskjin
subjects:
b
X
s
j=X
s
−j
bw
s,λ,j
.
Cross-validated R-squared for taskjat locationi:
R
2
i(j) = 1−mean
s∈[n]
∥
b
X
s
i,j
−X
s
i,j
∥
2
∥X
s
i,j
∥
2
Pinho, A. L.et al.Hum Brain Mapp(2021)
n= 13
max R
2
Most of the brain regions
are covered by the
predicted functional
signatures.
n= 13
Pinho, A. L.et al.Hum Brain Mapp(2021)
Ridge-Regression model
for the scrambled case:
bw
s,λ,j
= argmin
w∈R
c−1
X
i,k̸=s
∥X
i
j
−X
k
−j
w∥
2
+λ∥w∥
2
Cross-validated R-squared:
R
2
i
(j) = 1−mean
s∈[n]
∥bX
s
i,j
−X
s
′
i,j
∥
2
∥X
s
′
i,j
∥
2
Permutations of subjects
decrease the proportion of
well-predicted voxels in all
tasks, showing that
topographies are driven by
subject-specific variability.
16/44
Reconstruction of Functional Contrasts
Train a Ridge-regression model with individual
contrast mapsiof tasks−jto predict taskjon
individual contrast-mapss̸=i:
bw
s,λ,j
= argmin
w∈R
c−1
X
i̸=s
∥X
i
j−X
i
−jw∥
2
+λ∥w∥
2
Prediction output for one contrast of taskjin
subjects:
b
X
s
j=X
s
−j
bw
s,λ,j
.
Cross-validated R-squared for taskjat locationi:
R
2
i(j) = 1−mean
s∈[n]
∥
b
X
s
i,j
−X
s
i,j
∥
2
∥X
s
i,j
∥
2
Pinho, A. L.et al.Hum Brain Mapp(2021)
n= 13
max R
2
Most of the brain regions
are covered by the
predicted functional
signatures.
n= 13
Pinho, A. L.et al.Hum Brain Mapp(2021)
Ridge-Regression model
for the scrambled case:
bw
s,λ,j
= argmin
w∈R
c−1
X
i,k̸=s
∥X
i
j
−X
k
−j
w∥
2
+λ∥w∥
2
Cross-validated R-squared:
R
2
i
(j) = 1−mean
s∈[n]
∥bX
s
i,j
−X
s
′
i,j
∥
2
∥X
s
′
i,j
∥
2
Permutations of subjects
decrease the proportion of
well-predicted voxels in all
tasks, showing that
topographies are driven by
subject-specific variability.
16/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Bayesian Framework (1/3)
Two key elements:
•Data-set Fusion framework
•Hierarchical Bayesian Parcellation(HBP) scheme
Two applications:
•Create cross-population functional atlases
•Derive an individualized-parcellation benchmark
Adapted from Zhi, D. Shahshahani, L., Net-
tekoven, C.,Pinho, A. L., Bzdok, D., & Diedrich-
sen, J. (2023) [preprint; under review]
The challenge:
We can derive reliable parcellations, even on new subjects based on existing ones.
(Thirion, B, Himanshu, A., Ponce, A. F.,Pinho, A. L., & Thual, A. Brain Struct Funct (2024)).
↓
How about when we have fewtrain data available?
17/44
The Hierarchical Bayesian Framework (1/3)
Two key elements:
•Data-set Fusion framework
•Hierarchical Bayesian Parcellation(HBP) scheme
Two applications:
•Create cross-population functional atlases
•Derive an individualized-parcellation benchmark
Adapted from Zhi, D. Shahshahani, L., Net-
tekoven, C.,Pinho, A. L., Bzdok, D., & Diedrich-
sen, J. (2023) [preprint; under review]
The challenge:
We can derive reliable parcellations, even on new subjects based on existing ones.
(Thirion, B, Himanshu, A., Ponce, A. F.,Pinho, A. L., & Thual, A. Brain Struct Funct (2024)).
↓
How about when we have fewtrain data available?
17/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Bayesian Framework (1/3)
Two key elements:
•Data-set Fusion framework
•Hierarchical Bayesian Parcellation(HBP) scheme
Two applications:
•Create cross-population functional atlases
•Derive an individualized-parcellation benchmark
Adapted from Zhi, D. Shahshahani, L., Net-
tekoven, C.,Pinho, A. L., Bzdok, D., & Diedrich-
sen, J. (2023) [preprint; under review]
The challenge:
We can derive reliable parcellations, even on new subjects based on existing ones.
(Thirion, B, Himanshu, A., Ponce, A. F.,Pinho, A. L., & Thual, A. Brain Struct Funct (2024)).
↓
How about when we have fewtrain data available?
17/44
The Hierarchical Bayesian Framework (1/3)
Two key elements:
•Data-set Fusion framework
•Hierarchical Bayesian Parcellation(HBP) scheme
Two applications:
•Create cross-population functional atlases
•Derive an individualized-parcellation benchmark
Adapted from Zhi, D. Shahshahani, L., Net-
tekoven, C.,Pinho, A. L., Bzdok, D., & Diedrich-
sen, J. (2023) [preprint; under review]
The challenge:
We can derive reliable parcellations, even on new subjects based on existing ones.
(Thirion, B, Himanshu, A., Ponce, A. F.,Pinho, A. L., & Thual, A. Brain Struct Funct (2024)).
↓
How about when we have fewtrain data available?
17/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Bayesian Framework (1/3)
Two key elements:
•Data-set Fusion framework
•Hierarchical Bayesian Parcellation(HBP) scheme
Two applications:
•Create cross-population functional atlases
•Derive an individualized-parcellation benchmark
Adapted from Zhi, D. Shahshahani, L., Net-
tekoven, C.,Pinho, A. L., Bzdok, D., & Diedrich-
sen, J. (2023) [preprint; under review]
The challenge:
We can derive reliable parcellations, even on new subjects based on existing ones.
(Thirion, B, Himanshu, A., Ponce, A. F.,Pinho, A. L., & Thual, A. Brain Struct Funct (2024)).
↓
How about when we have fewtrain data available?
17/44
The Hierarchical Bayesian Framework (1/3)
Two key elements:
•Data-set Fusion framework
•Hierarchical Bayesian Parcellation(HBP) scheme
Two applications:
•Create cross-population functional atlases
•Derive an individualized-parcellation benchmark
Adapted from Zhi, D. Shahshahani, L., Net-
tekoven, C.,Pinho, A. L., Bzdok, D., & Diedrich-
sen, J. (2023) [preprint; under review]
The challenge:
We can derive reliable parcellations, even on new subjects based on existing ones.
(Thirion, B, Himanshu, A., Ponce, A. F.,Pinho, A. L., & Thual, A. Brain Struct Funct (2024)).
↓
How about when we have fewtrain data available?
17/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Bayesian Framework (2/3)
•Group+Individual parcellation
outperforms Individual-only
parcellation.
•Both parcellation schemes
improve when using more
individual data.
•30min of individual data are
necessary to obtain a brain
parcellation significantly better
than group probability.
Nettekoven, C., Zhi, D., Shahshahani, L.,Pinho, A. L., Saadon-Grosman, N.,
Buckner, R. L., & Diedrichsen, J. (2023) [preprint; under review]
18/44
The Hierarchical Bayesian Framework (2/3)
•Group+Individual parcellation
outperforms Individual-only
parcellation.
•Both parcellation schemes
improve when using more
individual data.
•30min of individual data are
necessary to obtain a brain
parcellation significantly better
than group probability.
Nettekoven, C., Zhi, D., Shahshahani, L.,Pinho, A. L., Saadon-Grosman, N.,
Buckner, R. L., & Diedrichsen, J. (2023) [preprint; under review]
18/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Bayesian Framework (3/3)
•20min of trained data / 80min of test data
•HBPsignificantly outperforms competing
models (Dual RegressionandDictionary
Learning).
Pinho, A. L., Yoon, J., & Diedrichsen, J. CCN2024 (2024) [conference paper under review]
19/44
The Hierarchical Bayesian Framework (3/3)
•20min of trained data / 80min of test data
•HBPsignificantly outperforms competing
models (Dual RegressionandDictionary
Learning).
Pinho, A. L., Yoon, J., & Diedrichsen, J. CCN2024 (2024) [conference paper under review]
19/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Bayesian Framework (3/3)
•20min of trained data / 80min of test data
•HBPsignificantly outperforms competing
models (Dual RegressionandDictionary
Learning).
Pinho, A. L., Yoon, J., & Diedrichsen, J. CCN2024 (2024) [conference paper under review]
19/44
The Hierarchical Bayesian Framework (3/3)
•20min of trained data / 80min of test data
•HBPsignificantly outperforms competing
models (Dual RegressionandDictionary
Learning).
Pinho, A. L., Yoon, J., & Diedrichsen, J. CCN2024 (2024) [conference paper under review]
19/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Bayesian Framework (3/3)
•20min of trained data / 80min of test data
•HBPsignificantly outperforms competing
models (Dual RegressionandDictionary
Learning).
Pinho, A. L., Yoon, J., & Diedrichsen, J. CCN2024 (2024) [conference paper under review]
Find the optimalκfor a
specific network of regions.
19/44
The Hierarchical Bayesian Framework (3/3)
•20min of trained data / 80min of test data
•HBPsignificantly outperforms competing
models (Dual RegressionandDictionary
Learning).
Pinho, A. L., Yoon, J., & Diedrichsen, J. CCN2024 (2024) [conference paper under review]
Find the optimalκfor a
specific network of regions.
19/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Bayesian Framework (3/3)
•20min of trained data / 80min of test data
•HBPsignificantly outperforms competing
models (Dual RegressionandDictionary
Learning).
Pinho, A. L., Yoon, J., & Diedrichsen, J. CCN2024 (2024) [conference paper under review]
Find the optimalκfor a
specific network of regions.
Obtain the best individual
parcellation for a specific
cognitive domain.
19/44
The Hierarchical Bayesian Framework (3/3)
•20min of trained data / 80min of test data
•HBPsignificantly outperforms competing
models (Dual RegressionandDictionary
Learning).
Pinho, A. L., Yoon, J., & Diedrichsen, J. CCN2024 (2024) [conference paper under review]
Find the optimalκfor a
specific network of regions.
Obtain the best individual
parcellation for a specific
cognitive domain.
19/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: FastSRM (1/3)
Challenges:•Standard GLM applied to naturalistic stimuli leads to high-dimensional
controlled-design models.
↓
Unsupervised data-driven approach on fMRI timeseries where the design matrix and the
spatial maps are learnt jointly is more wieldy.
•High-dimensional data (many voxels) require a decomposition method with scalability.
Shared Response Modelby Chenet al.(2015)
Fast Shared Response Model(FastSRM)by Richardet al.(2019):
https://hugorichard.github.io/FastSRM/
20/44
Encoding Models for Naturalistic Stimuli: FastSRM (1/3)
Challenges:•Standard GLM applied to naturalistic stimuli leads to high-dimensional
controlled-design models.
↓
Unsupervised data-driven approach on fMRI timeseries where the design matrix and the
spatial maps are learnt jointly is more wieldy.
•High-dimensional data (many voxels) require a decomposition method with scalability.
Shared Response Modelby Chenet al.(2015)
Fast Shared Response Model(FastSRM)by Richardet al.(2019):
https://hugorichard.github.io/FastSRM/
20/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: FastSRM (1/3)
Challenges:•Standard GLM applied to naturalistic stimuli leads to high-dimensional
controlled-design models.
↓
Unsupervised data-driven approach on fMRI timeseries where the design matrix and the
spatial maps are learnt jointly is more wieldy.
•High-dimensional data (many voxels) require a decomposition method with scalability.
Shared Response Modelby Chenet al.(2015)
Fast Shared Response Model(FastSRM)by Richardet al.(2019):
https://hugorichard.github.io/FastSRM/
20/44
Encoding Models for Naturalistic Stimuli: FastSRM (1/3)
Challenges:•Standard GLM applied to naturalistic stimuli leads to high-dimensional
controlled-design models.
↓
Unsupervised data-driven approach on fMRI timeseries where the design matrix and the
spatial maps are learnt jointly is more wieldy.
•High-dimensional data (many voxels) require a decomposition method with scalability.
Shared Response Modelby Chenet al.(2015)
Fast Shared Response Model(FastSRM)by Richardet al.(2019):
https://hugorichard.github.io/FastSRM/
20/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: FastSRM (1/3)
Challenges:•Standard GLM applied to naturalistic stimuli leads to high-dimensional
controlled-design models.
↓
Unsupervised data-driven approach on fMRI timeseries where the design matrix and the
spatial maps are learnt jointly is more wieldy.
•High-dimensional data (many voxels) require a decomposition method with scalability.
Shared Response Modelby Chenet al.(2015)
Fast Shared Response Model(FastSRM)by Richardet al.(2019):
https://hugorichard.github.io/FastSRM/
20/44
Encoding Models for Naturalistic Stimuli: FastSRM (1/3)
Challenges:•Standard GLM applied to naturalistic stimuli leads to high-dimensional
controlled-design models.
↓
Unsupervised data-driven approach on fMRI timeseries where the design matrix and the
spatial maps are learnt jointly is more wieldy.
•High-dimensional data (many voxels) require a decomposition method with scalability.
Shared Response Modelby Chenet al.(2015)
Fast Shared Response Model(FastSRM)by Richardet al.(2019):
https://hugorichard.github.io/FastSRM/
20/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: FastSRM (1/3)
Challenges:•Standard GLM applied to naturalistic stimuli leads to high-dimensional
controlled-design models.
↓
Unsupervised data-driven approach on fMRI timeseries where the design matrix and the
spatial maps are learnt jointly is more wieldy.
•High-dimensional data (many voxels) require a decomposition method with scalability.
Shared Response Modelby Chenet al.(2015)
Fast Shared Response Model(FastSRM)by Richardet al.(2019):
https://hugorichard.github.io/FastSRM/
20/44
Encoding Models for Naturalistic Stimuli: FastSRM (1/3)
Challenges:•Standard GLM applied to naturalistic stimuli leads to high-dimensional
controlled-design models.
↓
Unsupervised data-driven approach on fMRI timeseries where the design matrix and the
spatial maps are learnt jointly is more wieldy.
•High-dimensional data (many voxels) require a decomposition method with scalability.
Shared Response Modelby Chenet al.(2015)
Fast Shared Response Model(FastSRM)by Richardet al.(2019):
https://hugorichard.github.io/FastSRM/
20/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: FastSRM (1/3)
Challenges:•Standard GLM applied to naturalistic stimuli leads to high-dimensional
controlled-design models.
↓
Unsupervised data-driven approach on fMRI timeseries where the design matrix and the
spatial maps are learnt jointly is more wieldy.
•High-dimensional data (many voxels) require a decomposition method with scalability.
Shared Response Modelby Chenet al.(2015)
Fast Shared Response Model(FastSRM)by Richardet al.(2019):
https://hugorichard.github.io/FastSRM/
20/44
Encoding Models for Naturalistic Stimuli: FastSRM (1/3)
Challenges:•Standard GLM applied to naturalistic stimuli leads to high-dimensional
controlled-design models.
↓
Unsupervised data-driven approach on fMRI timeseries where the design matrix and the
spatial maps are learnt jointly is more wieldy.
•High-dimensional data (many voxels) require a decomposition method with scalability.
Shared Response Modelby Chenet al.(2015)
Fast Shared Response Model(FastSRM)by Richardet al.(2019):
https://hugorichard.github.io/FastSRM/
20/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: FastSRM (2/3)
For all subjects and time frames, FastSRM can be formally defined as follows:
X=SW+E
X∈R
G×nv
→concatenation ofGbrain images withvvertices forn=12 subjects
S∈R
G×k
→shared response: concatenation of the weights across time frames
W∈R
k×nv
→concatenation of thekspatial components withvvertices for the
subjects
E →the additive noise
21/44
Encoding Models for Naturalistic Stimuli: FastSRM (2/3)
For all subjects and time frames, FastSRM can be formally defined as follows:
X=SW+E
X∈R
G×nv
→concatenation ofGbrain images withvvertices forn=12 subjects
S∈R
G×k
→shared response: concatenation of the weights across time frames
W∈R
k×nv
→concatenation of thekspatial components withvvertices for the
subjects
E →the additive noise
21/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: FastSRM (2/3)
For all subjects and time frames, FastSRM can be formally defined as follows:
X=SW+E
X∈R
G×nv
→concatenation ofGbrain images withvvertices forn=12 subjects
S∈R
G×k
→shared response: concatenation of the weights across time frames
W∈R
k×nv
→concatenation of thekspatial components withvvertices for the
subjects
E →the additive noise
CV scheme applied for each
task withK= 3for 12
subjects andK= 2forRruns
of a given task.
21/44
Encoding Models for Naturalistic Stimuli: FastSRM (2/3)
For all subjects and time frames, FastSRM can be formally defined as follows:
X=SW+E
X∈R
G×nv
→concatenation ofGbrain images withvvertices forn=12 subjects
S∈R
G×k
→shared response: concatenation of the weights across time frames
W∈R
k×nv
→concatenation of thekspatial components withvvertices for the
subjects
E →the additive noise
CV scheme applied for each
task withK= 3for 12
subjects andK= 2forRruns
of a given task.
21/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: FastSRM (3/3)
q⩽0.05
Pinho, A. L.et al.(2023)[preprint; accepted in Sci Data]
Top 10 regions of Glasser atlas w/ areas displaying
≥5%of significant voxels in both hemispheres
1
Auditory Association Cortex
2
Temporo-Parieto-Occipital Junction
3
Posterior Cingulate Cortex
4
Superior Parietal Cortex
5
Inferior Parietal Cortex
6
Early Auditory Cortex
7
Dorsal Stream Visual Cortex
8
Lateral Temporal Cortex
9
MT+Complex and Neighboring Visual Areas
10
Primary Visual Cortex (V1)
22/44
Encoding Models for Naturalistic Stimuli: FastSRM (3/3)
q⩽0.05
Pinho, A. L.et al.(2023)[preprint; accepted in Sci Data]
Top 10 regions of Glasser atlas w/ areas displaying
≥5%of significant voxels in both hemispheres
1
Auditory Association Cortex
2
Temporo-Parieto-Occipital Junction
3
Posterior Cingulate Cortex
4
Superior Parietal Cortex
5
Inferior Parietal Cortex
6
Early Auditory Cortex
7
Dorsal Stream Visual Cortex
8
Lateral Temporal Cortex
9
MT+Complex and Neighboring Visual Areas
10
Primary Visual Cortex (V1)
22/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: FastSRM (3/3)
q⩽0.05
Pinho, A. L.et al.(2023)[preprint; accepted in Sci Data]
Top 10 regions of Glasser atlas w/ areas displaying
≥5%of significant voxels in both hemispheres
1
Auditory Association Cortex
2
Temporo-Parieto-Occipital Junction
3
Posterior Cingulate Cortex
4
Superior Parietal Cortex
5
Inferior Parietal Cortex
6
Early Auditory Cortex
7
Dorsal Stream Visual Cortex
8
Lateral Temporal Cortex
9
MT+Complex and Neighboring Visual Areas
10
Primary Visual Cortex (V1)
Successful extraction of
cognitive networks using
FastSRM.
22/44
Encoding Models for Naturalistic Stimuli: FastSRM (3/3)
q⩽0.05
Pinho, A. L.et al.(2023)[preprint; accepted in Sci Data]
Top 10 regions of Glasser atlas w/ areas displaying
≥5%of significant voxels in both hemispheres
1
Auditory Association Cortex
2
Temporo-Parieto-Occipital Junction
3
Posterior Cingulate Cortex
4
Superior Parietal Cortex
5
Inferior Parietal Cortex
6
Early Auditory Cortex
7
Dorsal Stream Visual Cortex
8
Lateral Temporal Cortex
9
MT+Complex and Neighboring Visual Areas
10
Primary Visual Cortex (V1)
Successful extraction of
cognitive networks using
FastSRM.
22/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: CNN (1/2)
Encoding Pipeline
•Extract features:
•Resize frames
•Temporal downsampling oftsamples matching theTR
•CNN outputsdreduced intermediate representations
•Representations are convolved with thehrf
•Build a design matrix of sizeNt×Nd
•Fit the resulting GLM built from a CNN block with fMRI data
23/44
Encoding Models for Naturalistic Stimuli: CNN (1/2)
Encoding Pipeline
•Extract features:
•Resize frames
•Temporal downsampling oftsamples matching theTR
•CNN outputsdreduced intermediate representations
•Representations are convolved with thehrf
•Build a design matrix of sizeNt×Nd
•Fit the resulting GLM built from a CNN block with fMRI data
23/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: CNN (1/2)
Encoding Pipeline
•Extract features:
•Resize frames
•Temporal downsampling oftsamples matching theTR
•CNN outputsdreduced intermediate representations
•Representations are convolved with thehrf
•Build a design matrix of sizeNt×Nd
•Fit the resulting GLM built from a CNN block with fMRI data
LOO-run CV for every
subject and candidate
ridge-regression model.
23/44
Encoding Models for Naturalistic Stimuli: CNN (1/2)
Encoding Pipeline
•Extract features:
•Resize frames
•Temporal downsampling oftsamples matching theTR
•CNN outputsdreduced intermediate representations
•Representations are convolved with thehrf
•Build a design matrix of sizeNt×Nd
•Fit the resulting GLM built from a CNN block with fMRI data
LOO-run CV for every
subject and candidate
ridge-regression model.
23/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: CNN (2/2)
Results for theRaiderstask usingCORnet-ZCNNBlock 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theEarly Visual Cortex
Areas :
V1
V2
V3
V4 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Dorsal Pathway
Areas :
V3A
V3B
V7
IP0 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Ventral Pathway
Areas :
LO1
LO2
LO3
MT
MST
FST
PIT
24/44
Encoding Models for Naturalistic Stimuli: CNN (2/2)
Results for theRaiderstask usingCORnet-ZCNNBlock 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theEarly Visual Cortex
Areas :
V1
V2
V3
V4 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Dorsal Pathway
Areas :
V3A
V3B
V7
IP0 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Ventral Pathway
Areas :
LO1
LO2
LO3
MT
MST
FST
PIT
24/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: CNN (2/2)
Results for theRaiderstask usingCORnet-ZCNNBlock 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theEarly Visual Cortex
Areas :
V1
V2
V3
V4 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Dorsal Pathway
Areas :
V3A
V3B
V7
IP0 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Ventral Pathway
Areas :
LO1
LO2
LO3
MT
MST
FST
PIT
24/44
Encoding Models for Naturalistic Stimuli: CNN (2/2)
Results for theRaiderstask usingCORnet-ZCNNBlock 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theEarly Visual Cortex
Areas :
V1
V2
V3
V4 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Dorsal Pathway
Areas :
V3A
V3B
V7
IP0 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Ventral Pathway
Areas :
LO1
LO2
LO3
MT
MST
FST
PIT
24/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Encoding Models for Naturalistic Stimuli: CNN (2/2)
Results for theRaiderstask usingCORnet-ZCNNBlock 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theEarly Visual Cortex
Areas :
V1
V2
V3
V4 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Dorsal Pathway
Areas :
V3A
V3B
V7
IP0 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Ventral Pathway
Areas :
LO1
LO2
LO3
MT
MST
FST
PIT
Successful representation
of the hierarchy of the
visual system.
24/44
Encoding Models for Naturalistic Stimuli: CNN (2/2)
Results for theRaiderstask usingCORnet-ZCNNBlock 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theEarly Visual Cortex
Areas :
V1
V2
V3
V4 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Dorsal Pathway
Areas :
V3A
V3B
V7
IP0 Block 1 Block 2 Block 3 Block 4
Hierarchical Convolution Layers of the Model
0:04
0:06
0:08
0:10
0:12
Mean correlation across subjects (a.u)
CORNet-ZPredictions of theVisual Ventral Pathway
Areas :
LO1
LO2
LO3
MT
MST
FST
PIT
Successful representation
of the hierarchy of the
visual system.
24/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Encoding of Musical Descriptors (1/2)
How perceived musical features are encoded in the brain?
Categories of musical descriptors:
Low-level:loudness, timbre, pitch
Mid-level:rhythm, harmony, melody
High-level:genre, emotions, similarity
Goals:
•Feed a CNN with auditory naturalistic stimuli to obtain intermediate
representations and fit those layers with brain activity.
•Correlate individual predictive measures with metrics of music sophistication (e.g.
active engagement and perceptual abilities) and musical training.
25/44
The Hierarchical Encoding of Musical Descriptors (1/2)
How perceived musical features are encoded in the brain?
Categories of musical descriptors:
Low-level:loudness, timbre, pitch
Mid-level:rhythm, harmony, melody
High-level:genre, emotions, similarity
Goals:
•Feed a CNN with auditory naturalistic stimuli to obtain intermediate
representations and fit those layers with brain activity.
•Correlate individual predictive measures with metrics of music sophistication (e.g.
active engagement and perceptual abilities) and musical training.
25/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Encoding of Musical Descriptors (2/2)
Motivations:
CNN:It is a better predictive model of neural responses to natural sounds than
acoustic or even semantic models.
Naturalistic Stimuli:High-order cognitive features linked to a musical descriptor are
product of the interaction of low-order components.
↓
Benefit not to study them in isolation.
26/44
The Hierarchical Encoding of Musical Descriptors (2/2)
Motivations:
CNN:It is a better predictive model of neural responses to natural sounds than
acoustic or even semantic models.
Naturalistic Stimuli:High-order cognitive features linked to a musical descriptor are
product of the interaction of low-order components.
↓
Benefit not to study them in isolation.
26/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Encoding of Musical Descriptors (2/2)
Motivations:
CNN:It is a better predictive model of neural responses to natural sounds than
acoustic or even semantic models.
Naturalistic Stimuli:High-order cognitive features linked to a musical descriptor are
product of the interaction of low-order components.
↓
Benefit not to study them in isolation.
26/44
The Hierarchical Encoding of Musical Descriptors (2/2)
Motivations:
CNN:It is a better predictive model of neural responses to natural sounds than
acoustic or even semantic models.
Naturalistic Stimuli:High-order cognitive features linked to a musical descriptor are
product of the interaction of low-order components.
↓
Benefit not to study them in isolation.
26/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
The Hierarchical Encoding of Musical Descriptors (2/2)
Motivations:
CNN:It is a better predictive model of neural responses to natural sounds than
acoustic or even semantic models.
Naturalistic Stimuli:High-order cognitive features linked to a musical descriptor are
product of the interaction of low-order components.
↓
Benefit not to study them in isolation.
26/44
The Hierarchical Encoding of Musical Descriptors (2/2)
Motivations:
CNN:It is a better predictive model of neural responses to natural sounds than
acoustic or even semantic models.
Naturalistic Stimuli:High-order cognitive features linked to a musical descriptor are
product of the interaction of low-order components.
↓
Benefit not to study them in isolation.
26/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
DBP to investigate dissociations: a ‘Beat’ study (1/2)
Motivations
•Compare single task paradigms to dissociate
competing cognitive theories.
•Study temporal predictions
Hypothesis: •Contribution ofthe Dorsal Striatum (DS)and
the Cerebellumto the formation of
temporal predictions, depending on the
periodicity of the events.
↓
Beat Conditionregulated by theDS
Interval Conditionregulated by theCerebellum
27/44
DBP to investigate dissociations: a ‘Beat’ study (1/2)
Motivations
•Compare single task paradigms to dissociate
competing cognitive theories.
•Study temporal predictions
Hypothesis: •Contribution ofthe Dorsal Striatum (DS)and
the Cerebellumto the formation of
temporal predictions, depending on the
periodicity of the events.
↓
Beat Conditionregulated by theDS
Interval Conditionregulated by theCerebellum
27/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
DBP to investigate dissociations: a ‘Beat’ study (1/2)
Motivations
•Compare single task paradigms to dissociate
competing cognitive theories.
•Study temporal predictions
Hypothesis: •Contribution ofthe Dorsal Striatum (DS)and
the Cerebellumto the formation of
temporal predictions, depending on the
periodicity of the events.
↓
Beat Conditionregulated by theDS
Interval Conditionregulated by theCerebellum
27/44
DBP to investigate dissociations: a ‘Beat’ study (1/2)
Motivations
•Compare single task paradigms to dissociate
competing cognitive theories.
•Study temporal predictions
Hypothesis: •Contribution ofthe Dorsal Striatum (DS)and
the Cerebellumto the formation of
temporal predictions, depending on the
periodicity of the events.
↓
Beat Conditionregulated by theDS
Interval Conditionregulated by theCerebellum
27/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
DBP to investigate dissociations: a ‘Beat’ study (1/2)
Motivations
•Compare single task paradigms to dissociate
competing cognitive theories.
•Study temporal predictions
Hypothesis: •Contribution ofthe Dorsal Striatum (DS)and
the Cerebellumto the formation of
temporal predictions, depending on the
periodicity of the events.
↓
Beat Conditionregulated by theDS
Interval Conditionregulated by theCerebellum
27/44
DBP to investigate dissociations: a ‘Beat’ study (1/2)
Motivations
•Compare single task paradigms to dissociate
competing cognitive theories.
•Study temporal predictions
Hypothesis: •Contribution ofthe Dorsal Striatum (DS)and
the Cerebellumto the formation of
temporal predictions, depending on the
periodicity of the events.
↓
Beat Conditionregulated by theDS
Interval Conditionregulated by theCerebellum
27/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Single Dissociation: upper regulation of DS during Beat (2/2)
•2-way RM ANOVA
Independent variables: ROI (DS/Cerebellum), Dependent variable: PSC
Condition (Beat/Interval)
Ensemble of Individual ROIs
Fraction of participants represented at every voxel
28/44
Single Dissociation: upper regulation of DS during Beat (2/2)
•2-way RM ANOVA
Independent variables: ROI (DS/Cerebellum), Dependent variable: PSC
Condition (Beat/Interval)
Ensemble of Individual ROIs
Fraction of participants represented at every voxel
28/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Axis 2:
Toward an ontology of cognitive processes and
brain-network taxonomy
Axis 2:
Toward an ontology of cognitive processes and
brain-network taxonomy
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Cognitive Ontologies
What is an ontology?
An ontology is the specification of the elementary conceptual entities that are
postulated by a theory and their relationships.
30/44
Cognitive Ontologies
What is an ontology?
An ontology is the specification of the elementary conceptual entities that are
postulated by a theory and their relationships.
30/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Cognitive Ontologies
What is an ontology?
An ontology is the specification of the elementary conceptual entities that are
postulated by a theory and their relationships.
30/44
Cognitive Ontologies
What is an ontology?
An ontology is the specification of the elementary conceptual entities that are
postulated by a theory and their relationships.
30/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Cognitive Ontologies
What is an ontology?
An ontology is the specification of the elementary conceptual entities that are
postulated by a theory and their relationships.
30/44
Cognitive Ontologies
What is an ontology?
An ontology is the specification of the elementary conceptual entities that are
postulated by a theory and their relationships.
30/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Activation Similarity fits Task Similarity
Similarity between
activation maps
of elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Similarity between
cognitive description
of elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Pinho, A. L.et al.Sci Data(2018)
First Release
•ARCHI tasks
Pinel, P.et al.(2007)
Standard
Spatial
Social
Emotional
•HCP tasks
Barch, D. M.et al.(2013)
Emotion
Gambling
Motor
Language
Relational
Social
WM
•RSVP Language
31/44
Activation Similarity fits Task Similarity
Similarity between
activation maps
of elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Similarity between
cognitive description
of elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Pinho, A. L.et al.Sci Data(2018)
First Release
•ARCHI tasks
Pinel, P.et al.(2007)
Standard
Spatial
Social
Emotional
•HCP tasks
Barch, D. M.et al.(2013)
Emotion
Gambling
Motor
Language
Relational
Social
WM
•RSVP Language
31/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Similarity between
activation maps of
elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Similarity between
cognitive description
of elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Pinho, A. L.et al.Sci Data(2018)Pinho, A. L.et al.Sci Data(2020)
Second release:
•Mental Time Travel battery
Gauthier, B., & van Wassenhove, V. (2016a,b)
•Preference battery
Lebreton, M.et al.(2015)
•ToM + Pain Matrices battery
Dodell-Feder, D.et al.(2010)
Jacoby, N.et al.(2015)
Richardson, H.et al.(2018)
•Visual Short-Term Memory + Enumeration
tasks
Knops, A.et al.(2014)
•Self-Reference Effect task
Genon, S.et al.(2014)
•“Bang!” task
Campbell, K. L.et al.(2015)First + Second releases:
All contrasts:
Elementary contrasts:
32/44
Similarity between
activation maps of
elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Similarity between
cognitive description
of elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Pinho, A. L.et al.Sci Data(2018)Pinho, A. L.et al.Sci Data(2020)
Second release:
•Mental Time Travel battery
Gauthier, B., & van Wassenhove, V. (2016a,b)
•Preference battery
Lebreton, M.et al.(2015)
•ToM + Pain Matrices battery
Dodell-Feder, D.et al.(2010)
Jacoby, N.et al.(2015)
Richardson, H.et al.(2018)
•Visual Short-Term Memory + Enumeration
tasks
Knops, A.et al.(2014)
•Self-Reference Effect task
Genon, S.et al.(2014)
•“Bang!” task
Campbell, K. L.et al.(2015)First + Second releases:
All contrasts:
Elementary contrasts:
32/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Similarity between
activation maps of
elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Similarity between
cognitive description
of elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Pinho, A. L.et al.Sci Data(2018)Pinho, A. L.et al.Sci Data(2020)
Second release:
•Mental Time Travel battery
Gauthier, B., & van Wassenhove, V. (2016a,b)
•Preference battery
Lebreton, M.et al.(2015)
•ToM + Pain Matrices battery
Dodell-Feder, D.et al.(2010)
Jacoby, N.et al.(2015)
Richardson, H.et al.(2018)
•Visual Short-Term Memory + Enumeration
tasks
Knops, A.et al.(2014)
•Self-Reference Effect task
Genon, S.et al.(2014)
•“Bang!” task
Campbell, K. L.et al.(2015)First + Second releases:
All contrasts:
Elementary contrasts:
Spearman correlation
First Release:0.21(p≤10
−17
)
Second Release:0.21(p≤10
−13
)
First+Second Releases:0.23(p≤10
−72
)
32/44
Similarity between
activation maps of
elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Similarity between
cognitive description
of elementary contrastsarchi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gamblinghcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
archi emotional
archi social
archi spatial
archi standard
hcp emotion
hcp gambling
hcp language
hcp motor
hcp relational
hcp social
hcp wm
rsvp language
0
1
Pinho, A. L.et al.Sci Data(2018)Pinho, A. L.et al.Sci Data(2020)
Second release:
•Mental Time Travel battery
Gauthier, B., & van Wassenhove, V. (2016a,b)
•Preference battery
Lebreton, M.et al.(2015)
•ToM + Pain Matrices battery
Dodell-Feder, D.et al.(2010)
Jacoby, N.et al.(2015)
Richardson, H.et al.(2018)
•Visual Short-Term Memory + Enumeration
tasks
Knops, A.et al.(2014)
•Self-Reference Effect task
Genon, S.et al.(2014)
•“Bang!” task
Campbell, K. L.et al.(2015)First + Second releases:
All contrasts:
Elementary contrasts:
Spearman correlation
First Release:0.21(p≤10
−17
)
Second Release:0.21(p≤10
−13
)
First+Second Releases:0.23(p≤10
−72
)
32/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (1/2)
Online Survey to catalog nomenclature
practises:
•77 total respondents and 611 partial
respondents
•Randomized presentation of 93 brain
network images and 7lureensembles
•3 top consensual networks:
‘somatotopic’, ‘default’ and ‘visual’
•Least consensual networks: ‘salience’ and
‘fronto-parietal’
•The default network was the only one
identified reliably.
The WHATNET Group
Uddin,
L. Q., (...),Pinho, A. L., (...), Yeo, B. T., & Spreng, R. N. Netw
Neurosci (2023)
33/44
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (1/2)
Online Survey to catalog nomenclature
practises:
•77 total respondents and 611 partial
respondents
•Randomized presentation of 93 brain
network images and 7lureensembles
•3 top consensual networks:
‘somatotopic’, ‘default’ and ‘visual’
•Least consensual networks: ‘salience’ and
‘fronto-parietal’
•The default network was the only one
identified reliably.
The WHATNET Group
Uddin,
L. Q., (...),Pinho, A. L., (...), Yeo, B. T., & Spreng, R. N. Netw
Neurosci (2023)
33/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (1/2)
Online Survey to catalog nomenclature
practises:
•77 total respondents and 611 partial
respondents
•Randomized presentation of 93 brain
network images and 7lureensembles
•3 top consensual networks:
‘somatotopic’, ‘default’ and ‘visual’
•Least consensual networks: ‘salience’ and
‘fronto-parietal’
•The default network was the only one
identified reliably.
The WHATNET Group
Uddin, L. Q., (...),Pinho, A. L., (...), Yeo, B. T., & Spreng, R. N.
Netw Neurosci (2023)
33/44
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (1/2)
Online Survey to catalog nomenclature
practises:
•77 total respondents and 611 partial
respondents
•Randomized presentation of 93 brain
network images and 7lureensembles
•3 top consensual networks:
‘somatotopic’, ‘default’ and ‘visual’
•Least consensual networks: ‘salience’ and
‘fronto-parietal’
•The default network was the only one
identified reliably.
The WHATNET Group
Uddin, L. Q., (...),Pinho, A. L., (...), Yeo, B. T., & Spreng, R. N.
Netw Neurosci (2023)
33/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (1/2)
Online Survey to catalog nomenclature
practises:
•77 total respondents and 611 partial
respondents
•Randomized presentation of 93 brain
network images and 7lureensembles
•3 top consensual networks:
‘somatotopic’, ‘default’ and ‘visual’
•Least consensual networks: ‘salience’ and
‘fronto-parietal’
•The default network was the only one
identified reliably.
The WHATNET Group
Uddin, L. Q., (...),Pinho, A. L., (...), Yeo, B. T., & Spreng, R. N.
Netw Neurosci (2023)
33/44
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (1/2)
Online Survey to catalog nomenclature
practises:
•77 total respondents and 611 partial
respondents
•Randomized presentation of 93 brain
network images and 7lureensembles
•3 top consensual networks:
‘somatotopic’, ‘default’ and ‘visual’
•Least consensual networks: ‘salience’ and
‘fronto-parietal’
•The default network was the only one
identified reliably.
The WHATNET Group
Uddin, L. Q., (...),Pinho, A. L., (...), Yeo, B. T., & Spreng, R. N.
Netw Neurosci (2023)
33/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (1/2)
Online Survey to catalog nomenclature
practises:
•77 total respondents and 611 partial
respondents
•Randomized presentation of 93 brain
network images and 7lureensembles
•3 top consensual networks:
‘somatotopic’, ‘default’ and ‘visual’
•Least consensual networks: ‘salience’ and
‘fronto-parietal’
•The default network was the only one
identified reliably.
The WHATNET Group
Uddin, L. Q., (...),Pinho, A. L., (...), Yeo, B. T., & Spreng, R. N.
Netw Neurosci (2023)
33/44
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (1/2)
Online Survey to catalog nomenclature
practises:
•77 total respondents and 611 partial
respondents
•Randomized presentation of 93 brain
network images and 7lureensembles
•3 top consensual networks:
‘somatotopic’, ‘default’ and ‘visual’
•Least consensual networks: ‘salience’ and
‘fronto-parietal’
•The default network was the only one
identified reliably.
The WHATNET Group
Uddin, L. Q., (...),Pinho, A. L., (...), Yeo, B. T., & Spreng, R. N.
Netw Neurosci (2023)
33/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (1/2)
Online Survey to catalog nomenclature
practises:
•77 total respondents and 611 partial
respondents
•Randomized presentation of 93 brain
network images and 7lureensembles
•3 top consensual networks:
‘somatotopic’, ‘default’ and ‘visual’
•Least consensual networks: ‘salience’ and
‘fronto-parietal’
•The default network was the only one
identified reliably.
The WHATNET Group
Uddin, L. Q., (...),Pinho, A. L., (...), Yeo, B. T., & Spreng, R. N.
Netw Neurosci (2023)
1.What constitutes a brain network?
2.Are there universal and reproducible
brain networks that can be observed
across individuals?
33/44
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (1/2)
Online Survey to catalog nomenclature
practises:
•77 total respondents and 611 partial
respondents
•Randomized presentation of 93 brain
network images and 7lureensembles
•3 top consensual networks:
‘somatotopic’, ‘default’ and ‘visual’
•Least consensual networks: ‘salience’ and
‘fronto-parietal’
•The default network was the only one
identified reliably.
The WHATNET Group
Uddin, L. Q., (...),Pinho, A. L., (...), Yeo, B. T., & Spreng, R. N.
Netw Neurosci (2023)
1.What constitutes a brain network?
2.Are there universal and reproducible
brain networks that can be observed
across individuals?
33/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (2/2)
Some recommendations for reporting network results: •Task fMRI contrasts derived from univariate GLM analysis do not necessarily
comprise a network;
•To determine network affiliations of novel findings, use and reference one or more
existing parcellation schemes;
•Follow COBIDAS reporting guidelines for data sharing and analysis.34/44
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (2/2)
Some recommendations for reporting network results: •Task fMRI contrasts derived from univariate GLM analysis do not necessarily
comprise a network;
•To determine network affiliations of novel findings, use and reference one or more
existing parcellation schemes;
•Follow COBIDAS reporting guidelines for data sharing and analysis.34/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (2/2)
Some recommendations for reporting network results: •Task fMRI contrasts derived from univariate GLM analysis do not necessarily
comprise a network;
•To determine network affiliations of novel findings, use and reference one or more
existing parcellation schemes;
•Follow COBIDAS reporting guidelines for data sharing and analysis.34/44
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (2/2)
Some recommendations for reporting network results: •Task fMRI contrasts derived from univariate GLM analysis do not necessarily
comprise a network;
•To determine network affiliations of novel findings, use and reference one or more
existing parcellation schemes;
•Follow COBIDAS reporting guidelines for data sharing and analysis.34/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (2/2)
Some recommendations for reporting network results: •Task fMRI contrasts derived from univariate GLM analysis do not necessarily
comprise a network;
•To determine network affiliations of novel findings, use and reference one or more
existing parcellation schemes;
•Follow COBIDAS reporting guidelines for data sharing and analysis.34/44
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (2/2)
Some recommendations for reporting network results: •Task fMRI contrasts derived from univariate GLM analysis do not necessarily
comprise a network;
•To determine network affiliations of novel findings, use and reference one or more
existing parcellation schemes;
•Follow COBIDAS reporting guidelines for data sharing and analysis.34/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (2/2)
Some recommendations for reporting network results: •Task fMRI contrasts derived from univariate GLM analysis do not necessarily
comprise a network;
•To determine network affiliations of novel findings, use and reference one or more
existing parcellation schemes;
•Follow COBIDAS reporting guidelines for data sharing and analysis.34/44
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (2/2)
Some recommendations for reporting network results: •Task fMRI contrasts derived from univariate GLM analysis do not necessarily
comprise a network;
•To determine network affiliations of novel findings, use and reference one or more
existing parcellation schemes;
•Follow COBIDAS reporting guidelines for data sharing and analysis.34/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (2/2)
Some recommendations for reporting network results: •Task fMRI contrasts derived from univariate GLM analysis do not necessarily
comprise a network;
•To determine network affiliations of novel findings, use and reference one or more
existing parcellation schemes;
•Follow COBIDAS reporting guidelines for data sharing and analysis.34/44
WHATNET: Workgroup for HArmonized Taxonomy of NETworks (2/2)
Some recommendations for reporting network results: •Task fMRI contrasts derived from univariate GLM analysis do not necessarily
comprise a network;
•To determine network affiliations of novel findings, use and reference one or more
existing parcellation schemes;
•Follow COBIDAS reporting guidelines for data sharing and analysis.34/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
NCT: Network Correspondence Toolbox
Kong, R., Spreng, R. N., (...),Pinho, A. L., (...), Yeo, B. T., &
Uddin, L. Q. [under review]
•Computation ofDice Overlap
Coefficientbetween input data and
parcellation scheme.
•Spin-test permutation for statistical
significance
Checkout the toolbox!
cbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondence
35/44
NCT: Network Correspondence Toolbox
Kong, R., Spreng, R. N., (...),Pinho, A. L., (...), Yeo, B. T., &
Uddin, L. Q. [under review]
•Computation ofDice Overlap
Coefficientbetween input data and
parcellation scheme.
•Spin-test permutation for statistical
significance
Checkout the toolbox!
cbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondencecbignetworkcorrespondence
35/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Axis 3:
Open and Reproducible Science
Axis 3:
Open and Reproducible Science
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Public Repository of Behavioral Protocols
A public repository of task protocols is necessary for:
•reproducibility
•implementation of multi-task paradigms
•development of better ontologies
Hallmarks:
•storage of behavioral protocols (cross-platform compatible)
•guidelines with normatives about:
psychological domains
types of designs (block-design, event-related, naturalistic)
effects-of-interest (description of conditions, contrasts)
comparisons strategies (subtraction, factorial, parametric)
target population
37/44
Public Repository of Behavioral Protocols
A public repository of task protocols is necessary for:
•reproducibility
•implementation of multi-task paradigms
•development of better ontologies
Hallmarks:
•storage of behavioral protocols (cross-platform compatible)
•guidelines with normatives about:
psychological domains
types of designs (block-design, event-related, naturalistic)
effects-of-interest (description of conditions, contrasts)
comparisons strategies (subtraction, factorial, parametric)
target population
37/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
NeuroCausal
NeuroCausalis an online platform that allows open sharing, storage, and synthesis of
clinical, meta data, under the FAIR principles.
Bilgin, I. P., Paugam, F., Huang, R.,Pinho, A. L., Zhou, Y., Lukic, S., Pinheiro-Chagas, P., & Borghesani, V. ApertureNeuro (2024)
The goalis to create probabilistic maps synthesizing transdiagnostic information.
Structure
•Text-to-brain tool;
•Automated meta-analysis infrastructure based on brain-lesion information;
•LeveragesNeuroQueryto learn the causal mapping.
More information available on:https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/
38/44
NeuroCausal
NeuroCausalis an online platform that allows open sharing, storage, and synthesis of
clinical, meta data, under the FAIR principles.
Bilgin, I. P., Paugam, F., Huang, R.,Pinho, A. L., Zhou, Y., Lukic, S., Pinheiro-Chagas, P., & Borghesani, V. ApertureNeuro (2024)
The goalis to create probabilistic maps synthesizing transdiagnostic information.
Structure
•Text-to-brain tool;
•Automated meta-analysis infrastructure based on brain-lesion information;
•LeveragesNeuroQueryto learn the causal mapping.
More information available on:https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/
38/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
NeuroCausal
NeuroCausalis an online platform that allows open sharing, storage, and synthesis of
clinical, meta data, under the FAIR principles.
Bilgin, I. P., Paugam, F., Huang, R.,Pinho, A. L., Zhou, Y., Lukic, S., Pinheiro-Chagas, P., & Borghesani, V. ApertureNeuro (2024)
The goalis to create probabilistic maps synthesizing transdiagnostic information.
Structure
•Text-to-brain tool;
•Automated meta-analysis infrastructure based on brain-lesion information;
•LeveragesNeuroQueryto learn the causal mapping.
More information available on:https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/
38/44
NeuroCausal
NeuroCausalis an online platform that allows open sharing, storage, and synthesis of
clinical, meta data, under the FAIR principles.
Bilgin, I. P., Paugam, F., Huang, R.,Pinho, A. L., Zhou, Y., Lukic, S., Pinheiro-Chagas, P., & Borghesani, V. ApertureNeuro (2024)
The goalis to create probabilistic maps synthesizing transdiagnostic information.
Structure
•Text-to-brain tool;
•Automated meta-analysis infrastructure based on brain-lesion information;
•LeveragesNeuroQueryto learn the causal mapping.
More information available on:https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/
38/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
NeuroCausal
NeuroCausalis an online platform that allows open sharing, storage, and synthesis of
clinical, meta data, under the FAIR principles.
Bilgin, I. P., Paugam, F., Huang, R.,Pinho, A. L., Zhou, Y., Lukic, S., Pinheiro-Chagas, P., & Borghesani, V. ApertureNeuro (2024)
The goalis to create probabilistic maps synthesizing transdiagnostic information.
Structure
•Text-to-brain tool;
•Automated meta-analysis infrastructure based on brain-lesion information;
•LeveragesNeuroQueryto learn the causal mapping.
More information available on:https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/
38/44
NeuroCausal
NeuroCausalis an online platform that allows open sharing, storage, and synthesis of
clinical, meta data, under the FAIR principles.
Bilgin, I. P., Paugam, F., Huang, R.,Pinho, A. L., Zhou, Y., Lukic, S., Pinheiro-Chagas, P., & Borghesani, V. ApertureNeuro (2024)
The goalis to create probabilistic maps synthesizing transdiagnostic information.
Structure
•Text-to-brain tool;
•Automated meta-analysis infrastructure based on brain-lesion information;
•LeveragesNeuroQueryto learn the causal mapping.
More information available on:https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/
38/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
NeuroCausal
NeuroCausalis an online platform that allows open sharing, storage, and synthesis of
clinical, meta data, under the FAIR principles.
Bilgin, I. P., Paugam, F., Huang, R.,Pinho, A. L., Zhou, Y., Lukic, S., Pinheiro-Chagas, P., & Borghesani, V. ApertureNeuro (2024)
The goalis to create probabilistic maps synthesizing transdiagnostic information.
Structure
•Text-to-brain tool;
•Automated meta-analysis infrastructure based on brain-lesion information;
•LeveragesNeuroQueryto learn the causal mapping.
More information available on:https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/
38/44
NeuroCausal
NeuroCausalis an online platform that allows open sharing, storage, and synthesis of
clinical, meta data, under the FAIR principles.
Bilgin, I. P., Paugam, F., Huang, R.,Pinho, A. L., Zhou, Y., Lukic, S., Pinheiro-Chagas, P., & Borghesani, V. ApertureNeuro (2024)
The goalis to create probabilistic maps synthesizing transdiagnostic information.
Structure
•Text-to-brain tool;
•Automated meta-analysis infrastructure based on brain-lesion information;
•LeveragesNeuroQueryto learn the causal mapping.
More information available on:https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/
38/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
NeuroCausal
NeuroCausalis an online platform that allows open sharing, storage, and synthesis of
clinical, meta data, under the FAIR principles.
Bilgin, I. P., Paugam, F., Huang, R.,Pinho, A. L., Zhou, Y., Lukic, S., Pinheiro-Chagas, P., & Borghesani, V. ApertureNeuro (2024)
The goalis to create probabilistic maps synthesizing transdiagnostic information.
Structure
•Text-to-brain tool;
•Automated meta-analysis infrastructure based on brain-lesion information;
•LeveragesNeuroQueryto learn the causal mapping.
More information available on:https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/
38/44
NeuroCausal
NeuroCausalis an online platform that allows open sharing, storage, and synthesis of
clinical, meta data, under the FAIR principles.
Bilgin, I. P., Paugam, F., Huang, R.,Pinho, A. L., Zhou, Y., Lukic, S., Pinheiro-Chagas, P., & Borghesani, V. ApertureNeuro (2024)
The goalis to create probabilistic maps synthesizing transdiagnostic information.
Structure
•Text-to-brain tool;
•Automated meta-analysis infrastructure based on brain-lesion information;
•LeveragesNeuroQueryto learn the causal mapping.
More information available on:https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/https://neurocausal.github.io/
38/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Concluding Remarks
With Deep Behavioral Phenotyping, we can: •Extract shared representations of tasks and link elementary brain function to
mental function.•Obtain subject-specific parcellations that better account for individual brain
organization
•Capitalize on features extraction to:
understand the hierarchy of cognitive systems
dissociate competing cognitive theories
•Develop better cognitive ontologies and brain taxonomies.
39/44
Concluding Remarks
With Deep Behavioral Phenotyping, we can: •Extract shared representations of tasks and link elementary brain function to
mental function.•Obtain subject-specific parcellations that better account for individual brain
organization
•Capitalize on features extraction to:
understand the hierarchy of cognitive systems
dissociate competing cognitive theories
•Develop better cognitive ontologies and brain taxonomies.
39/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Concluding Remarks
With Deep Behavioral Phenotyping, we can: •Extract shared representations of tasks and link elementary brain function to
mental function.•Obtain subject-specific parcellations that better account for individual brain
organization
•Capitalize on features extraction to:
understand the hierarchy of cognitive systems
dissociate competing cognitive theories
•Develop better cognitive ontologies and brain taxonomies.
39/44
Concluding Remarks
With Deep Behavioral Phenotyping, we can: •Extract shared representations of tasks and link elementary brain function to
mental function.•Obtain subject-specific parcellations that better account for individual brain
organization
•Capitalize on features extraction to:
understand the hierarchy of cognitive systems
dissociate competing cognitive theories
•Develop better cognitive ontologies and brain taxonomies.
39/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Concluding Remarks
With Deep Behavioral Phenotyping, we can: •Extract shared representations of tasks and link elementary brain function to
mental function.•Obtain subject-specific parcellations that better account for individual brain
organization
•Capitalize on features extraction to:
understand the hierarchy of cognitive systems
dissociate competing cognitive theories
•Develop better cognitive ontologies and brain taxonomies.
39/44
Concluding Remarks
With Deep Behavioral Phenotyping, we can: •Extract shared representations of tasks and link elementary brain function to
mental function.•Obtain subject-specific parcellations that better account for individual brain
organization
•Capitalize on features extraction to:
understand the hierarchy of cognitive systems
dissociate competing cognitive theories
•Develop better cognitive ontologies and brain taxonomies.
39/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Concluding Remarks
With Deep Behavioral Phenotyping, we can: •Extract shared representations of tasks and link elementary brain function to
mental function.•Obtain subject-specific parcellations that better account for individual brain
organization
•Capitalize on features extraction to:
understand the hierarchy of cognitive systems
dissociate competing cognitive theories
•Develop better cognitive ontologies and brain taxonomies.
39/44
Concluding Remarks
With Deep Behavioral Phenotyping, we can: •Extract shared representations of tasks and link elementary brain function to
mental function.•Obtain subject-specific parcellations that better account for individual brain
organization
•Capitalize on features extraction to:
understand the hierarchy of cognitive systems
dissociate competing cognitive theories
•Develop better cognitive ontologies and brain taxonomies.
39/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Concluding Remarks
With Deep Behavioral Phenotyping, we can: •Extract shared representations of tasks and link elementary brain function to
mental function.•Obtain subject-specific parcellations that better account for individual brain
organization
•Capitalize on features extraction to:
understand the hierarchy of cognitive systems
dissociate competing cognitive theories
•Develop better cognitive ontologies and brain taxonomies.
39/44
Concluding Remarks
With Deep Behavioral Phenotyping, we can: •Extract shared representations of tasks and link elementary brain function to
mental function.•Obtain subject-specific parcellations that better account for individual brain
organization
•Capitalize on features extraction to:
understand the hierarchy of cognitive systems
dissociate competing cognitive theories
•Develop better cognitive ontologies and brain taxonomies.
39/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
Future Directions
•Functional Atlasing: Top-down approach to study functional specificity;•Functional Atlasing: Parameterize HBP model to improve individual parcellations
across brain regions;
•Music Cognition: Hierarchical encoding of musical features;•Open Science: Development of a repository of behavioral protocols and other
data-science tools (NCT, NeuroCausal);
•Open Science: Software development (Nilearn, HBP, RSA).
40/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Resources
41/44
Resources
41/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Thanks!
42/44
Thanks!
42/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments ○www.winrepo.org
○> 1900 profiles
○> 120 recommendations
○Easy search
Repository for Women in Neuroscience
@WINRePo1
Support the project:
○ Sign up
○ Spread the word
○ Submit recommendations
https://discord.gg/YHc9g3rN
https://www.linkedin.com/groups/12659766/
@[email protected]
43/44
○> 1900 profiles
○> 120 recommendations
○Easy search
Repository for Women in Neuroscience
@WINRePo1
Support the project:
○ Sign up
○ Spread the word
○ Submit recommendations
https://discord.gg/YHc9g3rN
https://www.linkedin.com/groups/12659766/
@[email protected]
43/44
SummaryOverviewMulti-task FrameworkOntologies and TaxonomiesOpen ScienceConclusion and Acknowledgments
Thank you for your attention.
Thank you for your attention.
Appendix
Appendix
Download
Download Slideshow Get the original presentation fileQuick Actions
Statistics
- Views 60
- Slides 124
- Age 546 days
Related Slideshows
11
8-top-ai-courses-for-customer-support-representatives-in-2025.pptx
JeroenErne2
48 views
10
7-essential-ai-courses-for-call-center-supervisors-in-2025.pptx
JeroenErne2
47 views
13
25-essential-ai-courses-for-user-support-specialists-in-2025.pptx
JeroenErne2
37 views
11
8-essential-ai-courses-for-insurance-customer-service-representatives-in-2025.pptx
JeroenErne2
34 views
21
Know for Certain
DaveSinNM
22 views
17
PPT OPD LES 3ertt4t4tqqqe23e3e3rq2qq232.pptx
novasedanayoga46
26 views