1	Mapping the automatic parallel computation with domain general control Walter Schneider University of Pittsburgh ONR Contractors Meeting June 5, 2005 Pittsburgh
2	Outline Dual Processing theory, computation and cortical processing Development of automatic processing in single and dual tasks training to enable processing Dynamic changes in CP in native and supported mode processing Processing of complex task AEGIS control ERP tracking of CP networks
3	Take Home Message #1 Two Systems of Cortex
4	Dual Nature of human processing
5	Dual Nature of human processing
6	Why does cognition utilize automatic and controlled processing? Complementary befits and costs Differential environmental demands Architectural constraints
7	1. Complementary Benefits & Deficits
8	2. Differential Environmental demands
9	3. Architectural constraints
10	Automatic Processing 10,000 modules providing representation and priority assessment of all active information Message Association
11
12	Take Home Messages #2 Development of automaticity leads to dramatic reductions in brain activity identifies the cortical structures of control processing.
13	Experiment 1 Developing automaticity in a search task In an fMRI study, we scanned before (Session 1) and after (Session 5) training in a search task
14	Experiment 1 Developing automaticity in a search task Subjects were scanned before (Session 1) and after (Session 5) training in a search task
15
16	Session 1
17
18	Take Home Messages #3 There is a single domain general control network that is active in all novel or control process tasks. Same brain areas active across wide variety of tasks Same areas for simple and complex tasks
19	Domain-generality across input modalities
20	Domain Specificity of Areas that Remain
21	Experiment 3 Line Search Mapping CP Network With Simple Task
22	Activation Relative to Control
23	Experiment 3 Complex Performance Does activation of a complex task utilize the domain general network we describe? AEGIS Navy Task Simulator developed ONR support by Ling Rothrock Pennsylvania State University Task involves Visually track radar screen Use keys to “hook” aircraft Sensor readings on radar to classify aircraft Identify unknown aircraft as friendly or hostile. If a hostile aircraft is within 50nm of your own ship, issue a warning. If a hostile aircraft is within 20nm of your own ship, assign or illuminate it. If a hostile aircraft is within 10nm of your own ship, engage (fire a missile) at it.
24	Task
25	Experiment Details AEGIS task: 8 subjects participated in two hours of behavioral training prior to a 1.5 hour fMRI scan. In the AEGIS task, subjects act as an Anti-Air Warfare Coordinator on board an AEGIS cruiser ship. The primary task is to protect your ship. There are four rules of engagement that must be followed throughout task performance Identify all unknown aircraft as friendly or hostile Warn any hostile aircraft that come within 50 nautical miles (nm) of your ship Assign weapons to hostile aircraft that come within 20nm of your ship Engage hostile aircraft that come within 10nm of your ship
26	Experiment Details In the second hour, subjects practiced training tasks in which they followed all rules of engagement together. Four different two minute tasks were practiced five times each. These same tasks were used in the fMRI session FMRI session 6 runs Control task: subjects passively viewed a 30s video of typical AEGIS task performance while randomly pressing keys Each run was structured as follows: 30s control task – 2 min AEGIS task – 30s control task – 2 min AEGIS task – 30s control task 7 subjects also performed two runs of a Line Search task in which they were shown a target line at the beginning of a block, and were told to respond whenever they saw the target line appear during a probe period. (Hard search, hard comparison and a fixation blink control)
27	RDLPFC (X=32, Y=45, Z=25) Goal Processor
28	ACC (X=1, Y=3, Z=49) Activity monitor/decision
29	Posterior Parietal Cortex PPPC Attention Control
30	Anterior Insula “Event evaluation”
31	Summary Areas
32
33	Follow on AEGIS analyses Temporal analysis (ICA, support vector machine) Identify temporal dynamics of each area and the operations in the complex task Determine if we can do “brain reading” of control areas to identify type of control operation, task loading, and performance quality
34	"Network of areas involved specifically..." Network of areas involved specifically in controlled processing Before practice, VM and CM use same control network The role of this control network is to scaffold performance in tasks that cannot (yet) be performed in an automatic processing state Only consistent practice produces a change in activity Control regions are domain general across input (visual & auditory), material types (verbal, spatial), and tasks (search, paired-associate learning, AEGIS radar control)
35	Take Home Messages #4 High workload training facilitates transition to fully automatic processing Single task training alone is not sufficient Extended dual task training does lead to great reductions
36	Background Garavan et al. (2000) found practice-related decreases with using a WM visuospatial delayed-match-to sample task (dot location task). Prefrontal and parietal areas reduced activation following brief (80 trials) and extended practice (880 trials). Hempel et al. (2004) found inverse U-shape practice function when training the n-back task over 4 weeks. Olsen et. al (2003) found that extensive VM training, however, prevents practice reductions in forward-backward (i.e. VM) sequence learning task. Five weeks of training resulted in practice-related increase. Chein and Schnieder (2003; in preparation) practice meta-analysis and several CM-VM fMR studies suggest that DLPFC, ACC, PPC, etc. are involved in learning without regard to domain (Presented at ONR 2004). Jansma et al., (2001) found practiced-related reductions using Sternberg (1966) search paradigm (both CM and VM search trained) only under consistently-mapped condition.
37	Visual Multi-Task (VMT)
38
39
40
41	Comment on Overlays & Peak effects
42	Domain General Overlay (DGO) Regions
43	DGO Analysis Yields Control Areas Designed to find areas involved in novel task performance. At Scan 1: single-task (ST) are extensively practiced but the dual-task (DT) is “unpracticed”. Both dual-task and single-task are consistently- mapped throughout study. At Scan 2: both ST and DT are practiced
44
45
46
47
48
49
50	Take Home Messages Domain General Overlay (DGO) analysis yields predicted network areas engaged in novel dual-task performance. Areas show greater response to “controlled” dual-task performance when contrasted with “automatic” single-task performance. This pattern is consistent across DGO regions and dual-task pairs. Furthermore, different subjects were used for the symbol-word task and the pattern-letter task.
51	Does the DGO Analysis elicit practice-related dual-task reductions at Scan 2?
52
53
54
55
56
57
58	Does the DGO Analysis elicit practice-related dual-task reductions at Scan 2? Yes…. but perhaps another approach would reveal practice-related increases in regions proximal to “DGO” regions? Or distal to DGO regions? learning-related reorganization
59
60	Any Practice related Increases? Direct comparison Scan 2> Scan 1
61	Take Home Messages DGO reveals wide-spread reductions in domain general learning network. Only one region found at scan 2, left PPC in only the SW condition: (Dual>Symbol) overlap (Dual> Word) A direct comparison looking for anything that is more active in scan 2 compared to scan 1 results in no activation. However… the subjects are not brain dead at scan 2. And the DGO areas are still active just to lesser extent when compared to novel performance. The lack of activation reflects the automatic performance of the dual-task at scan 2.
62	EEG/ERP Control Process Separation Gwen A. Frishkoff & Don M. Tucker It may be possible to develop monitoring devices to provide real-time readout of activity within and between subsystems of the control network.
63	Goal: Tracking Network Interactions with High-Density EEG
64	Geodesic Sensor Net (Electrical Geodesics, Inc.)
65	ERP Study Gwen Frishkoff, Ph.D. & Oregon Collaborators NeuroInformatics Center, University of Oregon Electrical Geodesics, Inc. (Eugene, Oregon) Pilot work with 13 subjects 2 hours of data on the CPS line task ERP/EEG Analyses Conventional ERPs (complete) 2-second, stimulus-locked averages Topographic mapping with dense array Advanced EEG & ERP analyses (ongoing) Mapping slow activity (15-second ERPs) Joint Time–Frequency (wavelet) analyses Spatiotemporal source localization Integration with fMRI results for individual subjects
66	CPS Line Search: 3 Factors
67
68
69	Major effects for Conventional ERP ERP analyses of line search show strong ERP activation with differential spatial and temporal patterns that are analogous to the FMRI results. Basic phenomena Effect of search difficulty has a parietal distribution Effect of comparison has a frontal distribution Task switching shows frontal midline distribution Time course effects over extended processing Drop out of frontal areas as processing continues Maintenance in the search condition of parietal activation
70	Search difficulty effect
71	Comparison difficulty effect
72	Dynamics in ERP Early versus late – we see substantial changes in CP network over time paralleling the fMRI results These results point to dynamic interactions between within the CP subareas and between the CP network and sensorimotor regions Individual differences in ERP dynamics are impressive, with some participants showing strong stimulus-evoked activity over all regions, and others showing oscillations that are confined to orbitofrontal regions These dynamics may be related to differences in “expertise” (CP on the line task). Further studies are planned to examine this hypothesis.
73	Work in Progress –1 Mapping sustained (blocked) versus phasic (stimulus-locked) activity Advanced signal analysis for artifact removal using APECS (Automated Protocol for Electromagnetic Component Separation) ICA, PCA, SOBI, Wavelets NeuroInformatics Center, University of Oregon Addressing the problem of superposition Techniques for ERP temporal and spatial component separation (PCA, ICA) Integration of ERP and fMRI data Spatiotemporal Source Analysis, seeding dipoles from centroids of activation in fMRI data (BESA & Brain Voyager)
74	Work in Progress–2 Joint Time-Frequency (wavelet) analyses allow us to track synronization of activity within particular frequency bands (alpha, gamma, etc.) may point to dynamic interactions (i.e., coherence) between CP regions Don Tucker, Electric Geodesics, Inc. Bob Frank, NeuroInformatics Center Vince Samar, University of Rochester
75	Future Prospects New studies to refine and elaborate ERP characterization of CP network activity Tracking degree of automaticity Training on CP strategies to track & enhance performance
76	Take Home Messages #6 (comment only) The control network is differentiated into specialized subsystems including:
77	Take Home Messages #7 (Comment only) Can Provide Feedback in Real Time of fMRI activation on single subjects
78	Activity Monitoring Is Ubiquitous In Complex Dynamic Systems Present in most complex systems Internet, electrical grid, communication grid, chemical plants, air traffic control, railroad systems, computer process scheduling, business management
79	Take Home Messages
80	Thanks & Links to More Information
81	ERP Tracking of Control Processing Function Real time distribution patterns of each function (work with Don Tucker U. Oregon)
82	Index Intro Imaging Exp 1 drop out Exp 1 domain general Exp 2 paired associate Exp 3 AEGIS Exp 4 Differentiation Models
83	What Are The Core Architectural Features Of The Brain That Support Complex Cognition?
84
85
86	Concluding Comment on Duality Duality of human performance Fast parallel automatic processing Slow serial flexible controlled processing Duality of computational models Parallel computation of connectionist hierarchies Serial processing of production system/recurrent net models Duality of cortical processing Many domain specific regions that work in parallel Single domain general control network that is serial