<< [[Human visual processing]], [[Working Memory]] > [[YousifEtAl-'21-SWM.pdf|Using Space to Remember]] ## Intro ### What is the relationship between space/spatial association and working memory? Two perspectives, not mutually exclusive: - Spatial interference - Items appearing in the same location in space interfere with one another - Visuospatial bootstrapping - Items presented with stable spatial mappings (ex: keypad) are better remembered than items without ### Spatial interference > Space supports the binding of features to objects, therefore objects overlapping in space interfere with one another "Feature binding error" - you see two shapes in the same location, and they interfere with each other in your memory Ex: you see a blue circle and green triangle together, then misremember a green circle or blue triangle Experiments: - People performed worse at remembering the orientation of a line when multiple lines appeared in the same location - Lines with overlapping colors do not have this effect (?) ### Visuospatial bootstrapping #### People are better at remembering verbal information when it is mapped onto consistent spatial locations - Digits are better remembered when presented in a keypad formation - Personal experience - Off the top of my head I can't remember my the digits of my phone passcode but I can easily unlock it without looking at the screen - This may only be true if the mapping is stored in long term memory - if the keypad is rearranged people do not remember digits better #### Items in working memory are automatically spatialized Example: People have to memorize a sequence of objects (`orange, apple, pear, banana, cherry`). - People appear to respond faster to **earlier** items in the list (`orange...`) with their **left** hand, and respond faster to **later** items in the list (`...cherry`) with their **right hand** - still confused about exactly what's happening in these studies - The lateralization of memory associations between the left and right hands implies the sequence may be mapped in space somehow, and maybe this is part of working memory function - "Spatial–Positional Association of Response Codes (SPoARC)" ### many models of the working memory system #### subsystem model - visuospatial sketchpad -> visual information - phonological loop -> verbal information - central executive - episodic buffer -> interacts with short and long term memory? #### there are just a lot of interactions between different [[systems]], shit is complex - Long term memory influences short term memory - Visuospatial information influences retention of verbal information ## What this study is studying > [!Hypothesis] > Short-term spatial structure supports working memory maintenance, even in tasks that pose no explicit spatial requirements ### Spatial structure and working memory - Spatial structure: consistent mapping between objects and space (visuospatial bootstrapping) - Low spatial structure: a grocery store whose aisles are rearranged every time you visit - High spatial structure: a meeting where people always sit in the same seats Over a short timescale - *are objects repeatedly appearing in the same location better remembered than objects repeatedly appearing in different locations*? - From personal experience, it's definitely true that a consistent layout of objects is embedded in my memory. I think the relationship between long term memory and working memory, and where visual representations come in, is the question here. How short is the time scale to learn a visual representation? Is it the visual itself that is in working memory (ie the keypad) or the information (the digits) are in working memory and the visual mapping (keypad) has to be in long term memory first? > [!Hypothesis] > Task-irrelevant spatial information will benefit visual working memory more than matched non-spatial visual information (i.e., color) and non-visual information (i.e., audio information) ### Specific goals/questions to answer The interaction of space, long-term memory, and working memory - Does visuospatial bootstrapping **depend on** spatial mappings held in long term memory? - Do mappings in long-term memory interfere with short-term mappings? - This study will test **short-term** spatial mappings over **6-10s** in a case where there are no long-term mappings How does spatial information influence working memory? - What if objects are mapped onto stable locations, but other objects are mapped onto the same locations? How do spatial structure and verbal rehearsal compete when people can rely on both? - verbal rehearsal = saying a list of shapes ### seven experiments around memorizing sequences of common shapes Shared features - Participants always remembered sequences they were free to verbally rehearse - Sequences were structured such that location, color, or audio information varied between the different shapes #### 1a and 1b: space vs color Participants saw a series of 5-7 shapes (**three unique shapes**) in random order, and had to recall what the shapes were and the order they were in ##### Space-structured trial - Any shape appearing multiple times appeared in the same location - Colors of all shapes were randomized ##### Color-structured trial - Any shape appearing multiple times appeared in the same color - No two shapes were the same color - Locations of all shapes were randomized ##### Visuals used and test method Sample size - 24 undergrads and then 16 undergrads Shapes: Circle, Pentagon, Diamond Colors: Red, Green, Blue, Yellow Participants watched a display with four boxes. 5-7 shapes appeared in boxes **one at a time**, for 1000ms, with 500ms in between. - The first three shapes were always unique Participants were told that the trials were color-structured and shape-structured, and they could use that information if they wanted ![[Screen Shot 2022-09-04 at 1.34.04 PM.jpg|lg]] After the shapes appeared, the three shapes appeared in white and the participants were asked to click the shapes in the order they saw them (clicking the same shape multiple times in a row if necessary) - The participants weren't shown how many shapes they had clicked, but the next round automatically started after enough shapes were clicked ##### Results - Accuracy was higher for space-structured trials than color-structured trials - Set size (5, 6 or 7 shapes) also had a strong effect - related to [[People can most effectively compare 4 things at a time]]? - ![[Screen Shot 2022-09-04 at 1.41.49 PM.jpg|med]] - People were surveyed and their responses sorted by whether they mentioned a verbal rehearsal (saying the shapes in order in your head) or a spatial rehearsal ("if I forgot the pattern, I would try to remember the locations"). - The majority of people described verbal rehearsal ##### Discussion/takeaways - Spatial structure benefits working memory even when people aren't aware of it and while they are actively using verbal rehearsal #### 2a and 2b: space vs color vs unstructured - Does color structure benefit accuracy in working memory less than spatial structure? - Does color structure adversely affect working memory of shapes? ##### Methods - 2a sample size - 12 undergrads - 2b sample size - 120 participants - some were excluded for failing an attention check - Unstructured baseline - both color and location are randomized for every shape ##### Results - Accuracy was higher for space-structured trials, and about the same for color-structured and unstructured trials. ![[Screen Shot 2022-09-04 at 1.54.32 PM.jpg|med]] Observation - although overall accuracy is higher for space-structured, a chunk of participants did better with color-structured than shape-structured ##### Discussion/takeaway - Looks like no evidence of color-structure benefits #### 3: space vs sound vs unstructured - Is spatial structure special? What is the benefit of spatial structure compared with audio structure? ##### Methods - A tone of a specific note: `A, C, E, G` was paired with each shape. Color and location were randomized. - Sample size - 18 undergrads ##### Results Audio structure doesn't have the same benefits as spatial structure ![[Screen Shot 2022-09-04 at 1.59.53 PM.jpg|med]] #### 4: what structure matters? - Previous studies report spatial overlap -> working memory interference - ie, items appearing in the same location were remembered worse than items of the same color appearing in different locations - How to decouple spatial interference from spatial structure? > Is the effect of spatial structure caused by the *presence* of structure (i.e., the fact that any given object appears in a consistent location) or the *absence* of overlap (i.e., the fact that no two objects appear in the same location)? ##### Methods - "Overlap" condition - different shapes appear in consistent locations but may overlap with each other - "Separate" condition - different shapes appear in multiple locations but will never overlap with each other ![[Screen Shot 2022-09-04 at 2.06.21 PM.jpg|lg]] ##### Results Accuracy was slightly higher for **overlapping** trials compared to **separate** trials ![[Screen Shot 2022-09-04 at 2.07.58 PM.jpg|lg]] ##### Discussion/takeaway > Memory benefits from binding information to *specific* locations, but not necessarily to *unique* locations - The overlapping condition only used 2 locations, the separate condition used up to 6 locations - Focusing on a smaller subset of locations could reduce demands on attention #### 5: how robust is the effect of spatial structure? - Is the higher accuracy in spatial structure trials based on people realizing they only need to pay attention to 3/4 locations? Changes to the test: - Reduced to only 3 locations and 3 colors - First three items always have unique color, unique location, and unique shape - 3 locations arranged in a line instead of a grid ![[Screen Shot 2022-09-04 at 2.15.18 PM.jpg|lg]] - Sample size - 120 - excluding people who failed attention check or refreshed the page ##### Results - Performance was worse - analysis was done including and excluding accuracy above 50% - Accuracy was slightly higher for space-structured - Space-structured and color-structured trials had closer results here ![[Screen Shot 2022-09-04 at 2.22.09 PM.jpg|lg]] ##### Results/discussion - The boost to color-structured performance could be from the change that the first three shapes all have both spatial structure and color structure - By random change, the color-structured trials could have more spatial structure ### Conclusion - Spatial structure has an effect on working memory even when the spatial information is totally irrelevant to what is being memorized - What could explain the difference between trials? - Simplicity of the task - Predictability of the sequences - Eye movements?