Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Virtual reality experiences promote autobiographical retrieval mechanisms: Electrophysiological correlates of laboratory and virtual experiences

Virtual reality experiences promote autobiographical retrieval mechanisms: Electrophysiological... Recent advancements in memory research indicate that virtual reality (VR) experiences are more vividly memorized as compared to conventional laboratory events. In contrast to the latter, VR experiences are highly immersive, simulating the multimodality, vividness and inclusiveness of real-life experiences. Therefore, VR might enable researchers to identify memory processes underlying events which participants have actually experienced, in contrast to conventional on-screen experiences. To differentiate the electrophysiological correlates of memory processes underlying VR experiences as compared to conventional laboratory experiences, participants watched videos either in a PC condition or in a VR condition, followed by an unannounced recognition memory test. As hypothesized, we replicated the well-established theta old/new effect for the PC condition, but remarkably, this effect was absent in the VR condition. Additionally, the latter was accompanied by significantly lower alpha activity as compared to the PC condition. As increases in theta-band responses are related to top- down control on, and memory load during retrieval, the observed theta responses might rather relate to retrieval effort than to retrieval success per se. Congruently, higher alpha activity measured over occipital sensor areas in the PC condition reflect visually guided search processes within episodic memory. The VR condition comes in with lower alpha activity, reflecting immediate and effortless memory access. Hence, our findings indicate that the retrieval of VR experiences promotes auto- biographical  retrieval mechanisms, whereas recalling conventional laboratory events comes in with higher effort, which might not reflect the mechanisms of everyday memory. Introduction in their respective spatial and temporal context (Tulving, 1983). Extending well beyond EM, AM encompasses highly How people behave in everyday life strongly depends on self-relevant information, especially beliefs and knowledge previous experiences either with a particular situation or about the self, experienced events and their relevance (see personal general knowledge, e.g. concerning the realiza- e.g. Conway, 2005; Greenberg & Rubin, 2003). Hence, AM tion of own goals, acting effectively and relating to other comprises episodic engrams, extending it by self-referential peoples (see Conway, 2005). This kind of information is and emotional processes. The retrieval of autobiographical predominantly encoded in and retrieved from autobiographi- memories is therefore not limited to temporal, spatial or con- cal memory (AM). Similar to episodic memory (EM), auto- textual information, but bears great personal significance biographical engrams encode personally experienced events (Svoboda, McKinnon, & Levine, 2006). The retrieval of such everyday memories promotes the re-experience of the associated emotions (Svoboda et al., 2006), coming in with Electronic supplementary material The online version of this vivid and conscious reliving, and foremost the belief that article (https ://doi.org/10.1007/s0042 6-020-01417 -x) contains they have actually occurred (Rubin, Schrauf & Greenberg supplementary material, which is available to authorized users. 2003; Greenberg & Rubin, 2003). While it is common practice to investigate everyday * Joanna Kisker joanna.kisker@uni-osnabrueck.de memory in the laboratory using paradigms that induce micro-events prior to recognition memory tests (see Cabeza Experimental Psychology I, Institute of Psychology, et al., 2004), these settings are often criticized for lacking Osnabrück University, Seminarstraße 20, 49074 Osnabrück, the complexity and variety of stimuli and response options Germany Vol.:(0123456789) 1 3 2486 Psychological Research (2021) 85:2485–2501 characteristic to real-life experiences (Pan & Hamilton, 2019a; Schöne et al., 2016, Schöne et al. 2019). Hence, VR 2018; Kvavilashvili & Ellis, 2004). Specifically, self-rele- might improve the possibilities to investigate the mecha- vance and self-involvement are rarely realized in laboratory nisms underlying real-life memory (see Parsons, 2015; settings (see e.g. McDermott, Szpunar, & Christ, 2009). Serino & Repetto, 2018; Schöne et al., 2016, Schöne et al. Obviously, such traditional approaches face a trade-off 2019; Kisker et al., 2019b; Burgess et al., 2001). between high experimental control and ecological validity, Initial studies of memory processes under immersive VR i.e. the validity of the results obtained in the laboratory and conditions found that retrieval of VR experiences is not only generalized to everyday life (see Parsons, 2015). enhanced compared to the retrieval of conventional labora- Potentially overcoming this gap between experimen- tory micro-events (see e.g. Serino & Repetto, 2018; Smith, tal control and ecological validity, virtual reality (VR) 2019; Schöne et  al., 2016, Schöne et  al. 2019; Krokos, has gained interest as a methodical tool in psychological Plaisant, & Varshney, 2019; Ernstsen, Mallam & Nazir, research (see e.g. Parsons, 2015; Pan & Hamilton, 2018; 2019; Harman, Joel, Brown, Ross & Johnson, 2017), but Schöne et al., 2017, Kisker, Gruber, & Schöne 2019a, b). For also provides a closer approximation to real-life memory memory research, VR experiences might provide a closer processes (Schöne et al., 2016; Schöne et al. 2019; Kisker approximation to real-life experiences as compared to con- et al., 2019b). In particular, a previous study found evidence ventional laboratory settings. The former is characterized that immersive VR experiences become part of an extensive by a high level of sensory cues and thus, by high fidelity of autobiographical associative network, whereas conventional the represented environment (Dan & Reiner, 2017). Accord- video experiences remain an isolated episodic event (Schöne ingly, VR environments are more pronounced regarding viv- et  al., 2019). Going one step further, the retrieval of VR idness as compared to classical setups (Slater & Wilbur, experiences is proposed to mainly rely on recollection, i.e. 1997), which is also characteristic for AM (Greenberg & vivid and accurate remembering of events (e.g. Atkinson Rubin, 2003). In particular, everyday experiences arise & Juola, 1973; Jacoby & Dallas, 1981) which is associated from the complex, multisensory 3D-environment of the real with AM (Roediger & Marsh, 2003; Conway, 2005). In con- world, while laboratory memories are generated by highly trast, retrieval of memories induced by conventional labora- controlled events rather poor in sensory information (Cabeza tory settings predominantly fall back on familiarity-based & St Jaques, 2007). Moreover, the formation of such memo- mnemonic processes (Kisker et al., 2019b), characterized as ries is accompanied by intuitive and quick monitoring and a subjective, vague feeling to remember a previous experi- closely linked to self-referential processing (Moscovitch & ence (e.g. Curran & Hancock, 2007; Rugg & Curran, 2007). Winocur, 2002; Cabeza & St Jaques, 2007). Importantly, Although both groups principally employed both, familiarity the latter is as well increased under VR conditions due to and recollection as non-exclusive retrieval mechanisms (see its immersive character: VR facilitates an increased sense Jones and Jacoby, 2001), one mechanism predominated over of presence, i.e. the subjective feeling of being within a vir- the other as a function of the encoding context. Accordingly, tual environment (VE; e.g. Slater & Wilbur, 1997; Schubert encoding in VR resulted in a more precise and vivid retrieval et al., 2001; Nilsson, Nordahl, & Serafin, 2016). Whereas than encoding the same scenario in a PC setup (Kisker et al., immersion predominantly determines the degree to which 2019b). the user is isolated from his physical surroundings by techni- Overall, these studies suggest that VR experiences are cal factors, like 3D-360° view and proprioceptive matching, not just observed, i.e. passively watching stimuli presented presence promotes the subjective feeling of actually being in on a screen, but experienced in a self-relevant manner. Even and acting within the VE (Slater & Wilbur, 1997; Nilsson, interactive PC setups designed as immersive as possible by et al., 2016). Consequently, the sensation of acting within means of active exploration of a desktop-based environ- the VE comes in with the impression of being subject to the ment, generate overall rather superficial engrams compared consequences of these actions and events in the VE (Slater to exactly the same VE explored as a VR experience (Kisker & Wilbur, 1997; Nilsson, et al., 2016). For example, par- et al., 2019b). Unlike conventional laboratory experiences, ticipant behave as if being in real danger when exposed to the latter become part of a personal experience like real-life dangerous situations in an immersive VE, even though their experiences would (Schöne et al., 2016, 2019). surroundings could not physically harm them (e.g. Kisker However, while the electrophysiological correlates of, et al., 2019a; Krijn et al., 2004; Gromer et al., 2019). In line, for example, the sense of presence (e.g. Bouchard et al., VR setups have been found to elicit the same emotional and 2009) and spatial memory (e.g. Rauchs et al., 2008) are physical reactions as compared to their real-life equivalents recently more widely investigated, findings regarding the (Gorini et al., 2010; Higuera-Trujillo et al., 2017). Given this electrophysiological correlates of retrieval of episodic and impression of mutual interaction with the virtual surround- autobiographical engrams encoded within VR are still rare ings, VR experiences are more personally and emotionally (cf. e.g. Smith, 2019; Serino & Repetto, 2018; Plancher & relevant than mere on-screen experiences (see Kisker et al., Polino, 2017; Bohil et al., 2011). Accordingly, it is the aim 1 3 Psychological Research (2021) 85:2485–2501 2487 of our study to differentiate the electrophysiological corre- 2008; Klimesch et al., 1997a, 2001a, b). In line, the alpha- lates of the retrieval of VR experiences as opposed to con- band response should significantly decrease for new pictures ventional laboratory experiences. Specifically, we examined as compared to old pictures. Concerning the VR condition, a well-established electrophysiological marker of recogni- different outcomes might be possible: Under the premise tion memory tasks by means of the theta old/new effect that the theta old/new effect is exclusively linked to suc- obtained from laboratory settings (for review see Nyhus & cessful memory retrieval, theta-band synchronization for old Curran 2010; Guderian & Düzel, 2005; Hsieh & Ranganath, stimuli should be higher for the VR condition as compared 2014; see also Gruber, Tsivilis, Giabbiconi & Müller, 2008; to the PC condition, as most studies indicate that VR set- Klimesch et al., 1997a, 2001a). Therefore, we examined ups enhance memory performance (e.g. Schöne et al. 2016, theta-oscillations (~ 4-8 Hz; e.g. Nyhus & Curran, 2010), 2019; Smith, 2019) and activate recollection-based engrams which are most prominent at sensors over frontal-midline (Kisker et al., 2019b). For the alpha-band, a similar pattern regions (e.g. Hsieh & Raganath, 2014). There is broad and of results might be expected. However, as theta-band oscil- stable consensus, that a characteristic theta-band synchro- lations are related to further memory-related processes, e.g. nization can be observed in these regions in response to the memory load (Nyhus & Curran, 2010; Jensen & Tesche, retrieval of old stimuli, which are correctly remembered, i.e. 2002), decision making (Nyhus & Curran, 2010) and work- in response to retrieval success. In contrast, new stimuli are ing memory (Hsieh & Ranganath, 2014), another outcome associated with theta-band desynchronization (e.g. Nyhus than the classical effect might be equally likely in the VR & Curran, 2010). This effect was observed both subsequent condition. to the stimulus presentation (e.g. Klimesch et al., 1997b; Klimesch et al., 2001a) and after a physical response of par- ticipants, e.g. key pressure (Gruber, Tsivilis, Giabbiconi, & Methods Müller, Gruber et al., 2008). Moreover, theta-oscillations are associated with recollection of personal events (Gude- Participants rian & Düzel, 2005) and hippocampal projections to neo- cortical frontal regions are regarded as possible generators 45 participants were recruited from Osnabrück University. of these oscillations during memory tasks (e.g. Hsieh & The sample size was determined on the basis of previous Ranganath, 2014). In conjunction with the characteristic studies with a similar study design (cf. Schöne et al., 2019; frontal-midline theta-band synchronization, a decrease of Kisker et al., 2019a). All participants were screened for psy- the alpha-band response (~ 8–13 Hz, e.g. Berger, 1929) can chological and neurological disorders and had normal or regularly be observed during memory recall (e.g. Klimesch, corrected-to-normal sight. Three participants were excluded et al., 1997b; Sauseng et al., 2009; Jacobs, Hwang, Curran during the anamnesis. When vision correction was neces- & Kahana, 2006). This decrease of alpha-band response is sary, only those participants who had contact lenses could regarded a reflection of visual processing (Clayton, Yeung participate, not those who wore glasses. It was ensured that & Cohen Kadosh, 2018), attentional processes (Klimesch the participants saw sharply on the screen as well as on the et al., 1997a) and memory load (Sauseng et al., 2009; Jacobs head-mounted display. Previous experience with VR envi- et al., 2006; Jensen & Tesche, 2002; Dan & Reiner, 2017). ronments was documented. All participants gave informed In short, the theta-band synchronizes in response to mental consent and were blind to the research question. The par- activity, whilst the alpha-band desynchronizes (Berger, 1929 ticipants received either partial course credits or 15€ for as cited in Klimesch et al., Klimesch, Doppelmayr, Schimke, participation. et al. 1997b). The participants were randomly assigned to both condi- To examine whether this well-established and robust tions (VR vs. PC). Three participants were excluded from effect occurs under VR conditions as well, we set up an analysis due to insufficient data quality (n = 2) and prior experiment in which participants incidentally encoded either knowledge of the stimulus material used for the unan- immersive 3D-360° videos or conventional 2D videos fol- nounced recognition memory test (n = 1). After exclusion, lowed by an unannounced recognition memory test. We we obtained 39 complete datasets for analysis (VR group: assume that the VR condition will result in a higher sense of n = 20, M=21.95, SD = 3.19, 15 female, 19 right- VR age age presence, better memory performance and higher accuracy handed; PC group: n = 19, M=22.16, SD = 2.32, 13 PC age age of memory judgements as compared to the conventional PC female, 18 right-handed). condition. Moreover, we hypothesize to replicate the theta old/new effect for the conventional PC condition, manifested significant difference between theta-band responses to old and new stimuli, including a synchronization for old, and a desynchronization for new stimuli (see e.g. Gruber et al., 1 3 2488 Psychological Research (2021) 85:2485–2501 constant at 80 cm. The videos were presented in 2D videos Encoding in full-screen resolution. Sound was presented over standard speakers placed on both sides of the monitor. Stimulus material For both conditions, the videos were presented in ran- domized sequences with the GoPro VR Player, providing One hundred 3D-360° videos from the Library for Universal Virtual Reality Experiments (luVRe, Schöne, Kisker, Syl- the same video resolution for both conditions (cf. stimuli). Each randomized sequence was presented to one partici- vester, Radtke & Gruber, 2020;  https ://www.psych o.uni- osnab r uec k .de/f ac hg ebie t e/allg e meine _psy c h ologi e_i/ pant per condition. Each video was preceded by one-sec- ond fixation on a fixation cross. To facilitate incidental luvre.html ) were used as stimulus material. All videos were recorded with the Insta360Pro VR-camera with a frame rate encoding, the presentation of each video clip was followed by a rating (10 s) as a distraction task (cf. Fig. 1). Partici- of 60 fps and 4 k resolution. Each video was 10 s long. The videos were randomly subdivided into targets and distrac- pants were instructed to separately rate the experienced valence, arousal and motivation, i.e. their desire to stay in tors for the unannounced recognition memory test in a 50:50 ratio. The themes of the videos were balanced between target or leave the presented scene for each video separately on a scale from one (bad/not at all) to six (good/very much; and distractor videos (e.g. nature footage, interiors, medi- cal facilities, sport events, social events; see supplementary cf. Kuhr et al., 2015). The ratings were consecutively pre- sented on the (virtual) screen for 3.33 s each. The partici- material for a detailed description of the video content). Only the target videos were presented during incidental pants were familiarized with the rating before the video presentation. To guarantee for similar visual experience learning. Distractor videos were unknown to the participants and only used for the unannounced recognition memory test. during the rating and maintain immersion, the rating took place in an exact virtual simulation of the laboratory in Procedure which the study actually took place, implemented as a 3D-360° video recording of the laboratory. In addition, the Participants were randomly assigned to the VR- or the PC- rating scales were displayed on the (virtual) monitor dur- ing rating phase. For the PC group, the simulation of the condition. For the VR-condition, participants were equipped with a wireless version of the HTC Vive Pro head-mounted laboratory was displayed as a 2D video as well. The par- ticipant’s answers were recorded with a dictation device. display. Video footage and sound were presented in 3D-360°, with a resolution of 1080 × 1200 pixels per display. Partici- The ratings regarding valence, arousal and motivation of the videos were collected for the validation of a database pants were allowed to look around, but not to turn 180° or walk around. and will not be further analyzed in this study. The pres- entation of the videos took a total of 19 min. To enhance For the PC-condition, participants were seated in front of a curved monitor (35″, 90 cm screen diagonal, 37 cm immersion, all test leaders left the lab until the end of the video presentation. The participant was given a bell to height). The participant’s distance to the screen was kept Fig. 1 Procedure of inciden- tal encoding. Each of the 50 target videos was preceded by a fixation on a virtual screen and followed by the rating of valence, arousal and motiva- tion. Each scale was faded in on the virtual screen separately for 3.33 s. During fixation and rating, a 3D-360° image of the laboratory in which the partici- pants were actually located was presented 1 3 Psychological Research (2021) 85:2485–2501 2489 alert the test leaders if they wanted to quit the experiment started with randomly 0.5–0.8 s fixation, followed by 1.5 s early or felt uncomfortable. presentation of the stimulus. The rating scale was then dis- To determine the sense of presence, participants were played until the participants responded via key pressure. The asked to fill in the German version of the Igroup Presence interstimulus interval lasted randomly between 1.0 s and Questionnaire (IPQ; Schubert, Friedmann & Regenbrecht, 1.5 s (see Fig. 2). The response options were defined during 2001) and were asked for their experience of physical symp- instruction as follows (translated from German): toms (vertigo, nausea). In addition, the participants were instructed not to discuss the videos with the test leaders until (1) Definitely unknown: I’m sure I’ve not seen this place the end of the experiment. (2) Rather unknown: I guess I haven’t seen this place (3) Familiar: This place looks familiar to me Unannounced recognition memory test (4) Vividly remembered: I remember this place precisely and vividly. Stimulus material Electrophysiological recordings and preprocessing Monoscopic screenshots from both, distractors (referred to as new pictures) and targets (referred to as old pictures), An electroencephalogram (EEG) with 128 electrodes, were used as stimulus material for the unannounced recogni- attached in accordance with the international 10-20-system tion memory test. Per video, one representative screenshot was recorded for the duration of the unannounced recogni- was utilized as stimulus, resulting in 100 trials. The stimuli tion memory test. The Active-Two amplifier system from were presented on a conventional 24″ monitor with a para- BioSemi (Amsterdam, Netherlands) was used. The sampling foveal visual angle of 2 × 5°. rate was 1024 Hz, the bandwidth (3 dB) 104 Hz. Addition- ally, horizontal electrooculogram (hEOG) and vertical elec- Procedure trooculogram (vEOG) were recorded and a common mode sense (CMS) and a driven right leg (DRL) electrode were The retention interval was set to 1 h during which the EEG applied. The EEG was recorded on the investigators’ com- was applied. If the participants mentioned the videos they puter using ActiView702 Lores. had seen during encoding, they were kindly interrupted and EEG data were analyzed using MATLAB. For further asked not to discuss the videos until the end of the experi- off-line analysis, the average reference was used. The ment. Participants were instructed about their task imme- EEG was segmented to obtain epochs starting 500  ms diately before the unannounced recognition memory test. prior and 1500  ms following stimulus onset (baseline The unannounced recognition test comprised of 100 tri- − 300 to − 100 ms). Artifact correction was performed als. Per trial, participants had to indicate as fast as possi- by means of ‘‘statistical correction of artifacts in dense ble whether they recognized the presented stimulus as (1) array studies’’ (SCADS; Junghöfer, Elbert, Tucker, & definitely unknown, (2) rather unknown, (3) familiar or (4) Rockstroh, 2000). In brief, this procedure uses a combi- vividly remembered (cf. Kisker et al., 2019b). Each trial nation of trial rejection and channel approximation based Fig. 2 Setup of the memory test trials: 0.5–0.8 s fixation, 1.5 s stimulus presentation, presentation of the scale until the participant’s response, 1.0–1.5 s inter stimulus interval (ISI). Participants were asked not to blink from fixation until the response scale appeared 1 3 2490 Psychological Research (2021) 85:2485–2501 on statistical parameters of the data. For each trial, con- Schwaiger, Winkler & Gruber, 2000; Klimesch al., 2001b; taminated electrodes are detected based on a threshold Jacobs et al., 2006). The alpha frequency band (8–13 Hz, criterion derived from the distribution of the amplitude, see e.g. Berger, 1929) was analyzed at electrodes sur- standard deviation, and gradient of the sensor across all rounding Oz, O1 and O2 in the time window from 0 to trials. The information of these electrodes is replaced 500 ms. with a spherical interpolation from the full channel set. The limit for the number of approximated channels was Statistical analysis set to 20. Epochs containing more than 20 channels with artifacts were rejected. Presence For demonstrating a robust signal at the frequency bands of interest, we first calculated a conventional fast The IPQ scales were determined as sum values of the respec- Fourier transform (FFT, see Fig. 5) per trial and averaged tive items (in total: 14 items; general presence: one item, across all electrodes, conditions and participants. spatial presence: five items, involvement: four items, real - For further analyses and a comparison between experi- ness: four items). Each item could reach values from − 3 and mental conditions, we considered it advantageous to take + 3 on a 7-step likert-scale, resulting in the following mini- the signal’s temporal evolution into account. Thus, for the mum and maximum sumscores per scale: General Presence subsequent examinations, spectral changes in oscillatory (− 3; 3), Spatial Presence (− 15; 15), Involvement (− 12; activity were analysed by means of Morlet wavelets with 12), Realness (− 12; 12). a width of 12 cycles per wavelet which is described in Shapiro–Wilk-test rejected normal distribution for one detail elsewhere (e.g., Tallon-Baudry & Bertrand, 1999; of the IPQ scales (General Presence, p < 0.05). Therefore, Bertrand & Pantev, 1994). In brief, the method provides the more robust Mann–Whitney U test as non-parametric a time-varying magnitude of the signal in each frequency equivalent of the unpaired t test was used for analysis. Cron- band, leading to a time-by-frequency (TF) representation bach’s α was calculated for each scale, with the exception of of the data. Due to the fact that induced oscillatory activ- the one-item-scale General Presence. ity occurs with a jitter in latency from one trial to another (Eckhorn et al., 1990), they tend to cancel out in the aver- aged evoked potential. Thus, TF amplitude is averaged Memory performance across single-trial frequency transformations, allowing one to analyze non-phase-locked components. Further- D′-prime (d′) was calculated separately for both groups as an operationalization of memory performance. D’ relates more, because we focused on the non-phase-locked com- ponents of the signal, the evoked response (i.e., the ERP) the hits, i.e. correct positive judgments, to the false-positive judgments (d’ = z(hit) −− z(false positive); Haatveit et al. was subtracted from each trial before frequency decompo- sition (for details, see Busch, Herrmann, Müller, Lenz, & 2010; Swets et al., 1961; as cited in Kisker et al., 2019b) and indicates how well participants are able to distinguish Gruber, 2006). Given our interest in the lower-frequency range, we used wavelets from 0.25 Hz to 30 Hz. between targets and distractors. D’-prime was calculated per group to assess the overall retrieval success (general Based upon prior literature (e.g. Nyhus & Curran, 2010) and our hypothesis, the frequency range from d’ = z(all hits) – z(all false positives)). Additionally, d’ was separately calculated for familiarity 4-7 Hz was included in the analyses and checked against visual inspection of the FFT (see Fig. 5). However, visual and recollection for each group, taking only the respective hits and false positives into account (cf. Kisker et al., 2019a: inspection of the FFT revealed high power for 2–4 Hz as well. This frequency range is commonly denoted as the d’-familiar ity score = z(familiarity hits) – z(familiarity false positives); d’-recollection score = z(recollection hits) delta-band, but was also identified as lower theta-band in some studies, indicating that the old-new effect might be – z(recollection false positives). Shapiro–Wilk-test rejected normal distribution for all d’ scores (all p < 0.05). Hence, ref lected in the 2–4 Hz frequency range as well (cf. Bur- gess & Gruzelier, 1997; Klimesch, Schimke & Schwaiger, Mann–Whitney U Test was used for analysis. Accuracy [(hits + correct rejection)/total number of trials] 1994; Klimesch et al., 2000). Hence, the 2–4 Hz response was included in the analyses as well. Electrodes around and error rate [(misses + false positives)/total number of tri- als] of recognition judgements were calculated per group. Fz covering for the frontal midline region were chosen. Based upon prior literature, an early latency range from Both were analyzed using the unpaired t test. 250 to 650 ms for the 2–4 Hz response (see Burgess & Gruzelier, 1997) and 200–600 ms for the 4–7 Hz band response were used for analyses (e.g. Guderian & Düzel, 2005; Klimesch et  al., 1997b; Klimesch, Doppelmayr, 1 3 Psychological Research (2021) 85:2485–2501 2491 Prior VR experience and cybersickness Dependent measures Prior experience with VR and cybersickness were assessed EEG data were analyzed using a 2 × 2 repeated-measure- as nominal variables (prior experience: “Have you already ments ANOVA (rmANOVA) with the between-factor “group” had any experience with virtual reality, e.g. studies, games (VR vs. PC) and the within-factor “condition” (new pictures or videos?”, [yes/no]; cybersickness: “Did you experience vs. old pictures). Significant effects of rmANOVA were com- physical symptoms such as nausea or dizziness during the plemented by post hoc t tests. experiment?”, [yes/no]; if yes: “How strongly did you feel nauseous/dizzy?” [1–10]; cf. Kisker et al., 2019a). Contin- gency tables and Pearson’s Chi square (X ) test were used Results for statistical analysis. Subjective measures Ratings of the videos Presence The ratings regarding valence, arousal and motivation of the videos were collected for the validation of a database and As hypothesized, the VR-group reported a higher feeling of will not be further analyzed in this study. To check that the presence during video presentation (see Fig. 3). This is valid target videos were perceived comparably emotive in both for all IPQ subscales (all p ≤ 0.005; see Table 1). Cronbach’s groups, arousal and valence averaged over all 50 target vid- α indicates acceptable reliability for all scales (all α ≥ 0.64). eos were compared between the groups using unpaired t test. Prior VR experience and cybersickness In both groups, about 70% of the participants had already gained experience with VR prior to the study, e.g. by partici- pating in other studies, watching VR videos or playing VR- games (X (1) = 0.011, p = 0.915). In total, nine subjects (n : VR six, n : three) reported experiencing physical symptoms PC like nausea and dizziness, but on a very mild level (nausea, in total: M = 2.55, SD = 2.13; VR: M = 3.33, SD = 2.25; VR VR PC: M = 1.0, SD = 0.0; dizziness, in total: M = 1.67, PC PC SD = 1.12; VR: M = 2.00, SD = 1.27; PC: M = 1.0, VR VR PC SD = 0.0), resulting in significantly stronger experiences PC of physical symptoms in the VR condition (X (1) = 4.91, p = 0.027). Ratings of the videos Fig. 3 Median scores of the IPQ scales General Presence, Spatial Participants of both groups reported equal levels of valence Presence, Involvement and Realness as evaluated by both groups. The and arousal averaged across all target videos (valence: error bars depict the standard error per scale. Minimum and maxi- mum sumscores per scale: General Presence (−  3; 3), Spatial Pres- M = 3.89, SD = 0.52, M = 3.61, SD = 0.47, VR VR PC PC ence(− 15; 15), Involvement (− 12; 12), Realness (− 12; 12) Table 1 Differences between VR- and PC-group regarding the sensa- and Cronbach’s α per scale. Cronbach’s α could not be calculated for tion of presence, assessed via the IPQ (Schubert et  al., 2001): Test the one-item-scale General Presence statistics of the one-tailed Mann–Whitney U test, descriptive values IPQ scale U z p Md Md Cronbach’s α VR PC General presence 41.00 − 4.29 < .001 2.00 − 2.00 Spatial PRESENCE 19.00 − 4.72 < .001 3.00 − 7.00 0.68 Involvement 98.00 − 2.59 0.005 1.50 − 4.00 0.70 Realness 64.50 − 3.55 < .001 -0.50 − 4.00 0.64 1 3 2492 Psychological Research (2021) 85:2485–2501 t(34) = 1.67, p = 0.103; arousal: M = 2.64, SD = 0.55, rates (t(37) = − 0.505, p = 0.31, M = 0.09, M = 0.08; see VR VR VR PC M = 2.65, SD = 0.47, t(34) = − 0.28, p = 0.978). Fig. 4), indicating a ceiling effect. PC PC Dependent measures Memory performance Since the behavioral data indicate no difference in memory Participants of both groups performed equally well on the performance between both groups, and since the high accu- unannounced recognition memory test, as none of the cal- racy indicates a ceiling effect, the latency range following culated d′ scores revealed significant differences (d’-general: stimulus onset was analyzed instead of the latency range U = 186.00, z = − 0.11, p = 0.462; d′-familiarity: U = 150.50, following the participants’ response (key pressure) to the z = − 1.11, p = 0.14; d’-recollection: U = 162.50, z = − 0.77, stimulus (cf. results, memory performance). p = 0.22; see Fig. 4). The visual inspection of the FFT validated the hypothe- Moreover, both groups achieved surprisingly high lev- sis-driven selection of the 4–7 Hz and 8–13 Hz frequency els of accuracy around 90% (t(37) = − 0.505, p = 0.308, ranges. In addition, the visual inspection also revealed a M = 0.91, M = 0.92) and correspondingly low error VR PC noticeable power of the 2–4 Hz frequency range, which is Fig. 4 Panel A depicts the accuracy as well as the respective error approximately 0.01 and therefore hardly visible in the figure. Panel rate of the judgement on the recognition or unknown character of the B depicts the retrieval success per group operationalized by general memory task trials in percent for both groups. The error bars depict d’ prime, as well as the d’-familiarity and d’-recollection scores. No the standard errors. For accuracy and error rate, the standard error is significant differences were found between both groups Fig. 5 Power spectra from fast Fourier transform (FFT) per group and condition. Visual inspection revealed a strong frequency peak from 2 to 4 Hz, which was hence included in the analyses 1 3 Psychological Research (2021) 85:2485–2501 2493 why it was also included in the analyses (see Fig.  5, see (t(37) = 2.06, p = 0.046; see Figs. 6 and 7). The theta-band methods). response to new pictures (t(37) = − 0.14, p = 0.889) and to old pictures (t(37) = 1.65, p = 0.107) did not differ between 4–7 Hz responses both groups (see Figs. 6, 7 and 8). Regarding frontal-midline theta-band responses, no sig- 2–4 Hz responses nificant main effects could be found (F (1,37) = 2.84, condition p = 0.10; F (1,37) = 0.38, p = 0.543), but a signifi- For the 2–4 Hz response, a significant main effect for the group cant interaction of the factors “group” and “condition” factor “condition” (F (1,37) = 11.61, p = 0.002), but condition (F (1,37) = 5.03, p = 0.046). not for the factor “group” (F (1,37) = 1.44, p = 0.239) interaction group Post-hoc t tests revealed a classical old/new effect in could be found. The main effect of “condition” was further the PC condition with a higher amplitude for old pictures characterized by a significant interaction of both factors than for new ones (t(18) = − 2.86, p = 0.010). However, (F (1,37) = 4.11, p = 0.049). Following the same interaction this difference effect was absent within the VR condi- trend as the 4-7 Hz responses, post hoc t tests revealed tion (t(19) = 0.25, p = 0.805). Furthermore, the observed a classical old/new effect across conditions (t (38) = 3.23, difference effect was comparably larger in the PC-group p = 0.003), as well as in the PC condition (t(18) = − 4.74, p < 0.001), but not in the VR condition (t(19) = − 0.85 p = 0.404). Again, the observed difference effect was com- parably larger in the PC-group (t(37) = 2.03, p = 0.049). But most importantly, old pictures elicited greater responses in the PC group compared to the VR-group (t(37) = 2.07, p = 0.046), whereas responses to new pic- tures did not differ between both groups (t (37) = − 0.05, p = 0.96; see Figs. 9, 10 and 11). Alpha‑band responses Regarding the alpha-band responses (8–13  Hz) at occipital electrodes, a main effect of the factors group (F (1,37) = 4.26, p = 0.046) and condition g roup Fig. 6 Mean amplitude in µV regarding the 4–7  Hz response in the (F (1,37) = 13.80, p < 0.001), but no significant inter - condition latency range from 200 to 600 ms after stimulus onset. The error bars action of both factors was found (F (1,37) = 1.21, depict the standard error of the mean amplitude. Significant differ - interaction ences are marked (*p < 0.05) p = 0.278). Fig. 7 Time-by-amplitude plot of the 4–7 Hz response from 200 ms before stimulus onset to 1200 ms after stimulus onset. While the classical old/ new-effect is also descriptively shown in the PC condition, there are no significant differ - ences between old and new pictures regarding the VR- group. The gray highlighted section  marks the latency range of significant interaction. The amplitude was averaged across the electrodes around Fz, covering for the frontal midline region 1 3 2494 Psychological Research (2021) 85:2485–2501 Fig. 8 Topography of the amplitude regarding the 4–7 Hz response separately for all com- binations of the factors group (VR vs. PC) and conditions (old vs. new) in the latency range from 200 to 600 ms after stimulus onset. Additionally, a difference plot of the old/new- effect is depicted. Black dots mark the electrodes which were included in the analyses More specifically, new pictures elicited lower alpha from the luVRe database (see methods), or watched the exact amplitudes as compared to old pictures (t (38) = 3.68, same stimulus material on a conventional 2D monitor (PC p < 0.001). In line, alpha amplitudes were significantly condition). In an unannounced recognition test, we com- lower for the PC group as compared to the VR group pared their memory performance, the mid-frontal theta old/ (t(76) = 2.75, p = 0.008; see Figs. 12, 13 and 14). new effect indexing mnemonic processing, as well as poste- rior alpha as a marker for visual processing load. As a result, both groups performed equally well in the recognition test, Discussion although the theta old/new effect could only be replicated for the PC condition and was absent in the VR condition. Addi- The aim of the study was to investigate the electrophysiolog- tionally, the theta effect was accompanied by a profound ical correlates of the retrieval of VR experiences as opposed reduction of posterior alpha in the PC condition, indicating to conventional laboratory experiences. To this end, par- a visually guided, effortful retrieval process. ticipants watched either 3D-360° VR videos (VR condition) Meeting our expectations, participants of the VR condi- tion felt more present during video presentation as compared to the PC condition, confirming that our video approach led to immersive VR experiences. Presence, as the most promi- nent feature of VR experiences (e.g. Schubert et al., 2001, Pan & Hamilton, 2018; Diemer et al., 2015; Alshaer, Regen- brecht, & O’Hare, 2017; Riva et al., 2007; Kisker et al., 2019a), is associated with increased emotional involvement (e.g. Gorini et al., 2010; Felnhofer et al., 2015), and stronger and more realistic behavioral responses as compared to con- ventional laboratory settings (Slobounov et al., 2015; Kisker et al., 2019a). Importantly, previous studies found that a high degree of presence aids memory recall: For example, both intentional encoding, as well as incidental encoding in a VE resulted in a more accurate memory recall as compared to conventional desktop conditions (e.g. Krokos, Plaisant & Fig. 9 Mean amplitude in µV regarding the 2–4  Hz response in the Varshney, 2019; Ernstsen, Mallam & Nazir, 2019). Hence, latency range from 250 to 650 ms after stimulus onset. The error bars presence might facilitate encoding processes constituting depict the standard error of the mean amplitude. Significant differ - the VR memory superiority effect (Makowski, Sperduti, ences are marked (*p < 0.05; ** p < 0.01) 1 3 Psychological Research (2021) 85:2485–2501 2495 Fig. 10 Time-by-amplitude plot of the 2–4 Hz response from 200 ms before stimulus onset to 1200 ms after stimulus onset. The gray highlighted section  marks the latency range of sig- nificant interaction. The ampli- tude was averaged across the electrodes around Fz, covering for the frontal midline region Fig. 11 Topography of the amplitude regarding the 2–4 Hz response separately for all com- binations of the factors group (VR vs. PC) and conditions (old vs. new) in the latency range from 250 to 650 ms after stimulus onset. Additionally, a difference plot of the old/new- effect is depicted. Black dots mark the electrodes which were included in the analyses Nicolas & Piolino, 2017; Serino & Repetto, 2018; Smith, 2019). In particular, visually detailed environments that pro- vide high realism and resemblance to the real world, such as 3D-360° videos (Pan & Hamilton, 2018; Lovett et al., 2015), facilitate more accurate judgments in old/new tasks (Smith, 2019). The resulting coherent egocentric perspective facili- tates recollection and reliving of such content (see Rubin & Umanath, 2015), which is crucial to form vivid, real- Fig. 12 Mean alpha amplitude (8-13  Hz) in µV in the latency range life memories (Conway, 2005; Roediger & Marsh, 2003). from 0 to 500 ms after stimulus onset. The error bars depict the stand- Hence, a high sense of presence—including sensations of ard error of the mean amplitude. Significant differences are marked spatial presence, involvement and realness—means that (*p < 0.05; **p < 0.01) 1 3 2496 Psychological Research (2021) 85:2485–2501 Fig. 13 Time-by-amplitude plot of the alpha-band response (8–13 Hz) from 200 ms before stimulus onset to 1200 ms after stimulus onset. Descriptively, a stronger reduction of the alpha amplitude was observed for both PC conditions compared to both VR conditions. The gray highlighted section  marks the latency range of both significant main effects Fig. 14 Topography plot of alpha amplitude in µV (8–13 Hz) separately for fac- tors group (VR vs. PC) and conditions (old vs. new) in the latency range from 0 to 500 ms after stimulus onset. Addition- ally, a difference plot of old minus new and PC minus VR is depicted. Black dots mark the electrodes which were included in the averaged amplitude these events are potentially significant for the participant, the VR group reported higher sensations of presence as com- consciously experienced and thus, might contribute to the pared to the PC group, we did not observe superior memory formation of autobiographical memory. recall performance. Our results, with both groups having However, at odds with previous research (Schöne et al., an accuracy of ca. 90%, indicate a ceiling effect, limiting 2019; Smith, 2019; Kisker et al., 2019b), our study did not the detection of group differences (Bortz & Döring, 2005). provide any behavioral evidence for this effect: Even though A possible cause of is effect might be the short retention 1 3 Psychological Research (2021) 85:2485–2501 2497 interval between encoding and retrieval. Previous stud- higher theta-band amplitudes to the retrieval of old, and ies, which did not apply EEG measurements, chose longer relatively lower amplitudes to the retrieval of new pictures retention intervals that included one or two sleeping peri- in conventional laboratory settings (e.g. Gruber et al., 2008; ods (Schöne et al., 2019; Kisker et al., 2019b). It is pos- Klimesch et al., 1997a, b, 2001a, b). The change of modal- sible that the process of forgetting irrelevant information ity, i.e. encoding videos, but retrieval in response to picture had not yet started at the time of the EEG measurement or presentation, did not markedly affect the theta old/new effect had at least not progressed very far (cf. Wang, Subagdja, in the PC condition. Tan & Starzyk, 2012). However, other studies have not been Remarkably, the theta old/new could not be observed able to demonstrate this overall memory superiority of VR in the VR condition. Specifically, new pictures led to the experiences either (LaFortune & Macuga, 2018; Dehn et al., same theta-band response in both groups, indicating that 2018; Kisker et al., 2019b). Differences regarding the find- the physical discrepancies between encoding in VR or under ings of VR studies might be related to varying implemen- conventional conditions did not affect the paradigm per se tations of VR technology, ranging from highly immersive or at least affected it to the same extent. Moreover, memory head-mounted displays and CAVE systems to less immersive success did not account for the different electrophysiological desktop-VR implementations (Smith, 2019). Additionally, responses as well, as both groups performed equally well in the level of multi-sensory sensations provided by the VR the recognition test. Accordingly, differences in the electro- system might influence memory performance as well: For physiological response must result from different underly - example, active navigation through a VR environment can ing retrieval mechanisms and thus, differences in mnemonic have an additional positive effect on spatial memory, but not processing of engrams encoded from either VR experiences necessarily on factual memory (Plancher, Barra, Orriols & or conventional laboratory events. Evidence that the absence Piolino, 2013). Moreover, some studies report a successful of the theta old/new ee ff ct under VR conditions results from transfer of content learned in an immersive VR environ- an altered mnemonic processing style as compared to the PC ment to real-life, and thus, to other than the encoding context condition is obtained from the comparison of the response (Ragan, Sowndararajan, Kopek & Bowman, 2010; as cited to old pictures between both groups. Regarding the 2–4 Hz in Smith, 2019), whereas other studies claim that knowledge frequency range, the presentation of old pictures led to a transfer comes with a loss of performance (Lanen & Lam- significant difference between relative synchronization in the ers, 2018). PC group and in the VR group. Descriptively, the 4–7 Hz Even though VR experiences do not necessarily increase frequency range follows the same trend but did not reach the retrieval success as measured by subjective reports, the significance. Hence, the theta old/new effect is modulated immersive nature of VR yet might alter the mode of opera- by the nature of the engram resulting from VR experiences tion of the mnemonic mechanisms. Specifically, Kisker and how these experiences are recalled. et  al., (2019a) demonstrated by means of a remember/ As aforementioned, immersive VR experiences are con- know paradigm that participants who explored a virtual vil- sidered to facilitate the formation of autobiographical mem- lage in an immersive VR condition report predominantly ory. Associative autobiographical engrams are generated by recollection-based memory. Interestingly, recollection is highly self-relevant experiences (Roediger & Marsh, 2003; hypothesized to be the associated retrieval mechanism of Conway, 2005). They are characterized by richer content autobiographical memory (Roediger & Marsh, 2003; Con- and are deeply interwoven into existing memory structures way, 2005). Participants exploring the very same village in (McDermott et al., 2009; Roediger & Marsh, 2003). Further- a PC condition reported predominantly familiarity-based more, they come with a broad set of functional properties, memories (Kisker et al., 2019a). However, both groups in namely self-reflection, emotional evaluation and semantic our experiment apparently employed the same retrieval strat- processes (Svoboda et al. 2006). Frontal-midline theta has egies as the d′-scores for recollection, familiarity and overall repeatedly been shown to reflect key-elements of autobio- performance do not differ significantly. graphical mnemonic processing. Specifically, it is associ - Nevertheless, modulations of the frontal-midline theta ated with the recollection of personal events and contextual effect might still indicate the involvement of different information (Guderian & Düzel, 2005; Hsieh & Ranganath, types of memory systems as well as associated encoding 2014; see also Roediger & Marsh, 2003; Conway, 2005). and retrieval strategies with respect to the encoding condi- In line with previous studies, our results indicate that the tion. As expected, we replicated the frontal-midline theta retrieval of immersive 3D-360° experiences differs from the old/new effect in the PC condition: Old pictures evoked retrieval of conventional 2D laboratory events (Schöne et al. an early theta-band synchronization, whereas new pictures 2016; Schöne et al. 2019; Kisker et al., 2019b). Hence, the resulted in theta-band desynchronization. Hence, our find- well-established theta old/new effect does not seem to be ings replicate broad and stable evidence relating relatively unrestrictedly applicable to VR experiences. It might rather 1 3 2498 Psychological Research (2021) 85:2485–2501 serve as an index for cue-matching of previously exog- has to be suppressed, while mental representation and cue enously processed pictorial stimuli: Experiences encoded are matched (Norman & Bobrow, 1979; Conway, 1996; Bur- in the laboratory are recalled and visually matched to the gess & Shallice, 1996). test stimuli, but are not inevitably associated with the vivid This interpretation of a visually guided matching process and multimodal character of autobiographical memories and gains further support from the difference in posterior alpha thus, might not provide a holistic representation of real-life oscillations, associated with visual processing (e.g. Clay- mnemonic processing. ton et al., 2018). Matching mental representation and cue The question remains, which processes change their mode is reflected by a generally reduced posterior alpha ampli - of operation in response to the recall of VR experiences. tude in the PC condition compared to the VR condition. The theta old/new effect is predominantly associated with This reduced alpha amplitude, commonly regarded as corti- retrieval success (e.g. Nyhus & Curran, 2010). However, the cal activity (e.g. Berger, 1929 as cited in Klimesch et al., VR and the PC group were likewise successful in the rec- 1997b), on the one hand reflects elevated attention (e.g. ognition task. As above mentioned, frontal-midline theta is Klimesch, et al. 1997a; Fries, Womelsdorf, Oostenveld & associated with autobiographical mnemonic processing, but Desimone, 2008) and, on the other hand, successful suppres- also regarded as an index for top-down control of memory sion of irrelevant information (Sauseng et al., 2009; Jensen retrieval (Klimesch et al., 1997b; Nyhus & Curran, 2010). & Mazaheri, 2010). Especially, the co-occurrence of higher Specifically, early theta-band increases indicate an attempt or frontal theta responses and posterior alpha activity has been the effort demands to retrieve engrams rather than success- interpreted as a response to higher cognitive load, with 2D ful retrieval per se (Klimesch et al., 2001a; Nyhus & Cur- environments exhibiting higher cognitive load as compared ran, 2010). Several studies investigating memory retrieval in to 3D environments (Dan & Reiner, 2017). Theta and alpha general as well as the classical old/new effect in particular, oscillations thus provide evidence for effortless and direct explicitly differentiate retrieval effort and retrieval success retrieval of immersive VR experience and a, in comparison, (Klimesch et al., 2001a; Nyhus & Curran, 2010; Rugg et al., effortful and strategic retrieval of conventionally presented 1998; Konishi, Wheeler, Donaldson & Buckner, 2000). In stimuli. particular, processes exclusively associated with retrieval Nevertheless, the finding that the retrieval mechanisms success are engaged only if an attempted retrieval is suc- underlying VR experiences and conventional laboratory cessful. In contrast, retrieval effort refers to those processes experiences differ, does not invalidate previous well-estab- engaged during a retrieval attempt per se, for example in lished knowledge gained from conventional setups. Rather, recognition tasks, regardless of whether this attempt is suc- it complements the immense insights from previous studies cessful or not (Rugg, Fletcher, Frith, Frackowiak & Dolan, and demonstrates the delicate balance between high experi- 1996). Accordingly, the absence of a difference in memory mental control and ecological validity. Thus, controlled success does not rule out that the effort required to achieve laboratory studies provide the foundations for understand- the very same retrieval outcome may vary. ing the complex mechanisms of human memory and are Hence, the difference in the theta-band response to old substantial for developing models. As a further refinement pictures between the VR condition and the PC condition of these foundations, VR settings facilitate the transfer of could reflect the two types of retrieval differing with respect experimental findings to everyday life and thus improve their to their effort demands (Conway, 1996; Haque & Conway, generalizability and practicability. 2001; Conway & Pleydell-Pearce, 2000). Immersive VR experiences as part of an extensive autobiographical asso- ciative network (PBM, Schöne et al., 2019) can be effortless Conclusions and, most of all, directly retrieved. In contrast, the retrieval of conventional stimuli triggers the iterative verification pro- As a conclusion, we replicated the well-established theta cess and the suppression of irrelevant information, thus com- old/new effect in a conventional laboratory setting, mani- ing in with higher effort to recall memories. Direct retrieval fested in relative theta-band synchronization for old, and of autobiographical memory is based upon a pronounced and relative desynchronization for new stimuli. However, this stable memory pattern (Conway & Pleydell-Pearce, 2000) effect could not be replicated for the immersive VR con- and enables spontaneous recall, which is rather automatic dition: Theta-band responses were equal for old and new and effortless (Conway & Pleydell-Pearce, 2000 as cited in stimuli. Hence, the canonical theta old/new effect might not Willander & Larsson, 2007). It thus allows immediate recall be unrestrictedly applicable to VR experiences and thus, of a cued memory. Generative or strategic retrieval of con- might not provide a holistic representation of real-life pro- ventional stimuli, as observed in the PC condition, relies cesses. Accompanied by higher alpha activity as compared on central control of memory recall (Willander & Larsson, to the VR condition, the theta-band synchronization in the 2007). To verify the cued memory, irrelevant information PC condition might rather reflect higher retrieval effort than 1 3 Psychological Research (2021) 85:2485–2501 2499 otherwise in a credit line to the material. If material is not included in retrieval success per se. In contrast to laboratory events, the article’s Creative Commons licence and your intended use is not memories obtained from VR experiences are spontaneous permitted by statutory regulation or exceeds the permitted use, you will and effortless retrieved. Additionally, participants of the VR need to obtain permission directly from the copyright holder. To view a condition reported a higher sense of presence, which might copy of this licence, visit http://creativ ecommons .or g/licenses/b y/4.0/. enhance the self-relevance of the VR experiences. Crucially, self-referential processing and a facile, effortless recall are characteristic of autobiographical memory. Therefore, the References effortless recall of VR experiences might approximate real-life memory more closely as compared to memories Alshaer, A., Regenbrecht, H., & O’Hare, D. (2017). Immersion fac- obtained from the laboratory. However, the VR group did tors affecting perception and behaviour in a virtual reality power not perform better in the memory test, as former research wheelchair simulator. Applied Ergonomics, 58, 1–12. https ://doi. org/10.1016/j.aperg o.2016.05.003. suggested. Hence, our results suggest that the memory pro- Atkinson, R. C., & Juola, J. F. (1973). Factors influencing speed and cesses underlying VR experiences are qualitatively different accuracy of word recognition. Attention and Performance, IV, from conventional laboratory experiences, but under which 583–612. conditions VR leads not only to altered mechanisms but also Berger, H. (1929). Über das Elektroenkephalogramm des Menschen. Archiv für Psychiatrie und Nervenkrankheiten, 87(1), 527–570. to a better memory performance compared to conventional Bertrand, O., & Pantev, C. (1994). Stimulus frequency dependence of settings should be the subject of further research. the transient oscillatory auditory evoked responses (40 Hz) stud- ied by electric and magnetic recordings in human. In: Oscillatory event-related brain dynamics (pp. 231–242). Boston: Springer. Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neu- Author contributions All authors contributed to the study design. Test- roscience research and therapy. Nature reviews neuroscience, ing and data collection were performed by JK. Data analyses were 12(12), 752–762. performed by JK under the supervision of BS and TG. JK and BS per- Bortz & Döring (2005). Forschungsmethoden und Evaluation (S. 182). formed the data interpretation under the supervision of TG. JK drafted Heidelberg: Springer-Verlag. ISBN 3-540-41940-3. the manuscript, and BS and TG provided critical revisions. All authors Bouchard, S., Talbot, J., Ledoux, A. A., Phillips, J., Cantamasse, M., approved the final version of the manuscript for submission. & Robillard, G. (2009). The meaning of being there is related to a specific activation in the brain located in the parahypocampus. Funding Open Access funding provided by Projekt DEAL. This In: Proceedings of Presence. research did not receive any specific grant from funding agencies in Burgess, A. P., & Gruzelier, J. H. (1997). Short duration synchroniza- the public, commercial, or not-for-profit sectors. tion of human theta rhythm during recognition memory. NeuroRe- port, 8(4), 1039–1042. Availability of data, material and code The datasets generated and Burgess, N., Maguire, E. A., Spiers, H. J., & O’Keefe, J. (2001). A analyzed during the current study are available in the Open Science temporoparietal and prefrontal network for retrieving the spatial Framework (OSF) repository, https://osf.io/q924w /?vie w_only=cc107 context of lifelike events. Neuroimage, 14(2), 439–453. f7a927f472 e8e68 e85aa a059e 97 . Stimulus material was obtained from Burgess, P. W., & Shallice, T. (1996). Response suppression, initiation the Library for Universal Virtual Reality Experiments (luVRe, https and strategy use following frontal lobe lesions. Neuropsychologia, ://www.psycho.uni-osnab r ueck.de/f achgebie te/allg emeine _psy chologi 34(4), 263–272. e_i/luvre .html) and can be accessed upon request. Busch, N. A., Herrmann, C. S., Müller, M. M., Lenz, D., & Gruber, T. (2006). A cross-laboratory study of event-related gamma activity in a standard object recognition paradigm. Neuroimage, 33(4), Compliance with ethical standards 1169–1177. Cabeza, R., Prince, S. E., Daselaar, S. M., Greenberg, D. L., Budde, M., Conflict of interest All authors declare that they have no conflict of Dolcos, F., & Rubin, D. C. (2004). Brain activity during episodic interest. retrieval of autobiographical and laboratory events: an fMRI study using a novel photo paradigm. Journal of Cognitive Neuroscience, Statement of ethical approval All procedures performed in this study 16(9), 1583–1594. involving human participants were in accordance with the ethical stand- Cabeza, R., & St Jacques, P. (2007). Functional neuroimaging of ards of the local ethic committee of Osnabrueck University and with autobiographical memory. Trends in Cognitive Sciences, 11(5), the 1964 Helsinki Declaration. 219–227. Clayton, M. S., Yeung, N., & Cohen Kadosh, R. (2018). The many Consent to participate and to publish All individual participants gave characters of visual alpha oscillations. European Journal of Neu- informed written consent to participate in the study and to anonymized roscience, 48(7), 2498–2508. publication of the datasets. Conway, M. A. (1996). Autobiographical knowledge and autobiograph- ical memories. In D. C. Rubin (Ed.), Remembering our past: Stud- ies in autobiographical memory (p. 67–93). Cambridge University Open Access This article is licensed under a Creative Commons Attri- Press. https ://doi.org/10.1017/CBO97 80511 52791 3.003. bution 4.0 International License, which permits use, sharing, adapta- Conway, M. A. (2005). Memory and the self. Journal of Memory and tion, distribution and reproduction in any medium or format, as long Language, 53(4), 594–628. as you give appropriate credit to the original author(s) and the source, Conway, M. A., & Pleydell-Pearce, C. W. (2000). The construction of provide a link to the Creative Commons licence, and indicate if changes autobiographical memories in the self-memory system. Psycho- were made. The images or other third party material in this article are logical Review, 107(2), 261. included in the article’s Creative Commons licence, unless indicated 1 3 2500 Psychological Research (2021) 85:2485–2501 Curran, T., & Hancock, J. (2007). The FN400 indexes familiarity-based Hsieh, L.-T., & Ranganath, C. (2014). Frontal midline theta oscilla- recognition of faces. Neuroimage, 36(2), 464–471. tions during working memory maintenance and episodic encoding Dan, A., & Reiner, M. (2017). EEG-based cognitive load of processing and retrieval. Neuroimage. https ://doi.org/10.1016/j.neuro image events in 3D virtual worlds is lower than processing events in 2D .2013.08.003. displays. International Journal of Psychophysiology, 122, 75–84. Jacobs, J., Hwang, G., Curran, T., & Kahana, M. J. (2006). EEG oscil- Dehn, L. B., Kater, L., Piefke, M., Botsch, M., Driessen, M., & Beblo, lations and recognition memory: Theta correlates of memory T. (2018). Training in a comprehensive everyday-like virtual retrieval and decision making. Neuroimage, 32(2), 978–987. reality environment compared to computerized cognitive train- Jacoby, L. L., & Dallas, M. (1981). On the relationship between auto- ing for patients with depression. Computers in Human Behavior, biographical memory and perceptual learning. Journal of Experi- 79, 40–52. mental Psychology: General, 110(3), 306. Diemer, J., Alpers, G. W., Peperkorn, H. M., Shiban, Y., & Mühlberger, Jensen, O., & Mazaheri, A. (2010). Shaping functional architecture A. (2015). The impact of perception and presence on emotional by oscillatory alpha activity: gating by inhibition. Frontiers in reactions: a review of research in virtual reality. Frontiers in Psy- Human Neuroscience, 4, 186. chology, 6, 26. Jensen, O., & Tesche, C. D. (2002). Frontal theta activity in humans Eckhorn, R., Reitboeck, H. J., Arndt, M. T., & Dicke, P. (1990). Fea- increases with memory load in a working memory task. European ture linking via synchronization among distributed assemblies: Journal of Neuroscience, 15(8), 1395–1399. Simulations of results from cat visual cortex. Neural Computation, Jones, T. C., & Jacoby, L. L. (2001). Feature and conjunction errors in 2(3), 293–307. recognition memory: Evidence for dual-process theory. Journal Ernstsen, J., Mallam, S. C., & Nazir, S. (2019). incidental memory of Memory and Language, 45(1), 82–102. recall in virtual reality: An empirical investigation. In: Proceed- Junghöfer, M., Elbert, T., Tucker, D. M., & Rockstroh, B. (2000). ings of the Human Factors and Ergonomics Society Annual Meet- Statistical control of artifacts in dense array EEG/MEG studies. ing (Vol. 63, No. 1, pp. 2277-2281). Sage CA: SAGE Publications. Psychophysiology, 37(4), 523–532. Felnhofer, A., Kothgassner, O. D., Schmidt, M., Heinzle, A. K., Beutl, Kisker, J., Gruber, T. & Schöne, B. (2019a). Behavioral realism and L., Hlavacs, H., & Kryspin-Exner, I. (2015). Is virtual reality lifelike psychophysiological responses in virtual reality by the emotionally arousing? Investigating five emotion inducing vir - example of a height exposure. Psychological Research. https :// tual park scenarios. International Journal of Human-Computer doi.org/10.1007/s0042 6-019-01244 -9. Studies, 82, 48–56. Kisker, J., Gruber, T. & Schöne, B. (2019b). Experiences in virtual Fries, P., Womelsdorf, T., Oostenveld, R., & Desimone, R. (2008). reality entail different processes of retrieval as opposed to con- The effects of visual stimulation and selective visual attention on ventional laboratory settings: A study on human memory. Curr rhythmic neuronal synchronization in macaque area V4. Journal Psychol. https ://doi.org/10.1007/s1214 4-019-00257 -2 of Neuroscience, 28, 4823–4835. Klimesch, W., Doppelmayr, M., Pachinger, T., & Ripper, B. (1997a). Gorini, A., Pallavicini, F., Algeri, D., Repetto, C., Gaggioli, A., & Brain oscillations and human memory: EEG correlates in the Riva, G. (2010). Virtual reality in the treatment of generalized upper alpha and theta band. Neuroscience Letters, 238(1–2), 9–12. anxiety disorders. Studies in Health Technology and Informatics, Klimesch, W., Doppelmayr, M., Schimke, H., & Ripper, B. (1997b). 154, 39–43. Theta synchronization and alpha desynchronization in a memory Greenberg, D. L., & Rubin, D. C. (2003). The neuropsychology of task. Psychophysiology, 34(2), 169–176. autobiographical memory. Cortex, 39(4–5), 687–728. Klimesch, W., Doppelmayr, M., Schwaiger, J., Winkler, T., & Gruber, Gromer, D., Reinke, M., Christner, I., & Pauli, P. (2019). Causal W. (2000). Theta oscillations and the ERP old/new effect: inde- interactive links between presence and fear in virtual reality pendent phenomena? Clinical Neurophysiology, 111(5), 781–793. height exposure. Frontiers in Psychology, 10, 141. https ://doi. Klimesch, W., Doppelmayr, M., Stadler, W., Pöllhuber, D., Sauseng, P., org/10.3389/fpsyg .2019.00141 . & Röhm, D. (2001a). Episodic retrieval is reflected by a process Gruber, T., Tsivilis, D., Giabbiconi, C. M., & Müller, M. M. (2008). specific increase in human electroencephalographic theta activity. Induced electroencephalogram oscillations during source mem- Neuroscience Letters, 302(1), 49–52. ory: familiarity is reflected in the gamma band, recollection in the Klimesch, W., Doppelmayr, M., Yonelinas, A., Kroll, N. E., Lazzara, theta band. Journal of cognitive neuroscience, 20(6), 1043–1053. M., Roehm, D., & Gruber, W. (2001b). Theta synchronization dur- Guderian, S., & Düzel, E. (2005). Induced theta oscillations mediate ing episodic retrieval: neural correlates of conscious awareness. large-scale synchrony with mediotemporal areas during recol- Cognitive Brain Research, 12(1), 33–38. lection in humans. Hippocampus, 15(7), 901–912. https ://doi. Klimesch, W., Schimke, H., & Schwaiger, J. (1994). Episodic and org/10.1002/hipo.20125. semantic memory: an analysis in the EEG theta and alpha band. Haatveit, B. C., Sundet, K., Hugdahl, K., Ueland, T., Melle, I., & Electroencephalography and Clinical Neurophysiology, 91(6), Andreassen, O. A. (2010). The validity of d prime as a working 428–441. memory index: results from the “Bergen n-back task”. Journal Konishi, S., Wheeler, M. E., Donaldson, D. I., & Buckner, R. L. (2000). of Clinical and Experimental Neuropsychology, 32(8), 871–880. Neural correlates of episodic retrieval success. Neuroimage, 12(3), Haque, S., & Conway, M. A. (2001). Sampling the process of autobio- 276–286. graphical memory construction. European Journal of Cognitive Krijn, M., Emmelkamp, P. M., Olafsson, R. P., & Biemond, R. (2004). Psychology, 13(4), 529–547. Virtual reality exposure therapy of anxiety disorders: A review. Harman, J., Brown, R., & Johnson, D. (2017). Improved memory elici- Clinical Psychology Review, 24(3), 259–281. tation in virtual reality: new experimental results and insights. In: Krokos, E., Plaisant, C., & Varshney, A. (2019). Virtual memory pal- IFIP Conference on human-computer interaction (pp. 128–146). aces: immersion aids recall. Virtual Reality, 23(1), 1–15. Springer, Cham. Kvavilashvili, L., & Ellis, J. (2004). Ecological validity and twenty Higuera-Trujillo, J. L., López-Tarruella Maldonado, J., & Llinares Mil- years of real-life/laboratory controversy in memory research: A lán, C. (2017). Psychological and physiological human responses critical (and historical) review. History and Philosophy of Psy- to simulated and real environments: A comparison between photo- chology, 9.   graphs, 360° panoramas, and virtual reality. Applied Ergonomics, LaFortune, J., & Macuga, K. L. (2018). Learning movements from a 65, 398–409. https ://doi.org/10.1016/j.aperg o.2017.05.006. virtual instructor: Effects of spatial orientation, immersion, and 1 3 Psychological Research (2021) 85:2485–2501 2501 expertise. Journal of Experimental Psychology: Applied, 24(4), Rugg, M. D., Fletcher, P. C., Allan, K., Frith, C. D., Frackowiak, R. S. 521. J., & Dolan, R. J. (1998). Neural correlates of memory retrieval Lanen, M., & Lamers, M. H. (2018). Context-dependent memory in during recognition memory and cued recall. Neuroimage, 8(3), real and virtual reality. In: International Conference on virtual 262–273. reality and augmented reality (pp. 177–189). Cham: Springer. Rugg, M. D., Fletcher, P. C., Frith, C. D., Frackowiak, R. S. J., & Lovett, A., Appleton, K., Warren-Kretzschmar, B., & Von Haaren, C. Dolan, R. J. (1996). Differential activation of the prefrontal cortex (2015). Using 3D visualization methods in landscape planning: in successful and unsuccessful memory retrieval. Brain, 119(6), An evaluation of options and practical issues. Landscape and 2073–2083. Urban Planning, 142, 85–94. Sauseng, P., Klimesch, W., Heise, K. F., Gruber, W. R., Holz, E., Makowski, D., Sperduti, M., Nicolas, S., & Piolino, P. (2017). “Being Karim, A. A., & Hummel, F. C. (2009). Brain oscillatory sub- there” and remembering it: Presence improves memory encoding. strates of visual short-term memory capacity. Current Biology, Consciousness and cognition, 53, 194–202. 19(21), 1846–1852. McDermott, K. B., Szpunar, K. K., & Christ, S. E. (2009). Laboratory- Schöne, B., Schomberg, J., Gruber, T., & Quirin, M. (2016). Event- based and autobiographical retrieval tasks differ substantially in related frontal alpha asymmetries: electrophysiological correlates their neural substrates. Neuropsychologia, 47(11), 2290–2298. of approach motivation. Experimental brain research, 234(2), Moscovitch, M., & Winocur, G. (2002). The frontal cortex and work- 559–567. ing with memory. Principles of frontal lobe function, 188, 209. Schöne, B., Wessels, M., & Gruber, T. (2017). Differences between Nilsson, N. C., Nordahl, R., & Serafin, S. (2016). Immersion revisited: real-life and laboratory memory: Evidence from EEG and virtual a review of existing definitions of immersion and their relation to reality. In Psychophysiology (Vol. 54, pp. S128–S128). 111 River different theories of presence. Human Technology, 12(2). ST, Hoboken 07030-5774, NJ USA: Wiley. Norman, D. A., & Bobrow, D. G. (1979). Descriptions: An interme- Schöne, B., Wessels, M., & Gruber, T. (2019). Experiences in virtual diate stage in memory retrieval. Cognitive Psychology, 11(1), reality: A window to autobiographical memory. Current Psychol- 107–123. ogy, 38(3), 715–719. Nyhus, E., & Curran, T. (2010). Functional role of gamma and theta Schöne, B., Kisker, J., Sylvester, R.S., Radtke, E.L., Gruber, T. (2020). oscillations in episodic memory. Neuroscience and Biobehavio- Library for Universal Virtual Reality Experiments (luVRe): A ral Reviews, 34(7), 1023–1035. https ://doi.org/10.1016/j.neubi standardized immersive 3D/360° picture and video database for orev.2009.12.014. VR based research. under review Pan, X., & Hamilton, A. F. D. C. (2018). Why and how to use vir- Schubert, T., Friedmann, F., & Regenbrecht, H. (2001). The experience tual reality to study human social interaction: The challenges of of presence: Factor analytic insights. Presence: Teleoperators & exploring a new research landscape. British Journal of Psychol- Virtual Environments, 10(3), 266–281. ogy, 109(3), 395–417. Serino, S., & Repetto, C. (2018). New trends in episodic memory Parsons, T. D. (2015). Virtual reality for enhanced ecological validity assessment: immersive 360 ecological videos. Frontiers in Psy- and experimental control in the clinical, affective and social neu- chology, 9, 1878. rosciences. Frontiers in Human Neuroscience, 9, 660. Slater, M., & Wilbur, S. (1997). A framework for immersive virtual Plancher, G., Barra, J., Orriols, E., & Piolino, P. (2013). The influence environments (FIVE): Speculations on the role of presence in vir- of action on episodic memory: a virtual reality study. Quarterly tual environments. Presence: Teleoperators & Virtual Environ- Journal of Experimental Psychology, 66(5), 895–909. ments, 6(6), 603–616. Plancher, G., & Piolino, P. (2017). Virtual reality for assessment of Slobounov, S. M., Ray, W., Johnson, B., Slobounov, E., & Newell, episodic memory in normal and pathological aging. The role of K. M. (2015). Modulation of cortical activity in 2D versus 3D technology in clinical neuropsychology, pp. 237–260. virtual reality environments: an EEG study. International Journal Ragan, E. D., Sowndararajan, A., Kopper, R., & Bowman, D. A. of Psychophysiology, 95(3), 254–260. (2010). The effects of higher levels of immersion on procedure Smith, S. A. (2019). Virtual reality in episodic memory research: A memorization performance and implications for educational vir- review. Psychonomic Bulletin & Review, 26(4), 1213–1237. tual environments. Presence: Teleoperators and Virtual Environ- Svoboda, E., McKinnon, M. C., & Levine, B. (2006). The functional ments, 19(6), 527–543. neuroanatomy of autobiographical memory: A meta-analysis. Rauchs, G., Orban, P., Balteau, E., Schmidt, C., Degueldre, C., Luxen, Neuropsychologia, 44(12), 2189–2208. A., & Peigneux, P. (2008). Partially segregated neural networks Swets, J. A., Tanner, W. P., Jr., & Birdsall, T. G. (1961). Decision pro- for spatial and contextual memory in virtual navigation. Hip- cesses in perception. Psychological Review, 68(5), 301. pocampus, 18(5), 503–518. Tallon-Baudry, C., & Bertrand, O. (1999). Oscillatory gamma activity Riva, G., Mantovani, F., Capideville, C. S., Preziosa, A., Morganti, in humans and its role in object representation. Trends in Cogni- F., Villani, D., & Alcañiz, M. (2007). Affective Interactions tive Sciences, 3(4), 151–162. Using Virtual Reality: The Link between Presence and Emo- Tulving, E. (1983). Ecphoric processes in episodic memory. Philo- tions. CyberPsychology & Behavior, 10(1), 45–56. https ://doi. sophical Transactions of the Royal Society of London B Biological org/10.1089/cpb.2006.9993. Sciences, 302(1110), 361–371. Roediger, H. L. III, & Marsh, E. J. (2003). Episodic and autobiographi- Wang, W., Subagdja, B., Tan, A. H., & Starzyk, J. A. (2012). Neural cal memory. In A. F. Healy & R. W. Proctor (Eds.), Handbook of modeling of episodic memory: Encoding, retrieval, and forgetting. psychology: Experimental psychology, Vol. 4 (p. 475–497). John IEEE Transactions on Neural Networks and Learning Systems, Wiley & Sons Inc. 23(10), 1574–1586. Rubin, D. C., Schrauf, R. W., & Greenberg, D. L. (2003). Belief and Willander, J., & Larsson, M. (2007). Olfaction and emotion: The recollection of autobiographical memories. Memory & Cognition, case of autobiographical memory. Memory & Cognition, 35(7), 31(6), 887–901. 1659–1663. Rubin, D. C., & Umanath, S. (2015). Event memory: A theory of memory for laboratory, autobiographical, and fictional events. Publisher’s Note Springer Nature remains neutral with regard to Psychological Review, 122(1), 1. jurisdictional claims in published maps and institutional affiliations. Rugg, M. D., & Curran, T. (2007). Event-related potentials and rec- ognition memory. Trends in cognitive sciences, 11(6), 251–257. 1 3 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Psychological Research Springer Journals

Virtual reality experiences promote autobiographical retrieval mechanisms: Electrophysiological correlates of laboratory and virtual experiences

Download PDF
17 pages

Loading next page...
 
/lp/springer-journals/virtual-reality-experiences-promote-autobiographical-retrieval-P68wGhx1nT

References (111)

Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2020
ISSN
0340-0727
eISSN
1430-2772
DOI
10.1007/s00426-020-01417-x
Publisher site
See Article on Publisher Site

Abstract

Recent advancements in memory research indicate that virtual reality (VR) experiences are more vividly memorized as compared to conventional laboratory events. In contrast to the latter, VR experiences are highly immersive, simulating the multimodality, vividness and inclusiveness of real-life experiences. Therefore, VR might enable researchers to identify memory processes underlying events which participants have actually experienced, in contrast to conventional on-screen experiences. To differentiate the electrophysiological correlates of memory processes underlying VR experiences as compared to conventional laboratory experiences, participants watched videos either in a PC condition or in a VR condition, followed by an unannounced recognition memory test. As hypothesized, we replicated the well-established theta old/new effect for the PC condition, but remarkably, this effect was absent in the VR condition. Additionally, the latter was accompanied by significantly lower alpha activity as compared to the PC condition. As increases in theta-band responses are related to top- down control on, and memory load during retrieval, the observed theta responses might rather relate to retrieval effort than to retrieval success per se. Congruently, higher alpha activity measured over occipital sensor areas in the PC condition reflect visually guided search processes within episodic memory. The VR condition comes in with lower alpha activity, reflecting immediate and effortless memory access. Hence, our findings indicate that the retrieval of VR experiences promotes auto- biographical  retrieval mechanisms, whereas recalling conventional laboratory events comes in with higher effort, which might not reflect the mechanisms of everyday memory. Introduction in their respective spatial and temporal context (Tulving, 1983). Extending well beyond EM, AM encompasses highly How people behave in everyday life strongly depends on self-relevant information, especially beliefs and knowledge previous experiences either with a particular situation or about the self, experienced events and their relevance (see personal general knowledge, e.g. concerning the realiza- e.g. Conway, 2005; Greenberg & Rubin, 2003). Hence, AM tion of own goals, acting effectively and relating to other comprises episodic engrams, extending it by self-referential peoples (see Conway, 2005). This kind of information is and emotional processes. The retrieval of autobiographical predominantly encoded in and retrieved from autobiographi- memories is therefore not limited to temporal, spatial or con- cal memory (AM). Similar to episodic memory (EM), auto- textual information, but bears great personal significance biographical engrams encode personally experienced events (Svoboda, McKinnon, & Levine, 2006). The retrieval of such everyday memories promotes the re-experience of the associated emotions (Svoboda et al., 2006), coming in with Electronic supplementary material The online version of this vivid and conscious reliving, and foremost the belief that article (https ://doi.org/10.1007/s0042 6-020-01417 -x) contains they have actually occurred (Rubin, Schrauf & Greenberg supplementary material, which is available to authorized users. 2003; Greenberg & Rubin, 2003). While it is common practice to investigate everyday * Joanna Kisker joanna.kisker@uni-osnabrueck.de memory in the laboratory using paradigms that induce micro-events prior to recognition memory tests (see Cabeza Experimental Psychology I, Institute of Psychology, et al., 2004), these settings are often criticized for lacking Osnabrück University, Seminarstraße 20, 49074 Osnabrück, the complexity and variety of stimuli and response options Germany Vol.:(0123456789) 1 3 2486 Psychological Research (2021) 85:2485–2501 characteristic to real-life experiences (Pan & Hamilton, 2019a; Schöne et al., 2016, Schöne et al. 2019). Hence, VR 2018; Kvavilashvili & Ellis, 2004). Specifically, self-rele- might improve the possibilities to investigate the mecha- vance and self-involvement are rarely realized in laboratory nisms underlying real-life memory (see Parsons, 2015; settings (see e.g. McDermott, Szpunar, & Christ, 2009). Serino & Repetto, 2018; Schöne et al., 2016, Schöne et al. Obviously, such traditional approaches face a trade-off 2019; Kisker et al., 2019b; Burgess et al., 2001). between high experimental control and ecological validity, Initial studies of memory processes under immersive VR i.e. the validity of the results obtained in the laboratory and conditions found that retrieval of VR experiences is not only generalized to everyday life (see Parsons, 2015). enhanced compared to the retrieval of conventional labora- Potentially overcoming this gap between experimen- tory micro-events (see e.g. Serino & Repetto, 2018; Smith, tal control and ecological validity, virtual reality (VR) 2019; Schöne et  al., 2016, Schöne et  al. 2019; Krokos, has gained interest as a methodical tool in psychological Plaisant, & Varshney, 2019; Ernstsen, Mallam & Nazir, research (see e.g. Parsons, 2015; Pan & Hamilton, 2018; 2019; Harman, Joel, Brown, Ross & Johnson, 2017), but Schöne et al., 2017, Kisker, Gruber, & Schöne 2019a, b). For also provides a closer approximation to real-life memory memory research, VR experiences might provide a closer processes (Schöne et al., 2016; Schöne et al. 2019; Kisker approximation to real-life experiences as compared to con- et al., 2019b). In particular, a previous study found evidence ventional laboratory settings. The former is characterized that immersive VR experiences become part of an extensive by a high level of sensory cues and thus, by high fidelity of autobiographical associative network, whereas conventional the represented environment (Dan & Reiner, 2017). Accord- video experiences remain an isolated episodic event (Schöne ingly, VR environments are more pronounced regarding viv- et  al., 2019). Going one step further, the retrieval of VR idness as compared to classical setups (Slater & Wilbur, experiences is proposed to mainly rely on recollection, i.e. 1997), which is also characteristic for AM (Greenberg & vivid and accurate remembering of events (e.g. Atkinson Rubin, 2003). In particular, everyday experiences arise & Juola, 1973; Jacoby & Dallas, 1981) which is associated from the complex, multisensory 3D-environment of the real with AM (Roediger & Marsh, 2003; Conway, 2005). In con- world, while laboratory memories are generated by highly trast, retrieval of memories induced by conventional labora- controlled events rather poor in sensory information (Cabeza tory settings predominantly fall back on familiarity-based & St Jaques, 2007). Moreover, the formation of such memo- mnemonic processes (Kisker et al., 2019b), characterized as ries is accompanied by intuitive and quick monitoring and a subjective, vague feeling to remember a previous experi- closely linked to self-referential processing (Moscovitch & ence (e.g. Curran & Hancock, 2007; Rugg & Curran, 2007). Winocur, 2002; Cabeza & St Jaques, 2007). Importantly, Although both groups principally employed both, familiarity the latter is as well increased under VR conditions due to and recollection as non-exclusive retrieval mechanisms (see its immersive character: VR facilitates an increased sense Jones and Jacoby, 2001), one mechanism predominated over of presence, i.e. the subjective feeling of being within a vir- the other as a function of the encoding context. Accordingly, tual environment (VE; e.g. Slater & Wilbur, 1997; Schubert encoding in VR resulted in a more precise and vivid retrieval et al., 2001; Nilsson, Nordahl, & Serafin, 2016). Whereas than encoding the same scenario in a PC setup (Kisker et al., immersion predominantly determines the degree to which 2019b). the user is isolated from his physical surroundings by techni- Overall, these studies suggest that VR experiences are cal factors, like 3D-360° view and proprioceptive matching, not just observed, i.e. passively watching stimuli presented presence promotes the subjective feeling of actually being in on a screen, but experienced in a self-relevant manner. Even and acting within the VE (Slater & Wilbur, 1997; Nilsson, interactive PC setups designed as immersive as possible by et al., 2016). Consequently, the sensation of acting within means of active exploration of a desktop-based environ- the VE comes in with the impression of being subject to the ment, generate overall rather superficial engrams compared consequences of these actions and events in the VE (Slater to exactly the same VE explored as a VR experience (Kisker & Wilbur, 1997; Nilsson, et al., 2016). For example, par- et al., 2019b). Unlike conventional laboratory experiences, ticipant behave as if being in real danger when exposed to the latter become part of a personal experience like real-life dangerous situations in an immersive VE, even though their experiences would (Schöne et al., 2016, 2019). surroundings could not physically harm them (e.g. Kisker However, while the electrophysiological correlates of, et al., 2019a; Krijn et al., 2004; Gromer et al., 2019). In line, for example, the sense of presence (e.g. Bouchard et al., VR setups have been found to elicit the same emotional and 2009) and spatial memory (e.g. Rauchs et al., 2008) are physical reactions as compared to their real-life equivalents recently more widely investigated, findings regarding the (Gorini et al., 2010; Higuera-Trujillo et al., 2017). Given this electrophysiological correlates of retrieval of episodic and impression of mutual interaction with the virtual surround- autobiographical engrams encoded within VR are still rare ings, VR experiences are more personally and emotionally (cf. e.g. Smith, 2019; Serino & Repetto, 2018; Plancher & relevant than mere on-screen experiences (see Kisker et al., Polino, 2017; Bohil et al., 2011). Accordingly, it is the aim 1 3 Psychological Research (2021) 85:2485–2501 2487 of our study to differentiate the electrophysiological corre- 2008; Klimesch et al., 1997a, 2001a, b). In line, the alpha- lates of the retrieval of VR experiences as opposed to con- band response should significantly decrease for new pictures ventional laboratory experiences. Specifically, we examined as compared to old pictures. Concerning the VR condition, a well-established electrophysiological marker of recogni- different outcomes might be possible: Under the premise tion memory tasks by means of the theta old/new effect that the theta old/new effect is exclusively linked to suc- obtained from laboratory settings (for review see Nyhus & cessful memory retrieval, theta-band synchronization for old Curran 2010; Guderian & Düzel, 2005; Hsieh & Ranganath, stimuli should be higher for the VR condition as compared 2014; see also Gruber, Tsivilis, Giabbiconi & Müller, 2008; to the PC condition, as most studies indicate that VR set- Klimesch et al., 1997a, 2001a). Therefore, we examined ups enhance memory performance (e.g. Schöne et al. 2016, theta-oscillations (~ 4-8 Hz; e.g. Nyhus & Curran, 2010), 2019; Smith, 2019) and activate recollection-based engrams which are most prominent at sensors over frontal-midline (Kisker et al., 2019b). For the alpha-band, a similar pattern regions (e.g. Hsieh & Raganath, 2014). There is broad and of results might be expected. However, as theta-band oscil- stable consensus, that a characteristic theta-band synchro- lations are related to further memory-related processes, e.g. nization can be observed in these regions in response to the memory load (Nyhus & Curran, 2010; Jensen & Tesche, retrieval of old stimuli, which are correctly remembered, i.e. 2002), decision making (Nyhus & Curran, 2010) and work- in response to retrieval success. In contrast, new stimuli are ing memory (Hsieh & Ranganath, 2014), another outcome associated with theta-band desynchronization (e.g. Nyhus than the classical effect might be equally likely in the VR & Curran, 2010). This effect was observed both subsequent condition. to the stimulus presentation (e.g. Klimesch et al., 1997b; Klimesch et al., 2001a) and after a physical response of par- ticipants, e.g. key pressure (Gruber, Tsivilis, Giabbiconi, & Methods Müller, Gruber et al., 2008). Moreover, theta-oscillations are associated with recollection of personal events (Gude- Participants rian & Düzel, 2005) and hippocampal projections to neo- cortical frontal regions are regarded as possible generators 45 participants were recruited from Osnabrück University. of these oscillations during memory tasks (e.g. Hsieh & The sample size was determined on the basis of previous Ranganath, 2014). In conjunction with the characteristic studies with a similar study design (cf. Schöne et al., 2019; frontal-midline theta-band synchronization, a decrease of Kisker et al., 2019a). All participants were screened for psy- the alpha-band response (~ 8–13 Hz, e.g. Berger, 1929) can chological and neurological disorders and had normal or regularly be observed during memory recall (e.g. Klimesch, corrected-to-normal sight. Three participants were excluded et al., 1997b; Sauseng et al., 2009; Jacobs, Hwang, Curran during the anamnesis. When vision correction was neces- & Kahana, 2006). This decrease of alpha-band response is sary, only those participants who had contact lenses could regarded a reflection of visual processing (Clayton, Yeung participate, not those who wore glasses. It was ensured that & Cohen Kadosh, 2018), attentional processes (Klimesch the participants saw sharply on the screen as well as on the et al., 1997a) and memory load (Sauseng et al., 2009; Jacobs head-mounted display. Previous experience with VR envi- et al., 2006; Jensen & Tesche, 2002; Dan & Reiner, 2017). ronments was documented. All participants gave informed In short, the theta-band synchronizes in response to mental consent and were blind to the research question. The par- activity, whilst the alpha-band desynchronizes (Berger, 1929 ticipants received either partial course credits or 15€ for as cited in Klimesch et al., Klimesch, Doppelmayr, Schimke, participation. et al. 1997b). The participants were randomly assigned to both condi- To examine whether this well-established and robust tions (VR vs. PC). Three participants were excluded from effect occurs under VR conditions as well, we set up an analysis due to insufficient data quality (n = 2) and prior experiment in which participants incidentally encoded either knowledge of the stimulus material used for the unan- immersive 3D-360° videos or conventional 2D videos fol- nounced recognition memory test (n = 1). After exclusion, lowed by an unannounced recognition memory test. We we obtained 39 complete datasets for analysis (VR group: assume that the VR condition will result in a higher sense of n = 20, M=21.95, SD = 3.19, 15 female, 19 right- VR age age presence, better memory performance and higher accuracy handed; PC group: n = 19, M=22.16, SD = 2.32, 13 PC age age of memory judgements as compared to the conventional PC female, 18 right-handed). condition. Moreover, we hypothesize to replicate the theta old/new effect for the conventional PC condition, manifested significant difference between theta-band responses to old and new stimuli, including a synchronization for old, and a desynchronization for new stimuli (see e.g. Gruber et al., 1 3 2488 Psychological Research (2021) 85:2485–2501 constant at 80 cm. The videos were presented in 2D videos Encoding in full-screen resolution. Sound was presented over standard speakers placed on both sides of the monitor. Stimulus material For both conditions, the videos were presented in ran- domized sequences with the GoPro VR Player, providing One hundred 3D-360° videos from the Library for Universal Virtual Reality Experiments (luVRe, Schöne, Kisker, Syl- the same video resolution for both conditions (cf. stimuli). Each randomized sequence was presented to one partici- vester, Radtke & Gruber, 2020;  https ://www.psych o.uni- osnab r uec k .de/f ac hg ebie t e/allg e meine _psy c h ologi e_i/ pant per condition. Each video was preceded by one-sec- ond fixation on a fixation cross. To facilitate incidental luvre.html ) were used as stimulus material. All videos were recorded with the Insta360Pro VR-camera with a frame rate encoding, the presentation of each video clip was followed by a rating (10 s) as a distraction task (cf. Fig. 1). Partici- of 60 fps and 4 k resolution. Each video was 10 s long. The videos were randomly subdivided into targets and distrac- pants were instructed to separately rate the experienced valence, arousal and motivation, i.e. their desire to stay in tors for the unannounced recognition memory test in a 50:50 ratio. The themes of the videos were balanced between target or leave the presented scene for each video separately on a scale from one (bad/not at all) to six (good/very much; and distractor videos (e.g. nature footage, interiors, medi- cal facilities, sport events, social events; see supplementary cf. Kuhr et al., 2015). The ratings were consecutively pre- sented on the (virtual) screen for 3.33 s each. The partici- material for a detailed description of the video content). Only the target videos were presented during incidental pants were familiarized with the rating before the video presentation. To guarantee for similar visual experience learning. Distractor videos were unknown to the participants and only used for the unannounced recognition memory test. during the rating and maintain immersion, the rating took place in an exact virtual simulation of the laboratory in Procedure which the study actually took place, implemented as a 3D-360° video recording of the laboratory. In addition, the Participants were randomly assigned to the VR- or the PC- rating scales were displayed on the (virtual) monitor dur- ing rating phase. For the PC group, the simulation of the condition. For the VR-condition, participants were equipped with a wireless version of the HTC Vive Pro head-mounted laboratory was displayed as a 2D video as well. The par- ticipant’s answers were recorded with a dictation device. display. Video footage and sound were presented in 3D-360°, with a resolution of 1080 × 1200 pixels per display. Partici- The ratings regarding valence, arousal and motivation of the videos were collected for the validation of a database pants were allowed to look around, but not to turn 180° or walk around. and will not be further analyzed in this study. The pres- entation of the videos took a total of 19 min. To enhance For the PC-condition, participants were seated in front of a curved monitor (35″, 90 cm screen diagonal, 37 cm immersion, all test leaders left the lab until the end of the video presentation. The participant was given a bell to height). The participant’s distance to the screen was kept Fig. 1 Procedure of inciden- tal encoding. Each of the 50 target videos was preceded by a fixation on a virtual screen and followed by the rating of valence, arousal and motiva- tion. Each scale was faded in on the virtual screen separately for 3.33 s. During fixation and rating, a 3D-360° image of the laboratory in which the partici- pants were actually located was presented 1 3 Psychological Research (2021) 85:2485–2501 2489 alert the test leaders if they wanted to quit the experiment started with randomly 0.5–0.8 s fixation, followed by 1.5 s early or felt uncomfortable. presentation of the stimulus. The rating scale was then dis- To determine the sense of presence, participants were played until the participants responded via key pressure. The asked to fill in the German version of the Igroup Presence interstimulus interval lasted randomly between 1.0 s and Questionnaire (IPQ; Schubert, Friedmann & Regenbrecht, 1.5 s (see Fig. 2). The response options were defined during 2001) and were asked for their experience of physical symp- instruction as follows (translated from German): toms (vertigo, nausea). In addition, the participants were instructed not to discuss the videos with the test leaders until (1) Definitely unknown: I’m sure I’ve not seen this place the end of the experiment. (2) Rather unknown: I guess I haven’t seen this place (3) Familiar: This place looks familiar to me Unannounced recognition memory test (4) Vividly remembered: I remember this place precisely and vividly. Stimulus material Electrophysiological recordings and preprocessing Monoscopic screenshots from both, distractors (referred to as new pictures) and targets (referred to as old pictures), An electroencephalogram (EEG) with 128 electrodes, were used as stimulus material for the unannounced recogni- attached in accordance with the international 10-20-system tion memory test. Per video, one representative screenshot was recorded for the duration of the unannounced recogni- was utilized as stimulus, resulting in 100 trials. The stimuli tion memory test. The Active-Two amplifier system from were presented on a conventional 24″ monitor with a para- BioSemi (Amsterdam, Netherlands) was used. The sampling foveal visual angle of 2 × 5°. rate was 1024 Hz, the bandwidth (3 dB) 104 Hz. Addition- ally, horizontal electrooculogram (hEOG) and vertical elec- Procedure trooculogram (vEOG) were recorded and a common mode sense (CMS) and a driven right leg (DRL) electrode were The retention interval was set to 1 h during which the EEG applied. The EEG was recorded on the investigators’ com- was applied. If the participants mentioned the videos they puter using ActiView702 Lores. had seen during encoding, they were kindly interrupted and EEG data were analyzed using MATLAB. For further asked not to discuss the videos until the end of the experi- off-line analysis, the average reference was used. The ment. Participants were instructed about their task imme- EEG was segmented to obtain epochs starting 500  ms diately before the unannounced recognition memory test. prior and 1500  ms following stimulus onset (baseline The unannounced recognition test comprised of 100 tri- − 300 to − 100 ms). Artifact correction was performed als. Per trial, participants had to indicate as fast as possi- by means of ‘‘statistical correction of artifacts in dense ble whether they recognized the presented stimulus as (1) array studies’’ (SCADS; Junghöfer, Elbert, Tucker, & definitely unknown, (2) rather unknown, (3) familiar or (4) Rockstroh, 2000). In brief, this procedure uses a combi- vividly remembered (cf. Kisker et al., 2019b). Each trial nation of trial rejection and channel approximation based Fig. 2 Setup of the memory test trials: 0.5–0.8 s fixation, 1.5 s stimulus presentation, presentation of the scale until the participant’s response, 1.0–1.5 s inter stimulus interval (ISI). Participants were asked not to blink from fixation until the response scale appeared 1 3 2490 Psychological Research (2021) 85:2485–2501 on statistical parameters of the data. For each trial, con- Schwaiger, Winkler & Gruber, 2000; Klimesch al., 2001b; taminated electrodes are detected based on a threshold Jacobs et al., 2006). The alpha frequency band (8–13 Hz, criterion derived from the distribution of the amplitude, see e.g. Berger, 1929) was analyzed at electrodes sur- standard deviation, and gradient of the sensor across all rounding Oz, O1 and O2 in the time window from 0 to trials. The information of these electrodes is replaced 500 ms. with a spherical interpolation from the full channel set. The limit for the number of approximated channels was Statistical analysis set to 20. Epochs containing more than 20 channels with artifacts were rejected. Presence For demonstrating a robust signal at the frequency bands of interest, we first calculated a conventional fast The IPQ scales were determined as sum values of the respec- Fourier transform (FFT, see Fig. 5) per trial and averaged tive items (in total: 14 items; general presence: one item, across all electrodes, conditions and participants. spatial presence: five items, involvement: four items, real - For further analyses and a comparison between experi- ness: four items). Each item could reach values from − 3 and mental conditions, we considered it advantageous to take + 3 on a 7-step likert-scale, resulting in the following mini- the signal’s temporal evolution into account. Thus, for the mum and maximum sumscores per scale: General Presence subsequent examinations, spectral changes in oscillatory (− 3; 3), Spatial Presence (− 15; 15), Involvement (− 12; activity were analysed by means of Morlet wavelets with 12), Realness (− 12; 12). a width of 12 cycles per wavelet which is described in Shapiro–Wilk-test rejected normal distribution for one detail elsewhere (e.g., Tallon-Baudry & Bertrand, 1999; of the IPQ scales (General Presence, p < 0.05). Therefore, Bertrand & Pantev, 1994). In brief, the method provides the more robust Mann–Whitney U test as non-parametric a time-varying magnitude of the signal in each frequency equivalent of the unpaired t test was used for analysis. Cron- band, leading to a time-by-frequency (TF) representation bach’s α was calculated for each scale, with the exception of of the data. Due to the fact that induced oscillatory activ- the one-item-scale General Presence. ity occurs with a jitter in latency from one trial to another (Eckhorn et al., 1990), they tend to cancel out in the aver- aged evoked potential. Thus, TF amplitude is averaged Memory performance across single-trial frequency transformations, allowing one to analyze non-phase-locked components. Further- D′-prime (d′) was calculated separately for both groups as an operationalization of memory performance. D’ relates more, because we focused on the non-phase-locked com- ponents of the signal, the evoked response (i.e., the ERP) the hits, i.e. correct positive judgments, to the false-positive judgments (d’ = z(hit) −− z(false positive); Haatveit et al. was subtracted from each trial before frequency decompo- sition (for details, see Busch, Herrmann, Müller, Lenz, & 2010; Swets et al., 1961; as cited in Kisker et al., 2019b) and indicates how well participants are able to distinguish Gruber, 2006). Given our interest in the lower-frequency range, we used wavelets from 0.25 Hz to 30 Hz. between targets and distractors. D’-prime was calculated per group to assess the overall retrieval success (general Based upon prior literature (e.g. Nyhus & Curran, 2010) and our hypothesis, the frequency range from d’ = z(all hits) – z(all false positives)). Additionally, d’ was separately calculated for familiarity 4-7 Hz was included in the analyses and checked against visual inspection of the FFT (see Fig. 5). However, visual and recollection for each group, taking only the respective hits and false positives into account (cf. Kisker et al., 2019a: inspection of the FFT revealed high power for 2–4 Hz as well. This frequency range is commonly denoted as the d’-familiar ity score = z(familiarity hits) – z(familiarity false positives); d’-recollection score = z(recollection hits) delta-band, but was also identified as lower theta-band in some studies, indicating that the old-new effect might be – z(recollection false positives). Shapiro–Wilk-test rejected normal distribution for all d’ scores (all p < 0.05). Hence, ref lected in the 2–4 Hz frequency range as well (cf. Bur- gess & Gruzelier, 1997; Klimesch, Schimke & Schwaiger, Mann–Whitney U Test was used for analysis. Accuracy [(hits + correct rejection)/total number of trials] 1994; Klimesch et al., 2000). Hence, the 2–4 Hz response was included in the analyses as well. Electrodes around and error rate [(misses + false positives)/total number of tri- als] of recognition judgements were calculated per group. Fz covering for the frontal midline region were chosen. Based upon prior literature, an early latency range from Both were analyzed using the unpaired t test. 250 to 650 ms for the 2–4 Hz response (see Burgess & Gruzelier, 1997) and 200–600 ms for the 4–7 Hz band response were used for analyses (e.g. Guderian & Düzel, 2005; Klimesch et  al., 1997b; Klimesch, Doppelmayr, 1 3 Psychological Research (2021) 85:2485–2501 2491 Prior VR experience and cybersickness Dependent measures Prior experience with VR and cybersickness were assessed EEG data were analyzed using a 2 × 2 repeated-measure- as nominal variables (prior experience: “Have you already ments ANOVA (rmANOVA) with the between-factor “group” had any experience with virtual reality, e.g. studies, games (VR vs. PC) and the within-factor “condition” (new pictures or videos?”, [yes/no]; cybersickness: “Did you experience vs. old pictures). Significant effects of rmANOVA were com- physical symptoms such as nausea or dizziness during the plemented by post hoc t tests. experiment?”, [yes/no]; if yes: “How strongly did you feel nauseous/dizzy?” [1–10]; cf. Kisker et al., 2019a). Contin- gency tables and Pearson’s Chi square (X ) test were used Results for statistical analysis. Subjective measures Ratings of the videos Presence The ratings regarding valence, arousal and motivation of the videos were collected for the validation of a database and As hypothesized, the VR-group reported a higher feeling of will not be further analyzed in this study. To check that the presence during video presentation (see Fig. 3). This is valid target videos were perceived comparably emotive in both for all IPQ subscales (all p ≤ 0.005; see Table 1). Cronbach’s groups, arousal and valence averaged over all 50 target vid- α indicates acceptable reliability for all scales (all α ≥ 0.64). eos were compared between the groups using unpaired t test. Prior VR experience and cybersickness In both groups, about 70% of the participants had already gained experience with VR prior to the study, e.g. by partici- pating in other studies, watching VR videos or playing VR- games (X (1) = 0.011, p = 0.915). In total, nine subjects (n : VR six, n : three) reported experiencing physical symptoms PC like nausea and dizziness, but on a very mild level (nausea, in total: M = 2.55, SD = 2.13; VR: M = 3.33, SD = 2.25; VR VR PC: M = 1.0, SD = 0.0; dizziness, in total: M = 1.67, PC PC SD = 1.12; VR: M = 2.00, SD = 1.27; PC: M = 1.0, VR VR PC SD = 0.0), resulting in significantly stronger experiences PC of physical symptoms in the VR condition (X (1) = 4.91, p = 0.027). Ratings of the videos Fig. 3 Median scores of the IPQ scales General Presence, Spatial Participants of both groups reported equal levels of valence Presence, Involvement and Realness as evaluated by both groups. The and arousal averaged across all target videos (valence: error bars depict the standard error per scale. Minimum and maxi- mum sumscores per scale: General Presence (−  3; 3), Spatial Pres- M = 3.89, SD = 0.52, M = 3.61, SD = 0.47, VR VR PC PC ence(− 15; 15), Involvement (− 12; 12), Realness (− 12; 12) Table 1 Differences between VR- and PC-group regarding the sensa- and Cronbach’s α per scale. Cronbach’s α could not be calculated for tion of presence, assessed via the IPQ (Schubert et  al., 2001): Test the one-item-scale General Presence statistics of the one-tailed Mann–Whitney U test, descriptive values IPQ scale U z p Md Md Cronbach’s α VR PC General presence 41.00 − 4.29 < .001 2.00 − 2.00 Spatial PRESENCE 19.00 − 4.72 < .001 3.00 − 7.00 0.68 Involvement 98.00 − 2.59 0.005 1.50 − 4.00 0.70 Realness 64.50 − 3.55 < .001 -0.50 − 4.00 0.64 1 3 2492 Psychological Research (2021) 85:2485–2501 t(34) = 1.67, p = 0.103; arousal: M = 2.64, SD = 0.55, rates (t(37) = − 0.505, p = 0.31, M = 0.09, M = 0.08; see VR VR VR PC M = 2.65, SD = 0.47, t(34) = − 0.28, p = 0.978). Fig. 4), indicating a ceiling effect. PC PC Dependent measures Memory performance Since the behavioral data indicate no difference in memory Participants of both groups performed equally well on the performance between both groups, and since the high accu- unannounced recognition memory test, as none of the cal- racy indicates a ceiling effect, the latency range following culated d′ scores revealed significant differences (d’-general: stimulus onset was analyzed instead of the latency range U = 186.00, z = − 0.11, p = 0.462; d′-familiarity: U = 150.50, following the participants’ response (key pressure) to the z = − 1.11, p = 0.14; d’-recollection: U = 162.50, z = − 0.77, stimulus (cf. results, memory performance). p = 0.22; see Fig. 4). The visual inspection of the FFT validated the hypothe- Moreover, both groups achieved surprisingly high lev- sis-driven selection of the 4–7 Hz and 8–13 Hz frequency els of accuracy around 90% (t(37) = − 0.505, p = 0.308, ranges. In addition, the visual inspection also revealed a M = 0.91, M = 0.92) and correspondingly low error VR PC noticeable power of the 2–4 Hz frequency range, which is Fig. 4 Panel A depicts the accuracy as well as the respective error approximately 0.01 and therefore hardly visible in the figure. Panel rate of the judgement on the recognition or unknown character of the B depicts the retrieval success per group operationalized by general memory task trials in percent for both groups. The error bars depict d’ prime, as well as the d’-familiarity and d’-recollection scores. No the standard errors. For accuracy and error rate, the standard error is significant differences were found between both groups Fig. 5 Power spectra from fast Fourier transform (FFT) per group and condition. Visual inspection revealed a strong frequency peak from 2 to 4 Hz, which was hence included in the analyses 1 3 Psychological Research (2021) 85:2485–2501 2493 why it was also included in the analyses (see Fig.  5, see (t(37) = 2.06, p = 0.046; see Figs. 6 and 7). The theta-band methods). response to new pictures (t(37) = − 0.14, p = 0.889) and to old pictures (t(37) = 1.65, p = 0.107) did not differ between 4–7 Hz responses both groups (see Figs. 6, 7 and 8). Regarding frontal-midline theta-band responses, no sig- 2–4 Hz responses nificant main effects could be found (F (1,37) = 2.84, condition p = 0.10; F (1,37) = 0.38, p = 0.543), but a signifi- For the 2–4 Hz response, a significant main effect for the group cant interaction of the factors “group” and “condition” factor “condition” (F (1,37) = 11.61, p = 0.002), but condition (F (1,37) = 5.03, p = 0.046). not for the factor “group” (F (1,37) = 1.44, p = 0.239) interaction group Post-hoc t tests revealed a classical old/new effect in could be found. The main effect of “condition” was further the PC condition with a higher amplitude for old pictures characterized by a significant interaction of both factors than for new ones (t(18) = − 2.86, p = 0.010). However, (F (1,37) = 4.11, p = 0.049). Following the same interaction this difference effect was absent within the VR condi- trend as the 4-7 Hz responses, post hoc t tests revealed tion (t(19) = 0.25, p = 0.805). Furthermore, the observed a classical old/new effect across conditions (t (38) = 3.23, difference effect was comparably larger in the PC-group p = 0.003), as well as in the PC condition (t(18) = − 4.74, p < 0.001), but not in the VR condition (t(19) = − 0.85 p = 0.404). Again, the observed difference effect was com- parably larger in the PC-group (t(37) = 2.03, p = 0.049). But most importantly, old pictures elicited greater responses in the PC group compared to the VR-group (t(37) = 2.07, p = 0.046), whereas responses to new pic- tures did not differ between both groups (t (37) = − 0.05, p = 0.96; see Figs. 9, 10 and 11). Alpha‑band responses Regarding the alpha-band responses (8–13  Hz) at occipital electrodes, a main effect of the factors group (F (1,37) = 4.26, p = 0.046) and condition g roup Fig. 6 Mean amplitude in µV regarding the 4–7  Hz response in the (F (1,37) = 13.80, p < 0.001), but no significant inter - condition latency range from 200 to 600 ms after stimulus onset. The error bars action of both factors was found (F (1,37) = 1.21, depict the standard error of the mean amplitude. Significant differ - interaction ences are marked (*p < 0.05) p = 0.278). Fig. 7 Time-by-amplitude plot of the 4–7 Hz response from 200 ms before stimulus onset to 1200 ms after stimulus onset. While the classical old/ new-effect is also descriptively shown in the PC condition, there are no significant differ - ences between old and new pictures regarding the VR- group. The gray highlighted section  marks the latency range of significant interaction. The amplitude was averaged across the electrodes around Fz, covering for the frontal midline region 1 3 2494 Psychological Research (2021) 85:2485–2501 Fig. 8 Topography of the amplitude regarding the 4–7 Hz response separately for all com- binations of the factors group (VR vs. PC) and conditions (old vs. new) in the latency range from 200 to 600 ms after stimulus onset. Additionally, a difference plot of the old/new- effect is depicted. Black dots mark the electrodes which were included in the analyses More specifically, new pictures elicited lower alpha from the luVRe database (see methods), or watched the exact amplitudes as compared to old pictures (t (38) = 3.68, same stimulus material on a conventional 2D monitor (PC p < 0.001). In line, alpha amplitudes were significantly condition). In an unannounced recognition test, we com- lower for the PC group as compared to the VR group pared their memory performance, the mid-frontal theta old/ (t(76) = 2.75, p = 0.008; see Figs. 12, 13 and 14). new effect indexing mnemonic processing, as well as poste- rior alpha as a marker for visual processing load. As a result, both groups performed equally well in the recognition test, Discussion although the theta old/new effect could only be replicated for the PC condition and was absent in the VR condition. Addi- The aim of the study was to investigate the electrophysiolog- tionally, the theta effect was accompanied by a profound ical correlates of the retrieval of VR experiences as opposed reduction of posterior alpha in the PC condition, indicating to conventional laboratory experiences. To this end, par- a visually guided, effortful retrieval process. ticipants watched either 3D-360° VR videos (VR condition) Meeting our expectations, participants of the VR condi- tion felt more present during video presentation as compared to the PC condition, confirming that our video approach led to immersive VR experiences. Presence, as the most promi- nent feature of VR experiences (e.g. Schubert et al., 2001, Pan & Hamilton, 2018; Diemer et al., 2015; Alshaer, Regen- brecht, & O’Hare, 2017; Riva et al., 2007; Kisker et al., 2019a), is associated with increased emotional involvement (e.g. Gorini et al., 2010; Felnhofer et al., 2015), and stronger and more realistic behavioral responses as compared to con- ventional laboratory settings (Slobounov et al., 2015; Kisker et al., 2019a). Importantly, previous studies found that a high degree of presence aids memory recall: For example, both intentional encoding, as well as incidental encoding in a VE resulted in a more accurate memory recall as compared to conventional desktop conditions (e.g. Krokos, Plaisant & Fig. 9 Mean amplitude in µV regarding the 2–4  Hz response in the Varshney, 2019; Ernstsen, Mallam & Nazir, 2019). Hence, latency range from 250 to 650 ms after stimulus onset. The error bars presence might facilitate encoding processes constituting depict the standard error of the mean amplitude. Significant differ - the VR memory superiority effect (Makowski, Sperduti, ences are marked (*p < 0.05; ** p < 0.01) 1 3 Psychological Research (2021) 85:2485–2501 2495 Fig. 10 Time-by-amplitude plot of the 2–4 Hz response from 200 ms before stimulus onset to 1200 ms after stimulus onset. The gray highlighted section  marks the latency range of sig- nificant interaction. The ampli- tude was averaged across the electrodes around Fz, covering for the frontal midline region Fig. 11 Topography of the amplitude regarding the 2–4 Hz response separately for all com- binations of the factors group (VR vs. PC) and conditions (old vs. new) in the latency range from 250 to 650 ms after stimulus onset. Additionally, a difference plot of the old/new- effect is depicted. Black dots mark the electrodes which were included in the analyses Nicolas & Piolino, 2017; Serino & Repetto, 2018; Smith, 2019). In particular, visually detailed environments that pro- vide high realism and resemblance to the real world, such as 3D-360° videos (Pan & Hamilton, 2018; Lovett et al., 2015), facilitate more accurate judgments in old/new tasks (Smith, 2019). The resulting coherent egocentric perspective facili- tates recollection and reliving of such content (see Rubin & Umanath, 2015), which is crucial to form vivid, real- Fig. 12 Mean alpha amplitude (8-13  Hz) in µV in the latency range life memories (Conway, 2005; Roediger & Marsh, 2003). from 0 to 500 ms after stimulus onset. The error bars depict the stand- Hence, a high sense of presence—including sensations of ard error of the mean amplitude. Significant differences are marked spatial presence, involvement and realness—means that (*p < 0.05; **p < 0.01) 1 3 2496 Psychological Research (2021) 85:2485–2501 Fig. 13 Time-by-amplitude plot of the alpha-band response (8–13 Hz) from 200 ms before stimulus onset to 1200 ms after stimulus onset. Descriptively, a stronger reduction of the alpha amplitude was observed for both PC conditions compared to both VR conditions. The gray highlighted section  marks the latency range of both significant main effects Fig. 14 Topography plot of alpha amplitude in µV (8–13 Hz) separately for fac- tors group (VR vs. PC) and conditions (old vs. new) in the latency range from 0 to 500 ms after stimulus onset. Addition- ally, a difference plot of old minus new and PC minus VR is depicted. Black dots mark the electrodes which were included in the averaged amplitude these events are potentially significant for the participant, the VR group reported higher sensations of presence as com- consciously experienced and thus, might contribute to the pared to the PC group, we did not observe superior memory formation of autobiographical memory. recall performance. Our results, with both groups having However, at odds with previous research (Schöne et al., an accuracy of ca. 90%, indicate a ceiling effect, limiting 2019; Smith, 2019; Kisker et al., 2019b), our study did not the detection of group differences (Bortz & Döring, 2005). provide any behavioral evidence for this effect: Even though A possible cause of is effect might be the short retention 1 3 Psychological Research (2021) 85:2485–2501 2497 interval between encoding and retrieval. Previous stud- higher theta-band amplitudes to the retrieval of old, and ies, which did not apply EEG measurements, chose longer relatively lower amplitudes to the retrieval of new pictures retention intervals that included one or two sleeping peri- in conventional laboratory settings (e.g. Gruber et al., 2008; ods (Schöne et al., 2019; Kisker et al., 2019b). It is pos- Klimesch et al., 1997a, b, 2001a, b). The change of modal- sible that the process of forgetting irrelevant information ity, i.e. encoding videos, but retrieval in response to picture had not yet started at the time of the EEG measurement or presentation, did not markedly affect the theta old/new effect had at least not progressed very far (cf. Wang, Subagdja, in the PC condition. Tan & Starzyk, 2012). However, other studies have not been Remarkably, the theta old/new could not be observed able to demonstrate this overall memory superiority of VR in the VR condition. Specifically, new pictures led to the experiences either (LaFortune & Macuga, 2018; Dehn et al., same theta-band response in both groups, indicating that 2018; Kisker et al., 2019b). Differences regarding the find- the physical discrepancies between encoding in VR or under ings of VR studies might be related to varying implemen- conventional conditions did not affect the paradigm per se tations of VR technology, ranging from highly immersive or at least affected it to the same extent. Moreover, memory head-mounted displays and CAVE systems to less immersive success did not account for the different electrophysiological desktop-VR implementations (Smith, 2019). Additionally, responses as well, as both groups performed equally well in the level of multi-sensory sensations provided by the VR the recognition test. Accordingly, differences in the electro- system might influence memory performance as well: For physiological response must result from different underly - example, active navigation through a VR environment can ing retrieval mechanisms and thus, differences in mnemonic have an additional positive effect on spatial memory, but not processing of engrams encoded from either VR experiences necessarily on factual memory (Plancher, Barra, Orriols & or conventional laboratory events. Evidence that the absence Piolino, 2013). Moreover, some studies report a successful of the theta old/new ee ff ct under VR conditions results from transfer of content learned in an immersive VR environ- an altered mnemonic processing style as compared to the PC ment to real-life, and thus, to other than the encoding context condition is obtained from the comparison of the response (Ragan, Sowndararajan, Kopek & Bowman, 2010; as cited to old pictures between both groups. Regarding the 2–4 Hz in Smith, 2019), whereas other studies claim that knowledge frequency range, the presentation of old pictures led to a transfer comes with a loss of performance (Lanen & Lam- significant difference between relative synchronization in the ers, 2018). PC group and in the VR group. Descriptively, the 4–7 Hz Even though VR experiences do not necessarily increase frequency range follows the same trend but did not reach the retrieval success as measured by subjective reports, the significance. Hence, the theta old/new effect is modulated immersive nature of VR yet might alter the mode of opera- by the nature of the engram resulting from VR experiences tion of the mnemonic mechanisms. Specifically, Kisker and how these experiences are recalled. et  al., (2019a) demonstrated by means of a remember/ As aforementioned, immersive VR experiences are con- know paradigm that participants who explored a virtual vil- sidered to facilitate the formation of autobiographical mem- lage in an immersive VR condition report predominantly ory. Associative autobiographical engrams are generated by recollection-based memory. Interestingly, recollection is highly self-relevant experiences (Roediger & Marsh, 2003; hypothesized to be the associated retrieval mechanism of Conway, 2005). They are characterized by richer content autobiographical memory (Roediger & Marsh, 2003; Con- and are deeply interwoven into existing memory structures way, 2005). Participants exploring the very same village in (McDermott et al., 2009; Roediger & Marsh, 2003). Further- a PC condition reported predominantly familiarity-based more, they come with a broad set of functional properties, memories (Kisker et al., 2019a). However, both groups in namely self-reflection, emotional evaluation and semantic our experiment apparently employed the same retrieval strat- processes (Svoboda et al. 2006). Frontal-midline theta has egies as the d′-scores for recollection, familiarity and overall repeatedly been shown to reflect key-elements of autobio- performance do not differ significantly. graphical mnemonic processing. Specifically, it is associ - Nevertheless, modulations of the frontal-midline theta ated with the recollection of personal events and contextual effect might still indicate the involvement of different information (Guderian & Düzel, 2005; Hsieh & Ranganath, types of memory systems as well as associated encoding 2014; see also Roediger & Marsh, 2003; Conway, 2005). and retrieval strategies with respect to the encoding condi- In line with previous studies, our results indicate that the tion. As expected, we replicated the frontal-midline theta retrieval of immersive 3D-360° experiences differs from the old/new effect in the PC condition: Old pictures evoked retrieval of conventional 2D laboratory events (Schöne et al. an early theta-band synchronization, whereas new pictures 2016; Schöne et al. 2019; Kisker et al., 2019b). Hence, the resulted in theta-band desynchronization. Hence, our find- well-established theta old/new effect does not seem to be ings replicate broad and stable evidence relating relatively unrestrictedly applicable to VR experiences. It might rather 1 3 2498 Psychological Research (2021) 85:2485–2501 serve as an index for cue-matching of previously exog- has to be suppressed, while mental representation and cue enously processed pictorial stimuli: Experiences encoded are matched (Norman & Bobrow, 1979; Conway, 1996; Bur- in the laboratory are recalled and visually matched to the gess & Shallice, 1996). test stimuli, but are not inevitably associated with the vivid This interpretation of a visually guided matching process and multimodal character of autobiographical memories and gains further support from the difference in posterior alpha thus, might not provide a holistic representation of real-life oscillations, associated with visual processing (e.g. Clay- mnemonic processing. ton et al., 2018). Matching mental representation and cue The question remains, which processes change their mode is reflected by a generally reduced posterior alpha ampli - of operation in response to the recall of VR experiences. tude in the PC condition compared to the VR condition. The theta old/new effect is predominantly associated with This reduced alpha amplitude, commonly regarded as corti- retrieval success (e.g. Nyhus & Curran, 2010). However, the cal activity (e.g. Berger, 1929 as cited in Klimesch et al., VR and the PC group were likewise successful in the rec- 1997b), on the one hand reflects elevated attention (e.g. ognition task. As above mentioned, frontal-midline theta is Klimesch, et al. 1997a; Fries, Womelsdorf, Oostenveld & associated with autobiographical mnemonic processing, but Desimone, 2008) and, on the other hand, successful suppres- also regarded as an index for top-down control of memory sion of irrelevant information (Sauseng et al., 2009; Jensen retrieval (Klimesch et al., 1997b; Nyhus & Curran, 2010). & Mazaheri, 2010). Especially, the co-occurrence of higher Specifically, early theta-band increases indicate an attempt or frontal theta responses and posterior alpha activity has been the effort demands to retrieve engrams rather than success- interpreted as a response to higher cognitive load, with 2D ful retrieval per se (Klimesch et al., 2001a; Nyhus & Cur- environments exhibiting higher cognitive load as compared ran, 2010). Several studies investigating memory retrieval in to 3D environments (Dan & Reiner, 2017). Theta and alpha general as well as the classical old/new effect in particular, oscillations thus provide evidence for effortless and direct explicitly differentiate retrieval effort and retrieval success retrieval of immersive VR experience and a, in comparison, (Klimesch et al., 2001a; Nyhus & Curran, 2010; Rugg et al., effortful and strategic retrieval of conventionally presented 1998; Konishi, Wheeler, Donaldson & Buckner, 2000). In stimuli. particular, processes exclusively associated with retrieval Nevertheless, the finding that the retrieval mechanisms success are engaged only if an attempted retrieval is suc- underlying VR experiences and conventional laboratory cessful. In contrast, retrieval effort refers to those processes experiences differ, does not invalidate previous well-estab- engaged during a retrieval attempt per se, for example in lished knowledge gained from conventional setups. Rather, recognition tasks, regardless of whether this attempt is suc- it complements the immense insights from previous studies cessful or not (Rugg, Fletcher, Frith, Frackowiak & Dolan, and demonstrates the delicate balance between high experi- 1996). Accordingly, the absence of a difference in memory mental control and ecological validity. Thus, controlled success does not rule out that the effort required to achieve laboratory studies provide the foundations for understand- the very same retrieval outcome may vary. ing the complex mechanisms of human memory and are Hence, the difference in the theta-band response to old substantial for developing models. As a further refinement pictures between the VR condition and the PC condition of these foundations, VR settings facilitate the transfer of could reflect the two types of retrieval differing with respect experimental findings to everyday life and thus improve their to their effort demands (Conway, 1996; Haque & Conway, generalizability and practicability. 2001; Conway & Pleydell-Pearce, 2000). Immersive VR experiences as part of an extensive autobiographical asso- ciative network (PBM, Schöne et al., 2019) can be effortless Conclusions and, most of all, directly retrieved. In contrast, the retrieval of conventional stimuli triggers the iterative verification pro- As a conclusion, we replicated the well-established theta cess and the suppression of irrelevant information, thus com- old/new effect in a conventional laboratory setting, mani- ing in with higher effort to recall memories. Direct retrieval fested in relative theta-band synchronization for old, and of autobiographical memory is based upon a pronounced and relative desynchronization for new stimuli. However, this stable memory pattern (Conway & Pleydell-Pearce, 2000) effect could not be replicated for the immersive VR con- and enables spontaneous recall, which is rather automatic dition: Theta-band responses were equal for old and new and effortless (Conway & Pleydell-Pearce, 2000 as cited in stimuli. Hence, the canonical theta old/new effect might not Willander & Larsson, 2007). It thus allows immediate recall be unrestrictedly applicable to VR experiences and thus, of a cued memory. Generative or strategic retrieval of con- might not provide a holistic representation of real-life pro- ventional stimuli, as observed in the PC condition, relies cesses. Accompanied by higher alpha activity as compared on central control of memory recall (Willander & Larsson, to the VR condition, the theta-band synchronization in the 2007). To verify the cued memory, irrelevant information PC condition might rather reflect higher retrieval effort than 1 3 Psychological Research (2021) 85:2485–2501 2499 otherwise in a credit line to the material. If material is not included in retrieval success per se. In contrast to laboratory events, the article’s Creative Commons licence and your intended use is not memories obtained from VR experiences are spontaneous permitted by statutory regulation or exceeds the permitted use, you will and effortless retrieved. Additionally, participants of the VR need to obtain permission directly from the copyright holder. To view a condition reported a higher sense of presence, which might copy of this licence, visit http://creativ ecommons .or g/licenses/b y/4.0/. enhance the self-relevance of the VR experiences. Crucially, self-referential processing and a facile, effortless recall are characteristic of autobiographical memory. Therefore, the References effortless recall of VR experiences might approximate real-life memory more closely as compared to memories Alshaer, A., Regenbrecht, H., & O’Hare, D. (2017). Immersion fac- obtained from the laboratory. However, the VR group did tors affecting perception and behaviour in a virtual reality power not perform better in the memory test, as former research wheelchair simulator. Applied Ergonomics, 58, 1–12. https ://doi. org/10.1016/j.aperg o.2016.05.003. suggested. Hence, our results suggest that the memory pro- Atkinson, R. C., & Juola, J. F. (1973). Factors influencing speed and cesses underlying VR experiences are qualitatively different accuracy of word recognition. Attention and Performance, IV, from conventional laboratory experiences, but under which 583–612. conditions VR leads not only to altered mechanisms but also Berger, H. (1929). Über das Elektroenkephalogramm des Menschen. Archiv für Psychiatrie und Nervenkrankheiten, 87(1), 527–570. to a better memory performance compared to conventional Bertrand, O., & Pantev, C. (1994). Stimulus frequency dependence of settings should be the subject of further research. the transient oscillatory auditory evoked responses (40 Hz) stud- ied by electric and magnetic recordings in human. In: Oscillatory event-related brain dynamics (pp. 231–242). Boston: Springer. Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neu- Author contributions All authors contributed to the study design. Test- roscience research and therapy. Nature reviews neuroscience, ing and data collection were performed by JK. Data analyses were 12(12), 752–762. performed by JK under the supervision of BS and TG. JK and BS per- Bortz & Döring (2005). Forschungsmethoden und Evaluation (S. 182). formed the data interpretation under the supervision of TG. JK drafted Heidelberg: Springer-Verlag. ISBN 3-540-41940-3. the manuscript, and BS and TG provided critical revisions. All authors Bouchard, S., Talbot, J., Ledoux, A. A., Phillips, J., Cantamasse, M., approved the final version of the manuscript for submission. & Robillard, G. (2009). The meaning of being there is related to a specific activation in the brain located in the parahypocampus. Funding Open Access funding provided by Projekt DEAL. This In: Proceedings of Presence. research did not receive any specific grant from funding agencies in Burgess, A. P., & Gruzelier, J. H. (1997). Short duration synchroniza- the public, commercial, or not-for-profit sectors. tion of human theta rhythm during recognition memory. NeuroRe- port, 8(4), 1039–1042. Availability of data, material and code The datasets generated and Burgess, N., Maguire, E. A., Spiers, H. J., & O’Keefe, J. (2001). A analyzed during the current study are available in the Open Science temporoparietal and prefrontal network for retrieving the spatial Framework (OSF) repository, https://osf.io/q924w /?vie w_only=cc107 context of lifelike events. Neuroimage, 14(2), 439–453. f7a927f472 e8e68 e85aa a059e 97 . Stimulus material was obtained from Burgess, P. W., & Shallice, T. (1996). Response suppression, initiation the Library for Universal Virtual Reality Experiments (luVRe, https and strategy use following frontal lobe lesions. Neuropsychologia, ://www.psycho.uni-osnab r ueck.de/f achgebie te/allg emeine _psy chologi 34(4), 263–272. e_i/luvre .html) and can be accessed upon request. Busch, N. A., Herrmann, C. S., Müller, M. M., Lenz, D., & Gruber, T. (2006). A cross-laboratory study of event-related gamma activity in a standard object recognition paradigm. Neuroimage, 33(4), Compliance with ethical standards 1169–1177. Cabeza, R., Prince, S. E., Daselaar, S. M., Greenberg, D. L., Budde, M., Conflict of interest All authors declare that they have no conflict of Dolcos, F., & Rubin, D. C. (2004). Brain activity during episodic interest. retrieval of autobiographical and laboratory events: an fMRI study using a novel photo paradigm. Journal of Cognitive Neuroscience, Statement of ethical approval All procedures performed in this study 16(9), 1583–1594. involving human participants were in accordance with the ethical stand- Cabeza, R., & St Jacques, P. (2007). Functional neuroimaging of ards of the local ethic committee of Osnabrueck University and with autobiographical memory. Trends in Cognitive Sciences, 11(5), the 1964 Helsinki Declaration. 219–227. Clayton, M. S., Yeung, N., & Cohen Kadosh, R. (2018). The many Consent to participate and to publish All individual participants gave characters of visual alpha oscillations. European Journal of Neu- informed written consent to participate in the study and to anonymized roscience, 48(7), 2498–2508. publication of the datasets. Conway, M. A. (1996). Autobiographical knowledge and autobiograph- ical memories. In D. C. Rubin (Ed.), Remembering our past: Stud- ies in autobiographical memory (p. 67–93). Cambridge University Open Access This article is licensed under a Creative Commons Attri- Press. https ://doi.org/10.1017/CBO97 80511 52791 3.003. bution 4.0 International License, which permits use, sharing, adapta- Conway, M. A. (2005). Memory and the self. Journal of Memory and tion, distribution and reproduction in any medium or format, as long Language, 53(4), 594–628. as you give appropriate credit to the original author(s) and the source, Conway, M. A., & Pleydell-Pearce, C. W. (2000). The construction of provide a link to the Creative Commons licence, and indicate if changes autobiographical memories in the self-memory system. Psycho- were made. The images or other third party material in this article are logical Review, 107(2), 261. included in the article’s Creative Commons licence, unless indicated 1 3 2500 Psychological Research (2021) 85:2485–2501 Curran, T., & Hancock, J. (2007). The FN400 indexes familiarity-based Hsieh, L.-T., & Ranganath, C. (2014). Frontal midline theta oscilla- recognition of faces. Neuroimage, 36(2), 464–471. tions during working memory maintenance and episodic encoding Dan, A., & Reiner, M. (2017). EEG-based cognitive load of processing and retrieval. Neuroimage. https ://doi.org/10.1016/j.neuro image events in 3D virtual worlds is lower than processing events in 2D .2013.08.003. displays. International Journal of Psychophysiology, 122, 75–84. Jacobs, J., Hwang, G., Curran, T., & Kahana, M. J. (2006). EEG oscil- Dehn, L. B., Kater, L., Piefke, M., Botsch, M., Driessen, M., & Beblo, lations and recognition memory: Theta correlates of memory T. (2018). Training in a comprehensive everyday-like virtual retrieval and decision making. Neuroimage, 32(2), 978–987. reality environment compared to computerized cognitive train- Jacoby, L. L., & Dallas, M. (1981). On the relationship between auto- ing for patients with depression. Computers in Human Behavior, biographical memory and perceptual learning. Journal of Experi- 79, 40–52. mental Psychology: General, 110(3), 306. Diemer, J., Alpers, G. W., Peperkorn, H. M., Shiban, Y., & Mühlberger, Jensen, O., & Mazaheri, A. (2010). Shaping functional architecture A. (2015). The impact of perception and presence on emotional by oscillatory alpha activity: gating by inhibition. Frontiers in reactions: a review of research in virtual reality. Frontiers in Psy- Human Neuroscience, 4, 186. chology, 6, 26. Jensen, O., & Tesche, C. D. (2002). Frontal theta activity in humans Eckhorn, R., Reitboeck, H. J., Arndt, M. T., & Dicke, P. (1990). Fea- increases with memory load in a working memory task. European ture linking via synchronization among distributed assemblies: Journal of Neuroscience, 15(8), 1395–1399. Simulations of results from cat visual cortex. Neural Computation, Jones, T. C., & Jacoby, L. L. (2001). Feature and conjunction errors in 2(3), 293–307. recognition memory: Evidence for dual-process theory. Journal Ernstsen, J., Mallam, S. C., & Nazir, S. (2019). incidental memory of Memory and Language, 45(1), 82–102. recall in virtual reality: An empirical investigation. In: Proceed- Junghöfer, M., Elbert, T., Tucker, D. M., & Rockstroh, B. (2000). ings of the Human Factors and Ergonomics Society Annual Meet- Statistical control of artifacts in dense array EEG/MEG studies. ing (Vol. 63, No. 1, pp. 2277-2281). Sage CA: SAGE Publications. Psychophysiology, 37(4), 523–532. Felnhofer, A., Kothgassner, O. D., Schmidt, M., Heinzle, A. K., Beutl, Kisker, J., Gruber, T. & Schöne, B. (2019a). Behavioral realism and L., Hlavacs, H., & Kryspin-Exner, I. (2015). Is virtual reality lifelike psychophysiological responses in virtual reality by the emotionally arousing? Investigating five emotion inducing vir - example of a height exposure. Psychological Research. https :// tual park scenarios. International Journal of Human-Computer doi.org/10.1007/s0042 6-019-01244 -9. Studies, 82, 48–56. Kisker, J., Gruber, T. & Schöne, B. (2019b). Experiences in virtual Fries, P., Womelsdorf, T., Oostenveld, R., & Desimone, R. (2008). reality entail different processes of retrieval as opposed to con- The effects of visual stimulation and selective visual attention on ventional laboratory settings: A study on human memory. Curr rhythmic neuronal synchronization in macaque area V4. Journal Psychol. https ://doi.org/10.1007/s1214 4-019-00257 -2 of Neuroscience, 28, 4823–4835. Klimesch, W., Doppelmayr, M., Pachinger, T., & Ripper, B. (1997a). Gorini, A., Pallavicini, F., Algeri, D., Repetto, C., Gaggioli, A., & Brain oscillations and human memory: EEG correlates in the Riva, G. (2010). Virtual reality in the treatment of generalized upper alpha and theta band. Neuroscience Letters, 238(1–2), 9–12. anxiety disorders. Studies in Health Technology and Informatics, Klimesch, W., Doppelmayr, M., Schimke, H., & Ripper, B. (1997b). 154, 39–43. Theta synchronization and alpha desynchronization in a memory Greenberg, D. L., & Rubin, D. C. (2003). The neuropsychology of task. Psychophysiology, 34(2), 169–176. autobiographical memory. Cortex, 39(4–5), 687–728. Klimesch, W., Doppelmayr, M., Schwaiger, J., Winkler, T., & Gruber, Gromer, D., Reinke, M., Christner, I., & Pauli, P. (2019). Causal W. (2000). Theta oscillations and the ERP old/new effect: inde- interactive links between presence and fear in virtual reality pendent phenomena? Clinical Neurophysiology, 111(5), 781–793. height exposure. Frontiers in Psychology, 10, 141. https ://doi. Klimesch, W., Doppelmayr, M., Stadler, W., Pöllhuber, D., Sauseng, P., org/10.3389/fpsyg .2019.00141 . & Röhm, D. (2001a). Episodic retrieval is reflected by a process Gruber, T., Tsivilis, D., Giabbiconi, C. M., & Müller, M. M. (2008). specific increase in human electroencephalographic theta activity. Induced electroencephalogram oscillations during source mem- Neuroscience Letters, 302(1), 49–52. ory: familiarity is reflected in the gamma band, recollection in the Klimesch, W., Doppelmayr, M., Yonelinas, A., Kroll, N. E., Lazzara, theta band. Journal of cognitive neuroscience, 20(6), 1043–1053. M., Roehm, D., & Gruber, W. (2001b). Theta synchronization dur- Guderian, S., & Düzel, E. (2005). Induced theta oscillations mediate ing episodic retrieval: neural correlates of conscious awareness. large-scale synchrony with mediotemporal areas during recol- Cognitive Brain Research, 12(1), 33–38. lection in humans. Hippocampus, 15(7), 901–912. https ://doi. Klimesch, W., Schimke, H., & Schwaiger, J. (1994). Episodic and org/10.1002/hipo.20125. semantic memory: an analysis in the EEG theta and alpha band. Haatveit, B. C., Sundet, K., Hugdahl, K., Ueland, T., Melle, I., & Electroencephalography and Clinical Neurophysiology, 91(6), Andreassen, O. A. (2010). The validity of d prime as a working 428–441. memory index: results from the “Bergen n-back task”. Journal Konishi, S., Wheeler, M. E., Donaldson, D. I., & Buckner, R. L. (2000). of Clinical and Experimental Neuropsychology, 32(8), 871–880. Neural correlates of episodic retrieval success. Neuroimage, 12(3), Haque, S., & Conway, M. A. (2001). Sampling the process of autobio- 276–286. graphical memory construction. European Journal of Cognitive Krijn, M., Emmelkamp, P. M., Olafsson, R. P., & Biemond, R. (2004). Psychology, 13(4), 529–547. Virtual reality exposure therapy of anxiety disorders: A review. Harman, J., Brown, R., & Johnson, D. (2017). Improved memory elici- Clinical Psychology Review, 24(3), 259–281. tation in virtual reality: new experimental results and insights. In: Krokos, E., Plaisant, C., & Varshney, A. (2019). Virtual memory pal- IFIP Conference on human-computer interaction (pp. 128–146). aces: immersion aids recall. Virtual Reality, 23(1), 1–15. Springer, Cham. Kvavilashvili, L., & Ellis, J. (2004). Ecological validity and twenty Higuera-Trujillo, J. L., López-Tarruella Maldonado, J., & Llinares Mil- years of real-life/laboratory controversy in memory research: A lán, C. (2017). Psychological and physiological human responses critical (and historical) review. History and Philosophy of Psy- to simulated and real environments: A comparison between photo- chology, 9.   graphs, 360° panoramas, and virtual reality. Applied Ergonomics, LaFortune, J., & Macuga, K. L. (2018). Learning movements from a 65, 398–409. https ://doi.org/10.1016/j.aperg o.2017.05.006. virtual instructor: Effects of spatial orientation, immersion, and 1 3 Psychological Research (2021) 85:2485–2501 2501 expertise. Journal of Experimental Psychology: Applied, 24(4), Rugg, M. D., Fletcher, P. C., Allan, K., Frith, C. D., Frackowiak, R. S. 521. J., & Dolan, R. J. (1998). Neural correlates of memory retrieval Lanen, M., & Lamers, M. H. (2018). Context-dependent memory in during recognition memory and cued recall. Neuroimage, 8(3), real and virtual reality. In: International Conference on virtual 262–273. reality and augmented reality (pp. 177–189). Cham: Springer. Rugg, M. D., Fletcher, P. C., Frith, C. D., Frackowiak, R. S. J., & Lovett, A., Appleton, K., Warren-Kretzschmar, B., & Von Haaren, C. Dolan, R. J. (1996). Differential activation of the prefrontal cortex (2015). Using 3D visualization methods in landscape planning: in successful and unsuccessful memory retrieval. Brain, 119(6), An evaluation of options and practical issues. Landscape and 2073–2083. Urban Planning, 142, 85–94. Sauseng, P., Klimesch, W., Heise, K. F., Gruber, W. R., Holz, E., Makowski, D., Sperduti, M., Nicolas, S., & Piolino, P. (2017). “Being Karim, A. A., & Hummel, F. C. (2009). Brain oscillatory sub- there” and remembering it: Presence improves memory encoding. strates of visual short-term memory capacity. Current Biology, Consciousness and cognition, 53, 194–202. 19(21), 1846–1852. McDermott, K. B., Szpunar, K. K., & Christ, S. E. (2009). Laboratory- Schöne, B., Schomberg, J., Gruber, T., & Quirin, M. (2016). Event- based and autobiographical retrieval tasks differ substantially in related frontal alpha asymmetries: electrophysiological correlates their neural substrates. Neuropsychologia, 47(11), 2290–2298. of approach motivation. Experimental brain research, 234(2), Moscovitch, M., & Winocur, G. (2002). The frontal cortex and work- 559–567. ing with memory. Principles of frontal lobe function, 188, 209. Schöne, B., Wessels, M., & Gruber, T. (2017). Differences between Nilsson, N. C., Nordahl, R., & Serafin, S. (2016). Immersion revisited: real-life and laboratory memory: Evidence from EEG and virtual a review of existing definitions of immersion and their relation to reality. In Psychophysiology (Vol. 54, pp. S128–S128). 111 River different theories of presence. Human Technology, 12(2). ST, Hoboken 07030-5774, NJ USA: Wiley. Norman, D. A., & Bobrow, D. G. (1979). Descriptions: An interme- Schöne, B., Wessels, M., & Gruber, T. (2019). Experiences in virtual diate stage in memory retrieval. Cognitive Psychology, 11(1), reality: A window to autobiographical memory. Current Psychol- 107–123. ogy, 38(3), 715–719. Nyhus, E., & Curran, T. (2010). Functional role of gamma and theta Schöne, B., Kisker, J., Sylvester, R.S., Radtke, E.L., Gruber, T. (2020). oscillations in episodic memory. Neuroscience and Biobehavio- Library for Universal Virtual Reality Experiments (luVRe): A ral Reviews, 34(7), 1023–1035. https ://doi.org/10.1016/j.neubi standardized immersive 3D/360° picture and video database for orev.2009.12.014. VR based research. under review Pan, X., & Hamilton, A. F. D. C. (2018). Why and how to use vir- Schubert, T., Friedmann, F., & Regenbrecht, H. (2001). The experience tual reality to study human social interaction: The challenges of of presence: Factor analytic insights. Presence: Teleoperators & exploring a new research landscape. British Journal of Psychol- Virtual Environments, 10(3), 266–281. ogy, 109(3), 395–417. Serino, S., & Repetto, C. (2018). New trends in episodic memory Parsons, T. D. (2015). Virtual reality for enhanced ecological validity assessment: immersive 360 ecological videos. Frontiers in Psy- and experimental control in the clinical, affective and social neu- chology, 9, 1878. rosciences. Frontiers in Human Neuroscience, 9, 660. Slater, M., & Wilbur, S. (1997). A framework for immersive virtual Plancher, G., Barra, J., Orriols, E., & Piolino, P. (2013). The influence environments (FIVE): Speculations on the role of presence in vir- of action on episodic memory: a virtual reality study. Quarterly tual environments. Presence: Teleoperators & Virtual Environ- Journal of Experimental Psychology, 66(5), 895–909. ments, 6(6), 603–616. Plancher, G., & Piolino, P. (2017). Virtual reality for assessment of Slobounov, S. M., Ray, W., Johnson, B., Slobounov, E., & Newell, episodic memory in normal and pathological aging. The role of K. M. (2015). Modulation of cortical activity in 2D versus 3D technology in clinical neuropsychology, pp. 237–260. virtual reality environments: an EEG study. International Journal Ragan, E. D., Sowndararajan, A., Kopper, R., & Bowman, D. A. of Psychophysiology, 95(3), 254–260. (2010). The effects of higher levels of immersion on procedure Smith, S. A. (2019). Virtual reality in episodic memory research: A memorization performance and implications for educational vir- review. Psychonomic Bulletin & Review, 26(4), 1213–1237. tual environments. Presence: Teleoperators and Virtual Environ- Svoboda, E., McKinnon, M. C., & Levine, B. (2006). The functional ments, 19(6), 527–543. neuroanatomy of autobiographical memory: A meta-analysis. Rauchs, G., Orban, P., Balteau, E., Schmidt, C., Degueldre, C., Luxen, Neuropsychologia, 44(12), 2189–2208. A., & Peigneux, P. (2008). Partially segregated neural networks Swets, J. A., Tanner, W. P., Jr., & Birdsall, T. G. (1961). Decision pro- for spatial and contextual memory in virtual navigation. Hip- cesses in perception. Psychological Review, 68(5), 301. pocampus, 18(5), 503–518. Tallon-Baudry, C., & Bertrand, O. (1999). Oscillatory gamma activity Riva, G., Mantovani, F., Capideville, C. S., Preziosa, A., Morganti, in humans and its role in object representation. Trends in Cogni- F., Villani, D., & Alcañiz, M. (2007). Affective Interactions tive Sciences, 3(4), 151–162. Using Virtual Reality: The Link between Presence and Emo- Tulving, E. (1983). Ecphoric processes in episodic memory. Philo- tions. CyberPsychology & Behavior, 10(1), 45–56. https ://doi. sophical Transactions of the Royal Society of London B Biological org/10.1089/cpb.2006.9993. Sciences, 302(1110), 361–371. Roediger, H. L. III, & Marsh, E. J. (2003). Episodic and autobiographi- Wang, W., Subagdja, B., Tan, A. H., & Starzyk, J. A. (2012). Neural cal memory. In A. F. Healy & R. W. Proctor (Eds.), Handbook of modeling of episodic memory: Encoding, retrieval, and forgetting. psychology: Experimental psychology, Vol. 4 (p. 475–497). John IEEE Transactions on Neural Networks and Learning Systems, Wiley & Sons Inc. 23(10), 1574–1586. Rubin, D. C., Schrauf, R. W., & Greenberg, D. L. (2003). Belief and Willander, J., & Larsson, M. (2007). Olfaction and emotion: The recollection of autobiographical memories. Memory & Cognition, case of autobiographical memory. Memory & Cognition, 35(7), 31(6), 887–901. 1659–1663. Rubin, D. C., & Umanath, S. (2015). Event memory: A theory of memory for laboratory, autobiographical, and fictional events. Publisher’s Note Springer Nature remains neutral with regard to Psychological Review, 122(1), 1. jurisdictional claims in published maps and institutional affiliations. Rugg, M. D., & Curran, T. (2007). Event-related potentials and rec- ognition memory. Trends in cognitive sciences, 11(6), 251–257. 1 3

Journal

Psychological Research – Springer Journals

Published: Oct 1, 2021

There are no references for this article.