Planning for the Future of Psychological Research on Aging

Planning for the Future of Psychological Research on Aging The first manuscript I ever submitted was to Journal of Gerontology: Psychological Sciences (but do not look for it on my CV—let us just say it did not ever get published at the journal). I still remember trying to staple all the copies together—this was still during the era of hard-copy manuscript submission. What I remember just as well is that there was no discussion with my advisor/co-author about where we would submit the manuscript first: we had a finding we were excited about, and we sent it to JG:PS. Fast forward more years than you might guess, and now I am extremely honored to have the opportunity to serve as editor-in-chief of the journal. I am excited to keep up the amazing momentum coming from Bob Knight’s time as editor. I am very grateful to have the help of four Associate Editors: Nicole Anderson, Angela Gutchess, Lynn Martire, and Shevaun Neupert, as well as Managing Editor Kathy Jackson. I am also cognizant that this is a moment of great introspection, and some critical changes in practices, in the field of psychological science. Although psychologists interested in aging and gerontology have not typically been in the forefront of recent discussions about research practices, I believe it is time for us as a field to think seriously and strategically about how we will align our practices with changing norms in the field more generally. I also feel that JG:PS is the ideal venue for open discussions about how we as a field want to move forward. And I am serious about discussion; my agenda is to push ahead scientific practices in our field rather than to necessarily adopt a specific set of rules. However, I do want to raise a number of starting points for these conversations. One discussion point involves null age effects: I think every reader of the journal has had the experience of doing a well-designed study, analyzing the data, and finding no age differences—then thinking “what a waste! I’ll never be able to publish this study anywhere good!” I am sure we have also all had the experience of trying to submit a manuscript like this to a top journal and having it rejected due to “no interesting age differences.” I have certainly had these experiences myself. I even had an editor-in-chief of a non-aging journal instruct me to only include manuscripts with significant age differences in a special section on aging. We are all aware that null age differences in a single study may not be very informative—but there are also likely processes that we are interested in as psychologists that do not vary by age. How can we make sure that studies that find informative null age effects can be included in the literature? First, our expectations about sample sizes to test for age differences need to be larger. We have grown familiar with rules of thumb for numbers of participants per age group, and those may be appropriate for studies investigating phenomena with large effect sizes, but are likely too small for the types of phenomena most psychologists study. In order to make sure we are being thoughtful about our sample sizes, all manuscripts should explicitly justify how they determine the sample size (i.e., by reporting a power analysis). This is a change I believe we can implement right away. Second, when we find null age effects in adequately-powered studies, we should be utilizing newly developed tools for quantifying the strength of evidence that there is actually no age difference. Methods like Two One-Sided Tests (TOST: Lakens, 2017) and Bayes Factors (Dienes, 2014) should be in the statistical toolbox of aging researchers to help determine when a null age effect is meaningful and when it is ambiguous—and potentially meaningful null age effects should be fair game for the journal. My hope is that making meaningful null age effects publishable will also reduce the temptation to engage in so-called Questionable Research Practices (QRPs) like p-hacking to try to ensure a p <.05 age difference in order to get a manuscript published. Using these techniques will not be required to submit, but instead can be an important tool to make stronger arguments about some null age effects. Look out for tutorials and examples of studies using these methods in upcoming issues of the journal. Another discussion point involves preregistration and replication. This is a standard that will be increasingly common in the coming years (e.g., expectations for Rigor and Reproducibility at NIH). So much of our knowledge as a field relies on the incredible large survey data sets available to us, such as HRS and MIDUS. Although the sample sizes of these studies are set and thus cannot be “justified” as in a lab study, researchers using these data sets can still take steps to ensure healthy research practices. For example, power analyses can be reported to demonstrate that the available sample size is adequate for testing the main hypotheses of interest. Although it would be unrealistic to expect investigators to collect an independent data set for replication, evaluating the robustness of the research findings in similar samples or independent samples drawn from the original data may give us more confidence in the results. A more challenging issue for us as a field to consider is how to reduce the possible influence of so-called Hypothesizing After Results are Known (HARKing) in secondary analyses. It is relatively easy for researchers working with existing data sets to check out different combinations of variables and see what leads to significant age (or other) effects. If our interest is in knowing what is true about age similarities and differences, rather than simply publishing significant age effects, then we need to know that studies were designed to ask the right questions using the right methods, rather than only evaluating the results. Thus, for studies using existing data sets, preregistration may offer a way to increase confidence in the robustness of the methods. Preregistration has already caught on in other disciplines that use existing data sets, such as personality psychology. Preregistration can be done easily using a site such as Open Science Framework (https://osf.io/)—this allows a time-stamped posting of preregistered hypotheses and analysis plans. Although this will not be a journal requirement, it is an option I believe journal authors should strongly consider. As a flagship journal for psychological aging, I believe JG:PS is the right venue for our field to have dialogue about research practices and to make changes that are right for us, that can also align us with psychology more generally. I hope my editorial will jumpstart this discussion, and I am excited to work with all of you to help craft the future of our field. References Dienes, Z. ( 2014). Using Bayes to get the most out of non-significant results. Frontiers in Psychology , 5, 781. doi: 10.3389/fpsyg.2014.00781 Google Scholar CrossRef Search ADS PubMed  Lakens, D. ( 2017). Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science , 8, 355– 362. doi: 10.1177/1948550617697177 Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The Journals of Gerontology Series B: Psychological Sciences and Social Sciences Oxford University Press

Planning for the Future of Psychological Research on Aging

Loading next page...
 
/lp/ou_press/planning-for-the-future-of-psychological-research-on-aging-myPYpUgtEp
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
ISSN
1079-5014
eISSN
1758-5368
D.O.I.
10.1093/geronb/gbx142
Publisher site
See Article on Publisher Site

Abstract

The first manuscript I ever submitted was to Journal of Gerontology: Psychological Sciences (but do not look for it on my CV—let us just say it did not ever get published at the journal). I still remember trying to staple all the copies together—this was still during the era of hard-copy manuscript submission. What I remember just as well is that there was no discussion with my advisor/co-author about where we would submit the manuscript first: we had a finding we were excited about, and we sent it to JG:PS. Fast forward more years than you might guess, and now I am extremely honored to have the opportunity to serve as editor-in-chief of the journal. I am excited to keep up the amazing momentum coming from Bob Knight’s time as editor. I am very grateful to have the help of four Associate Editors: Nicole Anderson, Angela Gutchess, Lynn Martire, and Shevaun Neupert, as well as Managing Editor Kathy Jackson. I am also cognizant that this is a moment of great introspection, and some critical changes in practices, in the field of psychological science. Although psychologists interested in aging and gerontology have not typically been in the forefront of recent discussions about research practices, I believe it is time for us as a field to think seriously and strategically about how we will align our practices with changing norms in the field more generally. I also feel that JG:PS is the ideal venue for open discussions about how we as a field want to move forward. And I am serious about discussion; my agenda is to push ahead scientific practices in our field rather than to necessarily adopt a specific set of rules. However, I do want to raise a number of starting points for these conversations. One discussion point involves null age effects: I think every reader of the journal has had the experience of doing a well-designed study, analyzing the data, and finding no age differences—then thinking “what a waste! I’ll never be able to publish this study anywhere good!” I am sure we have also all had the experience of trying to submit a manuscript like this to a top journal and having it rejected due to “no interesting age differences.” I have certainly had these experiences myself. I even had an editor-in-chief of a non-aging journal instruct me to only include manuscripts with significant age differences in a special section on aging. We are all aware that null age differences in a single study may not be very informative—but there are also likely processes that we are interested in as psychologists that do not vary by age. How can we make sure that studies that find informative null age effects can be included in the literature? First, our expectations about sample sizes to test for age differences need to be larger. We have grown familiar with rules of thumb for numbers of participants per age group, and those may be appropriate for studies investigating phenomena with large effect sizes, but are likely too small for the types of phenomena most psychologists study. In order to make sure we are being thoughtful about our sample sizes, all manuscripts should explicitly justify how they determine the sample size (i.e., by reporting a power analysis). This is a change I believe we can implement right away. Second, when we find null age effects in adequately-powered studies, we should be utilizing newly developed tools for quantifying the strength of evidence that there is actually no age difference. Methods like Two One-Sided Tests (TOST: Lakens, 2017) and Bayes Factors (Dienes, 2014) should be in the statistical toolbox of aging researchers to help determine when a null age effect is meaningful and when it is ambiguous—and potentially meaningful null age effects should be fair game for the journal. My hope is that making meaningful null age effects publishable will also reduce the temptation to engage in so-called Questionable Research Practices (QRPs) like p-hacking to try to ensure a p <.05 age difference in order to get a manuscript published. Using these techniques will not be required to submit, but instead can be an important tool to make stronger arguments about some null age effects. Look out for tutorials and examples of studies using these methods in upcoming issues of the journal. Another discussion point involves preregistration and replication. This is a standard that will be increasingly common in the coming years (e.g., expectations for Rigor and Reproducibility at NIH). So much of our knowledge as a field relies on the incredible large survey data sets available to us, such as HRS and MIDUS. Although the sample sizes of these studies are set and thus cannot be “justified” as in a lab study, researchers using these data sets can still take steps to ensure healthy research practices. For example, power analyses can be reported to demonstrate that the available sample size is adequate for testing the main hypotheses of interest. Although it would be unrealistic to expect investigators to collect an independent data set for replication, evaluating the robustness of the research findings in similar samples or independent samples drawn from the original data may give us more confidence in the results. A more challenging issue for us as a field to consider is how to reduce the possible influence of so-called Hypothesizing After Results are Known (HARKing) in secondary analyses. It is relatively easy for researchers working with existing data sets to check out different combinations of variables and see what leads to significant age (or other) effects. If our interest is in knowing what is true about age similarities and differences, rather than simply publishing significant age effects, then we need to know that studies were designed to ask the right questions using the right methods, rather than only evaluating the results. Thus, for studies using existing data sets, preregistration may offer a way to increase confidence in the robustness of the methods. Preregistration has already caught on in other disciplines that use existing data sets, such as personality psychology. Preregistration can be done easily using a site such as Open Science Framework (https://osf.io/)—this allows a time-stamped posting of preregistered hypotheses and analysis plans. Although this will not be a journal requirement, it is an option I believe journal authors should strongly consider. As a flagship journal for psychological aging, I believe JG:PS is the right venue for our field to have dialogue about research practices and to make changes that are right for us, that can also align us with psychology more generally. I hope my editorial will jumpstart this discussion, and I am excited to work with all of you to help craft the future of our field. References Dienes, Z. ( 2014). Using Bayes to get the most out of non-significant results. Frontiers in Psychology , 5, 781. doi: 10.3389/fpsyg.2014.00781 Google Scholar CrossRef Search ADS PubMed  Lakens, D. ( 2017). Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science , 8, 355– 362. doi: 10.1177/1948550617697177 Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Journal

The Journals of Gerontology Series B: Psychological Sciences and Social SciencesOxford University Press

Published: Mar 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off