Planning for the Future of Psychological Research on Aging

Planning for the Future of Psychological Research on Aging The first manuscript I ever submitted was to Journal of Gerontology: Psychological Sciences (but do not look for it on my CV—let us just say it did not ever get published at the journal). I still remember trying to staple all the copies together—this was still during the era of hard-copy manuscript submission. What I remember just as well is that there was no discussion with my advisor/co-author about where we would submit the manuscript first: we had a finding we were excited about, and we sent it to JG:PS. Fast forward more years than you might guess, and now I am extremely honored to have the opportunity to serve as editor-in-chief of the journal. I am excited to keep up the amazing momentum coming from Bob Knight’s time as editor. I am very grateful to have the help of four Associate Editors: Nicole Anderson, Angela Gutchess, Lynn Martire, and Shevaun Neupert, as well as Managing Editor Kathy Jackson. I am also cognizant that this is a moment of great introspection, and some critical changes in practices, in the field of psychological science. Although psychologists interested in aging and gerontology have not typically been in the forefront of recent discussions about research practices, I believe it is time for us as a field to think seriously and strategically about how we will align our practices with changing norms in the field more generally. I also feel that JG:PS is the ideal venue for open discussions about how we as a field want to move forward. And I am serious about discussion; my agenda is to push ahead scientific practices in our field rather than to necessarily adopt a specific set of rules. However, I do want to raise a number of starting points for these conversations. One discussion point involves null age effects: I think every reader of the journal has had the experience of doing a well-designed study, analyzing the data, and finding no age differences—then thinking “what a waste! I’ll never be able to publish this study anywhere good!” I am sure we have also all had the experience of trying to submit a manuscript like this to a top journal and having it rejected due to “no interesting age differences.” I have certainly had these experiences myself. I even had an editor-in-chief of a non-aging journal instruct me to only include manuscripts with significant age differences in a special section on aging. We are all aware that null age differences in a single study may not be very informative—but there are also likely processes that we are interested in as psychologists that do not vary by age. How can we make sure that studies that find informative null age effects can be included in the literature? First, our expectations about sample sizes to test for age differences need to be larger. We have grown familiar with rules of thumb for numbers of participants per age group, and those may be appropriate for studies investigating phenomena with large effect sizes, but are likely too small for the types of phenomena most psychologists study. In order to make sure we are being thoughtful about our sample sizes, all manuscripts should explicitly justify how they determine the sample size (i.e., by reporting a power analysis). This is a change I believe we can implement right away. Second, when we find null age effects in adequately-powered studies, we should be utilizing newly developed tools for quantifying the strength of evidence that there is actually no age difference. Methods like Two One-Sided Tests (TOST: Lakens, 2017) and Bayes Factors (Dienes, 2014) should be in the statistical toolbox of aging researchers to help determine when a null age effect is meaningful and when it is ambiguous—and potentially meaningful null age effects should be fair game for the journal. My hope is that making meaningful null age effects publishable will also reduce the temptation to engage in so-called Questionable Research Practices (QRPs) like p-hacking to try to ensure a p <.05 age difference in order to get a manuscript published. Using these techniques will not be required to submit, but instead can be an important tool to make stronger arguments about some null age effects. Look out for tutorials and examples of studies using these methods in upcoming issues of the journal. Another discussion point involves preregistration and replication. This is a standard that will be increasingly common in the coming years (e.g., expectations for Rigor and Reproducibility at NIH). So much of our knowledge as a field relies on the incredible large survey data sets available to us, such as HRS and MIDUS. Although the sample sizes of these studies are set and thus cannot be “justified” as in a lab study, researchers using these data sets can still take steps to ensure healthy research practices. For example, power analyses can be reported to demonstrate that the available sample size is adequate for testing the main hypotheses of interest. Although it would be unrealistic to expect investigators to collect an independent data set for replication, evaluating the robustness of the research findings in similar samples or independent samples drawn from the original data may give us more confidence in the results. A more challenging issue for us as a field to consider is how to reduce the possible influence of so-called Hypothesizing After Results are Known (HARKing) in secondary analyses. It is relatively easy for researchers working with existing data sets to check out different combinations of variables and see what leads to significant age (or other) effects. If our interest is in knowing what is true about age similarities and differences, rather than simply publishing significant age effects, then we need to know that studies were designed to ask the right questions using the right methods, rather than only evaluating the results. Thus, for studies using existing data sets, preregistration may offer a way to increase confidence in the robustness of the methods. Preregistration has already caught on in other disciplines that use existing data sets, such as personality psychology. Preregistration can be done easily using a site such as Open Science Framework (https://osf.io/)—this allows a time-stamped posting of preregistered hypotheses and analysis plans. Although this will not be a journal requirement, it is an option I believe journal authors should strongly consider. As a flagship journal for psychological aging, I believe JG:PS is the right venue for our field to have dialogue about research practices and to make changes that are right for us, that can also align us with psychology more generally. I hope my editorial will jumpstart this discussion, and I am excited to work with all of you to help craft the future of our field. References Dienes, Z. ( 2014). Using Bayes to get the most out of non-significant results. Frontiers in Psychology , 5, 781. doi: 10.3389/fpsyg.2014.00781 Google Scholar CrossRef Search ADS PubMed  Lakens, D. ( 2017). Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science , 8, 355– 362. doi: 10.1177/1948550617697177 Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The Journals of Gerontology Series B: Psychological Sciences and Social Sciences Oxford University Press

Planning for the Future of Psychological Research on Aging

Loading next page...
 
/lp/ou_press/planning-for-the-future-of-psychological-research-on-aging-myPYpUgtEp
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
ISSN
1079-5014
eISSN
1758-5368
D.O.I.
10.1093/geronb/gbx142
Publisher site
See Article on Publisher Site

Abstract

The first manuscript I ever submitted was to Journal of Gerontology: Psychological Sciences (but do not look for it on my CV—let us just say it did not ever get published at the journal). I still remember trying to staple all the copies together—this was still during the era of hard-copy manuscript submission. What I remember just as well is that there was no discussion with my advisor/co-author about where we would submit the manuscript first: we had a finding we were excited about, and we sent it to JG:PS. Fast forward more years than you might guess, and now I am extremely honored to have the opportunity to serve as editor-in-chief of the journal. I am excited to keep up the amazing momentum coming from Bob Knight’s time as editor. I am very grateful to have the help of four Associate Editors: Nicole Anderson, Angela Gutchess, Lynn Martire, and Shevaun Neupert, as well as Managing Editor Kathy Jackson. I am also cognizant that this is a moment of great introspection, and some critical changes in practices, in the field of psychological science. Although psychologists interested in aging and gerontology have not typically been in the forefront of recent discussions about research practices, I believe it is time for us as a field to think seriously and strategically about how we will align our practices with changing norms in the field more generally. I also feel that JG:PS is the ideal venue for open discussions about how we as a field want to move forward. And I am serious about discussion; my agenda is to push ahead scientific practices in our field rather than to necessarily adopt a specific set of rules. However, I do want to raise a number of starting points for these conversations. One discussion point involves null age effects: I think every reader of the journal has had the experience of doing a well-designed study, analyzing the data, and finding no age differences—then thinking “what a waste! I’ll never be able to publish this study anywhere good!” I am sure we have also all had the experience of trying to submit a manuscript like this to a top journal and having it rejected due to “no interesting age differences.” I have certainly had these experiences myself. I even had an editor-in-chief of a non-aging journal instruct me to only include manuscripts with significant age differences in a special section on aging. We are all aware that null age differences in a single study may not be very informative—but there are also likely processes that we are interested in as psychologists that do not vary by age. How can we make sure that studies that find informative null age effects can be included in the literature? First, our expectations about sample sizes to test for age differences need to be larger. We have grown familiar with rules of thumb for numbers of participants per age group, and those may be appropriate for studies investigating phenomena with large effect sizes, but are likely too small for the types of phenomena most psychologists study. In order to make sure we are being thoughtful about our sample sizes, all manuscripts should explicitly justify how they determine the sample size (i.e., by reporting a power analysis). This is a change I believe we can implement right away. Second, when we find null age effects in adequately-powered studies, we should be utilizing newly developed tools for quantifying the strength of evidence that there is actually no age difference. Methods like Two One-Sided Tests (TOST: Lakens, 2017) and Bayes Factors (Dienes, 2014) should be in the statistical toolbox of aging researchers to help determine when a null age effect is meaningful and when it is ambiguous—and potentially meaningful null age effects should be fair game for the journal. My hope is that making meaningful null age effects publishable will also reduce the temptation to engage in so-called Questionable Research Practices (QRPs) like p-hacking to try to ensure a p <.05 age difference in order to get a manuscript published. Using these techniques will not be required to submit, but instead can be an important tool to make stronger arguments about some null age effects. Look out for tutorials and examples of studies using these methods in upcoming issues of the journal. Another discussion point involves preregistration and replication. This is a standard that will be increasingly common in the coming years (e.g., expectations for Rigor and Reproducibility at NIH). So much of our knowledge as a field relies on the incredible large survey data sets available to us, such as HRS and MIDUS. Although the sample sizes of these studies are set and thus cannot be “justified” as in a lab study, researchers using these data sets can still take steps to ensure healthy research practices. For example, power analyses can be reported to demonstrate that the available sample size is adequate for testing the main hypotheses of interest. Although it would be unrealistic to expect investigators to collect an independent data set for replication, evaluating the robustness of the research findings in similar samples or independent samples drawn from the original data may give us more confidence in the results. A more challenging issue for us as a field to consider is how to reduce the possible influence of so-called Hypothesizing After Results are Known (HARKing) in secondary analyses. It is relatively easy for researchers working with existing data sets to check out different combinations of variables and see what leads to significant age (or other) effects. If our interest is in knowing what is true about age similarities and differences, rather than simply publishing significant age effects, then we need to know that studies were designed to ask the right questions using the right methods, rather than only evaluating the results. Thus, for studies using existing data sets, preregistration may offer a way to increase confidence in the robustness of the methods. Preregistration has already caught on in other disciplines that use existing data sets, such as personality psychology. Preregistration can be done easily using a site such as Open Science Framework (https://osf.io/)—this allows a time-stamped posting of preregistered hypotheses and analysis plans. Although this will not be a journal requirement, it is an option I believe journal authors should strongly consider. As a flagship journal for psychological aging, I believe JG:PS is the right venue for our field to have dialogue about research practices and to make changes that are right for us, that can also align us with psychology more generally. I hope my editorial will jumpstart this discussion, and I am excited to work with all of you to help craft the future of our field. References Dienes, Z. ( 2014). Using Bayes to get the most out of non-significant results. Frontiers in Psychology , 5, 781. doi: 10.3389/fpsyg.2014.00781 Google Scholar CrossRef Search ADS PubMed  Lakens, D. ( 2017). Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science , 8, 355– 362. doi: 10.1177/1948550617697177 Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Journal

The Journals of Gerontology Series B: Psychological Sciences and Social SciencesOxford University Press

Published: Mar 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 12 million articles from more than
10,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Unlimited reading

Read as many articles as you need. Full articles with original layout, charts and figures. Read online, from anywhere.

Stay up to date

Keep up with your field with Personalized Recommendations and Follow Journals to get automatic updates.

Organize your research

It’s easy to organize your research with our built-in tools.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

Monthly Plan

  • Read unlimited articles
  • Personalized recommendations
  • No expiration
  • Print 20 pages per month
  • 20% off on PDF purchases
  • Organize your research
  • Get updates on your journals and topic searches

$49/month

Start Free Trial

14-day Free Trial

Best Deal — 39% off

Annual Plan

  • All the features of the Professional Plan, but for 39% off!
  • Billed annually
  • No expiration
  • For the normal price of 10 articles elsewhere, you get one full year of unlimited access to articles.

$588

$360/year

billed annually
Start Free Trial

14-day Free Trial