Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

To be or not to be in the sample? On using manipulation checks in experimental accounting research

To be or not to be in the sample? On using manipulation checks in experimental accounting research Manipulation checks are a recommended for experimental accounting research. Usage of information gained by manipulation checks varies. In some studies, participants who failed the manipulation check are removed from the sample. Other studies report the results of the manipulation checks but still use the full sample. Some authors recommend removing participants who failed the manipulation check as a means to increase the power of the statistical analysis. Others warn that removing these participants endangers the randomization as a crucial precondition for gaining valid insights from experimental research. Until now, there is little research on how sensitive results react to exclusion of participants. The purpose of this paper is to study the effects of removing participants failing the manipulation checks on the evaluation of a hypothesis and the development of alternative usages of the information gained from manipulation checks.Design/methodology/approachBased on an analytical model and a simulation, the authors show how removing participants who fail the manipulation check affects experimental findings.FindingsSimulations show that statistical results and conclusions drawn from an experiment differ substantially, depending on whether participants who failed the manipulation check are removed from the sample. As the participants who are removed are no random sub-sample, but share a certain property, the experimental results react strongly, typically showing significant results, where there are actually none.Originality/valueThis paper is, to the best of the authors’ knowledge, the first to address the sensitivity of experimental results to removing participants who fail the manipulation check from the sample and the implications for the validity of conclusions drawn from experimental accounting research. This paper’s contribution is a better way of using information gained in the manipulation check in the statistical analysis of the experimental data. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Accounting Research Journal Emerald Publishing

To be or not to be in the sample? On using manipulation checks in experimental accounting research

Download PDF
14 pages

Loading next page...
 
/lp/emerald-publishing/to-be-or-not-to-be-in-the-sample-on-using-manipulation-checks-in-X0OE1H2Gvc

References (34)

Publisher
Emerald Publishing
Copyright
© Emerald Publishing Limited
ISSN
1030-9616
DOI
10.1108/arj-06-2019-0128
Publisher site
See Article on Publisher Site

Abstract

Manipulation checks are a recommended for experimental accounting research. Usage of information gained by manipulation checks varies. In some studies, participants who failed the manipulation check are removed from the sample. Other studies report the results of the manipulation checks but still use the full sample. Some authors recommend removing participants who failed the manipulation check as a means to increase the power of the statistical analysis. Others warn that removing these participants endangers the randomization as a crucial precondition for gaining valid insights from experimental research. Until now, there is little research on how sensitive results react to exclusion of participants. The purpose of this paper is to study the effects of removing participants failing the manipulation checks on the evaluation of a hypothesis and the development of alternative usages of the information gained from manipulation checks.Design/methodology/approachBased on an analytical model and a simulation, the authors show how removing participants who fail the manipulation check affects experimental findings.FindingsSimulations show that statistical results and conclusions drawn from an experiment differ substantially, depending on whether participants who failed the manipulation check are removed from the sample. As the participants who are removed are no random sub-sample, but share a certain property, the experimental results react strongly, typically showing significant results, where there are actually none.Originality/valueThis paper is, to the best of the authors’ knowledge, the first to address the sensitivity of experimental results to removing participants who fail the manipulation check from the sample and the implications for the validity of conclusions drawn from experimental accounting research. This paper’s contribution is a better way of using information gained in the manipulation check in the statistical analysis of the experimental data.

Journal

Accounting Research Journal – Emerald Publishing

Published: Jun 15, 2020

Keywords: Experimental accounting research; Manipulation checks; Replicability

There are no references for this article.