TY - JOUR
AU - Orero,, Pilar
AB - Abstract The United Nations Convention on the Rights of Persons with Disabilities requests “Nothing about us without us.” User-centered methodological research is the way to comply with this convention. Interaction with the deaf community must be in their language; hence sign language questionnaires are one of the tools to gather data. While in the past interacting with an online video questionnaire was out of the question, today it is a reality. This article focuses on the design of an interactive video questionnaire for sign language users. From a historical review of the existing literature on research methods and previous sign language questionnaire, the article examines the design features affected in the process of making accessible questionnaires with sign language videos: format and layout. The article finishes with the solution developed toward mainstreaming sign language questionnaires in order to contribute to a diverse and inclusive society for all citizens. Questionnaires are a common instrument used for research. A set of questions extracts relevant information given to a sample of respondents to complete (Harkness, 2008). They are used in research over other data collection methods and frequently have standardized answers that provide easy to compile and analyze data. They are relatively cheap to implement, compared to other surveying or assessment instruments, such as interviews or focus groups (Survey Research Center, 2016). As with any research instrument involving deaf participants, questionnaires need to incorporate a deaf-friendly perspective toward equity for the deaf population in both surveys and testing research. Questionnaires are frequently administered through the official, written languages and rarely present accessibility features, such as using Easy to Read (Bernabé & Orero, 2019). Cultural and linguistic lack of perspective in instrument design has a negative impact on both the appropriateness of the instrument itself as well as the validity of the results (Freeman, 1989). More recently, accessibility through computerized video technologies in questionnaire designs has promoted the possibility of incorporating sign languages. The application of information and communication technologies (ICTs) allows for deaf-friendlier design and also to implement translation protocols toward access to information for all users (Fontaine, 2012; Young & Hunt, 2011; Young & Temple, 2014). Moreover, designs are crucial when creating a fully accessible questionnaire to develop robust instruments to collect data on sign language from sign language users, within the sign language communities (De Meulder, Krausneker, Turner, & Conama, 2018; Harris, Holmes, & Mertens, 2009). In this article we review the evolution of methods in filmed-based approaches that include sign language to make questionnaires accessible. Specifically, we look into the language accessibility features, design features, and administration modes implemented in previous approaches. We present our own methodological approach to create a sign language-friendly questionnaire design, taking into consideration time-efficient and cost-efficient limitations. The implementation is a computerized video questionnaire fully accessible in sign language that can robustly deploy a small-scale, research-oriented, sign language recall and comprehension test. Toward a Deaf-Friendly Questionnaire Design Fully accessible questionnaire designs in sign language are important to guarantee the reliability of the research and the validity of the results. They are also relevant both to protect ethical standards in human subject research within the deaf population and to promote accessibility to comply with the new, inclusive “human right” paradigm (Berghs, Atkin, Graham, Hatton, & Thomas, 2016; Ewart & Snowden, 2012). This is not only to control the bias that may arise from the lack of linguistic and cultural concordance between the researchers and the signing deaf participants (McKee, Schlehofer, & Thew, 2013) but also to promote inclusive research that undertakes and integrates the diversity inherent within the Deaf communities, especially regarding language accessibility: language choice and literacy (Guardino & Cannon, 2016). Deaf-Friendly Research Methods The literature for sign language research methods tackles specific knowledge areas in both theoretical and applied sign language linguistics (Meurant, Sinte, Vermeerbergen, & Van Herreweghen, 2013) such as sign language acquisition (Baker, van den Bogaerde, & Woll, 2006), sociolinguistics (Lucas, Mirus, Palmer, Roessler, & Frost, 2013), and sign language assessment (Haug, 2011). Both hearing and deaf researchers have focused on ethical considerations when conducting research within the deaf communities, particularly on communication accessibility, truly informed consent, and anonymity (Baker-Shenk & Kyle, 1990; Benedict & Sass-Lehrer, 2007; Gutman, 2005; Harris et al., 2009; McKee et al., 2013; Pollard, 2002; Singleton, Jones, & Hanumantha, 2012; Singleton, Martin, & Morgan, 2015; Sign Language Linguistics Society, 2016). Accessible communication and informed consent should be a priority for researchers. Additionally, privacy issues may arise from the fact that it is almost impossible to separate the video-recorded sign language data from the participants’ identity; hence signed, accessible image release forms should be provided. Some of these ethical considerations and guidelines on deaf-friendly research have been compiled by Orfanidou, Woll, & Morgan (2015) in their handbook Research Methods in Sign Language Studies: A Practical Guide. This text is a landmark on sign language research devoted to data collection methods and new instruments and tools for sign language research in different fields of linguistics, paying special attention to ethical considerations. Unfortunately, deaf-friendly questionnaire design using sign language as a mode of administration is not discussed. More recently, Cawthon & Garberoglio (2017) edited the text Research in Deaf Education: Context, Challenges, and Considerations, featuring an array of diverse research methodologies for the Deaf community. This volume includes a chapter on accessible, large-scale survey design in education (Cawthon, 2017). The chapter describes some considerations when developing large-scale surveys in deaf education, for example, how to control the heterogeneous demographics within the Deaf communities or how to enhance the recruitment process. In the section on accessible survey designs, the author argues that dual-language designs not only provide better access but also promote engagement within the deaf community. Although she advocates the use of technology platforms that fully support dual-language/dual-modality surveys is a requirement, technical design features are not further discussed. A number of authors have included deaf-friendly methodological approaches and good practices in their research to promote cultural and linguistically adequate research methods. This means involving members of the Deaf community in the research process in meaningful ways following emancipatory or transformative paradigm approaches, not only as research participants (Harris et al., 2009; Mertens, 2005, 2010; Munger & Mertens, 2011; Mertens, Sullivan, & Stace, 2011). The participation of deaf native signers and deaf organizations in research teams is subsequently regarded as a key concern. Research teams should include different roles such as deaf researchers, deaf research assistants, deaf research facilitators, deaf interpreters or support communicators, and deaf-signing models (Ladd, 2003; McKee et al., 2013; Pollard, 2002; Singleton et al., 2012, 2015). On the other hand, developing deaf-friendly research methods involves merging accessibility and communication best practices with different research materials, documents, and the dissemination of research projects and results. The potential of eAccessibility through ICT for social inclusion, and participation of all citizens, is increasingly allowing full integration in everyday life for all (Orero, 2017). The use of video protocols in computer-based technologies is an opportunity to represent the modality-specific visual features of sign languages (Haug, 2011, p. 46). Video technologies have also had an impact on the type of data and data collection methods for research involving sign language users, such as face-to-face video chatting or user-generated video sharing (Lucas et al., 2013, p. 548). Language Accessibility in Questionnaire Designs The language and culture of the participants can affect their perception of questions and therefore their answers (Choi & Pak, 2005). Ethnic minority groups have a higher rate of nonrespondence (Erens, 2013). This can be related to “language barriers, lack of trust, wariness of government authorities, perception that the research is unimportant or that their contribution is unimportant, reluctance to have their information written down, and a feeling that they have been over-researched” (Erens, 2013, p. 59). Similar factors have been described for signing deaf participants, though their low participation may be partially linked to participant fatigue (Cawthon & Garberoglio, 2017). In this article we examine traditional monolingual-written text questionnaires using signing participants, because they are widely used in many research fields with sign language users. Hearing researchers may be unaware of the cultural and linguistic characteristics of the deaf community and vice versa. Understanding questions will depend on the respondent’s literacy skills that can lead to uncontrolled variation in question misunderstanding bias and participant cognitive burden. Adequate language options and question wording are important toward the validity of the questionnaire design. Questions must be phrased appropriately for the target audience with the information acquired avoiding ambiguity and connotative meaning (Boynton & Greenhalgh, 2004). Even though it is still not common practice, our literature review includes several examples on explicit explanations, or overt discussion, on how data collection instruments and materials are made available to signing deaf participants and the language choices in which data and questions are presented. Translation procedures in questionnaire design. Most research is still developed in written languages. Translation should be performed according to standards to guarantee the quality of the instrument and to control the effects of translation (Harkness, 2008). The process of preparing, performing, reviewing, and assessing translated questionnaires is thus quite complex and costly (Erens, 2013). There are different translation procedures and protocols devised to control translation quality and minimize translation bias that may jeopardize research results. One of the early documented translation approaches in sign-translated questionnaires is back translation. Back translation was a recommended standard for translating survey instruments (Erens, 2013; Survey Research Center, 2016). This procedure involves a specialist translating the text into the target language, which is then independently translated back into the source language. The original and back translated source language texts are compared to check if there are any translation issues in the target text. Even today, back translation is one of the most common translation techniques. Some of the reviewed questionnaires for deaf populations included professional sign language interpreters and/or signing deaf research team members carrying out the role of back translators in the development of the target video sign language questionnaires (VSL questionnaire; Brauer, 1993; Cohen & Jones, 1990; Goldstein, Eckhardt, & Joyner, 2004; Lipton, Goldstein, Fahnbulleh, & Gertz, 1996; Rojba, 2016). Nowadays, best practices and guidelines recommend team translation approaches for survey instrument production. Currently, the common best practice in survey translation is the five-phase iterative team approach model called translation, review, adjudication, pretesting, and documentation (TRAPD; Harkness, 2008). The TRAPD method involves several translators, survey experts, and consultants and is pretested with a small-scale target population group before producing the final translation. The team members involved may take on one or more roles as translators, reviewers, or adjudicators. During the first stage, translators produce initial parallel translations independently. Then an expanded team including the translators and a reviewer go through the draft translations, discuss the versions, and agree on a review translation. Adjudicators decide if the reviewed version moves to pretesting. Adjudication may take place at the review session, or at a different meeting between the reviewer and the adjudicator. All steps are fully documented. Documentation is a monitoring tool for quality assurance that provides information for further analysis. For example, team members take notes on compromised decisions, unresolved issues, or consultant feedback. The procedures are partially iterative. For example, pretesting may trigger further modifications of the translation before the adjudicator decides on the final version for fielding (Harkness, 2008; Survey Research Center, 2016). Adaptation in sign language questionnaire design. Sign linguists in the field of sign language tests have addressed methodological issues concerning translation and adaptation procedures to develop reliable and valid assessment instruments (see, e.g., Enns & Herman, 2011; Haug, 2011, 2015; Haug & Mann, 2008; Hermans, Knoors, & Verhoeven, 2010). Within the field, the term adaptation is used to emphasize that the goal is to create a test in the target language that parallels the source test not only in its language contents but also in its functionality. Although the translation process is assumed to be centered in faithfully rendering the source contents into the language of the target test, the adaptation process involves more flexibility in test construction and may introduce further alterations according to the cultural and social needs of the target language users (Haug & Mann, 2008, p. 139). The goal of adaptation in cross-cultural questionnaire design is to match the needs of the target population, also known as “localization” in translation studies, as is the case in Pollard, Dean, O’Hearn, and Haynes (2009) for health education material for deaf audiences.The modifications “may be made to the content, format, response scale, or visual presentation of any part of the question, questionnaire, or instrument” (Survey Research Center, 2016). Sign language linguists have outlined several phases in the adaptation process of sign language assessment tests to ensure the standardization of the developed tools. These adaptation protocols aim to guarantee the validity and reliability of the test instrument (Enns & Herman, 2011; Haug, 2011; Hauser et al., 2016; Mann, Roy, & Morgan, 2016) New language approaches in sign language questionnaire designs. More recently, Cawthon (2017) suggests the inclusion of bimodal–bilingual for the development of research materials. The author favors a dual-language development process rather than designing survey items in the written form and the implementation of translation or adaption procedures. “Beginning the development of survey items with an understanding of how they may be represented in different linguistic formats holds promise in making large-scale surveys more accessible by design” (Cawthon, 2017, p.31). Dual-language development of survey items would only include concepts that are equally easy to express in both languages. They would control potential sources of bias and also provide a more accessible design for all participants. A different innovative language approach to survey item development has recently been designed and implemented by Napier, Lloyd, Skinner, Turner, and Wheatly (2018), targeting multilingual deaf signers using multiple national sign languages from various European countries. Their goal is to implement a large-scale survey to engage a larger number of international participants from a broader range of linguistically diverse signing backgrounds. Because of time and cost restrictions, the authors choose to use International Sign Language for those users who lack literacy or prefer to use a signed language as opposed to a multicultural multilingual approach including many national sign languages. This solution also avoids prioritizing certain national sign languages over others, potentially excluding users of minority sign languages (Napier, et al., 2018, p. 104). On balance, time and cost constraints will greatly influence the choice of language approach, translation, and adaptation protocols and procedures, as will study specifications such as sample size and standardization (Kappelhof, 2015). Adaptation and translation procedures may be determined according to the nature of the material being implemented and the project characteristics. Lastly, a successful questionnaire translation is expected to keep the content of the questions semantically similar or maintain the same stimulus and measurement options. It should also keep the question format similar between the source and target questionnaires, within the bounds of the target language (Survey Research Center, 2016). For sign languages that differ greatly in modality and have no established written form, much more research is required. A Review of the Literature on Deaf-Friendly Questionnaires Computerized sign language implementations for questionnaire research have rapidly changed, thanks to the advances of new technologies especially in the development of video technology and online survey platforms (Lucas et al., 2013, p. 54). When reviewing early research methods in sign language studies linguists often relied on written-only questionnaires for data collection on sign language structure and use. To present sign language stimuli, written forms of the signs or sign structures being analyzed were used in paper format questionnaires (see, e.g., Woodward, 1973, and Woodward & DeSantis, 1977). During the following decade, video protocols were introduced using the different video formats available. Early video implementations involved a written questionnaire with a video-taped sign-translated version that was played alongside (Brauer, 1993). Although this solution would give access to the content, the signed and written versions differed largely in format and layout. The first attempt of a self-administered sign language questionnaire that did not require literacy was the Interactive Video Questionnaire, developed to question deaf people about drug abuse (Lipton et al., 1996). The questionnaire was implemented using a Laserdisc player on a standard television monitor displaying questions in American Sign Language. In Phase I of the study, the participants had a bar code reader that served to scan the responses and add them on the paper questionnaire answer sheet. In Phase II, touchscreen technology for automatic data capture and storage was used, which proved to be a simple mode of administration. Responses were recorded directly to the hard disk and then downloaded to a cassette tape. For interviewer-administered tests, Haug (2015, p. 45), in his exploratory study based on an international survey on the usage of ICT for sign language test delivery, listed different test modes of administration including noncomputerized and computerized formats. Noncomputerized formats included, for instance, videotape (VHS and other formats) or DVD either with or without an accompanying booklet. Haug (2015) also listed some computerized modes of administration, for example, web-based tests or sign language tests implemented into existing learning tools such as Moodle. There are many deaf-friendly designs implementing video protocols in order to give full access to signing participants. The term VSL questionnaire is used to refer to all instrument designs and modes of administration using video protocols that grant accessibility in sign language to a certain extent. This includes both computerized and noncomputerized modes of administration that have used different video protocols and technologies available since the 1980s. There are three types of computerized survey instruments classified according to their mode of administration: Computer-assisted surveys using video clips are called video computer-assisted personal interviews, or video-CAPI. This is when a physically present interviewer administers the questionnaire and enters the answers on a computer or similar device. A video computer-assisted self-administered interview, or video-CASI, is when the respondent enters the answers on a computer provided by an interviewer who is physically present. Computer-assisted video interviewing, or CAVI, is when the communication between the interviewer and the respondent is established remotely via video chat. In addition there are other terms that have been used in the literature to refer to video questionnaires specifically designed to include the use of sign languages as the language of interview administration. These video questionnaires are listed here in chronological order: interactive video-questionnaire (Lipton et al., 1996); animated questionnaire for deaf and hard of hearing (DHH) people, or ANIMAQU (Gerich & Lehner, 2006); dual-mode (CATI/CAPI) survey-computer-assisted telephone interview/computer-assisted personal interview (Sloan, Wright, & Barrett, 2012); virtual video survey (Lucas et al., 2013); and dual-language, dual-modality survey (Cawthon & Garberoglio, 2017). In this article, the term used is VSL questionnaire to refer to the computerized deaf-friendly questionnaire designs implementing full access in sign language using video protocols. When it is necessary to distinguish between the modes of administration we use the short forms VSL-CASI and VSL-CAPI, for convenience. In the 21st century, sign language assessment tests have been another important source of advances in computer- and web-based video sign language implementations (Enns et al., 2016; Haug, 2011; Haug, Herman, & Woll, 2015; Haug & Mann, 2008; Hauser et al., 2016; Herman, Holmes, & Woll, 1999; Hermans et al., 2010; Lara-Escudero, 2017; Mann et al., 2016). New methodologies have started to adopt VSL questionnaires as a mode of administration, such as sign language comprehension and video quality evaluations on technological devices (Cavender, Ladner, & Riskin, 2006; Tran, 2014; Tran, Kim, Riskin, Ladner, & Jacob, 2011; Tran, Riskin, Ladner, & Wobbrock, 2015) including synthesized sign language played by signing avatars or animations (Huenerfauth & Kacorri, 2015). Other VSL questionnaires are described in some research-led surveys and tests in different fields, such as Deaf education and sign language teaching applications (Hansen et al., 2018; Higgins, Famularo, Bownman, & Hall, 2015; Hussein & Al-Bayati, 2009), cross-modal language user tests (Lucas et al., 2013), cross-cultural social surveys (Fontaine, 2012), sign language teaching surveys (Pyfers, 2017), sign language interpretation (De Wit, 2011; Lang, 2015), or accessibility services and use of technology (Maiorana-Basas & Pagliaro, 2014; Napier et al., 2018), among others. Even though all these researchers acknowledged the use of a computer-based VSL questionnaire, the technical design of the implementations varies greatly, ranging from specially created, interactive applications to the use of existing online survey platforms that play video sign language content alongside it. Moreover, for self-administered interviews, participants should go back-and-forth between the two versions, which would increase both time burden and cognitive burden, and therefore affect response time and nonresponse rate. Together with participant burden, this mode of administration still required some degree of literacy to match the sign language to the written in order to submit the responses. The written source questionnaire was laid out in a clear manner, containing not only the source language but also the main language of interaction with the test platform. The sign language version contained an additional target language with a less prominent or lower position, thus subordinating sign language to the official written language. Nowadays, new technologies have allowed sign language interlocutors to interact virtually using their own language (Lucas et al., 2013). Deaf community members have rapidly become extensive users of mobile and computer applications that implement video technologies, such as video telephones, video chats, video calls, video blogs, video messages, or video sharing. At the same time, computerized, web-based instruments containing sign language have started to become more frequent. New technologies not only introduce the possibility of embedding video clips to develop web-based questionnaire application, but also existing online survey platforms (such as MonkeySurvey or Google forms), which introduce the possibility of inserting videos from video-sharing platforms (such as YouTube) and upload user video files in their features. These platforms and applications allow sign language on-screen setups following different technical designs that directly impact accessibility features, language status, and usability. Accessible technical designs: question format and layout. Most studies including a computer-based VSL questionnaire do not provide detailed descriptions, if any at all, particularly in regard to technical design, delivery format, and usability of the instrument. Some exceptions are Gerich and Lehner (2006), Haug et al. (2015), Lipton et al. (1996), Napier et al. (2018), and Samar, Barnett, Oyzon, Mowl, and Sutter (2012). After conducting a survey among sign language test developers, Haug (2015, p.37) acknowledges that none of the studies that had a computerized format addressed the usage of ICTs for sign language test development. According to the language of on-screen presentation, we distinguish two types of VSL questionnaires: One-clip VSL questionnaires and multiclip VSL questionnaires One-clip VSL questionnaires One-clip VSL questionnaires show a written online survey with one video screen embedded, showing a video clip of the sign-translated version (see Figure 1). Video clips contain both questions/stimuli and answers in a single video sequence. Answers can be identified by using numbers, letters, and vertical or horizontal lists. In order to make the video clips and question sequence more usable in the signed version, some authors use one clip per questionnaire item; only one item is shown on screen at a given time (Barnett et al., 2017; Goldstein et al., 2010; Pyfers, 2017; Tran, 2014; cf. Huenerfauth & Kacorri, 2015, simultaneously display four questions and their response items on one screen). Figure 1 Open in new tabDownload slide One-clip layout for cross-modal bilingual VSL questionnaire. Figure 1 Open in new tabDownload slide One-clip layout for cross-modal bilingual VSL questionnaire. Figure 2 Open in new tabDownload slide Multiclip layout monolingual VSL questionnaire. Figure 2 Open in new tabDownload slide Multiclip layout monolingual VSL questionnaire. The common layout shows the video at the top of the screen with the text version at the bottom or on the left. Typically, responses have to be submitted using the written format. For closed-ended questions the answer has to be selected from a set of predefined written responses, for example, yes/no or multiple choice. For open-ended questions, frequently, a text box requires the participant to write the answer (see, e.g., Fontaine, 2012; Goldstein et al., 2010; Huenerfauth & Kacorri, 2015; Pyfers, 2017). As with the noncomputerized video-taped questionnaire in Brauer (1993), this is an accessibility option that adds too much strain to the signing participant, as it only translates the linguistic content but not the format of the questionnaire. Additionally, it requires literacy and has limited interaction features with the signed video version. This might affect usability in an attempt to remove written language literacy requirements and enhance accessibility. Some studies have used color codes and symbols instead of written options to submit responses in multiple choice and Likert scale answers (Gerich & Lehner, 2006; Kipp, Nguyen, Heloir, & Matthes, 2011; Napier et al., 2018; Pardo-Guijarro et al., 2015). Multiclip VSL questionnaires The second type of VSL questionnaire format is multiclip VSL questionnaire, which displays a combination of more than one set of videos simultaneously to present both the question and the selected answers in separate video sequences. Multiclip layout designs aim to reproduce standard, computerized written instruments, both in content and in layout to provide better adaptations. One of the most prominent advantages over other designs is that they are fully accessible in sign language and do not require written language literacy. Multiclip VSL questionnaires can implement monolingual questionnaires in the signed modality. Thus, they can provide improved reliability and validity of the survey or research instrument for sign language users, especially in the case of language assessment tests. Furthermore, multiclip formats are a way of implementing fully translated bilingual questionnaires, where sign languages can be laid out in a prominent on-screen position, removing the hegemonic subordination to the written formats. Similarly, developing multilingual cross-modal questionnaires allows one to implement visual communication systems toward universal design. They grant access to a greater number and a wider range of potential deaf respondents within the diverse deaf population and their language choices in a number of ways (Graybill et al., 2010; Napier, et al. 2018; Tran, 2014). Multiclip formats present not only advantages but also limitations. Among the former they can reduce burden and nonresponse in questionnaires targeting deaf populations by offering choice of language and mode of questionnaire completion that might suit groups with differing communication skills. They may help to solve the issue of underrepresentation of the signing Deaf community in general population surveys. Finally, they provide better representation of languages in the signed modality and promote equality. Regarding limitations, they commonly display only one item—one question and the set of responses—at a time. Thus, they cannot provide an equivalent format for standardized questionnaires that present a whole set of questions (Dillman & Christian, 2005, as cited in Gerich & Lehner, 2006, p.271). Be that as it may, using computer-assisted VSL questionnaires has the potential to create better questionnaire adaptations and enhances accessibility for sign language users.1 Implementing Accessibility and Usability Features in Sign Language-Friendly Questionnaires Our multiclip video sign language questionnaires were first developed withn the HBB4ALL project funded by the European Commission (CIP-ICT-621014; 2013-2016) as a web-based data collection tool for sign language interpretation pilot tests. The tests aimed to collect data about perception and processing of information of the contents, usability, and user preferences regarding size and position of the sign language interpreter on TV in four different screen configurations. The detailed design and procedures, results, and findings of the experimental pilot tests are further presented in Bosch-Baliarda, Soler-Vilageliu, and Orero (2019). For the purposes of this paper we will first briefly describe the experimental procedure to provide a context to the questionnaire and later focus on the description of the design and development of the tool itself. Experimental Settings Participants in the pilot test were 32 deaf users (16 men/16 women) from the metropolitan area of Barcelona in Catalonia (Spain). Their ages ranged from 17 to 76 years (mean, 40 years). They were recruited through the Catalan Federation of the Deaf (Federació de Persones Sordes de Catalunya, FESOCA). All of them use Catalan Sign Language (LSC) to communicate in their everyday lives and have self-identified as being proficient signers. The experimental task consisted of watching four clips extracted from the documentary “Joining the Dots” (Romero-Fresco, 2012). Each clip was displayed in a different screen configuration and an Eye tracker Tobii 60, controlled by a Toshiba Portable personal computer recording participant’s eye movements. Users were individually tested in different local Deaf association offices. After being welcomed, they had to fill out the consent and image release forms and answer the demographics questionnaire. Following these preliminary steps, the experimental procedure began. Each participant watched one clip and answered three questionnaires immediately after a visual recall test, a language recall test, and a user experience test to tackle usability and user preferences for each screen configuration. We did not include any questions to gather data regarding the usability of questionnaire itself. All tests were administered and recorded on a MacBook Air personal computer. This process was repeated four times, one for every clip. Questionnaire Design The design approach to develop the target questionnaires was a mixed mode. The source questionnaires were first created in written Spanish and were later translated into LSC for the target questionnaire. The initial approach in question design involved asking the same questions and translating, or the ASQT approach. The questionnaires developed for captioning user tests were taken as the source question sets. Later some questions were adapted to the visual culture of the Deaf community in Catalonia and others had to be newly created to address specific features of the sign language interpreter on screen presentation (asking different questions, or the ADQ approach; Harkness, 2008; Survey Research Center, 2016). The translation/adaptation was dealt with in a one-team framework by a small team comprised of three members: one deaf native signer and two hearing nonnative signers. The deaf team member was a highly skilled sign language teacher for sign language interpreters and communication support workers. She has a vast experience as a deaf consultant in research and in creating and adapting educational materials into LSC. The second team member was a certified hearing sign language interpreter with more than 15 years of intensive working experience in many areas and skilled areas, such as TV. The third member of the translation team was the bilingual bimodal researcher assistant, trained in deaf studies, sign language linguistics, and sign language interpretation. The translation procedures included two translations from the hearing team members reviewed by the deaf consultant. She fine-tuned the translations and also pointed out which questions should be further adapted. The final translation was thus collaborative and agreed on by the three team members. The team translated both the video clips for the stimulus and all the questionnaire items. The signing model for the translated documentary clips was down to the input of the professional hearing interpreter, providing a parallel input to the most frequent use of sign language on TV, whereas the signing model for the questionnaire items was the deaf signer, in order to provide a native input. Special care was taken when creating the linguistic recall questions to use the same lexical items in both the interpreted clips and the questionnaire items, questions and responses. Thus, the linguistic input was controlled in both dialect and phonological variation of the targeted signs. Failing to do so would make the evaluation of the participants’ performance impossible to evaluate and would have compromised the validity of the results. Finally, all questionnaire items were video-recorded using a chroma key. A video protocol was the selected format so that the questionnaire input, accuracy, and consistency could be guaranteed throughout the pilot test. Technical design features implemented in HBB4ALL questionnaires. Even though the test was designed to be conducted with a small sample of 32 participants, the research team wanted to create a technical instrument design that allowed full accessibility of the questionnaire contents in sign language. The researchers wanted, on the one hand, to create a sign language-friendly data collection tool that could include video protocols to enhance validity and reliability of the instrument, especially for the sign language recall test. On the other, they wanted to create a deaf-friendly instrument that would enhance research engagement and usability of research tools by the targeted respondents, deaf sign language users. Additionally, we aimed to design a web-based on-screen layout that would attribute the same social and linguistic status to both language modalities, LSC and the written languages of the territory, namely Spanish and Catalan. This methodology was innovative because it had never been used in the country before nor in the field of accessibility research. Because no existing online survey platforms allowed multiclip layout customization for the different types of questions (multiple choice, yes–no, etc.), the technical team members developed a multiscreen, computer-assisted data collection program that could implement the desired features. Both local and online versions were tested in the different pilot tests. Even though developing tools that allow implementations of questionnaires fully accessible in sign language was more time- and cost-consuming, it was an effort that was not to be avoided to grant research standards. For a more detailed description of the technical platform implementation, question technical designs, and technical formats see López et al. (2019). Regarding language on-screen presentation, we wanted the research tool to be fully accessible in both signed and written modalities. The aim was to improve its usability within the diverse deaf signing communities, from skilled deaf signers regardless of their literacy levels to nonnative deaf signers that may rely on the written form or that prefer the written form to access certain questionnaire items, especially in the demographic questions. The platform was designed so that it could include a computerized sign language version of the informed consent and image release forms, so that these crucial aspects of the research tools were also accessible to signers who preferred this modality. Both forms were played or read before conducting the test and answering the questionnaires. As in most of the previous multiclip VSL questionnaire designs, only one question was displayed on-screen at a time so that no scrolling was necessary. The multiclip question layout was designed to make LSC prominent on the screen (see Figure 3). The video clips for the questions were placed top center of the screen, and the different possible responses were displayed horizontally underneath. Each answer video clip was identified with a thumbnail showing the fingerspelling hand shapes for A, B, C, and D, respectively—or number hand shapes in the Spanish Sign Language test—to make them visually more distinct from the question video clips and to provide consistency throughout the different question designs. The written versions were displayed under each video clip. Figure 3 Open in new tabDownload slide Screenshot of the multiclip layout design from the LSC HBB4ALL cross-modal bilingual questionnaire. Figure 3 Open in new tabDownload slide Screenshot of the multiclip layout design from the LSC HBB4ALL cross-modal bilingual questionnaire. To give the same status to both language modalities in the technical design we introduced an innovation for submitting the answers. As mentioned in Implementing Accessibility and Usability Features in Sign Language-Friendly Questionnaires section, most VSL questionnaires require participants to submit their responses using the written items, generally by a single click on the selected response or a bullet in line with the response item. In our questionnaire we wanted sign language to be the language of interaction too, so that both language modalities could play the same central role. To select the desired answers, participants could click on the video clip. Once the answer was selected, a blue frame would appear and the next button would be enabled. See Figure 4. Figure 4 Open in new tabDownload slide Screenshot of a multiple choice question design with four answers selected for submission in the Spanish Sign Language HBB4ALL cross-modal bilingual questionnaire. Figure 4 Open in new tabDownload slide Screenshot of a multiple choice question design with four answers selected for submission in the Spanish Sign Language HBB4ALL cross-modal bilingual questionnaire. Thus, participants’ responses would not rely on their literacy, and they could avoid going back and forth between the written and signed versions of the questionnaire items in order to respond. This innovation was designed not only to improve the language status but also to reduce cognitive burden, response time, and overall time, which was one of our concerns regarding test duration. A reproduction feature to allow “skimming” and “scanning” the videos was designed. The function was to make the user experience more comparable within the two languages in the fully cross-modal bilingual questionnaire. By swiping on the available answers the video clips were played automatically. Both the question and answer clips could be replayed, paused, forwarded, and rewound by clicking on the symbol buttons in a standardized video player bar. Other features were introduced to improve the experience of older users, blind or partially sighted signers. For example, the speed rate and size of the video clips could be adjusted according to the user needs in order to improve usability and accessibility. Each video clip item could be played full-screen if selected. Results The participants acknowledged that it was the first time they had used a questionnaire fully accessible in sign language and positively valued the language choice offered in the bilingual–bimodal approach. They also valued the use of deaf signers. Although we did not include any formal controlled questions to gather feedback on the strategies used, some were discussed informally at the end of the test. All participants used both language modalities to some extent when completing the questionnaire. Participants with higher literacy skills skipped most of the video items, especially in the demographic and user experience tests, because it was much faster than going through the response items. However, they all watched the sign language version for the recall tests. Strategies regarding the language chosen to access the questionnaire items differed greatly. Some participants first read the written items and used the signed version to double-check comprehension, whereas some used the opposite strategy. For example, some users would watch the video first and then read the responses to “learn” the words for some of the signs they did not know previously. However, the signed version was mostly used for response submission. The “skimming” feature was used both to previsualize the possible responses and to double-check the response before submitting the final choice. Younger participants exploited this feature more than older subjects, which helped them going much faster through the tests. Additionally, three older participants asked the interviewer to help them submit the responses, because they were not confident enough to interact with the computerized interface. The other built-in accessibility features were scarcely used. None of the participants used the speed adjustment, and only one used the full-screen video feature in two questionnaire items. The size and color contrast of the filmed items was considered to grant accessibility to the signed content. The web-based questionnaire design allowed automatically saving the results on the server. However due to server connectivity issues 5 recall test responses and 11 individual response items were not saved automatically. Discussion of the Current Approach The present questionnaire design aims to encompass previous findings in the method designs developed for sign language assessment tests and in deaf-friendly surveying instruments. The resulting interface is an appropriate tool that can successfully implement a cross-modal bilingual questionnaire design. It not only is fully accessible in both sign language and in writing, but also provides a similar status of the two language modalities in both their on-screen representation and in their functionality, such as allowing participants to submit their answers through the interactive signed version of the responses. It has shown to be a valid research tool to implement an interviewer-administered sign language recall test that can match the modality representation of the stimuli, the questions, and the response items, and thus enhances the reliability of the results. However, schedule and budget limitations together with the scope of the study impeded the implementation of more features that have been described for deaf-friendly surveying instruments, such as offering choice from different signing models using a wider range of sign language styles or other accessibility features in the written versions such as including Easy to Read. Similarly, other customization features such as offering background color choice were discussed but could not be implemented in this prototype version. Still, background color is a feature that could both reduce eye fatigue and improve accessibility and readability among signers with different sight status. As already mentioned, computerized, sign-friendly questionnaires are the most affordable and reliable mode of administration that give access to sign language as a prerequisite to comply with ethical standards and achieve human rights for sign language communities in research. Nevertheless, this mode of administration holds some disadvantages when compared to traditional, written paper-and-pencil questionnaires. Namely, there are at least three major drawbacks: they are more expensive to produce, they are bound to technical malfunctioning, and sometimes there is increased time burden, as they take longer to be conducted. Regarding the detrimental time factor, Graybill et al. (2010) acknowledge that access to answer choices in a sign language survey is necessarily sequential, unlike what occurs in a written survey. Participants in a written survey have a near-simultaneous access to all response items. They are able to scan and skim answer choices and reread them repeatedly, in any order, before making a final choice, whereas respondents to a VSL questionnaire have to watch and consider each answer in turn before selecting a response or replay any of the answer video clips. These authors admit that the modality differences greatly affect the experience of completing a survey, although VSL questionnaires would be more comparable to the experience of a telephone survey. When designing our interface, we discussed some features in the questionnaire design to reduce the overall interview time. On the one hand, reducing time burden was one of the arguments for choosing a cross-modal bilingual design for all the questionnaire sets over a monolingual sign language design. This option clearly reduces time burden but introduces new sources of uncontrolled bias among and within participant responses. Furthermore we wanted to implement some technical innovation that would allow simultaneous access to all the video responses. A first attempt in the design displayed all the signed response items playing simultaneously, nonstop along with the written items. However, this version produced too much noise and was not deployed. In this same direction, we finally implemented the “skimming” function that had to be activated by the respondent. This feature helped manage the sign-based version faster. Conclusion This article revised existing literature toward the design of a new sign language questionnaire. From the revision, the first recommendation is that data collection from end users should be performed in their preferred language and language modality. This is the safest way to avoid any polluted resulting data due to misunderstanding, biased replies, and lack of participation. Moreover, dealing with deaf sign language users means combining the language and culture minority paradigm of the sign language communities. Including sign language in all steps of research to grant full accessibility is a requirement to comply with the human rights of this minority group. In the human rights paradigm, authors generally agree that accessibility plays a central role, either as a human right per se or as an instrument for the fulfillment of human rights (Greco, 2016). Therefore, removing communication barriers and granting accessibility to question content in sign language is a prerequisite in order to conduct ethical research with signing deaf individuals. Yet it is not enough to meet research standards. Specifically in questionnaire-based research, such as surveys or tests, it is equally important to develop robust instruments in terms of question design, format, and mode of administration to reduce affliction and participant burden and measure effects due to the language visual modality. Although the production of video sequences is time-consuming and costly (Gerich & Lehner, 2006), this is a prerequisite for result validity and reliability. We have argued that multiclip VSL questionnaires are currently the best mode of administration for sign-friendly video-CASI. These questionnaires can deliver adapted designs for both signed monolingual and cross-modal bilingual tools. Whereas instruments fully implementing a sign language are necessary when carrying out sign language testing. Further adaptations may be necessary when the questionnaire is designed for surveying a wider range of deaf populations with different communication needs and language preferences. Deaf-friendly research instruments are crucial to any survey research within the social sciences that target a more general population. Deaf-friendly questionnaires should capture the inherent heterogeneity within deaf populations regarding the diversity of communication and literacy skills along with language choice and preference. Failing to do so would entail underrepresentation of deaf people, both due to noncontact (such as in telephone surveying) and a higher rate of nonresponse. The literature review led to the finding that most VSL questionnaires happen in very specific field areas, such as health surveys or sign language testing. VSL questionnaires are not mainstreamed across any democratic citizen participation portal or any general topic. Moreover, differences are even greater when we have a look at the sign languages that have been implemented. The vast majority of computerized VSL questionnaires have been developed in American Sign Language, and just a few are sparsely found in other national sign languages: de Wit (2011) for Dutch Sign Language, Fontaine (2012) for Belgium Sign Language, Gerich and Lehner (2006) for Austrian Sign Language, Kipp et al. (2011) in German Sign Language, Pardo-Guijarro et al., 2015 for Spanish Sign Language, Napier et al. (2018) and Pyfers (2017) in International Sign System, or our questionnaires developed in LSC and Spanish Sign Language. An encouraging exception is found in the field of sign language assessment tests, where the collaboration between researchers and a well-documented tradition of procedures in test adaptation have favored the development of tools in multiple sign languages.2 Cross-modal VSL questionnaires, displaying both a written language and a signed language, enhance the linguistic and social status of both languages represented, thus raising the status of the minority language. Additionally, they introduce new possibilities for universal design questionnaires that can simultaneously include different forms of communication. Cross-modal bilingual designs can thus provide better language accessibility and language choice and avoid or bridge literacy requirements. In fact, advances and innovations on computerized VSL questionnaires are promising and can benefit not only sign language users and researchers. Offering the possibility to choose and combine one screen several language variants, language modalities or other adapted visual communication systems, is a benefit not only for the diverse deaf population but also for the diverse general population. However, there is still much need for research, collaboration, and development. Haug (2015) acknowledged that exploring ICT for sign language testing is a barely researched and indeed almost neglected area. This is still applicable to surveying and general questionnaire design in other field areas too. In our pilot questionnaires we implemented skimming advance and other customizations such as size and speed of the video. These are examples of better adaptations aiming to reduce both time and cognitive burden. However, more research is needed to promote new advances related to video processing and usability for video questionnaires. These advances should be widely available through cost-efficient technological platforms that should be developed collaboratively in research teams involving deaf professionals and consultants at all stages. All advances toward mainstreaming sign language-friendly and deaf-friendly questionnaires will thus contribute to a more diverse and inclusive society not only for deaf populations but also for all citizens. Funding This work was partially funded by the European Commission through the H2020 research program funded projects [HBB4ALL: 621014, ImAC GA: 761974, EasyTV GA: 761999] the Catalan Government funding scheme 2017SGR113. Conflict of Interest No conflicts of interest were reported. Acknowledgments We gratefully acknowledge Vanessa Beroy, Noelia Hernández, and Sílvia Pujol for the translation and adaptation of the signed versions of the questionnaire items; RTVE, the Spanish Radio and Television Corporation, and WEBVISUAL TV for video production; Juan Pedro López-Velasco and Carlos Alberto Martín-Edo for video postproduction and developing the questionnaire technical implementation and delivery system; Idoia Vallverdú-Segura for the graphic design of Figures 2 and 3; FESOCA, the National Association of the Deaf in Catalonia, and CNLSE, the Centre for Linguistic Normalization of Spanish Sign Language, for their collaboration and support. Notes 1 See Gerich and Lehner (2006) and Haug (2015) for a review of the advantages and disadvantages of video-supported CASI compared to traditional noncomputerized data collection methods in the context of the research in sign languages. 2 Visit Sign Language Assessment Instruments hosted by Tobias Haug at http://www.signlang-assessment.info for a fully comprehensive resource; see Lara-Escudero (2017) for an updated short list of the languages available. References Baker-Shenk , C. , & Kyle , J. G. ( 1990 ). Research with deaf people: Issues and conflicts . Disability, Handicap & Society , 5 , 65 – 75 . doi:10.1080/02674649066780051 Google Scholar Crossref Search ADS WorldCat Baker , A. , van den Bogaerde , B. , & Woll , B. ( 2006 ). Methods and procedures in sign language acquisition studies . Sign Language & Linguistics , 8 , 7 – 59 . doi:10.1075/sll.8.1.03bak Google Scholar Crossref Search ADS WorldCat Barnett , S. L. , Matthews , K. A. , Sutter , E. J. , DeWindt , L. A. , Pransky , J. A. , O’Hearn , A. M. , & Pearson , T. A. ( 2017 ). Collaboration with deaf communities to conduct accessible health surveillance . American Journal of Preventive Medicine , 52 , 250 – S254 . doi:10.1016/j.amepre.2016.10.011. Link to American Sign Language video adaptation of this article retrieved from https://www.urmc.rochester.edu/ncdhr/research/current-research-2/deaf-health-survey-2013/ajpm-video.aspx Google Scholar Crossref Search ADS WorldCat Benedict , B. S. , & Sass-Lehrer , M. ( 2007 ). Deaf hearing partnerships: Ethical and communication considerations . American Annals of the Deaf , 152 , 275 – 282 . doi:10.1353/aad.2007.0023 Google Scholar Crossref Search ADS PubMed WorldCat Berghs , M. , Atkin , K. , Graham , H. , Hatton , C. , & Thomas , C. ( 2016 ). Implications for public health research of models and theories of disability: A scoping study and evidence synthesis . Public Health Research , 4 , 1 – 166 . doi:10.3310/phr04080 Google Scholar Crossref Search ADS WorldCat Bernabé , R. , & Orero , P. ( 2019 forthcoming ). Easy to Read as multimode accessibility service . Hermeneus , 21 . WorldCat Bosch-Baliarda , M. , Soler-Vilageliu , O. , & Orero , P. ( 2019 ). TV screen exploration in sign language users . Unpublished manuscript . Universitat Autònoma de Barcelona, Bellaterra, Catalonia . Google Preview WorldCat COPAC Boynton , P. M. , & Greenhalgh , T. ( 2004 ). Hands-on guide to questionnaire research: Selecting, designing, and developing your questionnaire . BMJ , 328 , 1312 – 1315 . doi:10.1136/bmj.328.7451.1312 Google Scholar Crossref Search ADS PubMed WorldCat Brauer , B. A. ( 1993 ). Adequacy of a translation of the MMPI into American Sign Language for use with deaf individuals: Linguistic equivalency issues . Rehabilitation Psychology , 38 , 247 – 260 . doi:10.1037/h0080302 Google Scholar Crossref Search ADS WorldCat Cavender , A. , Ladner , R. E. , & Riskin , E. A. ( 2006 ). MobileASL: Intelligibility of sign language video as constrained by mobile phone technology. In Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility—Assets ‘06 (Vol. 71 ). doi:10.1145/1168987.1169001 . Portland, Oregon, USA Cawthon , S. W. ( 2017 ). Large-scale survey design in deaf education research. In S. W. Cawthon & C. Lou Garberoglio (Eds.), Research in deaf education: Contexts, challenges, and considerations (pp. 1 – 28 ). New York, NY : Oxford University Press . doi:10.1093/oso/9780190455651.001.0001 . Google Preview WorldCat COPAC Cawthon , S. W. , & Garberoglio , C. L. ( 2017 ). Research in deaf education: Contexts, challenges, and considerations . New York, NY : Oxford University Press . doi:10.1093/oso/9780190455651.001.0001 . Google Preview WorldCat COPAC Choi , B. C. K. , & Pak , A. W. P. ( 2005 ). A catalog of biases in questionnaires . Preventing Chronic Disease , 2 , A13 . doi:A13 Google Scholar PubMed WorldCat Cohen , H. , & Jones , E. G. ( 1990 ). Interpreting for cross-cultural research: Changing written English to American Sign Language . Journal of the American Deafness and Rehabilitation Association , 24 , 41 – 48 . WorldCat De Meulder , M. , Krausneker , V. , Turner , G. H. , & Conama , J. B. ( 2018 ). Sign language communities. In G. Hogan-Brun & B. O’Rourke (Eds.), The handbook of minority languages and communities (pp. 207 – 232 ). London, UK : Palgrave Macmillan . Google Preview WorldCat COPAC De Wit , M. ( 2011 ). A sign language interpreter in inclusive education: The view of deaf persons on their quality of life (Master’s thesis) . Edinburgh, Scotland : Heriot-Watt University . Google Preview WorldCat COPAC Dillman , D. A. & Christian , L. M. ( 2005 ). Survey mode as a source of instability in responses across surveys . Field Methods 17 , 30 – 51 . doi:10.1177/1525822X04269550 . Google Preview WorldCat COPAC Enns , C. J. , Haug , T. , Herman , R. , Hoffmeister , R. , Mann , W. , & McQuarrie , L. ( 2016 ). Exploring signed language assessment tools around the world. In M. Marschark , V. Lampropoulou & E. K. Skordilis (Eds.), Diversity in deaf education (pp. 171 – 218 ). New York, NY : Oxford University Press . Google Preview WorldCat COPAC Enns , C. J. , & Herman , R. C. ( 2011 ). Adapting the assessing British Sign Language development: Receptive skills test into American Sign Language . Journal of Deaf Studies and Deaf Education , 16 , 362 – 374 . doi:10.1093/deafed/enr004 Google Scholar Crossref Search ADS PubMed WorldCat Erens , B. ( 2013 ). Designing high-quality surveys of ethnic minority groups in the United Kingdom. In J. Font & M. Méndez (Eds.), Surveying ethnic minorities and immigrant populations. Methodological challenges and researches strategies (pp. 45 – 68 ). Amsterdam, the Netherlands : Amsterdam University Press . Google Preview WorldCat COPAC Ewart , J. , & Snowden , C. ( 2012 ). The media’s role in social inclusion and exclusion . Media International Australia , 142 , 61 – 63 . doi:10.1177/1329878X1214200108 Google Scholar Crossref Search ADS WorldCat Fontaine , S. ( 2012 ). Surveying populations with disabilities. Specific mixed-mode methodologies to include sensory disabled people in quantitative surveys . Paper presented at the . American Statistical Association, Survey Methods for Hard to Reach Populations, . New Orleans, Louisiana, October–November : International Conference on Methods for Surveying and Enumerating Hard-to-Reach Populations . Google Preview WorldCat COPAC Freeman , S. T. ( 1989 ). Cultural and linguistic bias in mental health evaluations of deaf people . Rehabilitation Psychology , 34 , 51 – 63 . doi:10.1037/h0091705 WorldCat Gerich , J. , & Lehner , R. ( 2006 ). Video computer-assisted self-administered interviews for deaf respondents . Field Methods , 18 , 267 – 283 . doi:10.1177/1525822X06287535 Google Scholar Crossref Search ADS WorldCat Goldstein , M. E. , Eckhardt , E. A. , & Joyner , P. ( 2004 ). The inclusion of deaf individuals in survey research . Paper presented at the Washington, DC, April : Federal Interagency Committee on Disability Research Conference on Best Practices for Surveying People with Disabilities . Interagency Commitee on Disability Research Google Preview WorldCat COPAC Goldstein , M. F. , Eckhardt , E. A. , Joyner-Creamer , P. , Berry , R. , Paradise , H. , & Cleland , C. M. ( 2010 ). What do deaf high school students know about HIV? AIDS Education and Prevention , 22 , 523 – 537 . doi:10.1521/aeap.2010.22.6.523 Google Scholar Crossref Search ADS PubMed WorldCat Graybill , P. , Aggas , J. , Dean , R. K. , Demers , S. , Finigan , E. G. , & Pollard , R. Q. ( 2010 ). A community-participatory approach to adapting survey items for deaf individuals and American Sign Language . Field Methods , 22 , 429 – 448 . doi:10.1177/1525822X10379201 Google Scholar Crossref Search ADS WorldCat Greco , G. M. ( 2016 ). On accessibility as a human right, with an application to media accessibility. In A. Matamala & P. Orero (Eds.), Researching audio description (pp. 11 – 33 ). London, UK : Palgrave Macmillan . Google Preview WorldCat COPAC Guardino , C. , & Cannon , J. E. ( 2016 ). Deafness and diversity: Reflections and directions . American Annals of the Deaf , 161 , 104 – 112 . doi:10.1353/aad.2016.0016 Google Scholar Crossref Search ADS PubMed WorldCat Gutman , V. ( 2005 ). Ethical reasoning and mental health services with deaf clients . Journal of Deaf Studies and Deaf Education , 10 , 171 – 183 . doi:10.1093/deafed/eni017 Google Scholar Crossref Search ADS PubMed WorldCat Hansen , E. G. , Loew , R. C. , Laitusis , C. C. , Kushalnagar , P. , Pagliaro , C. M. , & Kurz , C. ( 2018 ). Usability of American Sign Language videos for presenting mathematics assessment content . Journal of Deaf Studies and Deaf Education , 23 , 284 – 294 . doi:10.1093/deafed/eny008 Google Scholar Crossref Search ADS PubMed WorldCat Harkness , J. A. ( 2008 ). Comparative survey research: Goal and challenges. In E. D. de Leeuw , J. J. Hox & D. A. Dillman (Eds.), International handbook of survey methodology (pp. 56 – 77 ). London, UK : Routledge . Google Preview WorldCat COPAC Harris , R. , Holmes , H. M. , & Mertens , D. M. ( 2009 ). Research ethics in sign language communities . Sign Language Studies , 9 , 104 – 131 . doi:10.1353/sls.0.0011 Google Scholar Crossref Search ADS WorldCat Haug , T. ( 2011 ). Adaptation and evaluation of a German Sign Language test. A computer-based receptive skills test for deaf children ages 4–8 years old . Hamburg, Germany : Hamburg University Press . Google Preview WorldCat COPAC Haug , T. ( 2015 ). Use of information and communication technologies in sign language test development: Results of an international survey . Deafness and Education International , 17 , 33 – 48 . doi:10.1179/1557069X14Y.0000000041 Google Scholar Crossref Search ADS WorldCat Haug , T. , Herman , R. , & Woll , B. ( 2015 ). Constructing an online test framework, using the example of a sign language receptive skills test . Deafness and Education International , 17 , 3 – 7 . doi:10.1179/1557069X14Y.0000000035 Google Scholar Crossref Search ADS WorldCat Haug , T. , & Mann , W. ( 2008 ). Adapting tests of sign language assessment for other sign languages-a review of linguistic, cultural, and psychometric problems . Journal of Deaf Studies and Deaf Education , 13 , 138 – 147 . doi:10.1093/deafed/enm027 Google Scholar Crossref Search ADS PubMed WorldCat Hauser , P. C. , Paludneviciene , R. , Riddle , W. , Kurz , K. B. , Emmorey , K. , & Contreras , J. ( 2016 ). American Sign Language comprehension test: A tool for sign language researchers . Journal of Deaf Studies and Deaf Education , 21 , 64 – 69 . doi:10.1093/deafed/env051 Google Scholar Crossref Search ADS PubMed WorldCat Herman , R. C. , Holmes , S. , & Woll , B. ( 1999 ). Assessing BSL development—Receptive skills test . Coleford, UK : The Forest Bookshop . Google Preview WorldCat COPAC Hermans , D. , Knoors , H. , & Verhoeven , L. ( 2010 ). Assessment of sign language development: The case of deaf children in the Netherlands . Journal of Deaf Studies and Deaf Education , 15 , 107 – 119 . doi:10.1093/deafed/enp030 Google Scholar Crossref Search ADS PubMed WorldCat Higgins , J. , Famularo , L. , Bownman , T. , & Hall , R. ( 2015 ). Research and development of audio and American Sign Language guidelines for creating accessible computer-based assessments . Retrieved from http://gaap.measuredprogress.org/gaap/ . Huenerfauth , M. , & Kacorri , H. ( 2015 ). Best practices for conducting evaluations of sign language animation . Journal on Technology and Persons with Disabilities , 3 , 20 – 32 . Retrieved from http://scholarworks.rit.edu/article/1787 WorldCat Hussein , K. Q. , & Al-Bayati , M. A. ( 2009 ). A multiple kinds of e-exam system for the deaf & dumb: Developing & evaluating “view points of experts in review” . International Journal of Computer Science and Network Security , 9 , 309 – 317 . Retrieved from http://paper.ijcsns.org/07_book/200905/20090541.pdf WorldCat Kappelhof , J. ( 2015 ). Surveying ethnic minorities: The impact of survey design on data quality . Den Haag, the Netherlands : SCP Publications . Retrieved from http://www.scp.nl/Publicaties/Alle_publicaties/Publicaties_2015/Surveying_ethnic_minorities . Google Preview WorldCat COPAC Kipp , M. , Nguyen , Q. , Heloir , A. , & Matthes , S. ( 2011 ). Assessing the deaf user perspective on sign language avatars . Assets ‘11 (Xiii) , 107 – 114 . doi:10.1145/2049536.2049557 WorldCat Ladd , P. ( 2003 ). Understanding deaf culture . Clevedon, UK : Multilingual Matters . Google Preview WorldCat COPAC Lang , C. ( 2015 ). Language use at RID conferences: A survey on behaviors and perceptions . Journal of Interpretation , 24 . Retrieved from https://digitalcommons.unf.edu/joi/vol24/iss1/4 WorldCat Lara-Escudero , C. ( 2017 ). Adapting the British Sign Language receptive skills test into Catalan Sign Language: Preliminary studies ( Master’s thesis ). Barcelona, Catalonia : University of Barcelona . Google Preview WorldCat COPAC Lipton , D. S. , Goldstein , M. F. , Fahnbulleh , F. W. , & Gertz , E. N. ( 1996 ). The interactive video-questionnaire: A new technology for interviewing deaf persons . American Annals of the Deaf , 141 , 370 – 378 . Google Scholar Crossref Search ADS PubMed WorldCat López , J. P. , Bosch-Baliarda , M. , Martín , C. A. , Menéndez , J. M. , Orero , P. , Soler-Vilageliu , O. , & Álvarez , F. ( 2019 ). Design and development of sign language questionnaires based on video and web interfaces . Manuscript submitted for publication . Lucas , C. , Mirus , G. , Palmer , J. L. , Roessler , N. J. , & Frost , A. ( 2013 ). The effect of new technologies on sign language research . Sign Language Studies , 13 , 541 – 564 . doi:10.1353/sls.2013.0018 Google Scholar Crossref Search ADS WorldCat Maiorana-Basas , M. , & Pagliaro , C. M. ( 2014 ). Technology use among adults who are deaf and hard of hearing: A national survey . Journal of Deaf Studies and Deaf Education , 19 , 400 – 410 . doi:10.1093/deafed/enu005 Google Scholar Crossref Search ADS PubMed WorldCat Mann , W. , Roy , P. , & Morgan , G. ( 2016 ). Adaptation of a vocabulary test from British Sign Language to American Sign Language . Language Testing , 33 , 3 – 22 . doi:10.1177/0265532215575627 Google Scholar Crossref Search ADS WorldCat McKee , M. , Schlehofer , D. , & Thew , D. ( 2013 ). Ethical issues in conducting research with deaf populations . American Journal of Public Health , 103 , 2174 – 2178 . doi:10.2105/AJPH.2013.301343 Google Scholar Crossref Search ADS PubMed WorldCat Mertens , D. M. ( 2005 ). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (2nd Ed., ). Thousand Oaks, CA : Sage . Google Preview WorldCat COPAC Mertens , D. M. ( 2010 ). Transformative mixed methods research . Qualitative Inquiry , 16 , 469 – 474 . doi:10.1177/1077800410364612 Google Scholar Crossref Search ADS WorldCat Mertens , D. M. , Sullivan , M. , & Stace , H. ( 2011 ). Disability communities: Transformative research and social justice. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (4th Ed., ). Thousand Oaks, CA : Sage . Google Preview WorldCat COPAC Meurant , L. , Sinte , A. , Vermeerbergen , M. , & Van Herreweghe , M. ( 2013 ). Sign language research, uses and practices: A Belgian perspective. In L. Meurant , A. Sinte , M. Van Herreweghe & M. Vermeerbergen (Eds.), Sign language research, uses and practices: Crossing views on theoretical and applied sign language linguistics (pp. 1 – 14 ). Berlin, Germany : De Gruyter Mouton . Google Preview WorldCat COPAC Munger , K. M. , & Mertens , D. M. ( 2011 ). Conducting research with the disability community: A rights-based approach. In T. S. Rocco (Ed.), Challenging ableism, understanding disability, including adults with disabilities in workplaces and learning spaces: New directions for adult and continuing education (pp. 23 – 33 ). New York, NY : Wiley Periodicals . doi:10.1002/ace . Google Preview WorldCat COPAC Napier , J. , Lloyd , K. , Skinner , R. , Turner , G. H. , & Wheatley , M. ( 2018 ). Using video technology to engage deaf sign language users in survey research: An example from the Insign project . The International Journal for Translation & Interpreting Research , 10 , 101 – 121 . doi:10.12807/ti.110202.2018.a08 WorldCat Orero , P. ( 2017 ). The professional profile of the expert in media accessibility for the scenic arts . Rivista Internazionale di Tecnica della Traduzione/International Journal of Translation , 19 , 143 – 161 . doi:10.13137/2421-6763/17356 WorldCat Orfanidou , E. , Woll , B. , & Morgan , G. (Eds.) ( 2015 ). Research methods in sign language studies: A practical guide . London, UK : Wiley-Blackwell . Google Preview WorldCat COPAC Pardo-Guijarro , M. J. , Martinez-Andres , M. , Notario-Pacheco , B. , Solera-Martinez , M. , Sanchez-Lopez , M. , & Martinez-Vizcaino , V. ( 2015 ). Self-reports versus parental perceptions of health-related quality of life among deaf children and adolescents . Journal of Deaf Studies and Deaf Education , 20 , 275 – 282 . doi:10.1093/deafed/env018 Google Scholar Crossref Search ADS PubMed WorldCat Pollard , R. ( 2002 ). Ethics in mental health and deafness. In V. Gutman (Ed.), Ethical conduct in research involving deaf people (pp. 162 – 178 ). Washington, DC : Gallaudet University Press . Google Preview WorldCat COPAC Pollard , R. Q. , Dean , R. K. , O’Hearn , A. M. , & Haynes , S. L. ( 2009 ). Adapting health education material for deaf audiences . Rehabilitation Psychology , 54 , 232 – 238 . Retrieved from https://www.northeastern.edu/cali/wp-content/uploads/2017/03/Adapting-health-education-material-for-deaf-audiences.pdf Google Scholar Crossref Search ADS PubMed WorldCat Pyfers , L. (Ed.) ( 2017 ). SignTeach: The survey. In Sign language teaching in Europe. Report & recommendations . Brussels, Belgium : European Union of the Deaf . Retrieved from https://www.signteach.eu/index.php/4-signteach-the-survey . Google Preview WorldCat COPAC Rojba , F. ( 2016 ). Fieldwork conducted by deaf sign language users. In I. Lathinen & P. Rainò (Eds.), Deaf people in Albania in 2015. A survey study (pp. 16 – 20 ). Tirana, Albania : Finnish Association of the Deaf . Google Preview WorldCat COPAC Romero-Fresco , P. ( 2012 ). Joining the dots . The Journal of Specialised Translation , 20 . Retrieved from http://www.jostrans.org/issue20/int_romero.php WorldCat Samar , V. J. , Barnett , S. , Oyzon , E. , Mowl , C. , & Sutter , E. ( 2012 ). Modality-independent survey tool: Imagine the potential . NTID Research Bulletin , 15 , 1 – 5 . WorldCat Sign Language Linguistics Society ( 2016 ). SLLS Ethics Statement for Sign Language Research . Retrieved from https://slls.eu/slls-ethics-statement/ . Google Preview WorldCat COPAC Singleton , J. , Jones , G. , & Hanumantha , S. ( 2012 ). Deaf friendly research? Toward ethical practice in research involving deaf participants . Deaf Studies Digital Journal , 3 . Retrieved from http://dsdj.gallaudet.edu/index.php?view=entry&issue=4&entry_id=123 WorldCat Singleton , J. L. , Martin , A. , & Morgan , G. ( 2015 ). Ethics, deaf-friendly research, and good practice when studying sign languages. In E. Orfanidou , B. Woll & G. Morgan (Eds.), Research methods in sign language studies: A practical guide First Edition (pp. 7 – 20 ). London, UK : Wiley-Blackwell . Google Preview WorldCat COPAC Sloan , M. , Wright , D. , & Barrett , K. ( 2012 ). Data comparability in a mixed mode telephone and face to face survey of persons with disabilities . Mathematica Policy Research Reports 51e4878b2264c7793ccb74dd, Mathematica Policy Research . Retrieved from: https://ideas.repec.org/p/mpr/mprres/51e48478b2264c7793ccb74ddc73f3f3.html Google Preview WorldCat COPAC Survey Research Center ( 2016 ). Guidelines for best practice in cross-cultural surveys. (S. R. C. Institute for Social Research, Ed.) (4th Ed., ). Ann Arbor, MI : University of Michigan . Retrieved from http://www.ccsg.isr.umich.edu/ . Google Preview WorldCat COPAC Tran , J. J. ( 2014 ). Human-centered optimization of mobile sign language video communication (Doctoral dissertation) . Washington, DC : University of Washington . Google Preview WorldCat COPAC Tran , J. J. , Kim , J. , Riskin , A. , Ladner , R. E. , & Jacob , O. W. ( 2011 ). Evaluating quality and comprehension of real-time sign language video on mobile phones. In ACM SIGACCESS Conference on Computers and Accessibility (pp. 115 – 122 ). New York, NY : ACM . Google Preview WorldCat COPAC Tran , J. J. , Riskin , E. A. , Ladner , R. E. , & Wobbrock , J. O. ( 2015 ). Evaluating intelligibility and battery drain of mobile sign language video transmitted at low frame rates and bit rates . ACM Transactions on Accessible Computing , 7 , 1 – 26 . doi:10.1145/2797142 Google Scholar Crossref Search ADS WorldCat Woodward , J. C. ( 1973 ). Interrule implication in American Sign Language . Sign Language Studies , 3 , 47 – 56 . doi:10.1353/sls.1973.0010 Google Scholar Crossref Search ADS WorldCat Woodward , J. C. , & DeSantis , S. ( 1977 ). Two to one it happens: Dynamic phonology in two sign languages . Sign Language Studies , 17 , 329 – 346 . doi:10.1353/sls.1977.0013 Google Scholar Crossref Search ADS WorldCat Young , A. , & Hunt , R. ( 2011 ). Research with d/Deaf sign language users . London, UK : NIHR School for Social Care Research . Retrieved from www.lse.ac.uk/LSEHealthAndSocialCare/pdf/SSCR%20Methods%20Review_9_web.pdf . Google Preview WorldCat COPAC Young , A. , & Temple , B. ( 2014 ). Approaches to social research: The case of deaf studies . New York, NY : Oxford University Press . Google Preview WorldCat COPAC © The Author(s) 2019. Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Toward a Sign Language-Friendly Questionnaire Design
JO - The Journal of Deaf Studies and Deaf Education
DO - 10.1093/deafed/enz021
DA - 2019-10-01
UR - https://www.deepdyve.com/lp/oxford-university-press/toward-a-sign-language-friendly-questionnaire-design-CELtex27jF
SP - 333
VL - 24
IS - 4
DP - DeepDyve
ER -