Evaluation & the Health Professions

Bausell, R. Barker

doi: 10.1177/016327878200500301pmid: N/A

Slotnick, Henry B.

doi: 10.1177/016327878200500302pmid: 10261554

Best Liked/Least Liked (BLILL) is a straightforward technique for collecting information used in evaluation. The technique is composed of stages (data collection, data analysis, and data interpretation) that are characterized by the ease with which they are accomplished. Further, the technique admits of flexibility' in both range of applicability (some achievement testing and the measuring of effect) and scope of use (it can be narrowly focused or used to collect information on unintended outcomes). The technique is described here and examples are presented of its use both narrowly and broadly focused, and in service of both achievement testing and affective measurement.

Petersen, Cheryl Bowser

doi: 10.1177/016327878200500303pmid: 10261555

This article represents the results of an evaluation of a continuing education program for dietitians. The evaluation was built into the program at its inception and involved an assessment of participant satisfaction, short-term knowledge gain and a follow-up assessment of self-reported behavior change. The results of this evaluation indicate that the course did, indeed, impact on both knowledge and behavior. In addition, however, the evaluation also uncovered some problems in planning which minimized the potential impact of this course. These results suggest that there should be increased attention to rigorous educational needs assessment and both process and impact program evaluations in order to determine approaches to education that could make more effective use of the resources availablefor continuing education in health care.

Niebuhr, Bruce R.; Lanier, Roger A.

doi: 10.1177/016327878200500304pmid: 10314573

Physician's assistant educational programs have used surveys of their graduates as one method of evaluating educational objectives and curricula. A concern is the validitY' of phisician's assistant self-ratings as measures of job performance. Ratings by supervising phy sicians have been suggested as more valid measures. In the present stud) ratings of physician's assistants and their supervising phy sicians were compared. Physician's assistants and their supervising physicians were interviewed using an interview instrument developed to cover the performance of the physician's assistant in the major activities ofprimary' care practice. While the physicians and physician's assistants disagreed on several measures, in all cases the ratings of the physician's assistants were more conservative. Thus, the physician's assistants did not show any tendency to inflate ratings of their own performance.

Repicky, Paul A.; Mendenhall, Robert C.; Neville, Richard E.

doi: 10.1177/016327878200500305pmid: 10314574

This article describes the methodology and findings of a national survey of Physician's Assistants (PAs) in adult, ambulatory care practices. Data on patient care roles and other professional activities were collected for a three-day period via a comprehensive self-reporting, log-diary instrument. Completing the instrument were 356 (50.4%o) PAs. Survey results address thefollowing questions: What is the typical work week for PAs? How do PAs allocate their time in a professional day? What direct patient care services do PAs provide? How productive are PAs with respect to number of patients seen and dollar income generated? In general, the data are consistent with the PA role model of a primary health care professional who provides basic health care services with an emphasis on patient counseling and disease prevention.

Majchrzak, Ann

doi: 10.1177/016327878200500306pmid: 10299134

In an attempt to understand the role that program evaluation plays in a community mental health center (CMHC), a conceptual framework of the organizational context in which CMHC self-evaluation efforts occur is presented. The conceptual framework consists of three major propositions. (I) Five dimensions of a CMHC are proposed to have significant effects on the center's evaluation process: research capabilitY, administrative support for evaluation, organizational structure, readiness for innovation, and the environmental context of the center. (2) The evaluation process is perceived to consist of three tYpes of evaluation activities corresponding to the input, process, and output elements of the center. (3) The five organizational parameters are proposed to differentially affect the individual components of the evaluation process.Stemming from the proposed conceptual framework, a theoretical model of the effects of organizational parameters on the evaluation process was tested on a purposive sample of CMHCs. The results were generally supportive of the model, indicating that the role of evaluation in CMHCs is not a simple or singular phenomenon. Specific conclusions and implications for policy and research arenas are discussed.

Weber, David J.; Hamer, Robert M.

doi: 10.1177/016327878200500307pmid: N/A

This study investigated the relationship of three medical college policies and procedures on the results of Part I of the standardized National Board of Medical Examiners (NBME) medical college examination. The three variables included (1) whether a school required its students to pass Part Ifor promotion or required its students to pass Part Ito record a score, (2) whether or not a school offered or recommended a review course for Part I, and (3) the number of days between the end of formal classwork and the administration of Part I. Correlation studies between these variables and the schools' performance on NBME Part I were conducted using the schools' mean Science MCA T scores as a covariate. A statistically significant result was found only for the review course variable. The covariate, Science MCA T, accounted for over 50% of the variance on Part I. The results indicate that a school policy of offering or recommending a review course for Part I is associated with a small but statistically significant increment in Part I scores.

Boodoo, Gwyneth M.; O'Sullivan, Patricia

doi: 10.1177/016327878200500308pmid: 10261556

This article discusses the determination of reliability coefficients for clinical evaluation forms by use of generalizability theory. This method is contrasted to some of the commonly used measures of interobserver agreement, interobserver reliability, and classical reliability. The value of this method in terms of making recommendations, such as the number ofjudges and the number of items on a form, in order to improve reliability is illustrated by two examples. Also, the difference between norm-referenced decisions and criterion-referenced decisions in determining reliability with this method is presented.