Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Routine measurement of outcomes in Australia's public sector mental health services

Routine measurement of outcomes in Australia's public sector mental health services Objective: This paper describes the Australian experience to date with a national 'roll out' of routine outcome measurement in public sector mental health services. Methods: Consultations were held with 123 stakeholders representing a range of roles. Results: Australia has made an impressive start to nationally implementing routine outcome measurement in mental health services, although it still has a long way to go. All States/Territories have established data collection systems, although some are more streamlined than others. Significant numbers of clinicians and managers have been trained in the use of routine outcome measures, and thought is now being given to ongoing training strategies. Outcome measurement is now occurring 'on the ground'; all States/Territories will be reporting data for 2003–04, and a number have been doing so for several years. Having said this, there is considerable variability regarding data coverage, completeness and compliance. Some States/Territories have gone to considerable lengths to 'embed' outcome measurement in day-to-day practice. To date, reporting of outcome data has largely been limited to reports profiling individual consumers and/or aggregate reports that focus on compliance and data quality issues, although a few States/Territories have begun to turn their attention to producing aggregate reports of consumers by clinician, team or service. Conclusion: Routine outcome measurement is possible if it is supported by a co-ordinated, strategic approach and strong leadership, and there is commitment from clinicians and managers. The Australian experience can provide lessons for other countries. Page 1 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 informed use of casemix to understand the role of pro- Introduction Internationally, there is an increasing emphasis on rou- vider variation in differences between agencies' costs and tine outcome measurement in mental health. A push to outcomes. For the purposes of the current paper, however, improve quality of care for consumers has prompted the focus is routine outcome measurement only, rather interest in monitoring outcomes at an individual level, than casemix development. and financial pressures and a need to demonstrate value- for-money have led to the use of aggregate reports that For its part, the Australian Government has established allow comparisons between services [1]. In the United three Expert Groups (Adult, Child/Adolescent, and Older States, there are examples of routine outcome measure- Persons) to advise on the implementation and use of rou- ment being 'rolled-out' across mental health services in tine outcome data in mental health services. It has also entire states, such as the Ohio Mental Health Consumer provided resources to support training in the use of out- Outcomes System [2]. In Europe, there are also some come measures, and arrangements to receive, process, examples of individual services monitoring outcomes, as analyse and report on outcome data submitted by States/ in the South Verona Outcomes Project [3] and the Territories. MECCA Study [4], but the routine collection of outcome data has not extended to larger areas. The latter arrangements have been established through the 'Australian Mental Health Outcomes and Classifica- Australia's commitment to routine outcome measure- tion Network' (AMHOCN), a consortium contracted from ment is evidenced in its National Mental Health Strategy late 2003 to provide national leadership in the develop- [5-7]. Since its inception in 1992, the continued improve- ment of outcome measurement in mental health. ment of the quality and effectiveness of the treatment of AMHOCN is pursuing a work program with three compo- people with a mental illness has been a key objective of nents, each being undertaken by a different member of the Strategy. The Strategy has recognised that this objec- the consortium: data management (Strategic Data Pty Ltd, tive can only be achieved through the development of Victoria); analysis and reporting (Queensland Centre for sound information to support service planning and deliv- Mental Health Services Research, The University of ery, and consequently the systematic implementation of Queensland, Queensland); and training and service devel- routine outcome measurement in all public sector mental opment (New South Wales Institute of Psychiatry, New health services is one of its priorities. South Wales). State/Territory governments and the Australian Govern- An immediate concern for AMHOCN was determining ment are collaborating in a coherent national approach. States/Territories' progress with respect to 'rolling out' For their part, all States/Territories have signed Agree- routine outcome measurement, so each State/Territory ments that require them to routinely submit two sets of was invited to participate in a consultation with data from public sector mental health services to the Aus- AMHOCN. This paper reports on the findings from these tralian Government. Firstly, they are required to submit consultations. de-identified, patient-level outcome data, referred to as the 'National Outcomes and Casemix Collection' Method (NOCC) [8]. These outcome data are collected via a range The consultations occurred in March/April 2004. The intention was to seek a range of views, rather than to try to of instruments that incorporate clinician and consumer perspectives on a range of mental health related constructs achieve a representative sample, and States/Territories (e.g., symptomatology, level of functioning, degree of dis- were asked to nominate relevant stakeholders. They could ability) relevant to adults, children/adolescents and older choose whomever they wished, but they were advised to people. The idea is that administration of these instru- consider including policy-makers and technical personnel ments at specific points in time will allow services to mon- from central mental health units and mainstream health itor changes in individual consumers and in groups of information sections, as well as consumers/carers. Most consumers, and, ultimately, to make comparisons with States/Territories sent representatives from all of these similar consumers in like services. Table 1 shows the spe- groups, and many also sent service managers, clinicians, cific instruments that comprise the NOCC dataset. individuals responsible for supporting routine outcome measurement at a site level, and members of the Expert Secondly, States/Territories are required to submit data on Groups. In total, 123 individuals attended the consulta- inpatient episodes of care and community contacts, tions: 10 from New South Wales; 21 from Victoria; 28 termed the 'National Minimum Data Set – Mental Health from Queensland; 23 from Western Australia; 17 from Care' (NMDS) [9-11]. These data provide information on South Australia; six from both Tasmania and the Austral- resource use by consumers, and, when combined with the ian Capital Territory; and 12 from the Northern Territory. above outcome data will promote the development and Page 2 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 Table 1: Data comprising the NOCC collection Adults Older persons Children and adolescents Clinician-rated Principal and additional diagnoses √√ √ Mental health legal status √√ √ Health of the Nation Outcomes Scale (HoNOS) [16] √ Health of the Nation Outcomes Scale 65+ (HoNOS65+) [17] √ Health of the Nation Outcomes Scale for Children and Adolescents √ (HoNOSCA) [18] Life Skills Profile 16 (LSP-16) [19, 20] √√ Resource Utilisation Groups – Activities of Daily Living Scale (RUG- √ ADL) [21] Focus of Care [20] √√ Children's Global Assessment Scale (CGAS) [22] √ Factors Influencing Health Status (FIHS) [20] √ Consumer-rated Mental Health Inventory (MHI) [23] or Behaviour and Symptom √√ Identification Scale (BASIS-32) [24] or Kessler-10 Plus (K-10+) [25] Consumer- and parent- Strengths and Difficulties Questionnaire (SDQ)[26] √ rated Source: Department of Health and Ageing (2003) [27] Many 'wore several hats', rendering it difficult to provide Results States/Territories' progress regarding data collection a breakdown of their roles. systems and infrastructure The consultations sought answers to questions regarding Ultimately, States/Territories are aiming to have stream- progress in four domains: (a) data collection systems and lined data collection systems that allow the outcome data infrastructure; (b) training and retraining of staff; (c) the collected via the NOCC dataset to be linked to the admit- implementation of routine outcome measurement; and ted and non-admitted activity data in the NMDS. This will (d) and analysis, reporting and use of data. allow outcome data to be 'attached' to given inpatient and community episodes of care. This has advantages in terms The majority of consultations took place over a full day, of allowing outcomes for consumers to be 'tracked' across with the shortest being half a day. All of the consultations episodes, and is necessary for progressing casemix devel- began with a brief presentation from AMHOCN, and then opment work that requires outcome data and resource use elicited information from participants. Some States/Terri- data to be combined within episodes. tories chose to split the consultation in two, inviting pol- icy makers, planners and clinicians to attend one session All States/Territories have developed data collection sys- and technical personnel to attend the other. Some States/ tems, or are in the final stages of doing so. For some, this Territories gave formal presentations responding to spe- has involved 'starting from scratch'; for others it has cific questions; others took a more informal approach. In required modifications to existing systems. For example, some cases, the information presented at the consulta- the systems used in Queensland to capture admitted and tions was supplemented by a written response. non-admitted NMDS information did not have the func- tionality to incorporate outcome measures, so an addi- Each consultation was transcribed. The transcription was tional system was developed to do so. By contrast, in the combined with any other written material (e.g., formal Australian Capital Territory, the system used by all com- responses and presentations), and examined at a global munity teams to collect non-admitted NMDS data, was level to identify major themes within each domain. Indi- modified to collect outcome data and extended to inpa- vidual responses were classified according to these tient services, where it runs alongside a separate patient themes. Each State/Territory was given the opportunity to administration system for the collection of inpatient activ- comment on the accuracy of the written interpretation of ity data. the consultation. States/Territories differ in terms of the number of systems that are currently involved in the collection of routine out- come data. The simplest scenario is one where outcome measurement functionality has been added to an existing Page 3 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 system for recording activity in community mental health This figure is consistent with that of 10,000 reported by settings, and has been extended into inpatient settings (as the Department of Health and Ageing, which is estimated with the system in the Australian Capital Territory, to represent approximately 60% the public sector mental described above). This also occurs in Victoria, Tasmania health workforce [12]. and the Northern Territory. Other States rely on as many as four statewide systems to collect NOCC and NMDS The direct training approach is seen as having the benefit information, sometimes with further degrees of complex- of consistency, while the train-the-trainer approach is ity between areas or metropolitan/country settings. seen as fostering capacity building and being less labour intensive and cheaper. Some States/Territories have con- Linking NOCC and admitted and non-admitted NMDS sidered accrediting trainers, so that the advantages of both datasets is impeded in most States/Territories by the lack approaches can be combined. Managers are also more of a unique identifier. Typically, linkage is only possible commonly being recruited as trainers, as part of a move to for parts of the data (usually NOCC and non-admitted secure their commitment in leading the change process. NMDS data) and/or by conducting quite complex record South Australia has been innovative here, building capac- linkage tasks. The exception is the Northern Territory, ity by training staff as trainers through the Certificate 4 in which has a client master index that allocates each con- Workplace Training and Assessment, and investing in sumer a unique identifier that allows him/her to be training in content knowledge around outcome measures 'tracked' across episodes, across services, and over time. in the NOCC collection. In this way, South Australia has Other States/Territories are working towards improve- addressed some of the difficulties inherent in more stand- ments, but have some way to go. Western Australia's data ard train-the-trainer approaches. collection system has a unique identifier that will allow episodes of care to be attributed to the same individual, Many States/Territories are now beginning to consider regardless of location or time, but its 'roll-out' is not yet issues of ongoing training and support. High levels of staff completed. Queensland and New South Wales have plans turnover in some States/Territories mean that there are to reconcile their unique identifier systems via specific new staff who have not been trained, and lags between projects. This will mean that States will assign a unique training and implementation in some jurisdictions have identifier to a given individual that he or she will 'carry' resulted in skills being lost. In addition, many States/Ter- across all health services, including mental health services, ritories are recognising the need for a second wave of but this will not occur in the near future. training that goes beyond how to use the outcome meas- ures and focuses more on how to interpret the results of States/Territories have differing levels of infrastructure to specific measures (at individual and aggregate levels). support the NOCC and NMDS collections. Human resources vary, with some States/Territories having a Some States/Territories have implemented ongoing train- number of personnel deployed to train and support clini- ing strategies. Western Australia, for example, has begun cians and managers, and others relying on one or two core refresher training. Tasmania has implemented a second individuals. So, for example, Queensland has Zonal Out- round of training, focusing on the outcome measures that comes Co-ordinators and Mental Health Information were not covered in the original training (i.e., the LSP-16 Support Officers providing 'on the ground' support, and the BASIS-32). Queensland has established an ongo- whereas Tasmania has a small, centrally-located team per- ing training program that emphasises sustainability, clin- forming the same function. Physical resources also vary, ical utility and building capacity, and involves its Zonal with some States/Territories having sophisticated online Outcomes Co-ordinators modelling for clinicians how data entry systems (e.g., the Australian Capital Territory), outcome data can be used in clinical management. Most others relying on batch entry of paper-based forms (e.g., other jurisdictions have plans in place to implement a sec- Tasmania), and still others using a combination of the ond wave of training that focuses on the clinical and man- two (e.g., New South Wales and South Australia). agement utility of outcome measurement. States/Territories' progress regarding training and Novel, clinician-focused approaches, such as the use of retraining of staff vignettes and interactive case studies in Victoria and West- All States/Territories have implemented comprehensive ern Australia, have underpinned the initial and ongoing training programs and have trained substantial propor- training in many States/Territories. Training has also typi- tions of their mental health workforces in routine out- cally involved the development of resources (e.g., guides come measurement. According to stakeholders, well over and glossaries for specific measures, consumer/carer bro- 7,000 clinicians and managers across Australia have chures), many of which are located on individual State/ received direct training, and possibly half as many again Territory websites. have received training under a train-the-trainer model. Page 4 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 States/Territories' progress regarding the implementation sumer's scores on a range of outcome measures, either at of routine outcome measurement a single point in time or over time. For example, in the States/Territories are now implementing routine outcome Australian Capital Territory, the data capture system pro- measurement, albeit with very variable degrees of duces an electronic management plan, similar to the New progress. By May 2004, Victoria had provided data for South Wales module described above, which incorporates 2000–01, 2001–02 and 2002–03; New South Wales for areas that the clinician and consumer might want to 2001–02 and 2002–03; Tasmania for 2001–02; and West- address, given the consumer's profile on the outcome ern Australia, Queensland and the Northern Territory for measures. Similarly, in Western Australia, HoNOS scores 2002–03 (partial year only in the latter two). For South of greater than 2 on Items 1 (Overactive, aggressive, dis- Australia and the Australian Capital Territory, 2003–04 ruptive or agitated behaviour) and 2 (Non-accidental self data will constitute the first report. Within States/Territo- injury) trigger a risk assessment, and an alert is registered ries, there is considerable patchiness in terms of coverage, on the system. compliance and completeness. There is variability by set- ting (with community services generally having higher Other States/Territories are generating aggregate-level coverage than inpatient services) and by outcome meas- reports about compliance. For instance, Western Australia ure (with clinician-rated measures being completed to a generates Statewide compliance reports that are distrib- greater extent than consumer-rated measures). Strong uted to mental health services every six weeks, and the leadership at all levels has been associated with high levels Office of Mental Health works with services that are expe- of overall performance in terms of implementation. riencing difficulties with compliance to review the systems in place for monitoring the NOCC collection. Beyond initial training and rollout, some States/Territo- ries have considered how to sustain and build upon cur- A few States/Territories have started producing some rudi- rent efforts with regard to routine outcome measurement. mentary, aggregate-level reports that provide information There is recognition by these (and other) States/Territories about groups of consumers under the care of a given clini- that unless routine outcome measurement becomes cian, team or service. Tasmania, for example, has pro- embedded in the process of clinical care, it will not be duced monthly reports for its Southern Region, which seen as a priority by clinicians and managers. So, for include aggregate-level data on average HoNOS scores at example, in New South Wales outcome measurement has admission, review and discharge. Some States/Territories been embedded in a standard protocol, which involves have begun to consider how best to provide these reports triage, assessment, review and discharge documentation. to areas and services. New South Wales, for example, has Specifically, a suite of clinical modules has been devel- conducted a project involving workshops in all area oped that not only includes an outcomes module but also health services, using their own data to demonstrate the includes the incorporation of outcome measures into the clinical and management utility of the information. A process of care. For example, the collaborative care plan- similar process has been undertaken in Queensland. ning module encourages collaboration between the clini- cian and consumer, and prompts review of the clinician- A range of factors has hampered efforts at analysis and rated HoNOS and the consumer-rated K-10. The process reporting to date. These include resource issues (e.g., lack of embedding outcome measurement within the clinical of personnel and technological constraints), data quality, process of care is enhanced by providing clinical interpre- a lack of clarity about which reports will have greatest clin- tations of given scores on particular measures. All New ical and management utility, and the absence of relevant South Wales Area Mental Health Services will have the normative and/or benchmarking data. same modules, produced as standard medical record sta- tionery for use within clinical files. Discussion Summary of findings States/Territories' progress regarding analysis and Australia has made an impressive start to nationally reporting of data implementing routine outcome measurement in mental Some States/Territories have also begun to consider how health services, although it still has a long way to go. All best to provide feedback to staff. There is recognition that States/Territories have established data collection systems, without appropriate and timely feedback in the form of although some are more streamlined than others. Signifi- relevant reports that shed light on clinical and manage- cant numbers of clinicians and managers have been ment issues, the current momentum will falter and data trained in the use of routine outcome measures, and quality and comprehensiveness will be jeopardised. Feed- thought is now being given to ongoing training strategies. back in the form of reports is required at a variety of levels. Outcome measurement is now occurring 'on the ground'; Some States/Territories have developed individual-level all States/Territories will be reporting data for 2003–04, reports that allow clinicians to profile an individual con- and a number have been doing so for several years. Hav- Page 5 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 ing said this, there is considerable variability regarding as well as guidance to States/Territories about their own data coverage, completeness and compliance. Some reporting. Several principles are guiding this process, in an States/Territories have gone to considerable lengths to effort to ensure that feedback has maximum clinical and 'embed' outcome measurement in day-to-day practice. To management utility, and occurs as quickly as possible. date, reporting of outcome data has largely been limited Specifically, feedback should take the form of reports that to reports profiling individual consumers and/or aggre- are relevant and useful at a range of levels (e.g., individual, gate reports that focus on compliance and data quality team, service and State/Territory). The precise nature of issues, although a few States/Territories have begun to the reports should be informed by an iterative process, turn their attention to producing aggregate reports of con- where relevant recipients are given the opportunity to sumers by clinician, team or service. comment on reports, and subsequent reports are modi- fied accordingly. Reports should provide reference points Study limitations that allow individual scores to be compared with norma- Several limitations must be borne in mind in interpreting tive data, and service profiles to be benchmarked against the above findings. Firstly, the study was dependent upon those of their peers. For now, reports will be based on States/Territories selecting the most appropriate stake- NOCC data alone, in recognition of the difficulties in holders to attend the consultations. Guidance was pro- linking NOCC and NMDS data. This has implications for vided, but it is possible that States/Territories were more defining episodes of care, but provides scope for much to inclined to invite those who were in favour of routine out- be done regarding reporting outcome data in a manner come measurement, and that the views of some key stake- that is useful for clinicians and managers. holders were missed. In particular, the perspective of 'coalface' clinicians was not well captured. Anecdotal AMHOCN is also helping to consolidate the existing reports suggest that there is some apathy, cynicism and State/Territory training efforts. Specifically, it is working resistance towards outcome measurement among this towards: developing and disseminating resources that fill group. Secondly, the study relied almost exclusively on particular gaps; helping States/Territories to streamline subjective reports from the stakeholders who were present their training and re-training packages in a way that bal- at the consultations. Standard qualitative methodologies ances national consistency against the unique require- were used to record and analyse their responses, but there ments of the local context; fostering the skills and were few opportunities for their views to be checked knowledge required for interpreting and reflecting upon against any objective measures. Finally, routine outcome the meaning of outcome data, at a range of levels; encour- measurement is moving at a considerable pace in Aus- aging information-sharing across the board, taking advan- tralia, and further progress has been made since the time tage of its 'birds eye view' to identify good ideas and of the study. The study therefore provides a conservative approaches in given States/Territories and promote them picture of the status quo. in others; exploring processes for accrediting trainers, ensuring that national accreditation is consistent with and Interpretation of findings complementary to any existing accreditation efforts; and These limitations aside, some key messages emerge from engaging, nurturing and supporting clinical leaders, the study. Specifically, it shows that routine outcome champions and innovators. measurement is possible if it is supported by a co-ordi- nated, strategic approach and strong leadership. Equally Conclusion important is commitment from clinicians who are Australia has consistently been regarded as a world leader involved in the day-to-day collection of the outcome data, in routine mental health outcome measurement [15]. It is and from managers who must make it a priority within acknowledged that Australia still has some way to go their services. Stakeholders in the current study repeatedly before routine outcome measurement is 'bedded down', stressed that this commitment will only be sustained in and issues of data coverage, completeness and compliance the long term if clinicians and managers value routine are fully addressed. However, its achievements regarding outcome measurement. Feedback to these groups in the national implementation are significant, and may provide form of reports tailored to their specific needs is crucial, lessons for other countries. and has been identified by others as necessary for main- taining momentum [2-4,13,14]. Sources of financial support This work was funded by the Health Priorities and Suicide AMHOCN clearly has a role in taking routine outcome Prevention Branch of the Australian Government's measurement to its next level. As a priority, AMHOCN is Department of Health and Ageing. specifying a reporting framework for providing feedback to States/Territories. This involves considerations of the References 1. Slade M: The use of patient-level outcomes to inform nature and form of data that AMHOCN itself will provide, treatment. Epidemiologia e Psichiatria Sociale 2002, 11:20-27. Page 6 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 2. Brower LA: The Ohio Mental Health Consumer Outcomes 24. Eisen SV, Dickey B, Sederer LI: A self-report symptom and prob- System: reflections on a major policy initiative in the US. Clin- lem rating scale to increase inpatients' involvement in ical Psychology and Psychotherapy 2003, 10:400-406. treatment. Psychiatric Services 2000, 51:349-353. 3. Ruggeri M: Feasibility, usefulness, limitations and perspectives 25. Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand of routine outcome assessment: the South Verona Outcome SLT, Walters EE, Zaslavsky AM: Short screening scales to moni- Project. Epidemiologia e Psichiatria Sociale 2002, 11:177-185. tor population prevalences and trends in non-specific psy- 4. Priebe S, McCabe R, Bullenkamp J, Hansson L, Rossler W, Torres- chological distress. Psychological Medicine 2002, 32:959-976. Gonzales F, Wiersma D: The impact of routine outcome meas- 26. Goodman R: The Strengths and Difficulties Questionnaire: A urement on treatment processes in community mental research note. Journal of Child Psychology & Psychiatry & Allied health care: approach and methods of the MECCA study. Epi- Disciplines 1997, 38:581-586. demiologia e Psichiatria Sociale 2002, 11:198-204. 27. Department of Health and Ageing: Mental Health National Out- 5. Australian Health Ministers: National Mental Health Plan. Can- comes and Casemix Collection: Overview of clinician rated berra, Australian Government Publishing Service; 1992. and consumer self-report measures, Version 1.50. Canberra, 6. Australian Health Ministers: Second National Mental Health Department of Health and Ageing; 2003. Plan. Canberra: Mental Health Branch, Commonwealth Department of Health and Family Services; 1997. 7. Australian Health Ministers: National Mental Health Plan (2003- 2008). Canberra, Australian Government; 1998. 8. Department of Health and Ageing: National Outcomes and Casemix Collection: Overview of clinical measures and data items. Version 1.02. Canberra, Commonwealth Department of Health and Ageing; 2002. 9. Australian Institute of Health and Welfare: Admitted patient men- tal health care National Minimum Data Set. National Health Data Dictionary, Version 12. AIHW Cat. No. HWI 49. Can- berra, Australian Institute of Health and Welfare; 2003. 10. Australian Institute of Health and Welfare: Community mental health establishments National Minimum Data Set. National Health Data Dictionary. Version 12. Canberra, Australian Insti- tute of Health and Welfare; 2003. 11. Australian Institute of Health and Welfare: Community mental health care National Minimum Data Set. National Health Data Dictionary. Version 12. Canberra, Australian Institute of Health and Welfare; 2003. 12. Department of Health and Ageing: National Mental Health Infor- mation Priorities, 2nd Edition. Canberra, Australian Government. Forthcoming 13. Callaly T, Hallebone EL: Introducing the routine use of out- comes measurement to mental health services. Australian Health Review 2001, 24:43-50. 14. Rock D, Combrinck J, Groves A: Issues associated with the implementation of routine outcome measures in public mental health serivces. Australasian Psychiatry 2001, 9:43-46. 15. Slade M: Routine outcome assessment in mental health services. Psychological Medicine 2002, 32:1339-1343. 16. Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A: Health of the Nation Outcome Scales (HoNOS). Research and development. British Journal of Psychiatry 1998, 172:11-18. 17. Burns A, Beevor A, Lelliott P, Wing J, Blakey A, Orrell M, Mulinga J, Hadden S: Health of the Nation Outcome Scales for elderly people (HoNOS 65+). British Journal of Psychiatry 1999, 174:424-427. 18. Gowers SG, Harrington RC, Whitton A, Lelliott P, Beevor A, Wing J, Jezzard R: Brief scale for measuring the outcomes of emo- tional and behavioural disorders in children. Health of the Nation Outcome Scales for children and Adolescents (HoNOSCA). British Journal of Psychiatry 1999, 174:413-416. 19. Parker G, Rosen A, Emdur N, Hadzi-Pavlovic D: The Life Skills Pro- file: Psychometric properties of a measure assessing function and disability in schizophrenia. Acta Psychiatrica Scandinavica 1991, 83:145-152. Publish with Bio Med Central and every 20. Buckingham W, Burgess P, Solomon S, Pirkis J, Eagar K: Developing scientist can read your work free of charge Casemix a Classification for Mental Health Services. Volume 1: Main Report. Canberra, Commonwealth Department of Health "BioMed Central will be the most significant development for and Family Services; 1998. disseminating the results of biomedical researc h in our lifetime." 21. Fries BE, Schneider DP, Foley WJ, Gavazzi M, et al.: Refining a case- Sir Paul Nurse, Cancer Research UK mix measure for nursing homes: Resource utilization groups (RUG-III). Medical Care 1994, 32:668-685. Your research papers will be: 22. Schaffer D, Gould MS, Brasic J, Ambrosini P, Fisher P, Bird H, Alu- available free of charge to the entire biomedical community wahlia S: A children's global assessment scale (CGAS). Archives of General Psychiatry 1983, 40:1228-1231. peer reviewed and published immediately upon acceptance 23. Veit CT, Ware JE: The structure of psychological distress and cited in PubMed and archived on PubMed Central well-being in general populations. Journal of Consulting & Clinical Psychology 1983, 51:730-742. yours — you keep the copyright BioMedcentral Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 7 of 7 (page number not for citation purposes) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Australia and New Zealand Health Policy Springer Journals

Routine measurement of outcomes in Australia's public sector mental health services

Loading next page...
 
/lp/springer-journals/routine-measurement-of-outcomes-in-australia-s-public-sector-mental-0vAXOTSzyk
Publisher
Springer Journals
Copyright
Copyright © 2005 by Pirkis et al; licensee BioMed Central Ltd.
Subject
Medicine & Public Health; Public Health; Social Policy
ISSN
1743-8462
eISSN
1743-8462
DOI
10.1186/1743-8462-2-8
Publisher site
See Article on Publisher Site

Abstract

Objective: This paper describes the Australian experience to date with a national 'roll out' of routine outcome measurement in public sector mental health services. Methods: Consultations were held with 123 stakeholders representing a range of roles. Results: Australia has made an impressive start to nationally implementing routine outcome measurement in mental health services, although it still has a long way to go. All States/Territories have established data collection systems, although some are more streamlined than others. Significant numbers of clinicians and managers have been trained in the use of routine outcome measures, and thought is now being given to ongoing training strategies. Outcome measurement is now occurring 'on the ground'; all States/Territories will be reporting data for 2003–04, and a number have been doing so for several years. Having said this, there is considerable variability regarding data coverage, completeness and compliance. Some States/Territories have gone to considerable lengths to 'embed' outcome measurement in day-to-day practice. To date, reporting of outcome data has largely been limited to reports profiling individual consumers and/or aggregate reports that focus on compliance and data quality issues, although a few States/Territories have begun to turn their attention to producing aggregate reports of consumers by clinician, team or service. Conclusion: Routine outcome measurement is possible if it is supported by a co-ordinated, strategic approach and strong leadership, and there is commitment from clinicians and managers. The Australian experience can provide lessons for other countries. Page 1 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 informed use of casemix to understand the role of pro- Introduction Internationally, there is an increasing emphasis on rou- vider variation in differences between agencies' costs and tine outcome measurement in mental health. A push to outcomes. For the purposes of the current paper, however, improve quality of care for consumers has prompted the focus is routine outcome measurement only, rather interest in monitoring outcomes at an individual level, than casemix development. and financial pressures and a need to demonstrate value- for-money have led to the use of aggregate reports that For its part, the Australian Government has established allow comparisons between services [1]. In the United three Expert Groups (Adult, Child/Adolescent, and Older States, there are examples of routine outcome measure- Persons) to advise on the implementation and use of rou- ment being 'rolled-out' across mental health services in tine outcome data in mental health services. It has also entire states, such as the Ohio Mental Health Consumer provided resources to support training in the use of out- Outcomes System [2]. In Europe, there are also some come measures, and arrangements to receive, process, examples of individual services monitoring outcomes, as analyse and report on outcome data submitted by States/ in the South Verona Outcomes Project [3] and the Territories. MECCA Study [4], but the routine collection of outcome data has not extended to larger areas. The latter arrangements have been established through the 'Australian Mental Health Outcomes and Classifica- Australia's commitment to routine outcome measure- tion Network' (AMHOCN), a consortium contracted from ment is evidenced in its National Mental Health Strategy late 2003 to provide national leadership in the develop- [5-7]. Since its inception in 1992, the continued improve- ment of outcome measurement in mental health. ment of the quality and effectiveness of the treatment of AMHOCN is pursuing a work program with three compo- people with a mental illness has been a key objective of nents, each being undertaken by a different member of the Strategy. The Strategy has recognised that this objec- the consortium: data management (Strategic Data Pty Ltd, tive can only be achieved through the development of Victoria); analysis and reporting (Queensland Centre for sound information to support service planning and deliv- Mental Health Services Research, The University of ery, and consequently the systematic implementation of Queensland, Queensland); and training and service devel- routine outcome measurement in all public sector mental opment (New South Wales Institute of Psychiatry, New health services is one of its priorities. South Wales). State/Territory governments and the Australian Govern- An immediate concern for AMHOCN was determining ment are collaborating in a coherent national approach. States/Territories' progress with respect to 'rolling out' For their part, all States/Territories have signed Agree- routine outcome measurement, so each State/Territory ments that require them to routinely submit two sets of was invited to participate in a consultation with data from public sector mental health services to the Aus- AMHOCN. This paper reports on the findings from these tralian Government. Firstly, they are required to submit consultations. de-identified, patient-level outcome data, referred to as the 'National Outcomes and Casemix Collection' Method (NOCC) [8]. These outcome data are collected via a range The consultations occurred in March/April 2004. The intention was to seek a range of views, rather than to try to of instruments that incorporate clinician and consumer perspectives on a range of mental health related constructs achieve a representative sample, and States/Territories (e.g., symptomatology, level of functioning, degree of dis- were asked to nominate relevant stakeholders. They could ability) relevant to adults, children/adolescents and older choose whomever they wished, but they were advised to people. The idea is that administration of these instru- consider including policy-makers and technical personnel ments at specific points in time will allow services to mon- from central mental health units and mainstream health itor changes in individual consumers and in groups of information sections, as well as consumers/carers. Most consumers, and, ultimately, to make comparisons with States/Territories sent representatives from all of these similar consumers in like services. Table 1 shows the spe- groups, and many also sent service managers, clinicians, cific instruments that comprise the NOCC dataset. individuals responsible for supporting routine outcome measurement at a site level, and members of the Expert Secondly, States/Territories are required to submit data on Groups. In total, 123 individuals attended the consulta- inpatient episodes of care and community contacts, tions: 10 from New South Wales; 21 from Victoria; 28 termed the 'National Minimum Data Set – Mental Health from Queensland; 23 from Western Australia; 17 from Care' (NMDS) [9-11]. These data provide information on South Australia; six from both Tasmania and the Austral- resource use by consumers, and, when combined with the ian Capital Territory; and 12 from the Northern Territory. above outcome data will promote the development and Page 2 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 Table 1: Data comprising the NOCC collection Adults Older persons Children and adolescents Clinician-rated Principal and additional diagnoses √√ √ Mental health legal status √√ √ Health of the Nation Outcomes Scale (HoNOS) [16] √ Health of the Nation Outcomes Scale 65+ (HoNOS65+) [17] √ Health of the Nation Outcomes Scale for Children and Adolescents √ (HoNOSCA) [18] Life Skills Profile 16 (LSP-16) [19, 20] √√ Resource Utilisation Groups – Activities of Daily Living Scale (RUG- √ ADL) [21] Focus of Care [20] √√ Children's Global Assessment Scale (CGAS) [22] √ Factors Influencing Health Status (FIHS) [20] √ Consumer-rated Mental Health Inventory (MHI) [23] or Behaviour and Symptom √√ Identification Scale (BASIS-32) [24] or Kessler-10 Plus (K-10+) [25] Consumer- and parent- Strengths and Difficulties Questionnaire (SDQ)[26] √ rated Source: Department of Health and Ageing (2003) [27] Many 'wore several hats', rendering it difficult to provide Results States/Territories' progress regarding data collection a breakdown of their roles. systems and infrastructure The consultations sought answers to questions regarding Ultimately, States/Territories are aiming to have stream- progress in four domains: (a) data collection systems and lined data collection systems that allow the outcome data infrastructure; (b) training and retraining of staff; (c) the collected via the NOCC dataset to be linked to the admit- implementation of routine outcome measurement; and ted and non-admitted activity data in the NMDS. This will (d) and analysis, reporting and use of data. allow outcome data to be 'attached' to given inpatient and community episodes of care. This has advantages in terms The majority of consultations took place over a full day, of allowing outcomes for consumers to be 'tracked' across with the shortest being half a day. All of the consultations episodes, and is necessary for progressing casemix devel- began with a brief presentation from AMHOCN, and then opment work that requires outcome data and resource use elicited information from participants. Some States/Terri- data to be combined within episodes. tories chose to split the consultation in two, inviting pol- icy makers, planners and clinicians to attend one session All States/Territories have developed data collection sys- and technical personnel to attend the other. Some States/ tems, or are in the final stages of doing so. For some, this Territories gave formal presentations responding to spe- has involved 'starting from scratch'; for others it has cific questions; others took a more informal approach. In required modifications to existing systems. For example, some cases, the information presented at the consulta- the systems used in Queensland to capture admitted and tions was supplemented by a written response. non-admitted NMDS information did not have the func- tionality to incorporate outcome measures, so an addi- Each consultation was transcribed. The transcription was tional system was developed to do so. By contrast, in the combined with any other written material (e.g., formal Australian Capital Territory, the system used by all com- responses and presentations), and examined at a global munity teams to collect non-admitted NMDS data, was level to identify major themes within each domain. Indi- modified to collect outcome data and extended to inpa- vidual responses were classified according to these tient services, where it runs alongside a separate patient themes. Each State/Territory was given the opportunity to administration system for the collection of inpatient activ- comment on the accuracy of the written interpretation of ity data. the consultation. States/Territories differ in terms of the number of systems that are currently involved in the collection of routine out- come data. The simplest scenario is one where outcome measurement functionality has been added to an existing Page 3 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 system for recording activity in community mental health This figure is consistent with that of 10,000 reported by settings, and has been extended into inpatient settings (as the Department of Health and Ageing, which is estimated with the system in the Australian Capital Territory, to represent approximately 60% the public sector mental described above). This also occurs in Victoria, Tasmania health workforce [12]. and the Northern Territory. Other States rely on as many as four statewide systems to collect NOCC and NMDS The direct training approach is seen as having the benefit information, sometimes with further degrees of complex- of consistency, while the train-the-trainer approach is ity between areas or metropolitan/country settings. seen as fostering capacity building and being less labour intensive and cheaper. Some States/Territories have con- Linking NOCC and admitted and non-admitted NMDS sidered accrediting trainers, so that the advantages of both datasets is impeded in most States/Territories by the lack approaches can be combined. Managers are also more of a unique identifier. Typically, linkage is only possible commonly being recruited as trainers, as part of a move to for parts of the data (usually NOCC and non-admitted secure their commitment in leading the change process. NMDS data) and/or by conducting quite complex record South Australia has been innovative here, building capac- linkage tasks. The exception is the Northern Territory, ity by training staff as trainers through the Certificate 4 in which has a client master index that allocates each con- Workplace Training and Assessment, and investing in sumer a unique identifier that allows him/her to be training in content knowledge around outcome measures 'tracked' across episodes, across services, and over time. in the NOCC collection. In this way, South Australia has Other States/Territories are working towards improve- addressed some of the difficulties inherent in more stand- ments, but have some way to go. Western Australia's data ard train-the-trainer approaches. collection system has a unique identifier that will allow episodes of care to be attributed to the same individual, Many States/Territories are now beginning to consider regardless of location or time, but its 'roll-out' is not yet issues of ongoing training and support. High levels of staff completed. Queensland and New South Wales have plans turnover in some States/Territories mean that there are to reconcile their unique identifier systems via specific new staff who have not been trained, and lags between projects. This will mean that States will assign a unique training and implementation in some jurisdictions have identifier to a given individual that he or she will 'carry' resulted in skills being lost. In addition, many States/Ter- across all health services, including mental health services, ritories are recognising the need for a second wave of but this will not occur in the near future. training that goes beyond how to use the outcome meas- ures and focuses more on how to interpret the results of States/Territories have differing levels of infrastructure to specific measures (at individual and aggregate levels). support the NOCC and NMDS collections. Human resources vary, with some States/Territories having a Some States/Territories have implemented ongoing train- number of personnel deployed to train and support clini- ing strategies. Western Australia, for example, has begun cians and managers, and others relying on one or two core refresher training. Tasmania has implemented a second individuals. So, for example, Queensland has Zonal Out- round of training, focusing on the outcome measures that comes Co-ordinators and Mental Health Information were not covered in the original training (i.e., the LSP-16 Support Officers providing 'on the ground' support, and the BASIS-32). Queensland has established an ongo- whereas Tasmania has a small, centrally-located team per- ing training program that emphasises sustainability, clin- forming the same function. Physical resources also vary, ical utility and building capacity, and involves its Zonal with some States/Territories having sophisticated online Outcomes Co-ordinators modelling for clinicians how data entry systems (e.g., the Australian Capital Territory), outcome data can be used in clinical management. Most others relying on batch entry of paper-based forms (e.g., other jurisdictions have plans in place to implement a sec- Tasmania), and still others using a combination of the ond wave of training that focuses on the clinical and man- two (e.g., New South Wales and South Australia). agement utility of outcome measurement. States/Territories' progress regarding training and Novel, clinician-focused approaches, such as the use of retraining of staff vignettes and interactive case studies in Victoria and West- All States/Territories have implemented comprehensive ern Australia, have underpinned the initial and ongoing training programs and have trained substantial propor- training in many States/Territories. Training has also typi- tions of their mental health workforces in routine out- cally involved the development of resources (e.g., guides come measurement. According to stakeholders, well over and glossaries for specific measures, consumer/carer bro- 7,000 clinicians and managers across Australia have chures), many of which are located on individual State/ received direct training, and possibly half as many again Territory websites. have received training under a train-the-trainer model. Page 4 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 States/Territories' progress regarding the implementation sumer's scores on a range of outcome measures, either at of routine outcome measurement a single point in time or over time. For example, in the States/Territories are now implementing routine outcome Australian Capital Territory, the data capture system pro- measurement, albeit with very variable degrees of duces an electronic management plan, similar to the New progress. By May 2004, Victoria had provided data for South Wales module described above, which incorporates 2000–01, 2001–02 and 2002–03; New South Wales for areas that the clinician and consumer might want to 2001–02 and 2002–03; Tasmania for 2001–02; and West- address, given the consumer's profile on the outcome ern Australia, Queensland and the Northern Territory for measures. Similarly, in Western Australia, HoNOS scores 2002–03 (partial year only in the latter two). For South of greater than 2 on Items 1 (Overactive, aggressive, dis- Australia and the Australian Capital Territory, 2003–04 ruptive or agitated behaviour) and 2 (Non-accidental self data will constitute the first report. Within States/Territo- injury) trigger a risk assessment, and an alert is registered ries, there is considerable patchiness in terms of coverage, on the system. compliance and completeness. There is variability by set- ting (with community services generally having higher Other States/Territories are generating aggregate-level coverage than inpatient services) and by outcome meas- reports about compliance. For instance, Western Australia ure (with clinician-rated measures being completed to a generates Statewide compliance reports that are distrib- greater extent than consumer-rated measures). Strong uted to mental health services every six weeks, and the leadership at all levels has been associated with high levels Office of Mental Health works with services that are expe- of overall performance in terms of implementation. riencing difficulties with compliance to review the systems in place for monitoring the NOCC collection. Beyond initial training and rollout, some States/Territo- ries have considered how to sustain and build upon cur- A few States/Territories have started producing some rudi- rent efforts with regard to routine outcome measurement. mentary, aggregate-level reports that provide information There is recognition by these (and other) States/Territories about groups of consumers under the care of a given clini- that unless routine outcome measurement becomes cian, team or service. Tasmania, for example, has pro- embedded in the process of clinical care, it will not be duced monthly reports for its Southern Region, which seen as a priority by clinicians and managers. So, for include aggregate-level data on average HoNOS scores at example, in New South Wales outcome measurement has admission, review and discharge. Some States/Territories been embedded in a standard protocol, which involves have begun to consider how best to provide these reports triage, assessment, review and discharge documentation. to areas and services. New South Wales, for example, has Specifically, a suite of clinical modules has been devel- conducted a project involving workshops in all area oped that not only includes an outcomes module but also health services, using their own data to demonstrate the includes the incorporation of outcome measures into the clinical and management utility of the information. A process of care. For example, the collaborative care plan- similar process has been undertaken in Queensland. ning module encourages collaboration between the clini- cian and consumer, and prompts review of the clinician- A range of factors has hampered efforts at analysis and rated HoNOS and the consumer-rated K-10. The process reporting to date. These include resource issues (e.g., lack of embedding outcome measurement within the clinical of personnel and technological constraints), data quality, process of care is enhanced by providing clinical interpre- a lack of clarity about which reports will have greatest clin- tations of given scores on particular measures. All New ical and management utility, and the absence of relevant South Wales Area Mental Health Services will have the normative and/or benchmarking data. same modules, produced as standard medical record sta- tionery for use within clinical files. Discussion Summary of findings States/Territories' progress regarding analysis and Australia has made an impressive start to nationally reporting of data implementing routine outcome measurement in mental Some States/Territories have also begun to consider how health services, although it still has a long way to go. All best to provide feedback to staff. There is recognition that States/Territories have established data collection systems, without appropriate and timely feedback in the form of although some are more streamlined than others. Signifi- relevant reports that shed light on clinical and manage- cant numbers of clinicians and managers have been ment issues, the current momentum will falter and data trained in the use of routine outcome measures, and quality and comprehensiveness will be jeopardised. Feed- thought is now being given to ongoing training strategies. back in the form of reports is required at a variety of levels. Outcome measurement is now occurring 'on the ground'; Some States/Territories have developed individual-level all States/Territories will be reporting data for 2003–04, reports that allow clinicians to profile an individual con- and a number have been doing so for several years. Hav- Page 5 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 ing said this, there is considerable variability regarding as well as guidance to States/Territories about their own data coverage, completeness and compliance. Some reporting. Several principles are guiding this process, in an States/Territories have gone to considerable lengths to effort to ensure that feedback has maximum clinical and 'embed' outcome measurement in day-to-day practice. To management utility, and occurs as quickly as possible. date, reporting of outcome data has largely been limited Specifically, feedback should take the form of reports that to reports profiling individual consumers and/or aggre- are relevant and useful at a range of levels (e.g., individual, gate reports that focus on compliance and data quality team, service and State/Territory). The precise nature of issues, although a few States/Territories have begun to the reports should be informed by an iterative process, turn their attention to producing aggregate reports of con- where relevant recipients are given the opportunity to sumers by clinician, team or service. comment on reports, and subsequent reports are modi- fied accordingly. Reports should provide reference points Study limitations that allow individual scores to be compared with norma- Several limitations must be borne in mind in interpreting tive data, and service profiles to be benchmarked against the above findings. Firstly, the study was dependent upon those of their peers. For now, reports will be based on States/Territories selecting the most appropriate stake- NOCC data alone, in recognition of the difficulties in holders to attend the consultations. Guidance was pro- linking NOCC and NMDS data. This has implications for vided, but it is possible that States/Territories were more defining episodes of care, but provides scope for much to inclined to invite those who were in favour of routine out- be done regarding reporting outcome data in a manner come measurement, and that the views of some key stake- that is useful for clinicians and managers. holders were missed. In particular, the perspective of 'coalface' clinicians was not well captured. Anecdotal AMHOCN is also helping to consolidate the existing reports suggest that there is some apathy, cynicism and State/Territory training efforts. Specifically, it is working resistance towards outcome measurement among this towards: developing and disseminating resources that fill group. Secondly, the study relied almost exclusively on particular gaps; helping States/Territories to streamline subjective reports from the stakeholders who were present their training and re-training packages in a way that bal- at the consultations. Standard qualitative methodologies ances national consistency against the unique require- were used to record and analyse their responses, but there ments of the local context; fostering the skills and were few opportunities for their views to be checked knowledge required for interpreting and reflecting upon against any objective measures. Finally, routine outcome the meaning of outcome data, at a range of levels; encour- measurement is moving at a considerable pace in Aus- aging information-sharing across the board, taking advan- tralia, and further progress has been made since the time tage of its 'birds eye view' to identify good ideas and of the study. The study therefore provides a conservative approaches in given States/Territories and promote them picture of the status quo. in others; exploring processes for accrediting trainers, ensuring that national accreditation is consistent with and Interpretation of findings complementary to any existing accreditation efforts; and These limitations aside, some key messages emerge from engaging, nurturing and supporting clinical leaders, the study. Specifically, it shows that routine outcome champions and innovators. measurement is possible if it is supported by a co-ordi- nated, strategic approach and strong leadership. Equally Conclusion important is commitment from clinicians who are Australia has consistently been regarded as a world leader involved in the day-to-day collection of the outcome data, in routine mental health outcome measurement [15]. It is and from managers who must make it a priority within acknowledged that Australia still has some way to go their services. Stakeholders in the current study repeatedly before routine outcome measurement is 'bedded down', stressed that this commitment will only be sustained in and issues of data coverage, completeness and compliance the long term if clinicians and managers value routine are fully addressed. However, its achievements regarding outcome measurement. Feedback to these groups in the national implementation are significant, and may provide form of reports tailored to their specific needs is crucial, lessons for other countries. and has been identified by others as necessary for main- taining momentum [2-4,13,14]. Sources of financial support This work was funded by the Health Priorities and Suicide AMHOCN clearly has a role in taking routine outcome Prevention Branch of the Australian Government's measurement to its next level. As a priority, AMHOCN is Department of Health and Ageing. specifying a reporting framework for providing feedback to States/Territories. This involves considerations of the References 1. Slade M: The use of patient-level outcomes to inform nature and form of data that AMHOCN itself will provide, treatment. Epidemiologia e Psichiatria Sociale 2002, 11:20-27. Page 6 of 7 (page number not for citation purposes) Australia and New Zealand Health Policy 2005, 2:8 http://www.anzhealthpolicy.com/content/2/1/8 2. Brower LA: The Ohio Mental Health Consumer Outcomes 24. Eisen SV, Dickey B, Sederer LI: A self-report symptom and prob- System: reflections on a major policy initiative in the US. Clin- lem rating scale to increase inpatients' involvement in ical Psychology and Psychotherapy 2003, 10:400-406. treatment. Psychiatric Services 2000, 51:349-353. 3. Ruggeri M: Feasibility, usefulness, limitations and perspectives 25. Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand of routine outcome assessment: the South Verona Outcome SLT, Walters EE, Zaslavsky AM: Short screening scales to moni- Project. Epidemiologia e Psichiatria Sociale 2002, 11:177-185. tor population prevalences and trends in non-specific psy- 4. Priebe S, McCabe R, Bullenkamp J, Hansson L, Rossler W, Torres- chological distress. Psychological Medicine 2002, 32:959-976. Gonzales F, Wiersma D: The impact of routine outcome meas- 26. Goodman R: The Strengths and Difficulties Questionnaire: A urement on treatment processes in community mental research note. Journal of Child Psychology & Psychiatry & Allied health care: approach and methods of the MECCA study. Epi- Disciplines 1997, 38:581-586. demiologia e Psichiatria Sociale 2002, 11:198-204. 27. Department of Health and Ageing: Mental Health National Out- 5. Australian Health Ministers: National Mental Health Plan. Can- comes and Casemix Collection: Overview of clinician rated berra, Australian Government Publishing Service; 1992. and consumer self-report measures, Version 1.50. Canberra, 6. Australian Health Ministers: Second National Mental Health Department of Health and Ageing; 2003. Plan. Canberra: Mental Health Branch, Commonwealth Department of Health and Family Services; 1997. 7. Australian Health Ministers: National Mental Health Plan (2003- 2008). Canberra, Australian Government; 1998. 8. Department of Health and Ageing: National Outcomes and Casemix Collection: Overview of clinical measures and data items. Version 1.02. Canberra, Commonwealth Department of Health and Ageing; 2002. 9. Australian Institute of Health and Welfare: Admitted patient men- tal health care National Minimum Data Set. National Health Data Dictionary, Version 12. AIHW Cat. No. HWI 49. Can- berra, Australian Institute of Health and Welfare; 2003. 10. Australian Institute of Health and Welfare: Community mental health establishments National Minimum Data Set. National Health Data Dictionary. Version 12. Canberra, Australian Insti- tute of Health and Welfare; 2003. 11. Australian Institute of Health and Welfare: Community mental health care National Minimum Data Set. National Health Data Dictionary. Version 12. Canberra, Australian Institute of Health and Welfare; 2003. 12. Department of Health and Ageing: National Mental Health Infor- mation Priorities, 2nd Edition. Canberra, Australian Government. Forthcoming 13. Callaly T, Hallebone EL: Introducing the routine use of out- comes measurement to mental health services. Australian Health Review 2001, 24:43-50. 14. Rock D, Combrinck J, Groves A: Issues associated with the implementation of routine outcome measures in public mental health serivces. Australasian Psychiatry 2001, 9:43-46. 15. Slade M: Routine outcome assessment in mental health services. Psychological Medicine 2002, 32:1339-1343. 16. Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A: Health of the Nation Outcome Scales (HoNOS). Research and development. British Journal of Psychiatry 1998, 172:11-18. 17. Burns A, Beevor A, Lelliott P, Wing J, Blakey A, Orrell M, Mulinga J, Hadden S: Health of the Nation Outcome Scales for elderly people (HoNOS 65+). British Journal of Psychiatry 1999, 174:424-427. 18. Gowers SG, Harrington RC, Whitton A, Lelliott P, Beevor A, Wing J, Jezzard R: Brief scale for measuring the outcomes of emo- tional and behavioural disorders in children. Health of the Nation Outcome Scales for children and Adolescents (HoNOSCA). British Journal of Psychiatry 1999, 174:413-416. 19. Parker G, Rosen A, Emdur N, Hadzi-Pavlovic D: The Life Skills Pro- file: Psychometric properties of a measure assessing function and disability in schizophrenia. Acta Psychiatrica Scandinavica 1991, 83:145-152. Publish with Bio Med Central and every 20. Buckingham W, Burgess P, Solomon S, Pirkis J, Eagar K: Developing scientist can read your work free of charge Casemix a Classification for Mental Health Services. Volume 1: Main Report. Canberra, Commonwealth Department of Health "BioMed Central will be the most significant development for and Family Services; 1998. disseminating the results of biomedical researc h in our lifetime." 21. Fries BE, Schneider DP, Foley WJ, Gavazzi M, et al.: Refining a case- Sir Paul Nurse, Cancer Research UK mix measure for nursing homes: Resource utilization groups (RUG-III). Medical Care 1994, 32:668-685. Your research papers will be: 22. Schaffer D, Gould MS, Brasic J, Ambrosini P, Fisher P, Bird H, Alu- available free of charge to the entire biomedical community wahlia S: A children's global assessment scale (CGAS). Archives of General Psychiatry 1983, 40:1228-1231. peer reviewed and published immediately upon acceptance 23. Veit CT, Ware JE: The structure of psychological distress and cited in PubMed and archived on PubMed Central well-being in general populations. Journal of Consulting & Clinical Psychology 1983, 51:730-742. yours — you keep the copyright BioMedcentral Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 7 of 7 (page number not for citation purposes)

Journal

Australia and New Zealand Health PolicySpringer Journals

Published: Apr 19, 2005

References