Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Impact of telephone triage on emergency after hours GP Medicare usage: a time-series analysis

Impact of telephone triage on emergency after hours GP Medicare usage: a time-series analysis Background: The Australian government sponsored trials aimed at addressing problems in after hours primary medical care service use in five different parts of the country with different after hours care problems. The study's objective was to determine in four of the five trials where telephone triage was the sole innovation, if there was a reduction in emergency GP after hours service utilization (GP first call-out) as measured in Medicare Benefits Schedule claim data. Monthly MBS claim data in both the pre-trial and trial periods was monitored over a 3-year period in each trial area as well as in a national sample outside the trial areas (National comparator). Poisson regression analysis was used in analysis. Results: There was significant reduction in first call out MBS claims in three of the four study areas where stand-alone call centre services existed. These were the Statewide Call Centre in both its Metropolitan and Non-metropolitan areas in which it operated – Relative Risk (RR) = 0.87 (95% Confidence interval: 0.86 – 0.88) and 0.60 (95% CI: 0.54 – 0.68) respectively. There was also a reduction in the Regional Call Centre in the non-Metropolitan area in which it operated (RR = 0.46 (95% CI: 0.35 – 0.61) though a small increase in its Metropolitan area (RR = 1.11 (95% CI: 1.06 – 1.17). For the two telephone triage services embedded in existing organisations, there was also a significant reduction for the Deputising Service – RR = 0.62 (95% CI: 0.61 – 0.64) but no change in the Local Triage centre area. Conclusion: The four telephone triage services were associated with reduced GP MBS claims for first callout after hours care in most study areas. It is possible that other factors could be responsible for some of this reduction, for example, MBS submitted claims for after hours GP services being reclassified from 'after hours' to 'in hours'. The goals of stand-alone call centres which are aimed principally at meeting population needs rather than managing demand may be being met only in part. Page 1 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Previous research on the APHMCTs using population- Background This paper reports further on the national evaluation of based telephone survey data of after hours service utilisa- the After Hours Primary Medical Care Trials (AHPMCTs) tion found that the introduction of telephone triage serv- which were a recent initiative of the Australian Govern- ices, whether embedded in other services or stand-alone, ment. The goals of these trials were to improve the quality was not clearly associated with reduction in after hours of service delivery as well as consumer acceptability, con- use or shift towards the use of GP after hours clinics [7]. It sumer access (including affordability) and equity, appro- is important therefore to extend analysis to include not priateness of service mix, provider satisfaction with regard only self-reported data, but also administrative data rou- to their impact on service mix as well as service use more tinely recorded on service utilisation. This study presents generally [1]. Medicare Benefits Schedule (MBS) data from Medicare Australia (formerly the Health Insurance Commission The common feature of these trials was the use of tele- [HIC]) bearing in mind that this data too had limitations. phone triage. Telephone triage and advice services have an At the time of the study, MBS data recorded only GP emer- important place in the development of after hours care in gency (first doctor call-out whether to consulting rooms other countries. Most frequently these services are embed- or patient's home) after hours service use and not all other ded in other after hours services such as in GP coopera- routine after hours GP use. Thus the data forms one small tives in the UK, HMOs in the US and the county-based but important part of all after hours service use. service arrangements in Denmark introduced in the 1990s [2]. A small number of stand-alone services have also The aim of this study then is to determine if there is a been established. NHS Direct is a 24 hour, confidential reduction in emergency (GP first callout) after hours serv- telephone, online and interactive digital TV health advice ice use recorded in MBS data following the introduction and information service provided by the National Health of different forms of telephone triage in the AHPMCTs, Service in England and Wales (similar service in Scot- where these were the sole service innovation. land). NHS Direct was rolled out across England and Wales between 1998 and 2000. The telephone service The After Hours Primary Medical Care Trials aims to triage symptomatic callers to provide guidance on Telephone triage services used by the four study AHPM- which healthcare provider the caller should access. Nurses CTs varied both in the form of the telephone triage service using proprietary health call centre software also give offered and its service function within the individual advice on how to manage an episode of illness at home. AHPMCT as follows [1]. Health Information Advisors can provide information on Stand-alone services a wide range of medical conditions, treatments, medicines and NHS services. In some areas of England and Wales, These were call centres where nurses, using proprietary NHS Direct is commissioned by local Primary Care Trusts health call centre software, aimed at providing more (PCTs) to provide the gateway for out-of-hours access to accessible advice and promoting more appropriate after GP's surgeries and clinics. [3]. hours primary medical care (PMC) service use. They were: A structured review on the impact of after hours GP serv- • the stand-alone Statewide Call Centre (studied in both ices on clinical outcomes, medical workloads as well as Metropolitan and Non-metropolitan areas); and patient and GP satisfaction concluded that this growth in the use of telephone triage and advice services usually, but  the stand-alone Regional Call Centre (studied in both not always reduced immediate medical workload through inner metropolitan (Metropolitan) and rural satellite areas the substitution of telephone consultations for face-to- (Non-metropolitan); face consultations [4]. For example, a before and after study following the introduction of NHS Direct as a stand- Embedded services alone service in the UK found a small, but significant These triage and advice systems did not use proprietary reduction in use of GP co-operatives but no change in use health call centre software. They aimed more at managing of Emergency Department (ED) and ambulance services demand to support the GP workforce in terms of recruit- in the study area [5]. Considering embedded services, a ment and retention. They were: randomised controlled trial from the UK compared a nurse telephone consultation service integrated within a  a GP-based telephone triage and advice service (without GP co-operative with the usual practice of that co-opera- guidelines or software) in a well-established, pre-existing tive [6]. There was a 69% reduction in telephone advice Deputising Service; from a GP, a 38% reduction in patient attendance at pri- mary care centres and a 23% reduction in home visits.  the Local Triage and advice service using hospital nurses with locally developed, paper-based protocols to support Page 2 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 after hours services in GP own-practice arrangements in a sulting rooms (75.7% in 97/98, 76.4% in 99/00). Services small rural community. provided by non-VR GPs, compared to VR GPs were most common in unsociable hours. The AHPMCT services together with the regional context of the trials and their organisational settings are summa- There was a 6.7% reduction in after hours utilisation rates rised in Table 1. It should be noted that while the from 30.0 to 28.0 per 10,000 population per month Statewide Call Centre received funding as part of the AHP- across the study period outside of the trial areas. There was MCTs this was mainly for logistic and information sup- also a 3.8% decline in the in-hours utilisation rate during port. The service opened a short time before the other this time. The decline in number of after hours services AHPMCTs with separate Australian and State government occurred more in the consulting room (9.6%) than out- funding. While it provides 24-hour advice on a variety of side (5.7%). This decline also occurred in sociable hours health-related topics, its involvement in after hours pri- (8–11 pm) (11.9%) but not in unsociable hours (11 pm– mary care is the focus of this paper. The AHPMCTs com- 8 am) which increased substantially (20.8%). menced operation during 1999–2000. Variation in the Stand-alone services – see Tables 3, 4, 5, 6 date of commencement affected whether or not one or two years of pre-trial data was available. It also meant, for Statewide Call Centre – Metropolitan a few trials, that data covered less than 12 months of oper- The total monthly after hours utilisation rate in the area ation, introducing unavoidable seasonal bias. was low – approximately 63.7% of rates elsewhere in Aus- tralia judged by the National Comparator. Results National comparator – see Table 2 There was a 17.8% reduction in total monthly after hours After hours utilisation rates were very small in number utilisation from 19.1 to 15.7 per 10,000 population per compared to in-hours utilisation rates. VR GPs provided month following the introduction of the AHPMCT. The most after hours services (59.3% in 97/98, 62.9% in 99/ decline in number of in-hours service by comparison was 00). Most after hours services were provided in sociable quite small (2.9%). Reduction in both after hours consult- hours (84.0% in 97/98, 79.3% in 99/00) and outside con- ing room and non-consulting room services was substan- Table 1: Local context and services of individual trials. Trial Catchment Areas * Frequency of telephone calls to Notes service (calls/month/10,000 head of population) [1] 1. Statewide Call Capital city, small rural centre and Other 50 (Metropolitan) Data not available 1 Operator with prior international industry experience Centre rural area 1.42M (Non-metropolitan) under contract to the State Government) – phase-in (Observation period for trial Feb99–Jan00 period when calls to a public hospitals were transferred to and Feb00–Dec00). Rest of state – small rural the call centre centre and other rural area studied 42,500 2 Free to caller (Observation period for trial July00–Dec00) 3 Coincidentally, a number of capital city after hours walk- in GP clinics opened around the time the Statewide Call Centre was established. 2. Regional Call Capital city (inner region) 537,000 Non- 18 (Metropolitan) 1 Startup operator – phase-in period when calls to two Centre metropolitan satellite: Small rural centre 6 (Non-metropolitan) public hospitals were transferred to the call centre 21,000 2 Free to caller (Observation period for trial May00–Dec 00) 3 A GP after hours walk-in clinic operating with no fee to patients located adjacent to the ED of a regional hospital in the central metropolitan area operating for part of the study period but closed due to lack of demand. 4 Prior restricted access to after hours GP care in non- metropolitan satellite 3. Deputising Capital city and Other rural area 229,000 31 (approx) 1 GP-based telephone triage and advice service Service (Observation period for trial – October 1999 2 Pre-existing and continuing Deputising Service offering an to September 2000) after hours walk-in clinic and home visiting service. 3 Service offered to patients of previously enrolled GP practices (85% of all practices in capital city) and new enrolled GP practices beyond the metropolitan area (52% practice take-up). 4 Commonwealth government funding of the program derived from 'cashing out' historical MBS reimbursements to the service – home visits by Deputising Service staff to patients in capital city (only) during the trial not billed to patient. 4. Local Triage and Other rural area (ie small country town and 24 (approx) 1 Hospital nurses using locally developed paper-based advice service surrounding community) 21,000 protocols (Observation period for trial – October 1999 2 Located in a local hospital in a rural community. to September 2000) 3 Aim to support GP own-practice after hours arrangement to relieve GP after hours burden. * Based on Rural, Remote and Metropolitan Areas classification (RRMA) [11] † Source: Australian Bureau of Statistics population projections for 1999 supplied to the Department of Health and Ageing Page 3 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Table 2: Utilisation rates of MBS services (per 10,000 population): National comparator. TIME IN-HOURS AFTER VR GPS NON VR GPS PERIOD HOURS (POPULA TION) Not "UNSOCIABLE" "UNSOCIABLE" HOURS NOT "UNSOCIABLE" "UNSOCIABLE" HOURS HOURS HOURS Not CON CON NOT CON CON NOT CON CON Not CON CON ROOMS ROOMS ROOMS ROOMS ROOMS ROOMS ROOMS ROOMS 1 2 601 602 97 98 697 698 Oct97/Sep98 4025.3 30.0 11.0 4.2 2.0 0.7 7.8 2.2 1.9 0.2 (16,344,730) Oct98/Sep99 3975.5 29.2 11.4 4.1 2.4 0.9 6.4 1.5 2.3 0.4 (16,517,748) Oct99/Sep00 3871.7 28.0 10.7 3.9 2.0 1.0 6.3 1.3 2.4 0.4 (16,689,060) Notes: For the non-trial areas the period October 1997 to September 2000 was analysed as this represented the most common period over which the trials were conducted. Sociable hours (8–11 pm)and unsociable hours (11 pm–8 am) tial (7.7% and 14.6% respectively) in the first 12 months. tre. In-hours items were relatively stable across the 3-years There was a further large reduction in consulting room period by comparison. Non-consulting room services services (41.7%) but not non-consulting room (2.6%) in declined from 82.9% to 65.1% as a proportion of total the following 11 months. A higher proportion of after after hours services over the 3-year period (compared to hours services were outside of the consulting room (93.2 75.7 – 76.4% nationally outside the trial areas). – 95.5%) compared with 75.7 – 76.4% in the National comparator area. Following the introduction of the AHPMCT, there was a significant reduction in the total monthly after hours uti- Following the introduction of the AHPMCT, there was a lisation rate (RR = 0.60, 95% CI: 0.54 – 0.68) when com- significant reduction in the total monthly after hours uti- pared with the National comparator. lisation rate (RR = 0.87 (95% CI: 0.86 – 0.88)) when com- pared with the National comparator. Regional Call Centre – Metropolitan The total monthly after hours utilisation rate in the area Statewide Call Centre – Non-metropolitan was low – approximately 23% of rates elsewhere in Aus- The total monthly after hours utilisation rate in the area tralia judged by the National Comparator. The total was low, 50.3% of rates elsewhere in Australia judged by monthly after hours utilisation rate remained steady at 6.6 the National Comparator. After hours services were pre- per 10,000 population per month following the introduc- dominantly supplied by VR GPs (in excess of 80%). Very tion of the Regional Call Centre. Services in and out of the few (less than 10%) of total after hours services were sup- consulting room also remained steady. plied in unsociable hours and all of these were supplied by VR GPs. Following the introduction of the AHPMCT, there was a small increase in the total monthly after hours utilisation There was a 36.4% reduction in total monthly after hours rate (RR = 1.11, 95% CI: 1.06 – 1.17) when compared utilisation rate from 15.1 to 9.6 per 10,000 population with the National comparator. per month after the introduction of the Statewide Call Cen- Table 3: Utilisation rates of MBS services (per 10,000 population): AHPMCT stand-alone services – Statewide Call Centre (Metropolitan). Feb98/Jan99 (1,398,781) 3920.6 19.1 4.8 0.7 1.2 0.0 8.5 0.6 3.3 0.0 Trial period begins Feb99/Jan00 (1,424,036) 3881.6 16.3 4.9 0.7 1.3 0.0 6.4 0.5 2.6 0.0 Feb00/Dec00 (1,449,729) 3790.0 15.7 3.9 0.5 0.8 0.0 7.5 0.2 2.6 0.0 Page 4 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Table 4: Utilisation rates of MBS services (per 10,000 population): AHPMCT stand-alone services – Statewide Call Centre (Non- metropolitan). Jul98/Jun99 (118,162) 2892.9 15.1 10.2 2.1 1.4 0.2 1.0 0.3 0.0 0.0 Jul99/Jun00 (120370) 2870.9 9.6 4.7 3.4 0.7 0.1 0.7 0.2 0.0 0.0 Trial period begins Jul00/Dec00 (122,211) 2810.1 8.6 4.0 2.5 0.5 0.1 1.1 0.4 0.0 0.0 Regional Call Centre – Non-metropolitan There was a 9.7% reduction in total monthly after hours The total monthly after hours utilisation rate in the area utilisation rate from 49.5 to 44.7 per 10,000 population was low – approximately 42.3% of rates elsewhere in Aus- per month after the introduction of the AHPMCT. (Rates tralia judged by the National Comparator. and change in rates should be interpreted cautiously as numbers were small and confidence intervals wide.) The total monthly after hours utilisation rate reduced by Nearly all after hours services (98–99%) were delivered 58.3% from 12.7 to 5.3 per 10,000 population per month outside the consulting room and this did not change dur- after the introduction of the AHPMCT. Following the ing the trial (compared to 76–77% in the National Compa- introduction of the AHPMCT there was a significant rator area. Following the introduction of the AHPMCT the reduction in the total monthly after hours utilisation rate total monthly after hours utilisation rate did not change (RR = 0.46, 95% CI: 0.35 – 0.61) when compared with the (RR = 1.01, 0.94 – 1.08) and when compared with the National comparator. National comparator. Embedded services – see Tables 7, 8 Discussion Deputising Service Considering first stand-alone services, there was a signifi- The total monthly after hours utilisation rate in the area cant reduction in after hours utilisation in three of the was approximately 33% higher than rates elsewhere in four study areas – the Statewide Call Centre in both Metro- Australia judged by the National Comparator. politan and Non-metropolitan areas, (larger in the Non-met- ropolitan area) as well as the Non-metropolitan areas in the There was a 39.1% reduction in total monthly after hours Regional Call Centre. (There was a small increase in the utilisation rate from 39.9 to 24.3 per 10,000 population Metropolitan area). Considering embedded services, there per month after the introduction of the AHPMCT. The was a significant reductionin after hours utilisation fol- decline in number of in-hours service by comparison was lowing the introduction of the Deputising Service but no quite small in comparison. Following the introduction of change in the Local Triage centre area. Thus, after hours uti- the AHPMCT there was a significant reduction in the total lisation rates decreased in four of the six AHPMCT areas monthly after hours utilisation rate (RR = 0.62, 0.61 – (three substantially) – there was no change in one area 0.64) when compared with the National comparator. This and a small increase in the other. excluded the unbilled services provided by the Deputising Service (2.0 per 10,000 population per month), the effects The variation in after hours service utilisation rate in the of which should be very small. six areas at baseline from 49.5 per 10,000 monthly in the Local Triage centre to 6.6 per 10,000 per month in the Met- Local Triage centre ropolitan area of the Regional Call Centre is noteworthy The total monthly after hours utilisation rate in the Local (after hours service utilisation rate was 30.0 per 10,000 Triage centre area was 65.0% higher than rates elsewhere monthly in the rest of Australia at that time). in Australia judged by the National Comparator. Table 5: Utilisation rates of MBS services (per 10,000 population): AHPMCT stand-alone services – Regional Call Centre (Metropolitan): May 98/Apr99 (532,478) 3548.0 6.6 2.8 0.8 0.3 0.2 1.6 0.8 0.0 0.1 May99/Apr00 (536,886) 3468.4 6.6 3.2 0.9 0.4 0.1 1.5 0.5 0.0 0.0 Trial period begins May99/Dec00 (539,030) 3473.4 6.6 3.4 1.0 0.2 0.2 1.0 0.7 0.0 0.0 Page 5 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Table 6: Utilisation rates of MBS services (per 10,000 population): AHPMCT stand-alone services – Regional Call Centre (Non- metropolitan): May98/Apr99 (21,261) 3927.8 12.7 3.3 1.4 0.0 0.0 8.0 0.0 0.0 0.0 May 99/Apr00 3468.1 8.1 5.2 0.5 0.0 0.0 2.4 0.0 0.0 0.0 Trial period begins May 00/Dec00 3058.0 5.3 2.9 0.5 0.0 0.0 1.9 0.0 0.0 0.0 Some limitations of the study should be acknowledged. clarity concerning this issue as the phenomenon of shift in As noted, the impact of the trials was monitored using after hours GP care from emergency (ie first callout) after monthly data before and after the introduction of the tri- hours care and identified as such to non-emergency (ie als taking into account secular trend at the national level after first callout) and not identified as after hours would only. This is a relatively weak study design However it was be common. not possible nor intended to compare the success of indi- vidual trials which had different health service character- In the Deputising Service a 'cashing-out' arrangement was istics, and resident population characteristics and put in place – after hours GPs were paid on a salary basis, consequently population health needs, as well as some- patients paid no fee for a home visit. This shift in payment what different service aims and arrangements. In general, arrangement from fee-for-service to salary rather than tel- local data collection at EDs and ambulance services was ephone triage could have been responsible for reducing inconsistently recorded and it was not possible to esti- utilisation. mate robustly the effects of the AHPMCTs on these serv- ices. Immediate after hours GP service was only studied – In all AHPMCTs, the decrease in after hours utilisation not subsequent in-hours GP or specialist use. Finally, as rates after the introduction of the AHPMCT could be due noted, only emergency GP after hours usage was identi- to other factors unrelated to the AHPMCT, such as an fied in the MBS. acceleration of the longer-term decline in after hours uti- lisation rates in the trial areas. As the AHPMCTs were Some possible explanations for these changes in after established in response to local needs and difficulties in hours service utilisation, other than impact of the AHPM- relation to after hours services, it is possible that these CTs need to be considered. In the Statewide Call Centre – local needs and difficulties continued after the establish- Metropolitan area, a number of new after hours clinics, ment of the AHPMCT. For example in the Non-metropoli- unrelated to the AHPMCT, set up operation at the same tan area of the Regional Call Centre there had been a crisis time as the AHPMCT. They could be expected therefore to in the provision of after hours services with almost no have effects on after hours utilisation rates that were inde- unsociable hours care being provided by GPs. pendent of the AHPMCT. These clinics typically provided non-emergency services not reimbursed in the MBS as an Some patients may have crossed service boundaries to after hours care item, and their effects would not be receive their after hours care and this was not recorded in recorded as an (emergency) after hours items. However our data collection system. We only have estimates of the some effect on emergency after hours use and some con- magnitude of the opposite phenomenon where patients fusion with AHPMCT effects are possible. Since the com- beyond the boundaries received their after hours care in pletion of the AHPMCTs, a new MBS item has been AHPMCT areas. This was measured at around 5% of all introduced to reimburse routine after hours service use. Its local GP usage in AHPMCT areas (and included in our existence during the trial would have provided greater Table 7: Utilisation rates of MBS services (per 10,000 population) – AHPMCT embedded services – Deputising Service. Oct97/Sep98 (229,593) 3888.5 39.9 9.6 12.7 2.2 0.5 7.1 5.4 2.3 0.1 Oct98/Sep99 (229,223) 3799.4 35.4 8.6 12.3 1.8 0.5 4.9 4.4 2.7 0.1 Trial period begins* Oct99/Sep00 (229,019) 3721.4 22.3 5.8 10.9 0.6 0.3 1.4 3.2 0.1 0.2 * excludes unbilled services provided by the Deputising Service. If these are included there were 24.3 per 10,000 after hours services provided in Oct99/Sep00. Data including for unbilled services not available for other than total after hours services. Page 6 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Table 8: Utilisation rates of MBS services (per 10,000 population) – AHPMCT embedded services – Local Triage centre. Oct97/Sep98 (21,395) 2467.4 49.5 35.5 0.5 4.7 0.0 7.5 0.5 0.9 0.0 Oct98/Sep99 (21,309) 2488.6 43.2 32.8 0.5 5.6 0.0 4.2 0.0 0.5 0.0 Trial period begins Oct99/Sep00 (21,259) 2510.5 44.7 31.5 0.5 5.6 0.0 5.6 0.0 0.9 0.0 analysis). While this phenomenon could introduce some and population survey dataset for the Local Triage centre. 'noise', it should not affect the validity of results. In general, there were greater reductions in after hours uti- lisation in stand-alone services than embedded services in In assessing to what extent the AHPMCTs were responsi- this MBS study, the opposite being true in the population ble for the changes in GP after hours utilisation, it is survey study. worthwhile to consider to what extent there was take-up of AHPMCT services. The telephone triage call rate in the These results need to be interpreted in the light of the lim- four AHPMCTs varied between 50 and 6 after hours calls/ itations and differences of both databases – the self- month/10,000 population in the Metropolitan area of the reported nature of the population survey but very impor- Statewide Call Centre and the Non-metropolitan area of tantly the absence of non-emergency after hours usage in the Regional Call Centre respectively (possibly related to the MBS database. While the two data collection method- the level of marketing and promotion for the service). The ologies focus on after hours GP care, they are capturing uptake of telephone triage, with the exception of the quite different service usage. Regional Call Centre (Non-metropolitan area), prima facie should have been sufficient to produce an effect on utili- These Australian findings, presented for the first time, are sation levels and service mix. generally consistent with the conclusion of the structured review on after hours care – namely that the growth in tel- It is also worthwhile to consider to what extent these MBS ephone triage and advice services usually, but not always, after hours utilisation results were consistent with those reduces immediate medical workload through the substi- reported in our earlier population survey study popula- tution of telephone consultations for in-person consulta- tion [5]. Stand-alone services did not align well. There was tions [5]. a significant reduction in after hours GP service usage in the MBS dataset but a significant increase in the popula- In considering the policy significance of these findings, it tion dataset in the Statewide Call Centre – Metropolitan is relevant to note that after hours service provision has area. This might be explained as an effect of the opening moved on in Australia – as it has in other countries – since of the after hours GP clinics unrelated to the AHPMCT in the conduct of this study with the introduction of second Perth with this extra activity generating additional MBS in- and third phase trials and the recent decision by COAG to hours rather than after hours item numbers as these fund a National Health Call Centre Network provided by would not have been emergency in nature and not nurses on a 24 hour basis [8] and the launch, for example, recorded as an after hours MBS item. There was a signifi- of Nurse-On-Call in Victoria. cant reduction in after hours GP service usage in the MBS dataset in the Statewide Call Centre – Non-metropolitan area These results need to be considered in conjunction with but no change in the population dataset. There was a the results from other studies in this series. [4,7,9]. The small significant increase in after hours GP service usage first of these, the structured review concluded that the in the MBS dataset in the Regional Call Centre – Metropoli- beneficial effects of telephone triage services in reducing tan area but no change in the population dataset. There immediate medical workload has to be balanced with the was a significant reduction in after hours GP service usage finding of reduced patient satisfaction when in-person in the MBS dataset but a significant increase in the popu- consultations are replaced by telephone consultations. lation dataset in the Regional Call Centre – Non-metropoli- This may partly explain some inconsistent results for the tan area. stand-alone call centres that were established as a new type of service aimed at better addressing population MBS results for embedded services aligned better with needs. These inconsistent results include that population results. There was a reduction for MBS service use and less GP clinic use (both per person contact and  the stand-alone services are clearly well used (results frequency) in the population survey for the Deputising indicate that like the other triage services there is a non- Service. There was no change in use reported in both MBS Page 7 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 significant trend to reduce unmet population need for they were provided in the GP's consultation room and after hours care) whether they were provided in unsociable hours (11 pm– 7 am) or not – see Table 2 for details. MBS after hours  access to after hours GP services in the stand-alone serv- items cover only emergency (GP first call-out) not routine ices, however measured, did not improve; and after hours care, as previously noted.  results for utilisation effects of their services on GP after 2 In-hours items (3, 4, 19, 20, 23, 24, 25, 33, 35, 36, 37, 38, hours usage, unlike for embedded services, are not con- 40, 43, 44 and 47). These items contain 96% of services in sistent for population and MBS datasets. the General Practice Profile developed by the Australian Department of Health and Aged Care [10]. Conclusion The findings in general indicated that the telephone triage For States where the trials took place, the data was divided and advice services provided by the AHPMCTs usually, into two categories: trial areas and non-trial areas. For but not always reduced GP emergency after hours care. It States where no trials took place, the whole state was is possible that other events and circumstances could be deemed to be a non-trial area. Patients and GPs could be responsible for some of the reduction in after hours serv- assigned to the trial and non-trial areas using residential ice use. These include other services opening in one AHP- postcodes for patients and practice location for GPs. MCT area and new financial arrangements for AHPMCT care in another. The goals of stand-alone call centres to Monthly data for these MBS items was collected for the 3- meet population needs may be being met only in part. year period November 1997 (when new after hours MBS Being demonstration programs, at least in the Australian items were introduced) to December 2000 (allowing time context, these goals may be more fully met over time. for full collection of data at the end of this period). Con- sultation rates are reported for GPs practising in trial areas and all their patients irrespective where they resided. Methodology Study design The national evaluation constituted 'multiple trials' with A 'cashing-out' arrangement was put in place for the Dep- common questions and hypotheses, rather than a 'multi- utising Service – after hours GPs were paid on a salary basis centred' trial with common protocols [1]. This reflected and received no fee for a home visit. No claim by the the underlying goals of the AHPMCTs which were to meet patient or GP was made therefore for reimbursement local circumstances and needs, and to investigate different (payment) from the HIC. These services therefore were models and types of after hours PMC care most suitable not recorded and counted in the MBS database. They were given these local circumstances. The individual trials were counted separately under arrangements made with the therefore studied separately. While they are not directly trial operators. compared, their relative success is considered. ED and ambulance data A monitoring strategy was adopted, using a pre-post Data collection was attempted in all AHPMCT areas. design to detect changes in relevant variables in the trial However high quality data was only available for the area across the study period. To examine whether these Regional Call Centre and even here there were problems in changes could reflect period effects occurring nationally, the Metropolitan area in this trial as ED attendances that changes occurring in the trial area were compared with were recorded, included attendances at the GP after hours those occurring in the rest of Australia outside of AHPM- walk-in clinic operated by the Regional Call Centre adja- CTs areas (the National Comparator). cent to the ED for part of the study period – see Table 1. It was not possible therefore to conduct a useful analysis of Data collection the impact of the AHPMCTs on ED and ambulance service MBS data usage. MBS data from the HIC was provided by the Department Data analysis of Health and Aged Care (as it was then named) for after hours and in-hours GP services for all States and Territo- Monthly total after hours GP utilisation rates per 10,000 ries. After hours was defined by the HIC as 8 pm-8 am population for both pre-trial and trial periods were com- weeknights, 1 pm–12 pm Saturdays and all day Sunday pared in each AHPMCT area. Changes in each site were and public holidays. also compared with the National comparator (over the same monthly periods) to control for the effects of secular 1 After hours items (1, 2, 97, 98, 601, 602, 697 and 698). trend – see Table 9. Poisson regression analysis was used Their use depends on whether services were provided by as the data consisted of counts of relatively rare events in Vocationally Registered (VR) or non-VR GPs, on whether a cohort of subject over a defined time period. The princi- Page 8 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Table 9: Impact of AHPMCTs on GP After hours utilisation rates*. PRE TRIAL TRIAL PERIOD Period (Months) Average 95% CI Period (Months) Average 95% CI Statewide Call Centre (Metropolitan) 16 19.3 14.8 – 23.8 20 15.9 12.9 – 18.8 National Comparator 16 30.1 23.6 – 36.6 20 28.3 23.2 – 33.4 Relative Rates 0.87 (95% CI 0.86 – 0.88) Statewide Call Centre (Non-metropolitan) 33 14.2 4.6 – 23.8 3 8.4 4.7 – 12.2 National Comparator 33 29.1 23.0 – 35.3 3 28.8 28.6 – 29.0 Relative Rates 0.60 (95% CI 0.54 – 0.68) Regional Call Centre (Metropolitan) 31 6.7 4.5 – 8.8 5 7.1 5.2 – 8.9 National Comparator 31 29.3 23.4 – 35.5 5 28.0 25.3 – 30.6 Relative Rates 1.11 (95% CI 1.06 – 1.17) Regional Call Centre (Non-metropolitan) 31 11.5 0.4 – 22.6 5 5.1 1.7 – 8.4 National Comparator 31 29.3 23.4 – 35.5 5 28.0 25.3 – 30.6 Relative Rates 0.46 (95% CI 0.35 – 0.61) Deputising Service 24 38.5 27.4 – 49.5 12 23.0 14.8 – 31.3 National Comparator 24 29.9 24.2 – 35.6 12 28.4 23.1 – 33.7 Relative Rates 0.62 (95% CI 0.61 – 0.64) Local Triage service 24 48.7 26.2 – 71.2 12 45.8 31.2 – 60.4 National Comparator 24 29.9 24.2 – 35.6 12 28.4 23.1 – 33.7 Relative Rates 1.01 (95% CI 0.94 – 1.08) * There are minor discrepancies in rates presented in Tables 2 and 3 arising from different time periods being used. pal independent variables (additional to the constant) Competing interests were: time (pre/during trial), intervention status (trial The author(s) declare that they have no competing inter- ests. area/National comparator), and trial effect (time*inter- vention status). This generated a relative rate (and 95% confidence intervals) for the effect of the AHPMCT, Authors' contributions expressed below: DD made a substantial contribution to conception and design, acquisition of data, analysis and interpretation of data and was involved in drafting the manuscript. SED UU / TA.. T TA T Relative rate () RR = made a substantial contribution to acquisition of data, UU / NC.. T NC T analysis and interpretation of data and was involved in revising the manuscript critically for important intellec- Where U is the after hours utilisation rate per 10,000 tual content. RW made a substantial contribution to TA.T1 population per month in the Trial Area (TA) during the acquisition of data, analysis and interpretation of data. trial MK made a substantial contribution to data analysis and was involved in revising the manuscript critically for Where U is the after hours utilisation rate per 10,000 important intellectual content. LG made a substantial TA.T0 population per month in the Trial Area pre-trial contribution to data analysis made a substantial contribu- tion to conception and design and was involved in revis- Where U is the after hours utilisation rate per 10,000 ing the manuscript critically for important intellectual NC.T1 population per month in the National Comparator (NC) content. area during the trial Acknowledgements The study was funded by the Commonwealth Department of Health and Where U is the after hours utilisation rate per 10,000 NC.T0 Ageing (DoHA). It owned intellectual property of the study but granted to population per month in the National Comparator area The University of Melbourne a non-exclusive, royalty-free license to use pre-trial the study for academic purposes such as publishing in peer-refereed jour- nals. DD and LG are funded by The University of Melbourne. SED and RW SPSS for Windows version 12.0.1 was used for the analy- were funded by DoHA funding for the study and more recently by an Aus- sis. tralian Research Council scholarship. MK is funded by an NHMRC Mid Career Development Grant. The work was overseen by an Evaluation and Policy Advisory Group (EAG). Page 9 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 References 1. Dunt D, Day S, van Dort P: After Hours Primary MedicalCare Trials – National Evaluation Report. [http://www.health.gov.au/ internet/wcms/publishing.nsf/content/health-pcd-programs-ahpmc- publications-ahpmctne.htm]. 2. Hallam L: Primary medical care outside normal working hours : review of published work. BMJ 1994, 308(6923):249-253. 3. Developing NHS Direct: a strategy document for the next three years [http://www.dh.gov.uk/en/Publicationsandstatistics/ Publications/PublicationsPolicyAndGuidance/DH_4005397] 4. Leibowitz R, Day S, Dunt D: A systematic review of the effect of different models of after hours primary medical care serv- ices on clinical outcome, medical workload, and patient and GP satisfaction. Fam Pract 2003, 20:311-317. 5. Munro J, Nicholl J, O'Cathain A, Knowles E: Impact of NHS Direct on demand for immediate care : observational study. BMJ 2000, 321(7254):150-153. 6. Lattimer V, George S, Thompson F, Thomas E, Mullee M, Turnbull J, Smith H, Moore M, Bond H, Glasper A: Safety and effectiveness of nurse telephone consultation in out-of-hours primary care: randomised controlled trial. The South Wiltshire Out of Hours Project (SWOOP) Group. BMJ 1998, 317(7165):1054-1059. 7. Dunt D, Day SE, Kelaher M, Montalto M: Impact of stand-alone and embedded telephone triage systems on after hours pri- mary medical care service utilisation and mix in Australia. Australia and New Zealand Health Policy 2005, 2:30. 12 December 2005 8. Council of Australian Governments' Meeting 10 February 2006 Attachment D: Better health for all Australians Action Plan Promoting good health, prevention and early intervention. [http://www.coag.gov.au/meetings/100206/ attachment_d_better_health.pdf]. 9. Dunt D, Day SE, Kelaher M, Montalto M: The impact of stan- dalone call centres and GP cooperatives on access to after hours GP care: a before and after study adjusted for secular trend. Fam Prac 2006, 23(4):453-460. 10. Commonwealth Department of Health & Aged Care: The relative value study: Stage 3 modelling – a technical report. Canberra, Commonwealth Department of Health & Aged Care; 2001. 11. Department of Primary Industries and Energy and Department of Human Services and Health: Rural, Remote and Metropolitan Areas Classification. 1991 Census Edition. Department of Pri- mary Industries and Energy and Department of Human Services and Health Canberra: Australian Government Publishing Service; 1994. Publish with Bio Med Central and every scientist can read your work free of charge "BioMed Central will be the most significant development for disseminating the results of biomedical researc h in our lifetime." Sir Paul Nurse, Cancer Research UK Your research papers will be: available free of charge to the entire biomedical community peer reviewed and published immediately upon acceptance cited in PubMed and archived on PubMed Central yours — you keep the copyright BioMedcentral Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 10 of 10 (page number not for citation purposes) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Australia and New Zealand Health Policy Springer Journals

Impact of telephone triage on emergency after hours GP Medicare usage: a time-series analysis

Loading next page...
 
/lp/springer-journals/impact-of-telephone-triage-on-emergency-after-hours-gp-medicare-usage-mgEbCpPIvf
Publisher
Springer Journals
Copyright
Copyright © 2007 by Dunt et al; licensee BioMed Central Ltd.
Subject
Medicine & Public Health; Public Health; Social Policy
ISSN
1743-8462
eISSN
1743-8462
DOI
10.1186/1743-8462-4-21
pmid
17927836
Publisher site
See Article on Publisher Site

Abstract

Background: The Australian government sponsored trials aimed at addressing problems in after hours primary medical care service use in five different parts of the country with different after hours care problems. The study's objective was to determine in four of the five trials where telephone triage was the sole innovation, if there was a reduction in emergency GP after hours service utilization (GP first call-out) as measured in Medicare Benefits Schedule claim data. Monthly MBS claim data in both the pre-trial and trial periods was monitored over a 3-year period in each trial area as well as in a national sample outside the trial areas (National comparator). Poisson regression analysis was used in analysis. Results: There was significant reduction in first call out MBS claims in three of the four study areas where stand-alone call centre services existed. These were the Statewide Call Centre in both its Metropolitan and Non-metropolitan areas in which it operated – Relative Risk (RR) = 0.87 (95% Confidence interval: 0.86 – 0.88) and 0.60 (95% CI: 0.54 – 0.68) respectively. There was also a reduction in the Regional Call Centre in the non-Metropolitan area in which it operated (RR = 0.46 (95% CI: 0.35 – 0.61) though a small increase in its Metropolitan area (RR = 1.11 (95% CI: 1.06 – 1.17). For the two telephone triage services embedded in existing organisations, there was also a significant reduction for the Deputising Service – RR = 0.62 (95% CI: 0.61 – 0.64) but no change in the Local Triage centre area. Conclusion: The four telephone triage services were associated with reduced GP MBS claims for first callout after hours care in most study areas. It is possible that other factors could be responsible for some of this reduction, for example, MBS submitted claims for after hours GP services being reclassified from 'after hours' to 'in hours'. The goals of stand-alone call centres which are aimed principally at meeting population needs rather than managing demand may be being met only in part. Page 1 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Previous research on the APHMCTs using population- Background This paper reports further on the national evaluation of based telephone survey data of after hours service utilisa- the After Hours Primary Medical Care Trials (AHPMCTs) tion found that the introduction of telephone triage serv- which were a recent initiative of the Australian Govern- ices, whether embedded in other services or stand-alone, ment. The goals of these trials were to improve the quality was not clearly associated with reduction in after hours of service delivery as well as consumer acceptability, con- use or shift towards the use of GP after hours clinics [7]. It sumer access (including affordability) and equity, appro- is important therefore to extend analysis to include not priateness of service mix, provider satisfaction with regard only self-reported data, but also administrative data rou- to their impact on service mix as well as service use more tinely recorded on service utilisation. This study presents generally [1]. Medicare Benefits Schedule (MBS) data from Medicare Australia (formerly the Health Insurance Commission The common feature of these trials was the use of tele- [HIC]) bearing in mind that this data too had limitations. phone triage. Telephone triage and advice services have an At the time of the study, MBS data recorded only GP emer- important place in the development of after hours care in gency (first doctor call-out whether to consulting rooms other countries. Most frequently these services are embed- or patient's home) after hours service use and not all other ded in other after hours services such as in GP coopera- routine after hours GP use. Thus the data forms one small tives in the UK, HMOs in the US and the county-based but important part of all after hours service use. service arrangements in Denmark introduced in the 1990s [2]. A small number of stand-alone services have also The aim of this study then is to determine if there is a been established. NHS Direct is a 24 hour, confidential reduction in emergency (GP first callout) after hours serv- telephone, online and interactive digital TV health advice ice use recorded in MBS data following the introduction and information service provided by the National Health of different forms of telephone triage in the AHPMCTs, Service in England and Wales (similar service in Scot- where these were the sole service innovation. land). NHS Direct was rolled out across England and Wales between 1998 and 2000. The telephone service The After Hours Primary Medical Care Trials aims to triage symptomatic callers to provide guidance on Telephone triage services used by the four study AHPM- which healthcare provider the caller should access. Nurses CTs varied both in the form of the telephone triage service using proprietary health call centre software also give offered and its service function within the individual advice on how to manage an episode of illness at home. AHPMCT as follows [1]. Health Information Advisors can provide information on Stand-alone services a wide range of medical conditions, treatments, medicines and NHS services. In some areas of England and Wales, These were call centres where nurses, using proprietary NHS Direct is commissioned by local Primary Care Trusts health call centre software, aimed at providing more (PCTs) to provide the gateway for out-of-hours access to accessible advice and promoting more appropriate after GP's surgeries and clinics. [3]. hours primary medical care (PMC) service use. They were: A structured review on the impact of after hours GP serv- • the stand-alone Statewide Call Centre (studied in both ices on clinical outcomes, medical workloads as well as Metropolitan and Non-metropolitan areas); and patient and GP satisfaction concluded that this growth in the use of telephone triage and advice services usually, but  the stand-alone Regional Call Centre (studied in both not always reduced immediate medical workload through inner metropolitan (Metropolitan) and rural satellite areas the substitution of telephone consultations for face-to- (Non-metropolitan); face consultations [4]. For example, a before and after study following the introduction of NHS Direct as a stand- Embedded services alone service in the UK found a small, but significant These triage and advice systems did not use proprietary reduction in use of GP co-operatives but no change in use health call centre software. They aimed more at managing of Emergency Department (ED) and ambulance services demand to support the GP workforce in terms of recruit- in the study area [5]. Considering embedded services, a ment and retention. They were: randomised controlled trial from the UK compared a nurse telephone consultation service integrated within a  a GP-based telephone triage and advice service (without GP co-operative with the usual practice of that co-opera- guidelines or software) in a well-established, pre-existing tive [6]. There was a 69% reduction in telephone advice Deputising Service; from a GP, a 38% reduction in patient attendance at pri- mary care centres and a 23% reduction in home visits.  the Local Triage and advice service using hospital nurses with locally developed, paper-based protocols to support Page 2 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 after hours services in GP own-practice arrangements in a sulting rooms (75.7% in 97/98, 76.4% in 99/00). Services small rural community. provided by non-VR GPs, compared to VR GPs were most common in unsociable hours. The AHPMCT services together with the regional context of the trials and their organisational settings are summa- There was a 6.7% reduction in after hours utilisation rates rised in Table 1. It should be noted that while the from 30.0 to 28.0 per 10,000 population per month Statewide Call Centre received funding as part of the AHP- across the study period outside of the trial areas. There was MCTs this was mainly for logistic and information sup- also a 3.8% decline in the in-hours utilisation rate during port. The service opened a short time before the other this time. The decline in number of after hours services AHPMCTs with separate Australian and State government occurred more in the consulting room (9.6%) than out- funding. While it provides 24-hour advice on a variety of side (5.7%). This decline also occurred in sociable hours health-related topics, its involvement in after hours pri- (8–11 pm) (11.9%) but not in unsociable hours (11 pm– mary care is the focus of this paper. The AHPMCTs com- 8 am) which increased substantially (20.8%). menced operation during 1999–2000. Variation in the Stand-alone services – see Tables 3, 4, 5, 6 date of commencement affected whether or not one or two years of pre-trial data was available. It also meant, for Statewide Call Centre – Metropolitan a few trials, that data covered less than 12 months of oper- The total monthly after hours utilisation rate in the area ation, introducing unavoidable seasonal bias. was low – approximately 63.7% of rates elsewhere in Aus- tralia judged by the National Comparator. Results National comparator – see Table 2 There was a 17.8% reduction in total monthly after hours After hours utilisation rates were very small in number utilisation from 19.1 to 15.7 per 10,000 population per compared to in-hours utilisation rates. VR GPs provided month following the introduction of the AHPMCT. The most after hours services (59.3% in 97/98, 62.9% in 99/ decline in number of in-hours service by comparison was 00). Most after hours services were provided in sociable quite small (2.9%). Reduction in both after hours consult- hours (84.0% in 97/98, 79.3% in 99/00) and outside con- ing room and non-consulting room services was substan- Table 1: Local context and services of individual trials. Trial Catchment Areas * Frequency of telephone calls to Notes service (calls/month/10,000 head of population) [1] 1. Statewide Call Capital city, small rural centre and Other 50 (Metropolitan) Data not available 1 Operator with prior international industry experience Centre rural area 1.42M (Non-metropolitan) under contract to the State Government) – phase-in (Observation period for trial Feb99–Jan00 period when calls to a public hospitals were transferred to and Feb00–Dec00). Rest of state – small rural the call centre centre and other rural area studied 42,500 2 Free to caller (Observation period for trial July00–Dec00) 3 Coincidentally, a number of capital city after hours walk- in GP clinics opened around the time the Statewide Call Centre was established. 2. Regional Call Capital city (inner region) 537,000 Non- 18 (Metropolitan) 1 Startup operator – phase-in period when calls to two Centre metropolitan satellite: Small rural centre 6 (Non-metropolitan) public hospitals were transferred to the call centre 21,000 2 Free to caller (Observation period for trial May00–Dec 00) 3 A GP after hours walk-in clinic operating with no fee to patients located adjacent to the ED of a regional hospital in the central metropolitan area operating for part of the study period but closed due to lack of demand. 4 Prior restricted access to after hours GP care in non- metropolitan satellite 3. Deputising Capital city and Other rural area 229,000 31 (approx) 1 GP-based telephone triage and advice service Service (Observation period for trial – October 1999 2 Pre-existing and continuing Deputising Service offering an to September 2000) after hours walk-in clinic and home visiting service. 3 Service offered to patients of previously enrolled GP practices (85% of all practices in capital city) and new enrolled GP practices beyond the metropolitan area (52% practice take-up). 4 Commonwealth government funding of the program derived from 'cashing out' historical MBS reimbursements to the service – home visits by Deputising Service staff to patients in capital city (only) during the trial not billed to patient. 4. Local Triage and Other rural area (ie small country town and 24 (approx) 1 Hospital nurses using locally developed paper-based advice service surrounding community) 21,000 protocols (Observation period for trial – October 1999 2 Located in a local hospital in a rural community. to September 2000) 3 Aim to support GP own-practice after hours arrangement to relieve GP after hours burden. * Based on Rural, Remote and Metropolitan Areas classification (RRMA) [11] † Source: Australian Bureau of Statistics population projections for 1999 supplied to the Department of Health and Ageing Page 3 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Table 2: Utilisation rates of MBS services (per 10,000 population): National comparator. TIME IN-HOURS AFTER VR GPS NON VR GPS PERIOD HOURS (POPULA TION) Not "UNSOCIABLE" "UNSOCIABLE" HOURS NOT "UNSOCIABLE" "UNSOCIABLE" HOURS HOURS HOURS Not CON CON NOT CON CON NOT CON CON Not CON CON ROOMS ROOMS ROOMS ROOMS ROOMS ROOMS ROOMS ROOMS 1 2 601 602 97 98 697 698 Oct97/Sep98 4025.3 30.0 11.0 4.2 2.0 0.7 7.8 2.2 1.9 0.2 (16,344,730) Oct98/Sep99 3975.5 29.2 11.4 4.1 2.4 0.9 6.4 1.5 2.3 0.4 (16,517,748) Oct99/Sep00 3871.7 28.0 10.7 3.9 2.0 1.0 6.3 1.3 2.4 0.4 (16,689,060) Notes: For the non-trial areas the period October 1997 to September 2000 was analysed as this represented the most common period over which the trials were conducted. Sociable hours (8–11 pm)and unsociable hours (11 pm–8 am) tial (7.7% and 14.6% respectively) in the first 12 months. tre. In-hours items were relatively stable across the 3-years There was a further large reduction in consulting room period by comparison. Non-consulting room services services (41.7%) but not non-consulting room (2.6%) in declined from 82.9% to 65.1% as a proportion of total the following 11 months. A higher proportion of after after hours services over the 3-year period (compared to hours services were outside of the consulting room (93.2 75.7 – 76.4% nationally outside the trial areas). – 95.5%) compared with 75.7 – 76.4% in the National comparator area. Following the introduction of the AHPMCT, there was a significant reduction in the total monthly after hours uti- Following the introduction of the AHPMCT, there was a lisation rate (RR = 0.60, 95% CI: 0.54 – 0.68) when com- significant reduction in the total monthly after hours uti- pared with the National comparator. lisation rate (RR = 0.87 (95% CI: 0.86 – 0.88)) when com- pared with the National comparator. Regional Call Centre – Metropolitan The total monthly after hours utilisation rate in the area Statewide Call Centre – Non-metropolitan was low – approximately 23% of rates elsewhere in Aus- The total monthly after hours utilisation rate in the area tralia judged by the National Comparator. The total was low, 50.3% of rates elsewhere in Australia judged by monthly after hours utilisation rate remained steady at 6.6 the National Comparator. After hours services were pre- per 10,000 population per month following the introduc- dominantly supplied by VR GPs (in excess of 80%). Very tion of the Regional Call Centre. Services in and out of the few (less than 10%) of total after hours services were sup- consulting room also remained steady. plied in unsociable hours and all of these were supplied by VR GPs. Following the introduction of the AHPMCT, there was a small increase in the total monthly after hours utilisation There was a 36.4% reduction in total monthly after hours rate (RR = 1.11, 95% CI: 1.06 – 1.17) when compared utilisation rate from 15.1 to 9.6 per 10,000 population with the National comparator. per month after the introduction of the Statewide Call Cen- Table 3: Utilisation rates of MBS services (per 10,000 population): AHPMCT stand-alone services – Statewide Call Centre (Metropolitan). Feb98/Jan99 (1,398,781) 3920.6 19.1 4.8 0.7 1.2 0.0 8.5 0.6 3.3 0.0 Trial period begins Feb99/Jan00 (1,424,036) 3881.6 16.3 4.9 0.7 1.3 0.0 6.4 0.5 2.6 0.0 Feb00/Dec00 (1,449,729) 3790.0 15.7 3.9 0.5 0.8 0.0 7.5 0.2 2.6 0.0 Page 4 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Table 4: Utilisation rates of MBS services (per 10,000 population): AHPMCT stand-alone services – Statewide Call Centre (Non- metropolitan). Jul98/Jun99 (118,162) 2892.9 15.1 10.2 2.1 1.4 0.2 1.0 0.3 0.0 0.0 Jul99/Jun00 (120370) 2870.9 9.6 4.7 3.4 0.7 0.1 0.7 0.2 0.0 0.0 Trial period begins Jul00/Dec00 (122,211) 2810.1 8.6 4.0 2.5 0.5 0.1 1.1 0.4 0.0 0.0 Regional Call Centre – Non-metropolitan There was a 9.7% reduction in total monthly after hours The total monthly after hours utilisation rate in the area utilisation rate from 49.5 to 44.7 per 10,000 population was low – approximately 42.3% of rates elsewhere in Aus- per month after the introduction of the AHPMCT. (Rates tralia judged by the National Comparator. and change in rates should be interpreted cautiously as numbers were small and confidence intervals wide.) The total monthly after hours utilisation rate reduced by Nearly all after hours services (98–99%) were delivered 58.3% from 12.7 to 5.3 per 10,000 population per month outside the consulting room and this did not change dur- after the introduction of the AHPMCT. Following the ing the trial (compared to 76–77% in the National Compa- introduction of the AHPMCT there was a significant rator area. Following the introduction of the AHPMCT the reduction in the total monthly after hours utilisation rate total monthly after hours utilisation rate did not change (RR = 0.46, 95% CI: 0.35 – 0.61) when compared with the (RR = 1.01, 0.94 – 1.08) and when compared with the National comparator. National comparator. Embedded services – see Tables 7, 8 Discussion Deputising Service Considering first stand-alone services, there was a signifi- The total monthly after hours utilisation rate in the area cant reduction in after hours utilisation in three of the was approximately 33% higher than rates elsewhere in four study areas – the Statewide Call Centre in both Metro- Australia judged by the National Comparator. politan and Non-metropolitan areas, (larger in the Non-met- ropolitan area) as well as the Non-metropolitan areas in the There was a 39.1% reduction in total monthly after hours Regional Call Centre. (There was a small increase in the utilisation rate from 39.9 to 24.3 per 10,000 population Metropolitan area). Considering embedded services, there per month after the introduction of the AHPMCT. The was a significant reductionin after hours utilisation fol- decline in number of in-hours service by comparison was lowing the introduction of the Deputising Service but no quite small in comparison. Following the introduction of change in the Local Triage centre area. Thus, after hours uti- the AHPMCT there was a significant reduction in the total lisation rates decreased in four of the six AHPMCT areas monthly after hours utilisation rate (RR = 0.62, 0.61 – (three substantially) – there was no change in one area 0.64) when compared with the National comparator. This and a small increase in the other. excluded the unbilled services provided by the Deputising Service (2.0 per 10,000 population per month), the effects The variation in after hours service utilisation rate in the of which should be very small. six areas at baseline from 49.5 per 10,000 monthly in the Local Triage centre to 6.6 per 10,000 per month in the Met- Local Triage centre ropolitan area of the Regional Call Centre is noteworthy The total monthly after hours utilisation rate in the Local (after hours service utilisation rate was 30.0 per 10,000 Triage centre area was 65.0% higher than rates elsewhere monthly in the rest of Australia at that time). in Australia judged by the National Comparator. Table 5: Utilisation rates of MBS services (per 10,000 population): AHPMCT stand-alone services – Regional Call Centre (Metropolitan): May 98/Apr99 (532,478) 3548.0 6.6 2.8 0.8 0.3 0.2 1.6 0.8 0.0 0.1 May99/Apr00 (536,886) 3468.4 6.6 3.2 0.9 0.4 0.1 1.5 0.5 0.0 0.0 Trial period begins May99/Dec00 (539,030) 3473.4 6.6 3.4 1.0 0.2 0.2 1.0 0.7 0.0 0.0 Page 5 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Table 6: Utilisation rates of MBS services (per 10,000 population): AHPMCT stand-alone services – Regional Call Centre (Non- metropolitan): May98/Apr99 (21,261) 3927.8 12.7 3.3 1.4 0.0 0.0 8.0 0.0 0.0 0.0 May 99/Apr00 3468.1 8.1 5.2 0.5 0.0 0.0 2.4 0.0 0.0 0.0 Trial period begins May 00/Dec00 3058.0 5.3 2.9 0.5 0.0 0.0 1.9 0.0 0.0 0.0 Some limitations of the study should be acknowledged. clarity concerning this issue as the phenomenon of shift in As noted, the impact of the trials was monitored using after hours GP care from emergency (ie first callout) after monthly data before and after the introduction of the tri- hours care and identified as such to non-emergency (ie als taking into account secular trend at the national level after first callout) and not identified as after hours would only. This is a relatively weak study design However it was be common. not possible nor intended to compare the success of indi- vidual trials which had different health service character- In the Deputising Service a 'cashing-out' arrangement was istics, and resident population characteristics and put in place – after hours GPs were paid on a salary basis, consequently population health needs, as well as some- patients paid no fee for a home visit. This shift in payment what different service aims and arrangements. In general, arrangement from fee-for-service to salary rather than tel- local data collection at EDs and ambulance services was ephone triage could have been responsible for reducing inconsistently recorded and it was not possible to esti- utilisation. mate robustly the effects of the AHPMCTs on these serv- ices. Immediate after hours GP service was only studied – In all AHPMCTs, the decrease in after hours utilisation not subsequent in-hours GP or specialist use. Finally, as rates after the introduction of the AHPMCT could be due noted, only emergency GP after hours usage was identi- to other factors unrelated to the AHPMCT, such as an fied in the MBS. acceleration of the longer-term decline in after hours uti- lisation rates in the trial areas. As the AHPMCTs were Some possible explanations for these changes in after established in response to local needs and difficulties in hours service utilisation, other than impact of the AHPM- relation to after hours services, it is possible that these CTs need to be considered. In the Statewide Call Centre – local needs and difficulties continued after the establish- Metropolitan area, a number of new after hours clinics, ment of the AHPMCT. For example in the Non-metropoli- unrelated to the AHPMCT, set up operation at the same tan area of the Regional Call Centre there had been a crisis time as the AHPMCT. They could be expected therefore to in the provision of after hours services with almost no have effects on after hours utilisation rates that were inde- unsociable hours care being provided by GPs. pendent of the AHPMCT. These clinics typically provided non-emergency services not reimbursed in the MBS as an Some patients may have crossed service boundaries to after hours care item, and their effects would not be receive their after hours care and this was not recorded in recorded as an (emergency) after hours items. However our data collection system. We only have estimates of the some effect on emergency after hours use and some con- magnitude of the opposite phenomenon where patients fusion with AHPMCT effects are possible. Since the com- beyond the boundaries received their after hours care in pletion of the AHPMCTs, a new MBS item has been AHPMCT areas. This was measured at around 5% of all introduced to reimburse routine after hours service use. Its local GP usage in AHPMCT areas (and included in our existence during the trial would have provided greater Table 7: Utilisation rates of MBS services (per 10,000 population) – AHPMCT embedded services – Deputising Service. Oct97/Sep98 (229,593) 3888.5 39.9 9.6 12.7 2.2 0.5 7.1 5.4 2.3 0.1 Oct98/Sep99 (229,223) 3799.4 35.4 8.6 12.3 1.8 0.5 4.9 4.4 2.7 0.1 Trial period begins* Oct99/Sep00 (229,019) 3721.4 22.3 5.8 10.9 0.6 0.3 1.4 3.2 0.1 0.2 * excludes unbilled services provided by the Deputising Service. If these are included there were 24.3 per 10,000 after hours services provided in Oct99/Sep00. Data including for unbilled services not available for other than total after hours services. Page 6 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Table 8: Utilisation rates of MBS services (per 10,000 population) – AHPMCT embedded services – Local Triage centre. Oct97/Sep98 (21,395) 2467.4 49.5 35.5 0.5 4.7 0.0 7.5 0.5 0.9 0.0 Oct98/Sep99 (21,309) 2488.6 43.2 32.8 0.5 5.6 0.0 4.2 0.0 0.5 0.0 Trial period begins Oct99/Sep00 (21,259) 2510.5 44.7 31.5 0.5 5.6 0.0 5.6 0.0 0.9 0.0 analysis). While this phenomenon could introduce some and population survey dataset for the Local Triage centre. 'noise', it should not affect the validity of results. In general, there were greater reductions in after hours uti- lisation in stand-alone services than embedded services in In assessing to what extent the AHPMCTs were responsi- this MBS study, the opposite being true in the population ble for the changes in GP after hours utilisation, it is survey study. worthwhile to consider to what extent there was take-up of AHPMCT services. The telephone triage call rate in the These results need to be interpreted in the light of the lim- four AHPMCTs varied between 50 and 6 after hours calls/ itations and differences of both databases – the self- month/10,000 population in the Metropolitan area of the reported nature of the population survey but very impor- Statewide Call Centre and the Non-metropolitan area of tantly the absence of non-emergency after hours usage in the Regional Call Centre respectively (possibly related to the MBS database. While the two data collection method- the level of marketing and promotion for the service). The ologies focus on after hours GP care, they are capturing uptake of telephone triage, with the exception of the quite different service usage. Regional Call Centre (Non-metropolitan area), prima facie should have been sufficient to produce an effect on utili- These Australian findings, presented for the first time, are sation levels and service mix. generally consistent with the conclusion of the structured review on after hours care – namely that the growth in tel- It is also worthwhile to consider to what extent these MBS ephone triage and advice services usually, but not always, after hours utilisation results were consistent with those reduces immediate medical workload through the substi- reported in our earlier population survey study popula- tution of telephone consultations for in-person consulta- tion [5]. Stand-alone services did not align well. There was tions [5]. a significant reduction in after hours GP service usage in the MBS dataset but a significant increase in the popula- In considering the policy significance of these findings, it tion dataset in the Statewide Call Centre – Metropolitan is relevant to note that after hours service provision has area. This might be explained as an effect of the opening moved on in Australia – as it has in other countries – since of the after hours GP clinics unrelated to the AHPMCT in the conduct of this study with the introduction of second Perth with this extra activity generating additional MBS in- and third phase trials and the recent decision by COAG to hours rather than after hours item numbers as these fund a National Health Call Centre Network provided by would not have been emergency in nature and not nurses on a 24 hour basis [8] and the launch, for example, recorded as an after hours MBS item. There was a signifi- of Nurse-On-Call in Victoria. cant reduction in after hours GP service usage in the MBS dataset in the Statewide Call Centre – Non-metropolitan area These results need to be considered in conjunction with but no change in the population dataset. There was a the results from other studies in this series. [4,7,9]. The small significant increase in after hours GP service usage first of these, the structured review concluded that the in the MBS dataset in the Regional Call Centre – Metropoli- beneficial effects of telephone triage services in reducing tan area but no change in the population dataset. There immediate medical workload has to be balanced with the was a significant reduction in after hours GP service usage finding of reduced patient satisfaction when in-person in the MBS dataset but a significant increase in the popu- consultations are replaced by telephone consultations. lation dataset in the Regional Call Centre – Non-metropoli- This may partly explain some inconsistent results for the tan area. stand-alone call centres that were established as a new type of service aimed at better addressing population MBS results for embedded services aligned better with needs. These inconsistent results include that population results. There was a reduction for MBS service use and less GP clinic use (both per person contact and  the stand-alone services are clearly well used (results frequency) in the population survey for the Deputising indicate that like the other triage services there is a non- Service. There was no change in use reported in both MBS Page 7 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 significant trend to reduce unmet population need for they were provided in the GP's consultation room and after hours care) whether they were provided in unsociable hours (11 pm– 7 am) or not – see Table 2 for details. MBS after hours  access to after hours GP services in the stand-alone serv- items cover only emergency (GP first call-out) not routine ices, however measured, did not improve; and after hours care, as previously noted.  results for utilisation effects of their services on GP after 2 In-hours items (3, 4, 19, 20, 23, 24, 25, 33, 35, 36, 37, 38, hours usage, unlike for embedded services, are not con- 40, 43, 44 and 47). These items contain 96% of services in sistent for population and MBS datasets. the General Practice Profile developed by the Australian Department of Health and Aged Care [10]. Conclusion The findings in general indicated that the telephone triage For States where the trials took place, the data was divided and advice services provided by the AHPMCTs usually, into two categories: trial areas and non-trial areas. For but not always reduced GP emergency after hours care. It States where no trials took place, the whole state was is possible that other events and circumstances could be deemed to be a non-trial area. Patients and GPs could be responsible for some of the reduction in after hours serv- assigned to the trial and non-trial areas using residential ice use. These include other services opening in one AHP- postcodes for patients and practice location for GPs. MCT area and new financial arrangements for AHPMCT care in another. The goals of stand-alone call centres to Monthly data for these MBS items was collected for the 3- meet population needs may be being met only in part. year period November 1997 (when new after hours MBS Being demonstration programs, at least in the Australian items were introduced) to December 2000 (allowing time context, these goals may be more fully met over time. for full collection of data at the end of this period). Con- sultation rates are reported for GPs practising in trial areas and all their patients irrespective where they resided. Methodology Study design The national evaluation constituted 'multiple trials' with A 'cashing-out' arrangement was put in place for the Dep- common questions and hypotheses, rather than a 'multi- utising Service – after hours GPs were paid on a salary basis centred' trial with common protocols [1]. This reflected and received no fee for a home visit. No claim by the the underlying goals of the AHPMCTs which were to meet patient or GP was made therefore for reimbursement local circumstances and needs, and to investigate different (payment) from the HIC. These services therefore were models and types of after hours PMC care most suitable not recorded and counted in the MBS database. They were given these local circumstances. The individual trials were counted separately under arrangements made with the therefore studied separately. While they are not directly trial operators. compared, their relative success is considered. ED and ambulance data A monitoring strategy was adopted, using a pre-post Data collection was attempted in all AHPMCT areas. design to detect changes in relevant variables in the trial However high quality data was only available for the area across the study period. To examine whether these Regional Call Centre and even here there were problems in changes could reflect period effects occurring nationally, the Metropolitan area in this trial as ED attendances that changes occurring in the trial area were compared with were recorded, included attendances at the GP after hours those occurring in the rest of Australia outside of AHPM- walk-in clinic operated by the Regional Call Centre adja- CTs areas (the National Comparator). cent to the ED for part of the study period – see Table 1. It was not possible therefore to conduct a useful analysis of Data collection the impact of the AHPMCTs on ED and ambulance service MBS data usage. MBS data from the HIC was provided by the Department Data analysis of Health and Aged Care (as it was then named) for after hours and in-hours GP services for all States and Territo- Monthly total after hours GP utilisation rates per 10,000 ries. After hours was defined by the HIC as 8 pm-8 am population for both pre-trial and trial periods were com- weeknights, 1 pm–12 pm Saturdays and all day Sunday pared in each AHPMCT area. Changes in each site were and public holidays. also compared with the National comparator (over the same monthly periods) to control for the effects of secular 1 After hours items (1, 2, 97, 98, 601, 602, 697 and 698). trend – see Table 9. Poisson regression analysis was used Their use depends on whether services were provided by as the data consisted of counts of relatively rare events in Vocationally Registered (VR) or non-VR GPs, on whether a cohort of subject over a defined time period. The princi- Page 8 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 Table 9: Impact of AHPMCTs on GP After hours utilisation rates*. PRE TRIAL TRIAL PERIOD Period (Months) Average 95% CI Period (Months) Average 95% CI Statewide Call Centre (Metropolitan) 16 19.3 14.8 – 23.8 20 15.9 12.9 – 18.8 National Comparator 16 30.1 23.6 – 36.6 20 28.3 23.2 – 33.4 Relative Rates 0.87 (95% CI 0.86 – 0.88) Statewide Call Centre (Non-metropolitan) 33 14.2 4.6 – 23.8 3 8.4 4.7 – 12.2 National Comparator 33 29.1 23.0 – 35.3 3 28.8 28.6 – 29.0 Relative Rates 0.60 (95% CI 0.54 – 0.68) Regional Call Centre (Metropolitan) 31 6.7 4.5 – 8.8 5 7.1 5.2 – 8.9 National Comparator 31 29.3 23.4 – 35.5 5 28.0 25.3 – 30.6 Relative Rates 1.11 (95% CI 1.06 – 1.17) Regional Call Centre (Non-metropolitan) 31 11.5 0.4 – 22.6 5 5.1 1.7 – 8.4 National Comparator 31 29.3 23.4 – 35.5 5 28.0 25.3 – 30.6 Relative Rates 0.46 (95% CI 0.35 – 0.61) Deputising Service 24 38.5 27.4 – 49.5 12 23.0 14.8 – 31.3 National Comparator 24 29.9 24.2 – 35.6 12 28.4 23.1 – 33.7 Relative Rates 0.62 (95% CI 0.61 – 0.64) Local Triage service 24 48.7 26.2 – 71.2 12 45.8 31.2 – 60.4 National Comparator 24 29.9 24.2 – 35.6 12 28.4 23.1 – 33.7 Relative Rates 1.01 (95% CI 0.94 – 1.08) * There are minor discrepancies in rates presented in Tables 2 and 3 arising from different time periods being used. pal independent variables (additional to the constant) Competing interests were: time (pre/during trial), intervention status (trial The author(s) declare that they have no competing inter- ests. area/National comparator), and trial effect (time*inter- vention status). This generated a relative rate (and 95% confidence intervals) for the effect of the AHPMCT, Authors' contributions expressed below: DD made a substantial contribution to conception and design, acquisition of data, analysis and interpretation of data and was involved in drafting the manuscript. SED UU / TA.. T TA T Relative rate () RR = made a substantial contribution to acquisition of data, UU / NC.. T NC T analysis and interpretation of data and was involved in revising the manuscript critically for important intellec- Where U is the after hours utilisation rate per 10,000 tual content. RW made a substantial contribution to TA.T1 population per month in the Trial Area (TA) during the acquisition of data, analysis and interpretation of data. trial MK made a substantial contribution to data analysis and was involved in revising the manuscript critically for Where U is the after hours utilisation rate per 10,000 important intellectual content. LG made a substantial TA.T0 population per month in the Trial Area pre-trial contribution to data analysis made a substantial contribu- tion to conception and design and was involved in revis- Where U is the after hours utilisation rate per 10,000 ing the manuscript critically for important intellectual NC.T1 population per month in the National Comparator (NC) content. area during the trial Acknowledgements The study was funded by the Commonwealth Department of Health and Where U is the after hours utilisation rate per 10,000 NC.T0 Ageing (DoHA). It owned intellectual property of the study but granted to population per month in the National Comparator area The University of Melbourne a non-exclusive, royalty-free license to use pre-trial the study for academic purposes such as publishing in peer-refereed jour- nals. DD and LG are funded by The University of Melbourne. SED and RW SPSS for Windows version 12.0.1 was used for the analy- were funded by DoHA funding for the study and more recently by an Aus- sis. tralian Research Council scholarship. MK is funded by an NHMRC Mid Career Development Grant. The work was overseen by an Evaluation and Policy Advisory Group (EAG). Page 9 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2007, 4:21 http://www.anzhealthpolicy.com/content/4/1/21 References 1. Dunt D, Day S, van Dort P: After Hours Primary MedicalCare Trials – National Evaluation Report. [http://www.health.gov.au/ internet/wcms/publishing.nsf/content/health-pcd-programs-ahpmc- publications-ahpmctne.htm]. 2. Hallam L: Primary medical care outside normal working hours : review of published work. BMJ 1994, 308(6923):249-253. 3. Developing NHS Direct: a strategy document for the next three years [http://www.dh.gov.uk/en/Publicationsandstatistics/ Publications/PublicationsPolicyAndGuidance/DH_4005397] 4. Leibowitz R, Day S, Dunt D: A systematic review of the effect of different models of after hours primary medical care serv- ices on clinical outcome, medical workload, and patient and GP satisfaction. Fam Pract 2003, 20:311-317. 5. Munro J, Nicholl J, O'Cathain A, Knowles E: Impact of NHS Direct on demand for immediate care : observational study. BMJ 2000, 321(7254):150-153. 6. Lattimer V, George S, Thompson F, Thomas E, Mullee M, Turnbull J, Smith H, Moore M, Bond H, Glasper A: Safety and effectiveness of nurse telephone consultation in out-of-hours primary care: randomised controlled trial. The South Wiltshire Out of Hours Project (SWOOP) Group. BMJ 1998, 317(7165):1054-1059. 7. Dunt D, Day SE, Kelaher M, Montalto M: Impact of stand-alone and embedded telephone triage systems on after hours pri- mary medical care service utilisation and mix in Australia. Australia and New Zealand Health Policy 2005, 2:30. 12 December 2005 8. Council of Australian Governments' Meeting 10 February 2006 Attachment D: Better health for all Australians Action Plan Promoting good health, prevention and early intervention. [http://www.coag.gov.au/meetings/100206/ attachment_d_better_health.pdf]. 9. Dunt D, Day SE, Kelaher M, Montalto M: The impact of stan- dalone call centres and GP cooperatives on access to after hours GP care: a before and after study adjusted for secular trend. Fam Prac 2006, 23(4):453-460. 10. Commonwealth Department of Health & Aged Care: The relative value study: Stage 3 modelling – a technical report. Canberra, Commonwealth Department of Health & Aged Care; 2001. 11. Department of Primary Industries and Energy and Department of Human Services and Health: Rural, Remote and Metropolitan Areas Classification. 1991 Census Edition. Department of Pri- mary Industries and Energy and Department of Human Services and Health Canberra: Australian Government Publishing Service; 1994. Publish with Bio Med Central and every scientist can read your work free of charge "BioMed Central will be the most significant development for disseminating the results of biomedical researc h in our lifetime." Sir Paul Nurse, Cancer Research UK Your research papers will be: available free of charge to the entire biomedical community peer reviewed and published immediately upon acceptance cited in PubMed and archived on PubMed Central yours — you keep the copyright BioMedcentral Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 10 of 10 (page number not for citation purposes)

Journal

Australia and New Zealand Health PolicySpringer Journals

Published: Oct 10, 2007

References