Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A multidimensional classification of public health activity in Australia

A multidimensional classification of public health activity in Australia Background: At present, we have very limited ability to compare public health activity across jurisdictions and countries, or even to ascertain differences in what is considered to be a public health activity. Existing standardised health classifications do not capture important dimensions of public health, which include its functions, the methods and interventions used to achieve these, the health issues and determinants of health that public health activities address, the resources and infrastructure they use, and the settings in which they occur. A classification that describes these dimensions will promote consistency in collecting and reporting information about public health programs, expenditure, workforce and performance. This paper describes the development of an initial version of such a classification. Methods: We used open-source Protégé software and published procedures to construct an ontology of public health, which forms the basis of the classification. We reviewed existing definitions of public health, descriptions of public health functions and classifications to develop the scope, domain, and multidimensional class structure of the ontology. These were then refined through a series of consultations with public health experts from across Australia, culminating in an initial classification framework. Results: The public health classification consists of six top-level classes: public health 'Functions'; 'Health Issues'; 'Determinants of Health'; 'Settings'; 'Methods' of intervention; and 'Resources and Infrastructure'. Existing classifications (such as the international classifications of diseases, disability and functioning and external causes of injuries) can be used to further classify large parts of the classes 'Health Issues', 'Settings' and 'Resources and Infrastructure', while new subclass structures are proposed for the classes of public health 'Functions', 'Determinants of Health' and 'Interventions'. Conclusion: The public health classification captures the important dimensions of public health activity. It will facilitate the organisation of information so that it can be used to address questions relating to any of these dimensions, either singly or in combination. The authors encourage readers to use the classification, and to suggest improvements. Page 1 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 tions use the alternative term 'population health' to refer Background One of the principal ways in which we make sense of the to the same domain, but this term is also poorly under- world is to group things and events into classes that share stood. common characteristics. Human beings learn to do this intuitively in early childhood, and quickly develop an The lack of widespread understanding about what consti- understanding of classes for commonly encountered tutes public health hampers efforts to advocate for more objects and concepts which is shared by all those around resources for the sector. If we as public health practitioners them. However, for more specialised areas, the descrip- cannot clearly describe the activities of our sector, the tion of things or events in terms of classes tends to be a far resources expended by it, and its outcomes, it will remain less intuitive process that demands a carefully thought- difficult to convince the public, politicians and other deci- out, explicitly articulated framework. Such classification sion-makers that greater investment is needed [4]. frameworks make it much easier to compare information about entities and concepts, and to discern their similari- The lack of basic conceptual development within public ties and differences. health has been recognised internationally. An expert panel convened by the United States (US) Centres for Dis- Within the domain of public health, little conceptual ease Control in 1999 identified the use of common defi- work has been done to develop shared definitions, termi- nitions and comparable data sources as being among the nologies or classifications. As a result, we have limited most important issues for achieving the goal of quality ability to compare public health activity across jurisdic- improvement in public health through performance tions and countries, or even to ascertain whether we share measurement [5]. Five years later, lack of terminological common notions of what constitutes 'public health'. This and conceptual consensus was cited as obstructing even in turn hinders our ability to collect comparable, time- basic work in the area of public health finance in that series data on expenditure, workforce, or performance, country [6]. In Australia, a 2002 project that set out to and to set and monitor benchmarks for these. develop a key set of performance indicators for public health practice recommended, as a priority, the develop- In Australia, the governments of the six states and two ter- ment of a common classification system that could be ritories are the major providers of public health services, used for measuring expenditure as well as for organising while the responsibility for funding these services is performance measurement activities [7]. A 2003 review of shared between the Australian (national) Government the financing of population health (defined as a subset of and state and territory governments [1]. Local government public health with a whole-of-population focus) in eight (municipal and shire councils) also plays a role in deliver- Organisation for Economic Co-operation and Develop- ing public health services, particularly in the areas of envi- ment (OECD) countries, noted that comparability was ronmental health, urban planning, food safety and hampered by differing definitions and categorisations of immunisation; this role varies among the states and terri- activity, lack of reliable data, and lack of uniformity in tories. methods for extracting information [8]. The National Public Health Partnership, a body set up in Several conceptual models describing 'core' or 'essential' 1996 to strengthen collaboration between the Australian functions of public health exist, including the framework Government and state and territory governments, described in the Institute of Medicine's 1988 report on the adopted the following definition for public health: status of public health in the US [9], the '10 essential pub- lic health services' proposed by an expert panel convened the organised response by society to protect and promote by the US Department of Health and Human Services health, and to prevent illness, injury and disability. The [10], the Australian National Public Health Partnership's starting point for identifying public health issues, problems 'core functions for public health practice' [2], a set of core and priorities, and for designing and implementing inter- functions promulgated by the Chief Medical Officer in the ventions, is the population as a whole, or population sub- UK [11], and another developed by the World Health groups[2]. Organization (WHO) in 1996 [12] as well as 'essential public health functions' developed from a three-country However, in Australia – as in other countries [3] – the study in WHO's Western Pacific Region in 2003 [13]. The term 'public health' is a source of confusion, because it is Pan American Health Organization developed 'essential also often used to refer to health services provided by the public health functions' and public health 'roles' in a con- state or otherwise paid for by taxpayers out of "the public ceptual renewal of public health in 2002 [14], and revised purse", as opposed to services provided by the private sec- these in 2007 [15]. A list of 'essential functions' was rec- tor, or paid for by individuals or nongovernmental health ommended by the Canadian National Advisory Commit- insurance or health maintenance funds. Some jurisdic- tee on Population Health in 2003 [16] (See Additional file Page 2 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 1). These conceptual models, in particular the '10 essen- public health programs, expenditure, workforce and per- tial public health services', have proved valuable for deriv- formance. It will facilitate aggregate reporting and analysis ing performance indicators, standards and associated of this information in ways that suit particular perspec- measurement instruments [17-23]. However, all are tives; for example, according to the health problem essentially 'flat' lists, or at best hierarchical taxonomies, addressed, or the setting where public health activity which conflate discrete dimensions such as the purpose of occurred. This paper presents an initial version of such a public health activities, the health issues and problems multidimensional classification, and describes the process addressed and the settings in which services are delivered, that we used to develop it [32]. into single 'functions'. None presents a well-defined theo- retical framework for multiple aspects of effective public Methods health practice [24]. We used an ontology-building process to develop the public health classification. The term "ontology" is used Some of the many standard classifications that are already in several ways, but here we use it in the computer science in use in health fields address aspects of public health. For or knowledge engineering sense of an explicit formal spec- example, WHO's international classifications of diseases ification of the concepts in a domain (in this case, public [25], functioning and disability [26], and external causes health), their attributes and the relationships among of injury [27] can be used to classify morbidity and mor- them, which allows people (and computers) to share a tality data in terms of diseases, disability and injury of common understanding of the structure of information. interest to public health. The recently created OECD sys- We chose this process, rather than a more traditional tem of health accounts [28,29] classifies health care in method for developing a classification, in recognition that three dimensions for the purposes of international com- a flexible, multidimensional classification structure parisons of health care spending: sources of funding; serv- would be needed in order to suit a variety of uses and user ice providers; and functions of care (the goals or purposes groups, and to exploit the near future capabilities of the of health care; e.g. disease prevention, health promotion). Semantic Web. The Semantic Web is an initiative which While these dimensions are clearly separate, the func- aims to give meaning (semantics), in a manner under- tional activity category of 'Prevention and public health standable by machines, to the content of documents on services' [29] consists of a list of only six, non-exclusive, the World Wide Web (and elsewhere) [33]. items (see Additional file 1). We used published methods for frame-based ontology A multidimensional approach was adopted by the Eastern development [34], and Protégé open-source, ontology- Region Public Health Observatory in the United Kingdom building software from Stanford University [35] as a (UK), for the construction of their Public Health Informa- development tool. After scanning available methods and tion Tagging Standard (PHITS). PHITS was developed to software, we chose Protégé because it is freely available to categorise and provide structure to information provided everyone and does not require a commercial license, sup- on websites, and to improve the efficiency of the retrieval ports current Semantic Web standards such as the of web-based public health resources. PHITS has seven Resource Description Framework (RDF) [36] and Web dimensions: 'Person'; 'Time'; 'Place'; 'Determinants'; 'Mor- Ontology Language (OWL) [37], and has an active com- bidity and Mortality'; 'Services'; and 'Policy' [30]. It has munity of interest, with strong representation from now been integrated into a UK National Public Health researchers in the health, biomedical and related sectors. Language thesaurus [31]. Although Protégé provides support for the three types of OWL, we chose to develop our ontology in simpler CLIPS PHITS was designed primarily to categorise web-based format in the first instance, as this is slightly easier to use, information resources, rather than as a multi-purpose with a view to transforming it to OWL format at a later classification for public health. Like other existing classifi- date. cations, it does not capture all the important dimensions of public health, which include its functions, the methods The four steps in the ontology-building process were as and interventions used to achieve these, the health issues follows: and determinants of health which public health activities address, the resources and infrastructure they use, and the 1. Determine the domain and scope of the ontology. settings in which the activities occur. 2. Consider reusing existing ontologies. A multidimensional classification of public health that describes all these dimensions and their relationships, 3. Enumerate important terms in the ontology. and adopts elements from existing classifications where appropriate, will serve multiple purposes. It will have util- 4. Define the classes and class hierarchy [34]. ity for standardising the collection of information about Page 3 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 A reference group of public health experts (see Acknowl- 1. The classification should be multidimensional. edgements) drafted initial responses to steps 1 to 4. Defi- nitions of public health and of core and essential public 2. A range of the most important dimensions need to health functions, including those used by the WHO [13], be considered and developed concurrently. the OECD system of health accounts [29], the Australian National Public Health Partnership [2], the US Depart- 3. Existing classification systems (including interna- ment of Health and Human Services [10], the US Associa- tional and Australian standards) should be used wher- tion of Schools of Public Health [38] and the Pan ever possible. American Health Organisation [14], the UK Department of Health [11] and the Canadian National Advisory Com- 4. The classification should be inclusive and deliber- mittee on SARS and Public Health [16] (see Additional ately broad at the top levels. Specific boundaries and file 1) were reviewed for step 2. restrictions to the scope of the classification should be defined in practical applications, rather than be arbi- We refined the classification through a series of consulta- trarily imposed during the development of the classi- tions with public health experts and practitioners across fication. Australia. The project consultation process sought to achieve agreement on a version of the classification that The public health classification was 'good enough', recognising that the classification of Version one of the public health classification consists of public health is a complex and technically difficult prob- six top-level classes: (public health) 'Functions'; 'Health lem, with no definitive formulation or solution. Early Issues'; 'Determinants of Health'; 'Settings'; 'Methods' (of consultations were informal, and designed to seek the intervention); and 'Resources and Infrastructure', which views of content experts in particular domains (e.g. envi- are shown as circles in Figure 1, together with a hierarchy ronmental health, health promotion). Later, more formal of subclasses, and, at the lowest level, instances. Each sub- consultations were organised through reference group class and instance should have a subsumption ("is-a") members representing various jurisdictions. relationship with its parent class. Prior to each consultation meeting, we sent material intro- Existing classifications (such as the international classifi- ducing the classification project to participants. All con- cations of diseases [25], functioning and disability [26], sultations were conducted face to face. The number of and external causes of injuries [27], can be used as sub- participants varied from one or two, to larger groups of up classes of the classes 'Health Issues', 'Settings' and to 15, and the duration varied from one to three hours. In 'Resources and Infrastructure', while new subclass struc- each consultation, an introduction and background to the tures are proposed for the classes of public health 'Func- classification project were given with the aid of a slide tions', 'Determinants of Health' and 'Methods' (see Figure presentation, after which an early version of the public 1). health classification, rendered through a Web browser, was demonstrated. This was followed by a 'live' session The working definitions for the six top-level classes are using Protégé, which allowed participants to explore the shown in Table 1, and their immediate subclasses are structure of the classification, to suggest additions and given in Table 2. changes, and to immediately see their effect on the overall classification. The 'Functions' class is currently the most highly devel- oped. Both primary and instrumental functions were con- Last, participants were asked to identify practical uses for sidered to be important in conceptualising public health a multidimensional public health classification. Follow- (See Additional file 2). Primary functions are ends in ing consultation meetings, the project reference group themselves, while instrumental functions are means to debated proposed changes to the classification before those ends. Public health practitioners also described deciding to adopt or reject them. The Australian National instrumental functions as 'supporting', 'underpinning' or Public Health Information Working Group – a committee 'cross-cutting' functions because all primary functions rely with representation from all states and territories as well on them – they do not belong solely to any one of the pri- as relevant national bodies – discussed and provided feed- mary functions. back on an early draft version of the classification. While there was reasonable agreement among public health experts regarding the subclasses for the top-level Results Principles of development classes 'Health Issues', 'Determinants of Health', and 'Set- Development of the public health classification was tings', the remaining classes are in earlier stages of devel- guided by the following principles: opment. Page 4 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 nym-hyponym) relationships among the current subclasses of this top-level class. A copy of the full report on this initial phase of the project has been included as an additional file to this paper (Additional file 3), together with copies of the underlying classification ontology as it stood at the time of writing. Two sets of ontology files are provided: a set of Protégé project files (Additional file 4), and a set of interlinked HTML files that can be explored using a Web browser (Additional file 5). Potential uses for the classification The classification is used by assigning zero or more attributes chosen from each of the top-level class hierar- chies to the "thing" being classified. A wide range of "things" can potentially be organised according to the classification, including (but not restricted to) public health policies, programs and interventions, the popula- tion groups they target, and their outcomes. Figure 1 depicts some examples of things that could be classified, shown as heptagons. A Figure 1 model of public health classification Many participants in the development process expressed a A model of public health classification. Source: Adapted view that the classification would assist them in describ- from Gruszin S, Jorm L, Churches T, Straton J: Public Health ing what public health is, and what its characteristics are. Classifications Project Phase One: Final Report: Report to the They also identified a range of questions that, currently, National Public Health Partnership Group. Melbourne: National are difficult or impossible to answer, but which poten- Public Health Partnership; 2005. tially could be answered if the multidimensional classifi- cation were used to facilitate aggregated reporting on public health activity. Examples of such questions are The 'Methods' class was the subject of some disagreement, given in Figure 2. with some experts preferring to narrow its scope to those methods that are peculiar to – or only used by – public Participants also suggested a range of potential practical health (e.g. population-based epidemiology, health pro- applications for the classification. Examples of these are motion, environmental risk assessment). Others favoured given in Figure 3. A knowledge base to support communi- an inclusive approach that would capture all methods used cable disease surveillance, also constructed using the Pro- by public health, including those that, while not specific tégé software suite, has been described [39]; it should be to it, are employed by public health workers in the normal possible to use the classification ontology to create analo- course of their work (e.g. administration, management, gous databases for other specific or general areas of public policy development). An inclusive approach was adopted, health practice. in line with development principle (4) above. Discussion Public health experts expressed diametrically opposed The process of developing the classification brought to views as to whether 'infrastructure' represented aggregates light several areas of basic disagreement among Australian of 'resources', or whether 'resources' were in fact a subclass public health practitioners regarding the nature of public of 'infrastructure'. An inclusive approach was adopted, in health practice. line with development principle (4) above, with the rele- vant class termed 'Resources and Infrastructure', and its The inclusion, or otherwise, of preventive services deliv- subclasses capturing both compound (e.g. administrative ered on a one-to-one basis to individuals was particularly infrastructure, information systems) and unitary elements contentious. Such preventive services include screening (e.g. funds, workforce). and detection, immunisation, and counselling and life- style advice to support healthy behaviour, as well as man- Further work is needed to disentangle the mixture of par- agement (through lifestyle changes or pharmacological titive (holonym-meronym) and subsumption (hyper- means) of disease risk factors such as high blood pressure and high cholesterol. Page 5 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 Table 1: Classification of public health: six top-level classes and their working definitions Class Working definition Functions Public health functions. The purpose of public health interventions, actions, activities and programs. Health Issues Health, and well-being issues that affect health ('issues' includes: concerns, topics, problems). Health is defined (by the WHO) as 'a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity'. Determinants of Health Factors that influence health status and determine health differentials or health inequalities. They include, for example, natural, biological factors, such as age, sex and ethnicity; behaviour and lifestyles, such as smoking, alcohol consumption, diet and physical activity; physical and social factors, including employment and education, housing quality, the workplace and the wider urban and rural environment; and access to health care [a]. Methods The methods used by organised public health interventions (actions, activities, programs, services) to protect and promote health and prevent illness, injury and disability, that are designed to change population exposure, behavioural or health status. Settings Settings in which public health activities and interventions take place, institutional and social environments, partnerships, and locations (e.g. schools, local government, hospitals, workplaces). Resources and Infrastructure Resources and infrastructure, the means available for the operation of health systems, including human resources, facilities, equipment and supplies, financial funds and knowledge [b]. It includes both person-time and calendar time. Source: Gruszin S, Jorm L, Churches T, Straton J: Public Health Classifications Project Phase One: Final Report: Report to the National Public Health Partnership Group. Melbourne: National Public Health Partnership; 2005. [a] World Health Organisation: Health Impact Assessment (HIA): Glossary of terms used. Geneva; 2006. http://www.who.int/hia/about/glos/en/ index.html [b] World Health Organization: Health Promotion Glossary. Geneva; 1998. [WHO/HPR/HEP/98.1] Many participants argued that those individual preventive ological condition (such as high blood pressure) or dis- services related to communicable disease (immunisation, ease marked the boundary of public health practice. contact tracing and treatment for sexually transmitted Others regarded this boundary as spurious, because the infections) are public health activities because they help pathophysiological processes which underlie the develop- to protect the health of the whole population, through ment of chronic disease are continuous, and because herd immunity and controlling the spread of infection. interventions such as anti-hypertensive and cholesterol- Others felt that immunisation is only a legitimate part of lowering drugs, or even the 'Polypill' [40], may have dra- public health activity when it is delivered as part of an matic benefits in terms of morbidity and mortality at the organised government-funded program, such as through population level. local government or school health services. By adopting an inclusive approach, the public health clas- Most public health practitioners agreed that early detec- sification allows decisions about boundaries, inclusions tion of disease through screening is a public health activity and exclusions, to be made at the level of individual appli- when it is delivered through an organised government- cations of the classification. This is especially useful for funded program (such as national breast and cervical can- those boundary issues – such as where public health ends cer screening programs). Less clear was whether screening and clinical practice begins – about which opinion may that is not part of an organised program, such as oppor- evolve with knowledge about preventive interventions, tunistic bone density screening for osteoporosis, should and how and by whom they are best delivered. be seen as part of public health. The issue of whether a public health classification should There was substantial disagreement among public health be restricted to a domain solely within the health sector or practitioners regarding the inclusion, or otherwise, of whether it should it include the health-related activities of activities relating to prevention and management of non- other sectors was also frequently raised. Most public communicable disease through individual counselling or health practitioners agreed, when pressed, that accounting other interventions directed at lifestyle risk factors (smok- for public health should include the activities of, and ing, poor nutrition, risky alcohol use and lack of physical investments by, the non-health portfolios (such as educa- activity), and the early detection and management of bio- tion and transport) of national and state governments, as logical risk factors such as high blood pressure and high well as the relevant activities of local governments and cholesterol. Many contended that the diagnosis of a path- non-government organisations (NGOs). This is consistent Page 6 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 Table 2: Classification of public health: top two levels of all classes Top-level class Level 2 subclasses Functions Assess health of populations Primary: Promote health and prevent disease, disability and injury Protect from threats to health Instrumental Ensure public health capability Build the evidence base for public health Health issues Health and well-being Injury Diseases and conditions Disability and functioning Determinants of Health Environmental Socioeconomic External causes of injury Person-level Health system Methods Advocacy and lobbying Health impact assessment Research and evaluation Communicable disease control specific Immunisation Road safety methods Community action Infection control Screening to detect disease/risk factors Community development Legislation and regulation Social action Counselling Lifestyle advice Social marketing Diagnosis Management of biological risk Training and workforce development methods Directed investment Monitoring and surveillance Treatment methods Environmental monitoring Personal skills development Urban planning methods Epidemiologic methods Political action Vector control methods Exercise of capabilities Public policy development Waste management methods Food safety methods Radiation safety methods Other methods of intervention Health education Remediation of environment methods Settings Educational settings Home settings Other settings Healthcare settings Workplace settings Includes LOCATIONS – classification of geographical areas (e.g. postcodes). Local government and communities Transport settings settings Resources and infrastructure Administrative infrastructure Organisational systems Technical infrastructure Funds Partnerships Time Information systems Physical infrastructure Workforce Legislative infrastructure Policies Workforce development capacity Source: Gruszin S, Jorm L, Churches T, Straton J: Public Health Classifications Project Phase One: Final Report: Report to the National Public Health Partnership Group. Melbourne: National Public Health Partnership; 2005. Page 7 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 with the definition of public health used by the US Insti- those of Canada [16] and the Americas [14,15]) are lim- tute of Medicine as: "what we, as a society, do collectively ited to primary functions, while others include both pri- to assure the conditions for people to be healthy" [9]. mary and instrumental functions. In the UK [11], both primary (e.g. health promotion and disease prevention However, in practice there are major difficulties in captur- programs) and instrumental (e.g. development and main- ing information on public health-related activities and tenance of a public health workforce) functions are prom- expenditure by non-health sectors. In Australia, current inent. Our classification captures the majority of public health expenditure reporting is limited to the functions that other nations have described, including the health portfolio expenditures of the state and national (instrumental) 'partnership' and 'research' functions that governments [1]. One view was that the activities of non- are present in both the UK core functions [11] and the 10 health sectors should only be considered 'in scope' when essential public health services of the USA [10]. public health is their primary purpose (e.g. immunisation organised by local government). The 'boundary' issues we A 'quality assurance' function does not currently feature in encountered reflect the way that public health activity is our classification, although it is specified in the published conceptualised, and organised, in Australia. Similar exer- functions for public health in the USA [10], the Americas cises conducted in other countries would doubtless high- [14,15], the UK [11] and in WHO's most recent work light different issues. For example, public health services [13,15]. in the US (which has a strongly privatised approach to health care) are seen as having a major role in filling 'gaps' Public health practitioners and experts in Australia at no in health service provision (such as maternal and child stage suggested that such a function was a critical part of health services) for those without access to health insur- their work. It is possible that their views may subsequently ance, as well as in evaluating the accessibility and quality have changed, particularly in light of several recent scan- of personal health care services [10]. In several European dals [41-43] relating to the safety and quality of the care countries, provision of many public health services is fully provided in Australian public hospitals. devolved to the level of local municipalities, which also have responsibility for issues such as air and water quality, The approach we adopted in developing the classification noise diminution and waste management. It is likely that should maximise its flexibility for application in other set- the organisation of services in these countries influences tings. The ontology-building process offered particular the way their public health practitioners conceptualise the advantages in dealing with divergent (and often strongly boundaries between primary health care, public health held) views regarding what was and was not 'in scope'. and environmental protection. Although defining and specifying classes was central to the process, the emphasis was on modelling the relation- A comparison of the published public health functions of ships among classes, rather than on the within-class hier- other nations (see Additional file 1) shows that some (e.g. archies. We were able to adopt an inclusive approach, leaving scope for decisions about rules and exclusions to ƒ How much was spent last year on the prevention of obesity? ƒ How is public health relevant to components of the human ƒ Explain what public health is services delivery system? ƒ Organise information to answer key public health ƒ Why do public health unit costs differ across jurisdictions? questions ƒ Can we describe screening in clinical settings (e.g. Pap smears taken in General Practice surgeries)? ƒ Promote consistency in describing public health ƒ How much did we spend on prevention of HIV/AIDS relative to ƒ Improve data capture processes and the quality of other preventable diseases? reporting ƒ What are the nature and cost of partnerships between public ƒ Contribute to higher-level classification and standards health and other sectors? activities ƒ How much was spent on social marketing last year? ƒ Structure and design information and communications ƒ How many staff work in environmental health, and in what settings do they work? ƒ Audit the spread of activity across the public health business cycle ƒ How much did we spend and what was the activity on specific risk factors? ƒ Build models of good public health practice ƒ Link research, policy and practice A li Figure 2 ke. public h .. ealth classification should help answer questions A public health classification should help answer Potential Figure 3uses for a public health classification questions like.... Source: Adapted from Gruszin S, Jorm L, Potential uses for a public health classification. Source: Churches T, Straton J: Public Health Classifications Project Phase Adapted from Gruszin S, Jorm L, Churches T, Straton J: Pub- One: Final Report: Report to the National Public Health Partner- lic Health Classifications Project Phase One: Final Report: Report ship Group. Melbourne: National Public Health Partnership; to the National Public Health Partnership Group. Melbourne: National Public Health Partnership; 2005. Page 8 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 be made at the level of specific practical applications of Additional file 2 the classification. It is to be hoped that such practical Classification of public health: detail of the functions class. Table pre- applications will make the areas of contention explicit, senting the "Functions" class of the public health classification, and its encourage debate, and offer a way to move towards a com- subclasses. mon language to describe public health activity in Aus- Click here for file tralia and elsewhere. [http://www.biomedcentral.com/content/supplementary/1743- 8462-6-9-S2.pdf] Conclusion Additional file 3 The public health classification is an initial attempt to Gruszin S, Jorm L, Churches T, Straton J: Public Health Classifica- describe the important dimensions of public health activ- tions Project Phase One: Final Report: Report to the National ity. It will facilitate the organisation of information about Public Health Partnership Group. Melbourne: National Public public health activity so that it can be used to address Health Partnership; 2005. Full report of Phase One of the Australian questions relating to any of these dimensions, either sin- Public Health Classifications Project. gly or in combination. The authors encourage readers to Click here for file use the classification, and to suggest further refinements. [http://www.biomedcentral.com/content/supplementary/1743- It is our intention to further refine and extend several of 8462-6-9-S3.pdf] the class hierarchies, and to convert the ontology into RDF Additional file 4 and OWL format to make it suitable for use in Semantic Public Health ontology (PHont) Version 1.0. Protégé version 3.1 Web applications. We welcome potential collaborators in project files for Public Health Ontology (PHont) Version 1.0. this endeavour. Click here for file [http://www.biomedcentral.com/content/supplementary/1743- Abbreviations 8462-6-9-S4.zip] HIV/AIDS: Human Immunodeficiency Virus/Acquired ImmunoDeficiency Syndrome; OECD: Organisation for Additional file 5 Public Health ontology (PHont) Version 1.0. HTML web page version Economic Co-operation and Development; UK: United of Public Health Ontology (PHont) Version 1.0 Kingdom; US: United States of America; WHO: World Click here for file Health Organization; RDF: Resource Description Frame- [http://www.biomedcentral.com/content/supplementary/1743- work; OWL: Web Ontology Language (sic). 8462-6-9-S5.zip] Competing interests The authors declare that they have no competing interests. Acknowledgements Authors' contributions The following people contributed to the development of the classification through participating in the Reference Group for the Public Health Classi- LJ compiled the argument and drafted the background, fications Project: Louisa Jorm, Chair (New South Wales Department of discussion and conclusions of the manuscript. SG Health); Michael Ackland (Department of Human Services, Victoria); And- reported the results, contributed to the background and rea Casasola (Queensland Health); Jenny Cleary, Pam Gollow (Department discussion, and prepared the figures and tables. TC of Health and Community Services, Northern Territory); Charles Guest informed the directions of the project, provided key con- (ACT Health, Australian Capital Territory); Paul Jelfs (South Australian ceptual and practical input to its results, and reviewed and Department of Health); Paul Magnus, Catherine Sykes, Robert Van der edited the manuscript. All authors worked on the project, Hoek (Australian Institute of Health and Welfare); Dean Martin (Australian and have co-authored, with Judy Straton (see Acknowl- Bureau of Statistics); Karen Roger (National Public Health Partnership Sec- edgements), the report of phase one of the project. All retariat); authors read and approved the final manuscript. Colin Sindall (Australian Government Department of Health and Ageing); Judy Straton (Department of Health Western Australia); Tony Woollacott Additional material (South Australian Department of Health). Professor Richard Madden (National Centre for Classification in Health, University of Sydney) made important conceptual contributions to the classification. Additional file 1 Comparison of published public health functions and roles. Table pre- The (Australian) National Public Health Partnership funded the Public senting public health roles and functions published in international Health Classifications Project, some results of which are reported in this reports. article. Additional sources of funding for the authors were the New South Click here for file Wales Department of Health (L Jorm and T Churches) and the Public [http://www.biomedcentral.com/content/supplementary/1743- Health Information Development Unit, The University of Adelaide (S 8462-6-9-S1.pdf] Page 9 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 Gruszin). The funding body was represented on the Reference Group, and 24. Roper WL, Mays GP: Performance measurement in public health: Conceptual and methodological issues in building the as such had a role in the project design, operation, interpretation and science base. J Public Health Manag Pract 2000, 6:66-77. reporting of results. 25. World Health Organization: International Statistical Classification of Dis- eases and Related Health Problems (ICD-10). 10th revision. Geneva 1992, 1–3:. References 26. World Health Organization: International Classification of Functioning, 1. Australian Institute of Health and Welfare: National Public Health Disability and Health (ICF). Geneva 2001. Expenditure Report 2000–01. Canberra; [Health and Welfare Expenditure 27. ICECI Coordination and Maintenance Group: International Classifica- Series, HWE 25] 2004. tion of External Causes of Injuries (ICECI). Version 1.2 Amsterdam, and 2. National Public Health Partnership: Public Health Practice in Australia Adelaide: Consumer Safety Institute, and Australian Institute of Today – A Statement of Core Functions. Melbourne 2000. Health and Welfare National Injury Surveillance Unit; 2004. 3. McCarthy M: Integrating public health practice in Europe. 28. Organisation for Economic Co-operation and Development: A System Eurohealth 2003, 9:19-23. of National Health Accounts. Geneva 2000. 4. Barry M, Bialek R: Tracking our investments in public health: 29. Organisation for Economic Co-operation and Development: Interna- What have we learned? J Public Health Manag Pract 2004, tional Classification of Health Accounts. Geneva 2000. 10:383-92. 30. Public Health Information Tagging Standard [http:// 5. Mays GP, Halverson PK: Conceptual and methodological issues www.phl.nhs.uk/] in public health performance measurement: Results from a 31. National Public Health Language [http://www.phl.nhs.uk/docs/ computer-assisted expert panel process. J Public Health Manag version_2_0_files/PHL_Version_2_0_alpha.pdf] Pract 2000, 6:59-65. 32. Gruszin S, Jorm L, Churches T, Straton J: Public Health Classifications 6. Moulton AD, Halverson PK, Honoré PA, Berkowitz B: Public health Project Phase One: Final Report: Report to the National Public Health Part- finance: A conceptual framework. J Public Health Manag Pract nership Group Melbourne: National Public Health Partnership; 2005. 2004, 10:377-382. 33. Berners-Lee T, Hendler J, Lassila O: The Semantic Web. Sci Am 7. National Public Health Partnership: Public Health Performance Project- 2001, 284:34-43. Report to the National Public Health Partnership Group. Melbourne 2002. 34. Noy NF, McGuinness DL: Ontology Development 101: A Guide to Creat- 8. Bennett J: Investment in Population Health in Five OECD Countries. [OECD ing your First Ontology 2001 [http://protege.stanford.edu/publications/ Health Working Papers No. 2.] Paris: OECD; 2003. ontology_development/ontology101-noy-mcguinness.html]. Stanford, 9. Institute of Medicine: The Future of Public Health Washington, DC: CA: Knowledge Systems Laboratory, Stanford University National Academy Press; 1988. 35. Protégé [http://protege.stanford.edu/] 10. Public Health Functions Steering Committee, US Department of 36. Resource description Framework (RDF) [http://www.w3.org/ Health and Human Services: Public Health in America Washington, DC: RDF/] US Department of Health and Human Services; 1994. 37. Web Ontology Language (OWL) [http://www.w3.org/2004/ 11. Chief Medical Officer, United Kingdom: Public Health in England Lon- OWL/] don: UK Dept of Health; 2003. 38. Association of Schools of Public Health: What is Public Health? Wash- 12. Bettcher DW, Sapirie S, Goon EHT: Essential public health func- ington, DC 2004. tions: Results of the international Delphi study. World Health 39. Doyle TJ, Ma H, Groseclose SL, Hopkins RS: PHSkb: A knowledge- Stat Q 1998, 51:44-54. base to support notifiable disease surveillance. BMC Med 13. World Health Organization, Regional Office for the Western Pacific: Inform Decis Mak 2005, 5:27 [http://www.biomedcentral.com/1472- Essential Public Health Functions: A Three-Country Study in the Western 6947/5/27]. Pacific Region. Manila, Philippines 2003. 40. Wald NJ, Law MR: A strategy to reduce cardiovascular disease 14. Pan American Health Organization: Public Health in the Americas: Con- by more than 80%. BMJ 2003, 326:1419-1424. ceptual Renewal, Performance Assessment, and Bases for Action. Washing- 41. Weyden MB Van Der: The Bundaberg Hospital scandal: The ton, DC 2002. need for reform in Queensland and beyond. Med J Aust 2005, 15. Ramagem C, Ruales J: The essential public health functions as a strategy 183:284-285. for improving overall health systems performance: trends and challenges 42. Joseph AP, Hunyor SN: The Royal North Shore Hospital since the Public Health in the Americas Initiative, 2000–2007 Washing- inquiry: An analysis of the recommendations and the impli- ton, DC: Pan American Health Organization/World Health Organiza- cations for quality and safety in Australian public hospitals. tion; 2008. Med J Aust 2008, 188:469-472. 16. National Advisory Committee on SARS and Public Health: Learning 43. Weyden MB Van Der: Good and safe doctors. Med J Aust 2008, from SARS: Renewal of Public Health in Canada Ottawa: Health Canada; 188:497. 17. Suen J, Magruder C: National profile: Overview of capabilities and core functions of local public health jurisdictions in 47 states, the District of Columbia, and 3 US territories, 2000– 2002. J Public Health Manag Pract 2004, 10:2-12. 18. Centers for Disease Control and Prevention: National Public Health Performance Standards Program – State Public Health System Performance Assessment Instrument. Atlanta 2001. 19. Centers for Disease Control and Prevention: National public health Performance Standards Program – Local Public Health System Performance Assessment Instrument. Atlanta 2001. Publish with Bio Med Central and every 20. Mays GP, McHugh MC, Shim K, Lenaway D, Halverson PK, Moones- scientist can read your work free of charge inghe R, Honoré P: Getting what you pay for: Public health "BioMed Central will be the most significant development for spending and the performance of essential public health services. J Public Health Manag Pract 2004, 10:435-443. disseminating the results of biomedical researc h in our lifetime." 21. Owen T, Jorm L: The national public health performance Sir Paul Nurse, Cancer Research UK project: How do we know whether Australian public health ser vices are performing? NSW Public Health Bulletin 2005, 16:2-8. Your research papers will be: 22. Lower T, Durham G, Bow D, Larson A: Implementation of the available free of charge to the entire biomedical community Australian core public health functions in rural Western Australia. Aust N Z J Public Health 2004, 28:418-25. peer reviewed and published immediately upon acceptance 23. Beitsch LM, Brooks RG, Grigg M, Menachemi N: Structure and cited in PubMed and archived on PubMed Central Functions of State Public Health Agencies. Am J Public Health yours — you keep the copyright 2006, 96:167-72. BioMedcentral Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 10 of 10 (page number not for citation purposes) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Australia and New Zealand Health Policy Springer Journals

A multidimensional classification of public health activity in Australia

Loading next page...
 
/lp/springer-journals/a-multidimensional-classification-of-public-health-activity-in-paOlYi4nAK
Publisher
Springer Journals
Copyright
Copyright © 2009 by Jorm et al; licensee BioMed Central Ltd.
Subject
Medicine & Public Health; Public Health; Social Policy
ISSN
1743-8462
eISSN
1743-8462
DOI
10.1186/1743-8462-6-9
pmid
19358715
Publisher site
See Article on Publisher Site

Abstract

Background: At present, we have very limited ability to compare public health activity across jurisdictions and countries, or even to ascertain differences in what is considered to be a public health activity. Existing standardised health classifications do not capture important dimensions of public health, which include its functions, the methods and interventions used to achieve these, the health issues and determinants of health that public health activities address, the resources and infrastructure they use, and the settings in which they occur. A classification that describes these dimensions will promote consistency in collecting and reporting information about public health programs, expenditure, workforce and performance. This paper describes the development of an initial version of such a classification. Methods: We used open-source Protégé software and published procedures to construct an ontology of public health, which forms the basis of the classification. We reviewed existing definitions of public health, descriptions of public health functions and classifications to develop the scope, domain, and multidimensional class structure of the ontology. These were then refined through a series of consultations with public health experts from across Australia, culminating in an initial classification framework. Results: The public health classification consists of six top-level classes: public health 'Functions'; 'Health Issues'; 'Determinants of Health'; 'Settings'; 'Methods' of intervention; and 'Resources and Infrastructure'. Existing classifications (such as the international classifications of diseases, disability and functioning and external causes of injuries) can be used to further classify large parts of the classes 'Health Issues', 'Settings' and 'Resources and Infrastructure', while new subclass structures are proposed for the classes of public health 'Functions', 'Determinants of Health' and 'Interventions'. Conclusion: The public health classification captures the important dimensions of public health activity. It will facilitate the organisation of information so that it can be used to address questions relating to any of these dimensions, either singly or in combination. The authors encourage readers to use the classification, and to suggest improvements. Page 1 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 tions use the alternative term 'population health' to refer Background One of the principal ways in which we make sense of the to the same domain, but this term is also poorly under- world is to group things and events into classes that share stood. common characteristics. Human beings learn to do this intuitively in early childhood, and quickly develop an The lack of widespread understanding about what consti- understanding of classes for commonly encountered tutes public health hampers efforts to advocate for more objects and concepts which is shared by all those around resources for the sector. If we as public health practitioners them. However, for more specialised areas, the descrip- cannot clearly describe the activities of our sector, the tion of things or events in terms of classes tends to be a far resources expended by it, and its outcomes, it will remain less intuitive process that demands a carefully thought- difficult to convince the public, politicians and other deci- out, explicitly articulated framework. Such classification sion-makers that greater investment is needed [4]. frameworks make it much easier to compare information about entities and concepts, and to discern their similari- The lack of basic conceptual development within public ties and differences. health has been recognised internationally. An expert panel convened by the United States (US) Centres for Dis- Within the domain of public health, little conceptual ease Control in 1999 identified the use of common defi- work has been done to develop shared definitions, termi- nitions and comparable data sources as being among the nologies or classifications. As a result, we have limited most important issues for achieving the goal of quality ability to compare public health activity across jurisdic- improvement in public health through performance tions and countries, or even to ascertain whether we share measurement [5]. Five years later, lack of terminological common notions of what constitutes 'public health'. This and conceptual consensus was cited as obstructing even in turn hinders our ability to collect comparable, time- basic work in the area of public health finance in that series data on expenditure, workforce, or performance, country [6]. In Australia, a 2002 project that set out to and to set and monitor benchmarks for these. develop a key set of performance indicators for public health practice recommended, as a priority, the develop- In Australia, the governments of the six states and two ter- ment of a common classification system that could be ritories are the major providers of public health services, used for measuring expenditure as well as for organising while the responsibility for funding these services is performance measurement activities [7]. A 2003 review of shared between the Australian (national) Government the financing of population health (defined as a subset of and state and territory governments [1]. Local government public health with a whole-of-population focus) in eight (municipal and shire councils) also plays a role in deliver- Organisation for Economic Co-operation and Develop- ing public health services, particularly in the areas of envi- ment (OECD) countries, noted that comparability was ronmental health, urban planning, food safety and hampered by differing definitions and categorisations of immunisation; this role varies among the states and terri- activity, lack of reliable data, and lack of uniformity in tories. methods for extracting information [8]. The National Public Health Partnership, a body set up in Several conceptual models describing 'core' or 'essential' 1996 to strengthen collaboration between the Australian functions of public health exist, including the framework Government and state and territory governments, described in the Institute of Medicine's 1988 report on the adopted the following definition for public health: status of public health in the US [9], the '10 essential pub- lic health services' proposed by an expert panel convened the organised response by society to protect and promote by the US Department of Health and Human Services health, and to prevent illness, injury and disability. The [10], the Australian National Public Health Partnership's starting point for identifying public health issues, problems 'core functions for public health practice' [2], a set of core and priorities, and for designing and implementing inter- functions promulgated by the Chief Medical Officer in the ventions, is the population as a whole, or population sub- UK [11], and another developed by the World Health groups[2]. Organization (WHO) in 1996 [12] as well as 'essential public health functions' developed from a three-country However, in Australia – as in other countries [3] – the study in WHO's Western Pacific Region in 2003 [13]. The term 'public health' is a source of confusion, because it is Pan American Health Organization developed 'essential also often used to refer to health services provided by the public health functions' and public health 'roles' in a con- state or otherwise paid for by taxpayers out of "the public ceptual renewal of public health in 2002 [14], and revised purse", as opposed to services provided by the private sec- these in 2007 [15]. A list of 'essential functions' was rec- tor, or paid for by individuals or nongovernmental health ommended by the Canadian National Advisory Commit- insurance or health maintenance funds. Some jurisdic- tee on Population Health in 2003 [16] (See Additional file Page 2 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 1). These conceptual models, in particular the '10 essen- public health programs, expenditure, workforce and per- tial public health services', have proved valuable for deriv- formance. It will facilitate aggregate reporting and analysis ing performance indicators, standards and associated of this information in ways that suit particular perspec- measurement instruments [17-23]. However, all are tives; for example, according to the health problem essentially 'flat' lists, or at best hierarchical taxonomies, addressed, or the setting where public health activity which conflate discrete dimensions such as the purpose of occurred. This paper presents an initial version of such a public health activities, the health issues and problems multidimensional classification, and describes the process addressed and the settings in which services are delivered, that we used to develop it [32]. into single 'functions'. None presents a well-defined theo- retical framework for multiple aspects of effective public Methods health practice [24]. We used an ontology-building process to develop the public health classification. The term "ontology" is used Some of the many standard classifications that are already in several ways, but here we use it in the computer science in use in health fields address aspects of public health. For or knowledge engineering sense of an explicit formal spec- example, WHO's international classifications of diseases ification of the concepts in a domain (in this case, public [25], functioning and disability [26], and external causes health), their attributes and the relationships among of injury [27] can be used to classify morbidity and mor- them, which allows people (and computers) to share a tality data in terms of diseases, disability and injury of common understanding of the structure of information. interest to public health. The recently created OECD sys- We chose this process, rather than a more traditional tem of health accounts [28,29] classifies health care in method for developing a classification, in recognition that three dimensions for the purposes of international com- a flexible, multidimensional classification structure parisons of health care spending: sources of funding; serv- would be needed in order to suit a variety of uses and user ice providers; and functions of care (the goals or purposes groups, and to exploit the near future capabilities of the of health care; e.g. disease prevention, health promotion). Semantic Web. The Semantic Web is an initiative which While these dimensions are clearly separate, the func- aims to give meaning (semantics), in a manner under- tional activity category of 'Prevention and public health standable by machines, to the content of documents on services' [29] consists of a list of only six, non-exclusive, the World Wide Web (and elsewhere) [33]. items (see Additional file 1). We used published methods for frame-based ontology A multidimensional approach was adopted by the Eastern development [34], and Protégé open-source, ontology- Region Public Health Observatory in the United Kingdom building software from Stanford University [35] as a (UK), for the construction of their Public Health Informa- development tool. After scanning available methods and tion Tagging Standard (PHITS). PHITS was developed to software, we chose Protégé because it is freely available to categorise and provide structure to information provided everyone and does not require a commercial license, sup- on websites, and to improve the efficiency of the retrieval ports current Semantic Web standards such as the of web-based public health resources. PHITS has seven Resource Description Framework (RDF) [36] and Web dimensions: 'Person'; 'Time'; 'Place'; 'Determinants'; 'Mor- Ontology Language (OWL) [37], and has an active com- bidity and Mortality'; 'Services'; and 'Policy' [30]. It has munity of interest, with strong representation from now been integrated into a UK National Public Health researchers in the health, biomedical and related sectors. Language thesaurus [31]. Although Protégé provides support for the three types of OWL, we chose to develop our ontology in simpler CLIPS PHITS was designed primarily to categorise web-based format in the first instance, as this is slightly easier to use, information resources, rather than as a multi-purpose with a view to transforming it to OWL format at a later classification for public health. Like other existing classifi- date. cations, it does not capture all the important dimensions of public health, which include its functions, the methods The four steps in the ontology-building process were as and interventions used to achieve these, the health issues follows: and determinants of health which public health activities address, the resources and infrastructure they use, and the 1. Determine the domain and scope of the ontology. settings in which the activities occur. 2. Consider reusing existing ontologies. A multidimensional classification of public health that describes all these dimensions and their relationships, 3. Enumerate important terms in the ontology. and adopts elements from existing classifications where appropriate, will serve multiple purposes. It will have util- 4. Define the classes and class hierarchy [34]. ity for standardising the collection of information about Page 3 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 A reference group of public health experts (see Acknowl- 1. The classification should be multidimensional. edgements) drafted initial responses to steps 1 to 4. Defi- nitions of public health and of core and essential public 2. A range of the most important dimensions need to health functions, including those used by the WHO [13], be considered and developed concurrently. the OECD system of health accounts [29], the Australian National Public Health Partnership [2], the US Depart- 3. Existing classification systems (including interna- ment of Health and Human Services [10], the US Associa- tional and Australian standards) should be used wher- tion of Schools of Public Health [38] and the Pan ever possible. American Health Organisation [14], the UK Department of Health [11] and the Canadian National Advisory Com- 4. The classification should be inclusive and deliber- mittee on SARS and Public Health [16] (see Additional ately broad at the top levels. Specific boundaries and file 1) were reviewed for step 2. restrictions to the scope of the classification should be defined in practical applications, rather than be arbi- We refined the classification through a series of consulta- trarily imposed during the development of the classi- tions with public health experts and practitioners across fication. Australia. The project consultation process sought to achieve agreement on a version of the classification that The public health classification was 'good enough', recognising that the classification of Version one of the public health classification consists of public health is a complex and technically difficult prob- six top-level classes: (public health) 'Functions'; 'Health lem, with no definitive formulation or solution. Early Issues'; 'Determinants of Health'; 'Settings'; 'Methods' (of consultations were informal, and designed to seek the intervention); and 'Resources and Infrastructure', which views of content experts in particular domains (e.g. envi- are shown as circles in Figure 1, together with a hierarchy ronmental health, health promotion). Later, more formal of subclasses, and, at the lowest level, instances. Each sub- consultations were organised through reference group class and instance should have a subsumption ("is-a") members representing various jurisdictions. relationship with its parent class. Prior to each consultation meeting, we sent material intro- Existing classifications (such as the international classifi- ducing the classification project to participants. All con- cations of diseases [25], functioning and disability [26], sultations were conducted face to face. The number of and external causes of injuries [27], can be used as sub- participants varied from one or two, to larger groups of up classes of the classes 'Health Issues', 'Settings' and to 15, and the duration varied from one to three hours. In 'Resources and Infrastructure', while new subclass struc- each consultation, an introduction and background to the tures are proposed for the classes of public health 'Func- classification project were given with the aid of a slide tions', 'Determinants of Health' and 'Methods' (see Figure presentation, after which an early version of the public 1). health classification, rendered through a Web browser, was demonstrated. This was followed by a 'live' session The working definitions for the six top-level classes are using Protégé, which allowed participants to explore the shown in Table 1, and their immediate subclasses are structure of the classification, to suggest additions and given in Table 2. changes, and to immediately see their effect on the overall classification. The 'Functions' class is currently the most highly devel- oped. Both primary and instrumental functions were con- Last, participants were asked to identify practical uses for sidered to be important in conceptualising public health a multidimensional public health classification. Follow- (See Additional file 2). Primary functions are ends in ing consultation meetings, the project reference group themselves, while instrumental functions are means to debated proposed changes to the classification before those ends. Public health practitioners also described deciding to adopt or reject them. The Australian National instrumental functions as 'supporting', 'underpinning' or Public Health Information Working Group – a committee 'cross-cutting' functions because all primary functions rely with representation from all states and territories as well on them – they do not belong solely to any one of the pri- as relevant national bodies – discussed and provided feed- mary functions. back on an early draft version of the classification. While there was reasonable agreement among public health experts regarding the subclasses for the top-level Results Principles of development classes 'Health Issues', 'Determinants of Health', and 'Set- Development of the public health classification was tings', the remaining classes are in earlier stages of devel- guided by the following principles: opment. Page 4 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 nym-hyponym) relationships among the current subclasses of this top-level class. A copy of the full report on this initial phase of the project has been included as an additional file to this paper (Additional file 3), together with copies of the underlying classification ontology as it stood at the time of writing. Two sets of ontology files are provided: a set of Protégé project files (Additional file 4), and a set of interlinked HTML files that can be explored using a Web browser (Additional file 5). Potential uses for the classification The classification is used by assigning zero or more attributes chosen from each of the top-level class hierar- chies to the "thing" being classified. A wide range of "things" can potentially be organised according to the classification, including (but not restricted to) public health policies, programs and interventions, the popula- tion groups they target, and their outcomes. Figure 1 depicts some examples of things that could be classified, shown as heptagons. A Figure 1 model of public health classification Many participants in the development process expressed a A model of public health classification. Source: Adapted view that the classification would assist them in describ- from Gruszin S, Jorm L, Churches T, Straton J: Public Health ing what public health is, and what its characteristics are. Classifications Project Phase One: Final Report: Report to the They also identified a range of questions that, currently, National Public Health Partnership Group. Melbourne: National are difficult or impossible to answer, but which poten- Public Health Partnership; 2005. tially could be answered if the multidimensional classifi- cation were used to facilitate aggregated reporting on public health activity. Examples of such questions are The 'Methods' class was the subject of some disagreement, given in Figure 2. with some experts preferring to narrow its scope to those methods that are peculiar to – or only used by – public Participants also suggested a range of potential practical health (e.g. population-based epidemiology, health pro- applications for the classification. Examples of these are motion, environmental risk assessment). Others favoured given in Figure 3. A knowledge base to support communi- an inclusive approach that would capture all methods used cable disease surveillance, also constructed using the Pro- by public health, including those that, while not specific tégé software suite, has been described [39]; it should be to it, are employed by public health workers in the normal possible to use the classification ontology to create analo- course of their work (e.g. administration, management, gous databases for other specific or general areas of public policy development). An inclusive approach was adopted, health practice. in line with development principle (4) above. Discussion Public health experts expressed diametrically opposed The process of developing the classification brought to views as to whether 'infrastructure' represented aggregates light several areas of basic disagreement among Australian of 'resources', or whether 'resources' were in fact a subclass public health practitioners regarding the nature of public of 'infrastructure'. An inclusive approach was adopted, in health practice. line with development principle (4) above, with the rele- vant class termed 'Resources and Infrastructure', and its The inclusion, or otherwise, of preventive services deliv- subclasses capturing both compound (e.g. administrative ered on a one-to-one basis to individuals was particularly infrastructure, information systems) and unitary elements contentious. Such preventive services include screening (e.g. funds, workforce). and detection, immunisation, and counselling and life- style advice to support healthy behaviour, as well as man- Further work is needed to disentangle the mixture of par- agement (through lifestyle changes or pharmacological titive (holonym-meronym) and subsumption (hyper- means) of disease risk factors such as high blood pressure and high cholesterol. Page 5 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 Table 1: Classification of public health: six top-level classes and their working definitions Class Working definition Functions Public health functions. The purpose of public health interventions, actions, activities and programs. Health Issues Health, and well-being issues that affect health ('issues' includes: concerns, topics, problems). Health is defined (by the WHO) as 'a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity'. Determinants of Health Factors that influence health status and determine health differentials or health inequalities. They include, for example, natural, biological factors, such as age, sex and ethnicity; behaviour and lifestyles, such as smoking, alcohol consumption, diet and physical activity; physical and social factors, including employment and education, housing quality, the workplace and the wider urban and rural environment; and access to health care [a]. Methods The methods used by organised public health interventions (actions, activities, programs, services) to protect and promote health and prevent illness, injury and disability, that are designed to change population exposure, behavioural or health status. Settings Settings in which public health activities and interventions take place, institutional and social environments, partnerships, and locations (e.g. schools, local government, hospitals, workplaces). Resources and Infrastructure Resources and infrastructure, the means available for the operation of health systems, including human resources, facilities, equipment and supplies, financial funds and knowledge [b]. It includes both person-time and calendar time. Source: Gruszin S, Jorm L, Churches T, Straton J: Public Health Classifications Project Phase One: Final Report: Report to the National Public Health Partnership Group. Melbourne: National Public Health Partnership; 2005. [a] World Health Organisation: Health Impact Assessment (HIA): Glossary of terms used. Geneva; 2006. http://www.who.int/hia/about/glos/en/ index.html [b] World Health Organization: Health Promotion Glossary. Geneva; 1998. [WHO/HPR/HEP/98.1] Many participants argued that those individual preventive ological condition (such as high blood pressure) or dis- services related to communicable disease (immunisation, ease marked the boundary of public health practice. contact tracing and treatment for sexually transmitted Others regarded this boundary as spurious, because the infections) are public health activities because they help pathophysiological processes which underlie the develop- to protect the health of the whole population, through ment of chronic disease are continuous, and because herd immunity and controlling the spread of infection. interventions such as anti-hypertensive and cholesterol- Others felt that immunisation is only a legitimate part of lowering drugs, or even the 'Polypill' [40], may have dra- public health activity when it is delivered as part of an matic benefits in terms of morbidity and mortality at the organised government-funded program, such as through population level. local government or school health services. By adopting an inclusive approach, the public health clas- Most public health practitioners agreed that early detec- sification allows decisions about boundaries, inclusions tion of disease through screening is a public health activity and exclusions, to be made at the level of individual appli- when it is delivered through an organised government- cations of the classification. This is especially useful for funded program (such as national breast and cervical can- those boundary issues – such as where public health ends cer screening programs). Less clear was whether screening and clinical practice begins – about which opinion may that is not part of an organised program, such as oppor- evolve with knowledge about preventive interventions, tunistic bone density screening for osteoporosis, should and how and by whom they are best delivered. be seen as part of public health. The issue of whether a public health classification should There was substantial disagreement among public health be restricted to a domain solely within the health sector or practitioners regarding the inclusion, or otherwise, of whether it should it include the health-related activities of activities relating to prevention and management of non- other sectors was also frequently raised. Most public communicable disease through individual counselling or health practitioners agreed, when pressed, that accounting other interventions directed at lifestyle risk factors (smok- for public health should include the activities of, and ing, poor nutrition, risky alcohol use and lack of physical investments by, the non-health portfolios (such as educa- activity), and the early detection and management of bio- tion and transport) of national and state governments, as logical risk factors such as high blood pressure and high well as the relevant activities of local governments and cholesterol. Many contended that the diagnosis of a path- non-government organisations (NGOs). This is consistent Page 6 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 Table 2: Classification of public health: top two levels of all classes Top-level class Level 2 subclasses Functions Assess health of populations Primary: Promote health and prevent disease, disability and injury Protect from threats to health Instrumental Ensure public health capability Build the evidence base for public health Health issues Health and well-being Injury Diseases and conditions Disability and functioning Determinants of Health Environmental Socioeconomic External causes of injury Person-level Health system Methods Advocacy and lobbying Health impact assessment Research and evaluation Communicable disease control specific Immunisation Road safety methods Community action Infection control Screening to detect disease/risk factors Community development Legislation and regulation Social action Counselling Lifestyle advice Social marketing Diagnosis Management of biological risk Training and workforce development methods Directed investment Monitoring and surveillance Treatment methods Environmental monitoring Personal skills development Urban planning methods Epidemiologic methods Political action Vector control methods Exercise of capabilities Public policy development Waste management methods Food safety methods Radiation safety methods Other methods of intervention Health education Remediation of environment methods Settings Educational settings Home settings Other settings Healthcare settings Workplace settings Includes LOCATIONS – classification of geographical areas (e.g. postcodes). Local government and communities Transport settings settings Resources and infrastructure Administrative infrastructure Organisational systems Technical infrastructure Funds Partnerships Time Information systems Physical infrastructure Workforce Legislative infrastructure Policies Workforce development capacity Source: Gruszin S, Jorm L, Churches T, Straton J: Public Health Classifications Project Phase One: Final Report: Report to the National Public Health Partnership Group. Melbourne: National Public Health Partnership; 2005. Page 7 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 with the definition of public health used by the US Insti- those of Canada [16] and the Americas [14,15]) are lim- tute of Medicine as: "what we, as a society, do collectively ited to primary functions, while others include both pri- to assure the conditions for people to be healthy" [9]. mary and instrumental functions. In the UK [11], both primary (e.g. health promotion and disease prevention However, in practice there are major difficulties in captur- programs) and instrumental (e.g. development and main- ing information on public health-related activities and tenance of a public health workforce) functions are prom- expenditure by non-health sectors. In Australia, current inent. Our classification captures the majority of public health expenditure reporting is limited to the functions that other nations have described, including the health portfolio expenditures of the state and national (instrumental) 'partnership' and 'research' functions that governments [1]. One view was that the activities of non- are present in both the UK core functions [11] and the 10 health sectors should only be considered 'in scope' when essential public health services of the USA [10]. public health is their primary purpose (e.g. immunisation organised by local government). The 'boundary' issues we A 'quality assurance' function does not currently feature in encountered reflect the way that public health activity is our classification, although it is specified in the published conceptualised, and organised, in Australia. Similar exer- functions for public health in the USA [10], the Americas cises conducted in other countries would doubtless high- [14,15], the UK [11] and in WHO's most recent work light different issues. For example, public health services [13,15]. in the US (which has a strongly privatised approach to health care) are seen as having a major role in filling 'gaps' Public health practitioners and experts in Australia at no in health service provision (such as maternal and child stage suggested that such a function was a critical part of health services) for those without access to health insur- their work. It is possible that their views may subsequently ance, as well as in evaluating the accessibility and quality have changed, particularly in light of several recent scan- of personal health care services [10]. In several European dals [41-43] relating to the safety and quality of the care countries, provision of many public health services is fully provided in Australian public hospitals. devolved to the level of local municipalities, which also have responsibility for issues such as air and water quality, The approach we adopted in developing the classification noise diminution and waste management. It is likely that should maximise its flexibility for application in other set- the organisation of services in these countries influences tings. The ontology-building process offered particular the way their public health practitioners conceptualise the advantages in dealing with divergent (and often strongly boundaries between primary health care, public health held) views regarding what was and was not 'in scope'. and environmental protection. Although defining and specifying classes was central to the process, the emphasis was on modelling the relation- A comparison of the published public health functions of ships among classes, rather than on the within-class hier- other nations (see Additional file 1) shows that some (e.g. archies. We were able to adopt an inclusive approach, leaving scope for decisions about rules and exclusions to ƒ How much was spent last year on the prevention of obesity? ƒ How is public health relevant to components of the human ƒ Explain what public health is services delivery system? ƒ Organise information to answer key public health ƒ Why do public health unit costs differ across jurisdictions? questions ƒ Can we describe screening in clinical settings (e.g. Pap smears taken in General Practice surgeries)? ƒ Promote consistency in describing public health ƒ How much did we spend on prevention of HIV/AIDS relative to ƒ Improve data capture processes and the quality of other preventable diseases? reporting ƒ What are the nature and cost of partnerships between public ƒ Contribute to higher-level classification and standards health and other sectors? activities ƒ How much was spent on social marketing last year? ƒ Structure and design information and communications ƒ How many staff work in environmental health, and in what settings do they work? ƒ Audit the spread of activity across the public health business cycle ƒ How much did we spend and what was the activity on specific risk factors? ƒ Build models of good public health practice ƒ Link research, policy and practice A li Figure 2 ke. public h .. ealth classification should help answer questions A public health classification should help answer Potential Figure 3uses for a public health classification questions like.... Source: Adapted from Gruszin S, Jorm L, Potential uses for a public health classification. Source: Churches T, Straton J: Public Health Classifications Project Phase Adapted from Gruszin S, Jorm L, Churches T, Straton J: Pub- One: Final Report: Report to the National Public Health Partner- lic Health Classifications Project Phase One: Final Report: Report ship Group. Melbourne: National Public Health Partnership; to the National Public Health Partnership Group. Melbourne: National Public Health Partnership; 2005. Page 8 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 be made at the level of specific practical applications of Additional file 2 the classification. It is to be hoped that such practical Classification of public health: detail of the functions class. Table pre- applications will make the areas of contention explicit, senting the "Functions" class of the public health classification, and its encourage debate, and offer a way to move towards a com- subclasses. mon language to describe public health activity in Aus- Click here for file tralia and elsewhere. [http://www.biomedcentral.com/content/supplementary/1743- 8462-6-9-S2.pdf] Conclusion Additional file 3 The public health classification is an initial attempt to Gruszin S, Jorm L, Churches T, Straton J: Public Health Classifica- describe the important dimensions of public health activ- tions Project Phase One: Final Report: Report to the National ity. It will facilitate the organisation of information about Public Health Partnership Group. Melbourne: National Public public health activity so that it can be used to address Health Partnership; 2005. Full report of Phase One of the Australian questions relating to any of these dimensions, either sin- Public Health Classifications Project. gly or in combination. The authors encourage readers to Click here for file use the classification, and to suggest further refinements. [http://www.biomedcentral.com/content/supplementary/1743- It is our intention to further refine and extend several of 8462-6-9-S3.pdf] the class hierarchies, and to convert the ontology into RDF Additional file 4 and OWL format to make it suitable for use in Semantic Public Health ontology (PHont) Version 1.0. Protégé version 3.1 Web applications. We welcome potential collaborators in project files for Public Health Ontology (PHont) Version 1.0. this endeavour. Click here for file [http://www.biomedcentral.com/content/supplementary/1743- Abbreviations 8462-6-9-S4.zip] HIV/AIDS: Human Immunodeficiency Virus/Acquired ImmunoDeficiency Syndrome; OECD: Organisation for Additional file 5 Public Health ontology (PHont) Version 1.0. HTML web page version Economic Co-operation and Development; UK: United of Public Health Ontology (PHont) Version 1.0 Kingdom; US: United States of America; WHO: World Click here for file Health Organization; RDF: Resource Description Frame- [http://www.biomedcentral.com/content/supplementary/1743- work; OWL: Web Ontology Language (sic). 8462-6-9-S5.zip] Competing interests The authors declare that they have no competing interests. Acknowledgements Authors' contributions The following people contributed to the development of the classification through participating in the Reference Group for the Public Health Classi- LJ compiled the argument and drafted the background, fications Project: Louisa Jorm, Chair (New South Wales Department of discussion and conclusions of the manuscript. SG Health); Michael Ackland (Department of Human Services, Victoria); And- reported the results, contributed to the background and rea Casasola (Queensland Health); Jenny Cleary, Pam Gollow (Department discussion, and prepared the figures and tables. TC of Health and Community Services, Northern Territory); Charles Guest informed the directions of the project, provided key con- (ACT Health, Australian Capital Territory); Paul Jelfs (South Australian ceptual and practical input to its results, and reviewed and Department of Health); Paul Magnus, Catherine Sykes, Robert Van der edited the manuscript. All authors worked on the project, Hoek (Australian Institute of Health and Welfare); Dean Martin (Australian and have co-authored, with Judy Straton (see Acknowl- Bureau of Statistics); Karen Roger (National Public Health Partnership Sec- edgements), the report of phase one of the project. All retariat); authors read and approved the final manuscript. Colin Sindall (Australian Government Department of Health and Ageing); Judy Straton (Department of Health Western Australia); Tony Woollacott Additional material (South Australian Department of Health). Professor Richard Madden (National Centre for Classification in Health, University of Sydney) made important conceptual contributions to the classification. Additional file 1 Comparison of published public health functions and roles. Table pre- The (Australian) National Public Health Partnership funded the Public senting public health roles and functions published in international Health Classifications Project, some results of which are reported in this reports. article. Additional sources of funding for the authors were the New South Click here for file Wales Department of Health (L Jorm and T Churches) and the Public [http://www.biomedcentral.com/content/supplementary/1743- Health Information Development Unit, The University of Adelaide (S 8462-6-9-S1.pdf] Page 9 of 10 (page number not for citation purposes) Australia and New Zealand Health Policy 2009, 6:9 http://www.anzhealthpolicy.com/content/6/1/9 Gruszin). The funding body was represented on the Reference Group, and 24. Roper WL, Mays GP: Performance measurement in public health: Conceptual and methodological issues in building the as such had a role in the project design, operation, interpretation and science base. J Public Health Manag Pract 2000, 6:66-77. reporting of results. 25. World Health Organization: International Statistical Classification of Dis- eases and Related Health Problems (ICD-10). 10th revision. Geneva 1992, 1–3:. References 26. World Health Organization: International Classification of Functioning, 1. Australian Institute of Health and Welfare: National Public Health Disability and Health (ICF). Geneva 2001. Expenditure Report 2000–01. Canberra; [Health and Welfare Expenditure 27. ICECI Coordination and Maintenance Group: International Classifica- Series, HWE 25] 2004. tion of External Causes of Injuries (ICECI). Version 1.2 Amsterdam, and 2. National Public Health Partnership: Public Health Practice in Australia Adelaide: Consumer Safety Institute, and Australian Institute of Today – A Statement of Core Functions. Melbourne 2000. Health and Welfare National Injury Surveillance Unit; 2004. 3. McCarthy M: Integrating public health practice in Europe. 28. Organisation for Economic Co-operation and Development: A System Eurohealth 2003, 9:19-23. of National Health Accounts. Geneva 2000. 4. Barry M, Bialek R: Tracking our investments in public health: 29. Organisation for Economic Co-operation and Development: Interna- What have we learned? J Public Health Manag Pract 2004, tional Classification of Health Accounts. Geneva 2000. 10:383-92. 30. Public Health Information Tagging Standard [http:// 5. Mays GP, Halverson PK: Conceptual and methodological issues www.phl.nhs.uk/] in public health performance measurement: Results from a 31. National Public Health Language [http://www.phl.nhs.uk/docs/ computer-assisted expert panel process. J Public Health Manag version_2_0_files/PHL_Version_2_0_alpha.pdf] Pract 2000, 6:59-65. 32. Gruszin S, Jorm L, Churches T, Straton J: Public Health Classifications 6. Moulton AD, Halverson PK, Honoré PA, Berkowitz B: Public health Project Phase One: Final Report: Report to the National Public Health Part- finance: A conceptual framework. J Public Health Manag Pract nership Group Melbourne: National Public Health Partnership; 2005. 2004, 10:377-382. 33. Berners-Lee T, Hendler J, Lassila O: The Semantic Web. Sci Am 7. National Public Health Partnership: Public Health Performance Project- 2001, 284:34-43. Report to the National Public Health Partnership Group. Melbourne 2002. 34. Noy NF, McGuinness DL: Ontology Development 101: A Guide to Creat- 8. Bennett J: Investment in Population Health in Five OECD Countries. [OECD ing your First Ontology 2001 [http://protege.stanford.edu/publications/ Health Working Papers No. 2.] Paris: OECD; 2003. ontology_development/ontology101-noy-mcguinness.html]. Stanford, 9. Institute of Medicine: The Future of Public Health Washington, DC: CA: Knowledge Systems Laboratory, Stanford University National Academy Press; 1988. 35. Protégé [http://protege.stanford.edu/] 10. Public Health Functions Steering Committee, US Department of 36. Resource description Framework (RDF) [http://www.w3.org/ Health and Human Services: Public Health in America Washington, DC: RDF/] US Department of Health and Human Services; 1994. 37. Web Ontology Language (OWL) [http://www.w3.org/2004/ 11. Chief Medical Officer, United Kingdom: Public Health in England Lon- OWL/] don: UK Dept of Health; 2003. 38. Association of Schools of Public Health: What is Public Health? Wash- 12. Bettcher DW, Sapirie S, Goon EHT: Essential public health func- ington, DC 2004. tions: Results of the international Delphi study. World Health 39. Doyle TJ, Ma H, Groseclose SL, Hopkins RS: PHSkb: A knowledge- Stat Q 1998, 51:44-54. base to support notifiable disease surveillance. BMC Med 13. World Health Organization, Regional Office for the Western Pacific: Inform Decis Mak 2005, 5:27 [http://www.biomedcentral.com/1472- Essential Public Health Functions: A Three-Country Study in the Western 6947/5/27]. Pacific Region. Manila, Philippines 2003. 40. Wald NJ, Law MR: A strategy to reduce cardiovascular disease 14. Pan American Health Organization: Public Health in the Americas: Con- by more than 80%. BMJ 2003, 326:1419-1424. ceptual Renewal, Performance Assessment, and Bases for Action. Washing- 41. Weyden MB Van Der: The Bundaberg Hospital scandal: The ton, DC 2002. need for reform in Queensland and beyond. Med J Aust 2005, 15. Ramagem C, Ruales J: The essential public health functions as a strategy 183:284-285. for improving overall health systems performance: trends and challenges 42. Joseph AP, Hunyor SN: The Royal North Shore Hospital since the Public Health in the Americas Initiative, 2000–2007 Washing- inquiry: An analysis of the recommendations and the impli- ton, DC: Pan American Health Organization/World Health Organiza- cations for quality and safety in Australian public hospitals. tion; 2008. Med J Aust 2008, 188:469-472. 16. National Advisory Committee on SARS and Public Health: Learning 43. Weyden MB Van Der: Good and safe doctors. Med J Aust 2008, from SARS: Renewal of Public Health in Canada Ottawa: Health Canada; 188:497. 17. Suen J, Magruder C: National profile: Overview of capabilities and core functions of local public health jurisdictions in 47 states, the District of Columbia, and 3 US territories, 2000– 2002. J Public Health Manag Pract 2004, 10:2-12. 18. Centers for Disease Control and Prevention: National Public Health Performance Standards Program – State Public Health System Performance Assessment Instrument. Atlanta 2001. 19. Centers for Disease Control and Prevention: National public health Performance Standards Program – Local Public Health System Performance Assessment Instrument. Atlanta 2001. Publish with Bio Med Central and every 20. Mays GP, McHugh MC, Shim K, Lenaway D, Halverson PK, Moones- scientist can read your work free of charge inghe R, Honoré P: Getting what you pay for: Public health "BioMed Central will be the most significant development for spending and the performance of essential public health services. J Public Health Manag Pract 2004, 10:435-443. disseminating the results of biomedical researc h in our lifetime." 21. Owen T, Jorm L: The national public health performance Sir Paul Nurse, Cancer Research UK project: How do we know whether Australian public health ser vices are performing? NSW Public Health Bulletin 2005, 16:2-8. Your research papers will be: 22. Lower T, Durham G, Bow D, Larson A: Implementation of the available free of charge to the entire biomedical community Australian core public health functions in rural Western Australia. Aust N Z J Public Health 2004, 28:418-25. peer reviewed and published immediately upon acceptance 23. Beitsch LM, Brooks RG, Grigg M, Menachemi N: Structure and cited in PubMed and archived on PubMed Central Functions of State Public Health Agencies. Am J Public Health yours — you keep the copyright 2006, 96:167-72. BioMedcentral Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 10 of 10 (page number not for citation purposes)

Journal

Australia and New Zealand Health PolicySpringer Journals

Published: Apr 9, 2009

References