Jumpstarting Monitoring and Evaluation for Global Health Engagement

Jumpstarting Monitoring and Evaluation for Global Health Engagement The U.S. Department of Defense (DoD) has made significant strides toward improving its efforts in global health engagement (GHE) to include an emphasized requirement for monitoring and evaluation (M&E) of GHE activities.1 However, a renewed push is needed to capitalize on over a decade of improved perspective on global health and turn it into action at the medical planning and execution levels. The lack of consistent and persistent, quality M&E has limited the ability to clearly demonstrate the objective value of the U.S. military application of the discipline of global health – aka, global health engagement. We suspect that GHE is undervalued in its ability to achieve goals in both security cooperation and global health but do not have well-documented evidence to prove that hypothesis. Multiple challenges and barriers stand in the way of high-quality M&E, to include a lack of dedicated resources; inconsistent and disparate data input and collection across the DoD; non-standardized after action reports for GHE that do not contribute to high-quality evaluation;2 and the challenges of incorporating relevant metrics into the joint planning process that can be followed up longitudinally. To simplify the approach, we can consider three categories of desired outcomes for M&E of global health engagement: (1) geopolitical/military outcomes for the USA, (2) operational readiness of U.S. military forces, and (3) positive health outcomes for the host nation’s health system or the involved community. Command surgeon’s offices and planners cannot get bogged down trying to measure everything but should begin by picking one S.M.A.R.T.3 metric in each of the three categories that can be extrapolated to a long-term outcome indicator – aka, second-level measurement of effectiveness,4 that is contributory to desired endstates. Potentially, these three outcome metrics can be assessed for all GHE missions for a year or two, then analyzed, decisions made upon that information, and new or additional measures could then be selected. Ideally, the joint health community should decide on an accepted initial set of the three measures and their collection. Then, all combatant commands and components would collect the same data at the same time for centralized analysis with a larger denominator. Geopolitical/military outcomes will remain the hardest and most nebulous to measure. However, we must measure effects that contribute to the military objective and to the desires of the government lead. For DoD GHE the lead may be the four-star GCC commander, a country Chief of Mission, a unique U.S. government funding authority, or all three. At the highest, strategic level the current Secretary of Defense (SECDEF) has touted these three lines of effort that commanders will support: restore military readiness, strengthen alliances, and bring business reforms.5 The final geopolitical/military endstate is beyond the reach of GHE alone, but proxy measures of effectiveness that contribute towards the endstate(s) should be the target for determination and measurement of GHE activities. On first blush, the second SECDEF line of effort “strengthen alliances” seems the best fit for GHE, nevertheless, the combatant command theater campaign plan will codify lower echelon targets that support this endstate via intermediate military objectives. Additionally, this category of outcomes is best measured via qualitative means such as structured observation, surveys, or key informant interviews which require disciplined design and follow-up to provide valid and reliable information. Historically, this type of longitudinal well designed, qualitative analysis has not been accomplished for DoD health engagements. Operational readiness training for U.S. forces is not a mandate for all foreign assistance funding authorities. However, any deployment of a team to execute a health engagement affords the opportunity to include learning and training objectives that can be tracked and measured capitalizing on readiness opportunities via peacetime operations is not a new concept, but not incidentally, this also happens to be the first SECDEF line of effort noted in the referenced memo. The operational readiness application should be the easiest to measure since we have pre-identified tasks on mission essential task lists (METL) from the highest level joint Universal Joint Task List, through service specific lists and down to unit level tasks. GHE activities can be employed across a variety of operations in permissive environments and thus offer opportunities to test and evaluate medical capabilities as they might be employed across the full spectrum and phases of operations. Therefore, any GHE activity should be able to be matched with a METL element that becomes that second-level measure of effectiveness for the readiness objective. A wide variety of operational training outcomes may be selected that could include confirmation of point of injury care capability in a disaster exercise or the ability to operate on a team that can accomplish health system change in the country. We cannot measure every single operational readiness element every time, but should be able to select at least one or two METL aligned outcomes for each engagement even for missions with appropriations that do not require the operational readiness to be measured. As mentioned, a standardized approach from the joint health community might make this less gargantuan. A simple, uniform process is needed to begin and then determine how to proceed with the increased measurement in the future. Military purists may argue that health outcomes for the host nation populations are secondary to the first two outcome categories when executing global health activities. On the other hand, health professionals may wonder why the health outcomes are not the preeminent priority. However, there needs to be a symbiotic perspective rather than an either/or situation. It is intuitive that better health outcomes will lead to greater positive impact on geopolitical, military, and operational readiness outcomes. Again, currently there is not enough evidence to support or oppose this claim, therefore we must intentionally and vigorously attempt to accomplish M&E that analyzes and documents significant health outcomes. Ultimately, foreign assistance intends for host nation to have enduring positive effects that benefit security cooperation and some level of interoperability for contingency operations. This requires knowledge of the host civilian and military health systems and their self-assessed gaps and needs. Addressing health outcomes cannot be grandiose or all-inclusive but must be specific and tailored to the local civilian health region or the military units involved. The capabilities of the U.S. medical team that is sent to accomplish the mission must also align with the targeted needs or gaps. Expectations can involve an immediate effect but should also aim at a long-term potential health effect that may not be realized by the deployed team during the initial engagement. The health outcome measured may apply to the civilian population being served or the force health protection or casualty care capability needs of the partner military. Sources that can help determine an appropriate, mutually beneficial, health outcome include Sphere standards for humanitarian response,6 Millennium/Sustainable Development goals, Global Health Security Agenda goals, USAID strategic priorities (formerly the Global Health Initiative), and perhaps, most importantly, collaborative assessment with the partner nation. A major challenge is incorporating formal measurement into the planning cycle of engagement from the very beginning to the end. Determining the way for the measures to be captured and collected in a relatively efficient, simple, and consistent manner that becomes the accepted modus operandi is also essential. Paramount to overcoming this challenge is developing medical planners who understand M&E and can incorporate the necessary elements into their regular duties. As the joint health community begins to incorporate this habit of measuring within these three categories it can develop a body of data and information that can support future decisions. Selection of simple measurements may then inform how DoD designates and employs health capabilities for future engagements. Another challenge is how to collect the data in a central, secure, reliable repository that is accepted across the AORs and stakeholder communities. The system must be easy to work with to extract desired data and reports, and be flexible to accommodate current and future needs. Several operational and lessons learned repositories exist but they have not been designed nor adequate for the analytical purposes needed. There may be some potential options for such software solutions out there that might warrant an in depth look by the joint health community. Going forward, the joint military health community must decide upon a few consistent S.M.A.R.T. measures within these three categories of geopolitical/military outcomes, operational readiness outcomes for U.S. forces, and health outcomes for host partner health systems. The focus should be on second-level outcome indicators (measures of long-term effectiveness) instead of only the process measures. It would be extremely valuable for a central, joint-oriented entity to then help combatant and component commands to track the M&E, advise on consistency across AORs, and conduct analysis and report on the findings. Once a maturing database is established over a few years, it may open the door for even more complex and valuable impact evaluation.7 Previous experts have had solid ideas on “mission-generic metrics”;8 nonetheless, even such a seemingly simple, binary checklist as recommended by those authors has not been tested. Perhaps, simplifying down to three outcomes, one from each of three categories could be a digestible start to M&E for the complex and burgeoning arena of global health engagement. The DoD health community must develop and employ sound methods that measure outcome indicators that provide valid and reliable information for assessment towards endstates and for sound decision making. There has been decades of significant trust in the value of health engagement activities across the spectrum of operations because of first-hand experience, anecdotal success, and clear association of health as a connecting bridge between people and nations. It is time to step up the analysis of global health projects and activities so that the measured outcomes can guide the future on the most effective engagements that support to security cooperation and global health. References 1 Department of Defense Instruction 2000.30 , Global Health Engagement Activities, July 12, 2017 , accessed on the Internet 2 March 2018 at https://fas.org/irp/doddir/dod/i2000_30.pdf. 2 Drifmeyer EJ , Llewellyn CH , Measuring the effectiveness of Department of Defense humanitarian assistance, Center for Disaster and Humanitarian Assistance Medicine (CDHAM) Publication 02-03, 2002. Uniformed Services University of the Health Sciences, Bethesda, MD. 3 S.M.A.R.T. Specific, Measurable, Achievable, Relevant, and Time-bound. 4 RAND , Center for Military Health Policy Research, Prototype Handbook for Monitoring and Evaluating Department of Defense Humanitarian Assistance Projects: A Primer on Monitoring and Evaluation and a User’s Guide for Project Assessment, 2011 accessible on the Internet on 2 March 2018 at https://www.rand.org/pubs/technical_reports/TR784.html 5 U.S Secretary of Defense , Guidance form from Secretary Jim Mattis, 5 Oct 2017 , accessed on the Internet 2 March 2018 at https://www.defense.gov/Portals/1/Documents/pubs/GUIDANCE-FROM-SECRETARY-JIM-MATTIS.pdf 6 Sphere Project : Humanitarian charter and minimum standards in humanitarian response, accessed on the Internet March 5 at http://www.sphereproject.org/ 7 Perrin B , Linking monitoring and evaluation to impact evaluation, Impact evaluation notes, No.2 April 2012 . Accessed February 6, 2018 at https://www.interaction.org/sites/default/files/Linking%20Monitoring%20and%20Evaluation%20to%20Impact%20Evaluation.pdf 8 Waller SG , Ward JB , Montalvo M , et al. : A new paradigm for military humanitarian medical missions: mission-generic metrics . Mil Med 2011 ; 176 ( 8 ): 845 . Google Scholar CrossRef Search ADS PubMed Author notes The views expressed are those of the authors and do not reflect the official policy or position of the Uniformed Services University of the Health Sciences, The US Air Force, The Department of Defense or the U.S. Government. Published by Oxford University Press on behalf of Association of Military Surgeons of the United States 2018. This work is written by (a) US Government employee(s) and is in the public domain in the US. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Military Medicine Oxford University Press

Jumpstarting Monitoring and Evaluation for Global Health Engagement

Military Medicine , Volume Advance Article (7) – Jun 28, 2018

Loading next page...
 
/lp/ou_press/jumpstarting-monitoring-and-evaluation-for-global-health-engagement-ofAQQMg6um
Publisher
Association of Military Surgeons of the United States
Copyright
Published by Oxford University Press on behalf of Association of Military Surgeons of the United States 2018.
ISSN
0026-4075
eISSN
1930-613X
D.O.I.
10.1093/milmed/usy054
Publisher site
See Article on Publisher Site

Abstract

The U.S. Department of Defense (DoD) has made significant strides toward improving its efforts in global health engagement (GHE) to include an emphasized requirement for monitoring and evaluation (M&E) of GHE activities.1 However, a renewed push is needed to capitalize on over a decade of improved perspective on global health and turn it into action at the medical planning and execution levels. The lack of consistent and persistent, quality M&E has limited the ability to clearly demonstrate the objective value of the U.S. military application of the discipline of global health – aka, global health engagement. We suspect that GHE is undervalued in its ability to achieve goals in both security cooperation and global health but do not have well-documented evidence to prove that hypothesis. Multiple challenges and barriers stand in the way of high-quality M&E, to include a lack of dedicated resources; inconsistent and disparate data input and collection across the DoD; non-standardized after action reports for GHE that do not contribute to high-quality evaluation;2 and the challenges of incorporating relevant metrics into the joint planning process that can be followed up longitudinally. To simplify the approach, we can consider three categories of desired outcomes for M&E of global health engagement: (1) geopolitical/military outcomes for the USA, (2) operational readiness of U.S. military forces, and (3) positive health outcomes for the host nation’s health system or the involved community. Command surgeon’s offices and planners cannot get bogged down trying to measure everything but should begin by picking one S.M.A.R.T.3 metric in each of the three categories that can be extrapolated to a long-term outcome indicator – aka, second-level measurement of effectiveness,4 that is contributory to desired endstates. Potentially, these three outcome metrics can be assessed for all GHE missions for a year or two, then analyzed, decisions made upon that information, and new or additional measures could then be selected. Ideally, the joint health community should decide on an accepted initial set of the three measures and their collection. Then, all combatant commands and components would collect the same data at the same time for centralized analysis with a larger denominator. Geopolitical/military outcomes will remain the hardest and most nebulous to measure. However, we must measure effects that contribute to the military objective and to the desires of the government lead. For DoD GHE the lead may be the four-star GCC commander, a country Chief of Mission, a unique U.S. government funding authority, or all three. At the highest, strategic level the current Secretary of Defense (SECDEF) has touted these three lines of effort that commanders will support: restore military readiness, strengthen alliances, and bring business reforms.5 The final geopolitical/military endstate is beyond the reach of GHE alone, but proxy measures of effectiveness that contribute towards the endstate(s) should be the target for determination and measurement of GHE activities. On first blush, the second SECDEF line of effort “strengthen alliances” seems the best fit for GHE, nevertheless, the combatant command theater campaign plan will codify lower echelon targets that support this endstate via intermediate military objectives. Additionally, this category of outcomes is best measured via qualitative means such as structured observation, surveys, or key informant interviews which require disciplined design and follow-up to provide valid and reliable information. Historically, this type of longitudinal well designed, qualitative analysis has not been accomplished for DoD health engagements. Operational readiness training for U.S. forces is not a mandate for all foreign assistance funding authorities. However, any deployment of a team to execute a health engagement affords the opportunity to include learning and training objectives that can be tracked and measured capitalizing on readiness opportunities via peacetime operations is not a new concept, but not incidentally, this also happens to be the first SECDEF line of effort noted in the referenced memo. The operational readiness application should be the easiest to measure since we have pre-identified tasks on mission essential task lists (METL) from the highest level joint Universal Joint Task List, through service specific lists and down to unit level tasks. GHE activities can be employed across a variety of operations in permissive environments and thus offer opportunities to test and evaluate medical capabilities as they might be employed across the full spectrum and phases of operations. Therefore, any GHE activity should be able to be matched with a METL element that becomes that second-level measure of effectiveness for the readiness objective. A wide variety of operational training outcomes may be selected that could include confirmation of point of injury care capability in a disaster exercise or the ability to operate on a team that can accomplish health system change in the country. We cannot measure every single operational readiness element every time, but should be able to select at least one or two METL aligned outcomes for each engagement even for missions with appropriations that do not require the operational readiness to be measured. As mentioned, a standardized approach from the joint health community might make this less gargantuan. A simple, uniform process is needed to begin and then determine how to proceed with the increased measurement in the future. Military purists may argue that health outcomes for the host nation populations are secondary to the first two outcome categories when executing global health activities. On the other hand, health professionals may wonder why the health outcomes are not the preeminent priority. However, there needs to be a symbiotic perspective rather than an either/or situation. It is intuitive that better health outcomes will lead to greater positive impact on geopolitical, military, and operational readiness outcomes. Again, currently there is not enough evidence to support or oppose this claim, therefore we must intentionally and vigorously attempt to accomplish M&E that analyzes and documents significant health outcomes. Ultimately, foreign assistance intends for host nation to have enduring positive effects that benefit security cooperation and some level of interoperability for contingency operations. This requires knowledge of the host civilian and military health systems and their self-assessed gaps and needs. Addressing health outcomes cannot be grandiose or all-inclusive but must be specific and tailored to the local civilian health region or the military units involved. The capabilities of the U.S. medical team that is sent to accomplish the mission must also align with the targeted needs or gaps. Expectations can involve an immediate effect but should also aim at a long-term potential health effect that may not be realized by the deployed team during the initial engagement. The health outcome measured may apply to the civilian population being served or the force health protection or casualty care capability needs of the partner military. Sources that can help determine an appropriate, mutually beneficial, health outcome include Sphere standards for humanitarian response,6 Millennium/Sustainable Development goals, Global Health Security Agenda goals, USAID strategic priorities (formerly the Global Health Initiative), and perhaps, most importantly, collaborative assessment with the partner nation. A major challenge is incorporating formal measurement into the planning cycle of engagement from the very beginning to the end. Determining the way for the measures to be captured and collected in a relatively efficient, simple, and consistent manner that becomes the accepted modus operandi is also essential. Paramount to overcoming this challenge is developing medical planners who understand M&E and can incorporate the necessary elements into their regular duties. As the joint health community begins to incorporate this habit of measuring within these three categories it can develop a body of data and information that can support future decisions. Selection of simple measurements may then inform how DoD designates and employs health capabilities for future engagements. Another challenge is how to collect the data in a central, secure, reliable repository that is accepted across the AORs and stakeholder communities. The system must be easy to work with to extract desired data and reports, and be flexible to accommodate current and future needs. Several operational and lessons learned repositories exist but they have not been designed nor adequate for the analytical purposes needed. There may be some potential options for such software solutions out there that might warrant an in depth look by the joint health community. Going forward, the joint military health community must decide upon a few consistent S.M.A.R.T. measures within these three categories of geopolitical/military outcomes, operational readiness outcomes for U.S. forces, and health outcomes for host partner health systems. The focus should be on second-level outcome indicators (measures of long-term effectiveness) instead of only the process measures. It would be extremely valuable for a central, joint-oriented entity to then help combatant and component commands to track the M&E, advise on consistency across AORs, and conduct analysis and report on the findings. Once a maturing database is established over a few years, it may open the door for even more complex and valuable impact evaluation.7 Previous experts have had solid ideas on “mission-generic metrics”;8 nonetheless, even such a seemingly simple, binary checklist as recommended by those authors has not been tested. Perhaps, simplifying down to three outcomes, one from each of three categories could be a digestible start to M&E for the complex and burgeoning arena of global health engagement. The DoD health community must develop and employ sound methods that measure outcome indicators that provide valid and reliable information for assessment towards endstates and for sound decision making. There has been decades of significant trust in the value of health engagement activities across the spectrum of operations because of first-hand experience, anecdotal success, and clear association of health as a connecting bridge between people and nations. It is time to step up the analysis of global health projects and activities so that the measured outcomes can guide the future on the most effective engagements that support to security cooperation and global health. References 1 Department of Defense Instruction 2000.30 , Global Health Engagement Activities, July 12, 2017 , accessed on the Internet 2 March 2018 at https://fas.org/irp/doddir/dod/i2000_30.pdf. 2 Drifmeyer EJ , Llewellyn CH , Measuring the effectiveness of Department of Defense humanitarian assistance, Center for Disaster and Humanitarian Assistance Medicine (CDHAM) Publication 02-03, 2002. Uniformed Services University of the Health Sciences, Bethesda, MD. 3 S.M.A.R.T. Specific, Measurable, Achievable, Relevant, and Time-bound. 4 RAND , Center for Military Health Policy Research, Prototype Handbook for Monitoring and Evaluating Department of Defense Humanitarian Assistance Projects: A Primer on Monitoring and Evaluation and a User’s Guide for Project Assessment, 2011 accessible on the Internet on 2 March 2018 at https://www.rand.org/pubs/technical_reports/TR784.html 5 U.S Secretary of Defense , Guidance form from Secretary Jim Mattis, 5 Oct 2017 , accessed on the Internet 2 March 2018 at https://www.defense.gov/Portals/1/Documents/pubs/GUIDANCE-FROM-SECRETARY-JIM-MATTIS.pdf 6 Sphere Project : Humanitarian charter and minimum standards in humanitarian response, accessed on the Internet March 5 at http://www.sphereproject.org/ 7 Perrin B , Linking monitoring and evaluation to impact evaluation, Impact evaluation notes, No.2 April 2012 . Accessed February 6, 2018 at https://www.interaction.org/sites/default/files/Linking%20Monitoring%20and%20Evaluation%20to%20Impact%20Evaluation.pdf 8 Waller SG , Ward JB , Montalvo M , et al. : A new paradigm for military humanitarian medical missions: mission-generic metrics . Mil Med 2011 ; 176 ( 8 ): 845 . Google Scholar CrossRef Search ADS PubMed Author notes The views expressed are those of the authors and do not reflect the official policy or position of the Uniformed Services University of the Health Sciences, The US Air Force, The Department of Defense or the U.S. Government. Published by Oxford University Press on behalf of Association of Military Surgeons of the United States 2018. This work is written by (a) US Government employee(s) and is in the public domain in the US.

Journal

Military MedicineOxford University Press

Published: Jun 28, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off