Measuring Partnership Effectiveness: evaluating partnerships
By Andrea Bodkin, HC Link Coordinator
This blog is one of a series profiling some of HC Link’s “oldie but goodie” resources from the past several years. This resource was published under HC Link’s previous name, the Healthy Communities Consortium, by former member organization OPHA.
Evaluate, celebrate and renew is one of 6 activities for successful partnerships, which Gillian Kranias and I presented in a June webinar. According to a survey conducted with participants of that webinar, it’s one of the activities that people find the most challenging. In 2008 I wrote that the world of evaluation can seem mystical, confusing and daunting. I will cheerfully admit that 9 years later, I still often feel like that! That’s why I delved back into a two-part resource that we published in 2011 on partnership evaluation, Measuring Partnership Effectiveness. Part one provides and overview of partnership evaluation, including the benefits of assessing your partnership and principles to guide your partnership evaluation efforts. Part Two profiles a number of tools that you can use to measure the effectiveness of your partnership.
A simple way to think about evaluation that’s relevant to those of you who work in partnership, is the three levels to partnership evaluation:
- Level 1: Measuring coalition infrastructure, functioning and processes: how well the partnership is working together. Level 1 evaluation is process evaluation that measures short-term outcomes.
- Level 2: Measuring partnership programs and interventions: what the partnership is accomplishing together, such as the activities delivered and populations impacted. Level 2 evaluation is outcome evaluation that measures short and intermediate outcomes.
- Level 3: Measuring community change and community outcomes: changes in health status, community capacity, the sustainability or institutionalization of programs and services. Level 3 evaluation is outcome evaluation that measures long term outcomes.
Conducting a Level 1 evaluation can provide valuable information on how your partnership is functioning, how each partner is contributing to the partnership, and the effectiveness and efficiency of the partnership. This information can be used to help your partnership improve and make adjustments over time, resulting in improved functioning and a greater likelihood of achieving the goals and activities of the partnership. Tools used to assess partnerships typically gather data on things like:
- Purpose of the partnership: vision, mission, goals
- Partnership structure and meetings
- Member diversity and involvement
- Sense of community/cohesion and identify
- Task focus and sustainability
- Use of resources
- Membership needs and benefits
So what does a Level 1 evaluation look like? Here’s a case study from HC Link’s efforts to assess and improve its partnership functioning:
As a partnership between three provincial organizations, HC Link understands the importance of assessing our partnership functioning as well as how we meet our goals and objectives. Our Level 1 partnership evaluation focusses on measuring the effectiveness and efficiency of HC Link, how well we are structured/organized, how well we communicate, and the quality of our relationships.
We evaluate annually, using a mixed-method approach, with different approaches each year: surveys, key informant interviews, focus groups, analysis of service statistics and document reviews. One of the more innovative approaches we use is “Evaluation Day!” where HC Link staff are paired up and answer a series of questions in a dialogue. Immediately following, individuals answer an online survey. We found that the paired interviews develop relationships and understanding, and provide a richness that is then reflected in the online results. This would be a great approach for steering committee members! You can read more about “Evaluation Day!” in this blog post.
We use the results of our partnership evaluation to celebrate our accomplishments and highlight areas for improvement. When an area(s) for improvement is identified, we develop a strategy to help us strengthen this quality. Over our 8 years of working in partnership, we have made a number of adjustments based on the results of our evaluation. For example, we make changes to our meeting structure on an annual basis, and have made several rounds of changes to our structure to adapt to our changing environment/contexts.
In addition to the ten tools analyzed in the Part Two resource, here are some additional resources you may find helpful:
The Wilder Collaboration Factors Inventory has an online collaboration assessment tool with research-tested success factors.
Bilodeau, Galarneau, Fournier & Potvin have developed an 18 indicator tool: Outil diagnostique de l’action en partenariat. Currently the tool is available in French only, however Health Nexus is adapting the tool for English audiences.
Collaborative Leadership in Practice(CLIP) was a project of Health Nexus and the Ontario Public Health Association. The aim of the project was to equip leaders to be able to collaborate with diverse partners and recognize the power structures that exist in our partnerships. On the resources page, you’ll find an excellent tool- with guide- called Equity Analysis Tool of Group Membership.
HC Link’s Participatory Evaluation Toolkit and The Power of Reflection: an introduction to participatory evaluation techniques describe techniques that can be used to help assess and strengthen your parntership.
This 2017 edition of HC Link’s Digest + is on evaluation, and contains many more helpful tools and resources.