Featured Advocate: Rhonda Schlangen

“A donor asked how my organization evaluated the effectiveness of our advocacy work, since counting votes supporting a desired policy does not speak to effectiveness.

“After that conversation, I started to set up some systems to evaluate our advocacy progress. ”

“Advocacy tends to focus on action—planning has been focused on what we're going to do—not anchored in an explicit theory of change or focused on outcomes. What will change as a result of those actions? Advocates usually intuitively know these things. Evaluation is making them explicit.”

“I thought evaluation was a responsibility— knowing whether you actually did what you started out to do and whether you did so effectively. So, in my time with PPFA's international division, I have been working to integrate evaluation with our advocacy efforts.”

Rhonda Schlangen works for the Planned Parenthood Federation of America’s (PPFA) International Division. The international Division of PPFA works to increase access to family planning and other reproductive health services through direct work with organizations in the developing world and by advocating for sound US foreign policy related to reproductive health and rights.

InnoNet: How and why did you become involved in your service area?

Rhonda: In the past, I have been involved in most elements of advocacy: I worked for a legislator; I have been a lobbyist; I have been involved in grassroots work, etc. At one point when I was a lobbyist, a donor asked how my organization evaluated the effectiveness of our advocacy work, since counting votes supporting a desired policy does not speak to effectiveness. After that conversation, I started to set up some systems to evaluate our advocacy progress.

In order to do this I started to educate myself on evaluation through courses (at the Evaluators Institute and Johns Hopkins) and other means and fell in love with evaluation. Coming at it from my background (as that of an advocate), I was puzzled to see advocacy evaluation really wasn't being done. I was even told by a donor that it was impossible—that you could not establish a relationship between a vote and an advocacy effort so it was not worth trying.

I thought evaluation was a responsibility—knowing whether you actually did what you started out to do and whether you did so effectively. So, in my time with PPFA's international division, and with very positive support from our leadership, I have been working to integrate evaluation with our advocacy efforts.

InnoNet: How does PPFA use advocacy? Can you share examples of advocacy successes?

Rhonda: PPFA works for policy change in support of sound sexual and reproductive health and rights policies throughout the world. In the US, the International Division works at two levels. We work in Washington, DC to educate policymakers and their staff about sexual and reproductive health policy, and we work around the country through Planned Parenthood Federation of America local affiliates to incorporate international issues into their public policy work. A majority of local affiliates are interested in supporting global sexual/reproductive health advocacy in their work—and we help them do that. We are working with them to reach out to their communities to generate activism and advocacy on the behalf of sound policies.

InnoNet: How do you think evaluation affects advocacy?

Rhonda: Evaluation absolutely affects our ability to plan for advocacy. In my own part of the organization—I am in my 8th year with PPFA—planning has been revolutionized. Advocacy tends to focus on action—advocacy work by nature is very nimble and fluid according to the external environment. Therefore, planning has been focused on what we're going to do—the action we're going to take. However, planning tended not to be anchored in an explicit theory of change or focused on outcomes—what will change as a result of those actions. Advocates usually intuitively know these things. Evaluation is making that explicit.

Planning that integrates evaluation fills in those blanks and gives us a mechanism to test our assumptions, ask the hard questions, and obviously to monitor our progress. If we set up our advocacy activities correctly, we can consistently monitor against benchmarks we have set for ourselves. Evaluation enables us to test our strategies and helps us to assure our resources—which are limited—are being used to the most effect. Evaluation tightens our approaches.

Since we are unable to claim causality (i.e. confidently know exactly which organization's advocacy efforts brought about success), at the end of the day, it is hard to attribute what was the factor that caused the policymaker to vote the way they did. But, we can use evaluation to review chain of influence that leads to policy change and test our strategies and methodologies.

InnoNet: Has your organization had any experience in evaluating advocacy efforts?

Rhonda: We conducted a formative evaluation of one area of our Division's work a few years ago—that was our first foray into advocacy evaluation. We started by developing a logic model for a program approach that we were using in the US. In the program we evaluated, we were implementing "global partnerships" between our local US affiliates and service providers in developing countries. The US-based affiliates and service providers in developing countries exchanged skills and learning. In addition to building the capacity of service providers in both the US affiliates and developing countries, the exchange was also a tool to develop and inform an active constituency in the US.

InnoNet: What did your organization learn from that experience?

Rhonda: The evaluation process was a really wonderful capacity building exercise for my colleagues and myself. About 50% of my time was spent working with my colleagues on basic evaluation concepts: what is an outcome vs. an objective; developing a log framework for the program; developing indicators for their areas of work and making sure we had corresponding tracking systems set up, etc. It sounds elementary but it really had not been part of our culture or way of working.

The evaluation itself was very interesting and had results that changed the way we implemented the program. Even more surprising and notable was the process and how it brought staff along and how that experience became the catalyst for changing the way we think about and work with evaluation and how it is integrated into our planning.

InnoNet: Did you use any specific advocacy evaluation tools?

Rhonda: As I said, we used a logic model. But really what we tried to do was make sure the evaluation process was grounded in sound evaluation practice. Even if it was not a particularly traditional evaluation such as that we might use with a healthcare program, it was important to us that the evaluation methodology would "hold water." I think we made our own way. At that point—about 5 or 6 years ago—I looked around for a toolkit and really couldn't find anything that was relevant to what we were doing. The process of developing our own tools and approaches was really what helped increase our capacity.

InnoNet: How useful were the specific advocacy evaluation tools?

Rhonda: We developed indicators, tracking mechanisms, and checklists that we put in place after the program was launched. We did surveys and internal work on processes. We did matrices to assess what a successful partnership looked like. Some tools we are still using and some we are not. We streamlined and evolved into using global indicators (opposed to organically defined indicators in different sites) against which all program activities are measured.

InnoNet: Which of the advocacy evaluation resources would you recommend to others?

Rhonda: I can not pick out a single one; Everyone's needs are so different. You may wish to take what you can from the advocacy evaluation resources that already exist and adapt them internally to meet your specific needs. The process of adapting or developing tools is very instructive. National advocacy organizations have more work to do developing great internal capacity for evaluation.

InnoNet: In what ways do you think the Advocacy Evaluation Resource Center has benefited PPFA's International Division?

Rhonda: My awareness of what is out there has increased because of the Resource Center. The Resource Center will become increasingly important. There seems to be a legitimacy issue related to advocacy evaluation—Is advocacy evaluation possible? Realistic? It is a legitimate field of inquiry? Having this Resource Center available and online—a collection point for this body of work—helps me. The virtual community is invaluable. It offers legitimacy, connections, and a chance to share resources.

InnoNet: Do you have any advice to share with others about evaluating their advocacy efforts?

Rhonda: Encouragement—advocacy evaluation is a rewarding undertaking. The process creates focus. It allows us to use our resources more efficiently. Though it is difficult to prove causation, evaluating your efforts is still important. Advocacy relies on influencing a chain of events that ultimately causes action in support of your policy target. It's a long chain of events with many external influences, but evaluation allows us to test our strategies. Is this approach attracting more supporters? What are effective ways to engage young advocates? At the other end of the chain of influence—the policymakers—it can be difficult or impossible in the US political context to assess why each policymaker voted the way they did.

Our European counterparts apparently have more access to policymakers and are able to get direct feedback on their efforts to help assess their work. Since that is not the case in the US—mostly because of the political context—it is more difficult to arrive at knowing how we influenced that final link of the chain. Evaluation helps us detect patterns and develop models. It is also an issue of fiscal responsibility. Advocacy is an important part of what PPFA's International Division does and, ultimately, evaluating those efforts helps ensure we are accountable to the people we are committed to serving.

InnoNet: What do you think is needed to move the field of advocacy evaluation forward?

Rhonda: I think two things are needed to move the field forward. First, there is the piece of needing evaluators out there who have experience doing advocacy—it is an alignment issue. The goal is a marriage of sound evaluation practice with the current culture of advocacy. We need people who can integrate those two pieces: being an evaluator and being an advocate. The second piece is that advocacy organizations need an increased understanding about the benefits of evaluation. For years it seems evaluation wasn't done because of the nature of advocacy—and advocacy planning focuses on being fluid and nimble. Evaluation has to be demonstrated to be beneficial and to add to the skill-set of staff people— something that helps advocates to work more effectively. It is a limited resource environment. Ultimately, you need to make the case that evaluation is worth it.

InnoNet: Is there anything else you'd like to share about the resources you've used in the Advocacy Evaluation Resource Center?

Rhonda: I am grateful for its presence and I have directed others to it. I think it is a really important, unusual, and much needed online resource. I am excited to see it grow and look forward to supporting it.