Session reporter: Julia Coffman, Harvard Family Research Project
Email contact: jcoffman [at] evaluationexchange [dot] org
Link to abstract

Key discussion points:

Advocates are increasingly harnessing the power of technology to achieve policy change and to communicate. Evaluators in the advocacy and policy field are challenged by how to utilize these electronic tools for evaluation purposes—both to assess their use by advocates and to take advantage of them in evaluation methodology.
Modern advocacy and social change efforts recognize the Internet’s value for engaging and mobilizing large numbers of individuals to action. These efforts generate a wealth of data about the effectiveness of Web-based strategies. Following is advice about how to sort out which statistics to pay attention to and how to use them:
- Web analytics tend to fall into two main categories:
  - who is using the website and what they are doing, and
  - how they got to the site.
- Web statistics to assess the “who” and “what” may include visits and visitors, page views, time on site, and that old warhorse: hits. Hits, however, is less useful now because sites are designed so that each time a visitor triggers a request from the server it registers as a “hit.” It makes more sense to look at visits and unique visitors now. Time on site also is now a popular measure. It isn’t advisable to benchmark time on site with other websites.
- Statistics to assess the “how” include referrers, (the external links that users follow to get to the site); search keywords (the words or phrases users typed into search engines to get to the site); visitor information (how many are new to the site, the country or region where they're located, the Web browser they're using, etc.); click paths (graphical representations of typical journeys through the site); and tracking registered users (if parts of the site require users to log in, tracking exactly what those users did during each visit to the site).
The “eNonprofit Benchmarks Study” from M+R Strategic Services and the Advocacy Institute was the first of its kind to look at the overall effectiveness of nonprofits using the Internet to raise money, build e-mail lists, and influence political causes. The study provides an across-the-board look at how well leading American nonprofits are performing online. It was based on an in-depth review of statistics from 15 nonprofit organizations. Key findings include:
- Greater online advocacy results: Organizations generating the most online advocacy actions had several key characteristics in common, including larger e-mail lists; longer-lived online advocacy programs; larger online communications budgets; and sending a higher volume of advocacy e-mail messages.
- Email open rates are declining: Email message open rates averaged 26 percent between September 2004 and September 2005, a decline from the previous 12-month average of 30 percent. Average response rates to email advocacy appeals were 10 percent, while average response rates to email fundraising appeals were just 0.3 percent.
- Online actions speak louder than dollars: Not surprisingly, more email subscribers took online action than made an online donation. Between September 2004 and September 2005, an average of 47 percent of all email subscribers took at least one online action, while just 6 percent of subscribers made an online donation.
- Blogs are a dynamic medium that allow any Internet user to have a voice and become an active information generator rather than a passive consumer. Blog tracking can be an innovative way of assessing the “buzz” of an advocacy or policy issue. Harvard Family Research Project (HFRP) piloted blog tracking as an advocacy evaluation method as part of its evaluation of the Packard Foundation’s Preschool for California’s Children grantmaking program. HFRP found:
  - Blog tracking is similar methodologically to media tracking.
  - Tools for tracking blogs reliably are now emerging, such as Google’s Blog Search (blogsearch.google.com), Technorati (www.technorati.com) and Blog Pulse (www.blogpulse.com).
  - Because bloggers often cover issues that are making print or broadcast news, patterns of blog tracking results will often mirror print media results.
  - Content analysis increases the value of blog tracking, although as is the case with media tracking, it can be extremely time consuming.

Related resources:

<< Back to Highlights from AEA 2007 Annual Conference