Evaluating Your Efforts

It is important to assess your efforts over time. Use the survey and corresponding rubric presented here to gauge your community’s level of awareness and support for student-centered.

It takes time to build genuine public understanding and support for student-centered learning. Think of it as a marathon, instead of a sprint. So how do you know you have been successful? In the communications field, there are four basic kinds of evaluation: formative, process, outcome, and impact.

Formative

What do people think about an issue? Which messages work with what audiences? Who are the best messengers? These are the kinds of questions that are asked in formative evaluation. The research conducted by the FrameWorks Institute about how Americans think about education and learning are formative in nature.

Process

Process evaluation measures the effort and direct outputs of an initiative. How widely and strategically have the materials been distributed? How much media time has been earned? How many people have we reached? Process evaluation is important, but it is not meaningful from a causal point of view.

Outcome

Outcome evaluation measures the effect and changes that result from the work. It assesses outcomes in target communities that come about as a result of the materials, strategies, and activities. We can track attitude or behavior changes, often through pre- and post- tracking surveys, and interim “pulse taking” of communities.

Impact

Impact evaluation measures state-level change or longer-term changes that are the result of an initiative’s effects on attitudes and behavior. Has the work resulted in greater public understanding of an issue? Has it resulted in greater public support?

Where to begin?

As you can probably gather, evaluation of the kinds described above can be costly and time-consuming, so we propose an evaluation framework that we believe will save you time and money. Think of public understanding and support as a five-point continuum illustrated below:

Toolkit_fivepointcontinuum

 

How do you measure where your stakeholders are on this continuum?

This survey was created by the Shaping Our Future Together statewide campaign in Vermont.  The campaign’s purpose is to build public understanding and support for Act 77, a landmark piece of legislation aimed at moving all schools in Vermont toward student-centered approaches to learning.

The survey was designed as a way to gauge the engagement in and support for student-centered learning in specific communities.  A team of youth and adults worked together to collect the data, and then employed the data to inform their public understanding efforts. The survey uses as its foundation the FrameWorks Institute research on the mental and cultural models Americans hold about education and learning. The survey is aligned with the following Public Understanding and Support Assessment Rubric. The numbers directly underneath the Criteria refer to the corresponding questions in the survey.  In other words, they are best used as a matched set.

Only two “criteria” in the survey and rubric are specific to Vermont: Components of Act 77 and Awareness of Act 77 Adoption in Your Community. All of the others are relevant to all communities, so both documents can be easily adapted for use anywhere. The goal is to conduct the survey at key mileposts in your change effort, and to carefully analyze the data so that you can tailor your strategies accordingly. It’s very important to establish a baseline of public understanding right at the beginning of your efforts to build support for student-centered learning.

What differentiates this survey from other surveys is that it aligns with the research on the mental models people carry in their heads about education and learning. In this survey, we are less interested in what people know and more interested in how people think about an issue. This is a fundamental distinction.

Taking the Temperature of the Media Landscape

Often people conduct media analyses to determine how an issue is playing in the newspapers, electronic media, and social media. Here is a tool for Conducting a Media Analysis in your community.

Further Reading

The following reports are good resources to learn more about evaluation of communications campaigns:

Toolkit Overview »