Cost-Effectiveness
In order to maximize the impact of donations, we recommend cost-effective programs, often measured in terms of the number of additional voters cast ballots (for nonpartisan programs) or the net Democratic votes produced per $1,000 (for programs that are partisan.)
We make cost-effectiveness estimates for our top recommendations because programs can have a wide range of costs.
Our Cost Effectiveness Approach
To estimate the cost-effectiveness of a program, we perform our own analysis.
Whenever possible, the cost-effectiveness estimates are based on randomized controlled trials (RCTs), the gold standard in determining causal inference. In most cases, the organizations we consider have conducted RCTs of their own programs. When an organization under consideration has not conducted its own RCT, we review other organizations’ RCTs showing the effects of similar tactics.
We rely on organizations’ providing us with their study design, methodology, raw data, cost of implementing the program, and results from randomized control trials (RCTs). Our evaluation team reviews the methodology and assumptions, and in most cases, we perform our own analysis of the raw data to verify results.
From this analysis, we estimate the cost per net Democratic vote for each program under consideration and we recommend the most cost-effective programs that can achieve scale in our key competitive races.
Our cost-effectiveness estimates typically assess a program’s cost per vote for a single election or net Democratic vote over multiple election cycles. We also quantify the number of races, up and down the ballot, on which a program may have an effect.
When the differences in estimated cost-effectiveness are not significant, we also consider factors such as confidence in an organization's track record, capability of their leadership, and the strength of the evidence supporting a particular election program.
After each election cycle, organizations provide us with data on the results of their RCTs, and we compare our projections to the actual cost of the program per net vote.
The Value of Testing
Testing not only provides us with confidence that programs will produce net votes, it also helps organizations improve their programs.
Some organizations have the ability to test within the election cycle to determine which messages and tactics persuade voters to turn out for a particular candidate.
By elevating the best-performing tactics and dropping the worst, their programs maximize the impact of donations.
Our Metrics Compared to Figures Many Organizations Report
Focus for Democracy reports on the unique and discrete contributions of each program we recommend for an election.
Some organizations overstate a program’s impact by not accounting for the contributions of other groups or factors. For example, an organization might claim, “Because of our work, voter turnout increased by 10% in the county.” However, they might neglect to mention other organizations that also encouraged voters to get to the polls in the county. Or they may have overlooked other factors like heightened public interest in the election that could have accounted for some of the gains.
It sometimes can be shown that if an organization had been absent, the election result would have been different—for example, if an organization’s program produced more votes than the margin by which the race was decided. That’s an important fact to report. Still, the work of other organizations and the candidate's campaign made the race close enough to be able to provide the winning margin.
Organizations often report the wrong metrics. For example, they might report the outputs of their program such as the number of mailers sent, digital ads produced, doors knocked on, phone calls made, and texts sent to persuade voters to cast a ballot.
Although many of these outputs are necessary to help win an election, Focus for Democracy’s analysis goes a critical step further, measuring the outcomes–the impact of election tactics.
For nonpartisan efforts, that means the number of people that voted who otherwise would not have done so.
For partisan organizations or evaluators, it means net Democratic votes.
We then account for the cost of using the tactic to provide donors with information on the scale of votes produced and the number of votes produced by a given contribution (votes per $1,000).