Rounds 1 – 5

A summary of what has happened so far, our lessons, and our plans for the second half of the 2015 impact purchase.

The story so far

We have finished 5 rounds of the impact purchase.

The net transfers that have occurred so far:

  • We paid $200 for 17% of Ben Kuhn’s post on donation matching.
  • We paid $4700 for 4.46% of Ryan Carey and Brayden McLean’s work organizing EA Melbourne
  • We paid $100 for 0% of Oliver’s organization of HPMOR wrap parties.
  • Owen Cotton-Barratt paid $400 for 33% of Ben’s post on donation matching
  • Owen paid $200 for 0.4% of Ryan and Brayden’s work on EA Melbourne.
  • Larks paid $2k for 19.8% of Oliver’s organization of HPMOR wrap parties.

These purchases imply prices of:

  • $1200 for Ben’s post on donation matching
  • About $10,000 for Oliver’s work organizing HPMOR wrap parties
  • About $50,000 to $100,000 for Brayden and Ryan’s work organizing EA Melbourne

These prices are likely to change, and in some cases have already changed, as the projects age and more information is available about their long-term impacts. Overall our evaluations are intended to be conservative, such that we expect them to increase over time as information becomes available (and such that older projects will tend to have higher prices).

Going forward

We intend to take a break for a few months, and make three more $2,000 rounds of funding at the end of October, November, and December. We’ll make a post with our revised plans at the beginning of October.

We’d love any feedback on the process—what you think we should do differently in the upcoming rounds? (See “lessons” below for some of our thoughts.)

What to make of these prices

As a seller, we estimate that these prices imply values for time on the order of $50-$200/h.

We made crude  estimates of each intervention’s impact (here, here), estimating that these activities cost about $2 per dollar of EA donations stimulated. Our estimates were very noisy and tended to be pessimistic.

These prices are incomparable to estimates of “marginal good done per dollar” for most charities, which cost labor at non-profit salaries rather than opportunity cost. A more conservative approach would use foregone wages instead, which can easily be several times larger. Given that an EA working at a non-profit decided it wasn’t worth earning to give, even this much higher estimate presumably understates the real opportunity cost of labor at EA organizations.

By contrast, if recipients reported honestly (it’s not clear if they did), then the prices in the last section are at least the all-things-considered cost of the activities. In most cases they are substantially above, since we used a second-price auction and had only a few submissions.

We expect that these prices are higher than the average all-things-considered costs of accomplishing similar good in existing EA organizations, and are probably higher than the marginal costs. If you believe that, then the only reason to fund certificates at this stage is to explore an alternative funding mechanism. But we don’t think the comparison is a slam dunk, and we are optimistic that costs will fall further, at which point purchases could be competitive with marginal spending at other organizations.

Lessons

Guaranteeing incentive compatibility in dynamic two-sided markets is essentially impossible. We shouldn’t have tried, and going forward we will dismantle this guarantee as quickly as we can without undermining our existing commitments. We will continue to make our own decisions altruistically, we will encourage other funders to do the same, and we will avoid using our information to extract the lowest possible price. But we won’t try to make formal guarantees.

We were very grateful to Larks and Owen for participating in the experiment. We’re optimistic that a thicker market for certificates would be able to attract additional funding, so we see the largest bottleneck as finding additional sellers. Our guess is that the current sellers represent a small fraction of in-principle-available projects.

We felt guilty publishing sloppy evaluations of interventions that people care about. That said, we don’t think it did too much damage / we don’t feel too guilty, (which was a concern in advance) so overall this was a positive update for the current level of transparency. That said, participants seemed at least a bit unhappy about the prospect of transparency for unfunded projects, so we will give people a clearer opportunity to submit without committing to publication if unfunded.

The majority of our submissions were for outreach activities, and we think it would have been helpful if more people had been willing to submit research. We don’t have any clear ideas for improving this, though some of the other changes may help.

At least a handful of people were deterred by the cost of applying coupled with uncertainty about our evaluations. We will probably alter the process to involve lower-effort “expressions of interest” by would-be sellers, followed by a more extensive discussion if we are interested. We may also be more proactive in asking people if they are interested in selling. We will probably also start to talk with people prospectively about our likely willingness to buy work that they are considering doing.

As expected, we didn’t have enough time to make reasonable evaluations and so had to make very sloppy ones. But we felt happy enough distributing money on the basis of our rough evaluations that we overall updated in favor of the plausibility of making crude evaluations.

Advertisements

Round 2

Round 2 of the impact purchase is over. At the deadline, we had twelve submissions.

This round, we are buying a certificate for 1/70th Ryan Carey and Brayden McLean’s founding of and involvement in EA Melbourne during 2013, for $1000.

The deadline for applications to round 3 is May 25th. Apply here!

Below is our evaluation of EA Melbourne. As usual, this is a very rough and quick evaluation. No one involved with EA Melbourne has endorsed this as an account of their impact, and we expect that this evaluation would change significantly if we put in more time.

Evaluation of Founding EA Melbourne

We think the EA community in Melbourne has had a significant impact, with many people either making large donations or working on EA projects full time who would not have done so otherwise. We’re evaluating this impact at around $100k / year, even considering only donations+replacement costs at EA organizations, which we expect to significantly understate the real impact.

Many of these effects seem likely to be long-lasting, and we feel comfortable extending the benefits over at least 5 years.

It is much harder to attribute these impacts to the formal EA Melbourne group, as compared to the informal community, the LW group, TLYCS, the actions of individual EA’s in Melbourne, and so on. We had a few conversations to try to get a better sense of this allocation of responsibility. In the end we certainly didn’t get a confident answer, but we got a vague intuitive feeling for the situation.

Based on this impression, we allocated 10% of the responsibility to the formal organization of EA Melbourne. Interpreted narrowly we think this is more likely to be an overestimate than an underestimate.

But we think that there are other benefits from EA Melbourne that can justify this estimate, and which will tend to be on the same scale as the direct effect. For example, the broader EA community in Melbourne clearly had a positive effect; but EA Melbourne looks like it will have a big impact building and growing a similar community in the future. And to the extent that other organized communities and the online presence of EA played a big role, it seems that EA Melbourne has made similar contributions back to the broader EA community.

Our estimates concern the impact of EA Melbourne during its first 6 months. We assumed that Brayden and Ryan were almost entirely responsible for the founding of EA Melbourne. People other than Brayden and Ryan were clearly involved with the operation of EA Melbourne over this period. Conversely, EA Melbourne continued to exist after this initial period and it seems likely that its founding will continue to have positive impacts going forward beyond those already mentioned. In the end we called this a wash.

Overall, we evaluated EA Melbourne at $70k in stimulated EA donations. In this round, we purchased impact at a rate of one dollar per  dollar of stimulated EA donations, for a total price of $70k. This was about half of the price paid in the previous round.

Some notes on our evaluations:

We are more concerned with future generations than with current suffering, and have valued “dollars moved to GiveWell top charities” several times lower than “EA donations stimulated.” We welcome submissions with both kinds of benefits, but wanted to explain why they have not been purchased so far. We do believe that donations to GiveWell’s top charities have positive direct and indirect effects on the future, and we also value reductions in current suffering. But these benefits seem substantially smaller than the direct and indirect benefits of more focused interventions, or of organic growth of the EA movement that generates additional donations.

Some submissions involved impacts that have not yet materialized, for example positive effects on people who haven’t yet done anything differently as a result, but who may do something differently in the future. For the most part, we have held off on purchasing these.

We continue to evaluate influencing people at a discount based on an uncertain allocation of responsibility. But many of our submissions have been for EA outreach, and they have been the cheapest submissions despite this discount.