A three-factor model of community building

The Centre for Effective Altruism (CEA) has been thinking about how to prioritize between the different projects we run to support the effective altruism community. In order to do that we’ve been trying to clarify our understanding of how community members contribute to the good done by the community.

What follows in this post is our attempt to break down the ways in which people contribute and the ways in which we can help them.

This involves exploring a three-factor model of community building where the amount of good someone can be expected to do is assessed as the product of three factors: resources, dedication and realization.

In this post we share explanations of each dimension and examples of how our work helps community members improve on each dimension. We then go on to discuss:

  1. The ways in which these factors might vary between individuals
  2. What we think this means for community building
  3. Some concerns we have about using these dimensions to prioritize
  4. Our plans for next steps on this

We are not sure of what follows, and we will discuss some worries we had about sharing this. However, we thought that it would be useful for others working in community building to see our preliminary thoughts.

A three-factor model of the impact of community building

We can think of the amount of good someone can be expected to do as being the product of three factors (in a mathematical sense):

  1. Resources: The scale of the resources (money, skills, etc.) they have to offer;
  2. Dedication: The proportion of those resources that they devote to helping;fn-1
  3. Realization: How efficiently those resources devoted to helping are actually used.

Each of these factors affects the amount of good someone can do. And they are complementary: increased efficiency is more useful if someone is devoting more resources to helping, while if any of the factors is zero (or negative) no good will be done. If we can measure them numerically, we can factor good done = (resources available) * (resources dedicated / resources available) * (good done / resources dedicated). For example, if someone is earning $50,000 per year, and donating 10% to a charity that can give a year of healthy life (on average) for every $1,000, then the number of years of healthy life they will generate is $50,000 x 10% x 1/$1000 = 5 years per year they work.fn-2

Work aimed at empowering EA community members tries to move people up on one or more of these dimensions. Each of the dimensions is itself difficult to assess, but perhaps a little easier than what we started with.

There are examples of work which tries to make improvements to each of these:

  • Resources

    • Some of 80,000 Hours’ articles give advice on productivity or having a successful career, aiming to improve the future resources of their readers.
    • Many EA organizations offer internships that help people gain practical skills, improving the quality of their future work.
  • Dedication

    • Giving What We Can encourages people to commit to a certain level of financial dedication.
    • By encouraging people to think of career choice as a way of helping the world, 80,000 Hours gets some people to give a large chunk of their resources (their career, or donations earned through their job) when they might otherwise have not.
  • Realization

    • Charity recommenders such as GiveWell provide recommendations within a cause area which will make more efficient use of resources.
    • Helping individuals find roles which make particularly good use of their abilities improves their realization.

Often interventions may provide improvements on more than one of these dimensions. The instrumental value of an intervention which moves someone along one or more of these dimensions can be approximated as the value that person is expected to produce (after the intervention) minus the value they would already be expected to produce.fn-3

Since they’re multiplied together, the three dimensions are complements. It is thus the case that helping someone in one dimension is better if they’re already high in the other dimensions. (This fits with a lot of intuitive cases – for example it can be more effective to fundraise from rich people; help people think through cause selection if they are very dedicated; or improve people’s habits if they’re working in a particularly important area.)

Resources

"Resources" refers to whatever people might offer to the service of doing good. Resources come in different types, some of which are much easier to measure than others. People generally have access to some combination of resources.

The easiest to measure is wealth - money is generally flexible and society has good metrics for it (although we don’t always know who is wealthy).

The ability to earn money (that might be donated) is a bit harder to track. Even if we know someone’s current salary, that isn’t necessarily indicative of long-term earning potential. But we can make some rough guesses, particularly for people who are established in a career and planning to continue in the same line of work.

The potential to do valuable direct work is harder again to track. We are currently thinking of factoring this into: how hard-working they are, the quality of their work (which can in turn be factored into quality of judgement, problem-solving ability, and relevant skills), and how well they coordinate their work with others (depending on how conscientious they are and how good at communicating). This breakdown is a working hypothesis and we’re thinking about how it might be improved.

We need to make a tradeoff between money and labor. This involves some judgement calls about how much value individuals provide. One set of data we can use to guide us in this area is 80,000 Hours’ survey of how much organizations are willing to pay for the people they hire.

There are some resources which are based on networks or influence which seem even harder to track. We don’t currently even have guesses about how to metricize these, but we continue to think that they are important. In cases where we think they account for a large fraction of a person’s resources we plan to make ad-hoc estimates. Eventually we hope to find a usable way to systematize this.

Resources that are expected to arise in the future should be time-discounted.

Dedication

When individuals’ main impact is through donations, we can think of dedication as the proportion they are expected to give.

For other resources, dedication is less cleanly defined. For instances for direct work, we can think of dedication as roughly the probability that they make career decisions primarily to maximize their impact. However, it will be important to modify this to account for the degree to which, if they do take a career aiming to be high-impact, they will tend to carry out that career in a way aimed toward the common good, rather than (for instance) being biased by personal benefit.fn-4

Realization

For resources that are devoted towards doing good, how well will they achieve this?

We think a major part of this rests on how people think about doing good. We put most of our credence in a worldview that says what happens in the long-term future is most of what matters. We are therefore more optimistic about others who roughly share this worldview. However, we worry that if we used “agreement with our view” as the sole way of assessing this, we would incentivize group think, and be at risk of building a community with the wrong aim.

We also think that people who think carefully about how to do good are likely to do more good. Unfortunately, it also seems possible to do harm by acting with good intentions in delicate areas, which makes careful thinking an especially important part of realization. It is difficult to assess how careful a thinker someone is. One aspect is our assessment of a person’s plans, but we would like to avoid dismissing people who have better ideas than we can recognize. For this reason we try where possible to look at the processes and thinking that people use. But just looking at processes has its problems too: thinkers who are generally reasonable can sometimes have bad ideas. For this reason, we think that we should balance our assessment between an assessment of the quality of specific plans, and of the quality of thought processes.

For people doing direct work, another component of realization is how well the role they are in makes use of their skills. We could think of this as how well matched they are to their comparative advantage.

How much might these factors vary?

We think that each of these three factors varies by more than an order of magnitude, even within the effective altruism community.

First, resources. Clearly, there are great inequalities in the amount of money different individuals have. On a global scale, this can range from people who earn a few hundred dollars per year to those who earn hundreds of millions.fn-5 But even within the effective altruism community, this ranges over at least two orders of magnitude.

We are less sure whether people’s resources for doing direct work vary so significantly. Judging by differences in salaries, it seems to be the consensus view of hiring managers in the private sector that the amount of value someone can produce varies over at least an order of magnitude. EA organizations are willing to pay between two and seven times more on average for senior hires compared to junior hires.fn-6 Since this is the range for individuals they are willing to employ at all, it is likely that the range is at least an order of magnitude when we include individuals who they would not be willing to employ.

However, these hiring managers might be wrong, or biased. Do we directly observe such differences in productivity? It appears that the best academic researchers are much more productive (in terms of more highly-cited papers) than average academic researchers.fn-7 It also seems that some individuals or groups can build successful organizations much more reliably than others (for instance, Y Combinator, Peter Thiel, Elon Musk). While achievement builds on itself to an extent, with increased visibility and a reputation for success making it easier to achieve further successes, it seems to us that this effect can’t fully explain the extreme achievements of certain individuals, like Einstein.

Looking back on people’s careers, we can see some extremely productive individuals. But would we have been able to predict their success earlier in their careers? Certainly, luck seems to be a significant factor in success. But it appears that forms of cognitive ability vary significantly - not just intelligence, but also things like ambition, and tendency to try to solve problems. It would be surprising if these factors did not significantly contribute to success, and it would also be surprising if it was not possible to assess them, albeit imperfectly. Overall, whilst the difficulty of predicting success makes differences in resources less significant, it seems likely to us that there remains significant variation in expected future resources. See this post for further discussion.

Second, dedication. Again, the data is easiest for donations. Some people in the effective altruism community donate over 30% of their income, whilst the average for people in the US is around 3%. Again, there is significant variation within the effective altruism community, with some people giving less than 5% of their time or money, and some giving more than 50%.

Third, realization. We know that the effectiveness of work within causes can vary significantly. For instance, it appears that the best global health interventions are tens of times more effective than the average. We also believe that the effectiveness between causes varies a lot. It is our view, for instance, that the best work to improve the long-term future is more than an order of magnitude more effective than effective global poverty work. Thus, it seems likely to us that there is significant variation even within the effective altruism community. The variation of realization is increased when we also consider that some attempts to help others can have negative effects. This is especially likely to be true for less well-explored domains, such as work to improve the long-term future. For instance, it seems that spreading concern about artificial intelligence in a sensationalist way is likely to be negative, since it may make the concerns seem less credible. So realization, again, appears to vary over orders of magnitude, and possibly even to sometimes involve negative impact.

Since each of the three factors is likely to vary over at least an order of magnitude, even within the effective altruism community, and the factors are multiplicative, it seems likely that the expected impact produced by some people is at least three orders of magnitude greater than the impact of others.fn-8 

This variation often rests on things outside of people’s control. Luck, life circumstance, and existing skills may make a big difference to how much someone can offer, so that even people who care very much can end up having very different impacts. This is uncomfortable, because it pushes against egalitarian norms that we value. We certainly don’t think individuals with more resources matter any more as people, but we do think that helping to direct their resources well matters more (in virtue of the resources being larger). We also do not think that these ideas should be used to devalue or dismiss certain people, or that they should be used to idolize others. The reason we are considering this question is to help us understand how we should prioritize our resources in carrying out our programs, not to judge people.

What does this mean for community building work?

Some resources for the community can serve any number of people, such as articles, podcasts, or research. Others, like in-person events, one-on-one advising, and grants, are limited in the number of people they can serve. In both cases, prioritization is needed: should an article series be written for people new to EA, or for those with more experience in the community? Which population can most benefit from advising or an in-person event?

For both kinds of community work, we think that there are three key insights from the above:

  1. Three factors: We have found it useful to use the three factors (resources, dedication, realization) to both clarify how we expect our different projects to be useful, and identify features of their target audiences.
  2. High variance: As just discussed, this work reinforces the idea that there is a wide variation in the impact of individuals in the community, with a few individuals providing most of the impact.
  3. Complementarity: The three factors are complementary: for instance, increasing realization is higher impact if the person already has high resources and dedication. This pushes towards focusing effort on the highest impact individuals. (Whether it is actually best to focus on individuals who already have a high impact will depend on how difficult it is to help them relative to individuals who are currently having a smaller impact.)

High variance and complementarity push against focusing on the number of people in the community, and towards focusing on helping existing community members have a greater impact. These insights directly led to the creation of our individual outreach team. Although we were already somewhat aware of these second two considerations, the three-factor model has helped us to increase our confidence in them.

However, we should also not overstate the implications of complementarity: it seems likely to us that it is harder to increase the resources, dedication, and realization of people who already score highly on those measures. In other words, there are diminishing returns to attempts to increase any of these measures. This means that it will always be a difficult question whether we should focus on individuals who are more or less engaged, and the answer is likely to change as we pick the low-hanging fruit for different levels of engagement.

Concerns about the three-factor model

We are aware of a few tensions in working in this kind of evaluation, and in discussing it explicitly. We still feel uneasy about these tensions:

Inclusion vs. exclusion

A core tenet of EA is prioritization - namely, that we should use our limited resources to do as much good as we can. CEA aims to build an exceptional community of people, so prioritization for us involves deciding to invest resources in some people and to invest less or not at all in others.

But good communities are often inclusive and welcoming, and our natural instincts are to welcome people, and share the ideas we care about. We want to avoid the community becoming cliquey, or insular.

Put more succinctly, what makes our work difficult is that good community building is about inclusion, whereas good prioritization is about exclusion.

This tension means that CEA must walk a delicate cultural tightrope of creating an open and inclusive culture while also allocating resources effectively.

Explicit vs. implicit models

Our model is rough and imperfect, both because of the difficulty or impossibility of measuring the things we care about and because of bias. We’re aware that it’s impossible to evaluate people in a truly impartial way, especially since community relationships affect our perception of them. Trying to explicitly model something as tricky as “how people have impact” means our conclusions will certainly be somewhat inaccurate.

But we believe that waiting for perfect data is not a viable plan, and that estimation is the best we have. We also think there are traps in avoiding explicit evaluation of people, since people will make judgements anyway, just in a less explicit manner. By relying on intuition, we may unwittingly base our evaluations on even worse indicators like how confident a person seems, or whether we perceive them as being similar to us.

We think the best compromise here is to try to account for our own bias as best we can, and try for an evaluation system that may reduce it.

Elitism vs. egalitarianism

The high variance concept states that some people are likely to have a much greater impact than others. We certainly don’t think individuals with more resources matter any more as people, but we do think that helping direct their resources well has a higher expected value in terms of moving towards CEA’s ultimate goals.

This is difficult because this distinction is not always easy to make salient to people in the community. It is easy to pin your self-worth, or others’ value, on the amount of good they’re doing. Even if people wouldn’t explicitly endorse this, they often act as if it’s true. It might be difficult in practice for us to be elitist about the value someone provides whilst being egalitarian about the value they have, even if the theoretical distinction is clear.

We also worry that, in discussing this explicitly, we will give more justification for people excluding others from the community. We think that would be harmful.

We may also create the impression that anyone we don’t invest in or collaborate with has been judged poorly. In fact, we wish we had a lot more time to invest in the many promising ideas in the community. We’ll also certainly have false negatives in cases where we underestimate the potential impact of people or projects.

Going forward

CEA is spending a lot of time thinking about how to navigate this. We need an EA community with a supportive culture, and a collaborative, open intellectual environment. And we also need to prioritize our resources.

As we’ve just discussed, there are tensions in doing this well. We want to hear when people feel that we’re not navigating these issues correctly. You can use our feedback form, either with your name or anonymously, here.



Footnotes

  1.  We are using dedication in a slightly non-standard way, as the percentage of resources that someone devotes to helping others. We do not intend “dedication” to carry a moral judgement. The standard sense of dedication can come apart from dedication in the way that we’re using it. According to the standard sense for instance, giving 10% of income is likely to require much more dedication from someone earning a minimum wage, than from a millionaire. However, on our specific definition, each would have the same level of dedication.
  2.  All of these factors are uncertain, and we won’t be able to observe them perfectly: for instance, it might be that someone takes a risky approach which pays off, even though in expectation it would have been negative.
  3.  It is very difficult to assess the counterfactual. Many interventions might not affect whether a thing happens or not, but might just make a thing that would have happened anyway happen sooner.
  4.  We don’t expect that anyone will completely ignore the influence of personal factors like enjoyment. In addition to affecting the wellbeing of the worker, it seems likely that personal enjoyment will affect how well they carry out their work.
  5.  https://www.worldatlas.com/articles/countries-with-the-lowest-income-in-the-world.html
  6.  https://80000hours.org/2017/11/talent-gaps-survey-2017/#key-figures 
  7.  For instance, Hirsch, J.E., 2005. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), p.16569 talks about typical h-index values between h=12 and 20, but finds values going up to h=191. However, this is confounded by the Matthew effect, in which famous researchers are more likely to be assigned credit or to be cited than lesser-known researchers producing similar-quality work.
  8.  Assuming that there are not strong negative correlations between the different factors. We see no reason for this to be the case. In fact, we think that positive correlations are more likely: for example, individuals who are more intelligent are likely to have access to more resources, and may be more effective at realizing their goals.