CEA's 2019 annual review (appendix)

Posted on Wednesday, April 22nd 2020
(last updated Thursday, January 14th 2021)

Contents

Introduction

This appendix is a less-polished addition to our post about our work in 2019. Please see that post for an overall summary of our year, and this appendix for additional detail.

Program-level progress in 2019

See the table of contents above to jump to details about the programs summarized below.

Direct costs include contractors, but exclude permanent staff salaries.

While we are confident in our overall expense count for 2019 in 2020, we did not have project-level financial codes for our expenses in 2019. As a result, we think this information is directionally correct at the project level, but not hyper-precise. We have project level financial codes for our expenses built into our 2020 system.

Big table 2019

Major events of our year

Larissa Hesketh-Rowe (previous CEO) left CEA in January 2019. Max Dalton was appointed Interim Executive Director, and the trustees assigned Owen Cotton-Barratt, Nick Beckstead, Will Macaskill, Jenna Peters, and Julia Wise to lead a search for a new candidate. The result: Max was appointed Executive Director, and Joan Gass joined as Managing Director. Our main focus from January to October 2019 was improving CEA’s execution of its existing projects, while keeping options open for the next Executive Director (i.e. not making irreversible changes).

Progress since leadership decision

Max Dalton was appointed executive director in October. Shortly after, Joan Gass joined as Managing Director. She reports to Max, and they work together on org-wide priorities as well as divide up responsibilities for managing teams within the organization.

Max and Joan are working with staff and trustees to develop a strategic plan for CEA. One of our main priorities for 2020 will be to improve our strategic plan, and get feedback from stakeholders and community members. We hope to share more later in the year.

During December 2019 and early January 2020, we also pursued and completed a major fundraising round for 2020.

Office changes

Together with the Future of Humanity Institute, the Global Priorities Project, and the Forethought Foundation, CEA will move to a new office in Oxford during 2020 to acquire adequate space after outgrowing our current location. Coordinating with the other organizations and with Oxford University was an ongoing task during 2019.

We have also decided to close our Berkeley office, based on anticipated money and time savings, as well as the increased number of staff based in the UK. We are trying to minimize impacts on current staff by providing assistance to staff who work remotely. We do not expect to lose any staff due to the change.

Project updates from 2019

The following section provides a recap of key activities, outcomes, challenges, opportunities, and case studies (as relevant) for our 11 programs + Operations team. For a summary of this section, please see the summary table above.

How we use Net Promoter Scores

We use Net Promoter Scores (NPS) to evaluate and compare our events. The metric was originally developed for evaluating customer satisfaction. Attendee satisfaction is not the only outcome we care about our events achieving, but we find NPS useful for comparing our events to each other and observing trends.

The score is calculated based on responses to a single question: “How likely is it that you would recommend our company/product/service to a friend or colleague?” The scoring for this answer is most often based on a 0-to-10 scale. The score is then based on how many people answer 9 or 10 (“promoters”), 7 or 8 (“passives”), and 6 or below (“detractors”).

Scores can range from -100 (entirely detractors) to 100 (entirely promoters). Guidance on how to evaluate a score varies, but one large dataset indicates that the median for companies is 44, and the upper quartile is 72 and above.

We also ask attendees of our events about the positive and negative elements of their experience, and about how we could improve. That more nuanced information is a big part of how we decide what changes to make.

Community Health

Summary

Our Community Health team aims to support the EA community and mitigate risks to the community. Key activities include community support (e.g. support to people in mental health crisis, mediating conflicts in the community), monitoring and moderating online groups, monitoring and responding to media inquiries, and supporting diversity and inclusion in EA.

Note that the activities of this team are often confidential, so we are not able to share many cases of impact.

Direct Costs: ~$3,000

Staffing: 1.7 full time equivalents; Sky Mayhew joined Julia Wise on the community health team part-time in January 2019 and came on full-time in May. Sky comes with a background in American Sign Language interpreting and intercultural communication, developing mentorship programs, and implementing diversity & inclusion improvements within training programs.

Community support

In 2019 we responded to 63 community concerns. These range from minor situations (request for help preparing a workshop about self-care at an event) to major ones (request for help working out what to do about serious sexual harassment in a local group).

The most common topics that community members seek advice about are:

  • Handling inappropriate behavior in local groups
  • Supporting community members or org staff members struggling with mental health problems
  • Advising about organizational health (conflicts of interest, or concerns about fair treatment of staff, at EA projects or orgs)
  • Improving diversity and welcomingness in local groups

Community health also supports EA Global (by making admissions decisions, guiding efforts to improve diversity, and making attendees feel welcome); that work is discussed further in the EA Global section.

Improving online discussion

Julia is a moderator of several Facebook groups (the main EA Facebook group, Diversity and Inclusion in EA, EA Parents, and EA Peer Support) and advises moderators of other online groups on handling difficult community situations.

Advising on media coverage of EA

It’s common for local groups, organizations, and individuals in EA to receive interview requests from journalists. They are often not sure whether to accept, or how to clearly and accurately convey key ideas about effective altruism. This year, we ramped up support for the community in this area.

Activities

  • Dedicated more staff time to handling media situations well
  • Julia and Sky received professional training on how to evaluate and prepare for media engagements
  • Consulted with PR advisors for feedback on our advice for responding to journalists, and significantly revised it based on their advice
  • Created a crisis communications plan for identifying, mitigating, or responding to major negative press
  • Designated Sky as a “contact person for media inquires in EA”
  • Wrote a Forum post publicizing the updated resource, emphasizing media awareness and risks, and informing the community of Sky’s role

Impact

  • During 2019, we were involved with 15 actual or possible media stories or other PR situations. For example, we worked with a journalist from a major publication to connect them with knowledgeable EA community members as they were working on a piece about EA. We educated local organizers about best practices around having journalists at events, such as being sure that attendees realize they are speaking to a journalist rather than speaking informally and only realizing later that they may be quoted.
  • We’ve coached community members on conveying their message clearly in interviews, for example before this piece in The Guardian.

Diversity and Inclusion

We would like the EA community to be more diverse and inclusive. In 2019, we began some preliminary work to expand our efforts at improving diversity and inclusion in EA. In 2019, Sky worked with staff to implement some initial changes within existing programs. In 2020, we expect to continue to support these efforts and explore additional avenues for improvement.

In 2019:

  • We published our statement on diversity and inclusion.
  • At EAGx Australia, Julia presented Building a diverse, welcoming, and healthy community.
  • In order to evaluate any future efforts on diversity and inclusion, we needed to first collect baseline data. We began to consistently provide the option across our programs and events for survey respondents to provide more demographic data (such as age, ethnicity, nationality, etc).
  • We piloted a Guides program at EA Global that matched first-time attendees with returning attendees. This program provides support to first-time attendees from many backgrounds, including those from backgrounds underrepresented in EA.

Groups

Summary

The Groups team supports city and university effective altruism groups around the world with advice, resources, and funding. In 2019, we split our time between Community Building Grants (mentioned in the next section), running various projects, and providing general support to EA groups that reached out to us.

Outcomes from general support:

  • The Groups team had mentoring conversations with more than 50 different group organizers, responded to hundreds of requests for help and advice, and funded 80 projects run by groups (such as group retreats and food at events)
  • The Groups team hosted a January retreat for the leaders of US groups. The team also hosted a summer residency program in which group organizers spent time at the CEA office in Oxford.

Direct Costs: $90,000 in basic groups funding + $58,500 for events hosted by CEA

Staffing: 1.3 full-time equivalents (excluding Community Building Grants); Alex Barry full-time, managed by Katie Glass

Activities

Consolidation

Communication

Data & information gathering

  • Collaborated with LEAN on the EA Groups Survey, in order to create an up-to-date list of all groups globally
  • We believe that this survey gathered useful data. We’ve identified opportunities for additional questions that would be helpful in the future across our teams (such as diversity data, information about PR inquiries, and questions related to which materials are highest priority for group leaders)
  • Conducted 21 interviews with group organizers & 8 interviews with national groups

Groups support

  • Katie and Alex had ~250 conversations with group organizers by call and in person.
  • We provided $93,000 in funding across 80 groups that applied for assistance. 53% of the money went to groups in the US, 21% to groups in Europe, 20% to groups in Asia, and 6% to groups in Australia and New Zealand.
  • We responded to ~500 incoming messages.
  • Alex attended the Oxford and Cambridge group retreats and gave a remote presentation at the German EA group organizers' retreat.
  • We hosted a summer residency for 16 group organizers (9 with Community Building Grants, 7 without) to spend one week in the CEA Oxford office over the summer; the residency received a Net Promoter Score (NPS) of 63%.
  • The January US group leaders’ retreat for 30 people had an NPS of 46%. 91% of respondents reported the retreat led to changes to their group plans. It also contributed to two successful Community Building Grant applications.

Challenges and mistakes

  • Lack of Groups Strategy: Due to spending much of the year with an interim executive, we didn’t have the mandate to set a clear groups strategy for 2019, meaning that the strategic advice we could provide to group organizers was limited.
  • Now that CEA has a confirmed executive director, we plan to develop a Groups team strategy based on our organization-wide priorities.
  • In 2019, we had one staff member (Alex) working on groups support plus events like retreats for organizers, meaning that the amount of hands-on support we could provide to all groups was limited. This left some organizers feeling frustrated and neglected.
  • In 2020, we intend to work with a contractor to add capacity on groups support as Alex has left, and to hire an additional staff member.
  • We didn’t reply promptly to some of the enquiries sent to groups@effectivealtruism.org. We’re planning to improve this in 2020 by assigning the contractor mentioned above to manage this inbox.
  • According to LEAN’s March 2019 groups survey, only around half of groups know about CEA’s available resources for groups. We’ve now increased the visibility of these resources in the group organizers’ newsletter and Facebook group.

Community Building Grants (CBGs)

Summary

Our Community Building Grants program provides grants to individuals and groups doing local effective altruism community building. In 2019, we granted a total of $874,962. This includes:

  • $229,270 in new grants for 11 groups or projects to which we have not provided Community Building Grant funding in previous years.
  • $625,067 in renewed grants for 10 groups or projects to which we have provided Community Building Grant funding in previous years.
  • $20,625 in exit grants to 5 groups or projects for which we discontinued our prior funding.

See this spreadsheet for a full list of grants made in 2019.

We began this program in early 2018. The program’s purpose is to strengthen a small number of EA groups by increasing the amount of time dedicated organizers are able to spend on their groups. (See more context from past updates.)

Outcomes: We’re still in the process of determining what outcomes we want to use to assess Community Building Grants. Across several of our grants, we’ve seen an increase in the number of people encountering EA ideas in a high-fidelity way and using these principles to take significant action. Some groups have developed new resources or run events that have received external funding, such as EAGx Nordics and the AI Governance Careers Workshop.

As an illustration of a component of CBG impact, we identified 25 people who we expect to have significant opportunities to do good in the future, and whose EA groups significantly influenced their ability to take these opportunities. Around half of these people have taken roles (including internships, fellowship programs and full-time positions) in organizations that they selected based on EA considerations, or in EA-aligned organizations. We believe that the rest of these people have a strong likelihood of doing so in the next few years. These preliminary case studies have not been thoroughly investigated, and may over- or under-estimate individual impact. Examples of significant group effects include introducing the person to effective altruism, connecting them with job opportunities, influencing their donations to effective charities, and helping them think through a change in their career plans.

Direct Costs: $874,962 in 2019

Staffing: 1.7 full-time equivalents, run by Harri Besceli with support from Katie Glass and Nicole Ross in 2019. Alex Barry ran the CBG retreat in October.

Activities

Assessing applications

Collecting applications

  • New applicants for funding complete a written application or submit a project proposal.
  • Applicants for renewed funding complete an evaluation process which includes surveying their group members, submitting case studies, submitting a membership overview, and writing a general report.

Evaluating applications

  • We have a call with all applicants for funding.
  • We often contact external advisers or other community members for input on the application
  • We sometimes request applicants to submit additional written information (for example, more detail on the project proposal).
  • We assess the suitability of the application for funding, particularly focusing on the quality of the group, the fit of the applicants, and the potential of the group's location.
  • The evaluations are conducted by Harri Besceli (Community Building Grants Specialist), and the evaluation is reviewed by at least one other member of CEA before a decision is confirmed. The evaluation process takes around 5-15 hours per application.
Updates to the process

As we learned more about how the program was going, we updated our application process and evaluation criteria. More detail on these changes is available in the October 2019 CBG post.

The main changes were:

  • We moved to accepting applications on a rolling basis, and to using a more tailored process in evaluating funding applications.
  • We created an expression of interest form applicants could use to receive early-stage input into their plans.
  • The threshold for successful grant applications increased, due to reasons such as wanting to increase the focus on the best grantmaking opportunities and ensuring the programme is sufficiently flexible to leave room for future strategic changes.

You can see past updates on the CBG program here.

Events

In October, we held a 4-day retreat for CBG recipients which had a very high net promoter score of 83%. This involved almost all CBG recipients, three full-time non-CBG organizers, and various staff from EA organizations, for a total of 28 non-staff attendees and 9 staff over a week.

Outcomes of the retreat

  • The retreat was focused on increasing connections between CBG recipients and encouraging them to share advice/knowledge. It seemed to succeed at this, with over 80% of attendees rating it as ‘good’ or ‘excellent’ at achieving these goals.
  • Almost all (22/23 who filled out the survey) of the organizers who attended reported planning to make some kind of change to how they run their group, such as building a database of group members, holding more one-on-one meetings with members, or changing their organizational structure.
  • In terms of connections built, attendees reported (on average) having 10 new people they would be comfortable ask for favors/advice, and 2/3rds reported that they expected to contact other groups more often.
  • EA organization employees said it was valuable to attend, with all rating the retreat at as-good-or-better than the counterfactual use of their time.
  • We are conducting a follow-up survey to assess longer-term impacts, but this is not yet completed.

Challenges and mistakes

Not fulfilling public commitments

  • We haven’t shared a public impact evaluation for CBGs (we previously said in a forum comment that we would do this by August). In November, we posted a notice that we wouldn’t be meeting this commitment.
  • We said in a December 2018 post that we would hold an open CBG round in summer 2019, but we reopened applications in October 2019.

Early rounds were too large We made a large number of grants, particularly in the first round in 2018. This meant that much of the program’s capacity was dedicated to evaluating grant renewals, leaving less capacity for providing support, developing program strategy, and other parts of the program. This likely contributed to other mistakes we made.

Program support We didn’t always provide a good grantee experience. For example, we were not always on schedule in evaluating grant renewals, making decisions on applications, and disbursing funds.

EA Global

Summary

EA Global is the main conference series in effective altruism. The main objectives are to:

  1. Encourage connections, goodwill, and coordination among EAs
  2. Improve EA community culture, and deal with problems such as lack of diversity
  3. Attract resources, including money, talent, and influence, to the EA movement

In 2019, we ran:

  • EA Global San Francisco, with ~500 attendees
  • EA Global London, with ~560 attendees

Key Outcomes

  • Improved network density: EA Global attendees reported having around eight new people that they could reach out to ask for a favor, on average.
  • ~90% of attendees said they felt welcome, ~70% said they felt attendees were humble, and ~70% said they felt attendees were open to exploring new ideas.
  • Of our speakers, a significantly higher portion were women or people of color than are represented in the overall EA population.
  • More than 2000 in-person meetings happened at the conferences.fn-1
  • ~350 attendees reported a minor plan change, and ~50 reported a major plan change (these are self-reports, and should be viewed skeptically).
  • Net Promoter Score (NPS) was roughly in line with previous years (slightly decreased in SF, slightly increased in London - which had the highest NPS of any EA Global yet).

NPS table

Net Cost: ~$270,000 for EA Global London; ~$210,000 for EA Global SF

Costs in 2019 were $390,000 for EA Global London, $330,000 for EA Global SF, and revenue consisted of $120,000 in ticket sales for each. This does not include CEA full-time staff costs (such as for Amy Labenz and full-time event contractors). The direct costs for EA Global are approximately one-third for venue, one-third for food and drink, and one-third for other costs such as audio-visual work, video recordings so talks can be placed online, and speaker travel and lodging..

Staffing: 1.6 full-time equivalent staff; 1.8 full-time equivalent contractors. Run by Amy Labenz (permanent staff), along with Barry Grimes and Kate Hitchcock (contractors, with Barry hired as a staff member in December 2019), with admissions support from Julia Wise and Sky Mayhew. Katie Glass was the project manager for EA Global in 2018, and during Amy’s maternity leave for the first month of 2019.

Objective #1: Encourage connections, goodwill, and coordination among EAs

Data

On average, SF attendees reported 7.2 new people and London attendees reported 9.6 new people they could reach out to ask for a favor as a result of the conference.

Activities

Event App

  • This year, we experimented with two different event apps to encourage one-on-one meetings between participants. ~510 meetings were scheduled via the Grip app in SF, and ~370 meetings and 46 unofficial meetups were scheduled via the Whova app in London.

Fellowship program (since renamed the Stewardship program)

  • The Fellowship program was an updated version of the Meet-ers program which took place in EAG London 2018.
    • This 2018 Meet-ers program served 400 “meet-ees” with 100 Meet-ers. The Meet-ers had a low NPS of -21% due to poor preparation and to the main coordinator falling ill. We thought the program had potential to be a lot better.
  • The Fellowship Program paired people who have similar interests so they could meet at EA Global.
    • We recruited ambassadors who had long experience in effective altruism and/or who worked at EA organizations.
    • Each ambassador met with a small number of fellows (attendees with whom they shared a professional interest, such as biosecurity or operations work).
  • At EA Global San Francisco, we had 10 Ambassadors and 35 Fellows, with an NPS for participating in the program of 43 for Ambassadors and 46 for Fellows. In London, we doubled the size of the program to 23 Ambassadors and 83 Fellows, with an NPS for participating in the program of -8 for Ambassadors and 33 for Fellows. (After getting this low NPS score from Ambassadors, we changed the matching process to give them more input on selecting the Fellows they met with.) We also let some Fellows have meetings with more than one Ambassador, resulting in 111 meetings.
Areas for improvement

Attendee feedback provides us useful information for improvements. These were the main points we’d like to change next time:

  • Attendees found the space for the career fair in London too loud. This made some conversations challenging or unpleasant.
    • We plan to work with the venue to reduce reverberation, and we may rent additional space for the Friday reception in order to allow attendees to spread out more.
  • In London, we allowed people to host unofficial meetups. This seems good from the perspective of having more targeted content and opportunities for 1-1s, but it made quality control more difficult. If we allow people to do this in the future, we need to improve our expectation-setting for the hosts and the participants and better differentiate between official and unofficial meetups.

Objective #2: Improve EA community culture (and improve problems such as a lack of diversity)

We tracked a few components of community culture this year: welcomingness, humility, and openness to different viewpoints.

EA Global culture stats

Welcomingness

Data

  • 87% of attendees felt welcome at EA Global San Francisco; 92% of attendees said they felt welcome at EA Global Londonfn-2

From one attendee’s feedback: “I was warmly greeted by 'big names' in my cause areas — people I have read and watched on YouTube videos — which made me feel like a valued member of the community. This helped me very quickly get over the strong sense of imposter syndrome I had after arriving.”

New initiative: The Guides program

  • This year, we launched a program to help first-time attendees feel more welcome by matching them with returning participants who could serve as a friendly face and help orient them.
  • In San Francisco, we had 20 first-timers and 16 guides; in London (our second test of the program), we had 13 first-timers and 18 guides matched in advance, with additional onsite matches. NPS was 33 for participants in the program at San Francisco, and NPS was 0 for first-timers and 10 for guides in London. Based on this feedback we’ve concluded that the experiment we tried with matching people on-site in London did not work well.

As in previous years, Julia Wise gave a welcome/orientation talk for first-time attendees in both San Francisco and London.

Humility & Open-mindedness

Data

  • 69% of EA Global London attendees (261 people) felt that other attendees were humble. (This question wasn’t asked at EA Global San Francisco.)fn-3
  • 77% of attendees (292 people) felt that other attendees were open to exploring new ideas, as did 69% of attendees at EA Global San Francisco (198 people).fn-4

Initiative

  • We expanded the advisory board for EA Global speakers, who advise us on topics and speakers in different cause areas for the conference series.

Positives

  • One attendee said “All the discussions I had around the posters had people being appropriately calibrated and exploring beliefs.”
  • Another said “I may have gotten lucky, but people seemed to express beliefs more clearly and with better calibration than at other EA events.”
  • Someone else said: “Nearly all conversations beyond a certain length (5 minutes) involve weighing the benefits and risks of one's ideas and approaches.”

Negatives

  • EA org staff member: “I'm trying to remember instances where I witnessed people actually change their mind and am drawing a blank.”
  • Participant: “I thought that some higher-ups in the community had profiles that were quite lacklustre, illustrating their unreachability. Not so humble if you ask me.”
  • Some individuals thought the cost of the food and venue did not reflect EA’s values:
    • EA Global London participant: “Personally, a more humble menu and venue would match our commitment to others (especially to others who are suffering horribly) better. Though I see that the nice menu and venue create a really good atmosphere — which is also helpful.”

Actions/improvements for next year

  • Amy Labenz will continue conducting speaker calls that encourage them to use reasoning transparency in their talks, and to engage with ideas using epistemic humility.
  • We plan to discontinue having sparkling wine at the main event; while the price is similar to that of other beverages, we think it is not worth the risk of signalling the wrong values.
Diversity

See also the section on diversity and inclusion in the Community Health section.

Our goal is to provide an excellent attendee experience for people from a variety of backgrounds. To see if we were achieving that, we gave survey respondents the option to share demographic data.

Data

  • 51% of speakers at EA Global London were women and 36% of speakers at EA Global San Francisco were women (compared to 29% of respondents to the 2018 EA Survey who identified as women).
  • 28% of speakers in London and 25% of speakers in San Francisco were people of color (compared to 22% of EA Survey 2018 respondents who did not identify as white).
  • EAG London was the first event for which we broke down NPS by demographic group. For people who reported demographic information, we were encouraged to see that NPS was as high or higher for attendees from underrepresented groups compared to groups that are overrepresented in EA:

NPS table 2

Initiatives

  • We added new meetups for people from underrepresented groups (e.g. people of color in EA, mid- and late-career professionals in EA, LGBTQ people in EA) in addition to supporting meetups we’ve run historically (e.g. people of faith in EA, women and nonbinary people in EA)
  • Demographic diversity is taken into account as one of many factors in admissions & when selecting meetup hosts.

Positives

  • Some attendees mentioned positive experiences with diversity.
  • “Before I felt cautious about involvement with the EA community. But now I want to get more involved […] I loved that there were so many female speakers. I am female. I was worried the conference would be hostile to me because I had the impression EA might be a community that valued men over women, and that all the speakers might be men. I actually wrote in to the organizers about this before the conference and never heard back, but then they took down the initial speakers page and didn’t put it up again until some women had been added. So I gave the conference a chance. And I’m really glad I did. People were very accepting of me and there were several female speakers.” - Senior staff person at EA-aligned charity [EA Global San Francisco 2019]

Negatives

  • Some attendees mentioned that lack of demographic diversity contributed to a negative experience for them.
    • “As a woman of color, it was really difficult to continually walk into rooms as the only one. Further, it was super disheartening to see lack of diversity at the career fair; the amount of minorities working at this event vs. attending was so disappointing.“ - Participant [EA Global San Francisco 2019]
  • We did not collect demographic data on the EA Global post-event survey until EA Global London 2019, so this is the first time we've evaluated people’s experience at EA Global based on demographic data.

Footnotes

  1. It’s unclear what fraction of these would have happened without EA Global, and we are still investigating this. We have a few heuristics for how many meetings were facilitated: the number of one-on-ones scheduled through the app (370/510 at SF/London), the number of new people each attendee reported feeling like they could reach out to for a favor (7.2/9.6), and the number of attendees multiplied by our estimates of how many meetings each attendee would have. 2,000 meetings seems like a lower bound (which would be about two meetings per attendee).↩
  2. Participants that stated ‘agree’ or ‘strongly agree’ when asked if EA Global was a place where they felt welcome.↩
  3. Attendees that stated “agree” or “strongly agree” when asked “EA Global is a place where individuals express their beliefs with humility”↩
  4. Attendees that stated “agree” or “strongly agree” when asked “EA Global is a place where others are open to exploring ideas that are different than those they already believe”↩