How did opinions toward artificial general intelligence (AGI) vary among different sets of people?
Before you look at the results, try to predict where (and if) we will see big differences. Will it be an individual’s sex or age that best predicts their opinion? Or will it be how trusting or willing to take risks an individual is? Maybe political party affiliation will be the schism which separates opinions.
Read on to find out!
3.1 Sex
Respondents self-reported Sex by choosing Female or Male. As you can see below, both sexes followed the same trajectory over time.
Code
library(tidyverse)library(scales)# The file prolific-agi-2024-results.csv contains responses from a US representative sample of 501 respondents.# Download the file from a public Open Science Framework repository.responses =read_csv("https://osf.io/download/cgx7m/")# Download the 2021 and 2023 data.responses2021 =read_csv("https://osf.io/download/r4xd9/")responses2023 =read_csv("https://osf.io/download/szkuq/")# Select the columns I want.responses2021 = responses2021 %>%select(Prompt, Response, Year, Sex, Age)responses2023 = responses2023 %>%select(Prompt, Response, Year, Sex, Age)# Stack the files.responses =bind_rows(responses, responses2021)responses =bind_rows(responses, responses2023)# Add the Short_Prompt column.responses = responses %>%mutate(Short_Prompt = Prompt) %>%mutate(Short_Prompt =ifelse(grepl("I personally believe it will be possible to build an AGI.", Short_Prompt), "Possible to build", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("If scientists determine AGI can be built, it should be built.", Short_Prompt), "Should be built", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("An AGI should have the same rights as a human being.", Short_Prompt), "Same rights as a human", Short_Prompt))# Explicitly set types and factor levels.responses$Prompt =factor(responses$Prompt, levels=c("I personally believe it will be possible to build an AGI.", "If scientists determine AGI can be built, it should be built.", "An AGI should have the same rights as a human being."), ordered =FALSE)responses$Sex =as.factor(responses$Sex)responses$Age =as.ordered(responses$Age)# Short_Prompt is a factor. Make 'Possible to build' the reference level.responses$Short_Prompt =factor(responses$Short_Prompt, levels=c("Possible to build", "Should be built", "Same rights as a human"), ordered =FALSE)# Create a temporal trends by Sex figureresponses %>%group_by(Short_Prompt, Year, Sex) %>%summarise(Mean_Response =mean(Response),sd =sd(Response),n =n(),se = sd /sqrt(n),error_low = Mean_Response - se,error_high = Mean_Response + se ) %>%ggplot(aes(x = Year, y = Mean_Response, shape=Sex, color=Sex, group=Sex)) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=0.0, ymax=Inf, fill="green", alpha=0.1) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=0.0, fill="red", alpha=0.1) +geom_pointrange(aes(ymin = error_low, ymax = error_high), position =position_dodge(width =0.02)) +geom_line(position =position_dodge(width =0.02)) +scale_x_continuous(breaks =2021:2024, minor_breaks =NULL, guide =guide_axis(angle =45)) +#scale_y_continuous(limits = c(-1.75,1.6), breaks = -1:1, labels = c("Somewhat disagree -1", "Neither agree nor disagree 0", "Somewhat agree 1")) +facet_wrap(~ Short_Prompt, nrow=1) +# Set color and fill values.scale_color_manual(values =c("Female"="#1FC3AA", "Male"="#8624F5")) +ggtitle("Estimates of attitudes toward Artificial General Intelligence (AGI)", "Separated by self-report Sex") +xlab("Year") +ylab("")
Estimates of AGI attitudes by Sex over time based on three representative sample surveys of American adults. Note that 0 on the y-axis represents ‘Neither agree nor disagree’. +1 corresponds to ‘Somewhat agree’, and -2 corresponds to ‘Disagree’. Refer to the previous chapter for aggregate results.
Code
# For each item, let's estimate the direction of effect over Year and Sex.fit_possible = responses %>%# Make 2021 year zero, so that the model isn't estimating early Christians' attitudes for the intercept.mutate(Year = Year -2021) %>%filter(Short_Prompt =="Possible to build") %>%lm(Response ~ Year * Sex, data = .)summary(fit_possible)confint(fit_possible)# See text for interpretation.fit_should = responses %>%# Make 2021 year zero, so that the model isn't estimating early Christians' attitudes for the intercept.mutate(Year = Year -2021) %>%filter(Short_Prompt =="Should be built") %>%lm(Response ~ Year * Sex, data = .)summary(fit_should)confint(fit_should)# See text for interpretation.fit_rights = responses %>%# Make 2021 year zero, so that the model isn't estimating early Christians' attitudes for the intercept.mutate(Year = Year -2021) %>%filter(Short_Prompt =="Same rights as a human") %>%lm(Response ~ Year * Sex, data = .)summary(fit_rights)confint(fit_rights)# See text for interpretation.
Further analysis supports the following claims:
On average, Males more strongly agreed building AGI was possible.
On average, Females agreed more weakly that AGI should be built.
On average, neither Males nor Females ever agreed that an AGI should have the same rights as a human. The slight difference due to Sex shrank from 2021 to 2024.
3.2 Age
Respondents self-reported their age in years. To reduce complexity, I bucketed ages by decades. Look at the graphs below, and focus on the movement of young people over time. The darker the line, the more recent the survey. Age is on the x-axis (younger to older).
Code
# Create a temporal trends by Age figureresponses %>%mutate(Year =as.character(Year)) %>%group_by(Short_Prompt, Year, Age) %>%summarise(Mean_Response =mean(Response),sd =sd(Response),n =n(),se = sd /sqrt(n),error_low = Mean_Response - se,error_high = Mean_Response + se ) %>%ggplot(aes(x = Age, y = Mean_Response, color=Year, group=Year)) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=0.0, ymax=Inf, fill="green", alpha=0.1) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=0.0, fill="red", alpha=0.1) +geom_pointrange(aes(ymin = error_low, ymax = error_high), position =position_dodge(width =0.25)) +geom_line(linetype ="dashed", position =position_dodge(width =0.25)) +scale_x_discrete(guide =guide_axis(angle =45) ) +#scale_y_continuous(limits = c(-2.25,1.6), breaks = -2:1, labels = c("Disagree -2", "Somewhat disagree -1", "Neither agree nor disagree 0", "Somewhat agree 1")) +facet_wrap(~ Short_Prompt, nrow=1) +# Set color and fill values.scale_color_manual(values =c("2021"="gray65", "2023"="gray40", "2024"="black")) +ggtitle("Estimates of attitudes toward Artificial General Intelligence (AGI)", "Separated by Age Range") +xlab("Age Range") +ylab("") +labs(caption ="Source: Thinking Machines, Pondering Humans by Dr. Jason Jeffrey Jones") +theme(plot.caption =element_text(size=10, color ="#666666"))
Estimates of AGI attitudes by Age over time based on three representative sample surveys of American adults. Note that 0 on the y-axis represents ‘Neither agree nor disagree’. +1 corresponds to ‘Somewhat agree’, and -2 corresponds to ‘Disagree’. Refer to the previous chapter for aggregate results.
Code
# For each item, let's estimate the direction of effect over Year and Sex.fit_possible_age = responses %>%# Make 2021 year zero, so that the model isn't estimating early Christians' attitudes for the intercept.mutate(Year = Year -2021) %>%filter(Short_Prompt =="Possible to build") %>%lm(Response ~ Year * Age, data = .)summary(fit_possible_age)confint(fit_possible_age)# See text for interpretation.fit_should_age = responses %>%# Make 2021 year zero, so that the model isn't estimating early Christians' attitudes for the intercept.mutate(Year = Year -2021) %>%filter(Short_Prompt =="Should be built") %>%lm(Response ~ Year * Age, data = .)summary(fit_should_age)confint(fit_should_age)# See text for interpretation.fit_rights_age = responses %>%# Make 2021 year zero, so that the model isn't estimating early Christians' attitudes for the intercept.mutate(Year = Year -2021) %>%filter(Short_Prompt =="Same rights as a human") %>%lm(Response ~ Year * Age, data = .)summary(fit_rights_age)confint(fit_rights_age)# See text for interpretation.
I know I said I had reduced complexity, but the figure above is still too busy. Let’s simplify by dichotomizing age into two buckets. We will call ages 18-44 Younger and 45 and above Older. With only two categories, we can gain some statistical power and inspect simpler plots similar to the one above for Sex and the one below for Trust.
Code
# Create a temporal trends by Oldness figureresponses %>%mutate(Oldness =ifelse(Age %in%c("18-24", "25-34", "35-44"), "Younger", "Older") ) %>%group_by(Short_Prompt, Year, Oldness) %>%summarise(Mean_Response =mean(Response),sd =sd(Response),n =n(),se = sd /sqrt(n),error_low = Mean_Response - se,error_high = Mean_Response + se ) %>%ggplot(aes(x = Year, y = Mean_Response, shape=Oldness, color=Oldness, group=Oldness)) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=0.0, ymax=Inf, fill="green", alpha=0.1) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=0.0, fill="red", alpha=0.1) +geom_pointrange(aes(ymin = error_low, ymax = error_high), position =position_dodge(width =0.02)) +geom_line(position =position_dodge(width =0.02)) +scale_x_continuous(breaks =2021:2024, minor_breaks =NULL, guide =guide_axis(angle =45)) +facet_wrap(~ Short_Prompt, nrow=1) +# Set color and fill values.scale_color_manual(values =c("Younger"="#D95F02", "Older"="#E7298A")) +ggtitle("Estimates of attitudes toward Artificial General Intelligence (AGI)", "Separated by Dichotomous Age") +xlab("Year") +ylab("") +labs(caption ="Source: Thinking Machines, Pondering Humans by Dr. Jason Jeffrey Jones") +theme(plot.caption =element_text(size=10, color ="#666666"))
Replotting Age over Time with only two categories: Younger is 18-44 while Older is 45 and above.
Code
# For each item, let's estimate the direction of effect over Year and Oldness.fit_possible_oldness = responses %>%# Make 2021 year zero, so that the model isn't estimating early Christians' attitudes for the intercept.mutate(Year = Year -2021) %>%mutate(Oldness =ifelse(Age %in%c("18-24", "25-34", "35-44"), "Younger", "Older") ) %>%filter(Short_Prompt =="Possible to build") %>%lm(Response ~ Year * Oldness, data = .)summary(fit_possible_oldness)confint(fit_possible_oldness)# See text for interpretation.fit_should_oldness = responses %>%# Make 2021 year zero, so that the model isn't estimating early Christians' attitudes for the intercept.mutate(Year = Year -2021) %>%mutate(Oldness =ifelse(Age %in%c("18-24", "25-34", "35-44"), "Younger", "Older") ) %>%filter(Short_Prompt =="Should be built") %>%lm(Response ~ Year * Oldness, data = .)summary(fit_should_oldness)confint(fit_should_oldness)# See text for interpretation.fit_rights_oldness = responses %>%# Make 2021 year zero, so that the model isn't estimating early Christians' attitudes for the intercept.mutate(Year = Year -2021) %>%mutate(Oldness =ifelse(Age %in%c("18-24", "25-34", "35-44"), "Younger", "Older") ) %>%filter(Short_Prompt =="Same rights as a human") %>%lm(Response ~ Year * Oldness, data = .)summary(fit_rights_oldness)confint(fit_rights_oldness)# See text for interpretation.
However we slice it, age is not actually moving the needle very much. As we saw for Males and Females above, Older and Younger Americans are more similar than not in their attitudes toward AGI.
There is an interesting (statistically significant, but just barely) crossover interaction of age and time for Should be built. Alarm bells should be (faintly) ringing for AGI proponents; it may be that younger people are crossing over into opposition of developing this technology.
In the 2024 survey, I presented respondents additional items after the AGI attitude items. Each was a standard measure from previous academic work. The first was a single-item measure of generalized trust.
3.3 Trust
Some people are more trusting than others. As social scientists, we call this trait Generalized trust. It has been measured many ways, but my favorite is very simple; it is one item with two possible responses:
Generally speaking, would you say that most people can be trusted or that you can’t be too careful in dealing with people?
For now, grant me that we can measure individuals’ willingness to trust others and that forcing a choice between the two poles is a valid, interesting method. We will call our respondents Trusting if they selected ‘Most people can be trusted.’ Those respondents who chose ‘You can’t be too careful in dealing with people,’ we will call Careful.
Note that we’ll look at results for 2024, because I did not include the Trust item in previous years.
Let’s look at some results!
Code
# The file agi-2024-wide-correlates.csv contains responses from a 2024 US representative sample of 501 respondents.# This file also contains demographics and responses to Trust, Risk and Political Party affiliation items.# Download the file from a public Open Science Framework repository.responsesWideCorrelates =read_csv("https://osf.io/download/jsc4n/")responsesWideCorrelates = responsesWideCorrelates %>%mutate(Generalized_Trust =ifelse(Generalized_Trust =="Most people can be trusted.", "Trusting", "Careful") ) %>%mutate(Generalized_Trust =factor(Generalized_Trust, levels =c("Careful", "Trusting")) )# View counts and proportions.proportions = responsesWideCorrelates %>%count(Generalized_Trust) %>%mutate(Proportion = n /sum(n)) %>%mutate(Generalized_Trust =factor(Generalized_Trust, levels =c("Trusting", "Careful"))) %>%# Add text annotation to replace Legend. Include label \n count.mutate(Bar_Label =paste0(Generalized_Trust, "\n", "n = ", n) )#proportionsproportions %>%ggplot( aes(x = Proportion, y ="", fill = Generalized_Trust)) +geom_bar(stat ="identity", width =0.1, show.legend =FALSE ) +ggtitle("Proportion of Generalized Trust Responses") +xlab("Proportion") +ylab(NULL) +scale_x_continuous(labels = scales::percent, expand =expansion(0,0) ) +scale_y_discrete(, expand =expansion(0,0)) +scale_fill_manual(values =c("Careful"="#0047AB", "Trusting"="darkorange4")) +geom_text(aes(label = Bar_Label), position =position_fill(vjust =0.5), size =24/.pt)
More respondents were ‘Careful’ (307) than ‘Trusting’ (194).
We see that more respondents were Careful (307) than Trusting (194).
Now let’s see what difference generalized trust made. (I only included the generalized trust item on the most recent wave of the survey; the following results are for 2024.)
Code
# Pivot longer and add the Short_Prompt column.generalizedTrust = responsesWideCorrelates %>%select(Generalized_Trust, Possible, Should, Rights) %>%pivot_longer(cols =c("Possible", "Should", "Rights"), names_to ="Short_Prompt", values_to ="Response" ) %>%mutate(Short_Prompt =ifelse(Short_Prompt =="Possible", "Possible to build", Short_Prompt) ) %>%mutate(Short_Prompt =ifelse(Short_Prompt =="Should", "Should be built", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(Short_Prompt =="Rights", "Same rights as a human", Short_Prompt)) %>%mutate(Short_Prompt =factor(Short_Prompt, levels=c("Possible to build", "Should be built", "Same rights as a human"), ordered =FALSE) )# Create a temporal trends by Sex figuregeneralizedTrust %>%group_by(Short_Prompt, Generalized_Trust) %>%summarise(Mean_Response =mean(Response),sd =sd(Response),n =n(),se = sd /sqrt(n),error_low = Mean_Response - se,error_high = Mean_Response + se ) %>%ggplot(aes(x = Generalized_Trust, y = Mean_Response, shape=Generalized_Trust, color=Generalized_Trust, group=Generalized_Trust)) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=0.0, ymax=Inf, fill="green", alpha=0.1) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=0.0, fill="red", alpha=0.1) +geom_pointrange(aes(ymin = error_low, ymax = error_high), linewidth =1 ) +#scale_y_continuous(limits = c(-1.75,1.6), breaks = -1:1, labels = c("Somewhat disagree -1", "Neither agree nor disagree 0", "Somewhat agree 1")) +scale_color_manual(values =c("Careful"="#0047AB", "Trusting"="darkorange4")) +ggtitle("Estimates of attitudes toward Artificial General Intelligence (AGI)", "Separated by self-report Generalized Trust") +xlab("") +ylab("") +theme(legend.position ="none") +facet_wrap(~ Short_Prompt, nrow=1)
Generalized trust category as predictor for AGI attitudes. Careful and Trusting individuals responded similarly - except for the item ‘Should be built.’ Trusting respondents were more likely to agree that AGI should be built (p < 0.05).
Code
# For each item, let's estimate the association with Generalized Trust.fit_possible = generalizedTrust %>%filter(Short_Prompt =="Possible to build") %>%lm(Response ~ Generalized_Trust, data = .)summary(fit_possible)confint(fit_possible)# See text for interpretation.fit_should = generalizedTrust %>%filter(Short_Prompt =="Should be built") %>%lm(Response ~ Generalized_Trust, data = .)summary(fit_should)confint(fit_should)# See text for interpretation.fit_rights = generalizedTrust %>%filter(Short_Prompt =="Same rights as a human") %>%lm(Response ~ Generalized_Trust, data = .)summary(fit_rights)confint(fit_rights)# See text for interpretation.
Careful and Trusting individuals responded similarly - except for the item Should be built. Trusting respondents agreed more strongly that AGI should be built (p < 0.05).
3.4 Risk
TODO risk analysis
How do you see yourself: are you generally a person who is fully prepared to take risks or do you try to avoid taking risks? Please choose a number, where the value 0 means: ‘not at all willing to take risks’ and the value 10 means: ‘very willing to take risks’.
