library(tidyverse)library(scales)# The file prolific-agi-2024-results.csv contains responses from a US representative sample of 501 respondents.# Download the file from a public Open Science Framework repository.responses =read_csv("https://osf.io/download/cgx7m/")# Add the Short_Prompt column.responses = responses %>%mutate(Short_Prompt = Prompt) %>%mutate(Short_Prompt =ifelse(grepl("I personally believe it will be possible to build an AGI.", Short_Prompt), "Possible to build", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("If scientists determine AGI can be built, it should be built.", Short_Prompt), "Should be built", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("An AGI should have the same rights as a human being.", Short_Prompt), "Same rights as a human", Short_Prompt))# Inspect Short_Prompt values.print(unique(responses$Short_Prompt))table(responses$Short_Prompt)# Explicitly set types and factor levels.responses$Prompt =factor(responses$Prompt, levels=c("I personally believe it will be possible to build an AGI.", "If scientists determine AGI can be built, it should be built.", "An AGI should have the same rights as a human being."), ordered =FALSE)responses$Sex =as.factor(responses$Sex)responses$Age =as.ordered(responses$Age)# Short_Prompt is a factor. Make 'Possible to build' the reference level.responses$Short_Prompt =factor(responses$Short_Prompt, levels=c("Possible to build", "Should be built", "Same rights as a human"), ordered =FALSE)# Set up some options for the figure.#agi_2024_summary_caption = paste0("US representative sample, N = 501\nResponses collected April 2024\nSource: Thinking Machines, Pondering Humans by Dr. Jason Jeffrey Jones")agi_2024_summary_caption =paste0("Source: Thinking Machines, Pondering Humans by Dr. Jason Jeffrey Jones")agi_2024_summary_colors =c("Possible to build"="#798E87", "Should be built"="#C27D38", "Same rights as a human"="#CCC591")# Summary AGI 2024 figureresponses %>%filter(Year ==2024) %>%group_by(Short_Prompt) %>%summarise(Mean_Response =mean(Response),sd =sd(Response),n =n(),se = sd /sqrt(n),error_low = Mean_Response - se,error_high = Mean_Response + se ) %>%ggplot(aes(x =reorder(Short_Prompt, Mean_Response), y = Mean_Response, color = Short_Prompt, fill = Short_Prompt)) +# Add green and red shading to demarcate agree vs disagree.annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=0.0, ymax=Inf, fill="green", alpha=0.1) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=0.0, fill="red", alpha=0.1) +# Annotations go first, so data elements are layered on top.geom_col() +geom_errorbar(aes(ymin = error_low, ymax = error_high), color="black", width=0.2) +ggtitle("Americans' Attitudes toward Artificial General Intelligence", "April 2024 Representative Sample, N = 501") +xlab("") +ylab("") +# Apply labels with wrapping.scale_x_discrete(labels =label_wrap(10)) +# Set color and fill values.scale_fill_manual(values = agi_2024_summary_colors) +scale_color_manual(values = agi_2024_summary_colors) +# Force y scale to -3 through 3. Put numbers on y-axis. Add low and high labels.scale_y_continuous(limits =c(-3,3), breaks =-3:3, labels =c("-3\nStrongly\ndisagree", "-2", "-1", "0\nNeither agree\nnor disagree", "1", "2", "3\nStrongly\nagree"), expand=expansion(mult =0.025)) +labs(caption = agi_2024_summary_caption) +theme(plot.caption =element_text(size=10, color ="#666666")) +# The legend has only redundant information. Get rid of it.theme(legend.position ="none") +coord_flip()
2024 AGI survey results. On average, Americans believed it was possible to build AGI. They were split on whether AGI should be built. If it were built, they disagreed that AGI should have the same rights as a human. Plotted is mean response on a 7-point Likert scale from Strongly disagree to Strongly agree. Black bars are +/- one standard error.
In April 2024, on average, Americans believed it was possible to build AGI. They were split on whether AGI should be built. If it were built, they did not agree AGI should have the same rights as a human.
Average values like those above are great for quickly and compactly summarizing a group’s opinion. But, an average might obscure how responses are distributed. For example, the mean value at the middle of the scale for ‘Should be built’ above could be due to most respondents being unsure (and choosing ‘Neither agree nor disagree’) or one could find the same average if half the group strongly disagreed while the other half strongly agreed. Let’s not guess; let’s see how the responses were distributed:
Code
# Create a vector of the response labels.responseLabels =c("Strongly disagree","Disagree","Somewhat disagree","Neither agree nor disagree","Somewhat agree","Agree","Strongly agree")responses %>%filter(Year ==2024) %>%ggplot(aes(x = Response, fill = Prompt)) +geom_histogram(binwidth =1, color ="black") +ggtitle("How much do you agree with the statement below?") +xlab("") +ylab("Number of Respondents") +scale_x_continuous(breaks =-3:3, minor_breaks =NULL, labels = responseLabels, guide =guide_axis(angle =45) ) +scale_fill_manual(values =c("I personally believe it will be possible to build an AGI."="#798E87", "If scientists determine AGI can be built, it should be built."="#C27D38", "An AGI should have the same rights as a human being."="#CCC591")) +theme(legend.position ="none") +facet_wrap(~ Prompt, nrow =1, , labeller =label_wrap_gen())
Histogram for the same 2024 AGI survey results as presented above. Most responses were above the midpoint for possible to build, near the midpoint for should be built and below the midpoint for same rights as a human.
Americans were unsure whether AGI should be built. The modal (that is, most frequent) response was ‘Neither agree nor disagree.’ Clearly, agreement and disagreement were stronger for the other items.
2.1.1 Change Over Time
The results above are a snapshot. They capture how Americans felt about AGI at one point in time: April 2024. In isolation, however, they don’t tell you how Americans’ attitudes changed over time or what direction to predict they will move in the future.
We can fix that. Let me show you the temporal trends in these attitudes. I can do that, because this April 2024 survey was the third wave in a set of repeated surveys. In a repeated survey, we ask exactly the same questions at different points in time. In each wave, we recruit a new representative sample (our N = 501 American adults) that stands in for the population (all American adults). Repeated samples are a time- and cost-efficient method to track public opinion.
I presented the three statements above to samples of American adults in 2021, 2023 and 2024. Let’s see - in one figure - how the average response has changed over time.
Code
# Download the 2021 and 2023 data.responses2021 =read_csv("https://osf.io/download/r4xd9/")responses2023 =read_csv("https://osf.io/download/szkuq/")# Select the columns I want.responses2021 = responses2021 %>%select(Prompt, Response, Year, Sex, Age)responses2023 = responses2023 %>%select(Prompt, Response, Year, Sex, Age)# Add Short_Prompt.responses2021 = responses2021 %>%mutate(Short_Prompt = Prompt) %>%mutate(Short_Prompt =ifelse(grepl("I personally believe it will be possible to build an AGI.", Short_Prompt), "Possible to build", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("If scientists determine AGI can be built, it should be built.", Short_Prompt), "Should be built", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("An AGI should have the same rights as a human being.", Short_Prompt), "Same rights as a human", Short_Prompt))responses2023 = responses2023 %>%mutate(Short_Prompt = Prompt) %>%mutate(Short_Prompt =ifelse(grepl("I personally believe it will be possible to build an AGI.", Short_Prompt), "Possible to build", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("If scientists determine AGI can be built, it should be built.", Short_Prompt), "Should be built", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("An AGI should have the same rights as a human being.", Short_Prompt), "Same rights as a human", Short_Prompt))# Explicitly set types and factor levels.responses2021$Prompt =factor(responses2021$Prompt, levels=c("I personally believe it will be possible to build an AGI.", "If scientists determine AGI can be built, it should be built.", "An AGI should have the same rights as a human being."), ordered =FALSE)responses2023$Prompt =factor(responses2023$Prompt, levels=c("I personally believe it will be possible to build an AGI.", "If scientists determine AGI can be built, it should be built.", "An AGI should have the same rights as a human being."), ordered =FALSE)responses2021$Sex =as.factor(responses2021$Sex)responses2023$Sex =as.factor(responses2023$Sex)responses2021$Age =as.ordered(responses2021$Age)responses2023$Age =as.ordered(responses2023$Age)# Short_Prompt is a factor. Make 'Possible to build' the reference level.responses2021$Short_Prompt =factor(responses2021$Short_Prompt, levels=c("Possible to build", "Should be built", "Same rights as a human"), ordered =FALSE)responses2023$Short_Prompt =factor(responses2023$Short_Prompt, levels=c("Possible to build", "Should be built", "Same rights as a human"), ordered =FALSE)# Stack the files.responses =bind_rows(responses, responses2021)responses =bind_rows(responses, responses2023)# Create the temporal trends figureresponses %>%group_by(Year, Short_Prompt) %>%summarise(Mean_Response =mean(Response),sd =sd(Response),n =n(),se = sd /sqrt(n),error_low = Mean_Response - se,error_high = Mean_Response + se ) %>%ggplot(aes(x = Year, y = Mean_Response, color = Short_Prompt, shape = Short_Prompt)) +# Add green and red shading to demarcate agree vs disagree.annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=0.0, ymax=Inf, fill="green", alpha=0.1) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=0.0, fill="red", alpha=0.1) +# Annotations go first, so data elements are layered on top.#geom_point(size=5) +geom_line(aes(color = Short_Prompt), arrow =arrow(angle =10, type ="closed") ) +#geom_errorbar(aes(ymin = error_low, ymax = error_high), color="black", width=0.1, alpha=0.7) +geom_errorbar(aes(ymin = error_low, ymax = error_high), width=0.1, alpha=0.7) +ggtitle("Comparison of Americans' Attitudes toward AGI", "Over three repeated surveys 2021, 2023 and 2024") +xlab("Year") +ylab("") +scale_x_continuous(breaks =2021:2024, minor_breaks =NULL) +scale_y_continuous(limits =c(-2,2), breaks =-2:2, labels =c("Disagree -2", "Somewhat disagree -1", "Neither agree\nnor disagree 0", "Somewhat agree 1", "Agree 2")) +# Keep the legend, but no title.theme(legend.title=element_blank()) +# Set color and fill values.# For this visualization we made #CCC591 25% darker to #b0a655scale_color_manual(values =c("Possible to build"="#798E87", "Should be built"="#C27D38", "Same rights as a human"="#b0a655"))
Agreement that AGI is possible to build has increased. Agreement that AGI should be built and that AGI should have the same rights as a human has decreased.
Code
# For each item, let's estimate the direction of effect over time.fit_possible = responses %>%filter(Short_Prompt =="Possible to build") %>%lm(Response ~ Year, data = .)summary(fit_possible)confint(fit_possible)# Reliably positive.fit_should = responses %>%filter(Short_Prompt =="Should be built") %>%lm(Response ~ Year, data = .)summary(fit_should)confint(fit_should)# Reliably negative.fit_rights = responses %>%filter(Short_Prompt =="Same rights as a human") %>%lm(Response ~ Year, data = .)summary(fit_rights)confint(fit_rights)# Reliably negative.# For each item, is the mean response reliably different 2024 versus 2023?responses %>%filter(Year >2021) %>%filter(Short_Prompt =="Possible to build") %>%t.test(Response ~ Year, data = .)responses %>%filter(Year >2021) %>%filter(Short_Prompt =="Should be built") %>%t.test(Response ~ Year, data = .)responses %>%filter(Year >2021) %>%filter(Short_Prompt =="Same rights as a human") %>%t.test(Response ~ Year, data = .)# Only 'Same rights as a human' passed a p < 0.05 threshold for statistically significant change 2023 to 2024.
From 2021 through 2024, Americans increasingly agreed that AGI was possible to build. This makes sense, given the advances in generative artificial intelligence that occurred and were widely reported during this period.
Americans became less likely to agree that AGI should be built. Recall that the center point of the scale (‘Neither agree nor disagree’) was the most frequently chosen response in 2024.
The greatest movement was clearly in response to the prompt: ‘An AGI should have the same rights as a human being.’ American adults disagreed with this statement - more and more so over these years.
2.2 Survey Items, Respondents and Costs
2.2.1 Survey Items
First, respondents read this definition of AGI: Artificial General Intelligence (AGI) refers to a computer system that could learn to complete any intellectual task that a human being could.
The items were the three statements below. Respondents were asked how much they agreed or disagreed with each statement.
I personally believe it will be possible to build an AGI.
If scientists determine AGI can be built, it should be built.
An AGI should have the same rights as a human being.
The respondents chose among these options:
Strongly disagree
Disagree
Somewhat disagree
Neither agree nor disagree
Somewhat agree
Agree
Strongly agree
2.2.2 Respondents
Respondents were recruited through Prolific Academic. I requested a representative sample of 500 American adults. Specifically, I chose the option “USA, Factors: Sex, Age, Ethnicity (Simplified US Census).”
To demonstrate the demographic coverage, below I provide the Sex and Age crosstab for the 2024 sample:
Code
library(knitr)# Generate percentage per demographic bin for each survey sample.demosTable2024AGI = responses %>%# Use only one row per respondent to count.filter(Short_Prompt =="Possible to build") %>%# Counting subgroups in the 2024 sample.filter(Year ==2024) %>%group_by(Sex, Age) %>%summarise(groupN =n() ) %>%# Add totalN.ungroup() %>%mutate(totalN =sum(groupN) ) %>%# Now we can divide across each row to calculate a percentage.mutate(percent =round(100* groupN / totalN, 0) ) %>%select(-totalN)kable(demosTable2024AGI, format ="markdown")
Sex
Age
groupN
percent
Female
18-24
31
6
Female
25-34
42
8
Female
35-44
42
8
Female
45-54
40
8
Female
55-64
66
13
Female
65+
35
7
Male
18-24
30
6
Male
25-34
46
9
Male
35-44
43
9
Male
45-54
37
7
Male
55-64
55
11
Male
65+
34
7
2.2.3 Costs
The following numbers are for the 2024 wave of the AGI survey.
Each respondent was paid $0.50. Thus, the total of payments to respondents was $250 = 500 * $0.50.
Prolific Academic charged a Service fee equal to 33% of respondent payments. This totaled $83.33.
Prolific Academic charges a special, large Representative sample fee to ensure that your sample more closely matches the US population. For 500 participants, the fee was $704.09.
Thus, the total cost of the 2024 wave was $1,037.42.
There was a clear split in public opinion that widened over time: Americans increasingly believed that Artificial General Intelligence was possible to build, while at the same time they became more opposed to the ideas that AGI should be built or should have the same rights as a human.
Continue on to AGI Results by Subpopulation if you are curious who agreed with which statements. I’ll break down the survey respondents into subsets (by age, for example) and contrast attitudes by group.
Ready for a new topic? Skip to AI Fear Scenarios for the results of a survey experiment contrasting ‘fear scenarios.’ For example, we will see if Americans were more likely to agree with: ‘I fear that Artificial Intelligence will lead to the extinction of human beings.’ or ‘I fear that Artificial Intelligence will lead to me losing my job involuntarily.’
