library(tidyverse)library(scales)# The file prolific-agi-2024-results.csv contains responses from a US representative sample of 501 respondents.# Download the file from a public Open Science Framework repository.responses =read_csv("https://osf.io/download/cgx7m/")# Add the Short_Prompt column.responses = responses %>%mutate(Short_Prompt = Prompt) %>%mutate(Short_Prompt =ifelse(grepl("I personally believe it will be possible to build an AGI.", Short_Prompt), "Possible to build", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("If scientists determine AGI can be built, it should be built.", Short_Prompt), "Should be built", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("An AGI should have the same rights as a human being.", Short_Prompt), "Same rights as a human", Short_Prompt))# Inspect Short_Prompt values.print(unique(responses$Short_Prompt))table(responses$Short_Prompt)# Explicitly set types and factor levels.responses$Prompt =factor(responses$Prompt, levels=c("I personally believe it will be possible to build an AGI.", "If scientists determine AGI can be built, it should be built.", "An AGI should have the same rights as a human being."), ordered =FALSE)responses$Sex =as.factor(responses$Sex)responses$Age =as.ordered(responses$Age)# Short_Prompt is a factor. Make 'Possible to build' the reference level.responses$Short_Prompt =factor(responses$Short_Prompt, levels=c("Possible to build", "Should be built", "Same rights as a human"), ordered =FALSE)# Set up some options for the figure.#agi_2024_summary_caption = paste0("US representative sample, N = 501\nResponses collected April 2024\nSource: Thinking Machines, Pondering Humans by Dr. Jason Jeffrey Jones")agi_2024_summary_caption =paste0("Source: Thinking Machines, Pondering Humans by Dr. Jason Jeffrey Jones")agi_2024_summary_colors =c("Possible to build"="#798E87", "Should be built"="#C27D38", "Same rights as a human"="#CCC591")# Summary AGI 2024 figureresponses %>%filter(Year ==2024) %>%group_by(Short_Prompt) %>%summarise(Mean_Response =mean(Response),sd =sd(Response),n =n(),se = sd /sqrt(n),error_low = Mean_Response - se,error_high = Mean_Response + se ) %>%ggplot(aes(x =reorder(Short_Prompt, Mean_Response), y = Mean_Response, color = Short_Prompt, fill = Short_Prompt)) +# Add green and red shading to demarcate agree vs disagree.annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=0.0, ymax=Inf, fill="green", alpha=0.1) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=0.0, fill="red", alpha=0.1) +# Annotations go first, so data elements are layered on top.geom_col() +geom_errorbar(aes(ymin = error_low, ymax = error_high), color="black", width=0.2) +ggtitle("Americans' Attitudes toward Artificial General Intelligence", "April 2024 Representative Sample, N = 501") +xlab("") +ylab("") +# Apply labels with wrapping.scale_x_discrete(labels =label_wrap(10)) +# Set color and fill values.scale_fill_manual(values = agi_2024_summary_colors) +scale_color_manual(values = agi_2024_summary_colors) +# Force y scale to -3 through 3. Put numbers on y-axis. Add low and high labels.scale_y_continuous(limits =c(-3,3), breaks =-3:3, labels =c("-3\nStrongly\ndisagree", "-2", "-1", "0\nNeither agree\nnor disagree", "1", "2", "3\nStrongly\nagree"), expand=expansion(mult =0.025)) +labs(caption = agi_2024_summary_caption) +theme(plot.caption =element_text(size=10, color ="#666666")) +# The legend has only redundant information. Get rid of it.theme(legend.position ="none") +coord_flip()
2024 AGI survey results. On average, Americans believed it was possible to build AGI. They were split on whether AGI should be built. If it were built, they disagreed that AGI should have the same rights as a human. Plotted is mean response on a 7-point Likert scale from Strongly disagree to Strongly agree. Black bars are +/- one standard error.
In April 2024, on average, Americans believed it was possible to build AGI. They were split on whether AGI should be built. If it were built, they did not agree AGI should have the same rights as a human.
Average values like those above are great for quickly and compactly summarizing a group’s opinion. But, an average might obscure how responses are distributed. For example, the mean value at the middle of the scale for ‘Should be built’ above could be due to most respondents being unsure (and choosing ‘Neither agree nor disagree’) or one could find the same average if half the group strongly disagreed while the other half strongly agreed. Let’s not guess; let’s see how the responses were distributed:
Code
# Create a vector of the response labels.responseLabels =c("Strongly disagree","Disagree","Somewhat disagree","Neither agree nor disagree","Somewhat agree","Agree","Strongly agree")responses %>%filter(Year ==2024) %>%ggplot(aes(x = Response, fill = Prompt)) +geom_histogram(binwidth =1, color ="black") +ggtitle("How much do you agree with the statement below?") +xlab("") +ylab("Number of Respondents") +scale_x_continuous(breaks =-3:3, minor_breaks =NULL, labels = responseLabels, guide =guide_axis(angle =45) ) +scale_fill_manual(values =c("I personally believe it will be possible to build an AGI."="#798E87", "If scientists determine AGI can be built, it should be built."="#C27D38", "An AGI should have the same rights as a human being."="#CCC591")) +theme(legend.position ="none") +facet_wrap(~ Prompt, nrow =1, , labeller =label_wrap_gen())
Histogram for the same 2024 AGI survey results as presented above. Most responses were above the midpoint for possible to build, near the midpoint for should be built and below the midpoint for same rights as a human.
Americans were unsure whether AGI should be built. The modal (that is, most frequent) response was ‘Neither agree nor disagree.’ Clearly, agreement and disagreement were stronger for the other items.
2.1.1 Change Over Time
The results above are a snapshot. They capture how Americans felt about AGI at one point in time: April 2024. In isolation, however, they don’t tell you how Americans’ attitudes changed over time or what direction to predict they will move in the future.
We can fix that. Let me show you the temporal trends in these attitudes. I can do that, because this April 2024 survey was the third wave in a set of repeated surveys. In a repeated survey, we ask exactly the same questions at different points in time. In each wave, we recruit a new representative sample (our N = 500 American adults) that stands in for the population (all American adults). Repeated samples are a time- and cost-efficient method to track public opinion.
I presented the three statements above to samples of American adults in 2021, 2023 and 2024. Let’s see - in one figure - how the average response has changed over time.
Code
# Download the 2021 and 2023 data.responses2021 =read_csv("https://osf.io/download/r4xd9/")responses2023 =read_csv("https://osf.io/download/szkuq/")# Select the columns I want.responses2021 = responses2021 %>%select(Prompt, Response, Year, Sex, Age)responses2023 = responses2023 %>%select(Prompt, Response, Year, Sex, Age)# Add Short_Prompt.responses2021 = responses2021 %>%mutate(Short_Prompt = Prompt) %>%mutate(Short_Prompt =ifelse(grepl("I personally believe it will be possible to build an AGI.", Short_Prompt), "Possible to build", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("If scientists determine AGI can be built, it should be built.", Short_Prompt), "Should be built", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("An AGI should have the same rights as a human being.", Short_Prompt), "Same rights as a human", Short_Prompt))responses2023 = responses2023 %>%mutate(Short_Prompt = Prompt) %>%mutate(Short_Prompt =ifelse(grepl("I personally believe it will be possible to build an AGI.", Short_Prompt), "Possible to build", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("If scientists determine AGI can be built, it should be built.", Short_Prompt), "Should be built", Short_Prompt)) %>%mutate(Short_Prompt =ifelse(grepl("An AGI should have the same rights as a human being.", Short_Prompt), "Same rights as a human", Short_Prompt))# Explicitly set types and factor levels.responses2021$Prompt =factor(responses2021$Prompt, levels=c("I personally believe it will be possible to build an AGI.", "If scientists determine AGI can be built, it should be built.", "An AGI should have the same rights as a human being."), ordered =FALSE)responses2023$Prompt =factor(responses2023$Prompt, levels=c("I personally believe it will be possible to build an AGI.", "If scientists determine AGI can be built, it should be built.", "An AGI should have the same rights as a human being."), ordered =FALSE)responses2021$Sex =as.factor(responses2021$Sex)responses2023$Sex =as.factor(responses2023$Sex)responses2021$Age =as.ordered(responses2021$Age)responses2023$Age =as.ordered(responses2023$Age)# Short_Prompt is a factor. Make 'Possible to build' the reference level.responses2021$Short_Prompt =factor(responses2021$Short_Prompt, levels=c("Possible to build", "Should be built", "Same rights as a human"), ordered =FALSE)responses2023$Short_Prompt =factor(responses2023$Short_Prompt, levels=c("Possible to build", "Should be built", "Same rights as a human"), ordered =FALSE)# Stack the files.responses =bind_rows(responses, responses2021)responses =bind_rows(responses, responses2023)# Create the temporal trends figureresponses %>%group_by(Year, Short_Prompt) %>%summarise(Mean_Response =mean(Response),sd =sd(Response),n =n(),se = sd /sqrt(n),error_low = Mean_Response - se,error_high = Mean_Response + se ) %>%ggplot(aes(x = Year, y = Mean_Response, color = Short_Prompt, shape = Short_Prompt)) +# Add green and red shading to demarcate agree vs disagree.annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=0.0, ymax=Inf, fill="green", alpha=0.1) +annotate(geom="rect", xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=0.0, fill="red", alpha=0.1) +# Annotations go first, so data elements are layered on top.#geom_point(size=5) +geom_line(aes(color = Short_Prompt), arrow =arrow(angle =10, type ="closed") ) +#geom_errorbar(aes(ymin = error_low, ymax = error_high), color="black", width=0.1, alpha=0.7) +geom_errorbar(aes(ymin = error_low, ymax = error_high), width=0.1, alpha=0.7) +ggtitle("Comparison of Americans' Attitudes toward AGI", "Over three repeated surveys 2021, 2023 and 2024") +xlab("Year") +ylab("") +scale_x_continuous(breaks =2021:2024, minor_breaks =NULL) +scale_y_continuous(limits =c(-2,2), breaks =-2:2, labels =c("Disagree -2", "Somewhat disagree -1", "Neither agree\nnor disagree 0", "Somewhat agree 1", "Agree 2")) +# Keep the legend, but no title.theme(legend.title=element_blank()) +# Set color and fill values.# For this visualization we made #CCC591 25% darker to #b0a655scale_color_manual(values =c("Possible to build"="#798E87", "Should be built"="#C27D38", "Same rights as a human"="#b0a655"))
Agreement that AGI is possible to build has increased. Agreement that AGI should be built and that AGI should have the same rights as a human has decreased.
Code
# For each item, let's estimate the direction of effect over time.fit_possible = responses %>%filter(Short_Prompt =="Possible to build") %>%lm(Response ~ Year, data = .)summary(fit_possible)confint(fit_possible)# Reliably positive.fit_should = responses %>%filter(Short_Prompt =="Should be built") %>%lm(Response ~ Year, data = .)summary(fit_should)confint(fit_should)# Reliably negative.fit_rights = responses %>%filter(Short_Prompt =="Same rights as a human") %>%lm(Response ~ Year, data = .)summary(fit_rights)confint(fit_rights)# Reliably negative.# For each item, is the mean response reliably different 2024 versus 2023?responses %>%filter(Year >2021) %>%filter(Short_Prompt =="Possible to build") %>%t.test(Response ~ Year, data = .)responses %>%filter(Year >2021) %>%filter(Short_Prompt =="Should be built") %>%t.test(Response ~ Year, data = .)responses %>%filter(Year >2021) %>%filter(Short_Prompt =="Same rights as a human") %>%t.test(Response ~ Year, data = .)# Only 'Same rights as a human' passed a p < 0.05 threshold for statistically significant change 2023 to 2024.
From 2021 through 2024, Americans increasingly agreed that AGI was possible to build. This makes sense, given the advances in generative artificial intelligence that occurred and were widely reported during this period.
Americans became less likely to agree that AGI should be built. Recall that the center point of the scale (‘Neither agree nor disagree’) was the most frequently chosen response in 2024.
The greatest movement was clearly in response to the prompt: ‘An AGI should have the same rights as a human being.’ American adults disagreed with this statement - more and more so over these years.
There was a clear split in public opinion that widened over time: Americans increasingly believed that Artificial General Intelligence was possible to build, while at the same time they became more opposed to the ideas that AGI should be built or should have the same rights as a human.
Continue on to Chapter TODO-link if you are curious who agreed with which statements. I’ll break down the survey respondents into subsets (by age, for example) and contrast attitudes by group.
Ready for a new topic? Skip to Chapter TODO-link for the results of a survey experiment contrasting ‘fear scenarios.’ For example, we will see if Americans were more likely to agree with: ‘I fear that Artificial Intelligence will lead to the extinction of human beings.’ or ‘I fear that Artificial Intelligence will lead to me losing my job involuntarily.’
