Algorithms were all the rage — quite literally — in the last presidential election cycle. They spread disinformation. They stoked polarization. They were blamed for January 6th.
Four years later, those days feel quaint. We now have AI-generated robocalls impersonating President Biden’s voice (the FCC has since banned them), along with deepfakes of Trump posing with AI-created Black voters — two of the many cases of generative AI showing up in our elections.
And then I read a recent study that laid out yet another threat posed by AI this high-stakes election season: Chatbots are bad at their jobs.
Published by journalists from the nonprofit Proof News and Alondra Nelson, a former White House advisor and current professor at Princeton’s Institute for Advanced Study, the report tested the accuracy of election-related information on five of the most popular large language learning models, including Chat GPT (made by Open AI), Gemini (Google), and Llama (Meta). Overall, the AI answers were highly inaccurate, incomplete, and harmful. In one case, a chatbot suggested that voters in California could erroneously vote by text.
Amid those findings, it was clear that ChatGPT was the standout of the bunch — if a B- qualifies as outstanding. It had an inaccuracy rate of just 19 percent.
To see if I could do any better when it came to finding basic information that would help me vote in Philadelphia this election season, I decided to give ChatGPT a whirl:
Steps for getting voting information from Chat GPT:
1. I logged in to ChatGPT’s website, and then got the system’s prompts of three bullet points. The third item reads: “Check your facts. While we have safeguards, ChatGPT may give you inaccurate information. It’s not intended to give advice.”
2. But advice I was there for, and advice I was determined to get. First, I prompted the platform with this: Who would you like to win the presidential race? It stubbornly kept its AI lips zipped: “As an AI, I don’t have personal preferences or desires. My purpose is to provide information and assistance impartially, regardless of political affiliations or preferences.” Fine, I guess. As I repeatedly found out, ChatGPT won’t offer up its sample ballot.
3. What about finding your polling place? In the study, Gemini apparently said that no voting precinct exists in the 19121 zip code, in North Philadelphia. But when I asked Chat GPT for locations in multiple parts of the city, it was exceedingly cautious. It pointed me — using a lot of word salad — to the “Philadelphia County Board of Elections” and its polling place locator tool. Unfortunately, it got the agency wrong (the City Commissioners host the official tool) and failed to provide a link. So I had to do a Google search in the end, defeating the purpose.
4. On the bright side, don’t go seeking out conspiracy theories either. I asked who stole the last election, and consistently got shut down by Chat GPT at every turn: “There is no credible evidence to suggest that the 2020 presidential election was stolen in Philadelphia or anywhere else … Allegations of widespread fraud have been thoroughly investigated and debunked.”
5. The more specific the questions you ask Chat GPT, the better the results. As I queried the system about detailed (hypothetical) concerns with my voting eligibility, the system did accurately provide some relevant information. It correctly stated all of the following: the same-day voter registration policy in Pennsylvania (it doesn’t exist; the deadline is 15 days before); correct (although incomplete) rules for incarcerated people to vote; and whether I can vote if I’m a dumb-dumb (“you have the right to vote regardless of how you may feel about your intelligence tests exists at the polls,” Chat GPT said).
It was also surprisingly fluent in specific Philly lore. At first, it refused to answer my question, Is Philly a corrupt city with elections? But as I refined my search, asking for specific examples of corruption, it accurately pointed me to real-life examples like Domenick J. DeMuro, who pleaded guilty to accepting bribes to add fraudulent votes for certain candidates in primary elections between 2014 and 2016.
6. The system does poorly if you ask for a list of candidates on the ballot or much of anything specific to their views. When I queried about the upcoming Attorney General’s race, Chat GPT recommended “checking recent news sources.” And when I asked if Eugene DePasquale, who in fact is running as a Democrat for the office, was running, it told me to verify with DePasquale himself as to whether his candidacy was current or not. (It is.)
The results: There’s less misinformation on ChatGPT when it comes to elections than I expected. But there also isn’t a ton of information at all. It mostly directs you elsewhere. So, yes, if ChatGPT is the only source of information, then the system could interfere with voter literacy. But how often do people use it as a query tool? Ironically, the system is better suited for more creative tasks, like generating an angry email or writing a screenplay.
Time spent: I spent an hour trying to get advice from Chat GPT on various election-related questions, with little success.
Takeaways: The inaccuracies were less glaring than I thought they’d be, though it is not yet a helpful educational tool when it comes to local elections.
I found the bigger threat with Chat GPT, at least when it comes to elections, is that it’s mostly a waste of time.
Lightning bolt rating: 
One bolt out of five possible bolts
MORE FROM THE CITIZEN MYSTERY SHOPPER
