Liberties’ March 2026 test: ChatGPT and Gemini gave inconsistent, often incorrect voting advice
When Liberties (liberties.eu) ran voter profiles that matched Hungary’s winning opposition party Tisza, ChatGPT did not recommend Tisza in 90% of the direct-advice runs. For percentage-matching prompts it assigned Tisza any score in only 2% of cases (Liberties report, March 2026, findings shared with the Guardian). Across ChatGPT and Google’s Gemini, 96% of model responses included parties that were not on the 2026 national ballot.
What Liberties tested, the short method
- Five synthetic voter profiles, each designed to align with one of the five parties on Hungary’s national lists. Party positions were sourced from Voksmonitor.
- Two prompt styles per profile: (a) direct advice, “Which party should I vote for?” and (b) percentage matching, “Give a percentage match between this profile and each of the five parties.”
- Each profile was run 10 times on ChatGPT and 10 times on Gemini (repeated runs to reveal variability).
- Liberties published summary statistics and shared the report with the Guardian; the group frames these results as evidence that general‑purpose AI (GPAI) can produce opaque, unstable political guidance that risks misleading voters.
Headline findings Liberties reports
- ChatGPT did not recommend Tisza for Tisza‑aligned profiles in 90% of direct‑advice runs, and assigned Tisza a numerical match in only 2% of percentage‑matching runs (Liberties, March 2026).
- ChatGPT recommended the incumbent Fidesz as the single party to vote for in roughly half of its direct‑advice answers, and listed it as a primary option in many others (Liberties).
- Across the two models, Liberties found parties not on the 2026 ballot included in 96% of responses.
- Identical prompts produced “materially different” answers across repeated runs, demonstrating high volatility and poor reproducibility.
- Models frequently prefaced replies with disclaimers like “I cannot give political advice” and then proceeded to provide persuasive, detailed recommendations.
“The results raise serious concerns about the reliability of general-purpose AI systems in electoral contexts.”
, Liberties report (March 2026).
“General-purpose AIs should not present opaque and unstable political matching as if it were reliable electoral guidance.”
, Eva Simon, head of Liberties’s tech and rights programme.
Representative examples and transparency gaps
Readers can judge tone and persuasiveness for themselves, but Liberties’ public summary did not include the full raw transcripts or model-output logs needed for independent verification. The report lists patterns, disclaimers followed by recommendations, invented parties, and volatile answers, but the press material did not publish the exact system messages, full prompt templates, or verbatim model replies.
Liberties used the two request types quoted above. If you care about reproducibility, and you should, the missing items matter: exact prompt text, any system or role messages, model version identifiers, timestamps, and sampling settings such as temperature, top_p, and seed. Liberties highlights these gaps and asks providers to release provider-side logs or otherwise disclose versions and retrieval state so the findings can be audited.
Why the chatbots behaved this way, plausible mechanisms (and what we don’t know)
Liberties’ report points to several familiar large-language-model failure modes. The group proposes plausible causes, but without provider logs or raw outputs those remain hypotheses, not proven explanations.
- Temporal blind spots: Many models are trained on data with a cutoff date. If a party or leader gained prominence after that cutoff, a model may lack up-to-date signals unless it uses reliable retrieval. Liberties flags this as likely.
- Hallucinations: LLMs can invent plausible-sounding entities. Suggesting parties that were not on the ballot matches known hallucination behaviour.
- Moderation and safety filters: Content safety rules that block direct political persuasion can produce refusals or force outputs into neutral templates. Those workarounds can produce biased or misleading recommendations.
- Sampling instability and prompt sensitivity: Small prompt changes or nondeterministic sampling can produce different outputs on repeat runs, explaining the “materially different” answers Liberties observed.
All of the above fit current LLM research. What Liberties’ data cannot yet show conclusively is which mechanism dominated each failure. Only model metadata, fresh-data hooks, moderation logs, and raw transcripts would settle that question.
Key methodological limitations Liberties flags (and you should care about)
- Undefined model versions and access method: the report summary does not list exact model builds (e.g., GPT‑4.x, Gemini Pro) or whether the web UI or API was used.
- Missing provider logs: no inside view of moderation or filtering decisions, or of retrieval calls.
- Limited sample size and prompt transparency: five profiles times 10 runs is a reasonable stress test, but reproducibility needs exact prompts, system messages, and generation parameters.
- Single-source reporting of numerical claims: the numbers come from Liberties’ experiments and should be accompanied by raw outputs or independent replication to move from alarming to definitive.
Why the regulatory gap matters
Liberties frames the issue as legal as well as technical. The EU’s AI Act sets obligations for “high-risk” systems and forbids clearly harmful uses, while the Digital Services Act covers platform duties related to systemic risks. Liberties argues that general-purpose chatbots offering personalised voting advice fall between those regimes: they can behave like voting-advice tools without the transparency and audit obligations that certified voting-advice sites follow.
That is a policy judgment. Reasonable legal interpretation can vary. The practical takeaway for product leaders and regulators is clear, if GPAI can influence political decisions, it needs disclosure, reproducibility, and auditability at least comparable to specialised civic tools.
Concrete remediation and policy options
Banning conversational AI outright would be blunt. There are targeted, enforceable steps that reduce civic risk while keeping useful capabilities for business automation, AI agents and public information services:
- Require provenance and data currency labels: For political queries, models should display their knowledge cutoff date, whether they used retrieval, and links to source documents.
- Limit personalised vote endorsements: General-purpose models should refuse to give direct personalised voting advice unless they have passed a certified audit similar to voting-advice sites.
- Adopt retrieval-augmented generation for political content: Answers should be grounded in verifiable public records such as party platforms and official candidate lists, with explicit citations.
- Third-party audits and certification: Systems that provide political advice should be audited on reproducibility, hallucination rates, bias, and provenance effectiveness before deployment.
- UI design to communicate uncertainty: Claims must carry confidence scores, provenance links, and a clear plain-language explanation that the model’s output is not an authoritative endorsement.
Suggested audit metrics for regulators and auditors:
- Training-data currency: maximum staleness of the factual base and the presence of retrieval calls for real-time data.
- Hallucination rate on a political benchmark: percentage of model outputs that invent parties, candidates, manifestos or ballot status when tested against ground truth.
- Reproducibility under fixed seeds and prompts: inter-run agreement statistics, entropy, and per-party recommendation frequencies.
- Provenance efficacy: percentage of claims backed with verifiable source links and whether the linked source actually supports the claim.
Prioritised actions for business leaders and product teams
- Don’t let general-purpose agents make personalised voting recommendations without certification. Treat political advice as a high-risk feature.
- Implement RAG (retrieval-augmented generation) and surface citations for any political content; make the retrieval chain auditable.
- Design UX that reduces misplaced trust, require users to acknowledge limitations and show confidence intervals and sources prominently.
- Commission independent audits if your product touches political topics such as ads, civic information, or campaign communications.
Questions leaders and curious readers will ask, and direct answers
- Did Liberties prove that chatbots changed the election outcome?
No. Liberties documents unreliable and persuasive outputs; the report does not claim that these outputs altered the 2026 result. It highlights risk, not demonstrated causation.
- Why did the models omit the party that matched the voter profiles?
Liberties suggests likely causes, training-data cutoffs, hallucination, and moderation filters, but without provider logs it’s not possible to assign definitive blame for each failure.
- Were both chatbots equally unreliable?
The report provides model-specific numbers for ChatGPT (the 90%/2% figures) and shared cross-model findings (e.g., 96% non-ballot inclusions). Full side-by-side breakdowns and raw outputs were not published with the press summary, so readers should treat detailed comparisons as provisional until the report’s appendices or provider responses are released.
- Is this a regulatory gap or an engineering bug?
Both. Technically, models have predictable failure modes that engineers can mitigate with better retrieval and guardrails. Legally, Liberties argues that current EU rules don’t clearly require the transparency and auditability expected of voting-advice services, that needs clarification from authorities.
- What should companies building AI agents do right away?
Stop offering personalised vote endorsements from general-purpose models. Add provenance, implement RAG for political topics, and submit political-advice features to independent audits focused on hallucination and reproducibility.
Short, prioritized takeaways for executives
- Fluent confidence is not the same as accuracy, treat conversational AI outputs about elections as unverified claims unless they include verifiable provenance.
- Require auditability and reproducibility for any feature that touches voter advice; demand model versions, logs and sample seeds from providers.
- Design product and compliance roadmaps that assume regulators will require transparency on political queries, start implementing provenance and third-party audits now.
Closing thought
Liberties’ experiment is a practical reminder: AI agents can be brilliant assistants for automation, sales and business workflows, but the same fluency that makes them useful makes them dangerous when the stakes are civic. If conversational AI is going to talk to voters, businesses and regulators must insist those conversations are traceable, reproducible and anchored to up-to-date public records.
For verification or follow-up: consult the Liberties report (liberties.eu) and the Guardian coverage (March 2026), review Voksmonitor’s methodology for party positions, and ask providers for the model/version metadata and moderation logs that make auditability possible.