Unraveling India’s Deepfake Quandary: AI, Politics, and the Battle for Election Integrity

As voters lined up early in the morning on November 30 last year to participate in legislative elections determining the next government of Telangana, a seven-second video clip began circulating rapidly on social media.

The video, posted on X by the opposition Congress party, purported to show KT Rama Rao, a leader of the ruling Bharat Rashtra Samiti in the state, urging voters to support the Congress. Widely shared on various WhatsApp groups “unofficially operated” by the party, the clip eventually found its way onto the party’s official X account, garnering over 500,000 views.

However, the video was fake.

“It was AI-generated, though it appeared entirely authentic,” admitted a senior Congress party leader speaking to Al Jazeera anonymously. “But to the average voter, it would have been indistinguishable; by the time it was posted, voting had already commenced, leaving no opportunity for the opposition campaign to mitigate the damage.”

This strategically timed deepfake incident serves as just one example of the deluge of AI-manipulated media that plagued recent elections in India’s states, now posing a significant threat to the upcoming general elections.

Scheduled between March and May, India’s massive electorate of nearly one billion people will determine the next national government. Concerns over deceptive AI-generated media were heightened after faked sexually explicit images of artist Taylor Swift surfaced on social media in January. In November, Ashwini Vaishnaw, India’s information technology minister, labeled deepfakes as a “threat to democracy,” a sentiment echoed by Prime Minister Narendra Modi.

However, despite the heightened awareness of the dangers posed by deepfakes, political parties across India, including Modi’s Bharatiya Janata Party and the Congress, are increasingly leveraging this technology to sway voters, according to managers of nearly 40 recent campaigns interviewed by Al Jazeera. With the accessibility of AI tools, some available for free and others for as little as 10 cents per video on subscription, the proliferation of deepfakes is rapidly reshaping the landscape of political influence in India.

The Rise of Deepfakes and Campaigning Tactics

The BJP, renowned as India’s most technologically advanced party, has long been a pioneer in utilizing innovative techniques for campaigning. As early as 2012, the party employed 3D hologram projections of Modi, enabling him to simultaneously campaign across numerous locations—a strategy widely employed during the 2014 general elections, which propelled Modi to power.

While these methods involved minimal deception, the BJP took a significant step in February 2020 when Manoj Tiwari, a BJP Member of Parliament, became one of the first politicians globally to utilize deepfakes for campaigning. Tiwari released three videos addressing voters in Delhi ahead of the capital’s legislative assembly elections, each in Hindi, Haryanvi, and English. Remarkably, while the Hindi video was authentic, the other two were deepfakes, employing AI to generate Tiwari’s voice, alter his expressions, and manipulate lip movements—making it incredibly challenging to discern their authenticity upon viewing.

In recent months, the Dravida Munnetra Kazhagam (DMK), which governs the southern state of Tamil Nadu, utilized AI to resurrect its iconic leader, M Karunanidhi, through lifelike videos at campaign events.

Now, consultants and campaign managers predict that the 2024 elections could witness a significant surge in the use of deepfakes.

“Politics revolves around perception; with AI tools for voice and video modulation, a mere click can completely alter that perception in an instant,” remarked Arun Reddy, the national coordinator for social media at the Congress. Reddy emphasized the party’s eagerness to incorporate AI into their campaigning strategies, acknowledging a shortage of skilled personnel to execute these ideas effectively.

As parties across the political spectrum bolster their teams, Reddy believes that AI will play a pivotal role in shaping the narrative of the upcoming elections. He anticipates a substantial increase in AI-manipulated political content, surpassing anything seen before.

AI Manipulation in Indian Politics

From the desert town of Pushkar in western India, 30-year-old Divyendra Singh Jadoun leads an AI startup called The Indian Deepfaker. Established in October 2020, his company made headlines for cloning the voice of Rajasthan state’s Congress chief ministerial candidate, Ashok Gehlot, during the November assembly elections. Using personalized messages on WhatsApp, addressed to each voter by name, the company aimed to enhance political outreach.

Currently, The Indian Deepfaker is collaborating with the team of Sikkim’s Chief Minister Prem Singh Tamang for upcoming campaign holograms. Situated in the Himalayas between India, Bhutan, and China, Sikkim represents one of India’s smallest states in the northeast.

While Jadoun prides himself on the company’s legitimate projects, he acknowledges being inundated with what he terms “unethical requests” from political campaigns. These requests, often received through international WhatsApp numbers, burner Instagram handles, or Telegram connections, seek alterations to videos and audio, including requests to incorporate pornography, to target political opponents.

Despite numerous such requests, Jadoun asserts that his company maintains strict ethical standards to avoid legal repercussions. However, he notes the prevalence of individuals offering such services at low prices, posing a significant ethical dilemma.

During recent election campaigns in Madhya Pradesh and Rajasthan, police registered multiple cases for deepfake videos targeting senior politicians from both the BJP and Congress. These videos, often produced by private consulting firms and distributed via social media networks like WhatsApp, contribute to the growing concern over deceptive political tactics.

A political consultant, speaking on condition of anonymity, revealed the use of ordinary citizens’ phone numbers to disseminate deepfakes via WhatsApp, complicating efforts to trace the origin back to parties, candidates, or AI firms.

In smaller constituencies, AI-manipulated audios play a significant role, with forged call recordings used to tarnish candidates’ reputations or fabricate evidence of corruption. Despite the ethical implications, manipulating voters through AI has become an integral part of campaign strategies for many political parties.

With India boasting over 760 million internet users—surpassing 50% of the population—AI-driven political manipulation poses a formidable challenge, with campaigns becoming increasingly bizarre and unconventional.

Challenges of Deepfakes in Indian Politics

Indian laws currently lack a clear definition of “deepfakes,” as noted by Anushka Jain, a policy researcher at the Digital Futures Lab in Goa. Law enforcement agencies resort to existing laws on defamation, fake news, or privacy violations, combined with the Information Technology Act, to address instances of deepfakes. However, this approach proves challenging, as authorities often focus on the consequences rather than the nature of deepfakes, leading to a reactive rather than proactive response.

Experts advocate for the Election Commission of India (ECI), responsible for overseeing elections, to adapt to the changing dynamics of political campaigns. During the Telangana state elections last year, leaders of the ruling Bharat Rashtra Samithi party cautioned their supporters about potential deepfake threats from the Congress party on social media. Despite appeals to the ECI regarding a deepfake video shared by the Congress on election day, the video remained online without any action from the ECI, as confirmed by two Congress leaders.

Former Chief Election Commissioner of India, SY Quraishi, underscored the severity of the issue, emphasizing that deepfakes have significantly exacerbated the spread of rumors during elections. He stressed the urgency for real-time monitoring of deepfakes to safeguard Indian democracy, urging the ECI to take proactive measures to mitigate potential harm.

“The ECI must intervene promptly,” he stressed. “Swift action is essential to address the risks posed by deepfakes.”

the Battle for Election Integrity

In recent times, the Indian government has been exerting pressure on major tech corporations like Google and Meta to take proactive measures in combating the proliferation of deepfakes on their platforms. IT Minister Rajeev Chandrasekhar’s engagements with officials from these companies underscore the gravity of the threats posed by deepfakes and the urgent need for action.

However, while this approach seems to absolve the government from accusations of selective censorship or broad crackdowns on emerging AI technologies, it also raises doubts about the sincerity of its regulatory intentions. Prateek Waghre, Executive Director of India’s Internet Freedom Foundation, highlights the inherent flaws in relying solely on tech companies to address the deepfake menace. He argues that the rise of AI exacerbates existing content moderation challenges, and the current approach fails to tackle the root of the problem.

The recent accord signed by major tech companies at the Munich Security Conference aimed to adopt reasonable precautions against the use of AI tools to disrupt democratic elections worldwide. Yet, the vague wording of the pact has left many advocates and critics disappointed, questioning its effectiveness.

In response, YouTube has announced measures to enable users to request the removal of AI-generated or altered content, aiming to safeguard election integrity. However, Ravi Iyer, Managing Director of the Neely Center for Ethical Leadership and Decision Making, expresses skepticism about the platform’s ability to detect deepfakes effectively, given low digital literacy levels and the increasing consumption of videos.

Identifying every AI-manipulated media is an arduous task, leading to calls for tech companies to redesign algorithms that discourage the spread of polarizing content. The Internet Freedom Foundation’s open letter urging electoral candidates and parties to refrain from using deepfake technology ahead of national elections reflects a proactive stance in addressing the issue, although the response remains uncertain.

Amidst these developments, political campaigns are ramping up their use of AI, signaling a potentially ominous future. As Reddy, the national coordinator for social media at Congress, acknowledges, the pervasive use of AI to distort facts threatens to obscure truth and manipulate public perception, posing significant challenges to the integrity of elections in India.