AI-created videos circulating on Elon Musk’s X depict American soldiers captured by Iran, an Israeli city in ruins, and US embassies ablaze — a surge of lifelike deepfakes despite a policy crackdown to curb wartime disinformation.
The Middle East war has unleashed an avalanche of AI-generated visuals, dwarfing anything seen in previous conflicts and often leaving social media users unable to distinguish fabrication from reality, researchers say.
In a bid to protect “authentic information” during conflicts, X announced last week that it would suspend creators from its revenue sharing program for 90 days if they post AI-generated war videos without disclosing they were artificially made.
Subsequent violations will result in permanent suspension, X’s head of product Nikita Bier warned in a post.
The new policy is a notable pivot for a platform heavily criticized for becoming a haven of disinformation since Musk completed his USD44 billion acquisition of the site in October 2022.
It also won praise from senior State Department official Sarah Rogers, who called it a “great complement” to X’s Community Notes — a crowd-sourced verification system — that results in “less reach (thus monetization)” for inaccurate content.
But disinformation researchers remain skeptical.
“The feeds I monitor are still flooded with AI-generated content about the war,” Joe Bodnar of the Institute for Strategic Dialogue told AFP.
“It doesn’t seem like creators have been dissuaded from pushing misleading AI-generated images and videos about the conflict,” he said.
Bodnar pointed to a post from a premier “blue check” X account — which is eligible for monetization — that shared an AI clip depicting an Iranian “nuclear-capable” strike on Israel.
The post garnered more views than Bier’s message about cracking down on AI content.
X did not respond when AFP asked how many accounts it had demonetized since Bier’s announcement.
AFP’s global network of fact-checkers — from Brazil to India — identified a stream of AI fakes about the Middle East war, many from X’s premium accounts with blue checkmarks that can be purchased.
They include AI videos depicting a tearful American soldier inside a bombed-out embassy, captured US troops on their knees beside Iranian flags, and a destroyed US navy fleet.
The flood of AI-fabricated visuals — mixed with authentic imagery from the Middle East — continues to grow faster than professional fact-checkers can debunk them.
Grok, X’s own AI chatbot, appeared to make the problem worse, wrongly telling users seeking fact-checks that numerous AI visuals from the war were real.
Researchers have also warned that X’s model — allowing premium accounts to earn payouts based on engagement — has turbocharged the financial incentive to peddle false or sensational content.
One premium account, which posted an AI video of Dubai’s Burj Khalifa skyscraper engulfed in flames, ignored a request from Bier that it label the content as AI.
The post remained online, racking up more than two million views.
Last month, a report from the Tech Transparency Project said X appeared to be profiting from more than two dozen premium accounts belonging to Iranian government officials and state-controlled news outlets pushing propaganda, potentially in violation of US sanctions.
Comments
Comments are closed for this article.