AI and the imitation game

“Maria, this is Fauzia. I’m having trouble reaching Ahmed at the moment, so can you please click on the link I’ve sent so that you can wire me 50,000 rupees? It’s urgent!”

So sixty-year-old Maria immediately “wires” the money and calls Fauzia a minute later to let her know she’s just wired it, only to find out that her sister never actually called her today.

And the rest is a scam artist’s dream courtesy artificial intelligence (AI) voice cloning.

At first this seems like an extreme situation.

However, there have been accounts of Microsoft’s Copilot, an AI assistant, mimicking user’s voices. Additionally, CNN interviewed a woman who claimed to have been scammed thousands of dollars after receiving a phone call claiming her daughter had been kidnapped and hearing what she believed was her daughter’s voice.

Even more troubling is that you can plug in a 10 second audio of yourself on apps like ElevenLabs that have Instant Voice Cloning currently listed as a paid starter feature for only $6 per month. Imagine what a 10 second audio claimed from social media or a video online would do.

International celebrities have taken the first hit and many have expressed anger over the unauthorised cloning of their voices. According to BBC, a voice-over artist based in the UK had her voice downloaded on US-based AI firm Fish Audio more than 900 times without her consent.

The unlikely silver lining for Pakistan is that although it may be behind when it comes to AI, maybe one advantage is that we can see the repercussions from the lack of proper legislation and control from the experiences of other countries. The more troubling reality is that developed countries are severely struggling with their patchwork of AI laws.

Targeting the vulnerable

This is particularly concerning for the elderly population. According to a research paper by Professor Saniya Sabzwari from Aga Khan University Hospital and Professor Gohar Azhar from University of Arkansas for Medical Sciences, there is a demographic increase of the elderly living alone in Pakistan due to brain drain - approximately 6-7% of patients according to their observations.

Senior citizens are usually targeted because criminals view them as soft targets especially as they may be unfamiliar with AI-generated voices. Social isolation and unknowing trust are also some factors that make this demographic the most vulnerable.

Another large group are the digitally illiterate masses, especially those who rely on digital banking like easypaisa, those who depend on financial support from relatives abroad or even those who use WhatsApp frequently. Because either their voices can be easily obtained for scamming purposes or they might be unaware of the cloning. A point to note is that most banks still use phone calls to verify identity or even to discuss important financial matters.

Losing identity

Voice is unfortunately only one of the few important biomarkers of who we are that is now imitable. Generative AI has produced highly convincing pictures and videos - so your face and appearance can be separated from your actual actions. While signature forgery is not new, AI creates a more sophisticated means to do so, and enough examples of your writing can provide means to emulate your writing style as well.

While identity has traditionally been established through things that are difficult to replicate - AI has made everything reproducible. The danger of AI may not simply be that machines can impersonate us. It is that the characteristics we use to distinguish one human being from another - our face, voice, handwriting, words and mannerisms - are increasingly becoming reproducible.

If my voice, face, handwriting, writing style and personality can all be replicated, what exactly remains that prove I am me?

Ultimately, we arrive at the same conclusion time and again: AI requires stronger legislation. And educating the public is a responsibility of the Pakistani government. There just seems significant scope for abuse and exploitation.

The article does not necessarily reflect the opinion of Business Recorder or its owners.