Saudi Authority Identifies Deepfake Detection as Key to Mitigating Gen-AI Risk
The Saudi Data and Artificial Intelligence Authority (SDAIA) has warned of significant risks associated with malicious deepfakes, categorized under three primary threats; from imposter scams, to non-consensual manipulation, to disinformation and propaganda.
As deepfake creation technology becomes more sophisticated and accessible, through democratized access to generative AI, associated risks are increasing in tandem, creating a profound sense of vulnerability among citizens.
In its report: “Deepfakes Guidelines: Mitigating Risks While Fostering Innovation”, the SDAIA highlights non-consensual manipulation as a key risk, where threat actors use deepfake technology to create explicit or compromising content of individuals without their consent, often for purposes of harassment, blackmail, or reputational damage.
Victims of these attacks are typically women and children, who face severe emotional distress, potential long-term mental health issues, strained relationships, and professional setbacks. This violation of privacy and trust underscores the urgent need for robust legal and technological safeguards, the Saudi authority said.
Malicious deepfakes versus non-malicious deepfakes
| Malicious Deepfakes | Non-Malicious Deepfakes | |
| Definition | AI-generated or manipulated media designed to deceive, harm, or exploit, often used in spreading false information, defamation, or fraud | AI-generated media created for benign purposes like entertainment, satire, or education, without the intent to deceive or cause harm |
| Transparency & Disclosure | Deepfake content that doesn’t disclose its synthetic nature. This lack of transparency can manipulate public perception and cause harm | Clearly labelled content to inform viewers that it is synthetically generated or altered |
| Authorization & Consent | Using individuals’ likeness without their consent. This violates their rights and privacy. Use cases include identity theft and defamation | Obtain explicit consent from individuals whose likeness is used in the creation of deepfakes |
| Application | Employ deepfakes for harmful purposes such as spreading false information, political manipulation, exploitation, harassment, or defamation | Use deepfakes for constructive purposes like education, entertainment, historical reconstructions, and accessibility |
| Compliance | Adherence to comprehensive ethical guidelines for the creation and use of deepfakes | Failing to follow ethical guidelines and regulations in the creation and use of deepfakes |
Applications of deepfake technology
| Image Generation Manipulate photos using AI, or generate fully synthetic realistic images of people or scenes | Training Simulations Creating realistic simulations for training purposes | Explicit Material Attackers use deepfakes to generate non-consensual explicit images to exploit, harass or blackmail individuals | Advertising Brands use deepfakes to create engaging and novel advertisements |
| Audio Generation Synthetic audio mimicking a person’s voice used to create realistic but fake speech | Financial Scams Synthetic voices of trusted people to trick victims into transferring money | Audio Deepfakes Realistic voiceovers and dubbing in different languages without the original voice actor | Voice Cloning Using cloned voices of real people to access bank information to authorize transactions or commands |
| Video Generation AI-generated video that alters the appearance of individuals or creates realistic but fully synthetic visual content | Film and Production Similar to image deepfakes, video deepfakes are used to create digital actors or manipulate video content | Political Manipulation Using deepfake technology, attackers can alter interviews and speeches, or even fabricate new scenes to discredit candidates | Corporate Espionage Using deepfakes to manipulate executive communications to steal sensitive information |
Analytical AI tools recommended for deepfake detection
One of the main technological safeguards identified in the report is to authenticate content using analytical AI tools. The SDAIA recommended Sensity AI as a detection tool that can analyze content for signs of manipulation.
The SDAIA specifically mentioned various detection techniques supported by Sensity AI, including:
- Pixel Analysis: This technique involves scrutinizing the pixel-level details of an image or video to uncover anomalies or artifacts that could suggest manipulation.
- Motion Inconsistencies: This technique can identify irregularities in the way objects or people move within a video.
- Audio-Visual Synchronization: In deepfake videos where the audio has been synthesized or manipulated, this approach can assess whether the audio matches the visual content.
- Content Provenance: By examining metadata and embedded digital watermarks like C2PA manifests, it is possible to trace content back to its source and verify its authenticity.
Some methods and approaches used for content provenance as identified by the SDAIA:
- Metadata Analysis: Metadata contains information about the file, such as the date and time it was created, the device used to capture the content, and the software employed. By analyzing this data, it’s possible to identify discrepancies or signs of tampering. For example, inconsistencies between the creation date of the file and the supposed event it depicts can be a red flag.
- Digital Watermarking: Watermarks or content credentials are embedded into the media to provide proof of origin. Advances in technology have enabled the embedding of imperceptible digital watermarks within images and videos that can be traced back to the content creator. Many of these watermarks are resistant to common editing techniques but they are not infallible.
