Deepfakes: Novel trend or novel threat?

Deepfakes are a machine-aided synthetic media technology used for the generation or manipulation of text and video that can appear quite realistic to the untrained eye, and they have the potential for considerable implications across culture, geopolitics, and security.

While advances in technologies that have the potential to be exploited by threat actors, like those used to produce deepfakes, are generally controlled through regulation, the cat may already be out of the bag where deepfakes are concerned. We are at the point where most legitimate uses of deepfake tech can be controlled to a degree by limiting legitimate access, but these controls likely will have little effect on the potential for malicious use.

It is important to delineate the three main types of deepfakes so we can understand their implications:

Mimicking fakes: this is a technique where video of one person is superimposed on a target video using AI to enhance and mask the manipulation. Think of this as an advanced “green screen” process that effectively lets one person do all the talking while it appears to be someone else, right down to movements and gestures.

Generative fakes: this technique also employs AI algorithms, but in this case, they are used to completely synthesize new audio and video from existing materials to produce ultra-realistic content.

Generative text fakes: this more common technology uses AI applications such as the OpenAI GPT-3 to allow computers to generate text content on almost any subject that is incredibly close to actual human writing.

Each of these applications poses a separate and distinct threat on their own if misused, but in combination, have the potential to produce “generative personas” that soon will be extremely difficult to distinguish from the real thing. This is where the security implications come into play.

Malicious deepfake applications
Generative text fakes using GPT-3 level tech have the potential to make phishing attacks far more convincing and much harder to detect in an automated fashion, as the newer iterations of these techniques can generate text in individualised speaking styles that closely resemble specific people. This means it can be used to create deep-phishing messages that will be impossible for the current generation of anti-phishing systems to identify, leaving targets at risk.

Right now, advanced generative text technology is largely confined to a defined group of people that OpenAI granted access to. They blocked the last generation of the tech from public release due to fear of abuse. So, for the moment, that channel isn’t a significant threat. That said, the massive knowledge base used in model training means the tech is getting very close to being indistinguishable from actual human authorship, and it’s not clear that a strategy exists to better detect these fakes in the future.

Mimicking fakes with superimposed video — while still an evolving technology — are already good enough to fool an unwitting viewer to a reasonable degree. If this technology can be applied in real-time and coupled with advanced audio fakes, there is the potential that you might one day find yourself in a video conference call with someone you think is your CEO but is an attacker. The technology is not quite there yet, but before long it may result in a very effective attack vector.

Generative fakes with synthesised video have the most potential to have serious implications if used for nefarious purposes, given the technology creates fake content that could convincingly have almost anyone saying and doing almost anything.

Think of the impact that the abuse of fake social media accounts has had on the success of malicious state-sponsored disinformation and influence campaigns in recent years. Now imagine it’s not just mono-dimensional sock puppet personas inundating media channels, but dynamic manifestations of what appear to be real people saying and doing real things.

Finally, consider the implications a convincing deepfake video of a high-profile figure, going uncontrollably viral on the Internet, could have on public safety or geopolitical relations. The potential for real harm is significant.

Identifying deepfakes
Generally speaking, the successful detection of fake generated media will be based on our ability to recognise repeated patterns generated by the algorithms used in their production. In the case of text fakes that leverage GTP-3, the machine learning relies on a massive collection of human-generated text, and the same machine learning techniques used to create the fakes would also be used to identify “true” text vs. “fake” text. This is a case where the better machine learning application has the advantage.

GTP-3 tech is so advanced that it could be abused to create deep-phishing messages that speak in the voices of many and will be impossible for the current generation of anti-phishing systems to identify. Algorithms for both generation and detection of deepfakes rely on the same data pool for training the models, so it’s very hard to distinguish between text generated by the newer algorithms.

As for video deepfakes, at this point they generally have a perceivable “artificial quality” to them, even really well-executed fakes. The best weapon we humans have for determining if a video is a deepfake for now is for us to learn to recognise and acknowledge that uneasy feeling that what we’re looking at doesn’t feel real and is somewhat alien. This “uncanny valley” is the key to unravelling fakes — they just feel wrong intuitively.

But as these technologies improve, it’s clear that more than human intuition will be required to make a solid determination on authenticity, and it may be the case that it is simply not going to be possible without the creators of the technologies or the fakes themselves intentionally leaving an indicator of some sort, like a virtual watermark.

Either way, deepfakes and our ability to ensure they cannot be utilised maliciously, are a subject worthy of further discussion before the technology advances to the point where we can’t put the genie back in the bottle.

Yossi Naar is the chief visionary officer and co-founder, Cybereason