HIDA Lecture: Exploits and Misuse of AI

Thursday, 25.07.2024 · 11 am

Speaker: Lea Schönherr, Challenges and Threats in Generative AI: Exploits and Misuse

Date: 25.07.2024, 11 am

Title: Challenges and Threats in Generative AI: Exploits and Misuse


Generative AI (genAI) is becoming increasingly embedded in our daily lives, prompting concerns about potential threats within genAI systems and their outputs. In this talk, we will examine the challenges and security threats associated with generative AI. These threats fall into two main categories: malicious inputs used to exploit generative models and the misuse of computer-generated output.

Firstly, specially crafted inputs can be used to exploit models like LLMs to disrupt alignment or steal sensitive information. Existing attacks show that content filters of LLMs can be easily bypassed with specific inputs, and private information can be leaked. Moreover, established methods in the adversarial machine learning field cannot be easily transferred to generative models. We demonstrate that an alternative for protecting intellectual property is the obfuscation of prompts, which, with minimal overhead, can maintain utility while safeguarding confidential data.

Secondly, generative models can be used to produce fake content indistinguishable from human-generated content. This fake content is often used for fraudulent and manipulative purposes, such as impersonation and creating realistic fake news. As these models evolve, detecting fraudulent activities will become increasingly difficult, while the attacks themselves become easier to automate and require less expertise. This talk will provide an overview of the current challenges in detecting fake media in human and machine interactions.

Register here!

Lea Schönherr

Lea Schönherr is a tenure-track faculty at CISPA Helmholtz Center for Information Security interested in information security with a focus on adversarial machine learning. She received her Ph.D. in 2021 from Ruhr University Bochum, where she was advised by Prof. Dr.-Ing. Dorothea Kolossa at the Cognitive Signal Processing group at Ruhr University Bochum (RUB), Germany. She received two scholarships from UbiCrypt (DFG Research Training Group) and CASA (DFG Cluster of Excellence).


Subscribe newsletter