Cybercriminals using malicious AI prompts to trick victims: Hyderabad Police
The crooks are tricking AI with 'prompt injection attack' by confusing it with an intention to extract internal documents, customer records and system details of companies that should not be disclosed normally
Hyderabad: Cybercriminals are found to be giving malicious prompts to AI to mislead and trick the victim and data security, said Hyderabad Police Commissioner V C Sajjanar in a post on X on Monday.
The crooks are tricking AI with 'prompt injection attack' by confusing it with an intention to extract internal documents, customer records and system details of companies that should not be disclosed normally, Sajjanar said.
With the increasing use of AI, start-ups to large multinational companies are now chanting 'AI chatbots' and various organizations are increasingly relying on these as they have benefits such as instant answers to customer queries, increased work speed, and cost reduction, Sajjanar said.
Data security is a big challenge for many organizations, which are now connecting their AI models and chatbots to key data systems within the organization. The information should not be accidentally visible to the end user.
The Commissioner of Police warned stating a single 'tricky prompt' by hackers can expose all this confidential information. The organizations need to immediately put in place 'prompt guardrails' to detect and prevent this threat. A single layer of security is not enough and a multi-layer defense approach should be adopted.
