Geek-Guy.com

Tag: prompt

New image-based prompt injection attack targets multimodal AI models

Security researchers have developed a new image-based prompt injection attack that can manipulate how multimodal AI systems interpret user instructions without modifying the original text prompt, potentially expanding security risks for AI agents and vision-language systems. In a research paper published this week, researchers from Xidian University described a technique called “CrossMPI,” which uses nearly…

Security agencies draw red lines around agentic AI deployments

With prompt injection and other attack pathways consistently surfacing across agentic AI deployments, security watchdogs have stepped in, collectively, to draw some hard boundaries. A joint advisory from the US Cybersecurity and Infrastructure Security Agency (CISA) and international partners has called for tighter control over permissions, stronger monitoring, and a more deliberate rollout strategy, urging…

Prompt injection turned Google’s Antigravity file search into RCE

Security researchers have revealed a prompt injection flaw in Google’s Antigravity IDE that could be weaponized to bypass its sandbox protections and achieve remote code execution (RCE). The issue came from Antigravity’s ability to allow AI agents to invoke native functions, like searching files, on behalf of the user. Designed to kill complexity, the feature…

Zero‑click Grafana AI attack can enable enterprise data exfiltration

Indirect prompt injection is possible on AI-powered dashboards, allowing exfiltration of sensitive enterprise data without user authentication. Security researchers are warning about a critical Grafana issue, dubbed GrafanaGhost, that allows attackers to leak sensitive data from Grafana environments, including financial metrics, infrastructure health data, private customer data, and operational logs, among others. Noma Security disclosed…

Google Workspace’s continuous approach to mitigating indirect prompt injections

Posted by Adam Gavish, Google GenAI Security Team Indirect prompt injection (IPI) is an evolving threat vector targeting users of complex AI applications with multiple data sources, such as Workspace with Gemini. This technique enables the attacker to influence the behavior of an LLM by injecting malicious instructions into the data or tools used by…

Check Point Research Reveals ChatGPT Data Exfiltration Flaw

A flaw in ChatGPT’s code execution environment shows how a single malicious prompt could quietly leak sensitive user data — without any warning or user approval needed. “Sensitive data shared with ChatGPT conversations could be silently exfiltrated without the user’s knowledge or approval,” said Check Point researchers. Inside the ChatGPT DNS Exfiltration Flaw The issue…

A nearly undetectable LLM attack needs only a handful of poisoned samples

Prompt engineering has become a standard part of how large language models are deployed in production, and it introduces an attack surface most organizations have not yet addressed. Researchers have developed and tested a prompt-based backdoor attack method, called ProAttack, that achieves attack success rates approaching 100% on multiple text classification benchmarks without altering sample…

Microsoft details AI prompt abuse techniques targeting AI assistants

Prompt abuse occurs when crafted inputs manipulate an AI system into producing unintended behavior, such as attempting to access sensitive information or overriding built-in safety instructions. Prompt injection is also recognized as one of the top risks in the 2025 OWASP guidance for LLM applications. “Detecting abuse is challenging because it exploits natural language, such…

Arcjet enables inline defense against prompt injection in production AI systems

Arcjet has released AI Prompt Injection Protection, a new capability designed to stop prompt injection attacks before they reach production AI models. The feature detects hostile prompts at the application boundary and gives developers a decision point inside the request lifecycle where malicious instructions can be blocked before inference occurs. Companies are shipping AI features…

Mend.io eliminates AI prompt weaknesses before production

Mend.io has launched System Prompt Hardening within Mend AI to detect, score, and automatically remediate weaknesses in AI system prompts. Hidden instructions in system prompts have emerged as a growing security concern that traditional AppSec tools do not fully address. System Prompt Hardening provides instant visibility into these behind-the-scenes instructions, identifies weaknesses, and automatically strengthens…

Single prompt breaks AI safety in 15 major language models

A single benign-sounding prompt can systematically strip safety guardrails from major language and image models, raising fresh questions about the durability of AI alignment when models are customized for enterprise use, according to Microsoft research. The technique, dubbed GRP-Obliteration, weaponizes a common AI training method called Group Relative Policy Optimization, normally used to make models…