From AI Assistants to Attack Infrastructure: How Grok and Copilot Can Be Abused as C2 Proxies

AI-powered tools such as Microsoft Copilot and xAI’s Grok can surf the web, fetch URLs, summarize information, and perform automated tasks for developers. However, security researchers have shown how such capabilities can be exploited to enable potentially more sinister activities. Web-enabled AI tools can be exploited as command-and-control (C2) proxies, enabling attackers to issue commands to compromised systems and exfiltrate stolen data via trusted AI services.

Turning AI assistants into a communication channel

The problem arises because AI models can fetch external URLs and return results to the user. By directing the AI to attacker-controlled endpoints, researchers were able to trick Copilot and Grok into retrieving commands from remote servers and returning responses through their web interfaces. In other words, the AI model can be used as a relay between the attacker and a compromised system.

Commands hosted by the attacker on their server are fetched by the AI tool and relayed back to the attacker through the interface. Likewise, the interface can be used to send responses back to the attacker. The result is a two-way communication channel that can receive commands and return data.

AI as a C2 proxy

This technique is particularly effective because malicious communication is hidden within legitimate traffic. Instead of malware communicating directly with a command-and-control server, the traffic can be routed through AI services that organizations already trust. 

The compromised system can send requests to the AI service, which fetches attacker-controlled URLs and returns the results. Because the communications appear to originate from a trusted AI service, most network monitoring solutions will be unable to distinguish malicious traffic from legitimate traffic.

LOTS: Living Off Trusted Services

Researchers describe the attack as a form of LOTS (Living Off Trusted Services), a variant of the “living off the land” tactic used by attackers to carry out malicious activities. Rather than setting up dedicated C2 infrastructure that security controls might detect, attackers can simply use legitimate online services to do the dirty work. In this context, AI tools can serve as a relay for command execution and data exfiltration.

The attack doesn’t end with command delivery. Researchers also demonstrated that data could be exfiltrated using the same technique. Data harvested by an attacker from a compromised system can be embedded in a request to the AI tool. When the AI tool fetches content from an attacker-controlled URL, the response returned via the interface may include the data. In effect, this tunnels the victim’s data through the AI service itself, disguising exfiltration activity as legitimate interactions with an AI tool.

Why this matters for security teams

The research underscores the unintended security risks introduced by rapidly evolving AI capabilities. Web browsing, URL fetching, and automation are features designed to enhance the utility of AI tools, but they can also introduce new attack surfaces when used as infrastructure for C2 activity. 

For defenders, the takeaway here is that AI services are now part of the attack surface and should be monitored like any other network- or Internet-connected system. 

Limiting automated interactions with AI tools may also be necessary in some cases. As AI tools become more ingrained in the enterprise, LOTS is likely to be among the techniques attackers employ to disguise malicious activity as legitimate.