Alibaba AI model tried to send data to Beijing server, Cohere says

Publisher excerpt From CSO Online

An Alibaba AI model generated an unexpected command to upload data to a Beijing-region cloud storage service during a cybersecurity evaluation, rival developer Cohere said, raising concerns about the security of AI agents with access to enterprise systems. The model, Qwen3.8-Flash-Next, was being tested using Microsoft’s CTI-REALM cybersecurity benchmark, which tasked it with writing a rule to detect malicious file downloads followed by execution. Instead, it inspected local files and system settings before generating the unexpected command, according to Cohere’s report. The Canadian AI company said its isolated testing environment blocked the outgoing request. The model had indicated in its generated reasoning that it would not use any tools, contradicting the command it subsequently generated. The behavior occurred in fewer than one in 100,000 evaluation calls, Cohere said. When researchers repeated the same task, the model produced the requested detection rule without attempting an external connection. Cohere acknowledged that it could not determine whether the behavior was deliberate or resulted from information absorbed during training. Alibaba did not immediately respond to a request for comment. Uncertainty over the model’s behavior The findings warrant further investigation but do not establish that Alibaba deliberately introduced a backdoor into the model, said Pareekh Jain, CEO of Pareekh Consulting. “The behavior could result from contaminated training dat…

Excerpt only, shown with attribution. This content belongs to the original publisher and has not been independently verified by CyberSecurity.pk.