A security researcher acquired 6TB of routing-session data from a China-based LLM router carrying Anthropic Claude traffic and found credentials sufficient to compromise Chinese government agencies and firms including Huawei and Xiaomi.
The specific victims matter less than the mechanism. LLM routers and proxies now sit as invisible middleware between applications and model providers, chosen for cost arbitrage, multi-model failover, and access to region-restricted APIs. Every one of them logs the full request-and-response stream by default. That stream is no longer just natural language. Modern prompts routinely carry API keys, OAuth tokens, database connection strings, internal endpoints, and customer records pasted in for context. A router quietly aggregates all of it into one indexed, searchable corpus. When that corpus leaks, an attacker skips reconnaissance entirely and inherits a pre-sorted credential inventory. This is a structurally worse exposure than a classic database breach, because the data is already contextualized: who owns the key, what it unlocks, and how it is used are all sitting in the same session log.
The strategic shift for executives is that AI adoption has created a new tier of shadow infrastructure that existing security programs do not see. CASBs and DLP tools were built for SaaS and file egress, not for prompt payloads flowing to an opaque third-party gateway. Procurement rarely vets these intermediaries with the rigor applied to a cloud provider, yet they now hold some of the most sensitive plaintext in the organization.
For Japanese enterprises and SIers, this is an acute and near-term risk. The rush to deploy generative AI across NT Data-, Fujitsu-, and Nomura-class integration projects frequently relies on aggregation layers to juggle OpenAI, Anthropic, and domestic models under one contract. Cost-conscious teams and offshore development partners often reach for the cheapest available router without confirming where sessions are logged or under whose jurisdiction. Combined with Japan's persistent culture of copy-pasting production configs and credentials directly into chat interfaces, and RPA bots that pipe unmasked business data into LLM calls, the leakage surface is enormous and largely unmonitored.
The defensive playbook is concrete. Treat every LLM gateway as a data processor requiring the same due diligence as a cloud vendor, mandate a company-approved routing layer with no third-party logging, deploy prompt-level secret scanning and redaction before egress, and rotate any credential that has ever transited an external router. SIers that can package this governance into their AI delivery offerings will find it a genuine differentiator, since most clients have not yet realized the gateway itself is the weakest link.
