What changes when customer data hits an LLM (high level)
Data minimization, residency, logging, vendor policies, and OWASP LLM risks—engineering checklist with Microsoft and regulatory references. Not legal advice.
Disclaimer: This is an engineering checklist, not legal advice. Involve security, privacy, and counsel for your contracts, jurisdictions, and industry (HIPAA, GDPR, PCI, etc.).
1. Know what “sending data to an LLM” implies
When you call a hosted model API, you are typically transmitting prompt and context to a vendor’s systems. You must align with:
- Your customer agreements (what subprocessors are allowed?).
- Vendor product terms (training use, retention, subprocessors).
- Regional requirements (data residency, cross-border transfers).
For Microsoft’s Azure OpenAI Service, start with Data, privacy, and security in the FAQ and the official product documentation for your deployment type. Re-read when you change region, SKU, or API version.
2. Data minimization (the practical rule)
Send the smallest text that still lets the model complete the task.
| Instead of | Prefer |
|---|---|
| Pasting an entire PDF | Chunk + retrieve top-k chunks (RAG) with citations |
| Full conversation forever | Sliding window + summarized older turns |
| Raw emails with signatures/footers | Strip boilerplate in a preprocessing step |
The EDPB publishes guidance on data protection principles (lawfulness, minimization)—useful framing even outside the EU.
3. Logging, retention, and access control
If Application Insights or CloudWatch captures prompts or completions, you have created a new data store subject to retention and access policies.
- Define who can read logs (break-glass only?).
- Set retention aligned with policy (e.g., delete or redact after N days).
- Consider redaction of structured PII before logging—patterns exist, but regex alone is fragile for names/addresses.
OWASP’s LLM Top 10 calls out Sensitive Information Disclosure—logging is a common leak path.
4. Security abuse cases beyond privacy
Include in threat modeling:
- Prompt injection — untrusted content in the same context as instructions. Mitigations: separate trusted/system content, tool allowlists, output filtering. See OWASP Prompt Injection entry.
- Insecure output handling — treating model output as safe to pass to shells, SQL, or browsers without validation.
- Excessive agency — giving the model tools that can mutate production without human gates.
NIST’s AI Risk Management Framework is a useful cross-industry reference for governance structure.
5. Human review gates
For high-risk outputs (financial advice, medical content, anything customer-facing in regulated sectors), define when a human must approve before send. The model is an accelerator, not an authorizer.
6. Vendor diligence checklist (questions to answer in writing)
- Are inputs used to train foundation models? Under what conditions?
- Where is data processed and stored (regions)?
- What subprocessors exist?
- What certifications apply (SOC 2, ISO, FedRAMP paths)?
- What SLAs and incident notification terms apply?
References
- Microsoft: Azure OpenAI Service documentation — authoritative for Azure
- OpenAI: Enterprise privacy — if you use OpenAI’s commercial offerings (verify current terms)
- GDPR.eu — general reader overview of GDPR themes (not a substitute for counsel)
Related: webhook idempotency (safe automation), evaluation practices (quality gates).