Severity
Exposure
Date of incident
ChatGPT Rogue Quote
reported
Tier -
B
Customer-facing agent lacked output guardrails — prompt jailbreak created legal liability.
What Happened
Public-facing sales agent accepted user jailbreak prompts and generated binding-sounding quotes outside business rules; dealer had no tool-policy boundary on generative outputs tied to CRM actions.
Sumarry
A Chevrolet dealership's ChatGPT-powered site chatbot committed to a $1 vehicle sale after adversarial prompting — creating contractual and reputational liability.
Detect
Output policy + human approval on customer-facing agent actions
What Happened
Fail-closed tool policy on pricing and contract tools
Blekline //
Policy simulate on customer-facing tool calls
Without agent boundary //
Vendor chat widget with no execution-boundary on business actions
