The Fireworks AI security breach exposed some unattributable Notion AI chat transcripts, and it shows how AI vendor risk now reaches directly into the tools regulated businesses use every day.
Notion published a security notice about the incident at one of its AI vendors, Fireworks AI, on its help site [1]. Fireworks published its own security bulletin, first on October 9, 2026 and updated afterward, describing the intrusion and its response [2]. Business and IT leaders in regulated industries should care because the incident is a case study in how a breach at an AI infrastructure provider can surface customer chat data even when the customer is a large, well-audited SaaS company.
Key Takeaways
- Notion determined that fewer than 1% of Notion workspaces may have been affected, and that some unattributable Notion AI chat sessions may have been exposed in the Fireworks incident [1].
- The exposed transcripts were not labeled with who wrote them or which workspace they came from, and they did not contain tool call results such as Notion page content, but Notion still advises treating the data as if it were exposed [1].
- Fireworks traces the intrusion to a third-party cloud provider that hosted its R&D machines, which experienced a security incident on October 23; the next day an unauthorized third party used that foothold to reach Fireworks development machines and run an unauthorized workload [2].
- Fireworks links the activity to a financially motivated cybercriminal group engaged in "LLMJacking," selling unauthorized access to large language model tokens to illicit routing gateways [2].
- Notion disconnected its systems from Fireworks as of November 3, deleted all data in the Fireworks system, stopped sending data and traffic to Fireworks, and disabled its Insights feature [1].
What happened: the facts in detail, in order
The intrusion path at Fireworks
According to Fireworks' updated bulletin, a third-party cloud provider hosting Fireworks R&D machines experienced a security incident on October 23. On October 24, the unauthorized third party used that access to reach Fireworks development machines and then ran an unauthorized workload that led to additional customer data impact. Fireworks detected the activity the same day and followed its incident response processes to take remediation measures and close known critical vulnerabilities. As of October 25, Fireworks states it has not detected further evidence of successful unauthorized access on the platform customers use [2].
Fireworks engaged Palo Alto Networks Unit 42 to conduct forensic analysis and notified law enforcement. The company's earlier bulletin, published October 9, 2026, had described a narrower impact: at the time of that publication, only Fireworks stored secrets, such as those used in evaluation jobs, were known to have been accessed [2].
What the attackers got
The updated bulletin says the affected internal application provided access to several categories of information. Resource metadata included Fireworks resource identifiers, display names, descriptions, and creation times. Billing information included billing and shipping addresses, invoice and payment history, account balance, and credit usage; Fireworks states payment card numbers are not stored by Fireworks and were not impacted. For a limited set of accounts, secrets, datasets, and incomplete model weights were also accessed [2].
Fireworks attributes the activity to a cybercriminal threat actor group that is financially motivated and aimed at selling tokens from unauthorized access to large language models to illicit routing gateways, a practice known as "LLMJacking." Fireworks cites a November 2 blog post by Google Threat Intelligence Group and Mandiant on the rise of LLMJacking, its motives, and its techniques [2].
How Notion was drawn in
Notion used Fireworks for two purposes: classification jobs supporting a feature called Insights, which helped customers understand their own AI usage, and as a general inference provider serving open-weight models inside the product. Fireworks is described by both companies as one of the most-used global AI infrastructure platforms, powering tens of thousands of customers and used by hundreds of thousands of developers [1].
Notion's timeline is specific. Earlier this month, Notion learned from Fireworks about potential incidents and shut down all connections to Fireworks' systems out of an abundance of caution. On November 6, Fireworks notified Notion that malicious actors may have accessed potential customer data. Notion began another internal investigation. On November 8, Fireworks confirmed the new dataset. Upon analysis, Notion found that it included unattributable Notion AI chat transcripts used for the Insights feature [1].
As of November 3, Notion had disconnected its systems from Fireworks, deleted all data in the Fireworks system, and stopped sending all data and traffic to Fireworks. It also halted all training runs. The Insights feature has been disabled while Notion investigates, and Notion is no longer serving open-weight models through Fireworks [1].
What was exposed, and what was not
Notion determined that fewer than 1% of Notion workspaces may have been affected. The exposed material was some unattributable chat sessions with Notion AI. "Unattributable" carries a precise meaning here: the chats were not labeled with who wrote them or which workspace they came from, and the transcripts did not contain tool call results, such as Notion page content or external data pulled in by Notion AI. Beyond the words in the chat transcripts themselves, Notion states the data contained no information that could specifically identify the workspace, user, or company. No other data from within a workspace is affected [1].
That qualifier matters, but so does Notion's own caution. The company recommends that affected customers treat the data as if it were exposed, and specifically warns that if anyone pasted sensitive information such as passwords, API keys, or other credentials into Notion AI during the timeframe, they should rotate those credentials right away [1].
What Notion is doing for affected workspaces
Notion has contacted the owners of affected workspaces. Those owners can review the affected data inside their workspace through a security incident review page, available on the desktop app or web browser on the latest version of Notion; the data is not accessible on mobile. They can download a CSV of the workspace's affected data, and an annotated CSV with additional metadata, such as the user associated with each chat transcript, to support their own impact analysis. Notion states these added annotations and metadata were not exposed as part of the incident. Workspace owners can also share a link so individual members can review their own affected data. Notion notes these resources are only available if the workspace has managed members [1].
Why the Fireworks AI security breach matters technically
The Fireworks AI security breach is instructive because of where the data sat and how it moved. Notion did not suffer a direct intrusion into its own workspace storage. Instead, chat transcripts flowed to a third-party AI infrastructure provider as part of normal processing, and the compromise happened on that provider's side, starting with a third-party cloud provider hosting Fireworks' R&D machines [1] [2].
This is the shape of modern AI supply chain risk. When you use an AI feature in a SaaS product, your prompts and the model's answers may transit multiple companies: the software vendor, the inference provider that runs the model, and the cloud provider hosting the inference provider's systems. Each hop is a place where data can be exposed. In this incident, the exposed transcripts were deliberately stripped of attribution, which limits what an attacker can do with them. But the transcript text itself is the user's words, and users routinely paste things into AI chat windows that they would never put in a document, which is exactly why Notion recommends rotating any credentials that were pasted in [1].
The attacker's motive is also notable. Fireworks links the activity to LLMJacking, the theft and resale of access to large language model tokens through illicit routing gateways, a trend Google Threat Intelligence Group and Mandiant published about on November 2 [2]. That means the objective was not necessarily the documents or the billing records. It was compute and model access itself. For a regulated business, that reframes the threat model: your AI vendor's credentials and API capacity are now a monetizable target in their own right.
There is also a lesson in what limited the damage. Fireworks states its Inference, Training SDK, and Nexus services are zero data retention by default, and it has not detected any impact to them [2]. Notion's own Trust Center describes zero data retention for LLM providers for Enterprise workspaces, with a maximum 30-day retention before deletion for non-Enterprise workspaces, and contractual agreements prohibiting the use of customer data for model training [4]. Retention limits and data boundaries did not prevent this incident, but they narrowed what was available to steal. The exposed dataset was a working set for a classification feature, not a full archive of customer content.
Finally, the incident shows that strong security programs at the direct vendor do not eliminate risk further down the chain. Notion's Trust Center lists annual third-party audits including SOC 2 Type II, ISO 27001/17/18/701, HIPAA, and BSI C5, annual penetration testing, a bug bounty program through HackerOne, and a formal AI Tools Usage and Governance Policy reviewed annually [4]. None of that stopped a compromise at a vendor's vendor. Regulated businesses should read their subprocessor lists with that in mind: every name on the list is another organization whose security posture you are inheriting.
What the evidence does and does not show
Both companies have published substantial detail, but several important questions remain open, and it is worth being precise about what is known and what is not.
What is established: the intrusion path, from a third-party cloud provider hosting Fireworks R&D machines on October 23, to Fireworks development machines and an unauthorized workload on October 24; the categories of data accessed, including resource metadata, billing information, and, for a limited set of accounts, secrets, datasets, and incomplete model weights; the suspected motive, LLMJacking; and the remediation steps both companies have taken [1] [2].
What is not established: Fireworks states its investigation is ongoing and may reveal additional information [2]. The company says it is notifying customers whose data was impacted and providing details, but it has not publicly stated which customers, or how many, fall into the "limited set of accounts" whose secrets, datasets, and incomplete model weights were accessed. Notion has not publicly stated the exact timeframe of chats included in the exposed dataset, only that affected workspace owners have been contacted and can review their own data [1].
There are also limits on the exposure assessment itself. Notion's figure, fewer than 1% of workspaces, is a "may have been affected" determination, not a confirmed count of compromised workspaces. The transcripts are described as unattributable, but unattributable is not the same as harmless: the text of a chat can contain sensitive material, which is why Notion's own guidance is to treat the data as if it were exposed and rotate any credentials pasted into Notion AI during the timeframe [1].
Fireworks' negative findings carry their own caveat. The company states it has not detected any impact to Inference, Training SDK, and Nexus, and has not detected further evidence of successful unauthorized access since October 25, but frames these as findings based on the investigation to date [2]. As a precaution, Fireworks has disabled self-serve dataset uploads and downloads used by Batch Inference and dataset-based training while the investigation continues [2].
For affected Notion workspace owners, the immediate practical step is the one Notion itself prescribes: review the exposed transcripts through the security incident review tools, identify anything sensitive, and rotate credentials accordingly [1]. For everyone else, the open question this incident leaves behind is how well you know the AI vendors sitting two and three hops away from your data, a question we take up in Part 2.
The Compliance and Risk Implications for Regulated Organizations
The Fireworks AI security breach is not, on its face, a compliance failure by Notion or by Fireworks. Both companies responded the way mature programs are supposed to respond: Fireworks engaged law enforcement and Palo Alto Networks Unit 42 for forensics, Notion disconnected its systems from Fireworks on November 3, and affected workspace owners were contacted directly. The compliance problem is different and harder. It is the problem of fourth parties.
When a regulated business uses a SaaS product, its data protection obligations do not stop at the vendor's front door. Notion used Fireworks for classification jobs behind its Insights feature and as a general inference provider for open-weight models. Most customers never knew Fireworks existed, because Fireworks was a subprocessor, one step removed. When Fireworks was breached through a third-party cloud provider that hosted its R&D machines, data that Notion customers had placed in Notion ended up in a dataset that malicious actors may have accessed. That is the chain this incident exposes: your vendor's vendor.
For defense contractors, the framework that governs this is explicit. NIST SP 800-171 requires an operational incident handling capability and the tracking, documenting, and reporting of incidents, and DFARS 252.204-7012 requires reporting a cyber incident affecting covered defense information to DoD within 72 hours of discovery. If your controlled unclassified information flows through a SaaS tool whose AI subprocessor is breached, your incident response obligations do not pause while you wait for the vendor to figure out what happened. The Verizon 2025 DBIR found that third-party involvement in breaches rose from 15% to 30%, which means this scenario is no longer an edge case.
For healthcare organizations, the HIPAA Breach Notification Rule at 45 CFR 164.404, 164.406, and 164.408 imposes notification duties when unsecured protected health information is breached. Business associates accounted for about 66% of breached healthcare records in 2024, and the Change Healthcare breach ultimately reached 192.7 million individuals in the final HHS filing of 2025-07-31. An AI feature that processes clinical questions is a business associate relationship whether or not anyone labeled it that way.
The deeper lesson is about data flow mapping. Notion's Trust Center publishes a subprocessor list, and its documentation states that user data is never used to train Notion AI models and that Enterprise workspaces get zero data retention from LLM providers, with a maximum 30-day retention before deletion for non-Enterprise workspaces. Those are meaningful controls. But zero data retention at the model layer did not prevent chat transcripts used for a classification job from sitting in a dataset that could be stolen. The control that mattered here, and the one most organizations lack, is knowing exactly which of your data goes where, through how many layers, and under what retention terms at each layer.
What This Means for Regulated Industries
Defense contractors
The DFARS 252.204-7012 question is the sharpest one: does your incident response plan contemplate a breach at a subprocessor of a subprocessor? If your staff uses Notion AI, or any AI feature inside a SaaS tool, on systems that touch covered defense information, you need to know whether that feature routes data to an external inference provider, and what that provider's security posture is. DFARS 252.204-7012 requires a cloud service provider handling covered defense information to meet security requirements equivalent to the FedRAMP Moderate baseline, and the DoD CIO's CMMC FAQ clarifications make clear that encrypted CUI is still CUI. An AI feature that ingests CUI into a prompt is handling CUI, full stop. The practical step is to inventory every AI feature your team uses, determine the data path behind each one, and either block the ones you cannot verify or confine them to systems outside your CUI boundary. Enclaving CUI into a narrowly scoped boundary reduces audit surface precisely because it makes questions like this one answerable.
Healthcare organizations
Clinicians and administrative staff paste things into AI assistants. Scheduling questions, coding questions, sometimes fragments of patient records. If your organization uses Notion or a similar tool with AI features, treat the Notion guidance as your own playbook: Notion told affected customers to treat the exposed data as if it were exposed and to rotate any credentials pasted into Notion AI. Apply the same logic to your own environment. The IBM 2026 finding that the average healthcare breach costs $6.64M is the financial backdrop, but the operational risk is simpler: an AI chat transcript can contain PHI even when the database behind it is fully compliant. Your risk analysis under 45 CFR 164.308(a)(1)(ii)(A) should account for AI features as data flows, not as productivity conveniences.
Legal practices
Attorneys routinely use general-purpose productivity tools, and AI features inside those tools now process client communications by default. The exposed Notion AI transcripts were unattributable, meaning they were not labeled with who wrote them or which workspace they came from, but the transcript text itself was exposed. A privileged communication pasted into a prompt is still a privileged communication, and the confidentiality obligation does not depend on whether the attacker can attribute the text. Law firms should add AI features to their client confidentiality risk assessments and their vendor diligence, and should be able to tell clients, specifically, which tools touch client data and which subprocessors sit behind them.
Financial services
Fireworks reported that the incident exposed billing information for a limited set of accounts, including billing and shipping addresses, invoice and payment history, account balance, and credit usage, though payment card numbers were not stored by Fireworks and were not impacted. For financial institutions, the relevant lesson is the LLMJacking motive: Fireworks attributed the attack to a financially motivated cybercriminal group aimed at selling unauthorized access to Large Language Models to illicit routing gateways. Google Threat Intelligence Group and Mandiant published a blog post about the rise of LLMJacking on November 2. Your AI spend is now an attack surface. Stolen model access is a monetizable commodity, which means the API keys, inference endpoints, and vendor accounts behind your AI features need the same credential governance as your payment systems.
An Action Plan for This Month
- Inventory every AI feature in your SaaS stack. List each product your team uses that has an AI assistant, summarizer, or classification feature. Notion's Insights feature ran through Fireworks without most customers knowing; assume you have equivalents.
- Map the subprocessor chain for each one. Check your vendors' trust centers for subprocessor lists and AI governance documentation. If a vendor cannot tell you which inference providers it uses and under what retention terms, that is a diligence finding.
- Identify what your staff has been pasting into AI features. Focus on credentials, API keys, client data, PHI, and CUI. Notion's own recommendation to affected customers was to rotate any credentials pasted into Notion AI; apply that standard to your own tools.
- Rotate exposed or suspected credentials now. If any team member may have pasted an API key or password into an AI assistant, rotate it before you finish the investigation. Rotation is cheap; the alternative is not.
- Update your incident response plan to cover fourth parties. NIST SP 800-61 Rev. 3 frames incident response as part of cybersecurity risk management aligned to the NIST Cybersecurity Framework 2.0, and its GOVERN function makes leadership accountability explicit. Your plan should specify who monitors vendor breach notices, who decides whether a vendor's breach triggers your own reporting duties, and on what timeline.
- Verify your reporting clocks. Defense contractors should confirm they can meet the DFARS 252.204-7012 requirement to report to DoD within 72 hours of discovery, including having a DoD-approved medium assurance certificate obtained in advance. Healthcare organizations should review their breach notification procedures against 45 CFR 164.404, 164.406, and 164.408.
- Run a tabletop exercise on this exact scenario. A vendor's vendor is breached, your data is in the exposed set, and you learn about it from a notice rather than from your own telemetry. Walking your team through that decision chain before it happens is the point of Cybersecurity Tabletop Exercise Services.
- Tighten third-party risk management. The Verizon 2025 DBIR's finding that third-party involvement rose from 15% to 30% justifies making subprocessor transparency a contractual requirement, not a courtesy. Third-Party Risk Management Services can formalize that diligence.
- Consider where sensitive AI workloads should actually run. If your AI features process regulated data, evaluate whether that data belongs in a shared multi-tenant inference chain at all. Open-weight models can run on infrastructure you control, which is the premise behind private AI deployment for CUI and PHI workloads.
- Document everything. Whatever you decide about AI features, write it down: the inventory, the decisions, the compensating controls. Auditors under CMMC, HIPAA, and SOC 2 will ask how you govern AI, and the answer needs to exist on paper.
How Petronella Technology Group, Inc. Helps
Petronella Technology Group, Inc. works with regulated businesses and defense contractors on exactly the problem this incident illustrates: sensitive data flowing through systems and vendors you do not fully control. Our managed detection and response service combines 24/7 threat monitoring, AI-powered analytics, and human-led investigation. Our AI-augmented hybrid SOC runs ten-plus production AI agents on our own private AI cluster alongside human analysts who hold CMMC-RP credentials, with a Digital Forensics Examiner (DFE #604180) on staff. That architecture exists because defense industrial base and healthcare clients cannot send CUI or PHI to a public-cloud SOC, and because we apply the same guardrails we recommend: the AI never closes a ticket on its own, never touches production systems without human authorization, and every action is logged for CMMC and HIPAA audit.
On the compliance side, our compliance practice maps AI data flows to the frameworks that govern them, including CMMC Levels 1 through 3, HIPAA, and DFARS 252.204-7012. Our vCISO engagements deliver a written current-state assessment within 30 days and a 90 to 180 day remediation roadmap written for the executive reader, and the named vCISO can serve as security officer of record for HIPAA, CMMC senior official authorization, SOC 2 defined roles, and PCI assigned responsibility. For organizations that decide sensitive AI workloads should move onto infrastructure they control, we design, build, and operate private AI clusters end to end, running open-weight models such as Llama, Qwen, Mistral, and DeepSeek on an inference stack you own, with network isolation, secrets handling, prompt and access logging, and egress control mapped to framework controls.
If you believe your organization's data was in the affected set, or you need help working through a vendor breach notice, our digital forensics and incident response team and our incident response retainer provide capacity before you need it. And if the inventory exercise above turns up more AI sprawl than you expected, that is normal; the first step is a conversation.
Related reading
- Chainalysis Breach Exposed Credentials and Tokens for All Users
- What to Do After a Data Breach
- Ernst & Young Data Breach Affects Personal, Financial Information
- SickKids data breach exposes employee and job applicant info
Sources
- Notice about the Fireworks AI incident that may impact some Notion workspaces
- Update on October 2026 security incident
- Own your specialized intelligence
- Notion Trust Center
- Cross-Sector Cybersecurity Performance Goals
- SP 800-61 Rev. 3, Incident Response Recommendations and Considerations for Cybersecurity R
Frequently Asked Questions
Was my Notion workspace affected by the Fireworks AI security breach?
Notion determined that fewer than 1% of Notion workspaces may have been affected, and it contacted the owners of affected workspaces directly. If neither you nor your workspace owner has heard from Notion, the company states that your workspace is not affected and no action is needed.
What data was actually exposed in the Fireworks AI security breach?
For Notion customers, the exposed data consisted of unattributable Notion AI chat transcripts used for the Insights feature. The chats were not labeled with who wrote them or which workspace they came from, and the transcripts did not contain tool call results such as Notion page content. Beyond the words in the transcripts themselves, the data contained no information tying chats back to specific workspaces, users, or companies. Separately, Fireworks reported impacts to account and billing information, and for a limited set of accounts, to secrets, datasets, and incomplete model weights.
What is LLMJacking and why does it matter here?
LLMJacking is the practice of stealing access to large language models and selling that access to illicit routing gateways. Fireworks attributed the incident to a financially motivated cybercriminal threat actor group pursuing exactly this scheme, and Google Threat Intelligence Group and Mandiant published a blog post about the rise of LLMJacking on November 2. It matters because it shows that AI infrastructure is now a direct target, not collateral damage.
Does this incident mean AI vendors cannot be trusted with regulated data?
It means AI vendors must be governed like any other subprocessor that touches regulated data. Notion's Trust Center documents SOC 2 Type II, ISO 27001/17/18/701, HIPAA, and BSI C5 annual third-party audits, a bug bounty program through HackerOne, and annual third-party penetration testing, and those controls coexisted with this incident. The takeaway is not that AI features are unusable; it is that you need to know the data path, the retention terms, and the subprocessor chain before regulated data enters a prompt.
What should I do if my team pasted credentials into Notion AI?
Rotate them now. Notion's own guidance to affected customers was to treat the data as if it were exposed and to rotate passwords, API keys, and other credentials pasted into Notion AI during the affected timeframe. That advice generalizes: credentials pasted into any AI assistant should be treated as potentially disclosed.
How do I find out which AI subprocessors my vendors use?
Start with their trust centers. Notion publishes a subprocessor list and its AI governance documentation publicly, and Fireworks publishes security bulletins detailing its incident response. If a vendor has no equivalent disclosure, ask directly in writing. The answer, or the absence of one, belongs in your vendor risk file either way.
If this incident has you rethinking how AI features fit into your compliance posture, call Penny, our AI assistant, at 919-348-4912 to book a consultation, or reach Petronella Technology Group, Inc. through our contact form. The first call is a free 30-minute scoping consultation with a CMMC Registered Practitioner, delivered remote-first across all 50 states.
Free, practical, and specific to regulated environments. We will email it to you.
No spam. Unsubscribe anytime.
