In heavy industry (oil refineries, nuclear plants, or chemical facilities) AI promises efficiency but introduces unprecedented risks. As discussed in our series’ introduction, large language models (LLMs) can struggle when making numerical and logical conclusions.
To understand these risks systematically, we turn to the industry standard Purdue model. It's a framework that organizes industrial control systems into six levels, from physical equipment to standard enterprise IT. By mapping AI-related security threats across these levels, we can categorize vulnerabilities based on potential impact.
This post explores direct threats like poisoned AI models and cyberattacks, alongside indirect risks from operator misuse and AI overreliance from engineers, setting the stage for stronger safeguards in critical industries.
Purdue Model
The Purdue model structures industrial systems into six levels, each with distinct roles. Here's a rough example.
Level 0: Physical processes (e.g., pumps, valves, sensors).
Level 1: Basic control (e.g., PLCs, DCS).
Level 2: Supervisory control (e.g., SCADA, HMIs).
Level 3: Operations DMZ (e.g., scheduling, maintenance).
Level 4: Intranet (e.g., internal servers, metrics dashboards, Sharepoint, HRP, etc.).
Level 5: Internet facing servers (e.g., email servers, customer/vendor APIs, etc.).
The general idea is the age-old "defense in depth" paradigm. As data access becomes unnecessary for a broader audience, continue restricting access at each logical layer. Note that flows can also be restricted via one-way diodes - a blessing and a curse in practice - which we'll visit in a future article. For now, let's look at threats posed by AI adoption.
AI threats - general
Before considering specific threats at each level, let's look at what "AI threats" consists of. We can consider two broad categories: direct attacks and indirect problems.
Direct attacks
Direct attacks consist of attacks intentionally conducted by threat actors.
Model Inversion - theft and training inference
Attackers can gain information on proprietary models and training datasets. This is likely to occur when a model can be queried by the attacker, and smaller models are much more susceptible to theft or loss of training data. In the case of OT/ICS, this remains a relatively low likelihood of occurrence, and training data is likely not to be sensitive proprietary information.
"Enabling" of threat actors
ICS/OT infrastructure is still relatively obscure technology. It's never been a good idea to rely on the obscurity as a defensive control, but it's undoubtedly been an advantage. Nomore. Attackers can now easily learn about various OT infrastructure. including vendor-specific vulnerabilities and esoteric protocols - hallmarks of OT/ICS. Hell, they can even ask for a full attack chain on any specific plant.
Malicious AI plugins
Coding has been forever changed by LLMs. Engineers who use LLMs for code generation should be aware of malicious 'code helpers' - plugins for VS Code, for example, can assist OT programming. Innocuous looking plugins are increasingly becoming a threat vector. Other avenues are likely to emerge - from agentic tools like Claude Code or other desktop tooling.
Poisoning
Models are trained on tons of data. If malicious data is introduced during the training phase, the model can be 'poisoned' to make certain predictions (or classifications).
Misclassification attacks
A misclassification attack occurs when an AI/ML model is tricked into coming to the wrong conclusion. In traditional models, a 'cat' might be misclassified as a 'dog'. This is often an artifact of a adversarial attack via 'gradient descent' - its an abuse of the way neural networks classify decision boundaries.
Indirect problems
Indirect problems are not conducted with intent - they're the natural outcome as a result of AI adoption.
Misaligned models
Misaligned models occur when an AI's objectives diverge from intended outcomes due to poor specification or emergent behaviors. In heavy industry, this might arise from training on historical data that embeds outdated safety assumptions (e.g., an LLM assisting in chemical plant scheduling might prioritize throughput over resource constraints, inadvertently increasing downtime risk). Unlike direct attacks, misalignment stems from design flaws, amplifying in high-stakes environments where "good enough" approximations can lead to cascading failures.
Overreliance
Overreliance happens when operators or engineers defer critical judgment to AI outputs, completely bypassing human expertise. In refineries, this could mean trusting an LLM-generated alarm response without verification, especially under fatigue or time pressure - potentially missing nuanced indicators like subtle vibration anomalies in turbines. Research shows this "automation bias" reduces situational awareness, heightening risks in critical scenarios, such as emergency shutdowns.
Hard to update
AI models, particularly large ones, are resource intensive to retrain, leading to outdated deployments vulnerable to evolving threats. In OT systems, where downtime is costly, updating a misbehaving predictive analytics model in a chemical plant might require halting operations. This inertia contrasts with traditional software patches and can exacerbate indirect risks, as models trained on pre-2025 data fail to account for new regulatory or environmental variables.
AI vibe code
Vibe coding is the term for having an LLM generate code for you. Expert programmers have caught, and in some cases have missed, serious security vulnerabilities generated as part of the vibe coding experience. As engineers are not typically known for superb coding skills, it stand to reason they may increasingly rely on generated code - everything from metrics dashboards to ladder logic.
Generated code should be kept barred from any critical processes (as is the case with IEC regulation).
AI threats by Purdue level
Levels 4 and 5 - Intranet and enterprise DMZ
Levels 4 and 5 include standard enterprise hardware and software - everything from domain controllers to custom web applications.
Levels 4 and 5, by virtue of size and exposure, is where we expect to see most problems. We consider AI related threats to be similar enough to categorize together.
Direct
| Category | Example Risk | Likelihood | Impact |
| Model Inversion | IP theft | Medium | Low |
| "Enabling" of threat actors | Generated attack plan for your specific company perimeter technology stack | High | Medium |
| Malicious AI plugins | Employees across the enterprise can open C2 channels to APT by using malicious coding plugins | High | High |
| Poisoning | Poisoned enterprise models recommend risk-inducing COAs | Low | High |
| Misclassification attacks | Malicious actor submits slightly altered input to "trick" model into wrong conclusion | Medium | Low |
Indirect
| Category | Example Risk | Likelihood | Impact |
| Misaligned models | Financial analyst trusts incorrect output from foundational LLM about operations metrics | High | Medium |
| Overreliance | Employees begin to lose domain-specific knowledge over time. | High | Medium |
| Hard to update | N/A - at level 5, models are generally outsourced or relatively easy to update. | - | - |
| AI vibe code | Engineers utilize LLMs to generate critical procedural documentation. | Certainty | High |
Level 3 - Operations DMZ
Direct
| Category | Example Risk | Likelihood | Impact |
| Model Inversion | - LLMs creating maintenance procedures that skip critical safety steps because they weren't emphasized in training data
- AI-generated emergency response plans that optimize for speed/efficiency rather than safety margins | Medium | High |
| "Enabling" of threat actors | Attackers become familiar with security TTPs, including deployment strategies, | Certainty | Medium |
| Malicious AI plugins | Vendor or open source project sells a 'supervisor helper AI', to help inform operations considerations. This integrates an unknown model into process management equipment. This could lead to anything from stealing credentials to automated downstream attacks. | Medium | High |
| Poisoning | - False maintenance records introduced during training. Years later, AI recommends avoiding maintenance, causing cascading equipment failure. | Medium | High |
| Misclassification attacks | - AI systems analyzing plant data and incorrectly categorizing dangerous conditions as routine
- AI anomaly detection that flags normal but unusual conditions as problems, while missing actual emergencies | Medium | Medium |
Indirect
| Category | Example Risk | Likelihood | Impact |
| Misaligned models | AI-driven scheduling that prioritizes equipment uptime over thorough inspections | Medium | Medium |
| Overreliance | Operators losing ability to naturally understand critical parameters (flow rates, pressure differentials) when AI systems fail | Medium | Medium |
| Reduced situational awareness as operators become "system monitors" rather than active process controllers | Certainty | High | |
| Hard to update | AI system optimizing plant operations becomes progressively less accurate as equipment ages or process conditions change. Operators gradually lose confidence in AI recommendations, but have already lost the expertise to make manual decisions effectively | Medium | High |
| AI vibe code | Plant engineers use ChatGPT to generate Python scripts for custom monitoring dashboards. Generated code looks professional but contains logical errors in alarm threshold calculations (or more direct security issues). | High | High |
Level 2 - SCADA & HMI
Level 2 encompasses supervisory systems like SCADA servers, HMIs, batch/recipe servers, and alarm/report servers. These components bridge operational oversight with lower-level controls (e.g., PLCs at Level 1), enabling real-time monitoring, command issuance, and data aggregation.
As AI integrates here, for anomaly detection in alarms, or optimized batch processing, it introduces risk that can cascade to physical processes.
Direct
| Category | Example Risk | Likelihood | Impact |
| Model Inversion | Discovery of critical alarm parameters learned by an AI model. | Minimal | Medium |
| "Enabling" of threat actors | LLMs will allow anyone to build SCADA-specific exploit chains. | High | High |
| Malicious AI plugins | Coding tools, from compilers to IDEs, are compromised with malicious backdoors. ICS related coding tools are proven to be a prime target. | Certainty | High |
| Poisoning | Malicious data introduced into training sets causes critical alarms to be bypassed. | Medium | High |
| Misclassification attacks | HMIs mislabel threats, e.g. a misclassification of a pressure spike as 'safe' via gradient informed manipulation. | Medium | High |
Indirect
| Category | Example Risk | Likelihood | Impact |
| Misaligned models | Models prioritize cost reduction over safety, leading to flawed maintenance recommendations from the AI. | Low | Medium |
| Overreliance | Automation bias increasingly erodes operator attention. | High | Medium |
| Hard to update | AI models resist patching due to downtime risk; as adoption increases, likelihood and period of downtime will increase. | Medium | Medium |
| AI vibe code | Generated code for alarm logic or HMI dashboards may introduce subtle vulnerabilities, especially if engineers lack coding expertise. This could manifest as unvetted scripts in batch servers, | High | High |
Level 1 - DCS
Level 1 encompasses the basic control layer, including Programmable Logic Controllers (PLCs), Safety Instrumented Systems (SIS), Variable Frequency Drives (VFDs), and Distributed Control System (DCS) controllers. These systems directly manage physical processes—sensors, actuators, and field devices.
AI integration at this level is emerging, often for predictive maintenance, control optimization, or sensor data analysis, but its proximity to physical operations amplifies risks.
This is the critical layer for industry and regulation to focus on.
Direct
| Category | Example Risk | Likelihood | Impact |
| Model Inversion | Models expose site specific training data. | Low | Low |
| "Enabling" of threat actors | LLMs expose PLC vulnerabilities (ladder logic flaws) facilitating targeted attacks - such as Stuxnet variants. | Medium | High |
| Malicious AI plugins | Vendors using AI to code PLC firmware introduce logic errors (or introduce security vulnerabilities). | Medium | Critical |
| Poisoning | PLC firmware is trained on malicious data, leading to incorrect actions taken under specific conditions. | Low | High |
| Misclassification attacks | Attackers feed slightly incorrect data to DCS controller (via wireless or other compromise), causing a misclassified state (e.g., a pressure spike as nominal). | Low | High |
Indirect
| Category | Example Risk | Likelihood | Impact |
| Misaligned models | "General purpose" models were not be tuned for site or unit specific variables. They can give contextually incorrect errors which would be correct elsewhere. | Medium | Medium |
| Overreliance | Engineers trust AI-generated ladder logic or SIS settings, missing numerical errors (e.g., incorrect pressure thresholds). | High | Critical |
| Hard to update | Embedded AI in PLCs or DCS likely require downtime, making it an option of last resort. | High | High |
| AI vibe code | Current regulations require verified code. | - | - |
Level 0 - Physical Controllers
Level 0 is for physical controllers - the actual valves, sensors, and actuators in the field. AI integration at this level is (as of now) rare. Realized issues however can be catastrophic - a supply chain attack on physical controllers causing a Deepwater Horizon style incident could easily be brainstormed by an AI.
Direct
| Category | Example Risk | Likelihood | Impact |
| Model Inversion | A vendor trains a "valve actuator AI" on a single unit, sells valve to other companies. The other company reverse engineers the original unit's operating metrics. | Low | Low |
| "Enabling" of threat actors | LLMs assist attackers in understanding fieldbus protocols or actuator behaviors, enabling targeted physical tampering (e.g., valve manipulation in refineries). | Medium | High |
| Malicious AI plugins | Valve suppliers utilize a backdoored code assistance tool, unknowingly introducing remote shutdown functionality directly to its wireless controller module. | Low | Critical |
| Poisoning | Tainted sensor data from compromised supply chains could poison upstream AI models. | Low | High |
| Misclassification attacks | Adversarial inputs to AI-optimized sensors (e.g., via manipulated fieldbus signals) misclassify physical states. | Low | High |
Indirect
| Category | Example Risk | Likelihood | Impact |
| Misaligned models | Valves are programmed with a model specific to another climate, causing erroneous actions when deployed elsewhere. | Low | Low |
| Overreliance | Future reliance on AI-enhanced sensors might reduce manual checks, risking missed anomalies (e.g., pressure drops in chemical tanks). | High | Low |
| Hard to update | Physical replacement of faulty AI-enabled sensors and actuators is extremely costly. | Medium | Medium |
| AI vibe code | Firmware programmed with AI has unknowingly introduced remote shutdown functionality tied directly to its wireless controller module. | Low | High |
Summary
AI models can be great. They can be fantastic. They are super helpful and one day may replace us all. But for now, let’s avoid them for usage in critical industry. That said, let’s clear up a few things.
First, AI ≠ LLM. The term AI encompasses everything from dedicated, site specific models trained on particular units for some small task to general purpose LLMs. LLM usage, in particular, is a huge risk in this industry for everything stated above. On the other hand, small dedicated models can be very useful - think maintenance prediction based on historian data for a specific site/unit. You’d want talented data engineers to build it, but the risk of this kind of model is outweighed by potential benefits.
Second - SIS. SIS is designed to prevent catastrophic problems through a series of regulations (V&V, code coverage analysis, unit testing, etc.). It’s also mandated from various standards (IEC 61508 and 61511) and is routinely audited. The issue I foresee is that audits themselves will become increasingly reliant on usage of AI. Engineers may use an LLM to generate some paperwork, auditors may use an LLM to check it. SIS systems engineers may code everything by hand, but have a development environment setup that has a malicious AI embedding hidden code.
Third, don’t discount the usage of LLMs to fuel attacks. Stuxnet was almost 20 years ago, but at the time required very specific knowledge. That knowledge is now easily obtainable.
The possibilities of using AI to attack heavy industry are endless.
