PipeDream/InController: From High-Level Alert to Hands-On Threat Hunting

We’re diving into a topic that caused quite a stir in the ICS (Industrial Control Systems) security world: the PipeDream (also known as InController) toolkit. This was a significant alert because it highlighted a sophisticated new arsenal designed to target control systems, and intriguingly, it was discovered before it was reportedly used in the wild.

Today, we’ll unpack what PipeDream/InController is all about, and then, more importantly, we’ll explore how you can leverage a fantastic open-source tool, Wireshark, to hunt for the types of network activity associated with the techniques described in the public reports. Our main focus will be on the network forensics side of things.

The Big Reveal: What Exactly is PipeDream/InController?

Back on April 13th, 2022 (yep, has to be 2022 for A22-103A to make sense as a naming convention for that year), the Cybersecurity and Infrastructure Security Agency (CISA) dropped alert A22-103A. On the very same day, industry giants Mandiant, Dragos, and Schneider Electric released their own complementary research. It seems like a coordinated effort, with multiple teams digging into this toolkit.

The CISA report was stark: it detailed how Advanced Persistent Threat (APT) actors—often nation-state backed—had developed capabilities to directly attack control systems. The toolkit, as we’ll see, was modular and packed a punch. A crucial point, however, was that there were no confirmed instances of this toolkit being used “in the wild” at the time of the alerts. It was a proactive discovery, which is a somewhat rare and welcome development in the cybersecurity landscape.

So, what did this PipeDream/InController toolkit contain? It was essentially a collection of modules, each with a specific nasty purpose. Depending on whether you’re reading the Mandiant or Dragos reports, you’ll see different names for these components, but functionally they covered five main areas:

  1. OPC UA Module: OPC Unified Architecture is a very common industrial protocol. This module could reportedly read/write tags and even brute-force credentials.
  2. CODESYS Module: CODESYS is a widely used IEC 61131-3 development environment for PLCs. This module targeted its proprietary communication. We’ll dig into this one.
  3. Omron FINS Module: Another proprietary protocol, used by Omron PLCs. This is our other main focus for today, as Wireshark has some good support here.
  4. ASRock Driver Vulnerability: This involved exploiting a vulnerability in ASRock motherboard drivers to load vulnerable drivers, likely to bypass driver signing enforcement and gain privileged access.
  5. Backdoor: A component for command and control (C2) and reconnaissance.

For our practical Wireshark exploration, we’re going to zero in on the CODESYS and Omron FINS aspects, as these involve network communications we can potentially observe and dissect.

Threat Hunting Omron FINS Traffic with Wireshark

Let’s kick things off with Omron FINS. Many Programmable Logic Controllers (PLCs) out there support this protocol. It typically communicates over TCP and UDP on port $9600$.

The good news? Wireshark comes with a built-in dissector for FINS! This dissector is pretty handy, recognizing commands like run/stop, memory writes, file transfers, log operations, and more. While the absolute accuracy of any dissector can be a deep topic, a quick look at its capabilities and some sample packet captures (PCAPs) suggests it’s quite comprehensive.

If you want to play along, Wireshark’s own wiki for Omron FINS (a quick search will find it) often links to sample PCAPs. These are invaluable because they usually contain examples of many function codes. One quirk I’ve noticed in some samples is that requests might appear early in the capture, with responses clustered towards the end. Keep an eye out for that if you’re analyzing.

Now, let’s take some of the capabilities mentioned in the CISA alert regarding PipeDream and think about what they might look like as FINS traffic in Wireshark. The reports were high-level, not giving byte-by-byte details, so we’re doing some educated interpretation here based on protocol knowledge. We don’t have the actual malware samples, so this is about understanding the protocol’s capabilities that could be used for such attacks.

1. Backing Up/Restoring Arbitrary Files to the PLC

CISA mentioned this capability. How could it be done with FINS?

  • Single File Write (Command $0x2203$): This command allows writing a single file. You’d expect to see the filename and the actual file data in the packet. This is a prime candidate for transferring malicious files or exfiltrating legitimate ones.
    • Wireshark field to watch: omronfins.command (for the command code), omronfins.filename, and the data payload.
  • File Data to Memory: FINS also allows writing file data directly into memory, not just to disk. This uses different mechanisms but achieves a similar outcome of getting attacker-controlled data onto the device.
    • What to look for: Ensure your hunting covers both file-to-disk writes and file-to-memory writes. Don’t just focus on one.

2. Copying Files

If an attacker gets a file onto the PLC via another method, or needs to move files around on the device itself (perhaps as part of staging an attack), they might use a file copy function.

  • File Copy (Command $0x2207$): This command, unsurprisingly, copies files.
    • Wireshark field to watch: omronfins.command. You’ll also see source and destination filenames, often in fields like omronfins.filename.source and omronfins.filename.dest. Be aware that sometimes the same Wireshark field might be used for both depending on the packet structure, so context is key.

3. Remotely Deleting Files

To cover their tracks, attackers might delete files.

  • File Delete (Command $0x2205$): This command can delete one or more files. The packet might specify the number of files being deleted.
    • Wireshark field to watch: omronfins.command and associated parameters indicating which files.

Key Questions for FINS Monitoring:

  • Function Codes: Are you monitoring for specific, potentially malicious FINS function codes related to file and memory operations?
  • Disk vs. Memory: Are your analytics distinguishing between (or covering both) writes to the PLC’s disk and writes directly into its memory?
  • File Metadata: Can you parse or at least identify suspicious filenames, file types, or file sizes being transferred?
  • Vendor Capabilities: If you use a commercial passive network monitoring tool, ask your vendor how deeply they dissect FINS and what specific threats or behaviors their analytics cover.
  • Alerting Thresholds: Are your alerts tuned to detect isolated suspicious commands, or do they only fire on high-volume activity? A single malicious file write is significant.
  • Broader Applicability: Many OT protocols have similar file operation capabilities. The principles here can be adapted.

The great thing about FINS, from a defender’s perspective, is that these file transfer operations often happen “in the clear” within the protocol, making them relatively straightforward to spot with a tool like Wireshark.

4. Wiping Program Memory and Device Resets (from Mandiant’s Report)

Mandiant’s report specifically called out the toolkit’s ability to wipe program memory and reset devices. Let’s think about the typical sequence of operations for this:

  1. Put PLC in Stop Mode: Before writing to protected program memory, a PLC usually needs to be put into a “stop” or “program” mode.

    • FINS Stop Command (Command $0x0402$): Look for this command.
    • Wireshark field: omronfins.command.
  2. Clear Program Area Protection (If Necessary): Program memory is often protected. Attackers might need to disable this protection.

    • FINS Program Area Protection Clear (Command $0x0305$): This command handles unprotecting program areas.
    • Wireshark fields: omronfins.command. Interestingly, you might also see the omronfins.program_number and even a omronfins.password field! This is gold for defenders. You could potentially detect brute-force attempts or the use of default/weak passwords to unprotect memory. Imagine an alert: “Default password used to unprotect program memory on PLC X!”
  3. Wipe/Overwrite Program Memory: This is the actual destructive action.

    • FINS Memory Area Write (Command $0x0102$): This command writes data to a specified memory area.
    • Wireshark fields: omronfins.command, omronfins.memory_area.block_number (or similar, indicating the address), and omronfins.command_data (which would contain the actual bytes being written – e.g., a string of $0xAAs or $0x00s to wipe an area).


Key Considerations for Detecting Memory Wipes:

  • Protection Changes: Are you monitoring for commands that change memory protection status, especially if accompanied by suspicious or default passwords?
  • Stop Commands: Do you see PLC stop commands immediately preceding memory writes to program areas?
  • The Wipe Itself: Can you detect large writes of non-standard data or patterns indicative of wiping (e.g., all nulls, all $0xFFs) to program memory locations?

Again, these principles can be applied to other OT protocols that have similar device control and memory access functionalities.

5. Loading a Custom Malicious Agent (Theoretical)

The reports suggested the ability to load custom malicious agents. Details on how this was achieved were sparse, so this section is more speculative, brainstorming ways it could happen via FINS:

  • Memory Area File Transfer (Command $0x220A$): If an attacker uploads their agent as a “file” but wants it loaded directly into a specific memory region to be executed. This command could facilitate that, specifying the memory space and the file data.
  • Program Area Write (Command $0x0307$): If the agent needs to reside in the program memory area, this command could be used. The destination memory address (omronfins.program_area_address or similar) would be critical here – writes to unexpected or unusual program memory locations would be highly suspicious. The omronfins.command_data would contain the agent’s code.
  • Memory Area Transfer (Command $0x0105$): This could be used to move data from one memory area to another. Perhaps an agent is loaded into a temporary buffer and then moved to its final execution location in memory. You’d see source and destination memory addresses.

Considerations for Detecting Agent Loading:

  • Write Locations: Where are writes occurring? Are they to standard file storage, general memory, or protected program memory? Baseline what “normal” looks like.
  • Payload Analysis: The holy grail is analyzing the content of what’s being written. Is it legitimate configuration data, or does it look like executable code? This is advanced but powerful.

Bonus FINS Analytic: Clearing the Error Log

While not explicitly tied to PipeDream in the reports I saw, good attackers cover their tracks.

  • Error Log Clear (Command $0x2103$): This tiny packet can clear the PLC’s error log. Error logs on PLCs aren’t always like verbose Windows Event Logs; they’re often rolling buffers of diagnostic codes. If an attacker’s actions generated errors, they might use this to wipe away the evidence. Seeing this command in conjunction with other suspicious activity would be a red flag.

Omron FINS Wrap-Up: The existing Wireshark dissector for FINS gives us a solid starting point. The protocol’s nature, with distinct commands for file, memory, and control operations, provides clear indicators to build detections and threat hunts around. Ensure your monitoring strategy accounts for the specific function codes and delves into parameters like filenames and memory locations.

Tackling CODESYS: A More Opaque Challenge

Now, let’s shift gears to CODESYS. This one is trickier. CODESYS is a very popular software platform for industrial automation, but its network protocol is proprietary. This often means “security by obscurity,” which isn’t real security.

Fortunately, researchers at Kaspersky ICS Lab did some phenomenal work dissecting the CODESYS protocol in a blog series. They even developed a Wireshark dissector. Unfortunately, and this is a common headache in the ICS world, it appears some of this research, including potentially source code for the dissector, was asked to be taken down. I’ve checked, and it’s not readily available on the Wayback Machine either. This is a setback for defenders, as open knowledge about protocols is crucial for security.

So, for CODESYS, we’re largely relying on Kaspersky’s public analysis. We haven’t independently reversed the protocol for this discussion.

Kaspersky’s three-part blog series is the go-to reference here (a search for “Kaspersky CODESYS protocol” should find it).

  • Protocol Basics: CODESYS communication can occur over TCP and UDP. It uses specific programming ports, including $1740$-$1743$ (TCP/UDP), (UDP), `$11740$ (TCP), and potentially other controller-specific ports. If you’re really keen, CODESYS’s own website has guides like “Your First CODESYS Program” that can inadvertently reveal some of these communication details.
  • Layered Structure: The protocol is layered. Kaspersky’s research showed screenshots of their dissector breaking it down. Even without their dissector, understanding this structure is key if you were to attempt to build your own or interpret raw hex.
    • PDU (Protocol Data Unit) Level: The basic wrapper.
    • Datagram Level: This is where it gets interesting for routing over TCP/UDP. Kaspersky identified Service ID and Message ID fields at this layer. These are critical for initial classification of traffic.
    • Channel Layer: Deeper still, this layer introduces concepts like Service Groups. For example, log services might be group <span class="math-inline">0x05</span>, shell services group ` (hex values from memory, check Kaspersky’s blog for specifics). Monitoring these service groups is vital.
    • Service Layer & Tags: This is where the specific functions, analogous to what we saw with FINS (file operations, device control), are invoked. This layer uses “tags” to define the precise action. Kaspersky showed an example of an authentication request at this layer, clearly showing the username “administrator.”

Building a CODESYS Dissector (The Challenge): Given that Kaspersky’s dissector isn’t open-source and their detailed code was taken down, if you have CODESYS in your environment and need deep packet inspection, you might be faced with the non-trivial task of developing your own dissector. This is a significant undertaking.

Key CODESYS Monitoring Considerations (Based on Kaspersky’s Work):

  • Metadata at Multiple Layers: Because it’s layered, you need to consider what metadata to extract at each level (Service ID, Message ID, Service Group, Tags).
  • Depth of Analysis: How deep does your (or your vendor’s) CODESYS analysis go? Are you just seeing CODESYS traffic, or can you identify specific high-risk operations?
  • Port Coverage: Ensure all known CODESYS communication ports are being monitored.

From Generic Intel to Actionable Hunts

The journey from a high-level alert like “PipeDream can target control systems” to “I should look for FINS command $0x2203$ with suspicious filenames on port $9600$” is what operationalizing threat intelligence is all about.

We’ve seen how, even with limited public details about the malware itself, by understanding the targeted protocols (Omron FINS and CODESYS), we can use a tool like Wireshark to:

  • Formulate threat hunting hypotheses.
  • Identify specific commands and data fields to monitor.
  • Understand what malicious activity could look like on the wire.

This approach allows us to move beyond just reading the headlines and actively start looking for potential threats in our own environments.

Thanks for sticking with me through this deep dive! It was a bit longer than usual, but I hope it provided some practical insights into how you can approach threat hunting for sophisticated ICS malware. Be sure to check out the Insane Cyber YouTube channel for more Tech Talks. 

See how Insane Cyber transforms security

Our products are designed to work with
you and keep your network protected.