Firewalls and anti-malware programs alone are not enough to protect an entire network from an attack. A well-rounded security strategy should also include an intrusion detection system (IDS) that pinpoints suspicious traffic once it passes the firewall and enters the network.

This article is an introduction to intrusion detection systems and the role IDSes play in network security. Read on to learn how these systems work and why they are vital in preventing costly downtime and data breaches.

What Is an Intrusion Detection System (IDS)?

An intrusion detection system (IDS) is an app or device that continuously monitors network traffic and sends alerts in case of suspicious activity. An administrator then reviews alarms and takes actions to remove the threat.

Regardless of whether you set up a physical device or an IDS program, the system can:

  • Recognize attack patterns within network packets.
  • Monitor user behavior.
  • Identify abnormal traffic activity.
  • Ensure user and system activity do not go against security policies.

The info from an intrusion detection system can also help the security team to:

  • Audit the network for vulnerabilities and poor configurations.
  • Assess the integrity of critical systems and files.
  • Create more effective controls and incident responses.
  • Analyze the quantity and types of cyber threats attacking the network.

Cybersecurity benefits aside, an IDS also helps achieve regularity compliance. Greater network visibility and better logging ensure network operations stay in line with all relevant regulations.

Goals of Intrusion Detection Systems

The main goal of an IDS is to detect anomalies before hackers complete their objective. Once the system detects a threat, the IDS informs the IT staff and provides the following info about the danger:

  • The source address of the intrusion.
  • Target and victim addresses.
  • The type of threat.

The secondary goal of an intrusion detection system is to observe intruders and identify:

  • What resources attackers try to access.
  • How hackers try to bypass security controls.
  • What types of cyberattacks intruders initiate.

The company’s security operations center (SOC) and analysts can use this info to improve the network security strategy.

Anomaly detection and reporting are the two primary functions of an intrusion detection system. However, some detection systems can respond to malicious activity, such as automatically blocking an IP address or shutting down access to sensitive files. Systems with these response capabilities are intrusion prevention systems (IPSs).

How Do Intrusion Detection Systems Work?

An IDS monitors traffic to and from all devices on a network. The system operates behind a firewall as a secondary filter for malicious packets and primarily looks for two suspicious clues:

  • Signatures of known attacks.
  • Deviations from regular activity.

An intrusion detection system typically relies on pattern correlation to identify threats. This method allows an IDS to compare network packets to a database with signatures of known cyberattacks. The most common attacks an IDS can flag with pattern correlation are:

  • Malware (worms, ransomware, trojans, viruses, bots, etc.).
  • Scanning attacks that send packets to the network to gather info about open or closed ports, types of permitted traffic, active hosts, and software versions.
  • Asymmetric routing that sends a malicious packet and bypasses security controls with different entry and exit routes.
  • Buffer overflow attacks that replace database content with malicious executable files.
  • Protocol-specific attacks that target a specific protocol (ICMP, TCP, ARP, etc.).
  • Traffic flooding breaches that overload the network, such as a DDoS attack.

Once an IDS discovers an anomaly, the system flags the issue and raises the alarm. The alert can range from a simple note in an audit log to an urgent message to an IT admin. The team then troubleshoots the problem and identifies the root cause of the issue.

What Are the Types of Intrusion Detection Systems?

There are two main types of IDSes based on where the security team sets them up:

  • Network intrusion detection system (NIDS).
  • Host intrusion detection system (HIDS).

The way an intrusion detection system detects suspicious activity also allows us to define two categories:

  • A signature-based intrusion detection system (SIDS).
  • An anomaly-based intrusion detection system (AIDS).

Depending on your use case and budget, you can deploy a NIDS or HIDS or rely on both main IDS types. The same applies to detection models as many teams set up a hybrid system with SIDS and AIDS capabilities.

Before you determine a strategy, you need to understand the differences between IDS types and how they complement each other. Let us look at each of the four main IDS types, their pros and cons, and when to use them.

Network Intrusion Detection System (NIDS)

A network-based intrusion detection system monitors and analyzes traffic coming to and from all network devices. A NIDS operates from a strategic point (or points, if you deploy multiple detection systems) within the network, typically at data chokepoints.

Pros of a NIDS:

  • Provides IDS security across the entire network.
  • A few strategically placed NIDSes can monitor an enterprise-size network.
  • A passive device that does not compromise network availability or throughput.
  • Relatively easy to secure and hide from intruders.
  • Covers networks parts where traffic is most vulnerable.

Cons of a NIDS:

  • Expensive to set up.
  • If a NIDS must monitor an extensive or busy network, the system can suffer from low specificity and an occasional unnoticed breach.
  • Detecting threats within encrypted traffic can be problematic.
  • Typically not an ideal fit with switch-based networks.

Host Intrusion Detection System (HIDS)

A HIDS operates from a specific endpoint where it monitors network traffic and system logs to and from a single device.

This type of IDS security relies on regular snapshots, file sets that capture the entire system’s state. When the system takes a snapshot, the IDS compares it with the previous state and checks for missing or altered files or settings.

Pros of a HIDS

  • Offers deep visibility into the host device and its activity (changes to the configuration, permissions, files, registry, etc.).
  • An excellent second line of defense against a malicious packet a NIDS failed to detect.
  • Good at detecting packets originating from inside the organization, such as unauthorized changes to files from a system console.
  • Effective at detecting and preventing software integrity breaches.
  • Better at analyzing encrypted traffic than a NIDS due to less packets.
  • Far cheaper than setting up a NIDS.

Cons of a HIDS

  • Limited visibility as the system only monitors one device.
  • Less available context for decision-making.
  • Hard to manage for large companies as the team needs to configure and handle info for every host.
  • More visible to attackers than a NIDS.
  • Not good at detecting network scans or other network-wide surveillance attacks.

Signature-Based Intrusion Detection System (SIDS)

A SIDS monitors packets moving through a network and compares them to a database of known attack signatures or attributes. This common type of IDS security looks for specific patterns, such as byte or instruction sequences.

Pros of a SIDS

  • Works well against attackers using known attack signatures.
  • Helpful for discovering low-skill attack attempts.
  • Effective at monitoring inbound network traffic.
  • Can efficiently process a high volume of network traffic.

Cons of a SIDS

  • Cannot identify a breach without a specific signature in the threat database.
  • A savvy hacker can modify an attack to avoid matching known signatures, such as changing lowercase to uppercase letters or converting a symbol to its character code.
  • Requires regular updates of the threat database to keep the system up to date with the latest risks.

Anomaly-Based Intrusion Detection System (AIDS)

An AIDS goes beyond the attack signature model and detects malicious behavior patterns instead of specific data patterns.

This type of IDS uses machine learning to establish a baseline of expected system behavior (trust model) in terms of bandwidth, protocols, ports, and device usage. The system can then compare any new behavior to verified trust models and discover unknown attacks a signature-based IDS cannot identify.

For example, someone in the Sales department trying to access the website’s backend for the first time may not be a red flag for a SIDS. For an anomaly-based setup, however, a person trying to access a sensitive system for the first time is a cause for investigation.

Pros of an AIDS

  • Can detect signs of unknown attack types and novel threats.
  • Relies on machine learning and AI to establish a model of trustworthy behavior.

Cons of an AIDS

  • Complex to manage.
  • Requires more processing resources than a signature-based IDS.
  • High amounts of alarms can overwhelm admins.

IDS Strengths and Limitations

Using an IDS to protect a network is a valid strategy to boost security. When paired with a robust anti-malware program and firewall, an IDS ensures the team:

  • Stays ahead of a large percentage of cyber problems, whether caused by a malicious actor, accident, or error.
  • Does not need to comb through thousands of system logs for critical info.
  • Can reliably enforce company’s security policies at the network level.

IDSes (and even IPSes) are also becoming cheaper and easier to administer, so even SMBs with smaller budgets and less IT staff can rely on this strategy. Despite all the benefits, however, IDSes also have some unique challenges:

  • Avoiding an IDS is the priority for a successful attack, making these systems the go-to target for hackers.
  • Detecting malicious activity within encrypted traffic is a common issue.
  • An IDS can be less effective in a network with high amounts of traffic.
  • Even the best system can have problems recognizing signs of a novel attack.

The biggest challenge of an IDS is avoiding mistakes as even the best system can:

  • Raise the alarm for something that is not an attack (a false positive).
  • Fail to raise the alarm when there is a real threat (a false negative).

Too many false positives mean the IT team will be less confident of the IDS’s warnings. False negatives, however, mean that malicious packets are entering the network without raising an alarm, so an oversensitive IDS is always a better option.

IDS Best Practices

Once you know what IDS type and detection model you need to set up, ensure your strategy follows these best practices:

  • Baseline what normal network behavior looks like before you set up the IDS. Defining a clear initial baseline helps prevent false positives and false negatives.
  • Deploy the IDS at the highest point of visibility to not overwhelm the system with data.
  • Install multiple IDSes across the network if you need to deal with intra-host traffic.
  • Ensure the team setting up the IDS has a thorough understanding of your device inventory and each machine’s role.
  • Combine NIDS setups and network segmentation to make detections more effective and easier to manage.
  • Set the IDS up to run in stealth mode to make the system hard to detect for malicious actors. The simplest way to do so is to ensure the IDS has two network interfaces, one for the network and the other for generating alerts. The IDS should use the monitored interface as input only.
  • Once the IDS is up and running, your team should continually update the threat database to keep the system effective.
  • Consider adding a secondary analysis platform to analyze threats after the IDS raises the alarm.
  • Ensure all your IDSes and threat databases follow the principle of zero-trust security.

Do Not Overlook the Value of IDS Security

A high-quality IDS (or IPS) is vital to maintaining acceptable levels of network security. Set up a reliable intrusion detection system to ensure the network does not become an exploitable weakness in your cybersecurity strategy.