What One Network Feature Taught Me About Context in IoT Intrusion Detection

Why an "important" feature in machine learning may stop being important when the network protocol changes

When a machine-learning model achieves more than 99% accuracy, it is tempting to make that the headline.

While working on our recent IoT intrusion-detection research, however, one result interested me even more than the overall model accuracy.

It involved a single network feature called state.

For one protocol group in the Bot-IoT dataset, state produced a CorrAUC score of 0.8765, making it the strongest discriminator in that subset.

For another protocol group, the same feature scored only 0.1090.

Almost an eight-fold difference.

The feature had not changed.

The protocol context had.

Why this caught my attention

Many machine-learning-based intrusion detection systems follow a familiar pipeline:

1. Collect network traffic.
2. Select the most useful features.
3. Train a classifier.
4. Evaluate its performance.
5. Deploy the model.

Feature selection is often performed across the complete dataset, producing one global list of "important" variables.

But computer networks are not homogeneous environments.

Different protocols have different behaviours, traffic characteristics, communication patterns, and attack surfaces.

That raises a simple question:

Should we ask only which features are important — or also which features are important for a particular protocol?

Our results suggest that the second question deserves more attention.

Looking at feature importance through protocol context

In our study, Protocol-Aware IoT Intrusion Detection Using Random Forest and CorrAUC, we investigated whether protocol-conditioned feature analysis could reveal information hidden by global feature ranking.

For one Bot-IoT protocol subset, the leading CorrAUC-ranked features included:

• state — 0.8765
• dur — 0.7825
• max — 0.7710

But when the protocol changed, the discriminatory behaviour of these features changed as well.

The clearest example was state, falling from 0.8765 to 0.1090.

This suggests that a globally strong network feature may not necessarily remain equally informative across different protocol contexts.

For intrusion detection, that matters.

A system that understands protocol context may be able to make more informed decisions about which traffic characteristics deserve attention.

The classifier still has to perform

Alongside the protocol-aware analysis, we trained and evaluated several machine-learning classifiers.

The optimized Random Forest achieved:

• 99.43% accuracy
• 99.71% precision
• 99.15% recall
• 99.43% F1-score
• 0.28% false-positive rate
• AUC = 0.9944

XGBoost achieved comparable performance, while Logistic Regression and SVM performed considerably worse under the experimental configuration.

But I think one metric deserves particular attention in cybersecurity:

The false-positive rate

An intrusion-detection system can have excellent headline accuracy and still create operational problems if it repeatedly flags legitimate traffic as malicious.

In our 704-sample balanced test set, the Random Forest produced:

• 351 true negatives
• 349 true positives
• 1 false positive
• 3 false negatives

For a security analyst, that distinction is important.

A useful IDS should not only detect attacks — it should also avoid constantly "crying wolf."

Moving beyond the notebook

Another question we considered was whether the model could move beyond offline experimentation.

We packaged the trained IDS as a Docker-based Flask API and tested prediction requests through a /predict endpoint.

The deployment validation successfully returned HTTP 200 responses while maintaining lightweight resource usage.

This was not intended to claim production readiness.

Instead, it was an important engineering checkpoint.

For me, applied cybersecurity research should increasingly move through three stages:

Model performance → reproducibility → deployment feasibility

A model that achieves excellent results in a notebook is valuable.

A model that can also be packaged, served, tested, and eventually integrated into a real monitoring environment becomes much more interesting.

What are the limitations?

High performance should always be interpreted carefully.

Our current experiment was validated exclusively on the Bot-IoT dataset, using a balanced 704-sample test set.

Bot-IoT represents cyber-range traffic rather than a live production IoT network.

The protocol-conditioned analysis also focused on selected protocol subsets rather than every possible network protocol.

These limitations naturally lead to the next research questions:

• Will protocol-conditioned feature behaviour remain consistent across other datasets?
• Can the approach generalise to previously unseen network environments?
• How should an IDS adapt when traffic behaviour changes over time?
• Can protocol awareness improve streaming and online intrusion detection?
• Can this approach reduce false positives under realistic network conditions?

These are directions I am continuing to explore.

The lesson I took from this experiment

The strongest lesson was not simply that Random Forest achieved 99.43% accuracy.

It was something more fundamental:

Context can change the meaning of a feature.

In cybersecurity machine learning, we often spend significant effort asking:

Which algorithm performs best?

But sometimes another question may be equally important:

What does the network context tell us about the data before the algorithm even makes its decision?

That is where protocol-aware intrusion detection becomes particularly interesting.

Based on my research: Protocol-Aware IoT Intrusion Detection Using Random Forest and CorrAUC