AI-Powered IDS
Mohammed Alshukaili
- 3 min read

AI-Powered IDS

On this page
Introduction

In this blog post, I will walk you through the implementation of an AI-powered Intrusion Detection System (IDS) using machine learning techniques. I will cover the preprocessing of the dataset, building a neural network model, training the model, and evaluating its performance.

Importing Necessary Libraries

First, I need to import the necessary libraries for our project. These include libraries for data manipulation, visualization, and building neural networks.

import torch
import torch.nn as nn
import torch.nn.functional as F
import matplotlib.pyplot as plt 
import pandas as pd
%matplotlib inline

Loading the Dataset

Next, I load the training and testing datasets. These datasets contain network traffic data which we will use to train and evaluate our model.

df = pd.read_csv('./archive/KDDTrain+.txt', header=None)  
test_df = pd.read_csv('./archive/KDDTest+.txt', header=None)  
columns = [
    'duration', 'protocol_type', 'service', 'flag', 'src_bytes', 'dst_bytes', 'land',
    'wrong_fragment', 'urgent', 'hot', 'num_failed_logins', 'logged_in', 'num_compromised',
    'root_shell', 'su_attempted', 'num_root', 'num_file_creations', 'num_shells',
    'num_access_files', 'num_outbound_cmds', 'is_host_login', 'is_guest_login', 'count',
    'srv_count', 'serror_rate', 'srv_serror_rate', 'rerror_rate', 'srv_rerror_rate',
    'same_srv_rate', 'diff_srv_rate', 'srv_diff_host_rate', 'dst_host_count',
    'dst_host_srv_count', 'dst_host_same_srv_rate', 'dst_host_diff_srv_rate',
    'dst_host_same_src_port_rate', 'dst_host_srv_diff_host_rate', 'dst_host_serror_rate',
    'dst_host_srv_serror_rate', 'dst_host_rerror_rate', 'dst_host_srv_rerror_rate',
    'attack', 'level'
]
df.columns = columns
test_df.columns = columns

Data Preprocessing

To prepare the data for training, I convert the attack column to a binary format, where 'normal' traffic is labeled as 0 and all other traffic is labeled as 1. I also encode categorical variables.

df['attack_binary'] = df.attack.map(lambda a: 0 if a == 'normal' else 1)
df.drop('attack', axis=1, inplace=True)

test_df['attack_binary'] = test_df.attack.map(lambda a: 0 if a == 'normal' else 1)
test_df.drop('attack', axis=1, inplace=True)

from sklearn import preprocessing
le = preprocessing.LabelEncoder()
clm = ['protocol_type', 'service', 'flag']
for x in clm:
    df[x] = le.fit_transform(df[x])
    test_df[x] = le.fit_transform(test_df[x])

Feature Selection

I select specific features for our model to train on.

features = ['service', 'flag', 'src_bytes', 'dst_bytes', 'logged_in', 'count',
       'serror_rate', 'srv_serror_rate', 'same_srv_rate', 'diff_srv_rate',
       'dst_host_srv_count', 'dst_host_same_srv_rate',
       'dst_host_diff_srv_rate', 'dst_host_serror_rate',
       'dst_host_srv_serror_rate']
X = df[features]
y = df['attack_binary']
X = X.values
y = y.values

Splitting the Dataset

I split the dataset into training and testing sets.

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=41)
X_train = torch.FloatTensor(X_train)
X_test = torch.FloatTensor(X_test)
y_train = torch.LongTensor(y_train)
y_test = torch.LongTensor(y_test)

Building the Neural Network Model

I define our neural network architecture.

class Model(nn.Module):
    def __init__(self, in_features=15, h1=30, h2=30, h3=30, out_features=2):
        super().__init__()
        self.fc1 = nn.Linear(in_features, h1)
        self.fc2 = nn.Linear(h1, h2)
        self.fc3 = nn.Linear(h2, h3)
        self.out = nn.Linear(h3, out_features)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = F.relu(self.fc3(x))
        x = self.out(x)
        return x

Training the Model

I initialize the model, define the loss function and optimizer, and then train the model.

torch.manual_seed(41)
model = Model()
weights = torch.tensor([0.5, 3.0], dtype=torch.float32)  # Increase the weight for the 'attack' class
criterion = nn.CrossEntropyLoss(weight=weights)
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
epochs = 600
losses = []
for i in range(epochs):
    y_pred = model.forward(X_train)
    loss = criterion(y_pred, y_train)
    losses.append(loss.detach().numpy())
    if i % 10 == 0:
        print(f'Epoch: {i} and loss: {loss}')
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
plt.plot(range(epochs), losses)
plt.ylabel("loss/errors")
plt.xlabel('Epoch')

Evaluating the Model

I evaluate the model's performance on the test set.

with torch.no_grad():
    y_eval = model.forward(X_test)
    loss = criterion(y_eval, y_test)
correct = 0 
with torch.no_grad():
    for i, data in enumerate(X_test):
     y_val = model.forward(data)
     print(f'{i+1}.) {str(y_val)} \t {y_test[i]} \t {y_val.argmax().item()}')
     if y_val.argmax().item() == y_test[i]:
        correct+=1
print(correct)

Model Metrics

I calculate accuracy, precision, recall, and plot the confusion matrix.

import torch
from sklearn.metrics import accuracy_score, classification_report, precision_score, recall_score
model.eval()
predictions = []
labels = []
with torch.no_grad():
    for data, label in zip(X_test, y_test):
        y_val = model(data.unsqueeze(0))  
        _, predicted = torch.max(y_val, dim=1)
        predictions.append(predicted.item())
        labels.append(label.item())

accuracy = accuracy_score(labels, predictions)
precision = precision_score(labels, predictions, average='weighted')
recall = recall_score(labels, predictions, average='weighted')
report = classification_report(labels, predictions)
print(f'Accuracy: {accuracy * 100:.2f}%')
print(f'Precision: {precision * 100:.2f}%')
print(f'Recall: {recall * 100:.2f}%')
print("Classification Report:")
print(report)

Confusion Matrix

Finally, I will visualize the confusion matrix to understand the performance of our model better.

from sklearn.metrics import classification_report, accuracy_score, precision_score, recall_score, confusion_matrix
import seaborn as sns

neuralnetwork = confusion_matrix(labels, predictions)
plt.figure(figsize=(8, 6))
sns.heatmap(neuralnetwork, annot=True, fmt='d', cmap='Blues', xticklabels=['Negative', 'Positive'], yticklabels=['Negative', 'Positive'])
plt.xlabel('Predicted Label')
plt.ylabel('True Label')
plt.title('Confusion Matrix')
plt.show()

Conclusion

In this blog post, I built an AI-powered Intrusion Detection System using machine learning techniques. I walked through the steps of data preprocessing, model building, training, and evaluation. By following these steps, you can create a robust IDS to enhance the security of network systems.

Post In:
AI