Neural Network Model

This page explains the deep neural network model in CIDS-Sim. The hidden layers follow a funnel-shaped architecture, progressively reducing the number of neurons by half in each subsequent layer, starting from a count equal to the dataset’s input features. This configuration allows initial layers to capture general patterns, while deeper layers distill these into higher-level, discriminative features. The hierarchical compression promotes efficient learning of abstract representations, enhancing the model’s ability to generalize to unseen data by focusing on essential patterns and discarding noise.

1. Function: create_model(input_shape)

Purpose:

Define and compile a sequential neural network model for binary classification tasks.

Code:

def create_model(input_shape):
    model = keras.Sequential([
        layers.Dense(20, activation='relu', input_shape=(input_shape,)),
        layers.Dense(10, activation='relu'),
        layers.Dense(5, activation='relu'),
        layers.Dense(3, activation='relu'),
        layers.Dense(1, activation='sigmoid')
    ])
    model.compile(
        loss='mean_squared_error',
        optimizer='sgd',
        metrics=['accuracy', Recall(), Precision()]
    )
    return model

Details:

2. Model Architecture

  • Layer 1:

    • Dense(20, activation='relu', input_shape=(input_shape,))

    • A fully connected layer with 20 neurons and ReLU activation.

    • input_shape parameter defines the feature dimensions of the input data (e.g., number of features in X_df_scl).

  • Layer 2:

    • Dense(10, activation='relu')

    • A hidden layer with 10 neurons and ReLU activation.

  • Layer 3:

    • Dense(5, activation='relu')

    • A hidden layer with 5 neurons and ReLU activation.

  • Layer 4:

    • Dense(3, activation='relu')

    • A hidden layer with 3 neurons and ReLU activation.

  • Output Layer:

    • Dense(1, activation='sigmoid')

    • A single neuron with sigmoid activation for binary classification (outputs probabilities between 0 and 1).

3. Model Compilation

  • Loss Function:

    • loss='mean_squared_error' (MSE)

    • Typically used for regression tasks, but here applied to binary classification.

    • For better performance in classification, consider replacing with binary_crossentropy.

  • Optimizer:

    • optimizer='sgd' (Stochastic Gradient Descent)

    • A basic optimizer for weight updates during training. Alternatives include adam or rmsprop.

  • Metrics:

    • metrics=['accuracy', Recall(), Precision()]

    • Tracks classification accuracy, recall (sensitivity), and precision during training/evaluation.

    • Assumes Recall and Precision are imported from tensorflow.keras.metrics.

4. Customization Notes

  • Adjusting Layers:

    • Modify the number of neurons (e.g., Dense(30, ...)) or add/remove layers to experiment with model capacity.

    • Example: Replace layers.Dense(3, activation='relu') with layers.Dense(8, activation='tanh').

  • Activation Functions:

    • Replace relu with alternatives like tanh, leaky_relu, or selu for experimentation.

    • Ensure the output layer uses sigmoid for binary classification.

  • Compilation Parameters:

    • Change the loss to binary_crossentropy for standard classification tasks.

    • Experiment with optimizers (e.g., optimizer='adam') or learning rate tuning.

5. Dependencies

  • Requires tensorflow or keras for model definition.

  • Ensure Recall and Precision are imported:

    from tensorflow.keras.metrics import Recall, Precision

  • Assumes input data (X_df_scl) has already been scaled and preprocessed.