Activation Layers

custom_neural_net_creator.activation_layer

class custom_neural_net_creator.activation_layer.ActivationLayer(activation_function: relu | sigmoid | tanh, activation_derivative: relu_derivative | sigmoid_derivative | tanh_prime)

Bases: object

backward_prop(dE_dY: ndarray[Any, dtype[_ScalarType_co]], learning_rate: float) → ndarray[Any, dtype[_ScalarType_co]]

This method performs backward propagation through the Activation Layer. It computes the gradient of the error with respect to the layer’s input by multiplying the activation derivative with ∂E/∂Y. Since there are no parameters to optimize in this layer, it directly returns the gradient of the error with respect to the input, which can be used for backpropagation in previous layers.

Parameters:

• dE_dY (npt.NDArray) – The gradient of the error function.

• learning_rate (float) – The learning rate for perorming gradient descent.

Returns:

The gradient of the error with respect to current layer’s input.

Return type:

npt.NDArray

forward_prop(x: ndarray[Any, dtype[_ScalarType_co]]) → ndarray[Any, dtype[_ScalarType_co]]

Perform forward propagation through the Activation Layer.

This method computes the forward propagation of input data through the Activation Layer. It saves the input x for later use in backpropagation and passes the input through the specified activation function. The result is returned as the output of the layer.

Parameters:

x (npt.NDArray) – The input data for the activation layer.

Returns:

The output of the activation layer.

Return type:

npt.NDArray