optiml.ml.neural_network.layers module

class optiml.ml.neural_network.layers.Layer

Bases: ABC

Base abstract class for all neural network layers. A layer implements the forward pass that maps its input to its output and the backward pass that back-propagates the error signal.

forward(X)

backward(delta)

class optiml.ml.neural_network.layers.ParamLayer(coef_shape, activation, coef_init, inter_init, coef_reg, inter_reg, fit_intercept, random_state=None)

Bases: Layer, ABC

Base abstract class for all layers with trainable parameters, i.e., a coefficient (weight) tensor and, optionally, an intercept (bias) tensor, each with its own initializer and regularizer.

Parameters:

• coef_shape (tuple of int) – Shape of the coefficient (weight) tensor.

• activation (Activation instance) – The activation function applied by the layer.

• coef_init (callable, array-like or None) – Initializer for the coefficient tensor. If None, glorot_uniform is used; if callable, it is called with coef_shape and random_state; otherwise it is used as the initial values.

• inter_init (callable, array-like or None) – Initializer for the intercept tensor. If None, zeros are used; if callable, it is called with the intercept shape; otherwise it is used as the initial values. Only used when fit_intercept is True.

• coef_reg (Regularizer instance or None) – Regularizer applied to the coefficient tensor. If None, l2 is used.

• inter_reg (Regularizer instance or None) – Regularizer applied to the intercept tensor. If None, l2 is used.

• fit_intercept (bool) – Whether the layer has an intercept (bias) term.

• random_state (int, RandomState instance or None, default=None) – Controls the pseudo random number generation for the parameters initialization.

backward(delta)

forward(X)

class optiml.ml.neural_network.layers.FullyConnected(n_in, n_out, activation=<optiml.ml.neural_network.activations.Linear object>, coef_init=<function glorot_uniform>, inter_init=<function primitive.<locals>.f_wrapped>, coef_reg=<optiml.ml.neural_network.regularizers.L2 object>, inter_reg=<optiml.ml.neural_network.regularizers.L2 object>, fit_intercept=True, random_state=None)

Bases: ParamLayer

Fully connected (dense) layer that computes activation(X @ W + b).

Parameters:

• n_in (int) – Number of input units (fan-in) of the layer.

• n_out (int) – Number of output units (fan-out) of the layer, i.e., the number of neurons.

• activation (Activation instance, default=linear) – The activation function applied by the layer.

• coef_init (callable or array-like, default=glorot_uniform) – Initializer for the coefficient (weight) tensor.

• inter_init (callable or array-like, default=np.zeros) – Initializer for the intercept (bias) tensor. Only used when fit_intercept is True.

• coef_reg (Regularizer instance, default=l2) – Regularizer applied to the coefficient tensor.

• inter_reg (Regularizer instance, default=l2) – Regularizer applied to the intercept tensor.

• fit_intercept (bool, default=True) – Whether to add an intercept (bias) term to the layer.

• random_state (int, RandomState instance or None, default=None) – Controls the pseudo random number generation for the parameters initialization.

forward(X)

backward(delta)