optiml.ml.neural_network package

Submodules

• optiml.ml.neural_network.activations module
  • Activation
    • Activation.function()
    • Activation.jacobian()
  • Linear
    • Linear.function()
    • Linear.jacobian()
  • ReLU
    • ReLU.function()
    • ReLU.jacobian()
  • Tanh
    • Tanh.function()
    • Tanh.jacobian()
  • Sigmoid
    • Sigmoid.function()
    • Sigmoid.jacobian()
  • SoftMax
    • SoftMax.function()
    • SoftMax.jacobian()
• optiml.ml.neural_network.initializers module
  • truncated_normal()
  • glorot_normal()
  • glorot_uniform()
  • he_normal()
  • he_uniform()
• optiml.ml.neural_network.layers module
  • Layer
    • Layer.forward()
    • Layer.backward()
  • ParamLayer
    • ParamLayer.backward()
    • ParamLayer.forward()
  • FullyConnected
    • FullyConnected.forward()
    • FullyConnected.backward()
• optiml.ml.neural_network.losses module
  • NeuralNetworkLoss
    • NeuralNetworkLoss.args()
    • NeuralNetworkLoss.loss()
    • NeuralNetworkLoss.delta()
    • NeuralNetworkLoss.function()
    • NeuralNetworkLoss.jacobian()
    • NeuralNetworkLoss.f_star()
    • NeuralNetworkLoss.function_jacobian()
    • NeuralNetworkLoss.hessian()
    • NeuralNetworkLoss.x_star()
  • MeanSquaredError
    • MeanSquaredError.x_star()
    • MeanSquaredError.f_star()
    • MeanSquaredError.loss()
    • MeanSquaredError.args()
    • MeanSquaredError.delta()
    • MeanSquaredError.function()
    • MeanSquaredError.function_jacobian()
    • MeanSquaredError.hessian()
    • MeanSquaredError.jacobian()
  • MeanAbsoluteError
    • MeanAbsoluteError.loss()
    • MeanAbsoluteError.delta()
    • MeanAbsoluteError.args()
    • MeanAbsoluteError.f_star()
    • MeanAbsoluteError.function()
    • MeanAbsoluteError.function_jacobian()
    • MeanAbsoluteError.hessian()
    • MeanAbsoluteError.jacobian()
    • MeanAbsoluteError.x_star()
  • BinaryCrossEntropy
    • BinaryCrossEntropy.loss()
    • BinaryCrossEntropy.args()
    • BinaryCrossEntropy.delta()
    • BinaryCrossEntropy.f_star()
    • BinaryCrossEntropy.function()
    • BinaryCrossEntropy.function_jacobian()
    • BinaryCrossEntropy.hessian()
    • BinaryCrossEntropy.jacobian()
    • BinaryCrossEntropy.x_star()
  • CategoricalCrossEntropy
    • CategoricalCrossEntropy.loss()
    • CategoricalCrossEntropy.delta()
    • CategoricalCrossEntropy.args()
    • CategoricalCrossEntropy.f_star()
    • CategoricalCrossEntropy.function()
    • CategoricalCrossEntropy.function_jacobian()
    • CategoricalCrossEntropy.hessian()
    • CategoricalCrossEntropy.jacobian()
    • CategoricalCrossEntropy.x_star()
  • SparseCategoricalCrossEntropy
    • SparseCategoricalCrossEntropy.loss()
    • SparseCategoricalCrossEntropy.delta()
    • SparseCategoricalCrossEntropy.args()
    • SparseCategoricalCrossEntropy.f_star()
    • SparseCategoricalCrossEntropy.function()
    • SparseCategoricalCrossEntropy.function_jacobian()
    • SparseCategoricalCrossEntropy.hessian()
    • SparseCategoricalCrossEntropy.jacobian()
    • SparseCategoricalCrossEntropy.x_star()
  • mean_squared_error
  • mean_absolute_error
  • binary_cross_entropy
  • categorical_cross_entropy
  • sparse_categorical_cross_entropy
• optiml.ml.neural_network.regularizers module
  • Regularizer
    • Regularizer.function()
    • Regularizer.jacobian()
  • L1
    • L1.function()
    • L1.jacobian()
  • L2
    • L2.function()
    • L2.jacobian()

Module contents

class optiml.ml.neural_network.NeuralNetworkClassifier(layers=(), loss=<class 'optiml.ml.neural_network.losses.MeanSquaredError'>, optimizer=<class 'optiml.opti.unconstrained.stochastic.gradient_descent.StochasticGradientDescent'>, learning_rate=0.01, max_iter=1000, momentum_type='none', momentum=0.9, tol=0.0001, validation_split=0.0, batch_size=None, max_f_eval=15000, early_stopping=False, patience=5, shuffle=True, random_state=None, mu=1, master_solver='clarabel', master_verbose=False, verbose=False)

Bases: ClassifierMixin, NeuralNetwork

Feed-forward neural network for classification. The output layer must be sigmoid (binary/multi-label) or softmax (multi-class), consistently with the chosen loss function.

fit(X, y)

predict(X)

score(X, y, sample_weight=None)

Return accuracy on provided data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Test samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.

• sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

Returns:

score – Mean accuracy of self.predict(X) w.r.t. y.

Return type:

float

backward(delta)

property coefs_

forward(X)

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routing – A MetadataRequest encapsulating routing information.

Return type:

MetadataRequest

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

property intercepts_

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → NeuralNetworkClassifier

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.

Returns:

self – The updated object.

Return type:

object

class optiml.ml.neural_network.NeuralNetworkRegressor(layers=(), loss=<class 'optiml.ml.neural_network.losses.MeanSquaredError'>, optimizer=<class 'optiml.opti.unconstrained.stochastic.gradient_descent.StochasticGradientDescent'>, learning_rate=0.01, max_iter=1000, momentum_type='none', momentum=0.9, tol=0.0001, validation_split=0.0, batch_size=None, max_f_eval=15000, early_stopping=False, patience=5, shuffle=True, random_state=None, mu=1, master_solver='clarabel', master_verbose=False, verbose=False)

Bases: RegressorMixin, NeuralNetwork

Feed-forward neural network for regression. The output layer must be linear or, for regression between 0 and 1, sigmoid. The number of output neurons must equal the number of targets.

fit(X, y)

predict(X)

backward(delta)

property coefs_

forward(X)

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routing – A MetadataRequest encapsulating routing information.

Return type:

MetadataRequest

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

property intercepts_

score(X, y, sample_weight=None)

Return coefficient of determination on test data.

The coefficient of determination, \(R^2\), is defined as \((1 - \frac{u}{v})\), where \(u\) is the residual sum of squares ((y_true - y_pred)** 2).sum() and \(v\) is the total sum of squares ((y_true - y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a \(R^2\) score of 0.0.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

Returns:

score – \(R^2\) of self.predict(X) w.r.t. y.

Return type:

float

Notes

The \(R^2\) score used when calling score on a regressor uses multioutput='uniform_average' from version 0.23 to keep consistent with default value of r2_score(). This influences the score method of all the multioutput regressors (except for MultiOutputRegressor).

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → NeuralNetworkRegressor

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.

Returns:

self – The updated object.

Return type:

object