optiml.ml.neural_network.initializers module

optiml.ml.neural_network.initializers.truncated_normal(shape, mean=0.0, std=1.0, random_state=None)

optiml.ml.neural_network.initializers.glorot_normal(shape, random_state=None)

Glorot normal initializer, also called Xavier normal initializer. It draws samples from a truncated normal distribution centered on 0 with

\[\text{std} = \sqrt{\frac{2}{\text{fan\_in} + \text{fan\_out}}}\]

where fan_in is the number of input units in the weight tensor and fan_out is the number of output units in the weight tensor.

optiml.ml.neural_network.initializers.glorot_uniform(shape, random_state=None)

Glorot uniform initializer, also called Xavier uniform initializer. It draws samples from a uniform distribution within \([-\text{limit}, \text{limit}]\) where

\[\text{limit} = \sqrt{\frac{6}{\text{fan\_in} + \text{fan\_out}}}\]

where fan_in is the number of input units in the weight tensor and fan_out is the number of output units in the weight tensor.

optiml.ml.neural_network.initializers.he_normal(shape, random_state=None)

He normal initializer. It draws samples from a truncated normal distribution centered on 0 with

\[\text{std} = \sqrt{\frac{2}{\text{fan\_in}}}\]

where fan_in is the number of input units in the weight tensor.

optiml.ml.neural_network.initializers.he_uniform(shape, random_state=None)

He uniform variance scaling initializer. It draws samples from a uniform distribution within \([-\text{limit}, \text{limit}]\) where

\[\text{limit} = \sqrt{\frac{6}{\text{fan\_in}}}\]

where fan_in is the number of input units in the weight tensor.