Keras: Sequential Model
The sequential API is used to create models layer-by-layer. It is limited in that it does not allow you to create models that share layers or have multiple inputs or outputs.
The Sequential model is a linear stack of layers.
We can create a Sequential model incrementally using the
model = keras.Sequential()
Also, it can be created using `Dense()` method as shown below:
layers.Dense(2, activation="relu", name="layer1"),
layers.Dense(3, activation="relu", name="layer2"),
What is Relu?
Relu stands for Rectified Linear Unit activation function.
If default values are passed, it returns the standard ReLU activation: max(x, 0), the element-wise maximum of 0 and the input tensor.
If default parameters are modified, it allows to use non-zero thresholds, change the max value of the activation, and to use a non-zero multiple of the input for values below the threshold.
Downsamples the input along its spatial dimensions (height and width) by taking the maximum value over an input window (of size defined by pool_size ) for each channel of the input. The window is shifted by strides along each dimension.
What is Covolution?
In simple words, a convolution is an operation that changes a function into something more meaningful. The convolution is used so that we can transform the original function so that we get more information.
2D convolution layer is the most common type of convolution that is used. It is a filter whose layer has a height and a width. They are generally smaller than the input image and so we move them across the whole image. The area where the filter is on the image is called the receptive field.
Batch normalization provides an elegant way of reparametrizing almost any deep network. The reparametrization significantly reduces the problem of coordinating updates across many layers.
Batch normalization can be implemented during training by calculating the mean and standard deviation of each input variable to a layer per mini-batch and using these statistics to perform the standardization.
Alternately, a running average of mean and standard deviation can be maintained across mini-batches, but may result in unstable training.
Generally, while fitting a model, a case occurs where the model overfits and the preiction made by the model often gets wrong. So, there is a technique used to prevent a model from overfitting which is Dropout. Dropout works by randomly setting the outgoing edges of hidden units to 0 at each update of the training phase.
Example, In passing 0.5, every hidden unit (neuron) is set to 0 with a probability of 0.5. In other words, there’s a 50% change that the output of a given neuron will be forced to 0.