Adding layers in Tensor Flow - Trabajando varias capas en Tensor Flow

Adding layers in Tensor Flow - Trabajando varias capas en Tensor Flow
https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#6

7. Lab: adding layers

To improve the recognition accuracy we will add more layers to the neural network. The neurons in the second layer, instead of computing weighted sums of pixels will compute weighted sums of neuron outputs from the previous layer. Here is for example a 5-layer fully connected neural network:

We keep softmax as the activation function on the last layer because that is what works best for classification. On intermediate layers however we will use the the most classical activation function: the sigmoid:

Your task in this section is to add one or two intermediate layers to your model to increase its performance.
The solution can be found in file mnist_2.0_five_layers_sigmoid.py. Use it if you are stuck only!

To add a layer, you need an additional weights matrix and an additional bias vector for the intermediate layer:

W1 = tf.Variable(tf.truncated_normal([28*28, 200] ,stddev=0.1))
B1 = tf.Variable(tf.zeros([200]))

W2 = tf.Variable(tf.truncated_normal([200, 10], stddev=0.1))
B2 = tf.Variable(tf.zeros([10]))

The shape of the weights matrix for a layer is [N, M] where N is the number of inputs and M of outputs for the layer. In the code above, we use 200 neurons in the intermediate layer and still 10 neurons in the last layer.

Tip: as you go deep, it becomes important to initialise weights with random values. The optimiser can get stuck in its initial position if you do not. tf.truncated_normal is a TensorFlow function that produces random values following the normal (Gaussian) distribution between -2*stddev and +2*stddev.

And now change your 1-layer model into a 2-layer model:

XX = tf.reshape(X, [-1, 28*28])

Y1 = tf.nn.sigmoid(tf.matmul(XX, W1) + B1)
Y  = tf.nn.softmax(tf.matmul(Y1, W2) + B2)

That's it. You should now be able to push your network above 97% accuracy with 2 intermediate layer with for example 200 and 100 neurons.