TensorFlow: how is dataset.train.next_batch defined?

The mnist object is returned from the read_data_sets() function defined in the tf.contrib.learn module. The mnist.train.next_batch(batch_size) method is implemented here, and it returns a tuple of two arrays, where the first represents a batch of batch_size MNIST images, and the second represents a batch of batch-size labels corresponding to those images. The images are returned … Read more

numpy : calculate the derivative of the softmax function

I am assuming you have a 3-layer NN with W1, b1 for is associated with the linear transformation from input layer to hidden layer and W2, b2 is associated with linear transformation from hidden layer to output layer. Z1 and Z2 are the input vector to the hidden layer and output layer. a1 and a2 represents the output of the hidden layer and output layer. a2 is your predicted output. delta3 and delta2 are the … Read more

How to initialize weights in PyTorch?

Single layer To initialize the weights of a single layer, use a function from torch.nn.init. For instance: Alternatively, you can modify the parameters by writing to conv1.weight.data (which is a torch.Tensor). Example: The same applies for biases: nn.Sequential or custom nn.Module Pass an initialization function to torch.nn.Module.apply. It will initialize the weights in the entire … Read more