How to initialize weights in PyTorch?

Single layer

To initialize the weights of a single layer, use a function from torch.nn.init. For instance:

conv1 = torch.nn.Conv2d(...)

Alternatively, you can modify the parameters by writing to (which is a torch.Tensor). Example:

The same applies for biases:

nn.Sequential or custom nn.Module

Pass an initialization function to torch.nn.Module.apply. It will initialize the weights in the entire nn.Module recursively.

apply(fn): Applies fn recursively to every submodule (as returned by .children()) as well as self. Typical use includes initializing the parameters of a model (see also torch-nn-init).


def init_weights(m):
    if isinstance(m, nn.Linear):

net = nn.Sequential(nn.Linear(2, 2), nn.Linear(2, 2))

Leave a Comment