Deep Learning Quick Start Guide

This section is intended as a quick start guide for deep learning users. It is based on PyTorch examples, but it should be easy to follow even for people working with other frameworks like TensorFlow.

Not into deep learning?

Confugue is absolutely not limited to machine learning applications. Python users unfamiliar with deep learning should check out the General Guide.

Tip

This guide is available as a Jupyter notebook. Open in Colab

Basic PyTorch example

We are going to start with a basic PyTorch model, adapted from the CIFAR-10 tutorial. First, let’s see what the model looks like without using Confugue:

from torch import nn

class Net(nn.Module):

    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.fc1 = nn.Linear(400, 120)
        self.fc2 = nn.Linear(120, 10)
        self.act = nn.ReLU()

    def forward(self, x):
        x = self.pool(self.act(self.conv1(x)))
        x = self.pool(self.act(self.conv2(x)))
        x = x.flatten(start_dim=1)
        x = self.act(self.fc1(x))
        x = self.fc2(x)
        return x

Making it configurable

Instead of hard-coding all the hyperparameters like above, we want to be able to specify them in a configuration file. To do so, we are going to decorate our class with the @configurable decorator. This provides it with a magic _cfg property, giving it access to the configuration. We can then rewrite our __init__ as follows:

from confugue import configurable, Configuration

@configurable
class Net(nn.Module):

    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = self._cfg['conv1'].configure(nn.Conv2d, in_channels=3)
        self.conv2 = self._cfg['conv2'].configure(nn.Conv2d)
        self.pool = self._cfg['pool'].configure(nn.MaxPool2d)
        self.fc1 = self._cfg['fc1'].configure(nn.Linear)
        self.fc2 = self._cfg['fc2'].configure(nn.Linear, out_features=10)
        self.act = self._cfg['act'].configure(nn.ReLU)

    def forward(self, x):
        x = self.pool(self.act(self.conv1(x)))
        x = self.pool(self.act(self.conv2(x)))
        x = x.flatten(start_dim=1)
        x = self.act(self.fc1(x))
        x = self.fc2(x)
        return x

Instead of creating each layer directly, we configure it with values from the corresponding section of the configuration file (which we will see in a moment). Notice that we can still specify arguments in the code (e.g. in_channels=3 for the conv1 layer), but these are treated as defaults and can be overridden in the configuration file if needed.

Loading configuration from a YAML file

Calling Net() directly would result in an error, since we haven’t specified defaults for all the required parameters of each layer. We therefore need to create a configuration file config.yaml to supply them:

conv1:
  out_channels: 6
  kernel_size: 5
conv2:
  in_channels: 6
  out_channels: 16
  kernel_size: 5
pool:
  kernel_size: 2
  stride: 2
fc1:
  in_features: 400
  out_features: 120
fc2:
  in_features: 120

Note

We do not need to include the activation function (act), since it does not have any required parameters. We could, however, override the type of the activation function itself.

We are now ready to load the file into a Configuration object and use it to configure our network:

>>> cfg = Configuration.from_yaml_file('config.yaml')
>>> cfg
Configuration({'conv1': {'out_channels': 6, 'kernel_size': 5}, 'conv2': {'in_channels': 6, 'out_channels': 16, 'kernel_size': 5}, 'pool': {'kernel_size': 2, 'stride': 2}, 'fc1': {'in_features': 400, 'out_features': 120}, 'fc2': {'in_features': 120}})
>>> cfg.configure(Net)
Net(
  (conv1): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1))
  (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))
  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (fc1): Linear(in_features=400, out_features=120, bias=True)
  (fc2): Linear(in_features=120, out_features=10, bias=True)
  (act): ReLU()
)

Tip

Instead of loading a YAML file, one can use any configuration dictionary by directly calling Configuration(cfg_dict).

Nested configurables

One of the most useful features of Confugue is that @configurable classes and functions can use other configurables, and the structure of the configuration file will naturally follow this hierarchy. To see this in action, we are going to write a configurable main function which trains our simple model on the CIFAR-10 dataset.

import torchvision
from torchvision import transforms

@configurable
def main(num_epochs=1, log_period=2000, *, _cfg):
    net = _cfg['net'].configure(Net)
    criterion = _cfg['loss'].configure(nn.CrossEntropyLoss)
    optimizer = _cfg['optimizer'].configure(torch.optim.SGD, params=net.parameters(),
                                            lr=0.001)

    transform = transforms.Compose(
        [transforms.ToTensor(),
         transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
    train_data = torchvision.datasets.CIFAR10(root='./data', train=True,
                                              download=True, transform=transform)
    train_loader = _cfg['data_loader'].configure(torch.utils.data.DataLoader,
                                                 dataset=train_data, batch_size=4,
                                                 shuffle=True, num_workers=2)

    for epoch in range(num_epochs):
        for i, batch in enumerate(train_loader):
            inputs, labels = batch
            optimizer.zero_grad()
            loss = criterion(net(inputs), labels)
            loss.backward()
            optimizer.step()

            if (i + 1) % log_period == 0:
                print(i + 1, loss.item())

Our config.yaml might then look like this:

net:
  conv1:
    out_channels: 6
    kernel_size: 5
  conv2:
    in_channels: 6
    out_channels: 16
    kernel_size: 5
  pool:
    kernel_size: 2
    stride: 2
  fc1:
    in_features: 400
    out_features: 120
  fc2:
    in_features: 120

optimizer:
  class: !!python/name:torch.optim.Adam
data_loader:
  batch_size: 8
num_epochs: 2
log_period: 1000

To create and train our model:

cfg = Configuration.from_yaml_file('config.yaml')
cfg.configure(main)

Configuring lists

The configure_list method allows us to configure a list of objects, with the parameters for each supplied from the configuration file. We are going to use this, in conjunction with nn.Sequential, to fully specify the model in the configuration file, so we won’t need our Net class anymore.

layers:
  - class: !!python/name:torch.nn.Conv2d
    in_channels: 3
    out_channels: 6
    kernel_size: 5
  - class: !!python/name:torch.nn.ReLU
  - class: !!python/name:torch.nn.MaxPool2d
    kernel_size: 2
    stride: 2
  - class: !!python/name:torch.nn.Conv2d
    in_channels: 6
    out_channels: 16
    kernel_size: 5
  - class: !!python/name:torch.nn.ReLU
  - class: !!python/name:torch.nn.MaxPool2d
    kernel_size: 2
    stride: 2
  - class: !!python/name:torch.nn.Flatten
  - class: !!python/name:torch.nn.Linear
    in_features: 400
    out_features: 120
  - class: !!python/name:torch.nn.ReLU
  - class: !!python/name:torch.nn.Linear
    in_features: 120
    out_features: 10

Creating the model then becomes a matter of two lines of code:

>>> cfg = Configuration.from_yaml_file('config.yaml')
>>> nn.Sequential(*cfg['layers'].configure_list())
Sequential(
  (0): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1))
  (1): ReLU()
  (2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (3): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))
  (4): ReLU()
  (5): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (6): Flatten()
  (7): Linear(in_features=400, out_features=120, bias=True)
  (8): ReLU()
  (9): Linear(in_features=120, out_features=10, bias=True)
)

This offers a lot of flexibility, but it should be used with care. If your configuration file is longer than your code, you might be overusing it.

See also

Advanced features are described in More Features.