Models

Pytorch Models for Sequential Data

from tsfast.datasets import create_dls_test

dls = create_dls_test()

Batchnorm

---

source

BatchNorm_1D_Stateful

 BatchNorm_1D_Stateful (hidden_size, seq_len=None, stateful=False,
                        batch_first=True, eps=1e-07, momentum=0.1,
                        affine=True, track_running_stats=True)

Batchnorm for stateful models. Stores batch statistics for for every timestep seperately to mitigate transient effects.

	Type	Default	Details
hidden_size
seq_len	NoneType	None
stateful	bool	False
batch_first	bool	True
eps	float	1e-07
momentum	float	0.1
affine	bool	True
track_running_stats	bool	True	num_features

Linear

---

source

SeqLinear

 SeqLinear (input_size, output_size, hidden_size=100, hidden_layer=1,
            act=<class 'torch.nn.modules.activation.Mish'>,
            batch_first=True)

*Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes::

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:to, etc.

.. note:: As per the example above, an __init__() call to the parent class must be made before assignment on the child.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool*

Autoregressive Models

---

source

Normalizer1D

 Normalizer1D (mean, std)

*Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes::

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:to, etc.

.. note:: As per the example above, an __init__() call to the parent class must be made before assignment on the child.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool*

---

source

AR_Model

 AR_Model (model, ar=True, stateful=False, model_has_state=False,
           return_state=False, out_sz=None)

Autoregressive model container which work autoregressively if the sequence y is not provided, otherwise it works as a normal model. This way it can be trained either with teacher forcing or with autoregression

model = AR_Model(SeqLinear(2,1),model_has_state=False,ar=True,out_sz=1)
model.init_normalize(dls.one_batch())
lrn = Learner(dls,model,loss_func=nn.MSELoss()).fit(1)

0.00% [0/1 00:00<?]

epoch	train_loss	valid_loss	time

0.00% [0/12 00:00<?]

Normalization Model

When we want to use a trained model in an environment without a dataloader the data given to the model has to be normalized by the model’s normalization layer.

---

source

NormalizedModel

 NormalizedModel (model, mean, std)

*Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes::

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:to, etc.

.. note:: As per the example above, an __init__() call to the parent class must be made before assignment on the child.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool*

model = SeqLinear(1,1)
model_norm = NormalizedModel.from_dls(model,dls)
lrn = Learner(dls,model_norm,loss_func=nn.MSELoss()).fit(1)

epoch	train_loss	valid_loss	time
0	0.101554	0.062500	00:00