Example 06: Custom Normalization¶
Neural networks train best when inputs are properly scaled. TSFast normalizes inputs by default using z-score normalization. This example shows all built-in scalers, how to swap and compare them on a benchmark, and how to create a custom scaler.
Setup¶
In [1]:
Copied!
import torch
from tsfast.tsdata.benchmark import create_dls_silverbox
from tsfast.models._core.scaling import Scaler, StandardScaler, MinMaxScaler, MaxAbsScaler
from tsfast.training import RNNLearner, fun_rmse
import torch
from tsfast.tsdata.benchmark import create_dls_silverbox
from tsfast.models._core.scaling import Scaler, StandardScaler, MinMaxScaler, MaxAbsScaler
from tsfast.training import RNNLearner, fun_rmse
Why Normalization Matters¶
Neural networks learn best when input features are on similar scales. When features have large or mismatched magnitudes -- say, a pressure in pascals (~1e5) next to a valve position in [0, 1] -- the large features dominate gradient updates, causing slow or unstable training. TSFast therefore normalizes input signals by default.
The Silverbox signals used below are already well-scaled (inputs within about +/-0.1, outputs within +/-0.22), so don't expect dramatic differences between scalers here. The comparison demonstrates the mechanism for swapping scalers and shows that on data like this the choice is minor -- the payoff comes when your signals arrive in raw physical units with very different magnitudes.
Load the Dataset¶
In [2]:
Copied!
dls = create_dls_silverbox(bs=16, win_sz=500, stp_sz=10)
dls = create_dls_silverbox(bs=16, win_sz=500, stp_sz=10)
Built-in Scalers¶
TSFast provides three built-in scalers:
StandardScaler(default): z-score normalization.x_norm = (x - mean) / stdMinMaxScaler: scales to [0, 1].x_norm = (x - min) / (max - min)MaxAbsScaler: scales to [-1, 1].x_norm = x / max(|min|, |max|)
Training with Different Scalers¶
Train with each scaler. All models use the same architecture and training
schedule, so the only difference is the scaler. Each run's validate()
result goes into a dict; a summary table at the end compares them all.
In [3]:
Copied!
results = {}
results = {}
In [4]:
Copied!
lrn_std = RNNLearner(dls, rnn_type='lstm', metrics=[fun_rmse])
lrn_std.fit_flat_cos(n_epoch=5, lr=3e-3)
results['StandardScaler'] = lrn_std.validate()
lrn_std = RNNLearner(dls, rnn_type='lstm', metrics=[fun_rmse])
lrn_std.fit_flat_cos(n_epoch=5, lr=3e-3)
results['StandardScaler'] = lrn_std.validate()
Epoch 1/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 1/5: 50%|█████ | 151/300 [00:00<00:00, 301.44it/s]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 301.44it/s, train=0.0111 | valid=0.0050 | fun_rmse=0.0101]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 345.17it/s, train=0.0111 | valid=0.0050 | fun_rmse=0.0101]
Epoch 2/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 2/5: 69%|██████▊ | 206/300 [00:00<00:00, 410.99it/s]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 410.99it/s, train=0.0053 | valid=0.0035 | fun_rmse=0.0095]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 407.44it/s, train=0.0053 | valid=0.0035 | fun_rmse=0.0095]
Epoch 3/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 3/5: 68%|██████▊ | 204/300 [00:00<00:00, 407.63it/s]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 407.63it/s, train=0.0054 | valid=0.0057 | fun_rmse=0.0104]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 403.87it/s, train=0.0054 | valid=0.0057 | fun_rmse=0.0104]
Epoch 4/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 4/5: 69%|██████▊ | 206/300 [00:00<00:00, 411.94it/s]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 411.94it/s, train=0.0051 | valid=0.0042 | fun_rmse=0.0097]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 409.97it/s, train=0.0051 | valid=0.0042 | fun_rmse=0.0097]
Epoch 5/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 5/5: 68%|██████▊ | 204/300 [00:00<00:00, 406.28it/s]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 406.28it/s, train=0.0036 | valid=0.0029 | fun_rmse=0.0095]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 405.50it/s, train=0.0036 | valid=0.0029 | fun_rmse=0.0095]
In [5]:
Copied!
lrn_mm = RNNLearner(dls, rnn_type='lstm', input_norm=MinMaxScaler, metrics=[fun_rmse])
lrn_mm.fit_flat_cos(n_epoch=5, lr=3e-3)
results['MinMaxScaler'] = lrn_mm.validate()
lrn_mm = RNNLearner(dls, rnn_type='lstm', input_norm=MinMaxScaler, metrics=[fun_rmse])
lrn_mm.fit_flat_cos(n_epoch=5, lr=3e-3)
results['MinMaxScaler'] = lrn_mm.validate()
Epoch 1/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 1/5: 67%|██████▋ | 202/300 [00:00<00:00, 403.89it/s]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 403.89it/s, train=0.0222 | valid=0.0061 | fun_rmse=0.0112]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 401.84it/s, train=0.0222 | valid=0.0061 | fun_rmse=0.0112]
Epoch 2/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 2/5: 67%|██████▋ | 202/300 [00:00<00:00, 402.72it/s]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 402.72it/s, train=0.0065 | valid=0.0066 | fun_rmse=0.0112]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 402.30it/s, train=0.0065 | valid=0.0066 | fun_rmse=0.0112]
Epoch 3/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 3/5: 68%|██████▊ | 205/300 [00:00<00:00, 408.95it/s]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 408.95it/s, train=0.0065 | valid=0.0094 | fun_rmse=0.0135]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 405.71it/s, train=0.0065 | valid=0.0094 | fun_rmse=0.0135]
Epoch 4/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 4/5: 68%|██████▊ | 205/300 [00:00<00:00, 409.83it/s]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 409.83it/s, train=0.0057 | valid=0.0066 | fun_rmse=0.0110]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 407.90it/s, train=0.0057 | valid=0.0066 | fun_rmse=0.0110]
Epoch 5/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 5/5: 68%|██████▊ | 203/300 [00:00<00:00, 404.37it/s]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 404.37it/s, train=0.0041 | valid=0.0030 | fun_rmse=0.0095]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 404.49it/s, train=0.0041 | valid=0.0030 | fun_rmse=0.0095]
In [6]:
Copied!
lrn_ma = RNNLearner(dls, rnn_type='lstm', input_norm=MaxAbsScaler, metrics=[fun_rmse])
lrn_ma.fit_flat_cos(n_epoch=5, lr=3e-3)
results['MaxAbsScaler'] = lrn_ma.validate()
lrn_ma = RNNLearner(dls, rnn_type='lstm', input_norm=MaxAbsScaler, metrics=[fun_rmse])
lrn_ma.fit_flat_cos(n_epoch=5, lr=3e-3)
results['MaxAbsScaler'] = lrn_ma.validate()
Epoch 1/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 1/5: 68%|██████▊ | 203/300 [00:00<00:00, 404.10it/s]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 404.10it/s, train=0.0136 | valid=0.0042 | fun_rmse=0.0100]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 406.41it/s, train=0.0136 | valid=0.0042 | fun_rmse=0.0100]
Epoch 2/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 2/5: 69%|██████▊ | 206/300 [00:00<00:00, 411.04it/s]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 411.04it/s, train=0.0056 | valid=0.0054 | fun_rmse=0.0105]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 409.18it/s, train=0.0056 | valid=0.0054 | fun_rmse=0.0105]
Epoch 3/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 3/5: 68%|██████▊ | 205/300 [00:00<00:00, 409.65it/s]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 409.65it/s, train=0.0054 | valid=0.0060 | fun_rmse=0.0108]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 407.36it/s, train=0.0054 | valid=0.0060 | fun_rmse=0.0108]
Epoch 4/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 4/5: 68%|██████▊ | 205/300 [00:00<00:00, 408.38it/s]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 408.38it/s, train=0.0052 | valid=0.0072 | fun_rmse=0.0115]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 406.56it/s, train=0.0052 | valid=0.0072 | fun_rmse=0.0115]
Epoch 5/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 5/5: 69%|██████▉ | 207/300 [00:00<00:00, 412.87it/s]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 412.87it/s, train=0.0034 | valid=0.0029 | fun_rmse=0.0096]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 408.60it/s, train=0.0034 | valid=0.0029 | fun_rmse=0.0096]
In [7]:
Copied!
lrn_none = RNNLearner(dls, rnn_type='lstm', input_norm=None, metrics=[fun_rmse])
lrn_none.fit_flat_cos(n_epoch=5, lr=3e-3)
results['None'] = lrn_none.validate()
lrn_none = RNNLearner(dls, rnn_type='lstm', input_norm=None, metrics=[fun_rmse])
lrn_none.fit_flat_cos(n_epoch=5, lr=3e-3)
results['None'] = lrn_none.validate()
Epoch 1/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 1/5: 68%|██████▊ | 204/300 [00:00<00:00, 406.47it/s]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 406.47it/s, train=0.0301 | valid=0.0077 | fun_rmse=0.0122]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 409.14it/s, train=0.0301 | valid=0.0077 | fun_rmse=0.0122]
Epoch 2/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 2/5: 68%|██████▊ | 205/300 [00:00<00:00, 408.71it/s]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 408.71it/s, train=0.0068 | valid=0.0058 | fun_rmse=0.0113]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 407.35it/s, train=0.0068 | valid=0.0058 | fun_rmse=0.0113]
Epoch 3/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 3/5: 69%|██████▉ | 207/300 [00:00<00:00, 412.57it/s]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 412.57it/s, train=0.0067 | valid=0.0055 | fun_rmse=0.0106]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 409.97it/s, train=0.0067 | valid=0.0055 | fun_rmse=0.0106]
Epoch 4/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 4/5: 69%|██████▉ | 208/300 [00:00<00:00, 415.40it/s]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 415.40it/s, train=0.0061 | valid=0.0045 | fun_rmse=0.0100]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 412.39it/s, train=0.0061 | valid=0.0045 | fun_rmse=0.0100]
Epoch 5/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 5/5: 69%|██████▊ | 206/300 [00:00<00:00, 410.79it/s]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 410.79it/s, train=0.0037 | valid=0.0032 | fun_rmse=0.0097]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 409.53it/s, train=0.0037 | valid=0.0032 | fun_rmse=0.0097]
Output Normalization¶
By default, only inputs are normalized and outputs stay in physical units. For multi-output systems where outputs have very different scales, you can also normalize outputs. Predictions are automatically denormalized back to physical units.
In [8]:
Copied!
lrn_out = RNNLearner(dls, rnn_type='lstm', output_norm=StandardScaler, metrics=[fun_rmse])
lrn_out.fit_flat_cos(n_epoch=5, lr=3e-3)
lrn_out = RNNLearner(dls, rnn_type='lstm', output_norm=StandardScaler, metrics=[fun_rmse])
lrn_out.fit_flat_cos(n_epoch=5, lr=3e-3)
Epoch 1/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 1/5: 68%|██████▊ | 205/300 [00:00<00:00, 409.65it/s]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 409.65it/s, train=0.0110 | valid=0.0036 | fun_rmse=0.0096]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 407.44it/s, train=0.0110 | valid=0.0036 | fun_rmse=0.0096]
Epoch 2/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 2/5: 68%|██████▊ | 203/300 [00:00<00:00, 404.97it/s]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 404.97it/s, train=0.0064 | valid=0.0055 | fun_rmse=0.0102]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 403.34it/s, train=0.0064 | valid=0.0055 | fun_rmse=0.0102]
Epoch 3/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 3/5: 68%|██████▊ | 203/300 [00:00<00:00, 405.18it/s]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 405.18it/s, train=0.0057 | valid=0.0050 | fun_rmse=0.0101]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 403.89it/s, train=0.0057 | valid=0.0050 | fun_rmse=0.0101]
Epoch 4/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 4/5: 67%|██████▋ | 201/300 [00:00<00:00, 400.89it/s]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 400.89it/s, train=0.0059 | valid=0.0056 | fun_rmse=0.0105]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 401.71it/s, train=0.0059 | valid=0.0056 | fun_rmse=0.0105]
Epoch 5/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 5/5: 68%|██████▊ | 203/300 [00:00<00:00, 404.61it/s]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 404.61it/s, train=0.0037 | valid=0.0030 | fun_rmse=0.0096]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 403.86it/s, train=0.0037 | valid=0.0030 | fun_rmse=0.0096]
Creating a Custom Scaler¶
To create a custom scaler, subclass Scaler and implement three methods:
normalize, denormalize, and the from_stats classmethod. Here is an
example that saturates outliers: values beyond half the largest training
amplitude are clipped before scaling to [-1, 1]. Placing the threshold
inside the data range (rather than at the extremes, where clamp would
never fire) makes the clipping actually do something.
In [9]:
Copied!
class ClipScaler(Scaler):
"""Clips values to [-clip_val, clip_val] then scales to [-1, 1].
Args:
clip_val: saturation threshold, set to half the largest absolute
value in the training data by ``from_stats``.
"""
def __init__(self, clip_val: torch.Tensor):
super().__init__()
self.register_buffer('clip_val', clip_val)
def normalize(self, x: torch.Tensor) -> torch.Tensor:
return torch.clamp(x, -self.clip_val, self.clip_val) / self.clip_val
def denormalize(self, x: torch.Tensor) -> torch.Tensor:
return x * self.clip_val
@classmethod
def from_stats(cls, stats):
max_abs = torch.max(torch.abs(torch.tensor(stats.min)),
torch.abs(torch.tensor(stats.max))).float()
return cls((0.5 * max_abs).unsqueeze(0).unsqueeze(0))
class ClipScaler(Scaler):
"""Clips values to [-clip_val, clip_val] then scales to [-1, 1].
Args:
clip_val: saturation threshold, set to half the largest absolute
value in the training data by ``from_stats``.
"""
def __init__(self, clip_val: torch.Tensor):
super().__init__()
self.register_buffer('clip_val', clip_val)
def normalize(self, x: torch.Tensor) -> torch.Tensor:
return torch.clamp(x, -self.clip_val, self.clip_val) / self.clip_val
def denormalize(self, x: torch.Tensor) -> torch.Tensor:
return x * self.clip_val
@classmethod
def from_stats(cls, stats):
max_abs = torch.max(torch.abs(torch.tensor(stats.min)),
torch.abs(torch.tensor(stats.max))).float()
return cls((0.5 * max_abs).unsqueeze(0).unsqueeze(0))
from_stats receives a NormPair object containing the dataset statistics
(mean, std, min, max as 1-D arrays) and must return a configured
scaler instance. register_buffer ensures the parameters move with the
model to GPU/CPU automatically.
Check how much the scaler actually clips. Building it from the input stats
(dls.norm_stats.u) and applying it to a raw validation batch counts the
values that saturate:
In [10]:
Copied!
scaler = ClipScaler.from_stats(dls.norm_stats.u)
xb, yb = next(iter(dls.valid))
n_clipped = int((xb.abs() > scaler.clip_val).sum())
print(f"clip_val: {scaler.clip_val.item():.4f}")
print(f"clipped: {n_clipped} of {xb.numel()} values ({100 * n_clipped / xb.numel():.1f}%)")
scaler = ClipScaler.from_stats(dls.norm_stats.u)
xb, yb = next(iter(dls.valid))
n_clipped = int((xb.abs() > scaler.clip_val).sum())
print(f"clip_val: {scaler.clip_val.item():.4f}")
print(f"clipped: {n_clipped} of {xb.numel()} values ({100 * n_clipped / xb.numel():.1f}%)")
clip_val: 0.0507 clipped: 214 of 8000 values (2.7%)
Train with the custom scaler:
In [11]:
Copied!
lrn_custom = RNNLearner(dls, rnn_type='lstm', input_norm=ClipScaler, metrics=[fun_rmse])
lrn_custom.fit_flat_cos(n_epoch=5, lr=3e-3)
results['ClipScaler'] = lrn_custom.validate()
lrn_custom = RNNLearner(dls, rnn_type='lstm', input_norm=ClipScaler, metrics=[fun_rmse])
lrn_custom.fit_flat_cos(n_epoch=5, lr=3e-3)
results['ClipScaler'] = lrn_custom.validate()
Epoch 1/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 1/5: 66%|██████▌ | 198/300 [00:00<00:00, 395.28it/s]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 395.28it/s, train=0.0133 | valid=0.0078 | fun_rmse=0.0120]
Epoch 1/5: 100%|██████████| 300/300 [00:00<00:00, 398.47it/s, train=0.0133 | valid=0.0078 | fun_rmse=0.0120]
Epoch 2/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 2/5: 69%|██████▉ | 207/300 [00:00<00:00, 412.39it/s]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 412.39it/s, train=0.0060 | valid=0.0052 | fun_rmse=0.0103]
Epoch 2/5: 100%|██████████| 300/300 [00:00<00:00, 408.45it/s, train=0.0060 | valid=0.0052 | fun_rmse=0.0103]
Epoch 3/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 3/5: 66%|██████▋ | 199/300 [00:00<00:00, 396.70it/s]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 396.70it/s, train=0.0057 | valid=0.0045 | fun_rmse=0.0101]
Epoch 3/5: 100%|██████████| 300/300 [00:00<00:00, 396.17it/s, train=0.0057 | valid=0.0045 | fun_rmse=0.0101]
Epoch 4/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 4/5: 67%|██████▋ | 201/300 [00:00<00:00, 400.81it/s]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 400.81it/s, train=0.0054 | valid=0.0070 | fun_rmse=0.0116]
Epoch 4/5: 100%|██████████| 300/300 [00:00<00:00, 399.08it/s, train=0.0054 | valid=0.0070 | fun_rmse=0.0116]
Epoch 5/5: 0%| | 0/300 [00:00<?, ?it/s]
Epoch 5/5: 67%|██████▋ | 202/300 [00:00<00:00, 402.82it/s]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 402.82it/s, train=0.0046 | valid=0.0044 | fun_rmse=0.0102]
Epoch 5/5: 100%|██████████| 300/300 [00:00<00:00, 400.58it/s, train=0.0046 | valid=0.0044 | fun_rmse=0.0102]
Visualize Results¶
In [12]:
Copied!
lrn_custom.show_results(max_n=2)
lrn_custom.show_results(max_n=2)
Comparison¶
All five runs share the same architecture and training schedule, so the scaler is the only difference:
In [13]:
Copied!
for name, (loss, metrics) in results.items():
rmse = metrics.get('fun_rmse', float('nan'))
print(f"{name:16s}: loss={loss:.4f}, RMSE={rmse:.4f}")
for name, (loss, metrics) in results.items():
rmse = metrics.get('fun_rmse', float('nan'))
print(f"{name:16s}: loss={loss:.4f}, RMSE={rmse:.4f}")
StandardScaler : loss=0.0029, RMSE=0.0095 MinMaxScaler : loss=0.0030, RMSE=0.0095 MaxAbsScaler : loss=0.0029, RMSE=0.0096 None : loss=0.0032, RMSE=0.0097 ClipScaler : loss=0.0044, RMSE=0.0102
The built-in scalers -- and even disabling normalization -- land at
essentially the same RMSE: Silverbox is insensitive to the choice because
its signals are already near unit scale. Even the lossy ClipScaler stays
close, since only a small fraction of values exceed its threshold. Treat
this table as evidence that on well-scaled data the scaler is not the
bottleneck -- and as the template for running this comparison on your own
data, where mismatched physical units can make the differences substantial.
Key Takeaways¶
StandardScaler(z-score) is the default and works well for most problems.MinMaxScalerandMaxAbsScalerare alternatives for bounded signals.input_norm=Nonedisables normalization (useful for pre-normalized data).output_norm=StandardScalernormalizes outputs for multi-scale training. Predictions are automatically denormalized.- Custom scalers subclass
Scalerwithnormalize,denormalize, andfrom_stats. - On data that is already near unit scale, the scaler choice makes little difference; normalization pays off when features have large or mismatched magnitudes.