Example 03: Prediction -- Using Output Feedback¶

In prediction mode, the model receives both the input signal u(t) and past measured outputs y(t-1) to predict the next output. This gives the model a "peek" at recent measurements, leading to more accurate predictions -- but requires measured outputs at inference time.

Prerequisites¶

This notebook builds on Examples 00-02. In particular, make sure you understand the simulation concept from Example 02 before reading on, because prediction mode is defined by how it differs from simulation.

Setup¶

In [1]:

from tsfast.tsdata.benchmark import create_dls_silverbox_prediction
from tsfast.prediction.fransys import FranSysLearner
from tsfast.training import fun_rmse

from tsfast.tsdata.benchmark import create_dls_silverbox_prediction from tsfast.prediction.fransys import FranSysLearner from tsfast.training import fun_rmse

/home/pheenix/Development/tsfast/.venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Simulation vs. Prediction¶

This is the core conceptual distinction in tsfast. Every modeling choice flows from this decision:

Simulation¶

Model input:  [ u(t) ]
Model output: y(t)

The model must simulate the system purely from the excitation signal. It never sees the real output. This is harder but more general -- at deployment time, you only need the input signal.

Prediction¶

Model input:  [ u(t),  y(t-1) ]
Model output: y(t)

The model gets past measured outputs as additional input. This is easier and typically more accurate, because the model can correct itself using recent measurements. The tradeoff: you need a sensor measuring y at deployment time.

How tsfast implements prediction¶

When you load a prediction dataset (e.g., create_dls_silverbox_prediction), the DataLoaders automatically provide [u(t), y(t-1)] as the input tensor and y(t) as the target. The input signal u is normalized (z-score), but the output feedback y stays in raw physical units. This mixed normalization is intentional -- the model learns to work with both scales.

Load a Prediction Dataset¶

create_dls_silverbox_prediction creates DataLoaders configured for prediction mode on the Silverbox benchmark. The prediction-mode factory handles the time-shifting of y(t) to y(t-1) and the concatenation automatically.

In [2]:

dls = create_dls_silverbox_prediction()

dls = create_dls_silverbox_prediction()

The FranSys Architecture¶

For prediction tasks, tsfast provides the FranSys (Framework for Nonlinear System identification) architecture. It separates the model into two phases:

1. Diagnosis (first init_sz timesteps): the model processes the initialization window -- both input u(t) and measured output y(t-1) -- to estimate the system's current internal state from measurement data.

2. Prognosis (remaining timesteps): using the estimated state from diagnosis as initial hidden state, the model predicts forward using only the input u(t). During training, the prognosis module sees the full sequence; during inference, it runs autoregressively from the estimated state.

The key insight is that init_sz controls how many timesteps are used to "warm up" the model's hidden state from real data. Predictions are only evaluated after this initialization window, so init_sz also acts like n_skip -- the first init_sz timesteps are excluded from the loss.

Train a FranSys Model¶

Key parameters:

• init_sz=50: use the first 50 timesteps for state estimation (diagnosis); predictions are evaluated on the remainder (prognosis)
• hidden_size=40: 40 hidden units in the RNN layers
• metrics=[fun_rmse]: track root mean squared error

In [3]:

lrn = FranSysLearner(dls, init_sz=50, hidden_size=40, metrics=[fun_rmse], attach_output=True)
lrn.show_batch(max_n=4)

lrn = FranSysLearner(dls, init_sz=50, hidden_size=40, metrics=[fun_rmse], attach_output=True) lrn.show_batch(max_n=4)

Notice that the input now has more channels than in the simulation example. The extra channels are the measured output values y(t-1) concatenated to u(t).

In [4]:

lrn.fit_flat_cos(n_epoch=10, lr=3e-3)

lrn.fit_flat_cos(n_epoch=10, lr=3e-3)

Training:   0%|          | 0/3000 [00:00<?, ?it/s]

Training:   0%|          | 1/3000 [00:00<33:12,  1.51it/s]

Training:   0%|          | 12/3000 [00:01<04:14, 11.74it/s]

Training:   1%|          | 23/3000 [00:01<03:12, 15.45it/s]

Training:   1%|          | 34/3000 [00:02<02:50, 17.35it/s]

Training:   1%|▏         | 43/3000 [00:03<03:29, 14.13it/s]

Training:   2%|▏         | 53/3000 [00:03<03:08, 15.59it/s]

Training:   2%|▏         | 63/3000 [00:04<03:01, 16.20it/s]

Training:   2%|▏         | 72/3000 [00:04<03:02, 16.05it/s]

Training:   3%|▎         | 81/3000 [00:05<02:56, 16.57it/s]

Training:   3%|▎         | 90/3000 [00:05<02:52, 16.89it/s]

Training:   3%|▎         | 100/3000 [00:06<02:46, 17.43it/s]

Training:   4%|▎         | 109/3000 [00:06<02:44, 17.58it/s]

Training:   4%|▍         | 118/3000 [00:07<02:43, 17.58it/s]

Training:   4%|▍         | 127/3000 [00:07<02:48, 17.04it/s]

Training:   5%|▍         | 136/3000 [00:08<02:46, 17.17it/s]

Training:   5%|▍         | 145/3000 [00:08<02:44, 17.32it/s]

Training:   5%|▌         | 154/3000 [00:09<02:43, 17.37it/s]

Training:   5%|▌         | 163/3000 [00:09<02:42, 17.43it/s]

Training:   6%|▌         | 173/3000 [00:10<02:38, 17.85it/s]

Training:   6%|▌         | 182/3000 [00:11<02:37, 17.85it/s]

Training:   6%|▋         | 192/3000 [00:11<02:35, 18.00it/s]

Training:   7%|▋         | 202/3000 [00:12<02:35, 17.98it/s]

Training:   7%|▋         | 212/3000 [00:12<02:30, 18.49it/s]

Training:   7%|▋         | 222/3000 [00:13<02:30, 18.50it/s]

Training:   8%|▊         | 232/3000 [00:13<02:29, 18.51it/s]

Training:   8%|▊         | 242/3000 [00:14<02:30, 18.27it/s]

Training:   8%|▊         | 252/3000 [00:14<02:30, 18.27it/s]

Training:   9%|▊         | 262/3000 [00:15<02:32, 17.95it/s]

Training:   9%|▉         | 271/3000 [00:15<02:32, 17.92it/s]

Training:   9%|▉         | 280/3000 [00:16<02:31, 17.90it/s]

Training:  10%|▉         | 289/3000 [00:16<02:32, 17.74it/s]

Training:  10%|▉         | 299/3000 [00:17<02:31, 17.89it/s]

Training:  10%|█         | 300/3000 [00:17<02:30, 17.89it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  10%|█         | 308/3000 [00:18<02:41, 16.64it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  11%|█         | 317/3000 [00:18<02:37, 16.99it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  11%|█         | 327/3000 [00:19<02:33, 17.41it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  11%|█         | 336/3000 [00:19<02:31, 17.55it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  12%|█▏        | 345/3000 [00:20<02:32, 17.39it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  12%|█▏        | 354/3000 [00:20<02:31, 17.52it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  12%|█▏        | 363/3000 [00:21<02:35, 16.96it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  12%|█▏        | 372/3000 [00:21<02:32, 17.19it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  13%|█▎        | 381/3000 [00:22<02:35, 16.89it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  13%|█▎        | 390/3000 [00:22<02:33, 16.98it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  13%|█▎        | 399/3000 [00:23<02:35, 16.76it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  14%|█▎        | 409/3000 [00:23<02:29, 17.30it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  14%|█▍        | 419/3000 [00:24<02:27, 17.49it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  14%|█▍        | 428/3000 [00:25<02:27, 17.45it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  15%|█▍        | 438/3000 [00:25<02:24, 17.74it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  15%|█▍        | 448/3000 [00:26<02:25, 17.57it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  15%|█▌        | 457/3000 [00:26<02:24, 17.65it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  16%|█▌        | 467/3000 [00:27<02:19, 18.18it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  16%|█▌        | 477/3000 [00:27<02:19, 18.14it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  16%|█▌        | 487/3000 [00:28<02:16, 18.39it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  17%|█▋        | 497/3000 [00:28<02:17, 18.15it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  17%|█▋        | 507/3000 [00:29<02:16, 18.27it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  17%|█▋        | 517/3000 [00:29<02:14, 18.45it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  18%|█▊        | 527/3000 [00:30<02:16, 18.09it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  18%|█▊        | 537/3000 [00:31<02:20, 17.55it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  18%|█▊        | 546/3000 [00:31<02:20, 17.47it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  19%|█▊        | 556/3000 [00:32<02:18, 17.71it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  19%|█▉        | 566/3000 [00:32<02:15, 17.92it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  19%|█▉        | 576/3000 [00:33<02:14, 18.02it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  20%|█▉        | 586/3000 [00:33<02:12, 18.20it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  20%|█▉        | 596/3000 [00:34<02:15, 17.75it/s, epoch 1 | train=0.0160 | valid=0.0033 | fun_rmse=0.0042]

Training:  20%|██        | 600/3000 [00:34<02:15, 17.75it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  20%|██        | 605/3000 [00:35<02:26, 16.39it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  20%|██        | 614/3000 [00:35<02:22, 16.70it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  21%|██        | 623/3000 [00:36<02:21, 16.77it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  21%|██        | 632/3000 [00:36<02:18, 17.06it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  21%|██▏       | 642/3000 [00:37<02:15, 17.40it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  22%|██▏       | 651/3000 [00:37<02:16, 17.21it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  22%|██▏       | 661/3000 [00:38<02:13, 17.58it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  22%|██▏       | 670/3000 [00:38<02:11, 17.67it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  23%|██▎       | 680/3000 [00:39<02:08, 18.03it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  23%|██▎       | 690/3000 [00:39<02:08, 17.93it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  23%|██▎       | 700/3000 [00:40<02:06, 18.21it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  24%|██▎       | 710/3000 [00:40<02:02, 18.65it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  24%|██▍       | 720/3000 [00:41<02:05, 18.21it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  24%|██▍       | 730/3000 [00:41<02:05, 18.11it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  25%|██▍       | 741/3000 [00:42<02:00, 18.67it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  25%|██▌       | 751/3000 [00:43<02:00, 18.72it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  25%|██▌       | 761/3000 [00:43<02:00, 18.61it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  26%|██▌       | 771/3000 [00:44<02:00, 18.57it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  26%|██▌       | 781/3000 [00:44<01:57, 18.90it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  26%|██▋       | 791/3000 [00:45<01:56, 18.90it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  27%|██▋       | 801/3000 [00:45<01:55, 19.01it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  27%|██▋       | 811/3000 [00:46<01:58, 18.54it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  27%|██▋       | 821/3000 [00:46<01:58, 18.35it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  28%|██▊       | 831/3000 [00:47<01:56, 18.55it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  28%|██▊       | 841/3000 [00:47<01:57, 18.44it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  28%|██▊       | 851/3000 [00:48<01:56, 18.47it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  29%|██▊       | 861/3000 [00:49<02:00, 17.72it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  29%|██▉       | 870/3000 [00:49<02:03, 17.20it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  29%|██▉       | 880/3000 [00:50<02:01, 17.48it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  30%|██▉       | 890/3000 [00:50<01:56, 18.10it/s, epoch 2 | train=0.0024 | valid=0.0023 | fun_rmse=0.0026]

Training:  30%|███       | 900/3000 [00:51<01:56, 18.10it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  30%|███       | 901/3000 [00:51<01:59, 17.52it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  30%|███       | 912/3000 [00:51<01:53, 18.35it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  31%|███       | 923/3000 [00:52<01:50, 18.87it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  31%|███       | 934/3000 [00:52<01:47, 19.22it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  32%|███▏      | 945/3000 [00:53<01:45, 19.51it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  32%|███▏      | 956/3000 [00:54<01:43, 19.68it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  32%|███▏      | 966/3000 [00:54<01:45, 19.35it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  33%|███▎      | 976/3000 [00:55<01:44, 19.42it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  33%|███▎      | 986/3000 [00:55<01:44, 19.33it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  33%|███▎      | 996/3000 [00:56<01:45, 18.98it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  34%|███▎      | 1006/3000 [00:56<01:45, 18.88it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  34%|███▍      | 1016/3000 [00:57<01:47, 18.54it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  34%|███▍      | 1026/3000 [00:57<01:48, 18.13it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  35%|███▍      | 1036/3000 [00:58<01:50, 17.72it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  35%|███▍      | 1045/3000 [00:58<01:51, 17.58it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  35%|███▌      | 1055/3000 [00:59<01:48, 17.85it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  36%|███▌      | 1065/3000 [01:00<01:45, 18.33it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  36%|███▌      | 1075/3000 [01:00<01:42, 18.74it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  36%|███▌      | 1085/3000 [01:01<01:41, 18.92it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  36%|███▋      | 1095/3000 [01:01<01:39, 19.13it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  37%|███▋      | 1105/3000 [01:02<01:39, 19.06it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  37%|███▋      | 1115/3000 [01:02<01:40, 18.75it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  38%|███▊      | 1125/3000 [01:03<01:44, 18.03it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  38%|███▊      | 1135/3000 [01:03<01:41, 18.35it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  38%|███▊      | 1145/3000 [01:04<01:39, 18.67it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  38%|███▊      | 1155/3000 [01:04<01:38, 18.64it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  39%|███▉      | 1165/3000 [01:05<01:39, 18.46it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  39%|███▉      | 1175/3000 [01:05<01:41, 17.90it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  40%|███▉      | 1185/3000 [01:06<01:39, 18.29it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  40%|███▉      | 1195/3000 [01:07<01:38, 18.37it/s, epoch 3 | train=0.0022 | valid=0.0011 | fun_rmse=0.0015]

Training:  40%|████      | 1200/3000 [01:07<01:37, 18.37it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  40%|████      | 1205/3000 [01:07<01:43, 17.42it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  40%|████      | 1214/3000 [01:08<01:43, 17.24it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  41%|████      | 1224/3000 [01:08<01:39, 17.94it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  41%|████      | 1234/3000 [01:09<01:36, 18.25it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  41%|████▏     | 1244/3000 [01:09<01:34, 18.58it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  42%|████▏     | 1254/3000 [01:10<01:32, 18.84it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  42%|████▏     | 1264/3000 [01:10<01:31, 18.98it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  42%|████▏     | 1274/3000 [01:11<01:30, 19.11it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  43%|████▎     | 1284/3000 [01:11<01:29, 19.08it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  43%|████▎     | 1294/3000 [01:12<01:32, 18.53it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  43%|████▎     | 1304/3000 [01:12<01:29, 18.85it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  44%|████▍     | 1314/3000 [01:13<01:28, 19.11it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  44%|████▍     | 1324/3000 [01:13<01:28, 18.86it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  44%|████▍     | 1334/3000 [01:14<01:28, 18.79it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  45%|████▍     | 1344/3000 [01:15<01:26, 19.06it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  45%|████▌     | 1354/3000 [01:15<01:25, 19.23it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  45%|████▌     | 1364/3000 [01:16<01:25, 19.10it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  46%|████▌     | 1374/3000 [01:16<01:25, 18.92it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  46%|████▌     | 1384/3000 [01:17<01:24, 19.09it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  46%|████▋     | 1395/3000 [01:17<01:22, 19.42it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  47%|████▋     | 1405/3000 [01:18<01:24, 18.94it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  47%|████▋     | 1415/3000 [01:18<01:22, 19.16it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  48%|████▊     | 1425/3000 [01:19<01:22, 19.12it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  48%|████▊     | 1435/3000 [01:19<01:23, 18.85it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  48%|████▊     | 1445/3000 [01:20<01:22, 18.90it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  48%|████▊     | 1455/3000 [01:20<01:23, 18.53it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  49%|████▉     | 1465/3000 [01:21<01:22, 18.59it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  49%|████▉     | 1475/3000 [01:21<01:21, 18.75it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  50%|████▉     | 1485/3000 [01:22<01:21, 18.50it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  50%|████▉     | 1495/3000 [01:23<01:21, 18.43it/s, epoch 4 | train=0.0019 | valid=0.0018 | fun_rmse=0.0024]

Training:  50%|█████     | 1500/3000 [01:23<01:21, 18.43it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  50%|█████     | 1505/3000 [01:23<01:29, 16.62it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  50%|█████     | 1514/3000 [01:24<01:27, 16.93it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  51%|█████     | 1523/3000 [01:24<01:26, 17.16it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  51%|█████     | 1533/3000 [01:25<01:23, 17.55it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  51%|█████▏    | 1543/3000 [01:25<01:20, 18.13it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  52%|█████▏    | 1553/3000 [01:26<01:21, 17.77it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  52%|█████▏    | 1563/3000 [01:26<01:19, 18.04it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  52%|█████▏    | 1573/3000 [01:27<01:20, 17.72it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  53%|█████▎    | 1582/3000 [01:28<01:28, 16.08it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  53%|█████▎    | 1591/3000 [01:28<01:27, 16.03it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  53%|█████▎    | 1600/3000 [01:29<01:26, 16.14it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  54%|█████▎    | 1610/3000 [01:29<01:23, 16.70it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  54%|█████▍    | 1619/3000 [01:30<01:22, 16.67it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  54%|█████▍    | 1628/3000 [01:31<01:22, 16.69it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  55%|█████▍    | 1637/3000 [01:31<01:20, 16.84it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  55%|█████▍    | 1646/3000 [01:32<01:21, 16.59it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  55%|█████▌    | 1655/3000 [01:32<01:21, 16.56it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  55%|█████▌    | 1664/3000 [01:33<01:22, 16.27it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  56%|█████▌    | 1674/3000 [01:33<01:18, 16.88it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  56%|█████▌    | 1683/3000 [01:34<01:18, 16.85it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  56%|█████▋    | 1692/3000 [01:34<01:17, 16.81it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  57%|█████▋    | 1701/3000 [01:35<01:16, 16.93it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  57%|█████▋    | 1710/3000 [01:35<01:15, 17.09it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  57%|█████▋    | 1719/3000 [01:36<01:14, 17.19it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  58%|█████▊    | 1728/3000 [01:36<01:14, 17.18it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  58%|█████▊    | 1737/3000 [01:37<01:12, 17.40it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  58%|█████▊    | 1746/3000 [01:37<01:12, 17.37it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  58%|█████▊    | 1755/3000 [01:38<01:11, 17.38it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  59%|█████▉    | 1765/3000 [01:39<01:10, 17.57it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  59%|█████▉    | 1774/3000 [01:39<01:11, 17.16it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  59%|█████▉    | 1783/3000 [01:40<01:11, 17.01it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  60%|█████▉    | 1792/3000 [01:40<01:10, 17.14it/s, epoch 5 | train=0.0018 | valid=0.0022 | fun_rmse=0.0028]

Training:  60%|██████    | 1800/3000 [01:41<01:10, 17.14it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  60%|██████    | 1801/3000 [01:41<01:16, 15.64it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  60%|██████    | 1810/3000 [01:41<01:15, 15.81it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  61%|██████    | 1818/3000 [01:42<01:14, 15.80it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  61%|██████    | 1828/3000 [01:42<01:11, 16.41it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  61%|██████▏   | 1838/3000 [01:43<01:07, 17.10it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  62%|██████▏   | 1848/3000 [01:44<01:06, 17.35it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  62%|██████▏   | 1857/3000 [01:44<01:06, 17.25it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  62%|██████▏   | 1868/3000 [01:45<01:02, 18.04it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  63%|██████▎   | 1878/3000 [01:45<01:00, 18.54it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  63%|██████▎   | 1888/3000 [01:46<00:59, 18.70it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  63%|██████▎   | 1898/3000 [01:46<00:59, 18.49it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  64%|██████▎   | 1908/3000 [01:47<00:57, 18.86it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  64%|██████▍   | 1919/3000 [01:47<00:56, 19.18it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  64%|██████▍   | 1930/3000 [01:48<00:55, 19.37it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  65%|██████▍   | 1940/3000 [01:48<00:55, 19.22it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  65%|██████▌   | 1950/3000 [01:49<00:54, 19.21it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  65%|██████▌   | 1960/3000 [01:49<00:56, 18.30it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  66%|██████▌   | 1970/3000 [01:50<00:58, 17.62it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  66%|██████▌   | 1979/3000 [01:51<00:58, 17.55it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  66%|██████▋   | 1988/3000 [01:51<00:58, 17.43it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  67%|██████▋   | 1997/3000 [01:52<00:57, 17.51it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  67%|██████▋   | 2006/3000 [01:52<00:58, 17.00it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  67%|██████▋   | 2016/3000 [01:53<00:56, 17.54it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  68%|██████▊   | 2025/3000 [01:53<00:55, 17.61it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  68%|██████▊   | 2034/3000 [01:54<00:55, 17.29it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  68%|██████▊   | 2043/3000 [01:54<00:56, 17.07it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  68%|██████▊   | 2052/3000 [01:55<00:55, 17.10it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  69%|██████▊   | 2061/3000 [01:55<00:55, 16.80it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  69%|██████▉   | 2070/3000 [01:56<00:56, 16.54it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  69%|██████▉   | 2079/3000 [01:57<00:56, 16.39it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  70%|██████▉   | 2088/3000 [01:57<00:54, 16.60it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  70%|██████▉   | 2097/3000 [01:58<00:56, 15.98it/s, epoch 6 | train=0.0017 | valid=0.0017 | fun_rmse=0.0020]

Training:  70%|███████   | 2100/3000 [01:58<00:56, 15.98it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  70%|███████   | 2106/3000 [01:58<01:00, 14.70it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  70%|███████   | 2114/3000 [01:59<01:00, 14.57it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  71%|███████   | 2122/3000 [02:00<01:01, 14.36it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  71%|███████   | 2130/3000 [02:00<00:59, 14.58it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  71%|███████▏  | 2139/3000 [02:01<00:57, 15.02it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  72%|███████▏  | 2147/3000 [02:01<00:56, 15.02it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  72%|███████▏  | 2156/3000 [02:02<00:54, 15.56it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  72%|███████▏  | 2164/3000 [02:02<00:53, 15.50it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  72%|███████▏  | 2173/3000 [02:03<00:51, 15.92it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  73%|███████▎  | 2181/3000 [02:03<00:51, 15.87it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  73%|███████▎  | 2189/3000 [02:04<00:52, 15.40it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  73%|███████▎  | 2197/3000 [02:04<00:51, 15.47it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  74%|███████▎  | 2205/3000 [02:05<00:50, 15.60it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  74%|███████▍  | 2213/3000 [02:05<00:50, 15.65it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  74%|███████▍  | 2221/3000 [02:06<00:51, 15.25it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  74%|███████▍  | 2229/3000 [02:06<00:50, 15.34it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  75%|███████▍  | 2238/3000 [02:07<00:48, 15.58it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  75%|███████▍  | 2246/3000 [02:07<00:48, 15.63it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  75%|███████▌  | 2255/3000 [02:08<00:47, 15.83it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  75%|███████▌  | 2263/3000 [02:09<00:46, 15.68it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  76%|███████▌  | 2272/3000 [02:09<00:45, 15.83it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  76%|███████▌  | 2280/3000 [02:10<00:46, 15.40it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  76%|███████▋  | 2288/3000 [02:10<00:47, 15.10it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  77%|███████▋  | 2296/3000 [02:11<00:46, 15.07it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  77%|███████▋  | 2304/3000 [02:11<00:46, 14.89it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  77%|███████▋  | 2312/3000 [02:12<00:46, 14.92it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  77%|███████▋  | 2320/3000 [02:12<00:45, 14.99it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  78%|███████▊  | 2328/3000 [02:13<00:44, 15.20it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  78%|███████▊  | 2336/3000 [02:13<00:44, 14.90it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  78%|███████▊  | 2344/3000 [02:14<00:43, 14.97it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  78%|███████▊  | 2352/3000 [02:15<00:44, 14.64it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  79%|███████▊  | 2360/3000 [02:15<00:43, 14.72it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  79%|███████▉  | 2368/3000 [02:16<00:42, 14.73it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  79%|███████▉  | 2376/3000 [02:16<00:42, 14.77it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  80%|███████▉  | 2385/3000 [02:17<00:40, 15.26it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  80%|███████▉  | 2393/3000 [02:17<00:39, 15.44it/s, epoch 7 | train=0.0016 | valid=0.0014 | fun_rmse=0.0016]

Training:  80%|████████  | 2400/3000 [02:18<00:38, 15.44it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  80%|████████  | 2401/3000 [02:18<00:41, 14.47it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  80%|████████  | 2409/3000 [02:18<00:40, 14.58it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  81%|████████  | 2418/3000 [02:19<00:38, 15.25it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  81%|████████  | 2427/3000 [02:19<00:36, 15.87it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  81%|████████  | 2436/3000 [02:20<00:35, 15.88it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  82%|████████▏ | 2446/3000 [02:21<00:33, 16.45it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  82%|████████▏ | 2455/3000 [02:21<00:32, 16.79it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  82%|████████▏ | 2464/3000 [02:22<00:31, 16.88it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  82%|████████▏ | 2474/3000 [02:22<00:30, 17.46it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  83%|████████▎ | 2484/3000 [02:23<00:29, 17.74it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  83%|████████▎ | 2493/3000 [02:23<00:28, 17.76it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  83%|████████▎ | 2502/3000 [02:24<00:28, 17.45it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  84%|████████▎ | 2512/3000 [02:24<00:27, 17.72it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  84%|████████▍ | 2521/3000 [02:25<00:27, 17.60it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  84%|████████▍ | 2530/3000 [02:25<00:26, 17.69it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  85%|████████▍ | 2539/3000 [02:26<00:27, 17.04it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  85%|████████▍ | 2548/3000 [02:26<00:26, 16.77it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  85%|████████▌ | 2557/3000 [02:27<00:26, 16.90it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  86%|████████▌ | 2566/3000 [02:28<00:26, 16.20it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  86%|████████▌ | 2575/3000 [02:28<00:25, 16.65it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  86%|████████▌ | 2585/3000 [02:29<00:24, 16.73it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  86%|████████▋ | 2595/3000 [02:29<00:23, 17.17it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  87%|████████▋ | 2604/3000 [02:30<00:23, 17.02it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  87%|████████▋ | 2613/3000 [02:30<00:22, 17.03it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  87%|████████▋ | 2622/3000 [02:31<00:22, 16.78it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  88%|████████▊ | 2631/3000 [02:31<00:22, 16.34it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  88%|████████▊ | 2640/3000 [02:32<00:21, 16.59it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  88%|████████▊ | 2649/3000 [02:32<00:20, 16.92it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  89%|████████▊ | 2659/3000 [02:33<00:19, 17.39it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  89%|████████▉ | 2669/3000 [02:33<00:18, 17.81it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  89%|████████▉ | 2679/3000 [02:34<00:17, 18.04it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  90%|████████▉ | 2689/3000 [02:35<00:17, 17.85it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  90%|████████▉ | 2698/3000 [02:35<00:17, 17.55it/s, epoch 8 | train=0.0017 | valid=0.0018 | fun_rmse=0.0024]

Training:  90%|█████████ | 2700/3000 [02:35<00:17, 17.55it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  90%|█████████ | 2707/3000 [02:36<00:18, 15.93it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  91%|█████████ | 2717/3000 [02:36<00:16, 16.65it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  91%|█████████ | 2727/3000 [02:37<00:15, 17.13it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  91%|█████████ | 2736/3000 [02:37<00:15, 17.18it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  92%|█████████▏| 2746/3000 [02:38<00:14, 17.43it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  92%|█████████▏| 2756/3000 [02:39<00:13, 17.78it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  92%|█████████▏| 2765/3000 [02:39<00:13, 17.69it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  92%|█████████▏| 2774/3000 [02:40<00:13, 16.97it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  93%|█████████▎| 2783/3000 [02:40<00:12, 16.97it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  93%|█████████▎| 2792/3000 [02:41<00:12, 17.23it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  93%|█████████▎| 2801/3000 [02:41<00:11, 17.41it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  94%|█████████▎| 2810/3000 [02:42<00:10, 17.35it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  94%|█████████▍| 2820/3000 [02:42<00:10, 17.52it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  94%|█████████▍| 2829/3000 [02:43<00:09, 17.57it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  95%|█████████▍| 2838/3000 [02:43<00:09, 17.51it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  95%|█████████▍| 2847/3000 [02:44<00:08, 17.44it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  95%|█████████▌| 2857/3000 [02:44<00:08, 17.41it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  96%|█████████▌| 2866/3000 [02:45<00:07, 17.34it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  96%|█████████▌| 2875/3000 [02:45<00:07, 16.66it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  96%|█████████▌| 2884/3000 [02:46<00:07, 15.48it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  96%|█████████▋| 2892/3000 [02:47<00:07, 14.95it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  97%|█████████▋| 2901/3000 [02:47<00:06, 15.67it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  97%|█████████▋| 2910/3000 [02:48<00:05, 15.92it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  97%|█████████▋| 2919/3000 [02:48<00:05, 15.99it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  98%|█████████▊| 2928/3000 [02:49<00:04, 16.38it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  98%|█████████▊| 2938/3000 [02:49<00:03, 17.01it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  98%|█████████▊| 2947/3000 [02:50<00:03, 16.80it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  99%|█████████▊| 2957/3000 [02:50<00:02, 17.64it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  99%|█████████▉| 2967/3000 [02:51<00:01, 17.70it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training:  99%|█████████▉| 2977/3000 [02:52<00:01, 17.94it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training: 100%|█████████▉| 2987/3000 [02:52<00:00, 18.22it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training: 100%|█████████▉| 2997/3000 [02:53<00:00, 17.41it/s, epoch 9 | train=0.0010 | valid=0.0005 | fun_rmse=0.0007]

Training: 100%|██████████| 3000/3000 [02:53<00:00, 17.41it/s, epoch 10 | train=0.0004 | valid=0.0003 | fun_rmse=0.0004]

Training: 100%|██████████| 3000/3000 [02:53<00:00, 17.29it/s, epoch 10 | train=0.0004 | valid=0.0003 | fun_rmse=0.0004]

Visualize Results¶

show_results overlays predictions against targets on validation windows. Note the initialization region (first 50 timesteps): the model uses this window to estimate the system state, so predictions in that region are zero-padded and excluded from evaluation.

In [5]:

lrn.show_results(max_n=3)

lrn.show_results(max_n=3)

Key Takeaways¶

• Prediction mode feeds past measured outputs back as model input, giving higher accuracy than simulation at the cost of requiring a sensor at deployment.
• The prediction DataLoaders concatenate [u_normalized, y_raw] -- mixed normalization is intentional and the model learns to use both scales.
• FranSys separates diagnosis from prognosis: the diagnosis phase estimates the system state from an initialization window of real measurements, and the prognosis phase predicts forward from that estimated state.
• init_sz controls the initialization window: more timesteps give better state estimates but leave fewer timesteps for evaluated predictions.
• Prediction requires measured outputs at inference time -- if you only have the input signal, use simulation mode (Example 02) instead.