Example 08: FranSys -- Diagnosis/Prognosis Architecture¶

FranSys (Framework for Nonlinear System identification) uses a two-phase approach: a diagnosis RNN estimates the system's hidden state from an initialization window, then a prognosis RNN predicts forward from that state. This architecture handles variable initial conditions gracefully and generalizes to unseen starting points.

Prerequisites¶

This notebook builds on concepts from Examples 00-03. Make sure you understand simulation mode (Example 02) and prediction mode (Example 03) before proceeding.

Setup¶

In [1]:

from tsfast.tsdata.benchmark import create_dls_cascaded_tanks
from tsfast.prediction.fransys import FranSysLearner
from tsfast.models.scaling import unwrap_model
from tsfast.training import fun_rmse, FranSysRegularizer, ActivationRegularizer, TemporalActivationRegularizer

from tsfast.tsdata.benchmark import create_dls_cascaded_tanks from tsfast.prediction.fransys import FranSysLearner from tsfast.models.scaling import unwrap_model from tsfast.training import fun_rmse, FranSysRegularizer, ActivationRegularizer, TemporalActivationRegularizer

/home/pheenix/Development/tsfast/.venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

The Diagnosis/Prognosis Concept¶

Traditional RNNs start from a zero hidden state, which doesn't match reality -- real systems are rarely at rest. FranSys addresses this with a two-phase architecture:

• Diagnosis phase (first init_sz timesteps): The diagnosis RNN processes the initialization window of measured input+output data to estimate the system's internal state. The final hidden state captures where the system "is" at the end of the window.

• Prognosis phase (remaining timesteps): The prognosis RNN receives the diagnosis hidden state and predicts forward. It processes incoming input (and optionally output feedback) to generate predictions.

The two RNNs share the same hidden dimension but are trained jointly end-to-end.

The Cascaded Tanks Benchmark¶

The Cascaded Tanks system is a benchmark where water flows between two tanks in series. It exhibits strong nonlinear behavior due to the square-root relationship between water level and flow rate. The system has one input (pump voltage) and one output (water level in the second tank).

In [2]:

dls = create_dls_cascaded_tanks()

dls = create_dls_cascaded_tanks()

Training a Basic FranSys Model¶

Key parameters:

• init_sz=50: use the first 50 timesteps for diagnosis (state estimation). Predictions are only evaluated after this window.
• attach_output=True: the model receives past measured outputs as additional input. This is standard for prediction-mode FranSys.
• hidden_size=40: dimension of the RNN hidden state for both diagnosis and prognosis.

In [3]:

lrn = FranSysLearner(
    dls, init_sz=50, attach_output=True,
    hidden_size=40, metrics=[fun_rmse]
)
lrn.show_batch(max_n=4)

lrn = FranSysLearner( dls, init_sz=50, attach_output=True, hidden_size=40, metrics=[fun_rmse] ) lrn.show_batch(max_n=4)

In [4]:

lrn.fit_flat_cos(n_epoch=10, lr=3e-3)

lrn.fit_flat_cos(n_epoch=10, lr=3e-3)

Training:   0%|          | 0/3000 [00:00<?, ?it/s]

Training:   0%|          | 1/3000 [00:00<26:26,  1.89it/s]

Training:   1%|          | 31/3000 [00:01<01:25, 34.61it/s]

Training:   2%|▏         | 61/3000 [00:01<01:04, 45.89it/s]

Training:   3%|▎         | 91/3000 [00:02<00:56, 51.35it/s]

Training:   4%|▍         | 122/3000 [00:02<00:52, 54.68it/s]

Training:   5%|▌         | 151/3000 [00:03<00:51, 55.75it/s]

Training:   6%|▌         | 181/3000 [00:03<00:49, 56.67it/s]

Training:   7%|▋         | 210/3000 [00:04<00:48, 56.94it/s]

Training:   8%|▊         | 239/3000 [00:04<00:48, 57.15it/s]

Training:   9%|▉         | 269/3000 [00:05<00:47, 57.76it/s]

Training:  10%|▉         | 298/3000 [00:05<00:46, 57.73it/s]

Training:  10%|█         | 300/3000 [00:05<00:46, 57.73it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  11%|█         | 327/3000 [00:06<00:46, 57.30it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  12%|█▏        | 356/3000 [00:06<00:46, 56.99it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  13%|█▎        | 386/3000 [00:07<00:45, 57.67it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  14%|█▍        | 416/3000 [00:07<00:44, 58.00it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  15%|█▍        | 446/3000 [00:08<00:53, 47.96it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  16%|█▌        | 473/3000 [00:09<00:51, 49.43it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  17%|█▋        | 502/3000 [00:09<00:48, 51.32it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  18%|█▊        | 529/3000 [00:10<00:49, 49.67it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  19%|█▊        | 558/3000 [00:10<00:47, 51.87it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  20%|█▉        | 586/3000 [00:11<00:45, 52.80it/s, epoch 1 | train=1.0082 | valid=0.3239 | fun_rmse=0.7568]

Training:  20%|██        | 600/3000 [00:11<00:45, 52.80it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  20%|██        | 613/3000 [00:11<00:44, 53.10it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  21%|██▏       | 640/3000 [00:12<00:44, 52.83it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  22%|██▏       | 667/3000 [00:12<00:44, 52.47it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  23%|██▎       | 695/3000 [00:13<00:43, 53.31it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  24%|██▍       | 722/3000 [00:13<00:43, 52.91it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  25%|██▌       | 750/3000 [00:14<00:42, 53.39it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  26%|██▌       | 777/3000 [00:14<00:41, 53.04it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  27%|██▋       | 805/3000 [00:15<00:41, 53.29it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  28%|██▊       | 832/3000 [00:15<00:40, 53.44it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  29%|██▊       | 860/3000 [00:16<00:39, 53.63it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  30%|██▉       | 888/3000 [00:16<00:39, 54.00it/s, epoch 2 | train=0.1643 | valid=0.2067 | fun_rmse=0.4740]

Training:  30%|███       | 900/3000 [00:16<00:38, 54.00it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  31%|███       | 916/3000 [00:17<00:38, 53.59it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  32%|███▏      | 945/3000 [00:17<00:37, 54.68it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  32%|███▏      | 974/3000 [00:18<00:36, 55.47it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  33%|███▎      | 1002/3000 [00:18<00:36, 54.91it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  34%|███▍      | 1030/3000 [00:19<00:36, 54.27it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  35%|███▌      | 1058/3000 [00:19<00:35, 54.71it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  36%|███▌      | 1086/3000 [00:20<00:35, 54.16it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  37%|███▋      | 1114/3000 [00:20<00:35, 53.67it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  38%|███▊      | 1142/3000 [00:21<00:34, 53.70it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  39%|███▉      | 1169/3000 [00:21<00:34, 53.69it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  40%|███▉      | 1197/3000 [00:22<00:33, 53.98it/s, epoch 3 | train=0.0872 | valid=0.1840 | fun_rmse=0.4146]

Training:  40%|████      | 1200/3000 [00:22<00:33, 53.98it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  41%|████      | 1225/3000 [00:22<00:32, 54.11it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  42%|████▏     | 1253/3000 [00:23<00:32, 53.90it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  43%|████▎     | 1281/3000 [00:24<00:31, 54.24it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  44%|████▎     | 1309/3000 [00:24<00:30, 54.58it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  45%|████▍     | 1337/3000 [00:25<00:30, 54.58it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  46%|████▌     | 1365/3000 [00:25<00:30, 53.17it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  46%|████▋     | 1392/3000 [00:26<00:30, 52.50it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  47%|████▋     | 1419/3000 [00:26<00:30, 51.28it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  48%|████▊     | 1446/3000 [00:27<00:29, 51.89it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  49%|████▉     | 1472/3000 [00:27<00:29, 51.62it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  50%|████▉     | 1498/3000 [00:28<00:29, 51.50it/s, epoch 4 | train=0.0717 | valid=0.1597 | fun_rmse=0.4064]

Training:  50%|█████     | 1500/3000 [00:28<00:29, 51.50it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  51%|█████     | 1524/3000 [00:28<00:28, 51.47it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  52%|█████▏    | 1552/3000 [00:29<00:27, 52.33it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  53%|█████▎    | 1579/3000 [00:29<00:26, 52.66it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  54%|█████▎    | 1607/3000 [00:30<00:26, 53.11it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  55%|█████▍    | 1635/3000 [00:30<00:25, 53.89it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  55%|█████▌    | 1662/3000 [00:31<00:24, 53.92it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  56%|█████▋    | 1690/3000 [00:31<00:24, 54.17it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  57%|█████▋    | 1719/3000 [00:32<00:23, 54.86it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  58%|█████▊    | 1747/3000 [00:32<00:22, 54.92it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  59%|█████▉    | 1776/3000 [00:33<00:22, 55.42it/s, epoch 5 | train=0.0636 | valid=0.1674 | fun_rmse=0.4090]

Training:  60%|██████    | 1800/3000 [00:33<00:21, 55.42it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  60%|██████    | 1804/3000 [00:33<00:21, 55.18it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  61%|██████    | 1832/3000 [00:34<00:21, 54.02it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  62%|██████▏   | 1860/3000 [00:34<00:20, 54.49it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  63%|██████▎   | 1888/3000 [00:35<00:20, 54.67it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  64%|██████▍   | 1916/3000 [00:35<00:19, 54.33it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  65%|██████▍   | 1944/3000 [00:36<00:19, 53.34it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  66%|██████▌   | 1971/3000 [00:36<00:19, 52.15it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  67%|██████▋   | 1999/3000 [00:37<00:18, 52.75it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  68%|██████▊   | 2026/3000 [00:38<00:18, 52.82it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  68%|██████▊   | 2053/3000 [00:38<00:17, 52.87it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  69%|██████▉   | 2081/3000 [00:39<00:17, 53.44it/s, epoch 6 | train=0.0600 | valid=0.1545 | fun_rmse=0.4131]

Training:  70%|███████   | 2100/3000 [00:39<00:16, 53.44it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  70%|███████   | 2108/3000 [00:39<00:16, 53.30it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  71%|███████   | 2136/3000 [00:40<00:15, 54.05it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  72%|███████▏  | 2164/3000 [00:40<00:15, 53.43it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  73%|███████▎  | 2191/3000 [00:41<00:15, 53.27it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  74%|███████▍  | 2218/3000 [00:41<00:14, 53.21it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  75%|███████▍  | 2245/3000 [00:42<00:14, 53.34it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  76%|███████▌  | 2272/3000 [00:42<00:13, 53.07it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  77%|███████▋  | 2300/3000 [00:43<00:13, 53.60it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  78%|███████▊  | 2328/3000 [00:43<00:12, 54.24it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  79%|███████▊  | 2356/3000 [00:44<00:11, 54.70it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  79%|███████▉  | 2384/3000 [00:44<00:11, 54.73it/s, epoch 7 | train=0.0563 | valid=0.1771 | fun_rmse=0.4132]

Training:  80%|████████  | 2400/3000 [00:44<00:10, 54.73it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  80%|████████  | 2412/3000 [00:45<00:10, 54.72it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  81%|████████▏ | 2441/3000 [00:45<00:10, 55.19it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  82%|████████▏ | 2470/3000 [00:46<00:09, 55.53it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  83%|████████▎ | 2498/3000 [00:46<00:09, 55.61it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  84%|████████▍ | 2526/3000 [00:47<00:08, 55.03it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  85%|████████▌ | 2555/3000 [00:47<00:08, 55.40it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  86%|████████▌ | 2584/3000 [00:48<00:07, 55.78it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  87%|████████▋ | 2613/3000 [00:48<00:06, 56.27it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  88%|████████▊ | 2642/3000 [00:49<00:06, 55.97it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  89%|████████▉ | 2671/3000 [00:49<00:05, 56.17it/s, epoch 8 | train=0.0522 | valid=0.1928 | fun_rmse=0.4178]

Training:  90%|█████████ | 2700/3000 [00:50<00:05, 56.17it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  90%|█████████ | 2701/3000 [00:50<00:05, 56.22it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  91%|█████████ | 2730/3000 [00:50<00:04, 55.70it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  92%|█████████▏| 2758/3000 [00:51<00:04, 55.37it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  93%|█████████▎| 2786/3000 [00:51<00:03, 55.54it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  94%|█████████▍| 2815/3000 [00:52<00:03, 55.77it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  95%|█████████▍| 2844/3000 [00:52<00:02, 56.05it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  96%|█████████▌| 2873/3000 [00:53<00:02, 55.52it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  97%|█████████▋| 2901/3000 [00:53<00:01, 55.16it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  98%|█████████▊| 2929/3000 [00:54<00:01, 52.11it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  99%|█████████▊| 2956/3000 [00:55<00:00, 52.42it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training:  99%|█████████▉| 2984/3000 [00:55<00:00, 53.25it/s, epoch 9 | train=0.0379 | valid=0.1941 | fun_rmse=0.4184]

Training: 100%|██████████| 3000/3000 [00:55<00:00, 53.25it/s, epoch 10 | train=0.0317 | valid=0.1885 | fun_rmse=0.4151]

Training: 100%|██████████| 3000/3000 [00:55<00:00, 53.74it/s, epoch 10 | train=0.0317 | valid=0.1885 | fun_rmse=0.4151]

Visualize Results¶

ds_idx=-1 shows the last validation/test set. The first 50 timesteps (diagnosis window) are zero-padded because the model uses that region for state estimation rather than prediction.

In [5]:

lrn.show_results(ds_idx=-1, max_n=2)

lrn.show_results(ds_idx=-1, max_n=2)

Adding Activation Regularization¶

FranSys models benefit significantly from activation regularization, which encourages smoother predictions. ActivationRegularizer penalizes large activations, TemporalActivationRegularizer penalizes abrupt changes between timesteps. We need to extract the prognosis RNN module from the model so the regularizers know which layer to hook into.

Pass them as auxiliary losses via lrn.add_aux_loss(...).

In [6]:

lrn_reg = FranSysLearner(
    dls, init_sz=50, attach_output=True,
    hidden_size=40, metrics=[fun_rmse]
)
model_reg = unwrap_model(lrn_reg.model)
lrn_reg.add_aux_loss(
    ActivationRegularizer(modules=[model_reg.rnn_prognosis], alpha=6.0)
)
lrn_reg.add_aux_loss(
    TemporalActivationRegularizer(modules=[model_reg.rnn_prognosis], beta=6.0)
)
lrn_reg.fit_flat_cos(n_epoch=10, lr=3e-3)
lrn_reg.show_results(ds_idx=-1, max_n=2)

lrn_reg = FranSysLearner( dls, init_sz=50, attach_output=True, hidden_size=40, metrics=[fun_rmse] ) model_reg = unwrap_model(lrn_reg.model) lrn_reg.add_aux_loss( ActivationRegularizer(modules=[model_reg.rnn_prognosis], alpha=6.0) ) lrn_reg.add_aux_loss( TemporalActivationRegularizer(modules=[model_reg.rnn_prognosis], beta=6.0) ) lrn_reg.fit_flat_cos(n_epoch=10, lr=3e-3) lrn_reg.show_results(ds_idx=-1, max_n=2)

Training:   0%|          | 0/3000 [00:00<?, ?it/s]

Training:   1%|          | 22/3000 [00:00<01:08, 43.72it/s]

Training:   2%|▏         | 51/3000 [00:01<00:57, 51.70it/s]

Training:   3%|▎         | 79/3000 [00:01<00:54, 53.37it/s]

Training:   4%|▎         | 106/3000 [00:02<00:54, 53.41it/s]

Training:   4%|▍         | 135/3000 [00:02<00:52, 54.53it/s]

Training:   5%|▌         | 164/3000 [00:03<00:51, 55.22it/s]

Training:   6%|▋         | 193/3000 [00:03<00:50, 55.59it/s]

Training:   7%|▋         | 221/3000 [00:04<00:50, 55.24it/s]

Training:   8%|▊         | 249/3000 [00:04<00:50, 54.97it/s]

Training:   9%|▉         | 277/3000 [00:05<00:49, 55.23it/s]

Training:  10%|█         | 300/3000 [00:05<00:48, 55.23it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  10%|█         | 305/3000 [00:05<00:48, 55.14it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  11%|█         | 333/3000 [00:06<00:49, 53.64it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  12%|█▏        | 361/3000 [00:06<00:48, 53.90it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  13%|█▎        | 389/3000 [00:07<00:48, 54.01it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  14%|█▍        | 417/3000 [00:07<00:47, 54.13it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  15%|█▍        | 445/3000 [00:08<00:47, 54.12it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  16%|█▌        | 473/3000 [00:08<00:46, 54.22it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  17%|█▋        | 501/3000 [00:09<00:45, 54.58it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  18%|█▊        | 529/3000 [00:09<00:45, 54.20it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  19%|█▊        | 557/3000 [00:10<00:45, 54.26it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  20%|█▉        | 585/3000 [00:10<00:44, 54.23it/s, epoch 1 | train=1.6830 | valid=0.3161 | fun_rmse=0.7831]

Training:  20%|██        | 600/3000 [00:11<00:44, 54.23it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  20%|██        | 613/3000 [00:11<00:43, 54.50it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  21%|██▏       | 641/3000 [00:11<00:43, 54.24it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  22%|██▏       | 669/3000 [00:12<00:43, 54.11it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  23%|██▎       | 697/3000 [00:12<00:42, 53.79it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  24%|██▍       | 724/3000 [00:13<00:42, 53.42it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  25%|██▌       | 751/3000 [00:13<00:42, 53.04it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  26%|██▌       | 779/3000 [00:14<00:41, 53.62it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  27%|██▋       | 806/3000 [00:14<00:41, 53.27it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  28%|██▊       | 834/3000 [00:15<00:40, 53.53it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  29%|██▊       | 861/3000 [00:15<00:39, 53.59it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  30%|██▉       | 888/3000 [00:16<00:39, 53.38it/s, epoch 2 | train=0.4560 | valid=0.2180 | fun_rmse=0.5742]

Training:  30%|███       | 900/3000 [00:16<00:39, 53.38it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  30%|███       | 915/3000 [00:16<00:39, 53.34it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  31%|███▏      | 942/3000 [00:17<00:38, 53.09it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  32%|███▏      | 970/3000 [00:17<00:37, 53.51it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  33%|███▎      | 998/3000 [00:18<00:37, 53.96it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  34%|███▍      | 1026/3000 [00:19<00:36, 54.14it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  35%|███▌      | 1054/3000 [00:19<00:36, 54.06it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  36%|███▌      | 1082/3000 [00:20<00:35, 54.35it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  37%|███▋      | 1110/3000 [00:20<00:35, 53.70it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  38%|███▊      | 1137/3000 [00:21<00:34, 53.58it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  39%|███▉      | 1165/3000 [00:21<00:33, 54.07it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  40%|███▉      | 1193/3000 [00:22<00:33, 54.54it/s, epoch 3 | train=0.3285 | valid=0.1785 | fun_rmse=0.4479]

Training:  40%|████      | 1200/3000 [00:22<00:33, 54.54it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  41%|████      | 1221/3000 [00:22<00:32, 54.30it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  42%|████▏     | 1249/3000 [00:23<00:32, 53.90it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  43%|████▎     | 1276/3000 [00:23<00:31, 53.90it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  43%|████▎     | 1304/3000 [00:24<00:31, 54.33it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  44%|████▍     | 1332/3000 [00:24<00:30, 54.77it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  45%|████▌     | 1360/3000 [00:25<00:29, 55.03it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  46%|████▋     | 1388/3000 [00:25<00:29, 54.39it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  47%|████▋     | 1417/3000 [00:26<00:28, 54.92it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  48%|████▊     | 1445/3000 [00:26<00:28, 55.08it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  49%|████▉     | 1473/3000 [00:27<00:27, 54.74it/s, epoch 4 | train=0.2664 | valid=0.1720 | fun_rmse=0.3974]

Training:  50%|█████     | 1500/3000 [00:27<00:27, 54.74it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  50%|█████     | 1501/3000 [00:27<00:27, 54.11it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  51%|█████     | 1529/3000 [00:28<00:27, 52.58it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  52%|█████▏    | 1556/3000 [00:28<00:27, 52.90it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  53%|█████▎    | 1583/3000 [00:29<00:26, 52.69it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  54%|█████▎    | 1610/3000 [00:29<00:26, 52.85it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  55%|█████▍    | 1637/3000 [00:30<00:25, 53.03it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  56%|█████▌    | 1665/3000 [00:30<00:24, 53.62it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  56%|█████▋    | 1692/3000 [00:31<00:24, 53.60it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  57%|█████▋    | 1719/3000 [00:31<00:24, 53.22it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  58%|█████▊    | 1746/3000 [00:32<00:23, 52.77it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  59%|█████▉    | 1773/3000 [00:32<00:23, 51.79it/s, epoch 5 | train=0.2261 | valid=0.1381 | fun_rmse=0.2687]

Training:  60%|██████    | 1800/3000 [00:33<00:23, 51.79it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  60%|██████    | 1801/3000 [00:33<00:23, 51.95it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  61%|██████    | 1827/3000 [00:34<00:23, 50.19it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  62%|██████▏   | 1853/3000 [00:34<00:22, 49.93it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  63%|██████▎   | 1880/3000 [00:35<00:21, 51.02it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  64%|██████▎   | 1906/3000 [00:35<00:21, 51.24it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  64%|██████▍   | 1932/3000 [00:36<00:20, 51.37it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  65%|██████▌   | 1959/3000 [00:36<00:19, 52.11it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  66%|██████▌   | 1986/3000 [00:37<00:19, 51.51it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  67%|██████▋   | 2012/3000 [00:37<00:20, 47.20it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  68%|██████▊   | 2038/3000 [00:38<00:19, 48.16it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  69%|██████▉   | 2063/3000 [00:38<00:19, 48.58it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  70%|██████▉   | 2090/3000 [00:39<00:18, 49.66it/s, epoch 6 | train=0.2032 | valid=0.1511 | fun_rmse=0.3128]

Training:  70%|███████   | 2100/3000 [00:39<00:18, 49.66it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  71%|███████   | 2116/3000 [00:39<00:17, 49.86it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  71%|███████▏  | 2142/3000 [00:40<00:17, 49.66it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  72%|███████▏  | 2167/3000 [00:40<00:16, 49.47it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  73%|███████▎  | 2193/3000 [00:41<00:16, 50.03it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  74%|███████▍  | 2219/3000 [00:41<00:15, 50.16it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  75%|███████▍  | 2245/3000 [00:42<00:15, 49.83it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  76%|███████▌  | 2270/3000 [00:42<00:14, 49.59it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  77%|███████▋  | 2296/3000 [00:43<00:14, 50.21it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  77%|███████▋  | 2322/3000 [00:43<00:13, 50.20it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  78%|███████▊  | 2348/3000 [00:44<00:12, 50.56it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  79%|███████▉  | 2374/3000 [00:45<00:12, 49.58it/s, epoch 7 | train=0.1856 | valid=0.1429 | fun_rmse=0.2489]

Training:  80%|████████  | 2400/3000 [00:45<00:12, 49.58it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  80%|████████  | 2401/3000 [00:45<00:11, 50.78it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  81%|████████  | 2427/3000 [00:46<00:11, 49.85it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  82%|████████▏ | 2453/3000 [00:46<00:10, 50.02it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  83%|████████▎ | 2479/3000 [00:47<00:10, 49.73it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  84%|████████▎ | 2505/3000 [00:47<00:09, 50.01it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  84%|████████▍ | 2531/3000 [00:48<00:09, 50.34it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  85%|████████▌ | 2557/3000 [00:48<00:08, 50.13it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  86%|████████▌ | 2583/3000 [00:49<00:08, 49.52it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  87%|████████▋ | 2608/3000 [00:49<00:07, 49.42it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  88%|████████▊ | 2634/3000 [00:50<00:07, 49.92it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  89%|████████▊ | 2659/3000 [00:50<00:06, 49.71it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  89%|████████▉ | 2684/3000 [00:51<00:06, 48.92it/s, epoch 8 | train=0.1703 | valid=0.1478 | fun_rmse=0.2899]

Training:  90%|█████████ | 2700/3000 [00:51<00:06, 48.92it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  90%|█████████ | 2710/3000 [00:51<00:05, 49.62it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  91%|█████████ | 2736/3000 [00:52<00:05, 49.88it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  92%|█████████▏| 2762/3000 [00:52<00:04, 50.44it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  93%|█████████▎| 2789/3000 [00:53<00:04, 50.71it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  94%|█████████▍| 2815/3000 [00:53<00:03, 50.86it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  95%|█████████▍| 2843/3000 [00:54<00:03, 52.20it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  96%|█████████▌| 2870/3000 [00:54<00:02, 52.31it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  97%|█████████▋| 2897/3000 [00:55<00:01, 52.69it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  97%|█████████▋| 2924/3000 [00:55<00:01, 52.40it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  98%|█████████▊| 2951/3000 [00:56<00:00, 52.43it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training:  99%|█████████▉| 2979/3000 [00:56<00:00, 52.94it/s, epoch 9 | train=0.1440 | valid=0.1273 | fun_rmse=0.2511]

Training: 100%|██████████| 3000/3000 [00:57<00:00, 52.94it/s, epoch 10 | train=0.1257 | valid=0.1331 | fun_rmse=0.2600]

Training: 100%|██████████| 3000/3000 [00:57<00:00, 52.43it/s, epoch 10 | train=0.1257 | valid=0.1331 | fun_rmse=0.2600]

FranSysRegularizer for State Synchronization¶

FranSysRegularizer adds an auxiliary loss that encourages the prognosis RNN to maintain state consistency: the hidden state at any point in the prognosis should be similar to what the diagnosis RNN would produce from the same data window. This improves long-horizon stability.

The regularizer requires the diagnosis and prognosis modules to be passed explicitly so it can hook into both and compare their hidden states.

In [7]:

lrn_sync = FranSysLearner(
    dls, init_sz=50, attach_output=True,
    hidden_size=40, metrics=[fun_rmse]
)
model_sync = unwrap_model(lrn_sync.model)
lrn_sync.add_aux_loss(
    ActivationRegularizer(modules=[model_sync.rnn_prognosis], alpha=6.0)
)
lrn_sync.add_aux_loss(
    TemporalActivationRegularizer(modules=[model_sync.rnn_prognosis], beta=6.0)
)
lrn_sync.add_aux_loss(
    FranSysRegularizer(
        modules=[model_sync.rnn_diagnosis, model_sync.rnn_prognosis],
        model=model_sync,
    )
)
lrn_sync.fit_flat_cos(n_epoch=10, lr=3e-3)
lrn_sync.show_results(ds_idx=-1, max_n=2)

lrn_sync = FranSysLearner( dls, init_sz=50, attach_output=True, hidden_size=40, metrics=[fun_rmse] ) model_sync = unwrap_model(lrn_sync.model) lrn_sync.add_aux_loss( ActivationRegularizer(modules=[model_sync.rnn_prognosis], alpha=6.0) ) lrn_sync.add_aux_loss( TemporalActivationRegularizer(modules=[model_sync.rnn_prognosis], beta=6.0) ) lrn_sync.add_aux_loss( FranSysRegularizer( modules=[model_sync.rnn_diagnosis, model_sync.rnn_prognosis], model=model_sync, ) ) lrn_sync.fit_flat_cos(n_epoch=10, lr=3e-3) lrn_sync.show_results(ds_idx=-1, max_n=2)

Training:   0%|          | 0/3000 [00:00<?, ?it/s]

Training:   1%|          | 17/3000 [00:00<01:28, 33.56it/s]

Training:   1%|▏         | 43/3000 [00:01<01:07, 43.57it/s]

Training:   2%|▏         | 66/3000 [00:01<01:06, 44.36it/s]

Training:   3%|▎         | 89/3000 [00:02<01:06, 44.09it/s]

Training:   4%|▎         | 112/3000 [00:02<01:06, 43.29it/s]

Training:   5%|▍         | 136/3000 [00:03<01:03, 44.78it/s]

Training:   5%|▌         | 160/3000 [00:03<01:02, 45.44it/s]

Training:   6%|▌         | 183/3000 [00:04<01:01, 45.47it/s]

Training:   7%|▋         | 206/3000 [00:04<01:04, 43.50it/s]

Training:   8%|▊         | 230/3000 [00:05<01:02, 44.64it/s]

Training:   8%|▊         | 254/3000 [00:05<01:00, 45.18it/s]

Training:   9%|▉         | 277/3000 [00:06<01:00, 44.87it/s]

Training:  10%|█         | 300/3000 [00:06<01:00, 44.87it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  10%|█         | 301/3000 [00:06<01:00, 44.71it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  11%|█         | 324/3000 [00:07<01:00, 43.88it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  12%|█▏        | 346/3000 [00:07<01:01, 43.15it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  12%|█▏        | 368/3000 [00:08<01:02, 41.92it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  13%|█▎        | 389/3000 [00:08<01:03, 41.09it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  14%|█▎        | 411/3000 [00:09<01:02, 41.51it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  14%|█▍        | 432/3000 [00:09<01:02, 41.35it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  15%|█▌        | 453/3000 [00:10<01:01, 41.37it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  16%|█▌        | 474/3000 [00:11<01:01, 40.78it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  16%|█▋        | 495/3000 [00:11<01:03, 39.59it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  17%|█▋        | 515/3000 [00:12<01:03, 39.10it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  18%|█▊        | 536/3000 [00:12<01:01, 39.84it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  19%|█▊        | 556/3000 [00:13<01:01, 39.49it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  19%|█▉        | 576/3000 [00:13<01:01, 39.28it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  20%|█▉        | 596/3000 [00:14<01:00, 39.44it/s, epoch 1 | train=8.8804 | valid=1.0636 | fun_rmse=1.3367]

Training:  20%|██        | 600/3000 [00:14<01:00, 39.44it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  21%|██        | 617/3000 [00:14<00:59, 39.85it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  21%|██▏       | 638/3000 [00:15<00:58, 40.48it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  22%|██▏       | 659/3000 [00:15<00:57, 40.64it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  23%|██▎       | 680/3000 [00:16<00:57, 40.51it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  23%|██▎       | 701/3000 [00:16<00:56, 40.49it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  24%|██▍       | 722/3000 [00:17<00:57, 39.81it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  25%|██▍       | 742/3000 [00:17<00:57, 39.45it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  25%|██▌       | 762/3000 [00:18<00:57, 38.60it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  26%|██▌       | 783/3000 [00:18<00:56, 39.15it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  27%|██▋       | 803/3000 [00:19<00:56, 39.19it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  27%|██▋       | 823/3000 [00:19<00:56, 38.67it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  28%|██▊       | 843/3000 [00:20<00:55, 38.95it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  29%|██▉       | 863/3000 [00:20<00:54, 39.04it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  29%|██▉       | 884/3000 [00:21<00:53, 39.87it/s, epoch 2 | train=1.8196 | valid=0.5267 | fun_rmse=0.7548]

Training:  30%|███       | 900/3000 [00:21<00:52, 39.87it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  30%|███       | 905/3000 [00:21<00:52, 40.21it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  31%|███       | 926/3000 [00:22<00:53, 38.49it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  32%|███▏      | 946/3000 [00:23<00:54, 38.00it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  32%|███▏      | 966/3000 [00:23<00:54, 37.63it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  33%|███▎      | 985/3000 [00:24<00:53, 37.56it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  33%|███▎      | 1004/3000 [00:24<00:53, 37.08it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  34%|███▍      | 1024/3000 [00:25<00:52, 37.67it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  35%|███▍      | 1043/3000 [00:25<00:51, 37.64it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  35%|███▌      | 1062/3000 [00:26<00:51, 37.65it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  36%|███▌      | 1081/3000 [00:26<00:52, 36.77it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  37%|███▋      | 1100/3000 [00:27<00:51, 36.95it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  37%|███▋      | 1119/3000 [00:27<00:50, 36.89it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  38%|███▊      | 1138/3000 [00:28<00:50, 37.01it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  39%|███▊      | 1159/3000 [00:28<00:48, 38.18it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  39%|███▉      | 1181/3000 [00:29<00:45, 39.59it/s, epoch 3 | train=0.7505 | valid=0.2486 | fun_rmse=0.5794]

Training:  40%|████      | 1200/3000 [00:29<00:45, 39.59it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  40%|████      | 1202/3000 [00:29<00:44, 40.12it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  41%|████      | 1223/3000 [00:30<00:44, 40.32it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  41%|████▏     | 1244/3000 [00:30<00:43, 40.46it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  42%|████▏     | 1265/3000 [00:31<00:42, 40.90it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  43%|████▎     | 1288/3000 [00:31<00:40, 42.01it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  44%|████▎     | 1311/3000 [00:32<00:39, 42.71it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  44%|████▍     | 1335/3000 [00:32<00:38, 43.81it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  45%|████▌     | 1358/3000 [00:33<00:37, 44.09it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  46%|████▌     | 1383/3000 [00:33<00:35, 45.52it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  47%|████▋     | 1407/3000 [00:34<00:34, 45.87it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  48%|████▊     | 1432/3000 [00:34<00:33, 46.79it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  49%|████▊     | 1456/3000 [00:35<00:33, 46.38it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  49%|████▉     | 1481/3000 [00:35<00:32, 47.16it/s, epoch 4 | train=0.4870 | valid=0.3183 | fun_rmse=0.7603]

Training:  50%|█████     | 1500/3000 [00:36<00:31, 47.16it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  50%|█████     | 1506/3000 [00:36<00:31, 47.87it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  51%|█████     | 1530/3000 [00:36<00:30, 47.50it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  52%|█████▏    | 1554/3000 [00:37<00:30, 46.83it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  53%|█████▎    | 1578/3000 [00:38<00:30, 47.00it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  53%|█████▎    | 1602/3000 [00:38<00:30, 45.71it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  54%|█████▍    | 1625/3000 [00:39<00:30, 44.58it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  55%|█████▍    | 1648/3000 [00:39<00:31, 43.53it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  56%|█████▌    | 1671/3000 [00:40<00:30, 44.19it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  56%|█████▋    | 1694/3000 [00:40<00:29, 44.33it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  57%|█████▋    | 1717/3000 [00:41<00:28, 44.35it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  58%|█████▊    | 1740/3000 [00:41<00:28, 43.99it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  59%|█████▉    | 1763/3000 [00:42<00:28, 43.40it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  60%|█████▉    | 1785/3000 [00:42<00:27, 43.45it/s, epoch 5 | train=0.4113 | valid=0.3224 | fun_rmse=0.7622]

Training:  60%|██████    | 1800/3000 [00:43<00:27, 43.45it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  60%|██████    | 1808/3000 [00:43<00:27, 43.93it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  61%|██████    | 1832/3000 [00:43<00:26, 44.87it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  62%|██████▏   | 1855/3000 [00:44<00:25, 44.64it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  63%|██████▎   | 1878/3000 [00:44<00:25, 44.61it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  63%|██████▎   | 1901/3000 [00:45<00:24, 44.91it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  64%|██████▍   | 1924/3000 [00:45<00:23, 44.97it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  65%|██████▍   | 1947/3000 [00:46<00:23, 44.29it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  66%|██████▌   | 1970/3000 [00:46<00:23, 44.50it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  66%|██████▋   | 1993/3000 [00:47<00:22, 44.44it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  67%|██████▋   | 2016/3000 [00:47<00:22, 44.03it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  68%|██████▊   | 2039/3000 [00:48<00:21, 43.97it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  69%|██████▊   | 2061/3000 [00:49<00:21, 43.94it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  69%|██████▉   | 2084/3000 [00:49<00:20, 44.03it/s, epoch 6 | train=0.3744 | valid=0.3119 | fun_rmse=0.7289]

Training:  70%|███████   | 2100/3000 [00:49<00:20, 44.03it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  70%|███████   | 2108/3000 [00:50<00:19, 44.63it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  71%|███████   | 2131/3000 [00:50<00:19, 44.83it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  72%|███████▏  | 2155/3000 [00:51<00:18, 45.72it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  73%|███████▎  | 2179/3000 [00:51<00:17, 46.06it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  73%|███████▎  | 2203/3000 [00:52<00:17, 45.24it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  74%|███████▍  | 2226/3000 [00:52<00:17, 44.62it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  75%|███████▍  | 2249/3000 [00:53<00:16, 44.87it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  76%|███████▌  | 2272/3000 [00:53<00:16, 44.88it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  76%|███████▋  | 2295/3000 [00:54<00:15, 44.46it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  77%|███████▋  | 2318/3000 [00:54<00:15, 44.72it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  78%|███████▊  | 2341/3000 [00:55<00:14, 44.03it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  79%|███████▉  | 2364/3000 [00:55<00:14, 44.33it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  80%|███████▉  | 2387/3000 [00:56<00:14, 43.18it/s, epoch 7 | train=0.3242 | valid=0.3214 | fun_rmse=0.6675]

Training:  80%|████████  | 2400/3000 [00:56<00:13, 43.18it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  80%|████████  | 2410/3000 [00:56<00:13, 43.76it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  81%|████████  | 2432/3000 [00:57<00:13, 43.66it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  82%|████████▏ | 2454/3000 [00:57<00:12, 42.61it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  83%|████████▎ | 2476/3000 [00:58<00:13, 40.10it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  83%|████████▎ | 2497/3000 [00:59<00:12, 39.64it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  84%|████████▍ | 2520/3000 [00:59<00:11, 41.30it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  85%|████████▍ | 2543/3000 [01:00<00:10, 42.41it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  86%|████████▌ | 2565/3000 [01:00<00:10, 42.53it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  86%|████████▌ | 2587/3000 [01:01<00:09, 42.26it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  87%|████████▋ | 2610/3000 [01:01<00:09, 42.92it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  88%|████████▊ | 2633/3000 [01:02<00:08, 43.46it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  89%|████████▊ | 2657/3000 [01:02<00:07, 44.77it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  89%|████████▉ | 2681/3000 [01:03<00:07, 45.28it/s, epoch 8 | train=0.3045 | valid=0.2802 | fun_rmse=0.6258]

Training:  90%|█████████ | 2700/3000 [01:03<00:06, 45.28it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  90%|█████████ | 2704/3000 [01:03<00:06, 45.08it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  91%|█████████ | 2728/3000 [01:04<00:05, 45.39it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  92%|█████████▏| 2751/3000 [01:04<00:05, 45.56it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  92%|█████████▎| 2775/3000 [01:05<00:04, 46.26it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  93%|█████████▎| 2799/3000 [01:05<00:04, 46.13it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  94%|█████████▍| 2823/3000 [01:06<00:03, 45.72it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  95%|█████████▍| 2848/3000 [01:06<00:03, 46.53it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  96%|█████████▌| 2872/3000 [01:07<00:02, 46.59it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  97%|█████████▋| 2896/3000 [01:07<00:02, 46.17it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  97%|█████████▋| 2920/3000 [01:08<00:01, 46.39it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  98%|█████████▊| 2944/3000 [01:08<00:01, 46.14it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training:  99%|█████████▉| 2968/3000 [01:09<00:00, 46.33it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training: 100%|█████████▉| 2992/3000 [01:09<00:00, 46.21it/s, epoch 9 | train=0.2687 | valid=0.2396 | fun_rmse=0.5690]

Training: 100%|██████████| 3000/3000 [01:10<00:00, 46.21it/s, epoch 10 | train=0.2401 | valid=0.2555 | fun_rmse=0.5634]

Training: 100%|██████████| 3000/3000 [01:10<00:00, 42.84it/s, epoch 10 | train=0.2401 | valid=0.2555 | fun_rmse=0.5634]

Key Takeaways¶

• FranSys separates state estimation (diagnosis) from forward prediction (prognosis).
• init_sz controls how many timesteps are used to initialize the hidden state from measured data.
• attach_output=True enables prediction mode (output feedback).
• ActivationRegularizer and TemporalActivationRegularizer are especially important for FranSys -- they encourage smooth, stable predictions. Pass them via lrn.add_aux_loss(...).
• FranSysRegularizer adds state synchronization regularization for improved long-horizon stability. It requires the diagnosis and prognosis modules to be passed so it can compare their hidden states.
• The architecture naturally handles variable initial conditions.