Metrics

Fill in a module description here

source

rmse

 rmse (y_true:numpy.ndarray, y_pred:numpy.ndarray, time_axis:int=0)

*Computes the Root Mean Square Error (RMSE) along a specified time axis.

Calculates RMSE = sqrt(mean((y_pred - y_true)**2)) separately for each channel defined by the remaining axes.*

Type Default Details
y_true ndarray Ground truth target values.
y_pred ndarray Estimated target values.
time_axis int 0 Axis representing time or samples.
Returns ndarray Root Mean Squared Error for each channel. 
# Example usage for rmse

# 1D case
y_t1 = np.array([1, 2, 3, 4, 5])
y_p1 = np.array([1.1, 2.1, 3.1, 4.1, 5.1]) # Small error
rmse_val_1d = rmse(y_t1, y_p1)
print(f"RMSE (1D): {rmse_val_1d}") # Should be ~0.1

# 2D case (e.g., time x features)
y_t2 = np.array([[1, 10], [2, 20], [3, 30]])
y_p2 = np.array([[1, 11], [2, 21], [3, 31]]) # Error=0 in col 0, Error=1 in col 1
rmse_val_2d = rmse(y_t2, y_p2, time_axis=0)
print(f"RMSE (2D, time_axis=0): {rmse_val_2d}") # Should be [0., 1.]

# 3D case (e.g., batch x time x features)
y_t3 = np.random.rand(2, 10, 3) # batch=2, time=10, features=3
y_p3 = y_t3 + np.random.randn(2, 10, 3) * 0.1
rmse_val_3d = rmse(y_t3, y_p3, time_axis=1) # Calculate RMSE over time axis
print(f"RMSE (3D, time_axis=1, shape={rmse_val_3d.shape}): {rmse_val_3d}") # Should have shape (2, 3)
RMSE (1D): 0.09999999999999991
RMSE (2D, time_axis=0): [0. 1.]
RMSE (3D, time_axis=1, shape=(2, 3)): [[0.09308798 0.11013668 0.11058777]
 [0.08573954 0.06386016 0.06679621]]

source

nrmse

 nrmse (y_true:numpy.ndarray, y_pred:numpy.ndarray, time_axis:int=0,
        std_tolerance:float=1e-09)

*Computes the Normalized Root Mean Square Error (NRMSE).

Calculates NRMSE = RMSE / std(y_true) separately for each channel. Returns NaN for channels where std(y_true) is close to zero (below std_tolerance).*

Type Default Details
y_true ndarray Ground truth target values.
y_pred ndarray Estimated target values.
time_axis int 0 Axis representing time or samples.
std_tolerance float 1e-09 Minimum standard deviation allowed for y_true to avoid division by zero.
Returns ndarray Normalized Root Mean Squared Error for each channel. 
# Example usage for nrmse

# 1D case
y_t1 = np.array([1, 2, 3, 4, 5])
y_p1 = y_t1 + 0.1 # Constant offset error
nrmse_val_1d = nrmse(y_t1, y_p1)
print(f"NRMSE (1D): {nrmse_val_1d}") # RMSE is 0.1, std(y_t1) is sqrt(2). NRMSE = 0.1 / sqrt(2)

# 2D case
y_t2 = np.array([[1, 10], [2, 20], [3, 30]])
y_p2 = np.array([[1, 11], [2, 21], [3, 31]]) # Error=0 in col 0, Error=1 in col 1
# Std dev col 0: ~0.816, Std dev col 1: ~8.165
# RMSE col 0: 0, RMSE col 1: 1
# NRMSE col 0: 0 / 0.816 = 0
# NRMSE col 1: 1 / 8.165 = ~0.122
nrmse_val_2d = nrmse(y_t2, y_p2, time_axis=0)
print(f"NRMSE (2D, time_axis=0): {nrmse_val_2d}") # Should be approx [0., 0.122]

# Case with zero standard deviation
y_t_const = np.array([5, 5, 5, 5])
y_p_const = np.array([5, 5, 5, 6]) # RMSE = 0.5
nrmse_val_const = nrmse(y_t_const, y_p_const)
print(f"NRMSE (Constant y_true): {nrmse_val_const}") # Should be NaN with a warning
NRMSE (1D): 0.07071067811865468
NRMSE (2D, time_axis=0): [0.         0.12247449]
NRMSE (Constant y_true): nan
/var/folders/pc/13zbh_m514n1tp522cx9npt00000gn/T/ipykernel_17861/2787241071.py:31: RuntimeWarning: Standard deviation of y_true is below tolerance (1e-09) for some channels. NRMSE set to NaN for these channels.
  warnings.warn(f"Standard deviation of y_true is below tolerance ({std_tolerance}) for some channels. NRMSE set to NaN for these channels.", RuntimeWarning)

source

fit_index

 fit_index (y_true:numpy.ndarray, y_pred:numpy.ndarray, time_axis:int=0,
            std_tolerance:float=1e-09)

*Computes the Fit Index (FIT) commonly used in System Identification.

Calculates FIT = 100 * (1 - NRMSE) separately for each channel. Returns NaN for channels where NRMSE could not be calculated (e.g., std(y_true) near zero).*

Type Default Details
y_true ndarray Ground truth target values.
y_pred ndarray Estimated target values.
time_axis int 0 Axis representing time or samples.
std_tolerance float 1e-09 Minimum standard deviation allowed for y_true.
Returns ndarray Fit index (in percent) for each channel. 
# Example usage for fit_index

# 1D case (using previous example)
# NRMSE = 0.1 / sqrt(2) approx 0.0707
# FIT = 100 * (1 - 0.0707) approx 92.93
fit_val_1d = fit_index(y_t1, y_p1)
print(f"Fit Index (1D): {fit_val_1d}")

# 2D case (using previous example)
# NRMSE approx [0., 0.122]
# FIT approx [100 * (1 - 0), 100 * (1 - 0.122)] = [100, 87.8]
fit_val_2d = fit_index(y_t2, y_p2, time_axis=0)
print(f"Fit Index (2D, time_axis=0): {fit_val_2d}")

# Constant case (using previous example)
# NRMSE is NaN
# FIT should also be NaN
fit_val_const = fit_index(y_t_const, y_p_const)
print(f"Fit Index (Constant y_true): {fit_val_const}")
Fit Index (1D): 92.92893218813452
Fit Index (2D, time_axis=0): [100.          87.75255129]
Fit Index (Constant y_true): nan
/var/folders/pc/13zbh_m514n1tp522cx9npt00000gn/T/ipykernel_17861/2787241071.py:31: RuntimeWarning: Standard deviation of y_true is below tolerance (1e-09) for some channels. NRMSE set to NaN for these channels.
  warnings.warn(f"Standard deviation of y_true is below tolerance ({std_tolerance}) for some channels. NRMSE set to NaN for these channels.", RuntimeWarning)

source

mae

 mae (y_true:numpy.ndarray, y_pred:numpy.ndarray, time_axis:int=0)

*Computes the Mean Absolute Error (MAE) along a specified time axis.

Calculates MAE = mean(abs(y_pred - y_true)) separately for each channel defined by the remaining axes.*

Type Default Details
y_true ndarray Ground truth target values.
y_pred ndarray Estimated target values.
time_axis int 0 Axis representing time or samples.
Returns ndarray Mean Absolute Error for each channel. 
# Example usage for mae

# 1D case
y_t1 = np.array([1, 2, 3, 4, 5])
y_p1 = np.array([1.1, 2.1, 3.1, 4.1, 5.1]) # Constant error of 0.1
mae_val_1d = mae(y_t1, y_p1)
print(f"MAE (1D): {mae_val_1d}") # Should be 0.1

y_p2 = np.array([0, 1, 2, 3, 4]) # Constant error of -1 -> abs error 1
mae_val_1d_neg = mae(y_t1, y_p2)
print(f"MAE (1D, negative err): {mae_val_1d_neg}") # Should be 1.0

# 2D case (e.g., time x features)
y_t2 = np.array([[1, 10], [2, 20], [3, 30]])
y_p3 = np.array([[1, 11], [2, 21], [3, 31]]) # Error=0 in col 0, Error=1 in col 1
mae_val_2d = mae(y_t2, y_p3, time_axis=0)
print(f"MAE (2D, time_axis=0): {mae_val_2d}") # Should be [0., 1.]
MAE (1D): 0.09999999999999991
MAE (1D, negative err): 1.0
MAE (2D, time_axis=0): [0. 1.]

source

r_squared

 r_squared (y_true:numpy.ndarray, y_pred:numpy.ndarray, time_axis:int=0,
            std_tolerance:float=1e-09)

*Computes the R-squared (coefficient of determination) score.

Calculates R^2 = 1 - NRMSE^2 separately for each channel. Returns NaN for channels where NRMSE could not be calculated (e.g., std(y_true) near zero). A constant model that always predicts the mean of y_true would get R^2=0.*

Type Default Details
y_true ndarray Ground truth target values.
y_pred ndarray Estimated target values.
time_axis int 0 Axis representing time or samples.
std_tolerance float 1e-09 Minimum standard deviation allowed for y_true.
Returns ndarray R-squared (coefficient of determination) for each channel. 
# Example usage for r_squared

# 1D case (using previous nrmse example)
# NRMSE = 0.1 / sqrt(2) approx 0.0707
# R^2 = 1 - (0.0707)^2 approx 1 - 0.005 = 0.995
y_t1 = np.array([1, 2, 3, 4, 5])
y_p1 = np.array([1.1, 2.1, 3.1, 4.1, 5.1])
r2_val_1d = r_squared(y_t1, y_p1)
print(f"R-squared (1D): {r2_val_1d}")

# Perfect prediction
r2_perfect = r_squared(y_t1, y_t1)
print(f"R-squared (1D, perfect): {r2_perfect}") # Should be 1.0

# 2D case (using previous nrmse example)
# NRMSE approx [0., 0.122]
# R^2 approx [1 - 0^2, 1 - 0.122^2] = [1, 1 - 0.0149] = [1, 0.9851]
y_t2 = np.array([[1, 10], [2, 20], [3, 30]])
y_p2 = np.array([[1, 11], [2, 21], [3, 31]])
r2_val_2d = r_squared(y_t2, y_p2, time_axis=0)
print(f"R-squared (2D, time_axis=0): {r2_val_2d}")

# Constant case (using previous nrmse example)
# NRMSE is NaN
# R^2 should also be NaN
y_t_const = np.array([5, 5, 5, 5])
y_p_const = np.array([5, 5, 5, 6])
r2_val_const = r_squared(y_t_const, y_p_const)
print(f"R-squared (Constant y_true): {r2_val_const}") # Should be NaN with a warning from nrmse
R-squared (1D): 0.995
R-squared (1D, perfect): 1.0
R-squared (2D, time_axis=0): [1.    0.985]
R-squared (Constant y_true): nan
/var/folders/pc/13zbh_m514n1tp522cx9npt00000gn/T/ipykernel_17861/2787241071.py:31: RuntimeWarning: Standard deviation of y_true is below tolerance (1e-09) for some channels. NRMSE set to NaN for these channels.
  warnings.warn(f"Standard deviation of y_true is below tolerance ({std_tolerance}) for some channels. NRMSE set to NaN for these channels.", RuntimeWarning)