Development documentation · 0.2.3 30c2f8ba · Intro examples tested with 0.2.3 · Version details · Known limitations

Preprocessing

Cleaning, normalization, and machine-learning preprocessing utilities, including MLPreprocessor, standardize / StandardizationModel, whiten / WhiteningModel, and impute.

Signal preprocessing algorithms.

class gwexpy.signal.preprocessing.WhiteningModel(mean, W)

Bases: object

Model resulting from whitening transformation.

Parameters:

• mean (ndarray) – Mean of the original data.

• W (ndarray) – Whitening matrix.

inverse_transform(X_w)

Project whitened data back to original space.

Parameters:

X_w (ndarray or array-like) – Whitened data with shape (n_samples, n_components).

Returns:

X_rec – Reconstructed data.

Return type:

ndarray

gwexpy.signal.preprocessing.whiten(X, *, method='pca', eps=None, n_components=None, return_model=True)

Whiten an array using PCA or ZCA whitening.

Parameters:

• X (ndarray) – Input data with shape (n_samples, n_features).

• method (str, optional) –

  Whitening method: ‘pca’ or ‘zca’. Default is ‘pca’.

  • 'pca': Principal Component Analysis whitening. Projects data onto principal components and scales to unit variance. The output may have a different orientation relative to the original feature space.

  • 'zca': Zero-phase Component Analysis whitening (also known as Mahalanobis whitening). The whitened data maintains maximum correlation with the original data while achieving decorrelation. This preserves the original axes alignment better than PCA.

• eps (float or str or None, optional) – Small constant added to eigenvalues to avoid division by zero. If None or ‘auto’ (default), the value is determined from data variance.

• n_components (int, optional) – Number of components to keep. If None, keep all components. For PCA, reduces dimensionality. For ZCA, reduces dimensionality but loses the channel-preserving property, and a warning is issued.

• return_model (bool, optional) – If True, return (X_whitened, model). If False, return only X_whitened.

Returns:

• X_whitened (ndarray) – Whitened data with shape (n_samples, n_components) or (n_samples, n_features) if n_components is None.

• model (WhiteningModel, optional) – Model for inverse transformation (if return_model=True).

Notes

Both PCA and ZCA whitening produce data with approximately identity covariance matrix (assuming sufficient samples). The difference is in the rotation:

• PCA: W = S^(-1/2) @ U^T where U and S are from SVD of covariance matrix.

• ZCA: W = U @ S^(-1/2) @ U^T, which applies the inverse rotation.

The inverse_transform method uses the pseudo-inverse of the whitening matrix to project back to the original space.

class gwexpy.signal.preprocessing.StandardizationModel(mean, scale, axis)

Bases: object

Model resulting from standardization transformation.

Parameters:

• mean (ndarray) – Mean of the original data.

• scale (ndarray) – Scale (std or MAD) of the original data.

• axis (int or str) – Axis along which standardization was performed.

inverse_transform(X_std)

Undo standardization: X = X_std * scale + mean.

Parameters:

X_std (ndarray or array-like) – Standardized data.

Returns:

X – Original-scale data.

Return type:

ndarray

gwexpy.signal.preprocessing.standardize(X, *, method='zscore', ddof=0, axis=-1, return_model=True)

Standardize an array using z-score or robust standardization.

Parameters:

• X (ndarray) – Input data.

• method (str, optional) –

  Standardization method: ‘zscore’ or ‘robust’ (alias: ‘mad’). Default is ‘zscore’.

  • 'zscore': Uses mean and standard deviation: (X - mean) / std.

  • 'robust' or 'mad': Uses median and MAD (median absolute deviation): (X - median) / (1.4826 * MAD). The constant 1.4826 is the reciprocal of the MAD of a standard normal distribution, which ensures that the resulting scale is equivalent to the standard deviation for Gaussian data.

• ddof (int, optional) – Delta degrees of freedom for std calculation. Default is 0.

• axis (int, optional) – Axis along which to standardize. Default is -1.

• return_model (bool, optional) – If True, return (X_standardized, model). If False, return only X_standardized.

Returns:

• X_standardized (ndarray) – Standardized data.

• model (StandardizationModel, optional) – Model for inverse transformation (if return_model=True).

Notes

NaN values are handled using nanmean/nanstd for zscore and nanmedian for robust methods, so NaN values in the input are ignored during computation but preserved in the output.

If the scale (std or MAD) is zero, a value of 1.0 is used to avoid division by zero.

gwexpy.signal.preprocessing.impute(values, *, method='interpolate', limit=None, times=None, max_gap=None, fill_value=nan)

Impute missing values in an array.

Parameters:

• values (ndarray) – 1D array with potential NaN values.

• method (str, optional) – Imputation method: ‘interpolate’, ‘ffill’, ‘bfill’, ‘mean’, ‘median’. Default is ‘interpolate’.

• limit (int, optional) – Maximum number of consecutive NaNs to fill. For ‘ffill’ and ‘bfill’, limits the forward/backward propagation. For ‘interpolate’, limits the number of consecutive NaNs that will be filled; any excess NaNs are restored to NaN after interpolation.

• times (ndarray, optional) – Time array corresponding to values. Used for time-based interpolation and max_gap calculation. Must be 1D and strictly increasing.

• max_gap (float, optional) – Maximum gap duration (in units of times) to fill. After interpolation, any NaN that was within a gap larger than this threshold is restored to NaN. This post-processing ensures that large temporal gaps are not bridged by interpolation.

• fill_value (float, optional) – Value to use for edge NaNs that cannot be interpolated or propagated. Only applies when there are no valid values to propagate from.

Returns:

imputed – Array with imputed values.

Return type:

ndarray

Notes

The max_gap parameter works as follows:

1. First, standard interpolation is performed.

2. Then, gaps in the original times array are identified.

3. Any interpolated values within gaps exceeding max_gap are reverted to NaN.

This “fill then restore” approach ensures that the interpolation algorithm can be applied uniformly while still respecting gap constraints.

Similarly, limit for ‘interpolate’ method:

1. Interpolation fills all internal NaNs.

2. Consecutive NaN runs longer than limit have their excess positions restored to NaN (forward direction).

class gwexpy.signal.preprocessing.MLPreprocessor(sample_rate: Quantity | float, freq_low: list[float] | None = None, freq_high: list[float] | None = None, filt_order: int = 8, valid_frac: float = 0.0, standardization_method: str = 'zscore')

Bases: object

Preprocessing pipeline for machine learning.

A scikit-learn-style Transformer that performs data splitting, band-pass filtering, and per-channel standardization. Can be used for noise removal tasks like DeepClean, as well as Random Forest, XGBoost, and other machine learning models.

Parameters:

• sample_rate (Quantity or float) – Sampling rate (in Hz).

• freq_low (list[float] or None, optional) – Low-frequency cutoff for the band-pass filter (supports multiple bands). If None, filtering is skipped.

• freq_high (list[float] or None, optional) – High-frequency cutoff for the band-pass filter (supports multiple bands). If None, filtering is skipped.

• filt_order (int, optional) – Order of the Butterworth filter (default: 8).

• valid_frac (float, optional) – Proportion of validation data (0.0 to 1.0, default: 0.0). If 0.0, no splitting is performed.

• standardization_method (str, optional) – Standardization method (‘zscore’ or ‘robust’, default: ‘zscore’).

Variables:

• X_scaler (StandardizationModel or None) – Standardization model for reference channels (set after fit).

• y_scaler (StandardizationModel or None) – Standardization model for target channel (set after fit).

• filter_coeffs (list[np.ndarray] or None) – Band-pass filter coefficients (SOS format, set after fit).

• is_fitted (bool) – Flag indicating if fit is complete.

Examples

Basic usage:

>>> from gwexpy.timeseries import TimeSeriesMatrix, TimeSeries
>>> from gwexpy.signal.preprocessing import MLPreprocessor
>>>
>>> # Load data
>>> witnesses = TimeSeriesMatrix(...)  # (n_channels, n_samples)
>>> strain = TimeSeries(...)            # (n_samples,)
>>>
>>> # Preprocessing pipeline
>>> preprocessor = MLPreprocessor(
...     sample_rate=4096,
...     freq_low=[55.0],
...     freq_high=[65.0],
...     valid_frac=0.2
... )
>>>
>>> # Split -> fit -> transform
>>> X_train, y_train, X_valid, y_valid = preprocessor.split(witnesses, strain)
>>> preprocessor.fit(X_train, y_train)
>>> X_train_proc, y_train_proc = preprocessor.transform(X_train, y_train)
>>> X_valid_proc, y_valid_proc = preprocessor.transform(X_valid, y_valid)

Notes

Processing order follows the DeepClean v2 implementation: 1. Data splitting (chronological) 2. Learn X standardization parameters (no filtering) 3. Design filter coefficients 4. Filter y -> Learn y standardization parameters

Important notes: - X is not filtered (reference channels are standardized as raw data). - y is filtered before standardization (target channel is band-limited).

fit(X: TimeSeriesMatrix, y: TimeSeries | None = None) → MLPreprocessor

Learn statistics and filter coefficients.

Parameters:

• X (TimeSeriesMatrix) – Reference channels (training data)

• y (TimeSeries or None, optional) – Target channel (training data) If None, skip y standardization.

Returns:

self – Fitted preprocessor

Return type:

MLPreprocessor

split(X: TimeSeriesMatrix, y: TimeSeries) → tuple[TimeSeriesMatrix, TimeSeries, TimeSeriesMatrix, TimeSeries]

Split data into training and validation sets.

Parameters:

• X (TimeSeriesMatrix) – Reference channels (shape: (n_channels, n_samples))

• y (TimeSeries) – Target channel (shape: (n_samples,))

Returns:

• X_train (TimeSeriesMatrix) – Training reference channels

• y_train (TimeSeries) – Training target channel

• X_valid (TimeSeriesMatrix) – Validation reference channels

• y_valid (TimeSeries) – Validation target channel

transform(X: TimeSeriesMatrix, y: TimeSeries | None = None) → tuple[TimeSeriesMatrix, TimeSeries] | TimeSeriesMatrix

Apply filtering and standardization.

Parameters:

• X (TimeSeriesMatrix) – Reference channels

• y (TimeSeries or None, optional) – Target channel If None, return only X.

Returns:

• X_proc (TimeSeriesMatrix) – Processed X (dimensionless_unscaled unit)

• y_proc (TimeSeries (if y is specified)) – Processed y (dimensionless_unscaled unit)