Loader#

class cdiutils.io.Loader(scan=None, sample_name=None, flat_field=None, alien_mask=None)[source]#

Bases: ABC

Abstract base class for beamline-specific data loaders.

Loaders handle experiment-specific data I/O operations including: - HDF5/NeXus/SPEC file parsing - Detector data extraction with ROI support - Motor angle retrieval - Energy and detector calibration parameter loading - Flat-field correction and bad pixel masking

Use the factory method from_setup() to instantiate the appropriate subclass for your beamline, or directly instantiate beamline-specific loaders (ID01Loader, P10Loader, etc.) for advanced configuration.

Supported beamlines:

  • ID01 (ESRF): ID01Loader

  • P10 (PETRA III): P10Loader

  • SIXS (SOLEIL): SIXSLoader

  • NanoMAX (MAX IV): NanoMAXLoader

  • CRISTAL (SOLEIL): CristalLoader

  • ID27 (ESRF): ID27Loader

scan#

Scan number identifier.

Type:

int

sample_name#

Sample identifier for file organisation.

Type:

str

flat_field#

Flat-field correction array for detector non-uniformity.

Type:

np.ndarray

alien_mask#

Mask for defective detector pixels.

Type:

np.ndarray

detector_name#

Detector type (set by subclass).

Type:

str

rocking_angle#

Name of rocking curve motor (beamline-specific).

Type:

str

See also

BcdiPipeline: Uses loaders automatically ID01Loader: ESRF ID01 beamline implementation P10Loader: PETRA III P10 beamline implementation

Examples

Using factory pattern (recommended):

>>> loader = Loader.from_setup(
...     beamline_setup="id01",
...     sample_name="PtNP",
...     scan=42,
...     data_dir="/data/id01/sample"
... )
>>> data, angles = loader.load_data()

Direct instantiation:

>>> from cdiutils.io import ID01Loader
>>> loader = ID01Loader(
...     sample_name="PtNP",
...     scan=42,
...     experiment_file_path="/data/sample.h5"
... )

Methods

load_energy()

Load X-ray beam energy for the scan.

load_det_calib_params()

Load detector calibration parameters from experiment file.

load_detector_shape()

Load detector's native pixel array shape.

get_detector_name()

Get canonical detector identifier for this beamline.
__init__(scan=None, sample_name=None, flat_field=None, alien_mask=None)[source]#

Initialise the base Loader.

Typically called by subclass constructors. Users should prefer from_setup() factory method or direct subclass instantiation.

Parameters:

  • scan (int) – Scan number identifier. Required for data loading.

  • sample_name (str) – Sample identifier used in file paths and logging.

  • flat_field (ndarray | str) – Flat-field correction array or path to .npy/.npz file. Applied as multiplicative correction to detector data. Shape must match detector dimensions. Defaults to None (no correction).

  • alien_mask (ndarray | str) – Bad pixel mask array or path to .npy/.npz file. Pixels with value 1 are masked (invalid), 0 are kept. Shape must match detector. Defaults to None (no masking).

Raises:

ValueError – If flat_field or alien_mask path is invalid or file format is unsupported.

get_alien_mask(roi=None)[source]#
classmethod from_setup(beamline_setup, **metadata)[source]#

Factory method to instantiate beamline-specific loader.

Automatically selects and returns the appropriate Loader subclass based on beamline name. This is the recommended way to create loaders as it handles beamline-specific initialisation automatically.

Parameters:

  • beamline_setup (str) –

    Beamline identifier (case-insensitive). Supported values:

    • "id01" or "id01bliss": ESRF ID01 (BLISS format)

    • "id01spec": ESRF ID01 (legacy SPEC format)

    • "sixs2019" or "sixs2022": SOLEIL SIXS (specify year)

    • "p10" or "p10eh2": PETRA III P10 (specify hutch)

    • "cristal": SOLEIL CRISTAL

    • "nanomax": MAX IV NanoMAX

    • "id27": ESRF ID27

  • **metadata –

    Beamline-specific keyword arguments passed to loader

    constructor. Common parameters include:

    • scan (int): Scan number

    • sample_name (str): Sample identifier

    • experiment_file_path (str): Path to experiment HDF5/SPEC

    • data_dir (str): Root data directory

    • flat_field (np.ndarray | str): flat-field

    correction

    • alien_mask (np.ndarray | str): bad pixel mask

Returns:

Beamline-specific Loader subclass instance.

Raises:

  • ValueError – If beamline_setup is not recognised.

  • NotImplementedError – If beamline version (e.g., SIXS year) is not specified or unsupported.

Return type:

Loader

Examples

Basic usage:

>>> loader = Loader.from_setup(
...     beamline_setup="id01",
...     scan=42,
...     sample_name="PtNP",
...     experiment_file_path="/data/id01/beamtile_id01.h5"
... )

With version specification:

>>> loader = Loader.from_setup(
...     beamline_setup="sixs2022",
...     scan=100,
...     sample_name="SrTiO3"
... )

With flat-field and mask:

>>> loader = Loader.from_setup(
...     beamline_setup="p10",
...     scan=15,
...     flat_field="/path/to/flatfield.npy",
...     alien_mask="/path/to/badpixels.npy"
... )
static bin_flat_mask(data, roi=None, flat_field=None, alien_mask=None, rocking_angle_binning=None, binning_method='sum')[source]#

Apply preprocessing: binning, flat-field, and masking.

Combines three common preprocessing steps in correct order:

  1. Bin along rocking curve (if requested)

  2. Apply flat-field correction (if provided)

  3. Apply alien mask (if provided)

Parameters:

  • data (ndarray) – 3D detector data with shape (n_frames, n_y, n_x).

  • roi (list) – Region of interest as tuple of slices or integers. If None, uses full array. See _check_roi() for format details.

  • flat_field (ndarray) – 2D array with detector efficiency correction. Shape must match data.shape[1:]. If None, no correction applied.

  • alien_mask (ndarray) – Binary mask of bad pixels (1 = bad, 0 = good). Shape must match data.shape (3D) or data.shape[1:] (2D). If None, no masking applied.

  • rocking_angle_binning (int) – Binning factor along rocking curve axis (frames). If None or 1, no binning performed.

  • binning_method (str) –

    Binning operation. Options:

    • "sum": Sum frames (default, preserves total counts)

    • "mean": Average frames (reduces noise)

    • "max": Maximum projection (peak intensity)

Returns:

Preprocessed 3D array with same dtype as input. Shape is (n_frames//binning, n_y, n_x) if binned.

Return type:

ndarray

Examples

ROI + flat-field + mask:

>>> roi = (slice(None), slice(100, 400), slice(150, 450))
>>> processed = Loader.bin_flat_mask(
...     data=raw_data,
...     roi=roi,
...     flat_field=flat,
...     alien_mask=mask
... )

Binning only:

>>> binned = Loader.bin_flat_mask(
...     data=raw_data,
...     rocking_angle_binning=2,
...     binning_method="sum"
... )
static bin_rocking_angle_values(values, binning_factor=None)[source]#

Bin rocking angle values to match binned detector frames.

Averages angle values when frames are binned together. Used to maintain synchronisation between data and motor positions.

Parameters:

  • values (list | ndarray) – Rocking angle values for each frame (e.g., delta, omega motor positions). Length must match original number of frames.

  • binning_factor (int) – Number of consecutive frames to average. If None or 1, returns input unchanged.

Returns:

Binned angle values with length len(values)//binning_factor. Uses mean binning to get average angle per binned frame.

Return type:

ndarray

abstract load_energy()[source]#

Load X-ray beam energy for the scan.

Must be implemented by beamline-specific subclass.

Returns:

Beam energy in keV.

abstract load_det_calib_params()[source]#

Load detector calibration parameters from experiment file.

Must be implemented by beamline-specific subclass. Typically reads values stored during detector alignment procedure.

Returns:

Calibration parameters with keys:

  • "direct_beam": (y, x) pixel coordinates of direct beam position

  • "detector_distance": sample-to-detector distance in metres

  • "outofplane_angle": detector rotation delta or gamma in degrees

  • "inplane_angle": detector rotation nu in degrees

Return type:

dict

See also

Detector Geometry Calibration for calibration procedures and parameter definitions.

abstract load_detector_shape()[source]#

Load detector’s native pixel array shape.

Must be implemented by beamline-specific subclass if detector shape cannot be determined from data files.

Returns:

Detector shape as (n_rows, n_columns) tuple, or None if shape is determined from data.

get_detector_name()[source]#

Get canonical detector identifier for this beamline.

Returns the first name from authorised_detector_names, which is the standard identifier for detector geometry calculations.

Returns:

Detector name string (e.g., "Eiger2M", "Maxipix", "Lambda750k").

Return type:

str

static get_rocking_angle(angles)[source]#

Identify which motor was scanned during rocking curve.

Determines whether out-of-plane or in-plane angle was varied based on which array has more than one unique value. Used to automatically detect scan geometry.

Parameters:

angles (dict) –

Dictionary with keys:

  • "sample_outofplane_angle": omega or eta values (scalar or array)

  • "sample_inplane_angle": chi or phi values (scalar or array)

Returns:

Name of scanned angle key, or None if neither angle was scanned (single-frame measurement).

Return type:

str | None

Examples

Out-of-plane scan (typical):

>>> angles = {
...     "sample_outofplane_angle": np.linspace(30.0, 30.5, 51),
...     "sample_inplane_angle": 0.0
... }
>>> Loader.get_rocking_angle(angles)
'sample_outofplane_angle'

In-plane scan (grazing incidence):

>>> angles = {
...     "sample_outofplane_angle": 2.0,
...     "sample_inplane_angle": np.linspace(-10, 10, 41)
... }
>>> Loader.get_rocking_angle(angles)
'sample_inplane_angle'
static format_scanned_counters(*counters, scan_axis_roi=None, rocking_angle_binning=None)[source]#

Preprocess motor positions to match ROI and binning of data.

Applies same binning and ROI selection to motor counter arrays as applied to detector data, maintaining synchronisation between intensity and position information.

Parameters:

  • *counters (float | ndarray | list) –

    One or more motor position values. Each can be:

    • Scalar: Fixed motor position (e.g., 30.0 degrees)

    • Array: Scanned motor positions (one per frame)

  • scan_axis_roi (tuple[slice]) – ROI slice along rocking curve axis (first dimension). Applied after binning. Typically (slice(start, stop),).

  • rocking_angle_binning (int) – Binning factor for scanned arrays. Scalar values are unaffected.

Returns:

Formatted counter(s) with same type as input. If multiple counters provided, returns tuple in same order. If single counter, returns that value directly.

Examples

Single scanned angle with binning:

>>> omega = np.linspace(30.0, 30.5, 100)
>>> formatted = Loader.format_scanned_counters(
...     omega,
...     rocking_angle_binning=2
... )
>>> # Returns array of length 50

Multiple counters with ROI:

>>> omega = np.linspace(30.0, 30.5, 100)
>>> energy = 8.5  # fixed
>>> omega_fmt, energy_fmt = Loader.format_scanned_counters(
...     omega, energy,
...     scan_axis_roi=(slice(10, 90),)
... )
>>> # omega_fmt has 80 values, energy_fmt is 8.5
classmethod get_mask(detector_name=None, channel=None, roi=None)[source]#

Generate detector-specific bad pixel mask.

Returns hardcoded masks for common BCDI detectors, marking chip gaps and known bad pixel regions. Masks are detector-specific due to different chip layouts and geometries.

Parameters:

  • detector_name (str) –

    Detector identifier (case-insensitive). Supported detectors:

    • Maxipix: "maxipix", "mpxgaas", "mpx4inr"

    • Eiger2M: "Eiger2M", "eiger2m"

    • Eiger4M: "Eiger4M", "eiger4m", "e4m"

    • Eiger9M: "eiger9m", "e9m"

    • Eiger500k: "eiger500k", "e2500"

    • Merlin: "merlin"

    If None and called as instance method, uses self.detector_name.

  • channel (int) – If provided, extends 2D mask to 3D by repeating along first axis (for 3D data). Specifies number of frames.

  • roi (tuple[slice]) – ROI applied after mask generation. See _check_roi() for format. Typically (slice(y1,y2), slice(x1,x2)) for 2D.

Returns:

  • 2D: detector_shape if no ROI

  • 2D: cropped to ROI if provided

  • 3D: (channel, n_y, n_x) if channel specified

Return type:

Binary mask array (1 = bad pixel, 0 = good pixel). Shape is

Raises:

ValueError – If detector_name is not recognized or if called as class method without providing detector_name.

Examples

Instance method (uses loader’s detector):

>>> loader = ID01Loader(scan=42, ...)
>>> mask = loader.get_mask(channel=100)
>>> # Returns (100, 2164, 1030) Eiger2M mask

Class method with explicit detector:

>>> mask = Loader.get_mask(detector_name="Maxipix")
>>> # Returns (516, 516) Maxipix mask

With ROI:

>>> roi = (slice(100, 400), slice(200, 800))
>>> mask = Loader.get_mask(
...     detector_name="Eiger2M",
...     roi=roi
... )
>>> # Returns (300, 600) cropped mask

Notes

Eiger masks include:

  • Chip gaps (horizontal and vertical)

  • Module boundaries

  • Known bad pixel clusters

Maxipix masks include central cross gaps (256±3 pixels).

static plot_detector_data(data, title=None, return_fig=False, equal_limits=False, **plot_params)[source]#

Quick visualisation of 2D or 3D detector data.

Creates diagnostic plots showing orthogonal slices (for 3D data) or single 2D image. Uses log-scale colouring by default for dynamic range typical of BCDI diffraction patterns.

Parameters:

  • data (ndarray) – Detector data array. If 3D, shape is (n_frames, n_y, n_x). If 2D, shape is (n_y, n_x).

  • title (str) – Plot title displayed above figure. If None, no title shown.

  • return_fig (bool) – If True, returns Figure object for further customisation. If False (default), displays figure interactively.

  • equal_limits (bool) – If True, uses same axis limits for all subplots (helpful for comparing slice scales). If False, each plot uses its own optimal limits.

  • **plot_params –

    Additional arguments passed to matplotlib.pyplot.imshow(). Defaults are:

    • norm="log": Logarithmic colour scale

    • origin="upper": [0,0] at top-left

    • cmap="turbo": Rainbow-like colourmap

Returns:

If return_fig=True, returns matplotlib Figure object. Otherwise, displays interactively and returns None.

Return type:

Figure

Examples

Quick 3D data check:

>>> data = loader.load_data(scan=42)
>>> Loader.plot_detector_data(data, title="Scan 42")

Custom colouring:

>>> Loader.plot_detector_data(
...     data,
...     cmap="viridis",
...     norm="linear",
...     vmin=0,
...     vmax=1e5
... )

Save figure for publication:

>>> fig = Loader.plot_detector_data(data, return_fig=True)
>>> fig.savefig("detector_scan42.png", dpi=300)

Notes

For 3D data, creates 2×3 subplot grid:

  • Top row: Central slices along each axis

  • Bottom row: Sum projections along each axis

This quickly reveals Bragg peak position and rocking curve quality.

Examples#

Use beamline-specific loaders:

from cdiutils.io import ID01Loader

# Create loader for ID01 beamline
loader = ID01Loader(
    sample_name="S123",
    scan=42,
    data_dir="/path/to/data"
)

# Loader is used automatically by BcdiPipeline
from cdiutils.pipeline import BcdiPipeline
pipeline = BcdiPipeline(param_file_path="config.yml")
pipeline.preprocess()  # loads data internally

See Also#

ID01Loader : ESRF ID01 beamline P10Loader : PETRA III P10 beamline SIXSLoader : SOLEIL SIXS beamline NanoMaxLoader : MAX IV NanoMAX beamline