dataset.core.rdr

rdr

MarsHiRISE dataset.

MarsHiRISE

MarsHiRISE(root: Path = '/scratch/mars_hirise', split: str = 'train', target: str | None = None, channels: list[Channel] | None = None, transforms: Callable[[Sample], Sample] | None = None, download: bool = False, bbox: tuple[float, float, float, float] | None = None, checksum: bool = False, reuse_cache: bool = True, normalize: bool = False, normalization_path: str | None = None, return_meta: bool = False)

Bases: MarsHiRISEBase

Mars HiRISE Reduced Data Records (RDR) dataset.

HiRISE <https://pds-imaging.jpl.nasa.gov/volumes/mro.html>__ is the High Resolution Imaging Science Experiment aboard the Mars Reconnaissance Orbiter (MRO). This dataset wraps the RDR products — radiometrically- corrected images resampled to a standard map projection — hosted on the NASA PDS Imaging Node.

Directory layout ~~~~~~~~~~~~~~~~ Files are stored preserving the PDS directory hierarchy under root::

<root>/
    RDRCUMINDEX.LBL
    RDRCUMINDEX.TAB
    images/
        PSP_001430_1780_COLOR.JP2
        PSP_001430_1780_COLOR.LBL
        PSP_001430_1780_RED.JP2
        PSP_001430_1780_RED.LBL

Each HiRISE observation produces two JP2 product files:

_COLOR.JP2 Three-band mosaic — in-file band order: NEAR-INFRARED (band 1), RED (band 2), BLUE-GREEN (band 3).

_RED.JP2 Single-band full-strip RED image. Uses more TDI lines; highest- quality RED source when colour context is not needed.

Channel selection ~~~~~~~~~~~~~~~~~ * Only "RED" requested → _RED.JP2 (higher fidelity). * Any "NEAR-INFRARED" or "BLUE-GREEN" → _COLOR.JP2.

Radiometric calibration ~~~~~~~~~~~~~~~~~~~~~~~ I/F = DN * SCALING_FACTOR + OFFSET, clipped to [0, 1].

Sampler units ~~~~~~~~~~~~~ self.crs is a geographic CRS; self.res is in degrees/pixel. Pass size to :class:~torchgeo.samplers.RandomGeoSampler in degrees (e.g. size=0.01 ≈ 1 185 px ≈ 593 m at the equator).

Dataset homepage

https://hirise-pds.lpl.arizona.edu/PDS/AAREADME.TXT

.. versionadded:: 0.7

Initialise the dataset.

Parameters:

Name	Type	Description	Default
root	Path	Root directory. Must contain the PDS index files and the observation JP2/LBL files, or download=True must be set.	'/scratch/mars_hirise'
split	str	Dataset split — informational.	'train'
target	str | None	Optional case-insensitive substring filter on all character columns of the cumulative index (e.g. "Olympus").	None
channels	list[Channel] | None	Which channels to include. Valid values: "NEAR-INFRARED", "RED", "BLUE-GREEN". Output tensor band order always follows :attr:all_channels. Defaults to all three.	None
transforms	Callable[[Sample], Sample] | None	Optional callable applied to each :class:Sample.	None
download	bool	Fetch index and images from the PDS server if absent.	False
bbox	tuple[float, float, float, float] | None	(lon_min, lat_min, lon_max, lat_max) bounding box filter in degrees ([-180, 180] longitude convention).	None
checksum	bool	Verify checksums after download (not yet implemented).	False
reuse_cache	bool	Reuse cached spatial index if present.	True
normalize	bool	If True, apply per-channel z-score normalisation using statistics from normalization_path.	False
normalization_path	str | None	Path to a JSON file with "channels", "mean", and "std" keys.	None

Raises:

Type	Description
ValueError	If channels contains an unrecognised name.
DatasetNotFoundError	If index files are absent and download=False.

Source code in src/dataset/core/rdr.py

def __init__(
        self,
        root: Path = "/scratch/mars_hirise",
        split: str = "train",
        target: str | None = None,
        channels: list[Channel] | None = None,
        transforms: Callable[[Sample], Sample] | None = None,
        download: bool = False,
        bbox: tuple[float, float, float, float] | None = None,
        checksum: bool = False,
        reuse_cache: bool = True,
        normalize: bool = False,
        normalization_path: str | None = None,
        return_meta: bool = False,
) -> None:
    """Initialise the dataset.

    Args:
        root: Root directory.  Must contain the PDS index files and the
            observation JP2/LBL files, or ``download=True`` must be set.
        split: Dataset split — informational.
        target: Optional case-insensitive substring filter on all character
            columns of the cumulative index (e.g. ``"Olympus"``).
        channels: Which channels to include.  Valid values:
            ``"NEAR-INFRARED"``, ``"RED"``, ``"BLUE-GREEN"``.  Output
            tensor band order always follows :attr:`all_channels`.
            Defaults to all three.
        transforms: Optional callable applied to each :class:`Sample`.
        download: Fetch index and images from the PDS server if absent.
        bbox: ``(lon_min, lat_min, lon_max, lat_max)`` bounding box filter
            in degrees ([-180, 180] longitude convention).
        checksum: Verify checksums after download (not yet implemented).
        reuse_cache: Reuse cached spatial index if present.
        normalize: If ``True``, apply per-channel z-score normalisation
            using statistics from *normalization_path*.
        normalization_path: Path to a JSON file with ``"channels"``,
            ``"mean"``, and ``"std"`` keys.

    Raises:
        ValueError: If *channels* contains an unrecognised name.
        DatasetNotFoundError: If index files are absent and
            ``download=False``.
    """
    self.return_meta = return_meta

    # ── Channel validation (before super().__init__ triggers _verify) ──
    if channels is None:
        self.channels: list[str] = list(ALL_CHANNELS)
    else:
        invalid = set(channels) - set(ALL_CHANNELS)
        if invalid:
            raise ValueError(
                f"Invalid channel(s): {invalid}. "
                f"Valid choices are: {ALL_CHANNELS}"
            )
        self.channels = [c for c in ALL_CHANNELS if c in set(channels)]

    # ── Normalisation ──
    self.normalize = normalize
    self.normalization_path = normalization_path
    self._normalizer = None

    if self.normalize:
        path = pathlib.Path(self.normalization_path or "")
        if not self.normalization_path or not path.exists():
            raise ValueError(
                f"normalization_path must be a valid path to "
                f"dataset_stats.json when normalize=True, currently "
                f"pointing to {path.absolute()}"
            )
        with open(self.normalization_path) as f:
            stats = json.load(f)

        stat_channels = stats["channels"]
        mean_dict = dict(zip(stat_channels, stats["mean"]))
        std_dict = dict(zip(stat_channels, stats["std"]))
        try:
            mean = [mean_dict[ch] for ch in self.channels]
            std = [std_dict[ch] for ch in self.channels]
        except KeyError as e:
            raise ValueError(
                f"Channel {e} not found in normalization stats."
            )

        self._normalizer = Normalize(mean=mean, std=std)

    # Native HiRISE RDR resolution: 1 / 118 502.26 pix/deg ≈ 8.44e-6 deg/pix
    super().__init__(
        root,
        split=split,
        target=target,
        transforms=transforms,
        download=download,
        bbox=bbox,
        checksum=checksum,
        reuse_cache=reuse_cache,
        res=1.0 / 118_502.26464032,
    )

plot

plot(sample: Sample, show_titles: bool = True, suptitle: str | None = None, eps: float = 1e-08, **kwargs) -> Figure

Visualise an RDR image sample with percentile stretch.

Source code in src/dataset/core/rdr.py

def plot(
        self,
        sample: Sample,
        show_titles: bool = True,
        suptitle: str | None = None,
        eps: float = 1e-8,
        **kwargs,
) -> Figure:
    """Visualise an RDR image sample with percentile stretch."""
    eps = abs(eps)

    image: torch.Tensor = sample["image"]
    if image.ndim == 4:
        image = image[0]

    ch = self.channels
    if set(ch) >= {"NEAR-INFRARED", "RED", "BLUE-GREEN"}:
        idx = [
            ch.index("NEAR-INFRARED"),
            ch.index("RED"),
            ch.index("BLUE-GREEN"),
        ]
        rgb = image[idx]
        # If only one channel has non-zero data (e.g. COLOR file
        # absent), fall back to grayscale.
        nonzero = [(rgb[i] > 0).any().item() for i in range(3)]
        if sum(nonzero) == 1:
            active_i = nonzero.index(True)
            active_name = ["NEAR-INFRARED", "RED", "BLUE-GREEN"][active_i]
            img_np = rgb[active_i].numpy()
            cmap = "grey"
            title = (
                f"MarsHiRISE — {active_name} (COLOR file unavailable)"
            )
        else:
            img_np = rgb.permute(1, 2, 0).numpy()
            cmap = None
            title = "MarsHiRISE — false colour (NIR→R, RED→G, BG→B)"
    else:
        img_np = image[0].numpy()
        cmap = "grey"
        title = f"MarsHiRISE — {ch[0]}"

    # Percentile stretch — only over non-zero (data) pixels
    img_out = img_np.copy()
    if img_out.ndim == 3:
        for c in range(img_out.shape[2]):
            band = img_out[..., c]
            mask = np.logical_or(band > eps, band < -eps)
            data_pixels = band[mask]
            if len(data_pixels) > 0:
                p2, p98 = np.percentile(data_pixels, [2, 98])
                if p98 > p2:
                    img_out[..., c] = np.clip(
                        (band - p2) / (p98 - p2), 0, 1
                    )
                    img_out[..., c][np.logical_not(mask)] = 0
    else:
        mask = np.logical_or(img_out > eps, img_out < -eps)
        data_pixels = img_out[mask]
        if len(data_pixels) > 0:
            p2, p98 = np.percentile(data_pixels, [2, 98])
            if p98 > p2:
                img_out = np.clip(
                    (img_out - p2) / (p98 - p2), 0, 1
                )
                img_out[np.logical_not(mask)] = 0

    fig, ax = plt.subplots(figsize=(8, 8))
    ax.imshow(img_out, cmap=cmap, interpolation="nearest")
    ax.axis("off")
    if show_titles:
        ax.set_title(title)
    if suptitle is not None:
        fig.suptitle(suptitle)
    fig.tight_layout()
    return fig