dataset.core.dtm

dtm

MarsHiRISE DTM (Digital Terrain Model) dataset.

MarsHiRISEDTM

MarsHiRISEDTM(root: Path = '/scratch/mars_hirise_dtm', *, split: str = 'train', target: str | None = None, include_ortho: bool = True, ortho_type: OrthoType | list[OrthoType] = 'RED', ortho_scale: str | None = None, transforms: Callable[[Sample], Sample] | None = None, download: bool = False, bbox: tuple[float, float, float, float] | None = None, checksum: bool = False, reuse_cache: bool = True, normalize_elevation: bool = False, elevation_stats_path: str | None = None, return_meta: bool = False)

Bases: MarsHiRISEBase

Mars HiRISE Digital Terrain Model (DTM) dataset.

HiRISE DTMs <https://www.uahirise.org/dtm/about.php>__ are derived from stereo pairs of HiRISE observations. Each DTM set consists of:

• A DTM (.IMG) — 32-bit floating-point raster where each pixel value is an areoid elevation (metres) or planetary radius (metres). 1 DN = 1 m.
• Orthoimages (.JP2) — the left and right stereo observations orthorectified onto the DTM, in RED (1-band) and/or IRB (3-band: near-IR, RED, blue-green) colour content, at one or more resolutions (A = 0.25 m, B = 0.5 m, C = 1.0 m, D = 2.0 m).

This class indexes the PDS DTM cumulative index (DTMCUMINDEX.TAB) and groups all products by stereo pair.

Directory layout ~~~~~~~~~~~~~~~~ ::

<root>/
    DTMCUMINDEX.LBL
    DTMCUMINDEX.TAB
    images/
        DTEEC_011265_1560_011331_1560_U01.IMG
        ESP_011265_1560_RED_A_01_ORTHO.JP2
        ESP_011265_1560_RED_A_01_ORTHO.LBL
        ...

Sample dict ~~~~~~~~~~~ __getitem__ returns a dict containing:

• "elevation" — (1, H, W) float32 in metres. Nodata = NaN.
• "left_red" — (1, H, W) float32 I/F [0, 1] (if requested)
• "right_red" — same, for the right observation
• "left_irb" — (3, H, W) float32 I/F [0, 1] (if requested)
• "right_irb" — same, for the right observation
• "bounds" tensor, "crs" WKT string

Sampler units ~~~~~~~~~~~~~ self.crs is geographic; self.res is in degrees/pixel (~1.69e-5 deg/px ≈ 1 m at the equator by default).

Dataset homepage

https://www.uahirise.org/dtm/about.php

.. versionadded:: 0.8

Initialise the dataset.

Parameters:

Name	Type	Description	Default
root	Path	Root directory containing PDS index files and data.	'/scratch/mars_hirise_dtm'
split	str	Dataset split — informational.	'train'
target	str | None	Case-insensitive substring filter on character columns of the cumulative index (e.g. "Eberswalde").	None
include_ortho	bool	If True, load orthoimage patches alongside elevation.	True
ortho_type	OrthoType | list[OrthoType]	Which orthoimage colour content to load: "RED" (1-band) and/or "IRB" (3-band).	'RED'
ortho_scale	str | None	Preferred scale letter ("A"–"D"). If None, the finest resolution available is used.	None
transforms	Callable[[Sample], Sample] | None	Optional transform applied to each sample.	None
download	bool	Fetch from PDS if absent.	False
bbox	tuple[float, float, float, float] | None	(lon_min, lat_min, lon_max, lat_max) in degrees.	None
checksum	bool	Verify checksums (not yet implemented).	False
reuse_cache	bool	Reuse cached spatial index if present.	True
normalize_elevation	bool	Z-score normalise elevation using stats.	False
elevation_stats_path	str | None	JSON file with "mean" and "std".	None

Raises:

Type	Description
ValueError	If ortho_type contains an unrecognised value.
DatasetNotFoundError	If index is absent and download=False.

Source code in src/dataset/core/dtm.py

def __init__(
        self,
        root: Path = "/scratch/mars_hirise_dtm",
        *,
        split: str = "train",
        target: str | None = None,
        include_ortho: bool = True,
        ortho_type: OrthoType | list[OrthoType] = "RED",
        ortho_scale: str | None = None,
        transforms: Callable[[Sample], Sample] | None = None,
        download: bool = False,
        bbox: tuple[float, float, float, float] | None = None,
        checksum: bool = False,
        reuse_cache: bool = True,
        normalize_elevation: bool = False,
        elevation_stats_path: str | None = None,
        return_meta: bool = False,
) -> None:
    """Initialise the dataset.

    Args:
        root: Root directory containing PDS index files and data.
        split: Dataset split — informational.
        target: Case-insensitive substring filter on character columns
            of the cumulative index (e.g. ``"Eberswalde"``).
        include_ortho: If ``True``, load orthoimage patches alongside
            elevation.
        ortho_type: Which orthoimage colour content to load:
            ``"RED"`` (1-band) and/or ``"IRB"`` (3-band).
        ortho_scale: Preferred scale letter (``"A"``–``"D"``).  If
            ``None``, the finest resolution available is used.
        transforms: Optional transform applied to each sample.
        download: Fetch from PDS if absent.
        bbox: ``(lon_min, lat_min, lon_max, lat_max)`` in degrees.
        checksum: Verify checksums (not yet implemented).
        reuse_cache: Reuse cached spatial index if present.
        normalize_elevation: Z-score normalise elevation using stats.
        elevation_stats_path: JSON file with ``"mean"`` and ``"std"``.

    Raises:
        ValueError: If ortho_type contains an unrecognised value.
        DatasetNotFoundError: If index is absent and ``download=False``.
    """
    # ── Ortho configuration (before super().__init__ calls _verify) ──
    self.return_meta = return_meta
    self.include_ortho = include_ortho
    if isinstance(ortho_type, str):
        ortho_type = [ortho_type]
    invalid = set(ortho_type) - {"RED", "IRB"}
    if invalid:
        raise ValueError(
            f"Invalid ortho_type(s): {invalid}. Valid: 'RED', 'IRB'"
        )
    self.ortho_types: list[str] = list(ortho_type)
    self.ortho_scale = ortho_scale.upper() if ortho_scale else None

    # ── Elevation normalisation ──
    self.normalize_elevation = normalize_elevation
    self._elev_mean: float | None = None
    self._elev_std: float | None = None

    if self.normalize_elevation:
        path = pathlib.Path(elevation_stats_path or "")
        if not elevation_stats_path or not path.exists():
            raise ValueError(
                f"elevation_stats_path must point to a valid stats JSON "
                f"when normalize_elevation=True (got: {path.absolute()})"
            )
        with open(path) as f:
            stats = json.load(f)
        self._elev_mean = float(stats["mean"])
        self._elev_std = float(stats["std"])

    # DTM typical resolution ≈ 1 m/pix → ~1/(59 251) deg/pix at equator.
    super().__init__(
        root,
        split=split,
        target=target,
        transforms=transforms,
        download=download,
        bbox=bbox,
        checksum=checksum,
        reuse_cache=reuse_cache,
        res=1.0 / 59_251.13,
    )

plot

plot(sample: Sample, show_titles: bool = True, suptitle: str | None = None, **kwargs) -> Figure

Visualise a sample: elevation + ortho panels.

Source code in src/dataset/core/dtm.py

def plot(
        self,
        sample: Sample,
        show_titles: bool = True,
        suptitle: str | None = None,
        **kwargs,
) -> Figure:
    """Visualise a sample: elevation + ortho panels."""
    has_elev = "elevation" in sample
    ortho_keys = [
        k for k in ("left_red", "right_red", "left_irb", "right_irb")
        if k in sample
    ]
    n_panels = max(int(has_elev) + len(ortho_keys), 1)

    fig, axes = plt.subplots(1, n_panels, figsize=(6 * n_panels, 6))
    if n_panels == 1:
        axes = [axes]

    panel = 0

    # ── Elevation ─────────────────────────────────────────────────
    if has_elev:
        ax = axes[panel]
        elev = sample["elevation"]
        if elev.ndim == 4:
            elev = elev[0]
        elev_np = elev[0].numpy().copy()
        valid = np.isfinite(elev_np)
        if valid.any():
            vmin, vmax = np.nanpercentile(elev_np[valid], [2, 98])
            im = ax.imshow(
                np.where(valid, elev_np, np.nan),
                cmap="terrain", vmin=vmin, vmax=vmax,
                interpolation="nearest",
            )
            fig.colorbar(im, ax=ax, fraction=0.046, pad=0.04,
                         label="Elevation (m)")
        else:
            ax.imshow(elev_np, cmap="terrain", interpolation="nearest")
        ax.axis("off")
        if show_titles:
            ax.set_title("Elevation (DTM)")
        panel += 1

    # ── Ortho panels ─────────────────────────────────────────────
    for key in ortho_keys:
        ax = axes[panel]
        img = sample[key]
        if img.ndim == 4:
            img = img[0]

        if img.shape[0] >= 3:
            img_np = img[:3].permute(1, 2, 0).numpy().copy()
            cmap = None
        else:
            img_np = img[0].numpy().copy()
            cmap = "grey"

        ax.imshow(
            self._percentile_stretch(img_np),
            cmap=cmap, interpolation="nearest",
        )
        ax.axis("off")
        if show_titles:
            ax.set_title(f"Ortho — {key.replace('_', ' ').title()}")
        panel += 1

    if suptitle is not None:
        fig.suptitle(suptitle)
    fig.tight_layout()
    return fig

plot3d

plot3d(sample: Sample, show_titles: bool = True, suptitle: str | None = None, **kwargs) -> Figure

Visualise a sample: 3D elevation panel.

Source code in src/dataset/core/dtm.py

def plot3d(
        self,
        sample: Sample,
        show_titles: bool = True,
        suptitle: str | None = None,
        **kwargs,
) -> Figure:
    """Visualise a sample: 3D elevation panel."""
    has_elev = "elevation" in sample

    fig, ax = plt.subplots(1, 1, figsize=(6, 6), subplot_kw={"projection": "3d"})
    ax: Axes3D

    if has_elev:
        elev = sample["elevation"]
        if elev.ndim == 4:
            elev = elev[0]
        elev_np = elev[0].numpy().copy()
        valid = np.isfinite(elev_np)

        x, y = np.meshgrid(range(elev_np.shape[1]), range(elev_np.shape[0]))

        if valid.any():
            stride = max(1, min(elev_np.shape[0], elev_np.shape[1]) // 100)

            im = ax.plot_surface(
                np.where(valid, x, np.nan),
                np.where(valid, y, np.nan),
                np.where(valid, elev_np, np.nan),
                cmap="terrain",
                rstride=stride,
                cstride=stride,
                linewidth=0,
                antialiased=False,
            )
            fig.colorbar(im, ax=ax, fraction=0.046, pad=0.04,
                         label="Elevation (m)")

        ax.axis("off")
        if show_titles:
            ax.set_title("3D Elevation (DTM)")

    if suptitle is not None:
        fig.suptitle(suptitle)
    fig.tight_layout()
    return fig