depth_fm.data.scalers¶
scalers ¶
DTM Normalization for Training AND Inference on Unseen Data¶
CRITICAL CONSTRAINT: At inference time, we have ONLY an ortho image. The network predicts a normalised DTM patch. We must be able to convert that prediction back to physical metres WITHOUT any per-patch metadata (no stored scales, no stored trend parameters).
This rules out adaptive_stored normalization — it produces outputs
where 0.8 could mean 0.5m or 15m depending on the unknown local scale.
The correct approach for this use case:
TRAINING TARGET = plane-detrended residual / global_fixed_scale
- The plane is removed so the network only predicts what the ortho
image actually encodes (local relief from shadow cues).
- The global fixed scale gives every prediction a CONSISTENT physical
meaning: network output * scale = metres of relief.
- At inference, the inverse is just: multiply by scale. No metadata.
RECONSTRUCTION = stitch trend planes from overlap consistency, then add the denormalised residuals.
The slope problem (dynamic range wasted on ramps) is solved by the DETRENDING, not by the scaling. After removing the plane, the residual distribution is tight enough that a single global scale works well.
The VAE precision concern for low-relief patches
After detrending, a 0.6m-relief patch maps to ~[-0.05, 0.05]. This is small but NOT zero — the VAE operates at float32 with 4 latent channels at 1/8 resolution. The effective precision is ~10-12 bits per channel, giving ~4000 distinguishable levels in [0, 0.05]. The signal IS there.
If you find empirically that low-relief patches are blurry, use the log-compressed variant (Strategy B) which expands small values by ~3x while remaining globally invertible with no stored metadata.
Three strategies ranked by recommendation:¶
GLOBAL FIXED (simplest, try first)
forward: normed = detrended_residual / global_scale inverse: residual = prediction * global_scale pros: Linear, trivial inverse, reconstruction is shift-only cons: Low-relief patches use narrow range
GLOBAL LOG (best compromise, recommended)
forward: normed = sign(r) * log1p(|r| / ref) / log1p(1) inverse: residual = sign(n) * ref * (expm1(|n| * ln2)) pros: Expands low-relief 3x, compresses high-relief, NO stored metadata needed, fully invertible cons: Non-linear — loss gradients weighted differently for large vs small features
ADAPTIVE + SCALE HEAD (most complex, best signal)
forward: normed = residual / local_p98 (full [-1,1] range) inverse: residual = prediction * predicted_scale The network has TWO outputs: 1. Normalised residual shape (3ch, H, W) 2. Predicted scale factor (scalar regression head) pros: Best VAE utilisation, network learns relief magnitude cons: Requires architecture modification (extra head), scale prediction errors propagate to reconstruction
For your DepthFM pipeline, I recommend starting with Strategy A (simplest to implement, changes only normalization code) and moving to Strategy B if low-relief patches are empirically blurry.
What about the trend plane at inference?¶
The network predicts ONLY the residual (what's visible in shadows). The trend plane (regional slope) cannot be predicted from nadir ortho. But for reconstruction, you NEED the plane to get absolute elevation.
Option 1 is the cleanest — it works with ortho-only input and the overlap constraints are well-conditioned since we're only solving for scalar offsets (1 unknown per patch, not 2 or 3).
TrainingNormResult
dataclass
¶
TrainingNormResult(normed_residual: Tensor, trend_params: Tensor, residual_scale: float, valid_mask: Tensor, raw_residual_rms: float, raw_residual_p98: float)
Output of normalization during training (GT elevation available).
normed_residual
instance-attribute
¶
(1, H, W) normalised residual in ~[-1, 1]. Network target.
trend_params
instance-attribute
¶
(3,) plane params [a, b, c] in normalised coords. Stored in manifest for analysis but NOT needed at inference time.
residual_scale
instance-attribute
¶
The global scale factor used. Same for all patches.
raw_residual_rms
instance-attribute
¶
RMS of physical residual (metres). For manifest/filtering.
raw_residual_p98
instance-attribute
¶
p98 of |residual| (metres). For manifest/filtering.
InferenceResult
dataclass
¶
Output of denormalization during inference (no GT available).
GlobalFixedNormalizer ¶
Plane detrend → divide by fixed global scale.
The simplest approach. Every network output has a fixed physical
meaning: prediction * global_scale = metres.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
global_scale
|
float
|
p98 of |detrended residual| across the dataset.
Compute once with |
required |
Source code in src/depth_fm/data/scalers.py
normalize_for_training ¶
Training-time normalization (GT elevation available).
Source code in src/depth_fm/data/scalers.py
denormalize_prediction ¶
Inference-time inverse. No metadata needed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prediction
|
Tensor
|
(1, H, W) or (B, 1, H, W) network output in [-1, 1]. |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
Physical residual in metres (same shape). |
Source code in src/depth_fm/data/scalers.py
GlobalLogNormalizer ¶
Plane detrend → signed log compression with global reference.
Expands low-relief details by ~3× compared to linear scaling while compressing high-relief patches. Fully invertible with NO stored metadata.
Forward
normed = sign(r) * log1p(|r| / ref_scale) / log1p(1)
Inverse
residual = sign(n) * ref_scale * expm1(|n| * ln(2))
At ref_scale (the reference magnitude): - Input ±ref_scale → output ±1.0 - Input ±ref_scale/10 → output ±0.14 (vs ±0.10 linear: 1.4× expansion) - Input ±ref_scale/100 → output ±0.014 (vs ±0.01 linear: 1.4× expansion)
The expansion ratio increases for smaller values. A 0.6m signal with ref_scale=12 gets 1.4× more range than linear.
Adjustment: to get MORE expansion of small values, use a smaller ref_scale. But this clips more high-relief patches. The p90 of residual distribution is often a good choice (clips the top 10% while expanding everything else).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ref_scale
|
float
|
Reference scale in metres. Values at ±ref_scale map to ±1.0 in the normalised space. Use the p90 or p95 of your detrended residual distribution. |
required |
clip
|
bool
|
Whether or not to clip the range to be withing [-1, 1] explicitly |
False
|
Source code in src/depth_fm/data/scalers.py
normalize_for_training ¶
Training-time normalization.
Source code in src/depth_fm/data/scalers.py
denormalize_prediction ¶
Inference-time inverse. No metadata needed.
inverse: sign(n) * ref * expm1(|n| * ln2)
Source code in src/depth_fm/data/scalers.py
denormalize_batch
staticmethod
¶
Vectorized per-sample denormalization using per-sample ref scales.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
normed
|
Tensor
|
(B, ...) log-compressed normalized tensor. |
required |
residual_scales
|
Tensor
|
(B,) ref scale per sample (metres). |
required |
Returns: Physical residual in metres, same shape as normed.
Source code in src/depth_fm/data/scalers.py
AdaptiveWithScaleHead ¶
AdaptiveWithScaleHead(min_scale: float = 0.1, log_scale_mean: float = 1.5, log_scale_std: float = 1.0)
Plane detrend → per-patch adaptive scaling.
The network must have TWO outputs
- Normalised residual (3, H, W) — the usual DepthFM output
- Log-scale prediction (scalar) — a regression head
During training, the target scale is log(local_p98).
During inference, the network predicts the scale, and we use:
physical_residual = predicted_residual * exp(predicted_log_scale)
This gives the best VAE signal (full [-1,1] range always) while remaining invertible on unseen data.
REQUIRES ARCHITECTURE CHANGES to your DepthFM model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
min_scale
|
float
|
Floor for scale factor (metres). |
0.1
|
log_scale_mean
|
float
|
Mean of log(scale) distribution for normalising the scale prediction target. Compute from training data. |
1.5
|
log_scale_std
|
float
|
Std of log(scale) distribution. |
1.0
|
Source code in src/depth_fm/data/scalers.py
normalize_for_training ¶
Returns both the normalised residual AND the scale target.
Returns dict with
"normed_residual": (1, H, W) in [-1, 1] "scale_target": scalar, normalised log-scale for regression "raw_scale": scalar, the actual local_p98 in metres "trend_params": (3,) plane parameters
Source code in src/depth_fm/data/scalers.py
denormalize_prediction ¶
Inference-time inverse using the network's scale prediction.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prediction
|
Tensor
|
(B, 1, H, W) normalised residual. |
required |
predicted_log_scale
|
Tensor
|
(B,) predicted normalised log-scale. |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
(B, 1, H, W) physical residual in metres. |
Source code in src/depth_fm/data/scalers.py
StripOrthoStats
dataclass
¶
StripOrthoStats(p02: float, p98: float, median: float, n_pixels_sampled: int, strip_index: int = -1)
Radiometric statistics for one strip.
GlobalStripOrthoNormalizer ¶
Normalises ortho patches using strip-level quantiles.
All patches from the same strip are normalised with the same (p02, p98), guaranteeing that: - The same physical pixel always gets the same normalised value regardless of which overlapping patch contains it - Shadow depth is preserved in relative terms across the strip - The transform is invertible (if you ever need to go back)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
p02
|
float
|
Strip-level 2nd percentile of valid pixel values. |
required |
p98
|
float
|
Strip-level 98th percentile of valid pixel values. |
required |
Source code in src/depth_fm/data/scalers.py
normalize ¶
Normalise an ortho patch to [-1, 1].
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ortho
|
Tensor
|
(C, H, W) raw ortho reflectance values. |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
(C, H, W) normalised to [-1, 1], clamped. |
Source code in src/depth_fm/data/scalers.py
denormalize ¶
Inverse: normalised [-1, 1] → raw reflectance.
Useful for visualization or re-rendering.
LocalStripOrthoNormalizer ¶
Normalises ortho patches using strip-level quantiles.
All patches from the same strip are normalised with the same (p02, p98), guaranteeing that: - The same physical pixel always gets the same normalised value regardless of which overlapping patch contains it - Shadow depth is preserved in relative terms across the strip - The transform is invertible (if you ever need to go back)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
p02
|
Strip-level 2nd percentile of valid pixel values. |
required | |
p98
|
Strip-level 98th percentile of valid pixel values. |
required |
normalize ¶
Normalise an ortho patch to [-1, 1].
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ortho
|
Tensor
|
(C, H, W) raw ortho reflectance values. |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
(C, H, W) normalised to [-1, 1], clamped. |
Source code in src/depth_fm/data/scalers.py
denormalize ¶
Inverse: normalised [-1, 1] → raw reflectance.
Useful for visualization or re-rendering.
fit_plane ¶
Fit z = ax + by + c to elevation using normalised [-1,1] coords.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
elevation
|
Tensor
|
(1, H, W) or (H, W). |
required |
valid_mask
|
Tensor
|
(1, H, W) or (H, W), binary. |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
(3,) tensor [a, b, c]. |
Source code in src/depth_fm/data/scalers.py
evaluate_plane ¶
evaluate_plane(params: Tensor, H: int, W: int, device: device = None, dtype: dtype = torch.float32) -> torch.Tensor
Evaluate plane on (H, W) grid. Returns (1, H, W).