Yuhan Liu

Yuhan Liu

Associate Professor

MFSTPT - Multi-Frame Tensor Model

Published:

Infrared Dim and Small Target Detection from Complex Scenes via Multi-Frame Spatial–Temporal Patch-Tensor Model
Y Hu, Y Ma, Z Pan, Y Liu
Published in Remote Sensing, May 2022
DOI | Code

Project Overview

Challenge: Single Frame Limitations

Single-frame detection methods suffer from:

  • Limited information for weak targets
  • Confusion with background clutter
  • Sensitivity to noise

Multi-frame information can help but is often underutilized.

Solution: Tensor-Based Multi-Frame Processing

MFSTPT leverages temporal consistency through:

  1. Patch-tensor construction across multiple frames
  2. Low-rank + sparse tensor decomposition
  3. Background-target separation via tensor properties

Key Innovations

1. Spatial-Temporal Patch-Tensor

Tensor construction:

  • Extract patches from consecutive frames
  • Stack as 3D tensor (spatial × spatial × temporal)
  • Preserve both spatial and temporal correlations

Advantage: Captures target motion patterns implicitly

2. Tensor Decomposition

Robust Principal Tensor Analysis (RPTA):

  • Background: Low-rank tensor (smooth across space and time)
  • Target: Sparse tensor (rare, small, moving)
  • Noise: Residual component

Optimization: Iterative soft-thresholding algorithm

3. Motion-Aware Patch Grouping

Advanced patch selection:

  • Group patches with similar motion patterns
  • Better tensor structure for decomposition
  • Improved separation of moving targets

Method

Processing Pipeline

T Consecutive Frames
    ↓
[Patch Extraction]
├── Sliding window patches
├── Motion estimation
└── Patch grouping
    ↓
[Tensor Construction]
├── 3D tensor formation
├── Tensor unfolding
└── Preprocessing
    ↓
[Tensor Decomposition]
├── Low-rank approximation
├── Sparse component extraction
└── Background suppression
    ↓
[Target Extraction]
├── Thresholding
├── Connected components
└── Final detection

Results

Performance highlights:

  • Superior detection in complex backgrounds
  • Robust to low SNR conditions
  • Effective use of temporal information

Advantages over single-frame methods:

  • Better background suppression
  • Improved weak target detection
  • Reduced false alarms

Applications

  • Infrared surveillance systems
  • Early warning radar
  • Maritime monitoring
  • Border security

Citation

@article{hu2022infrared,
  title={Infrared Dim and Small Target Detection from Complex Scenes via Multi-Frame Spatial--Temporal Patch-Tensor Model},
  author={Hu, Y and Ma, Y and Pan, Z and Liu, Y},
  journal={Remote Sensing},
  volume={14},
  number={9},
  pages={2234},
  year={2022},
  publisher={MDPI}
}