This is the official repository for the paper "LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift".

Authors: Haozhe Si, Yuxuan Wan, Yuqing Wang, Minh Do, Han Zhao.

Modeling hyperspectral imagery (HSI) across different sensors presents a fundamental challenge due to variations in wavelength coverage, band sampling, and channel dimensionality. We introduce LESSViT (Low-rank Efficient Spatial–Spectral Vision Transformer), a sensor-flexible architecture for cross-spectral generalization.

Our contributions are:

1. LESS Attention — a structured low-rank factorization of spatial–spectral attention that reduces complexity from O(N²C²) to O(rNC), where N is the number of spatial tokens, C is the number of spectral channels, and r is the approximation rank.
2. LESSViT — a channel-agnostic ViT with wavelength-aware positional encoding (SSRoPE) that enables consistent modeling under varying spectral configurations.
3. HyperMAE — a hyperspectral masked autoencoder pre-training strategy with decoupled spatial–spectral masking and hierarchical channel sampling for scalable and robust learning.

We pre-train LESSViT using HyperMAE on the SpectralEarth benchmark (EnMAP hyperspectral data) for 200 epochs.

To launch pre-training, run:

bash launch_train.sh

See GeospatialFM/scripts/args.py for full argument descriptions.

We evaluate LESSViT under a cross-spectral generalization setting on the SpectralEarth benchmark. Models are pre-trained and fine-tuned on a fixed channel configuration (C120_VNIR+) and evaluated across four spectral settings:

Downstream datasets: enmap_cdl, enmap_corine, enmap_eurocrops, enmap_bdforet, enmap_bnetd.

To launch fine-tuning on a SpectralEarth dataset, run:

bash launch_finetune.sh

which wraps:

python3 GeospatialFM/finetune/finetune.py \
    --dataset_name ${DATASET_NAME} \
    --task_type ${TASK_TYPE} \
    --data_dir ${DATA_DIR} \
    --gen_task ${GEN_TASK} \
    --model_name ${MODEL_NAME} \
    --pretrained_model_path ${PRETRAINED_MODEL_PATH} \
    --run_name ${RUN_NAME} \
    --output_dir ${OUTPUT_DIR}

• --dataset_name: One of enmap_cdl, enmap_corine, enmap_eurocrops, enmap_bdforet, enmap_bnetd.
• --gen_task: One of id, ood_a, ood_complement, ood_full.
• --model_name: The backbone to fine-tune. lessvit (this work) or one of the supported baselines: dofa, dinov3, specvit, spatsigma, channelvit, hyperfree (HyperFree).
• --pretrained_model_path: A directory, not a file — this convention is the same for every backbone. Each encoder's load_pretrained_weights globs for its own checkpoint file(s) inside the directory it's given (see GeospatialFM/models/wrappers/). By convention, checkpoints for a given backbone live under results/models/<model_name>/, the same tree that pre-training and fine-tuning runs save their own checkpoints to.

See GeospatialFM/finetune/args.py for full argument descriptions, and GeospatialFM/models/registry.py for the full set of supported baseline encoders.

Pre-trained model checkpoints will be released soon. Stay tuned!

If you find our work helpful, please cite our paper:

@misc{si2026lessvitrobusthyperspectralrepresentation,
      title={LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift}, 
      author={Haozhe Si and Yuxuan Wan and Yuqing Wang and Minh Do and Han Zhao},
      year={2026},
      eprint={2605.18541},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2605.18541}, 
}

Haozhe Si, Han Zhao