Files

T

basketballcantho a4ad11742a bổ sung mô tả dự án

2026-03-27 09:18:38 +07:00

28 KiB

Raw Blame History

GEMINI PROJECT CONTEXT - Land Classification & Remote Sensing System

Last Updated: March 26, 2026
Project Location: /home/x79/remote-sensing
Purpose: Complete land classification and environmental monitoring system using satellite remote sensing for Vietnam

📋 PROJECT OVERVIEW

High-Level Purpose & Problem Domain

Core Task: Classify land use/land cover (8 land classes) in Vietnam using multispectral Sentinel-2 and radar Sentinel-1 data from Microsoft Planetary Computer
Geographic Focus: Vietnam provinces/regions with bounding-box (bbox) based Area-of-Interest (AOI) selection
Key Capabilities:
- Dynamic training with user-selected regions and time periods
- Pixel-wise inference (prediction) on new regions
- Cloud removal using 7 different strategies
- NDVI time-series forecasting and change detection workflows
- Auto-generated HTML reports with visualizations
- Batch processing of multiple regions
- Model lifecycle management (save, load, validate, delete)

Data Pipeline

Sentinel-2 (optical) + Sentinel-1 (SAR) 
    ↓
[Feature Extraction: 4 modes - simple (3) / temporal (39) / extended (15) / odc (8)]
    ↓
[Model Training: XGBoost, RF, SVM, CNN, Swin-UNet, MobileNet-LRASPP]
    ↓
[Prediction: Pixel-wise classification]
    ↓
[Output: GeoTIFF + PNG preview + HTML report + JSON metadata]

🏗️ SYSTEM ARCHITECTURE

Core Technology Stack

Backend: FastAPI (~4200 lines in api_server.py)
ML Training: scikit-learn (XGBoost, RF, SVM, DT) + PyTorch (CNN, Swin-UNet, MobileNet)
Geospatial: rasterio, rioxarray, geopandas, xarray, odc.stac
Data Access: Microsoft Planetary Computer STAC API (Sentinel-2 L2A, Sentinel-1 RTC)
Frontend: HTML + Leaflet.js (map drawing) + Fetch API + Chart.js
GPU Support: PyTorch with CUDA 12.x (optional fallback to CPU)

Folder Structure

remote-sensing/
├── Core Backend
│   ├── api_server.py                 # FastAPI app (~4200 LOC, 70+ endpoints)
│   ├── train_module.py               # Training pipeline engine
│   ├── feature_extractor.py          # Unified feature extraction (4 modes)
│   ├── model_manager.py              # Model lifecycle management
│   ├── cloud_removal.py              # 7 cloud removal strategies
│   ├── report_generator.py           # Auto HTML/PNG report generation
│   ├── generate_previews.py          # GeoTIFF → PNG conversion
│   │
├── Utilities & Lookup
│   ├── vietnam_provinces.py          # Province bboxes & metadata
│   ├── vietnam_provinces_merged.py   # 32-province variant
│   ├── utils.py                      # Geospatial helper functions
│   ├── create_odc_metadata.py        # Metadata generator utility
│   │
├── Frontend Pages (HTML)
│   ├── index.html                    # Main dashboard hub
│   ├── training_interface.html       # Training UI
│   ├── prediction_interface.html     # Prediction UI
│   ├── batch_interface.html          # Batch processing UI
│   ├── ndvi_interface.html           # NDVI time-series UI
│   ├── dashboard.html                # Analytics dashboard
│   ├── reports_interface.html        # Reports management
│   ├── change_detection_interface.html # Change detection UI
│   ├── cloud_training_interface.html # Cloud removal training UI
│   │
├── Tests & Notebooks
│   ├── test_*.py                     # Unit & integration tests
│   ├── 01.train_ODC*.ipynb           # Training notebooks
│   ├── 02.predict_ODC.ipynb          # Prediction notebooks
│   ├── cloud_removal_train.ipynb     # Cloud removal training
│   │
├── Model Storage & Caches
│   ├── model_train/                  # Trained models (*.joblib, *.pth)
│   │   ├── model_odc.joblib          # Legacy GridSearchCV model
│   │   ├── model_*_info.json         # Metadata sidecar files
│   ├── cloud_removal_model/          # Cloud removal U-Net models (.pth)
│   ├── predictions/                  # Prediction output (GeoTIFF + PNG)
│   ├── reports/                      # Generated HTML reports
│   ├── dataset_cache/                # Cached Sentinel data (optional)
│   │
├── Config & Documentation
│   ├── requirement.txt               # Python dependencies
│   ├── requirements_api.txt          # API-specific deps
│   ├── IMPLEMENTATION_SUMMARY.md     # Model manager summary
│   ├── MODEL_MANAGER_GUIDE.md        # Full model management guide
│   ├── NDVI_FORECAST_METHODOLOGY.md  # NDVI algorithm docs
│   ├── CLOUD_TRAINING_GUIDE.md       # Cloud removal training guide
│   └── [Other guides & docs]

🔧 MAIN MODULES & RESPONSIBILITIES

Module	File(s)	Key Responsibility
API Server	`api_server.py`	FastAPI app with 70+ endpoints; routes all training, prediction, batch, cloud removal, dashboard, reports, model management tasks
Training Engine	`train_module.py`	Complete training pipeline: fetch data → feature extraction → train/test split → model training → evaluation → save with metadata
Feature Extraction	`feature_extractor.py`	Standardized feature extraction with 4 modes: simple, temporal, extended, odc; used by both training and prediction
Model Manager	`model_manager.py`	Lifecycle management: list, load, save, validate, delete models; handles metadata JSON; auto-detects CNN/PyTorch models
Cloud Removal	`cloud_removal.py`	7 cloud removal strategies: classic (3-step), temporal_only, median_composite, none, speckle filter, ML inpainting, deep learning U-Net
Report Generator	`report_generator.py`	Auto-generates HTML/PNG reports with confusion matrices, class distributions, accuracy trends
Preview Generator	`generate_previews.py`	Converts GeoTIFF outputs to PNG previews (NDVI or classification rasters)
Province Lookup	`vietnam_provinces*.py`	Lookup tables for 32+ Vietnamese provinces with bboxes and region grouping
Utilities	`utils.py`	Geospatial helper functions (load GeoDataFrames, etc.)

📊 END-TO-END WORKFLOWS

1. TRAINING WORKFLOW

User Input → Training Configuration
    ↓
API Endpoint: POST /api/training/start
    ↓
train_module.py: train_model()
    1. Fetch Sentinel-2 & Sentinel-1 from Planetary Computer STAC
    2. Apply cloud mask (SCL band: clouds, shadows, cirrus masked)
    3. Extract features via FeatureExtractor (mode: simple/temporal/extended/odc)
    4. Train/test split (default 0.2)
    5. Train selected model type (XGBoost, RF, CNN, Swin-UNet, MobileNet)
    6. Evaluate: accuracy, precision, recall, F1, confusion matrix
    ↓
model_manager.py: Save model + JSON metadata
    ↓
report_generator.py: Auto-generate HTML training report
    ↓
Return: {model_filename, accuracy_metrics, training_time}

Key Metadata Saved:

{
  "timestamp": "2026-03-26T14:30:00",
  "model_type": "xgboost",
  "feature_mode": "temporal",
  "n_features": 39,
  "n_classes": 8,
  "features": ["NDVI_t1", "NDVI_t2", ..., "NDWI_t1", ...],
  "test_accuracy": 0.85,
  "train_accuracy": 0.92,
  "bbox": [105.6, 9.3, 106.2, 9.8],
  "time_range": "2023-03-01/2023-05-31",
  "resolution": 20,
  "data_source": "Microsoft Planetary Computer STAC"
}

2. PREDICTION WORKFLOW

User Input → Prediction Configuration (model_filename, bbox, time_range, cloud_strategy)
    ↓
API Endpoint: POST /api/predict or POST /api/predict/with-ndvi
    ↓
run_prediction() function:
    1. Load model via model_manager.py (retrieves metadata, feature requirements)
    2. Fetch Sentinel-2 & Sentinel-1 for new region
    3. Apply chosen cloud_removal_method (classic/temporal_only/median_composite/none/deep_learning)
    4. Extract features matching model's metadata requirements
    5. Auto-adjust if feature count mismatch (pad/trim)
    6. Predict land class for each pixel
    7. (Optional) Calculate NDVI: (NIR - Red) / (NIR + Red)
    8. Save outputs: GeoTIFF + PNG preview
    ↓
generate_previews.py: Create PNG from GeoTIFF
    ↓
report_generator.py: Generate prediction report
    ↓
Return: {prediction_file, ndvi_file, class_distribution, statistics}

3. BATCH PROCESSING WORKFLOW

User uploads CSV with multiple regions:
  (name, min_lon, min_lat, max_lon, max_lat, start_date, end_date, max_scenes, cloud_cover, resolution)
    ↓
API Endpoint: POST /api/batch/start
    ↓
Enqueue all regions; process sequentially
    ↓
For each region: Run same prediction workflow
    ↓
Track status per region: Queued → Running → Completed/Failed
    ↓
UI shows progress bar, auto-retry on failure (max 3 retries)
    ↓
Return: Bulk results with per-region status & output files

4. CLOUD REMOVAL WORKFLOW

User selects cloud_removal_method in prediction config:
    ↓
cloud_removal.py: process_cloud_removal()
    ↓
Strategy Selection:
  • 'classic': temporal interpolation → median composite → spatial interpolation (3-step)
  • 'temporal_only': ffill + bfill across time dimension (fast, good for many scenes)
  • 'median_composite': Prioritize median across scenes (best for noise reduction)
  • 'none': Keep original, just fill NaN with 0
  • 'deep': Use trained U-Net model (S2 cloudy + S1 → clean S2)
  • 'ml_inpainting': KNN or Random Forest based inpainting
  • 'speckle_filter': Reduce radar noise
    ↓
Return cleaned Sentinel-2 data for subsequent feature extraction

5. NDVI TIME-SERIES WORKFLOW

User requests NDVI calculation (bbox + time_range + aggregation)
    ↓
API Endpoint: POST /api/ndvi/timeseries or /api/ndvi/predict-timeseries
    ↓
Load Sentinel-2 (B04 Red, B08 NIR)
    ↓
Calculate NDVI = (NIR - Red) / (NIR + Red + 0.00001)
    ↓
Resample to monthly or user-defined aggregation
    ↓
Export as GeoTIFF + PNG visualization
    ↓
Show time-series graph & statistics (mean, min, max, std, trend)

6. CHANGE DETECTION WORKFLOW

User selects: model + current_period + prediction_period
    ↓
API Endpoint: POST /api/change-detection/compare-periods
    ↓
Run prediction for both time periods
    ↓
Compute difference map (current - prediction)
    ↓
Classify changes: increased vegetation, decreased vegetation, stable
    ↓
Generate change map GeoTIFF + report with statistics

🌐 API ENDPOINTS SUMMARY (70+ endpoints)

Model Management

GET /api/models/list - List all trained models with metadata
GET /api/models/{filename}/info - Get model details
GET /api/models/{filename}/validate - Validate model integrity
DELETE /api/models/{filename} - Delete model file

Training APIs

POST /api/training/start - Start land classification training
GET /api/training/status - Get training progress
POST /api/training/stop - Cancel ongoing training
POST /api/cloud-removal/train - Train cloud removal U-Net

Prediction APIs

POST /api/predict - Standard prediction (classification only)
POST /api/predict/with-ndvi - Prediction with NDVI export
POST /api/change-detection/compare-periods - Change detection
GET /api/prediction/status - Check prediction progress
GET /api/predictions/list - List prediction outputs
GET /api/predictions/download/{filename} - Download prediction file
GET /api/predictions/preview/{filename} - View PNG preview

Batch Processing

POST /api/batch/start - Enqueue multiple predictions from CSV
GET /api/batch/status - Check batch queue
GET /api/batch/results/{batch_id} - Retrieve batch results
POST /api/batch/cancel/{batch_id} - Cancel batch job

Cloud Removal

GET /api/cloud-removal/methods - List available strategies
GET /api/cloud-removal/models - List trained .pth models
POST /api/cloud-removal/upload - Upload .pth cloud removal model
DELETE /api/cloud-removal/models/{filename} - Delete cloud removal model

Dashboard & Reports

GET /api/dashboard/statistics - Overall system stats
GET /api/dashboard/accuracy-trends - Accuracy over time
GET /api/dashboard/class-distribution/{model_filename} - Class distribution
GET /api/reports/list - List generated reports
GET /api/reports/view/{filename} - View HTML report
GET /api/reports/download/{filename} - Download report
DELETE /api/reports/delete/{filename} - Delete report

Provinces & Utilities

GET /api/provinces/list - List all Vietnamese provinces
GET /api/provinces/by-region - Group provinces by region
GET /api/provinces/{province_name}/bbox - Get province bbox
GET /api/provinces/search/{query} - Search province by name
GET /api/provinces-32/* - Alternative 32-province variant
GET /api/network/check - Check connectivity to Planetary Computer
GET /api/cache/info - Show cache statistics
POST /api/cache/clear - Clear local cache

NDVI & Time-Series

POST /api/ndvi/timeseries - Calculate NDVI time-series
POST /api/ndvi/predict-timeseries - NDVI prediction/forecast
POST /api/ndvi/forecast - NDVI forecasting

File Management

GET /api/training/files - List training files
GET /api/overlay/shapefiles - List available shapefiles
GET /api/training/shapefile/{filename}/labels - Get shapefile labels
POST /api/land-classification/upload - Upload custom model
POST /api/cloud-removal/upload - Upload cloud removal model

Frontend Routes (Serve HTML)

GET / - Main dashboard
GET /training - Training interface
GET /prediction - Prediction interface
GET /dashboard - Analytics dashboard
GET /batch - Batch processing UI
GET /ndvi - NDVI time-series UI
GET /reports - Reports management
GET /cloud-training - Cloud removal training
GET /change-detection - Change detection UI

💾 DATA INPUTS / OUTPUTS & FOLDER CONVENTIONS

Input Data Sources

Sentinel-2 L2A from Microsoft Planetary Computer STAC API
- Bands: B02 (blue), B03 (green), B04 (red), B08 (NIR), B11 (SWIR), SCL (cloud mask)
- Resolution: 10m or 20m (user selectable)
- Collection: sentinel-2-l2a
Sentinel-1 RTC from Planetary Computer
- Bands: VH, VV (radar polarizations)
- Converted to dB scale: 10 * log10(intensity)
- Collection: sentinel-1-rtc
Training Labels: User-provided shapefiles with pixel-level class labels

Output File Structure

predictions/
  ├── prediction_YYYYMMDD_HHMMSS.tif         # Classification GeoTIFF
  ├── prediction_YYYYMMDD_HHMMSS.png         # PNG preview
  ├── ndvi_YYYYMMDD_HHMMSS.tif              # NDVI raster
  ├── ndvi_YYYYMMDD_HHMMSS.png              # NDVI preview

reports/
  ├── training_report_*.html                 # Auto training reports
  ├── prediction_report_*.html                # Auto prediction reports

model_train/
  ├── model_odc.joblib                      # Legacy model
  ├── model_odc_info.json                    # Metadata
  ├── model_xgboost_*.joblib                 # XGBoost models
  ├── model_xgboost_*_info.json              # Metadata
  ├── model_cnn_*.joblib                     # CNN models
  ├── model_cnn_*_info.json                  # Metadata

cloud_removal_model/
  ├── cloud_removal_unet_best.pth            # Trained U-Net
  ├── *.pth                                  # Custom models
  ├── *.json                                 # Model metadata

🔌 EXTERNAL DEPENDENCIES & PLATFORMS

Critical External Services

Microsoft Planetary Computer (STAC API)
- Hosts Sentinel-2 L2A and Sentinel-1 RTC archives
- URL: https://planetarycomputer.microsoft.com/api/stac/v1
- Auto-signed access tokens via planetary_computer.sign_inplace
- Network connectivity check: GET /api/network/check

Key Python Libraries

Geospatial: rasterio, rioxarray, geopandas, shapely, Cartopy, folium, ipyleaflet
Data Processing: numpy, pandas, xarray, dask
ML: scikit-learn, xgboost
Deep Learning: torch, torchvision
Web: fastapi, uvicorn, pydantic
Visualization: matplotlib, Pillow (PIL)
Document Gen: markdown, Pillow

GPU Support

PyTorch with CUDA 12.x (optional; falls back to CPU)
Benefits Swin-UNet and CNN models (10-100x speedup)
CPU training for XGBoost/RF typically <1 hour; deep models need GPU for reasonable speed

⚙️ FEATURE EXTRACTION MODES (CRITICAL)

Train and prediction MUST use same feature mode and dimension; metadata auto-detects this.

Mode	# Features	Description	Best For	Training Time
simple	3	NDVI_mean, VH_db_mean, VV_db_mean	Fast iteration, baseline	~5-10 min
temporal	39	NDVI/NDWI/NDBI across 13 months + radar stats	High accuracy (~85%+)	~30-60 min
extended	15	NDVI/NDWI/NDBI stats (mean/std/min/max) + radar	Balanced speed/accuracy	~15-30 min
odc	8	NDVI stats + NDWI/NDBI/EVI mean (legacy ODC mode)	Legacy compatibility	~10-20 min

Critical: If feature mode = "temporal" (39 features) at training, prediction MUST extract 39 features. System auto-detects from metadata but will fail if mismatched.

🎯 OPERATIONAL NOTES & CONSTRAINTS

Performance Limits

Planetary Computer Timeout Issues
- Large bbox (>10km × 10km) + long time range (>1 month) + high max_scenes → timeouts
- Solution: Progressive loading (subdivide bbox), reduce time window, reduce max_scenes
- Safe Settings: bbox ≤ 10km × 10km, time ≤ 1 month, max_scenes ≤ 12
Memory Usage
- Temporal mode (39 features) requires ~2-3x RAM vs simple mode
- Large regions: reduce resolution (10m → 20m) or split into sub-tiles
- Batch processing: sequential (one region at a time due to API limits)
GPU Training
- Swin-UNet: ~15-60 min on GPU vs ~2-4 hours on CPU
- CNN: ~10-30 min on GPU vs ~1-2 hours on CPU
- XGBoost/RF: CPU-bound; GPU not beneficial

Data Quality Issues

Cloud Cover
- SCL band values: 3=cloud shadow, 8=cloud medium, 9=cloud high, 10=cirrus
- Recommend multiple scenes (≥5) for temporal aggregation
- Cloud removal strategy critical—test different approaches
Radar Data (Sentinel-1)
- Not always available for all regions/dates
- System gracefully falls back to zeros if unavailable
- Safe for "extended" & "odc" modes that have radar fallback
Feature Mode Mismatch
- Model trained with "temporal" (39 features) needs 39-dim input
- System auto-adjusts (pads/trims) from metadata but may degrade accuracy
- Best Practice: Align feature mode explicitly; don't mix

Known Caveats

Legacy Model (model_odc.joblib): Hardcoded 39 temporal features; auto-detected via model_odc_info.json
Metadata Consistency: Old models may lack .json sidecar; system generates default (may be incorrect)
Batch Processing: Sequential only; large batches (100+ regions) take hours
Change Detection: Simple differencing approach; requires same model & feature mode for both periods
Rate Limiting: Planetary Computer may rate-limit if too many concurrent requests

Recommended Best Practices

Test model on small bbox first (2km × 2km, 1 week, 3 scenes)
Use "simple" mode for fast iteration, "temporal" for best accuracy (85%+)
Store metadata JSON alongside model file (sidecar pattern)
Version control: record feature_mode & n_features in every training
Monitor training accuracy; retrain if <70% accuracy
Cache Sentinel data locally to avoid repeated downloads
Use "median_composite" cloud strategy if >5 scenes; "temporal_only" if 3-4 scenes

🔍 TEST COVERAGE MAP

Test File	Coverage	Status
`test_model_manager.py`	ModelManager lifecycle (list, load, validate)	✅ Well-tested
`test_feature_extractor.py`	All 4 feature extraction modes	✅ Well-tested
`test_training_api.py`	Training API endpoints	✅ Partial
`test_cloud_removal.py`	7 cloud removal strategies	✅ Well-tested
`test_cloud_training.py`	U-Net cloud removal training	✅ Partial
`test_shapefile_api.py`	Shapefile overlay feature	✅ Partial
`test_planetary_computer.py`	Planetary Computer STAC access	✅ Well-tested
`test_new_features.py`	Recent feature releases	✅ Partial
Jupyter Notebooks	Training & prediction workflows	✅ Mix of unit/integration/notebooks

Coverage Notes: Model management, feature extraction, and cloud removal well-tested; Dashboard UI, change detection, NDVI time-series mostly tested via notebooks.

📚 FILE REFERENCE MAP

Core Execution

api_server.py — Main FastAPI application (~4200 LOC)
train_module.py — Training logic (data fetch → feature extraction → training)
run_prediction_new.py — Prediction execution function
feature_extractor.py — Unified feature extraction (4 modes)
model_manager.py — Model lifecycle (load/save/validate/list)
cloud_removal.py — Cloud removal strategies (7 methods)
report_generator.py — HTML/PNG report auto-generation
generate_previews.py — GeoTIFF → PNG conversion

Data & Config

vietnam_provinces.py — 32+ province lookup tables & bboxes
vietnam_provinces_merged.py — Alternative 32-province variant
utils.py — Geospatial utility functions
create_odc_metadata.py — Legacy metadata generator

Frontend

index.html — Main dashboard hub (tab navigation)
training_interface.html — Training configuration UI
prediction_interface.html — Prediction configuration UI
batch_interface.html — Batch processing (CSV upload)
ndvi_interface.html — NDVI time-series visualization
dashboard.html — Analytics & model performance dashboard
reports_interface.html — Report management & viewing
change_detection_interface.html — Change detection visualization
cloud_training_interface.html — Cloud removal U-Net training

Documentation

IMPLEMENTATION_SUMMARY.md — Model manager & system overview
MODEL_MANAGER_GUIDE.md — Complete model management guide
NDVI_FORECAST_METHODOLOGY.md — NDVI algorithm documentation
CLOUD_TRAINING_GUIDE.md — Cloud removal training guide
NDVI_PREDICTION_GUIDE.md — NDVI prediction workflow
CLOUD_PROCESSING.md — Cloud processing notes
UPDATE_SUMMARY.md — Recent updates & features

🚀 BOOTSTRAP PROMPT FOR GEMINI

System Context (Copy & Paste for Gemini)

You are assisting a remote-sensing land-classification project for Vietnam.

## ARCHITECTURE SNAPSHOT
- **Backend**: FastAPI (~4200 LOC, 70+ endpoints) for orchestrating training, prediction, batch, cloud removal, reporting
- **Data Source**: Microsoft Planetary Computer STAC API (Sentinel-2 L2A + Sentinel-1 RTC)
- **Training**: scikit-learn (XGBoost/RF/SVM/DT) + PyTorch (CNN/Swin-UNet/MobileNet)
- **Feature Extraction**: 4 modes (simple 3-feat / temporal 39-feat / extended 15-feat / odc 8-feat)
- **Cloud Removal**: 7 strategies (classic, temporal_only, median_composite, none, ML inpainting, deep U-Net)
- **Output**: GeoTIFF + PNG + HTML report + JSON metadata

## CORE FILES TO UNDERSTAND (Priority Order)
1. api_server.py — Main API server (training, prediction, batch, models, reports)
2. train_module.py — Training pipeline (data fetch → feature extraction → train → save)
3. feature_extractor.py — Unified feature extraction with auto mode detection
4. model_manager.py — Model lifecycle (load/save/validate/list)
5. cloud_removal.py — Cloud removal strategies (7 methods)
6. report_generator.py — Auto-generate HTML reports
7. run_prediction_new.py — Prediction execution
8. vietnam_provinces.py — Province lookup & bbox tables

## CRITICAL CONSTRAINTS & GOTCHAS
1. **Feature Mode Consistency**: Training & prediction MUST use same mode (simple/temporal/extended/odc)
   → Auto-detected from metadata JSON
   → Mismatch causes dimension error or accuracy degradation

2. **Planetary Computer Limits**:
   → Timeout if bbox >10km×10km OR time range >1 month OR max_scenes >12
   → Solution: subdivide bbox, reduce time window, limit scenes

3. **Cloud Strategy Selection**:
   → ≥5 scenes → use "median_composite" (best noise reduction)
   → 3-4 scenes → use "temporal_only" (fast temporal interp)
   → <3 scenes → use "none" (skip cloud removal)

4. **Radar Data Fallback**:
   → Sentinel-1 may be unavailable for some regions
   → System gracefully falls back to zeros (safe for all modes)

5. **Model Metadata**:
   → Always stored as `model_name_info.json` sidecar file
   → Contains: n_features, feature_mode, features list, accuracy, bbox, time_range
   → Missing metadata → system uses defaults (may be incorrect)

6. **Legacy Model (model_odc.joblib)**:
   → Hardcoded 39 temporal features
   → Metadata in model_odc_info.json

## REASONING CHECKLIST (before answering)
□ Is feature_mode consistent between train and prediction?
□ Is metadata.json present and correct?
□ Does bbox exceed 10km×10km? (Planetary Computer timeout risk)
□ Is cloud_removal_strategy appropriate for # of scenes?
□ Is Sentinel-1 data available for this region/date?
□ Is model a joblib (scikit-learn) or .pth (PyTorch) file?
□ Is GPU available for deep models (CNN, Swin-UNet)?
□ Does memory allow temporal feature extraction (39-feat)?

## RESPONSE FORMAT
- Always cite api_server.py endpoint, function name, or module being discussed
- Verify feature_mode & n_features from metadata JSON
- Suggest cloud_removal_strategy based on # of scenes available
- For unknown issues: offer alternative approaches (reduce bbox, cache results, use simpler model)
- Explain reasoning using checklist above

## DATA FLOW SUMMARY
Sentinel-2/S1 → [Cloud Remove] → [Feature Extract] → [Train/Predict] → [GeoTIFF + PNG + Report]

📞 QUICK REFERENCE CHECKLIST

Before Troubleshooting Any Issue

Check feature_mode consistency (metadata JSON)
Verify metadata.json exists for the model
Check Planetary Computer connectivity (GET /api/network/check)
Review cloud_removal_method choice (≥5 scenes = median_composite)
Confirm Sentinel-1 availability (or fallback to zeros if missing)
Validate bbox size (≤10km×10km for safety)
Check memory usage for temporal feature mode
Verify GPU if using CNN/Swin-UNet models

Common Issues & Solutions

Issue	Likely Cause	Solution
Training timeout	Large bbox / long time / many scenes	Subdivide bbox, reduce time window, max_scenes ≤ 12
Feature dimension mismatch	Different feature_mode between train & predict	Check metadata.json, ensure same mode
Low prediction accuracy	Cloud cover, poor training data, feature mode too simple	Use "temporal" mode, increase training data, try cloud removal
Out of memory	Temporal features + large region	Reduce resolution (20m), split into sub-tiles, increase RAM
Model not found	Wrong filename or model_train/ path issue	`GET /api/models/list` to verify, check file path
Planetary Computer error	Network issue or API rate limit	Check DNS, retry later, reduce concurrent requests
Cloud removal failing	Strategy not suitable for scene count	Try "none" or "median_composite" depending on scenes

🎓 LEARNING RESOURCES IN REPO

Notebooks: 01.train_ODC.ipynb, 02.predict_ODC.ipynb, cloud_removal_train.ipynb
Tests: test_*.py files for unit test patterns
Docs: All *.md files for detailed guides and methodology
Code Comments: API server and modules heavily commented

Generated: March 26, 2026
For Use By: Gemini, Claude, GPT, or any AI system needing project context
Maintainer: Remote-Sensing Project Team

28 KiB Raw Blame History Unescape Escape