remote-sensing/CLOUD_REMOVAL_UPLOAD_GUIDE.md

Cloud Removal Model Upload Feature

Overview

Added functionality to upload and use custom deep learning cloud removal models (.pth files) during prediction.

Features Implemented

1. API Endpoints

Upload Cloud Removal Model

POST /api/cloud-removal/upload

• Upload .pth cloud removal model files
• Validates file extension (.pth only)
• Security checks for filename
• Returns file info (name, size)

Example:

curl -X POST -F "file=@cloud_removal_unet_best.pth" \
  http://localhost:8000/api/cloud-removal/upload

List Cloud Removal Models

GET /api/cloud-removal/models

Already existing - lists all .pth models in model_train/ directory

Delete Cloud Removal Model

DELETE /api/cloud-removal/models/{filename}

Already existing - deletes a specific cloud removal model

2. Prediction Configuration Updates

PredictionConfig

Added new optional field:

cloud_removal_model: Optional[str] = None  # .pth filename

PredictionWithNDVIConfig

Added new optional field:

cloud_removal_model: Optional[str] = None  # .pth filename

3. Prediction Function Integration

The run_prediction() function now:

1. Accepts cloud_removal_model parameter
2. Passes model path to process_cloud_removal()
3. Logs which model is being used

Code:

cloud_removal_method = config.cloud_removal_method
cloud_removal_model = config.cloud_removal_model

s2_data, cloud_metadata = process_cloud_removal(
    s2_data=s2_data,
    method=cloud_removal_method,
    model_path=f"model_train/{cloud_removal_model}" if cloud_removal_model else None,
    verbose=True
)

4. Web Interface Updates

Upload Button

• Added file input in "Deep Learning" cloud removal section
• Upload button appears when "Deep Learning" method is selected
• Real-time upload status feedback
• Auto-refreshes model list after successful upload

Model Selection

• Dropdown shows all available .pth models
• Auto-selects newly uploaded model
• Shows model metadata (epoch, loss)

Usage Guide

Step 1: Train or Obtain a Cloud Removal Model

Train using the cloud training interface or obtain a pre-trained .pth model.

Step 2: Upload Model

1. Go to Prediction Interface
2. Scroll to "Cloud Removal Method" section
3. Select "Deep Learning (U-Net)" from dropdown
4. Model upload section appears
5. Click "📤 Upload Cloud Removal Model (.pth)"
6. Select your .pth file
7. Wait for upload confirmation

Step 3: Use Model in Prediction

1. The uploaded model is automatically selected
2. Configure other prediction parameters (bbox, dates, etc.)
3. Click "🚀 Start Prediction (với NDVI)"
4. The system will use your custom model for cloud removal

File Structure

model_train/
├── cloud_removal_unet_best.pth         # User uploaded
├── cloud_removal_unet_epoch_10.pth     # User uploaded
├── model_mobilenet-lraspp_*.joblib     # Land classification models
└── ...

API Request Example

Using Uploaded Model

{
  "model_filename": "model_mobilenet-lraspp_20260105_225459.joblib",
  "min_lon": 105.80,
  "min_lat": 10.00,
  "max_lon": 105.82,
  "max_lat": 10.02,
  "start_date": "2024-01-15",
  "end_date": "2024-01-17",
  "max_scenes": 3,
  "cloud_cover": 30,
  "resolution": 20,
  "use_gpu": true,
  "export_ndvi": true,
  "export_classification": true,
  "cloud_removal_method": "deep",
  "cloud_removal_model": "cloud_removal_unet_best.pth"
}

Without Custom Model (Classical Methods)

{
  ...
  "cloud_removal_method": "hybrid",
  "cloud_removal_model": null
}

Security Features

• Filename validation (no path traversal)
• File extension validation (.pth only)
• File existence checks
• Duplicate filename detection

Error Handling

• Invalid file type → 400 Bad Request
• Duplicate filename → 400 Bad Request
• Upload failure → 500 Internal Server Error
• Missing model when "deep" selected → Falls back to "hybrid" method

Notes

• Uploaded models are stored in model_train/ directory
• Models must be PyTorch .pth files
• Compatible with cloud_removal.py module
• Works with both /api/prediction/start and /api/predict/with-ndvi endpoints

Testing

Test Upload

# Upload a model
curl -X POST -F "file=@my_cloud_model.pth" \
  http://localhost:8000/api/cloud-removal/upload

# List models
curl http://localhost:8000/api/cloud-removal/models

# Delete model
curl -X DELETE \
  http://localhost:8000/api/cloud-removal/models/my_cloud_model.pth

Test Prediction

curl -X POST http://localhost:8000/api/predict/with-ndvi \
  -H "Content-Type: application/json" \
  -d '{
    "model_filename": "model_mobilenet-lraspp_20260105_225459.joblib",
    "min_lon": 105.80, "min_lat": 10.00,
    "max_lon": 105.82, "max_lat": 10.02,
    "start_date": "2024-01-15", "end_date": "2024-01-17",
    "max_scenes": 2, "cloud_cover": 30, "resolution": 20,
    "use_gpu": false, "export_ndvi": true,
    "cloud_removal_method": "deep",
    "cloud_removal_model": "cloud_removal_unet_best.pth"
  }'

Future Enhancements

• Model metadata display (architecture, training date)
• Model validation on upload
• Multiple model format support (.pt, .onnx)
• Model performance metrics
• Batch upload support