remote-sensing/MODEL_UPLOAD_GUIDE.md

Model Upload Guide

Overview

This system now supports uploading custom models for both Cloud Removal and Land Classification tasks with full metadata tracking.

Directory Structure

remote-sensing/
├── cloud_removal_model/          # Cloud removal models (U-Net, GAN, etc.)
│   ├── *.pth                     # PyTorch model files
│   └── *.json                    # Metadata sidecar files
├── land_classification_model/    # Land use classification models
│   ├── *.pth, *.pkl, *.joblib   # Model files (various formats)
│   ├── *.h5, *.keras            # TensorFlow/Keras models
│   └── *.json                    # Metadata sidecar files
└── model_train/                  # Legacy training outputs (other models)

Cloud Removal Model Upload

Supported Format

• File Extension: .pth (PyTorch)
• Use Case: Remove clouds from Sentinel-2 imagery

Metadata Fields

• Epoch (int): Training epoch number
• Validation Loss (float): Best validation loss achieved
• Training Loss (float): Final training loss
• Input Channels (int): Number of input channels (e.g., 6 for S2+S1)
• Output Channels (int): Number of output channels (e.g., 4 for RGBN)
• Use Sentinel-1 (bool): Whether model uses SAR data
• Description (string): Optional notes about the model

API Endpoint

POST /api/cloud-removal/upload
Content-Type: multipart/form-data

{
  "file": <binary>,
  "epoch": 50,
  "val_loss": 0.0134,
  "train_loss": 0.0142,
  "in_channels": 6,
  "out_channels": 4,
  "use_s1": true,
  "description": "Trained on winter dataset"
}

Example Metadata File

cloud_removal_unet_winter.pth.json:

{
  "filename": "cloud_removal_unet_winter.pth",
  "epoch": 50,
  "train_loss": 0.0142,
  "val_loss": 0.0134,
  "in_channels": 6,
  "out_channels": 4,
  "use_s1": true,
  "description": "Trained on winter dataset, 50 epochs",
  "uploaded_at": "2026-01-26T15:30:00"
}

---

Land Classification Model Upload

Supported Formats

• PyTorch: .pth
• Scikit-learn: .pkl, .joblib
• TensorFlow/Keras: .h5, .keras

Metadata Fields

• Model Type: mobilenet, cnn, swin, xgboost, random_forest, other
• Epoch (int): Training epochs
• Train Accuracy (float %): Training accuracy percentage
• Val Accuracy (float %): Validation accuracy percentage
• Train Loss (float): Final training loss
• Val Loss (float): Final validation loss
• Number of Classes (int): Number of land use classes (e.g., 10)
• Input Size (int): Input image dimension (e.g., 64x64)
• Description (string): Optional notes

API Endpoint

POST /api/land-classification/upload
Content-Type: multipart/form-data

{
  "file": <binary>,
  "model_type": "mobilenet",
  "epoch": 100,
  "train_accuracy": 95.5,
  "val_accuracy": 93.2,
  "train_loss": 0.12,
  "val_loss": 0.18,
  "num_classes": 10,
  "input_size": 64,
  "description": "MobileNetV2 trained on Mekong Delta"
}

Example Metadata File

mobilenet_mekong_v2.pth.json:

{
  "filename": "mobilenet_mekong_v2.pth",
  "model_type": "mobilenet",
  "epoch": 100,
  "train_accuracy": 95.5,
  "val_accuracy": 93.2,
  "train_loss": 0.12,
  "val_loss": 0.18,
  "num_classes": 10,
  "input_size": 64,
  "description": "MobileNetV2 trained on Mekong Delta dataset",
  "uploaded_at": "2026-01-26T15:45:00"
}

---

Usage in Web Interface

Cloud Removal Models

1. Navigate to Prediction Interface
2. Select Cloud Removal Method → "Deep Learning (U-Net)"
3. Click 📤 Upload Cloud Removal Model (.pth)
4. Fill in metadata form
5. Click ✅ Upload with Metadata
6. Model appears in dropdown with epoch/loss info

Land Classification Models

1. Navigate to Prediction Interface
2. In Model Selection section
3. Click 📤 Upload Land Classification Model
4. Fill in metadata form (model type, accuracy, etc.)
5. Click ✅ Upload with Metadata
6. Model appears in main model dropdown

---

API Reference

List Models

Cloud Removal:

GET /api/cloud-removal/models

Land Classification:

GET /api/land-classification/models

Response:

{
  "models": [
    {
      "filename": "model.pth",
      "epoch": 50,
      "val_loss": 0.0134,
      "size_mb": 356.2,
      "has_metadata": true,
      "created": 1706284800
    }
  ],
  "count": 1
}

Delete Model

Cloud Removal:

DELETE /api/cloud-removal/models/{filename}

Land Classification:

DELETE /api/land-classification/models/{filename}

---

Best Practices

1. Naming Convention: Use descriptive names

   • ✅ cloud_removal_unet_winter_50ep.pth
   • ✅ mobilenet_v2_mekong_acc93.pth
   • ❌ model1.pth

2. Metadata Accuracy: Always fill in actual training metrics

   • Helps compare model performance
   • Enables informed model selection

3. Version Control: Include version/date in description

   • "v2.0 - Improved augmentation"
   • "2026-01-15 - Fixed class imbalance"

4. File Size: Monitor model sizes

   • Cloud removal models: 50-500 MB typical
   • Land classification: 5-200 MB typical
   • Large models may require more GPU memory

5. Testing: Always test uploaded model on small region first

   • Verify predictions are reasonable
   • Check for errors/crashes

---

Troubleshooting

Upload Fails with "Already Exists"

• Model filename is duplicate
• Delete old model first or rename new one

Model Shows Default Values (0, 0, 0)

• Server needs restart to load Form(...) imports
• Refresh page and try again

Model Not Appearing in Dropdown

• Click 🔄 Refresh button
• Check file extension is valid
• Verify model saved to correct folder

Metadata Not Displaying

• Check .json file exists alongside model
• Verify JSON format is valid
• Look for server errors in terminal

---

Migration from Old System

If you have models in model_train/:

1. Cloud Removal Models: Move to cloud_removal_model/

   mv model_train/cloud_removal_*.pth cloud_removal_model/
   mv model_train/*_unet*.pth cloud_removal_model/
   mv model_train/*GAN*.pth cloud_removal_model/
   

2. Land Classification Models: Move to land_classification_model/

   mv model_train/mobilenet*.pth land_classification_model/
   mv model_train/cnn*.pth land_classification_model/
   mv model_train/swin*.pth land_classification_model/
   mv model_train/*.pkl land_classification_model/
   

3. Create metadata files by re-uploading through web interface

---

Security Features

✅ File Extension Validation: Only allowed formats accepted
✅ Path Traversal Prevention: No ../ or / in filenames
✅ Duplicate Detection: Prevents overwriting existing models
✅ Size Limits: Prevents extremely large uploads
✅ JSON Sanitization: Metadata stored safely

---

Future Enhancements

• Batch model upload
• Model versioning system
• Automated benchmarking
• Model comparison tool
• Export/import model configs
• Cloud storage integration

---

Last Updated: January 26, 2026