remote-sensing/MODEL_UPLOAD_GUIDE.md

# Model Upload Guide

## Overview
This system now supports uploading custom models for both **Cloud Removal** and **Land Classification** tasks with full metadata tracking.

## Directory Structure

```
remote-sensing/
├── cloud_removal_model/          # Cloud removal models (U-Net, GAN, etc.)
│   ├── *.pth                     # PyTorch model files
│   └── *.json                    # Metadata sidecar files
├── land_classification_model/    # Land use classification models
│   ├── *.pth, *.pkl, *.joblib   # Model files (various formats)
│   ├── *.h5, *.keras            # TensorFlow/Keras models
│   └── *.json                    # Metadata sidecar files
└── model_train/                  # Legacy training outputs (other models)
```

## Cloud Removal Model Upload

### Supported Format
- **File Extension**: `.pth` (PyTorch)
- **Use Case**: Remove clouds from Sentinel-2 imagery

### Metadata Fields
- **Epoch** (int): Training epoch number
- **Validation Loss** (float): Best validation loss achieved
- **Training Loss** (float): Final training loss
- **Input Channels** (int): Number of input channels (e.g., 6 for S2+S1)
- **Output Channels** (int): Number of output channels (e.g., 4 for RGBN)
- **Use Sentinel-1** (bool): Whether model uses SAR data
- **Description** (string): Optional notes about the model

### API Endpoint
```http
POST /api/cloud-removal/upload
Content-Type: multipart/form-data

{
  "file": <binary>,
  "epoch": 50,
  "val_loss": 0.0134,
  "train_loss": 0.0142,
  "in_channels": 6,
  "out_channels": 4,
  "use_s1": true,
  "description": "Trained on winter dataset"
}
```

### Example Metadata File
`cloud_removal_unet_winter.pth.json`:
```json
{
  "filename": "cloud_removal_unet_winter.pth",
  "epoch": 50,
  "train_loss": 0.0142,
  "val_loss": 0.0134,
  "in_channels": 6,
  "out_channels": 4,
  "use_s1": true,
  "description": "Trained on winter dataset, 50 epochs",
  "uploaded_at": "2026-01-26T15:30:00"
}
```

---

## Land Classification Model Upload

### Supported Formats
- **PyTorch**: `.pth`
- **Scikit-learn**: `.pkl`, `.joblib`
- **TensorFlow/Keras**: `.h5`, `.keras`

### Metadata Fields
- **Model Type**: `mobilenet`, `cnn`, `swin`, `xgboost`, `random_forest`, `other`
- **Epoch** (int): Training epochs
- **Train Accuracy** (float %): Training accuracy percentage
- **Val Accuracy** (float %): Validation accuracy percentage
- **Train Loss** (float): Final training loss
- **Val Loss** (float): Final validation loss
- **Number of Classes** (int): Number of land use classes (e.g., 10)
- **Input Size** (int): Input image dimension (e.g., 64x64)
- **Description** (string): Optional notes

### API Endpoint
```http
POST /api/land-classification/upload
Content-Type: multipart/form-data

{
  "file": <binary>,
  "model_type": "mobilenet",
  "epoch": 100,
  "train_accuracy": 95.5,
  "val_accuracy": 93.2,
  "train_loss": 0.12,
  "val_loss": 0.18,
  "num_classes": 10,
  "input_size": 64,
  "description": "MobileNetV2 trained on Mekong Delta"
}
```

### Example Metadata File
`mobilenet_mekong_v2.pth.json`:
```json
{
  "filename": "mobilenet_mekong_v2.pth",
  "model_type": "mobilenet",
  "epoch": 100,
  "train_accuracy": 95.5,
  "val_accuracy": 93.2,
  "train_loss": 0.12,
  "val_loss": 0.18,
  "num_classes": 10,
  "input_size": 64,
  "description": "MobileNetV2 trained on Mekong Delta dataset",
  "uploaded_at": "2026-01-26T15:45:00"
}
```

---

## Usage in Web Interface

### Cloud Removal Models
1. Navigate to **Prediction Interface**
2. Select **Cloud Removal Method** → "Deep Learning (U-Net)"
3. Click **📤 Upload Cloud Removal Model (.pth)**
4. Fill in metadata form
5. Click **✅ Upload with Metadata**
6. Model appears in dropdown with epoch/loss info

### Land Classification Models
1. Navigate to **Prediction Interface**
2. In **Model Selection** section
3. Click **📤 Upload Land Classification Model**
4. Fill in metadata form (model type, accuracy, etc.)
5. Click **✅ Upload with Metadata**
6. Model appears in main model dropdown

---

## API Reference

### List Models

**Cloud Removal:**
```http
GET /api/cloud-removal/models
```

**Land Classification:**
```http
GET /api/land-classification/models
```

**Response:**
```json
{
  "models": [
    {
      "filename": "model.pth",
      "epoch": 50,
      "val_loss": 0.0134,
      "size_mb": 356.2,
      "has_metadata": true,
      "created": 1706284800
    }
  ],
  "count": 1
}
```

### Delete Model

**Cloud Removal:**
```http
DELETE /api/cloud-removal/models/{filename}
```

**Land Classification:**
```http
DELETE /api/land-classification/models/{filename}
```

---

## Best Practices

1. **Naming Convention**: Use descriptive names
   - ✅ `cloud_removal_unet_winter_50ep.pth`
   - ✅ `mobilenet_v2_mekong_acc93.pth`
   - ❌ `model1.pth`

2. **Metadata Accuracy**: Always fill in actual training metrics
   - Helps compare model performance
   - Enables informed model selection

3. **Version Control**: Include version/date in description
   - "v2.0 - Improved augmentation"
   - "2026-01-15 - Fixed class imbalance"

4. **File Size**: Monitor model sizes
   - Cloud removal models: 50-500 MB typical
   - Land classification: 5-200 MB typical
   - Large models may require more GPU memory

5. **Testing**: Always test uploaded model on small region first
   - Verify predictions are reasonable
   - Check for errors/crashes

---

## Troubleshooting

### Upload Fails with "Already Exists"
- Model filename is duplicate
- Delete old model first or rename new one

### Model Shows Default Values (0, 0, 0)
- Server needs restart to load `Form(...)` imports
- Refresh page and try again

### Model Not Appearing in Dropdown
- Click **🔄 Refresh** button
- Check file extension is valid
- Verify model saved to correct folder

### Metadata Not Displaying
- Check `.json` file exists alongside model
- Verify JSON format is valid
- Look for server errors in terminal

---

## Migration from Old System

If you have models in `model_train/`:

1. **Cloud Removal Models**: Move to `cloud_removal_model/`
   ```bash
   mv model_train/cloud_removal_*.pth cloud_removal_model/
   mv model_train/*_unet*.pth cloud_removal_model/
   mv model_train/*GAN*.pth cloud_removal_model/
   ```

2. **Land Classification Models**: Move to `land_classification_model/`
   ```bash
   mv model_train/mobilenet*.pth land_classification_model/
   mv model_train/cnn*.pth land_classification_model/
   mv model_train/swin*.pth land_classification_model/
   mv model_train/*.pkl land_classification_model/
   ```

3. **Create metadata files** by re-uploading through web interface

---

## Security Features

✅ **File Extension Validation**: Only allowed formats accepted  
✅ **Path Traversal Prevention**: No `../` or `/` in filenames  
✅ **Duplicate Detection**: Prevents overwriting existing models  
✅ **Size Limits**: Prevents extremely large uploads  
✅ **JSON Sanitization**: Metadata stored safely

---

## Future Enhancements

- [ ] Batch model upload
- [ ] Model versioning system
- [ ] Automated benchmarking
- [ ] Model comparison tool
- [ ] Export/import model configs
- [ ] Cloud storage integration

---

**Last Updated**: January 26, 2026