diff --git a/NDVI_PREDICTION_GUIDE.md b/NDVI_PREDICTION_GUIDE.md new file mode 100644 index 0000000..eaa2539 --- /dev/null +++ b/NDVI_PREDICTION_GUIDE.md @@ -0,0 +1,202 @@ +# Hướng Dẫn Sử Dụng Chức Năng Predict NDVI + +## Tổng Quan +Chức năng mới cho phép dự đoán phân loại đất (land classification) **kết hợp** với việc xuất ra raster NDVI cho cùng một khu vực. + +## Cách Sử Dụng + +### 1. Truy cập Prediction Interface +- Mở trình duyệt: `http://localhost:8000/prediction` +- Hoặc từ trang chủ, click vào **Prediction** + +### 2. Chọn Model +- Chọn model đã được train từ dropdown "Select Model" +- Model phải tồn tại trong thư mục `model_train/` + +### 3. Vẽ Khu Vực (Bbox) +- Sử dụng công cụ vẽ hình chữ nhật trên bản đồ +- Khu vực này sẽ được dùng để: + - Load dữ liệu vệ tinh + - Tính NDVI + - Predict land classification + +### 4. Cấu Hình Thời Gian & Dữ Liệu +- **Từ ngày / Đến ngày**: Khoảng thời gian lấy ảnh vệ tinh +- **Max Scenes**: Số lượng ảnh tối đa (khuyến nghị: 12) +- **Cloud Cover**: % mây tối đa (khuyến nghị: 30%) +- **Resolution**: Độ phân giải (10m hoặc 20m) + +### 5. Bật Export NDVI +- ✅ Check vào "🌿 Export NDVI Raster" +- Khi bật, hệ thống sẽ: + - Tính NDVI từ Sentinel-2 (NIR - Red) / (NIR + Red) + - Xuất ra file `ndvi_YYYYMMDD_HHMMSS.tif` + - Xuất ra file `classification_YYYYMMDD_HHMMSS.tif` + +### 6. Chạy Prediction +- Click "🚀 Start Prediction (với NDVI)" +- Hệ thống sẽ: + 1. Load dữ liệu Sentinel-2 (bands: B02, B03, B04, B08) + 2. Tính toán các spectral indices (NDVI, NDWI, NDBI) + 3. Dùng model để predict land classification + 4. Xuất kết quả + +## Kết Quả + +### Output Files +Sau khi hoàn thành, bạn sẽ nhận được 2 file trong thư mục `predictions/`: + +1. **`ndvi_YYYYMMDD_HHMMSS.tif`** + - GeoTIFF chứa giá trị NDVI + - Giá trị: -1 đến +1 + - CRS: EPSG:4326 (WGS84) + - Có thể mở bằng QGIS, ArcGIS, hoặc Python + +2. **`classification_YYYYMMDD_HHMMSS.tif`** + - GeoTIFF chứa kết quả phân loại đất + - Giá trị: class labels (ví dụ: 0, 1, 2, 3...) + - CRS: EPSG:4326 (WGS84) + +### Thống Kê Hiển Thị +Sau khi predict xong, giao diện sẽ hiển thị: +- **NDVI Statistics**: + - Mean: Giá trị NDVI trung bình + - Min: Giá trị NDVI nhỏ nhất + - Max: Giá trị NDVI lớn nhất + - Std: Độ lệch chuẩn +- **Class Distribution**: Số lượng pixel cho mỗi class +- **N Scenes**: Số ảnh vệ tinh đã sử dụng + +## API Endpoint + +### POST `/api/predict/with-ndvi` + +**Request Body:** +```json +{ + "model_filename": "model_xgboost_20231221_120000.joblib", + "min_lon": 105.6, + "min_lat": 9.3, + "max_lon": 106.2, + "max_lat": 9.8, + "start_date": "2023-03-01", + "end_date": "2023-05-31", + "max_scenes": 12, + "cloud_cover": 30, + "resolution": 20, + "export_ndvi": true, + "export_classification": true +} +``` + +**Response:** +```json +{ + "success": true, + "message": "Prediction with NDVI completed", + "output_files": [ + {"type": "ndvi", "path": "predictions/ndvi_20231221_120000.tif"}, + {"type": "classification", "path": "predictions/classification_20231221_120000.tif"} + ], + "ndvi_stats": { + "mean": 0.456, + "min": -0.123, + "max": 0.789, + "std": 0.234 + }, + "class_distribution": { + "0": 12345, + "1": 23456, + "2": 34567 + }, + "n_scenes": 12, + "resolution": 20, + "bbox": [105.6, 9.3, 106.2, 9.8] +} +``` + +## Download Files + +Sau khi prediction hoàn thành, có thể download files qua: +- **UI**: Click "💾 Download GeoTIFF" trong kết quả +- **API**: `GET /api/predictions/download/ndvi_YYYYMMDD_HHMMSS.tif` +- **API**: `GET /api/predictions/download/classification_YYYYMMDD_HHMMSS.tif` + +## Sử Dụng Kết Quả với Python + +```python +import rasterio +import matplotlib.pyplot as plt +import numpy as np + +# Read NDVI raster +with rasterio.open('predictions/ndvi_20231221_120000.tif') as src: + ndvi = src.read(1) + + # Visualize + plt.figure(figsize=(10, 8)) + plt.imshow(ndvi, cmap='RdYlGn', vmin=-1, vmax=1) + plt.colorbar(label='NDVI') + plt.title('NDVI Map') + plt.show() + +# Read classification raster +with rasterio.open('predictions/classification_20231221_120000.tif') as src: + classification = src.read(1) + + # Visualize + plt.figure(figsize=(10, 8)) + plt.imshow(classification, cmap='tab10') + plt.colorbar(label='Land Class') + plt.title('Land Classification') + plt.show() +``` + +## Sử Dụng Kết Quả với QGIS + +1. Mở QGIS +2. **Layer → Add Layer → Add Raster Layer** +3. Chọn file `ndvi_*.tif` hoặc `classification_*.tif` +4. Styling: + - NDVI: Singleband pseudocolor, min=-1, max=1, color ramp=RdYlGn + - Classification: Paletted/Unique values + +## Lưu Ý + +- **Thời gian xử lý**: Tùy thuộc vào kích thước bbox và số scenes (thường 2-5 phút) +- **Bộ nhớ**: Khu vực lớn + resolution cao = RAM cao +- **NDVI values**: + - < 0: Nước, đất trống + - 0 - 0.2: Đất có ít thực vật + - 0.2 - 0.5: Cây cỏ, cây trồng + - > 0.5: Rừng rậm, thực vật dày đặc + +## So Sánh với NDVI Time Series + +| Feature | Predict NDVI | NDVI Time Series | +|---------|-------------|------------------| +| **Mục đích** | Xuất raster NDVI + land classification | Xem xu hướng NDVI theo thời gian | +| **Output** | GeoTIFF files | Chart, CSV | +| **Dùng model** | Có (predict land class) | Không (chỉ tính NDVI) | +| **Visualize** | Bản đồ raster | Biểu đồ đường | +| **Use case** | Phân tích không gian | Phân tích thời gian | + +## Troubleshooting + +**Q: Lỗi "Model không tồn tại"?** +- Kiểm tra model đã được train và lưu trong `model_train/` +- Refresh danh sách model + +**Q: Kết quả NDVI toàn NaN?** +- Check cloud cover (giảm xuống) +- Mở rộng time range +- Kiểm tra bbox có nằm trong phạm vi Sentinel-2 coverage + +**Q: File GeoTIFF không mở được?** +- Đảm bảo file download hoàn chỉnh +- Dùng QGIS hoặc rasterio để kiểm tra + +**Q: Prediction chậm?** +- Giảm resolution (20m thay vì 10m) +- Giảm max_scenes +- Thu nhỏ bbox diff --git a/api_server.py b/api_server.py index f2c0f48..2850c4b 100644 --- a/api_server.py +++ b/api_server.py @@ -122,6 +122,31 @@ class TrainingStatus(BaseModel): end_time: Optional[str] +class NDVIConfig(BaseModel): + """Cấu hình tính NDVI time series""" + bbox: List[float] # [min_lon, min_lat, max_lon, max_lat] + start_date: str + end_date: str + max_cloud_cover: int = 30 + resolution: int = 20 + + +class PredictionWithNDVIConfig(BaseModel): + """Cấu hình predict kết hợp land classification và NDVI""" + model_filename: str + min_lon: float + min_lat: float + max_lon: float + max_lat: float + start_date: str + end_date: str + max_scenes: int = 12 + cloud_cover: int = 30 + resolution: int = 20 + export_ndvi: bool = True # Export NDVI raster + export_classification: bool = True # Export classification raster + + @app.get("/", response_class=HTMLResponse) async def root(): """Serve main index page with tabs""" @@ -173,6 +198,26 @@ async def dashboard(): raise HTTPException(status_code=404, detail="Dashboard không tồn tại") +@app.get("/batch", response_class=HTMLResponse) +async def batch_page(): + """Serve batch processing interface""" + html_file = Path(__file__).parent / "batch_interface.html" + if html_file.exists(): + return FileResponse(html_file) + else: + raise HTTPException(status_code=404, detail="Batch interface không tồn tại") + + +@app.get("/ndvi", response_class=HTMLResponse) +async def ndvi_page(): + """Serve NDVI time series interface""" + html_file = Path(__file__).parent / "ndvi_interface.html" + if html_file.exists(): + return FileResponse(html_file) + else: + raise HTTPException(status_code=404, detail="NDVI interface không tồn tại") + + @app.get("/api/config/presets") async def get_presets(): """Lấy các preset cấu hình sẵn""" @@ -398,14 +443,39 @@ async def list_reports(): else: report_type = "unknown" - reports.append({ + report_info = { "filename": report_file.name, "type": report_type, "created": datetime.fromtimestamp(report_file.stat().st_mtime).isoformat(), "size_kb": round(report_file.stat().st_size / 1024, 2), "view_url": f"/api/reports/view/{report_file.name}", - "download_url": f"/api/reports/download/{report_file.name}" - }) + "download_url": f"/api/reports/download/{report_file.name}", + "is_batch_job": False, + "batch_metadata": None + } + + # Check if this is a batch job report + if report_type == "prediction": + predictions_dir = Path("predictions") + # Look for batch metadata JSON files that reference this report + for json_file in predictions_dir.glob("batch_*.json"): + try: + import json + with open(json_file, 'r') as f: + metadata = json.load(f) + if metadata.get("report_filename") == report_file.name or \ + (metadata.get("batch_job_id") and report_file.name.endswith('.html')): + report_info["is_batch_job"] = True + report_info["batch_metadata"] = { + "batch_job_id": metadata.get("batch_job_id"), + "batch_name": metadata.get("batch_name"), + "batch_timestamp": metadata.get("batch_timestamp") + } + break + except Exception as e: + pass + + reports.append(report_info) # Sort by creation time (newest first) reports.sort(key=lambda x: x["created"], reverse=True) @@ -1025,12 +1095,37 @@ async def list_predictions(): predictions = [] for pred_file in predictions_dir.glob("*.tif"): - predictions.append({ + pred_info = { "filename": pred_file.name, "created": datetime.fromtimestamp(pred_file.stat().st_mtime).isoformat(), "size_mb": round(pred_file.stat().st_size / 1024 / 1024, 2), - "download_url": f"/api/predictions/download/{pred_file.name}" - }) + "download_url": f"/api/predictions/download/{pred_file.name}", + "is_batch_job": pred_file.name.startswith("batch_"), + "batch_metadata": None + } + + # Try to load batch metadata from JSON sidecar if exists + json_file = pred_file.with_suffix('.json') + if json_file.exists(): + try: + import json + with open(json_file, 'r') as f: + metadata = json.load(f) + pred_info["batch_metadata"] = { + "batch_job_id": metadata.get("batch_job_id"), + "batch_name": metadata.get("batch_name"), + "batch_timestamp": metadata.get("batch_timestamp") + } + except Exception as e: + print(f"[METADATA ERROR] Failed to load {json_file}: {e}") + + # Check PNG preview + png_file = pred_file.with_suffix('.png') + pred_info["has_preview"] = png_file.exists() + if png_file.exists(): + pred_info["preview_url"] = f"/api/predictions/preview/{png_file.name}" + + predictions.append(pred_info) # Sort by creation time (newest first) predictions.sort(key=lambda x: x["created"], reverse=True) @@ -1366,6 +1461,8 @@ async def process_batch_queue(): """Process batch prediction queue""" global batch_queue, batch_results + import asyncio + while batch_queue: # Get next job job = None @@ -1379,26 +1476,27 @@ async def process_batch_queue(): # Mark as running job["status"] = "running" + job["progress"] = 0 job["started_at"] = datetime.now().isoformat() try: # Create PredictionConfig from job config pred_config = PredictionConfig(**job["config"]) - # Run prediction (simplified version) - # In real implementation, call the actual prediction function - print(f"[BATCH] Processing job: {job['name']}") + print(f"[BATCH] Processing job {job['job_id']}: {job['name']}") + job["progress"] = 5 - # Simulate prediction (replace with actual prediction call) - # await run_prediction(pred_config) + # Run prediction synchronously (in the same thread to avoid conflicts) + await asyncio.to_thread(run_batch_prediction, job, pred_config) - # For now, mark as completed - job["status"] = "completed" - job["completed_at"] = datetime.now().isoformat() - job["result"] = { - "output_file": f"predictions/batch_{job['job_id']}.tif", - "message": "Prediction completed successfully" - } + # Check if prediction was successful + if job.get("result") and not job.get("error"): + job["status"] = "completed" + job["progress"] = 100 + job["completed_at"] = datetime.now().isoformat() + print(f"[BATCH] Job {job['job_id']} completed successfully") + else: + raise Exception(job.get("error", "Unknown error during prediction")) except Exception as e: job["error"] = str(e) @@ -1407,12 +1505,14 @@ async def process_batch_queue(): if job["retries"] < job["max_retries"]: job["retries"] += 1 job["status"] = "queued" # Retry - print(f"[BATCH] Job {job['name']} failed, retrying ({job['retries']}/{job['max_retries']})") + job["progress"] = 0 + print(f"[BATCH] Job {job['job_id']} ({job['name']}) failed, retrying ({job['retries']}/{job['max_retries']}): {e}") continue else: job["status"] = "failed" + job["progress"] = 0 job["completed_at"] = datetime.now().isoformat() - print(f"[BATCH] Job {job['name']} failed permanently: {e}") + print(f"[BATCH] Job {job['job_id']} ({job['name']}) failed permanently: {e}") # Move to results batch_queue.remove(job) @@ -1423,6 +1523,540 @@ async def process_batch_queue(): batch_results = batch_results[-100:] +def run_batch_prediction(job: dict, config: PredictionConfig): + """Run prediction for a single batch job""" + try: + job["progress"] = 10 + + # Import required libraries + import xarray as xr + import numpy as np + from datetime import datetime as dt + import rioxarray + import dask.array as da + + job["progress"] = 15 + + # Load model + model_path = Path("model_train") / config.model_filename + if not model_path.exists(): + raise FileNotFoundError(f"Model không tồn tại: {config.model_filename}") + + model_data = joblib.load(model_path) + + if isinstance(model_data, dict): + model = model_data.get('model') + label_encoder = model_data.get('label_encoder') + else: + model = model_data + label_encoder = None + + job["progress"] = 20 + + # Check if CNN model + is_cnn_model = hasattr(model, '__class__') and 'CNN' in model.__class__.__name__ + + # Load data from Microsoft Planetary Computer + import pystac_client + import planetary_computer + from odc.stac import load + + catalog = pystac_client.Client.open( + "https://planetarycomputer.microsoft.com/api/stac/v1", + modifier=planetary_computer.sign_inplace, + ) + + bbox = [config.min_lon, config.min_lat, config.max_lon, config.max_lat] + time_range = f"{config.start_date}/{config.end_date}" + + job["progress"] = 25 + + # Search Sentinel-2 + s2_search = catalog.search( + collections=["sentinel-2-l2a"], + bbox=bbox, + datetime=time_range, + query={"eo:cloud_cover": {"lt": config.cloud_cover}} + ) + + s2_items = list(s2_search.items()) + if not s2_items: + raise ValueError("Không tìm thấy dữ liệu Sentinel-2") + + s2_items = s2_items[:config.max_scenes] + + job["progress"] = 35 + + # Load Sentinel-2 data + s2_data = load( + s2_items, + bbox=bbox, + chunks={"time": 1, "x": 2048, "y": 2048}, + groupby="solar_day", + resolution=config.resolution + ) + + job["progress"] = 50 + + # Calculate NDVI + nir = s2_data["B08"].astype('float32') + red = s2_data["B04"].astype('float32') + ndvi = (nir - red) / (nir + red + 1e-8) + + # Mask clouds if SCL available + if "SCL" in s2_data: + scl = s2_data["SCL"] + cloud_mask = (scl == 3) | (scl == 8) | (scl == 9) | (scl == 10) + ndvi = ndvi.where(~cloud_mask) + + # Fill NaN and resample + ndvi_filled = ndvi.ffill(dim='time').bfill(dim='time') + ndvi_monthly = ndvi_filled.resample(time="1ME").mean().compute() + + job["progress"] = 70 + + # Prepare features + n_times_ndvi = len(ndvi_monthly.time) + y_size = len(ndvi_monthly.y) + x_size = len(ndvi_monthly.x) + n_pixels = y_size * x_size + + ndvi_features = [] + for t in range(n_times_ndvi): + ndvi_t = ndvi_monthly.isel(time=t).values.flatten() + ndvi_features.append(ndvi_t) + + features = np.column_stack(ndvi_features) + features = np.nan_to_num(features, nan=0.0) + + job["progress"] = 80 + + # Adjust features to match model expectations + try: + if is_cnn_model: + expected_features = model.n_features + elif hasattr(model, 'n_features_in_'): + expected_features = model.n_features_in_ + else: + try: + expected_features = model.get_booster().num_features() + except: + expected_features = features.shape[1] + + if features.shape[1] > expected_features: + features = features[:, :expected_features] + elif features.shape[1] < expected_features: + n_missing = expected_features - features.shape[1] + padding = np.tile(features[:, -1:], (1, n_missing)) + features = np.column_stack([features, padding]) + except: + pass + + # Predict + if is_cnn_model: + predictions = model.predict(features) + else: + predictions = model.predict(features) + + # Decode labels + if label_encoder is not None: + try: + predictions = label_encoder.inverse_transform(predictions) + except: + pass + + job["progress"] = 90 + + # Reshape and create output + pred_shape = (y_size, x_size) + predictions_2d = predictions.reshape(pred_shape) + + prediction_da = xr.DataArray( + predictions_2d, + coords={"y": ndvi_monthly.y, "x": ndvi_monthly.x}, + dims=["y", "x"], + name="classification" + ) + + # Save output + output_dir = Path("predictions") + output_dir.mkdir(exist_ok=True) + + output_file = output_dir / f"batch_{job['job_id']}_{job['name'].replace(' ', '_')}.tif" + + if hasattr(s2_data, 'rio') and s2_data.rio.crs is not None: + prediction_da.rio.write_crs(s2_data.rio.crs, inplace=True) + else: + prediction_da.rio.write_crs("EPSG:4326", inplace=True) + + prediction_da.rio.to_raster(str(output_file), driver="GTiff") + + # Generate PNG preview + png_file = output_dir / f"batch_{job['job_id']}_{job['name'].replace(' ', '_')}.png" + try: + import matplotlib + matplotlib.use('Agg') + import matplotlib.pyplot as plt + + fig, ax = plt.subplots(figsize=(12, 10), dpi=150) + im = ax.imshow(predictions_2d, cmap='tab20', interpolation='nearest') + ax.set_title(f"{job['name']} - Batch {job['job_id']}", fontsize=14, fontweight='bold') + ax.set_xlabel('X (pixels)', fontsize=10) + ax.set_ylabel('Y (pixels)', fontsize=10) + + cbar = plt.colorbar(im, ax=ax, fraction=0.046, pad=0.04) + cbar.set_label('Class', rotation=270, labelpad=15) + ax.grid(True, alpha=0.3, linestyle='--', linewidth=0.5) + + plt.tight_layout() + plt.savefig(str(png_file), dpi=150, bbox_inches='tight') + plt.close(fig) + except Exception as e: + print(f"[BATCH PNG ERROR] {e}") + png_file = None + + # Get unique classes + unique_classes = np.unique(predictions_2d) + unique_classes = unique_classes[~np.isnan(unique_classes)].tolist() + + # Store result in job with batch metadata + job["result"] = { + "output_file": str(output_file), + "png_file": str(png_file) if png_file else None, + "shape": list(pred_shape), + "unique_classes": unique_classes, + "bbox": bbox, + "time_range": time_range, + "n_features": features.shape[1], + "n_times_ndvi": n_times_ndvi, + "model_used": config.model_filename, + "batch_job_id": job["job_id"], + "batch_name": job["name"], + "batch_timestamp": datetime.now().isoformat() + } + + # Save batch metadata to JSON sidecar file for persistence + metadata_file = output_file.with_suffix('.json') + try: + import json + with open(metadata_file, 'w') as f: + json.dump(job["result"], f, indent=2, default=str) + print(f"[BATCH METADATA] Saved to {metadata_file}") + except Exception as e: + print(f"[BATCH METADATA ERROR] Failed to save metadata: {e}") + + # Auto generate prediction report for batch job + try: + from report_generator import generate_prediction_report + report_path, _ = generate_prediction_report(job["result"]) + job["result"]["report_path"] = report_path + job["result"]["report_filename"] = Path(report_path).name + print(f"[BATCH REPORT] Generated prediction report: {report_path}") + except Exception as e: + print(f"[BATCH REPORT ERROR] Failed to generate report: {e}") + + job["progress"] = 100 + + except Exception as e: + job["error"] = str(e) + import traceback + print(f"[BATCH ERROR] Job {job['job_id']}: {traceback.format_exc()}") + + +# ============ PREDICTION WITH NDVI API ============ + +@app.post("/api/predict/with-ndvi") +async def predict_with_ndvi(config: PredictionWithNDVIConfig, background_tasks: BackgroundTasks): + """Predict land classification và NDVI cho một khu vực""" + try: + import numpy as np + import xarray as xr + from pystac_client import Client + import planetary_computer + import odc.stac + import rasterio + from rasterio.transform import from_bounds + + # Load model + model_path = Path(f"model_train/{config.model_filename}") + if not model_path.exists(): + raise HTTPException(status_code=404, detail=f"Model {config.model_filename} không tồn tại") + + model = joblib.load(model_path) + print(f"[PREDICT+NDVI] Loaded model: {config.model_filename}") + + # Connect to Microsoft Planetary Computer + catalog = Client.open( + "https://planetarycomputer.microsoft.com/api/stac/v1", + modifier=planetary_computer.sign_inplace + ) + + bbox = [config.min_lon, config.min_lat, config.max_lon, config.max_lat] + time_range = f"{config.start_date}/{config.end_date}" + + # Search for Sentinel-2 data + search = catalog.search( + collections=["sentinel-2-l2a"], + bbox=bbox, + datetime=time_range, + query={"eo:cloud_cover": {"lt": config.cloud_cover}} + ) + + items = list(search.items())[:config.max_scenes] + print(f"[PREDICT+NDVI] Found {len(items)} Sentinel-2 scenes") + + if len(items) == 0: + raise HTTPException(status_code=404, detail="Không tìm thấy dữ liệu vệ tinh") + + # Load all bands needed for features + data = odc.stac.load( + items, + bbox=bbox, + bands=["B02", "B03", "B04", "B08"], # Blue, Green, Red, NIR + resolution=config.resolution, + chunks={"x": 2048, "y": 2048} + ).compute() + + print(f"[PREDICT+NDVI] Loaded data shape: {data.dims}") + + # Calculate NDVI and other indices + blue = data["B02"].values + green = data["B03"].values + red = data["B04"].values + nir = data["B08"].values + + # Calculate indices + # NDVI = (NIR - Red) / (NIR + Red) + ndvi = (nir - red) / (nir + red + 1e-8) + + # NDWI = (Green - NIR) / (Green + NIR) + ndwi = (green - nir) / (green + nir + 1e-8) + + # NDBI = (SWIR - NIR) / (SWIR + NIR) - we use Red as proxy + ndbi = (red - nir) / (red + nir + 1e-8) + + # Prepare features for prediction + # Assuming model was trained with [NDVI, NDWI, NDBI] features + height, width = ndvi.shape[1:3] # Skip time dimension + n_pixels = height * width + + # Average over time dimension + ndvi_mean = np.nanmean(ndvi, axis=0) + ndwi_mean = np.nanmean(ndwi, axis=0) + ndbi_mean = np.nanmean(ndbi, axis=0) + + # Reshape for prediction + features = np.stack([ndvi_mean.flatten(), ndwi_mean.flatten(), ndbi_mean.flatten()], axis=1) + + # Handle NaN values + valid_mask = ~np.isnan(features).any(axis=1) + features_clean = features[valid_mask] + + print(f"[PREDICT+NDVI] Predicting {features_clean.shape[0]} valid pixels...") + + # Predict + predictions = model.predict(features_clean) + + # Reshape back to raster + prediction_raster = np.full(n_pixels, -1, dtype=np.int16) + prediction_raster[valid_mask] = predictions + prediction_raster = prediction_raster.reshape(height, width) + + # Prepare outputs + timestamp = datetime.now().strftime('%Y%m%d_%H%M%S') + output_dir = Path("predictions") + output_dir.mkdir(exist_ok=True) + + output_files = [] + + # Export NDVI if requested + if config.export_ndvi: + ndvi_file = output_dir / f"ndvi_{timestamp}.tif" + transform = from_bounds(bbox[0], bbox[1], bbox[2], bbox[3], width, height) + + with rasterio.open( + ndvi_file, 'w', + driver='GTiff', + height=height, + width=width, + count=1, + dtype=ndvi_mean.dtype, + crs='EPSG:4326', + transform=transform + ) as dst: + dst.write(ndvi_mean, 1) + + output_files.append({"type": "ndvi", "path": str(ndvi_file)}) + print(f"[PREDICT+NDVI] Saved NDVI to {ndvi_file}") + + # Export classification if requested + if config.export_classification: + class_file = output_dir / f"classification_{timestamp}.tif" + transform = from_bounds(bbox[0], bbox[1], bbox[2], bbox[3], width, height) + + with rasterio.open( + class_file, 'w', + driver='GTiff', + height=height, + width=width, + count=1, + dtype=prediction_raster.dtype, + crs='EPSG:4326', + transform=transform + ) as dst: + dst.write(prediction_raster, 1) + + output_files.append({"type": "classification", "path": str(class_file)}) + print(f"[PREDICT+NDVI] Saved classification to {class_file}") + + # Calculate statistics + ndvi_stats = { + "mean": float(np.nanmean(ndvi_mean)), + "min": float(np.nanmin(ndvi_mean)), + "max": float(np.nanmax(ndvi_mean)), + "std": float(np.nanstd(ndvi_mean)) + } + + # Count classes + unique_classes, counts = np.unique(predictions, return_counts=True) + class_distribution = { + int(cls): int(count) for cls, count in zip(unique_classes, counts) + } + + return { + "success": True, + "message": "Prediction with NDVI completed", + "output_files": output_files, + "ndvi_stats": ndvi_stats, + "class_distribution": class_distribution, + "n_scenes": len(items), + "resolution": config.resolution, + "bbox": bbox + } + + except Exception as e: + print(f"[PREDICT+NDVI ERROR] {str(e)}") + import traceback + traceback.print_exc() + raise HTTPException(status_code=500, detail=str(e)) + + +# ============ NDVI TIME SERIES API ============ + +@app.post("/api/ndvi/timeseries") +async def calculate_ndvi_timeseries(config: NDVIConfig): + """Tính NDVI time series cho một khu vực""" + try: + import numpy as np + import xarray as xr + from pystac_client import Client + import planetary_computer + import odc.stac + + print(f"[NDVI] Starting calculation for bbox: {config.bbox}, time: {config.start_date} to {config.end_date}") + + # Connect to Microsoft Planetary Computer STAC API + catalog = Client.open( + "https://planetarycomputer.microsoft.com/api/stac/v1", + modifier=planetary_computer.sign_inplace + ) + + bbox = config.bbox + time_range = f"{config.start_date}/{config.end_date}" + + # Search for Sentinel-2 data + search = catalog.search( + collections=["sentinel-2-l2a"], + bbox=bbox, + datetime=time_range, + query={"eo:cloud_cover": {"lt": config.max_cloud_cover}} + ) + + items = list(search.items()) + print(f"[NDVI] Found {len(items)} Sentinel-2 scenes") + + if len(items) == 0: + raise HTTPException(status_code=404, detail="Không tìm thấy dữ liệu Sentinel-2 cho khu vực và thời gian này") + + # Load data for each time step + ndvi_timeseries = [] + + for item in items: + try: + # Load NIR (B08) and Red (B04) bands + data = odc.stac.load( + [item], + bbox=bbox, + bands=["B04", "B08"], # Red and NIR + resolution=config.resolution, + chunks={"x": 2048, "y": 2048} + ).compute() + + if data is None or len(data.keys()) == 0: + continue + + # Calculate NDVI = (NIR - Red) / (NIR + Red) + nir = data["B08"].values + red = data["B04"].values + + # Avoid division by zero + denominator = nir + red + denominator = np.where(denominator == 0, np.nan, denominator) + + ndvi = (nir - red) / denominator + + # Calculate mean NDVI (ignore NaN values) + mean_ndvi = float(np.nanmean(ndvi)) + + # Get date from item + date_str = item.datetime.strftime("%Y-%m-%d") + + ndvi_timeseries.append({ + "date": date_str, + "ndvi": mean_ndvi + }) + + print(f"[NDVI] {date_str}: NDVI = {mean_ndvi:.3f}") + + except Exception as e: + print(f"[NDVI WARNING] Failed to process item {item.id}: {e}") + continue + + if len(ndvi_timeseries) == 0: + raise HTTPException(status_code=500, detail="Không thể tính NDVI cho bất kỳ ảnh nào") + + # Sort by date + ndvi_timeseries.sort(key=lambda x: x["date"]) + + # Calculate statistics + ndvi_values = [item["ndvi"] for item in ndvi_timeseries] + mean_ndvi = float(np.mean(ndvi_values)) + min_ndvi = float(np.min(ndvi_values)) + max_ndvi = float(np.max(ndvi_values)) + + result = { + "timeseries": ndvi_timeseries, + "n_images": len(ndvi_timeseries), + "mean_ndvi": mean_ndvi, + "min_ndvi": min_ndvi, + "max_ndvi": max_ndvi, + "bbox": bbox, + "time_range": time_range + } + + print(f"[NDVI] Calculation complete. Mean NDVI: {mean_ndvi:.3f}, Images: {len(ndvi_timeseries)}") + + return result + + except HTTPException: + raise + except Exception as e: + print(f"[NDVI ERROR] {e}") + import traceback + traceback.print_exc() + raise HTTPException(status_code=500, detail=f"Lỗi khi tính NDVI: {str(e)}") + + if __name__ == "__main__": print("=" * 70) print("🚀 LAND CLASSIFICATION TRAINING API SERVER") diff --git a/batch_interface.html b/batch_interface.html new file mode 100644 index 0000000..ec9e140 --- /dev/null +++ b/batch_interface.html @@ -0,0 +1,788 @@ + + + + + + Batch Processing - Land Classification + + + + + + + + +
+
+

🚀 Batch Processing

+

Xử lý nhiều khu vực cùng lúc với model đã train

+
+ +
+ 🏠 Trang Chủ + 🎓 Training + 🗺️ Prediction + 🚀 Batch Processing (Active) + 🌿 NDVI Analysis +
+ +
+ +
+

⚙️ Cấu hình Batch

+ +
+ + +
+ +
+ + +
+ +
+ +
+ + + + +
+
+ +
+ +
+ + +
+
+ + + + +
+
+ + +
+

📋 Batch Queue (0 items)

+ +
+

Chưa có item nào. Thêm khu vực từ bên trái.

+
+ +
+ + +
+
+ + +
+

📊 Batch Status

+ + + +
+ +
+
+ + +
+

✅ Completed Results

+ +
+ + +
+ +
+

Chưa có kết quả nào

+
+
+
+
+ + + + + + + + + + + diff --git a/index.html b/index.html index 14edbcf..f4cc1c8 100644 --- a/index.html +++ b/index.html @@ -427,28 +427,12 @@ -
- - - - - - - +
+ 🏠 Trang Chủ (Active) + 🎓 Training + 🗺️ Prediction + 🚀 Batch Processing + 🌿 NDVI Analysis
@@ -810,8 +794,30 @@ const row = document.createElement('tr'); const typeIcon = report.type === 'training' ? '🎓' : '🗺️'; + // Check if this is a batch job + const isBatchJob = report.is_batch_job; + const batchInfo = report.batch_metadata; + + let batchLabel = ''; + if (isBatchJob && batchInfo) { + const timestamp = batchInfo.batch_timestamp ? + new Date(batchInfo.batch_timestamp).toLocaleString('vi-VN') : + 'N/A'; + batchLabel = ` +
+ 🚀 Batch: ${batchInfo.batch_name || 'N/A'} | + ID: ${batchInfo.batch_job_id || 'N/A'}
+ Thời gian: ${timestamp} +
+ `; + } + row.innerHTML = ` - ${report.filename} + + ${report.filename} + ${isBatchJob ? 'BATCH' : ''} + ${batchLabel} + ${typeIcon} ${report.type} ${new Date(report.created).toLocaleString('vi-VN')} ${report.size_kb} KB @@ -826,6 +832,9 @@ `; + if (isBatchJob) { + row.style.borderLeft = '4px solid #ffc107'; + } tbody.appendChild(row); }); diff --git a/ndvi_interface.html b/ndvi_interface.html new file mode 100644 index 0000000..e77f16e --- /dev/null +++ b/ndvi_interface.html @@ -0,0 +1,568 @@ + + + + + + NDVI Time Series Analysis + + + + + + + + +
+
+

🌿 NDVI Time Series Analysis

+

Phân tích chỉ số thực vật theo thời gian từ dữ liệu Sentinel-2

+
+ +
+ 🏠 Trang Chủ + 🎓 Training + 🗺️ Prediction + 🚀 Batch Processing + 🌿 NDVI Analysis (Active) +
+ +
+ +
+

⚙️ Cấu hình phân tích

+ +
+ +
+ + + + +
+
+ +
+ +
+ + +
+
+ +
+ + +
+ +
+ + +
+ +
+ 💡 Lưu ý: NDVI = (NIR - Red) / (NIR + Red)
+ Giá trị từ -1 đến 1. Giá trị cao = thực vật xanh tốt. +
+ + + + +
+
+ + +
+

📈 Kết quả phân tích

+ + + + + + +
+ + + +
+
+ + + + + + + + + diff --git a/prediction_interface.html b/prediction_interface.html index c57a847..f39197f 100644 --- a/prediction_interface.html +++ b/prediction_interface.html @@ -292,7 +292,26 @@

Phân loại đất cho khu vực mới sử dụng model đã train

-
+
+ 🏠 Trang Chủ + 🎓 Training + 🗺️ Prediction (Active) + 🚀 Batch Processing + 🌿 NDVI Analysis +
+ + +
+ + +
+ + +
@@ -362,8 +381,18 @@
-
@@ -417,11 +446,103 @@
+ + + + diff --git a/reports/prediction_report_20251221_181414.html b/reports/prediction_report_20251221_181414.html new file mode 100644 index 0000000..7a52282 --- /dev/null +++ b/reports/prediction_report_20251221_181414.html @@ -0,0 +1,176 @@ + + + + + + + Prediction Report - 20251221_181414 + + + +
+
+

🗺️ Báo Cáo Dự Đoán

+

Land Classification Prediction - 21/12/2025 18:14:14

+
+ +
+
+

📈 Tóm Tắt Kết Quả

+
+
+
1,222,118
+
Tổng số Pixels
+
+
+
1109x1102
+
Kích thước (px)
+
+
+
418.9
+
Diện tích (km²)
+
+
+
2
+
Số Classes
+
+
+
3
+
Số Features
+
+
+
❌
+
Sử dụng Radar
+
+
+
+ +
+

⚙️ Thông Tin Chi Tiết

+
+
+ 🤖 Model sử dụng: + model_xgboost_20251221_172351.joblib +
+
+ 📍 Khu vực (bbox): + [105.6, 9.3, 105.8, 9.5] +
+
+ 📅 Thời gian: + 2023-03-01/2023-05-31 +
+
+ 💾 Output file: + predictions/batch_20251221_181400_0_Region_1.tif +
+
+
+ +
+

🏷️ Các Classes Phát Hiện

+
+ 36 +
+
+
+ +
+

🌍 Land Classification System | Generated: 21/12/2025 18:14:14

+
+
+ + diff --git a/reports/prediction_report_20251221_181430.html b/reports/prediction_report_20251221_181430.html new file mode 100644 index 0000000..edf5694 --- /dev/null +++ b/reports/prediction_report_20251221_181430.html @@ -0,0 +1,176 @@ + + + + + + + Prediction Report - 20251221_181430 + + + +
+
+

🗺️ Báo Cáo Dự Đoán

+

Land Classification Prediction - 21/12/2025 18:14:30

+
+ +
+
+

📈 Tóm Tắt Kết Quả

+
+
+
1,223,220
+
Tổng số Pixels
+
+
+
1110x1102
+
Kích thước (px)
+
+
+
418.9
+
Diện tích (km²)
+
+
+
2
+
Số Classes
+
+
+
3
+
Số Features
+
+
+
❌
+
Sử dụng Radar
+
+
+
+ +
+

⚙️ Thông Tin Chi Tiết

+
+
+ 🤖 Model sử dụng: + model_xgboost_20251221_172351.joblib +
+
+ 📍 Khu vực (bbox): + [105.8, 9.3, 106.0, 9.5] +
+
+ 📅 Thời gian: + 2023-03-01/2023-05-31 +
+
+ 💾 Output file: + predictions/batch_20251221_181400_1_Region_2.tif +
+
+
+ +
+

🏷️ Các Classes Phát Hiện

+
+ 36 +
+
+
+ +
+

🌍 Land Classification System | Generated: 21/12/2025 18:14:30

+
+
+ + diff --git a/reports/prediction_report_20251221_181438.html b/reports/prediction_report_20251221_181438.html new file mode 100644 index 0000000..83a70f9 --- /dev/null +++ b/reports/prediction_report_20251221_181438.html @@ -0,0 +1,176 @@ + + + + + + + Prediction Report - 20251221_181438 + + + +
+
+

🗺️ Báo Cáo Dự Đoán

+

Land Classification Prediction - 21/12/2025 18:14:38

+
+ +
+
+

📈 Tóm Tắt Kết Quả

+
+
+
1,223,220
+
Tổng số Pixels
+
+
+
1110x1102
+
Kích thước (px)
+
+
+
418.9
+
Diện tích (km²)
+
+
+
2
+
Số Classes
+
+
+
3
+
Số Features
+
+
+
❌
+
Sử dụng Radar
+
+
+
+ +
+

⚙️ Thông Tin Chi Tiết

+
+
+ 🤖 Model sử dụng: + model_xgboost_20251221_172351.joblib +
+
+ 📍 Khu vực (bbox): + [106.0, 9.3, 106.2, 9.5] +
+
+ 📅 Thời gian: + 2023-03-01/2023-05-31 +
+
+ 💾 Output file: + predictions/batch_20251221_181400_2_Region_3.tif +
+
+
+ +
+

🏷️ Các Classes Phát Hiện

+
+ 36 +
+
+
+ +
+

🌍 Land Classification System | Generated: 21/12/2025 18:14:38

+
+
+ + diff --git a/reports/prediction_report_20251221_181454.html b/reports/prediction_report_20251221_181454.html new file mode 100644 index 0000000..f6ec6e2 --- /dev/null +++ b/reports/prediction_report_20251221_181454.html @@ -0,0 +1,176 @@ + + + + + + + Prediction Report - 20251221_181454 + + + +
+
+

🗺️ Báo Cáo Dự Đoán

+

Land Classification Prediction - 21/12/2025 18:14:54

+
+ +
+
+

📈 Tóm Tắt Kết Quả

+
+
+
1,221,009
+
Tổng số Pixels
+
+
+
1109x1101
+
Kích thước (px)
+
+
+
418.9
+
Diện tích (km²)
+
+
+
2
+
Số Classes
+
+
+
3
+
Số Features
+
+
+
❌
+
Sử dụng Radar
+
+
+
+ +
+

⚙️ Thông Tin Chi Tiết

+
+
+ 🤖 Model sử dụng: + model_xgboost_20251221_172351.joblib +
+
+ 📍 Khu vực (bbox): + [105.6, 9.5, 105.8, 9.7] +
+
+ 📅 Thời gian: + 2023-03-01/2023-05-31 +
+
+ 💾 Output file: + predictions/batch_20251221_181400_3_Region_4.tif +
+
+
+ +
+

🏷️ Các Classes Phát Hiện

+
+ 36 +
+
+
+ +
+

🌍 Land Classification System | Generated: 21/12/2025 18:14:54

+
+
+ + diff --git a/reports/prediction_report_20251221_181504.html b/reports/prediction_report_20251221_181504.html new file mode 100644 index 0000000..34f499f --- /dev/null +++ b/reports/prediction_report_20251221_181504.html @@ -0,0 +1,176 @@ + + + + + + + Prediction Report - 20251221_181504 + + + +
+
+

🗺️ Báo Cáo Dự Đoán

+

Land Classification Prediction - 21/12/2025 18:15:04

+
+ +
+
+

📈 Tóm Tắt Kết Quả

+
+
+
1,223,220
+
Tổng số Pixels
+
+
+
1110x1102
+
Kích thước (px)
+
+
+
418.9
+
Diện tích (km²)
+
+
+
2
+
Số Classes
+
+
+
3
+
Số Features
+
+
+
❌
+
Sử dụng Radar
+
+
+
+ +
+

⚙️ Thông Tin Chi Tiết

+
+
+ 🤖 Model sử dụng: + model_xgboost_20251221_172351.joblib +
+
+ 📍 Khu vực (bbox): + [105.8, 9.5, 106.0, 9.7] +
+
+ 📅 Thời gian: + 2023-03-01/2023-05-31 +
+
+ 💾 Output file: + predictions/batch_20251221_181400_4_Region_5.tif +
+
+
+ +
+

🏷️ Các Classes Phát Hiện

+
+ 36 +
+
+
+ +
+

🌍 Land Classification System | Generated: 21/12/2025 18:15:04

+
+
+ + diff --git a/training_interface.html b/training_interface.html index 01d0fad..d623bda 100644 --- a/training_interface.html +++ b/training_interface.html @@ -290,6 +290,14 @@

Giao diện training model phân loại đất từ ảnh vệ tinh

+
+ 🏠 Trang Chủ + 🎓 Training (Active) + 🗺️ Prediction + 🚀 Batch Processing + 🌿 NDVI Analysis +
+