================================================================================
BATHYMETRIX-AI: ADVANCED SDB TOOLKIT & COASTAL DYNAMICS
================================================================================

Bathymetrix-AI is a professional QGIS research toolkit for high-precision Satellite-Derived Bathymetry (SDB). The toolkit integrates multispectral satellite imagery with ICESat-2 (ATL24) LiDAR bathymetry through a modular and adaptive Machine Learning framework. It automates the end-to-end SDB processing pipeline while maintaining full control over data quality, model selection, spatial refinement, uncertainty estimation, and scientific validation.


--------------------------------------------------------------------------------
SCIENTIFIC METHODOLOGY — CORE 5-PHASE SYSTEM (SDB SINGLE MASTERFLOW)
--------------------------------------------------------------------------------
The SDB Single Masterflow is the core SDB workflow of Bathymetrix-AI, providing a complete end-to-end processing pipeline for a single satellite scene: Phase 01 -> Phase 02 -> Phase 03 -> Phase 04 -> Phase 05

Phase 01: Advanced Pre-processing
Prepares the satellite imagery for bathymetric modeling by isolating the aquatic domain, reducing radiometric interference, and generating depth-sensitive spectral features:
- Sun-Glint Removal: Removes surface reflections using the Hedley model, with robust handling of NaN and Inf values for processing stability.
- Water Segmentation: Uses NDWI, MNDWI, and NWI together with adaptive thresholding to isolate the aquatic domain.
- Deep Water OSW Filtering: Identifies and removes deep-water areas unsuitable for optically derived bathymetry using an advanced multi-method engine:
(1) Automated Knee-Point Extinction [Recommended] detecting the physical optical extinction cutoff on NIR absorption.
(2) Turbidity-Invariant Log-Ratio Extinction utilizing NDTI to protect coastal waters from sediment/mud plumes.
(3) Multi-Otsu / GMM 3D Spectral Clustering for complex reefs and heterogeneous benthos.
(4) Connected-Component Topological Cleaning removing open ocean speckles.
(5) Manual Polygon ROI & Custom OSW Polygon.
- OSW Boundary Extraction: Exports the Optically Shallow Water boundary as a GeoPackage vector while preserving the source imagery CRS.
- Log-Ratio Features & Spectral Indices: Generates physics-based depth-sensitive features such as Blue/Green Log-Ratio together with additional spectral indices.

Phase 02: Robust Filtering
Improves the quality of the training data by identifying and removing noisy observations, outliers, and environmental artifacts:
- Noise Removal: Supports Linear RANSAC, Least-Squares (LS) Variance Fit, and Huber Variance Fit with seamless user bypass.
- Dynamic Diagnostic Plotting: Uses robust percentile-based visualization to produce clearer variance and trend plots while reducing the effect of extreme noise.

Phase 03: Global Auto-ML & Feature Analysis
Automatically evaluates multiple machine-learning approaches instead of relying on a single predefined algorithm:
- Feature Analysis: Evaluates feature correlation, redundancy, and importance using Pearson/Spearman correlation, Auto-RANSAC, and Auto-Random Forest.
- Algorithm Benchmarking: Evaluates 15+ machine-learning regressors (Random Forest, Gradient Boosting, XGBoost, LightGBM, CatBoost, SVR, MLP, etc.).
- Model Selection & Winner Stability: Evaluates models using 7 distinct selection strategies (Winner Stability via Monte Carlo Sensitivity Simulation, SDB Composite Score, Max R², Min RMSE, Min wMAPE, Min |Bias|, Min MAE) with auto-balanced metric weighting and custom syntax parsing.
- Hyperparameter Optimization: Supports Bayesian Optimization, Grid Search, and Random Search.
- Spatial Cross-Validation: Provides independent spatial block cross-validation for model evaluation.
- Ensemble Blending: Supports Standard Average, Median, Stacking, and Uncertainty-Weighted Pixel Fusion.
- Memory-Efficient Prediction: Uses chunk-based raster prediction to reduce memory usage when processing large images.
- Customization: Provides advanced control over model and optimization parameters for research and fine-tuning.

Phase 04: Adaptive Refinement
Corrects local errors that may remain after global machine-learning prediction:
- Decoupled Refinement Architecture: Independent controls for Depth Variance Correction (Datum Mean Shift) and Spatial Residual Error Modeling (KNN / Kriging Grid).
- Spatial Residual Correction: Analyzes the differences between predicted and observed depths and models their spatial behavior.
- Zero-Mean Centered Spatial Residuals: Uses robust residual processing and spatial weighting to reduce local depth bias.
- Spatial Residual Modeling: Supports robust KNN, Standard KNN, and Gaussian Process / Kriging.
- Spatial Cross-Validation: Provides independent spatial validation for residual modeling.
- Adaptive Re-training & Pixel Fusion: Combines model information with spatial error information to generate a refined bathymetric surface.
- IHO Standards Assessment: Evaluates results against IHO S-44 Order 1a/2 Total Vertical Uncertainty (TVU) standards.

Phase 05: Validation & Reporting
Provides independent scientific assessment of the final bathymetric result:
- Independent Accuracy Assessment: Evaluates the final model using unseen validation points.
- Performance Metrics: R² (Coefficient of Determination), RMSE (Root Mean Square Error), wMAPE (Weighted Mean Absolute Percentage Error).
- IHO S-44 Assessment: Evaluates compliance with applicable Order 1a/2 TVU criteria.
- Interactive Validation Dashboard: Generates HTML dashboards containing model leaderboards, validation metrics, diagnostic plots, and summary information.


--------------------------------------------------------------------------------
MASTERFLOWS & STANDALONE MODULES
--------------------------------------------------------------------------------
1. SDB Single Masterflow:
The standard end-to-end SDB workflow for a single satellite scene (One Scene -> Pre-processing -> Data Filtering -> AI Modeling -> Adaptive Refinement -> Scientific Validation -> One Final SDB Map).

2. SDB SpatioSpectral Masterflow:
Designed for multiple satellite scenes covering the same area. Evaluates independent SDB results per scene and applies multi-scene consensus strategies (Best Scene Selection, Weighted Median, Weighted Mean, or Median/Mean) to produce one final consolidated SDB map.

3. SDB SpatioTemporal Masterflow:
Multi-year SDB modeling integrating Year/Time directly into a Global Spatiotemporal AI model, followed by year-specific adaptive refinement to produce a consistent temporal series of SDB maps.

4. Coastal Dynamics Analysis (Module 06):
Uses yearly SDB maps to analyze long-term bathymetric trends, Net Bathymetric Change, Morphological Stability Index (MSI), Statistical Level of Detection (StatCD), Erosion and Accretion, Shoreline Movement, Volumetric Sediment Tracking, and Target ROI Analytics.

5. ICESat-2 Downloader:
Specialized tool for querying, filtering, and downloading ICESat-2 ATL24 bathymetry data directly from NSIDC.

6. Tidal Datum Converter:
Dedicated tool for converting and vertically aligning bathymetric observations between different tidal reference datums (e.g., MSL to Chart Datum) integrating NASA GSFC GOT4.10c automated global tidal modeling.


--------------------------------------------------------------------------------
PERFORMANCE METRICS
--------------------------------------------------------------------------------
- R² (Coefficient of Determination): Measures goodness of fit between predicted and observed depths.
- RMSE (Root Mean Square Error): Measures average vertical error in meters.
- wMAPE (Weighted Mean Absolute Percentage Error): Measures relative error across depth intervals.
- IHO S-44 Standards: Evaluates compliance with Order 1a and Order 2 TVU criteria.


--------------------------------------------------------------------------------
MASTERFLOW SELECTION GUIDE
--------------------------------------------------------------------------------
- SDB Single Masterflow: One Scene -> One SDB.
- SDB SpatioSpectral Masterflow: Multiple Scenes -> Evaluate / Select / Aggregate -> One SDB.
- SDB SpatioTemporal Masterflow: Multiple Years -> Global Temporal AI -> SDB for Each Year.
- Coastal Dynamics Analysis: Yearly SDB -> Change & Morphological Analysis.


--------------------------------------------------------------------------------
SCIENTIFIC REFERENCES
--------------------------------------------------------------------------------
- Stumpf et al. (2003): Determination of shallow water depth with high-resolution satellite imagery.
- Hedley et al. (2005): Simple and robust removal of sun glint for high-resolution imagery.
- Fischler & Bolles (1981): Random Sample Consensus (RANSAC) for model fitting.
- Zhang et al. (2021): Adaptive variance fitting for satellite bathymetry noise reduction.
- Alevizos (2020): Spatial residual refinement and error compensation in shallow coastal waters.
- Wheaton et al. (2010): Accounting for uncertainty in DEMs from repeat topographic surveys.


================================================================================
Developed by Mohamed Aly Nasef for scientific research, coastal mapping, bathymetric analysis, and hydrographic applications.
Code architecture, algorithms, and technical documentation significantly enhanced and optimized using Google Gemini AI.
================================================================================