Development model of remote sensing data management for long-term storage using big data

Abstract

The growing volume of remote sensing data requires efficient approaches for long-term storage and data management. The National Remote Sensing Data Bank (BDPJN) stores multi-resolution satellite imagery acquired since 1993 to support disaster monitoring, climate studies, natural resource management, and other Earth observation applications. However, managing continuously growing archives remains challenging due to manual migration processes, inefficient storage utilization, and limited scalability. This study proposes an integrated remote sensing data management model that combines hierarchical storage management (HSM) with Apache Spark based parallel processing to support policy-driven data migration and long term storage optimization. The proposed model enables data migration across multiple storage tiers based on data characteristics and storage policies while improving migration efficiency through distributed processing. Experimental evaluation was conducted using Landsat-8, SPOT 6, SPOT-7, and Pleiades datasets under both 1 Gbps and 10 Gbps network environments. The results demonstrate migration performance improvements ranging from 80.14% to 91.04% compared with the existing manual migration system. These findings indicate that integrating HSM with Spark-based parallel processing can reduce migration bottlenecks, improve storage utilization, and provide a scalable and cost-effective approach for managing large-scale remote sensing archives while supporting long-term data governance and accessibility.