Towards Integrated and Efficient Scientific Sensor Data Processing: A Database Approach

In this work, we focus on managing scientific environmental data, which are measurement readings collected from wireless sensors. In environmental science applications, raw sensor data often need to be validated, interpolated, aligned and aggregated before they are used to construct meaningful result sets. Due to the lack of systems to integrate all the necessary processing steps, scientists often resort to multiple tools in reality to process the data, which can severely affect the efficiency of their work. In this paper, we propose a new data processing framework, HyperGrid, to address the problem. By following the way of DBMS, HyperGrid adopts a generic data model, a generic query processing and optimization framework, and offers an integrated environment to store, query, analyze and visualize scientific datasets. The experiments on real query set and data set show that the framework not only introduces little processing overhead, but also provides abundant opportunities to optimize the processing cost and thus significantly enhances the processing efficiency.