Advanced processing of sensing big data : a multicore architecture-Based

Abstract

Rising communication speeds, increased processing power, and the widespread use of hardware and software sensors have all contributed to our capacity for information creation rapidly expanding in recent years. This data is frequently available in the form of continuous streaming data, and the capacity to collect and analyze it in order to obtain insights and find trends represents tremendous potential for many enterprises and scientific applications. Recently, there has been an increase in activity in the academic world about Query Processing Over Data Stream (QPODS). From the development of computationally efficient algorithms to the design of programming and real-time systems to enable their execution, there are many difficulties to overcome while creating QPODS applications. Two major problems will be addressed in this thesis: Continuous Skyline Processing Over Data Stream: SPODS applications are long-running (24 hr/7 d), so they are exposed to changes in arrival rates and workload properties. The ability of applications to process incoming data in real time is crucial so that they can effectively manage the dynamic workload and provide a cost-effective overall performance. The necessity of high-performance skyline queries: When dealing with SPODS challenges, having both high throughput and low latency is essential. Software needs to efficiently utilize parallel hardware like multi-core processors. There is a lack of dynamic strategies with well-known properties of real-time processing, no delay response, and energy efficient performance in the current approaches to the development of SPODS applications, and there is inefficient exploitation of the parallelism of existing multicore architectures. This study makes an attempt to address these gaps by applying established methods like parallel programming and QPoDS. When it comes to data streams, sliding windowed queries are among the most common types. The most recent data that has been received is processed. The content and size of windows can change over time because they are dynamic data structures. Since various windowing methods (time- or count-based) have specific requirements in terms of data distribution and management policies, the SPODS domain necessitates particular expertise and enhanced features relative to conventional parallel techniques. The time and effort required for parallel programming can be minimized with the help of a well-organized strategy (ex: openMP API). It also makes it easier to understand the relationship between a parallel solution’s throughput, latency, and other performance metrics. While numerous articles have been published regarding the parallelization of regular skyline queries, there is a limited amount of research dedicated specifically to the parallel processing of continuous skyline queries. This study introduces a Parallel Range Search Skyline (PRSS), a continuous skyline technique for multicore processors specifically designed for sliding windowbased high dimensional data streams. The efficacy of the proposed parallel implementation is demonstrated through tests conducted on both real-world and synthetic datasets, encompassing various point distributions, arrival rates, and window widths. The experimental results for a dataset characterized by a large number of dimensions and cardinality demonstrate significant acceleration.