Big data classification based on improved parallel k-nearest neighbor

Abstract

In response to the rapid growth of many sorts of information, highway data has continued to evolve in the direction of big data in terms of scale, type, and structure, exhibiting characteristics of multi-source heterogeneous data. The k-nearest neighbor (KNN) join has received a lot of interest in recent years due to its wide range of applications. Processing KNN joins is time-consuming and inefficient due to the quadratic structure of the join method. As the number of applications dealing with vast amounts of data develops, KNN joins get more sophisticated. The authors seek to save money on computer resources by leveraging a large number of threads and multiprocessors. Six popular datasets are used to apply the method and evaluate the sequential and parallel performance of the KNN technique. These datasets are used to compare the sequential and parallel performance of the KNN method. When compared to a matching multi-core solution, the final implementation saves computing resources. It has been optimized to utilize as little RAM as possible, allowing it to manage high-resolution photo data without sacrificing efficiency. The authors will use the technique they presented using Spark Radoop. Our performance research validates the supplied method’s efficacy and scalability.