Multi-Objective Optimization Algorithm in Big Data Environment
- Resource Type
- Conference
- Authors
- Jie, Zhao
- Source
- 2023 IEEE 6th International Conference on Information Systems and Computer Aided Education (ICISCAE) Information Systems and Computer Aided Education (ICISCAE), 2023 IEEE 6th International Conference on. :968-975 Sep, 2023
- Subject
- Communication, Networking and Broadcast Technologies
Computing and Processing
Power, Energy and Industry Applications
Robotics and Control Systems
Software algorithms
Clustering algorithms
Big Data
Optical fiber communication
Hardware
System software
Sparks
computer
big data
algorithm
particle swarm
- Language
- ISSN
- 2770-663X
None of the basic system software in big data domain can identify and use such algorithms. So the study evaluates hardware compression the performance of the method in big data environment is of great significance. The algorithm takes into account both the completion time of the optimization task and the load equilibrium value of the heterogeneous fog cluster, and adopts a nonlinear decreasing method to update the inertial weight value. The effectiveness of the proposed algorithm is verified by MATLAB simulation. The experimental results show that the algorithm improves the convergence rate at the same time as obtaining more search space at the early stage. Based on the KAEzip experiment, the performance of the classic Spark benchmarking program was evaluated in Spark. Tests of KAEzip show that: 1) Hardware compression algorithm can effectively improve Spark performance, such as KAEzip has up to 13.8% compression performance advantage over snappy, up to 7% decompression advantage and up to 5.7% performance potential in the actual application scenario; 2) The mismatch between the data transfer rate of the disk and the performance of the hardware compression algorithm is an important factor that restricts the performance of the hardware compression algorithm.