Abstract:Real-world black-box optimization often involves time-consuming or costly experiments and simulations. Multi-fidelity optimization (MFO) stands out as a cost-effective strategy that balances high-fidelity accuracy with computational efficiency through a hierarchical fidelity approach. This survey presents a systematic exploration of MFO, underpinned by a novel text mining framework based on a pre-trained language model. We delve deep into the foundational principles and methodologies of MFO, focusing on three core components -- multi-fidelity surrogate models, fidelity management strategies, and optimization techniques. Additionally, this survey highlights the diverse applications of MFO across several key domains, including machine learning, engineering design optimization, and scientific discovery, showcasing the adaptability and effectiveness of MFO in tackling complex computational challenges. Furthermore, we also envision several emerging challenges and prospects in the MFO landscape, spanning scalability, the composition of lower fidelities, and the integration of human-in-the-loop approaches at the algorithmic level. We also address critical issues related to benchmarking and the advancement of open science within the MFO community. Overall, this survey aims to catalyze further research and foster collaborations in MFO, setting the stage for future innovations and breakthroughs in the field.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that in practical applications, black - box optimization usually involves time - consuming or costly experiments and simulations. Multi - fidelity optimization (MFO), as a cost - effective strategy, strikes a balance between high - fidelity accuracy and computational efficiency through a hierarchical fidelity approach. The paper aims to provide a comprehensive review through a systematic exploration of MFO, covering its basic principles, methodologies, and applications in different fields, such as machine learning, engineering design optimization, and scientific research, etc. In addition, the paper also discusses the challenges faced by the MFO field and future development directions, including scalability, low - fidelity combinations, and human - machine interaction methods at the algorithm level, etc.
Specifically, the paper focuses on the following aspects:
1. **Multi - fidelity surrogate models**: Explore how to construct surrogate models from multi - fidelity data, including single - model methods, space - mapping methods, correction - based methods, autoregressive methods, multi - task Gaussian process methods, etc.
2. **Fidelity management strategies**: Research how to transition between different fidelity levels to optimize computational efficiency.
3. **Optimization techniques**: Introduce various optimization techniques in multi - fidelity optimization, such as Bayesian optimization, assisted evolutionary optimization, etc.
4. **Application fields**: Demonstrate application examples of MFO in fields such as machine learning, engineering design optimization, and scientific research.
5. **Challenges and prospects**: Discuss the current challenges and future research directions in the MFO field, including scalability, combination of low - fidelity data, integration of human - machine interaction methods, etc.
Through the exploration of these aspects, the paper aims to promote further research and cooperation in the MFO field and lay the foundation for future innovation and breakthroughs.