| 摘要: |
| 贝叶斯优化是一种优化黑盒函数的技术, 高效的样本利用率使其在众多科学和工程领域中得到了广泛应用, 如深度模型调参、化合物设计、药物开发和材料设计等. 然而, 当输入空间维度较高时, 贝叶斯优化的性能会显著下降. 为了克服这一限制, 许多研究对贝叶斯优化方法进行了高维扩展. 为了深入剖析高维贝叶斯优化的研究方法, 根据不同工作的假设与特征将高维贝叶斯优化方法分为3类: 基于有效低维度假设的方法、基于加性假设的方法以及基于局部搜索的方法, 并对这些方法进行阐述和分析. 首先着重分析这3类方法的研究进展, 然后比较各类方法在贝叶斯优化应用中的优劣势, 最后总结当前阶段高维贝叶斯优化的主要研究趋势, 并对未来发展方向展开讨论. |
| 关键词: 高维贝叶斯优化 贝叶斯优化 黑盒优化 降维 变量选择 |
| DOI：10.13328/j.cnki.jos.007304 |
| 分类号: |
| 基金项目:国家自然科学基金(62192783, 62276128); 科技创新2030—“新一代人工智能”重大项目(2021ZD0113303); 中央高校基础研究基金(14380128) |
|
| Survey on High-dimensional Bayesian Optimization |
|
CHEN Quan-Lin1, CHEN Yi-Yu1, HUO Jing1, CAO Hong-Ye1, GAO Yang1, LI Dong2, HAO Jian-Ye2
|
|
1.State Key Laboratory for Novel Software Technology (Nanjing University), Nanjing 210023, China;2.Noah’s Ark Laboratory, Huawei Technologies Co. Ltd., Shenzhen 518129, China
|
| Abstract: |
| Bayesian optimization is a technique for optimizing black-box functions. Due to its high sample utilization efficiency, it is widely applied across various scientific and engineering fields, such as hyperparameters tuning of deep models, compound design, drug development, and material design. However, the performance of Bayesian optimization significantly deteriorates when the input space is of high dimensionality. To overcome this limitation, numerous studies carry out high-dimensional extensions on Bayesian optimization methods. To deeply analyze research methods of high-dimensional Bayesian optimization, this study categorizes these methods into three types based on assumptions and characteristics of different kinds of work: methods based on the effective low-dimensional hypothesis, methods based on additive assumptions, and methods based on local search. Then, this study elaborates on and analyzes these methods. This study first focuses on analyzing the research progress of these three types of methods. Then, the advantages and disadvantages of each method in the application of Bayesian optimization are compared. Finally, the main research trends in high-dimensional Bayesian optimization at the current stage are summarized, and future development directions are discussed. |
| Key words: high-dimensional Bayesian optimization Bayesian optimization black-box optimization dimensionality reduction variable selection |
