Tensorflow 的超参数调整 [英] Hyperparameter tune for Tensorflow

问题描述

我正在寻找直接用 Tensorflow(不是 Keras 或 Tflearn)编写的代码的超参数调整包.你能提点建议吗?

I am searching for a hyperparameter tune package for code written directly in Tensorflow (not Keras or Tflearn). Could you make some suggestion?

推荐答案

通常你不需要将你的超参数优化逻辑与优化模型结合起来(除非你的超参数优化逻辑是特定于你所使用的模型类型的)培训，在这种情况下，您需要告诉我们更多信息).有多种工具和包可用于该任务.这里是一篇关于这个主题的好论文，和此处是一篇带有示例的更实用的博客文章.

Usually you don't need to have your hyperparameter optimisation logic coupled with the optimised model (unless your hyperparemeter optimisation logic is specific to the kind of model that you are training, in which case you would need to tell us a bit more). There are several tools and packages available for the task. Here is a good paper on the topic, and here is a more practical blog post with examples.

• hyperopt 实现随机搜索和 parzen 估计器优化树.
• Scikit-Optimize 实现了其他一些功能，包括高斯过程贝叶斯优化.
• SigOpt 是一项针对超参数的便捷服务(付费，但为学生和研究人员提供免费套餐和额外津贴)优化.它基于 Yelp 的 MOE，它是开源的(虽然发布的版本似乎没有太多更新) 并且理论上可以单独使用，但需要付出一些额外的努力.
• Spearmint 也是一个常见的包，也是开源的，但不能免费用于商业目的(尽管你可以回退到 限制较少的旧版本).它看起来不错，但不是很活跃，并且可用版本与 Python 3 不兼容(即使已提交拉取请求以修复该问题).
• BayesOpt 似乎是贝叶斯优化的黄金标准，但它主要是 C++ 和 Python 接口看起来不是很清楚.

• hyperopt implements random search and tree of parzen estimators optimization.
• Scikit-Optimize implements a few others, including Gaussian process Bayesian optimization.
• SigOpt is a convenient service (paid, although with a free tier and extra allowance for students and researchers) for hyperparameter optimization. It is based upon Yelp's MOE, which is open source (although the published version doesn't seem to update much) and can, in theory, be used on its own, although it would take some additional effort.
• Spearmint is a commonly referred package too, also open source but not free for commercial purposes (although you can fall back to a less restrictive older version). It looks good, but not very active, and the available version is not compatible with Python 3 (even though pull requests have been submitted to fix that).
• BayesOpt seems to be the golden standard in Bayesian optimization, but it's mainly C++, and the Python interface doesn't look very documented.

在这些中，我只有真的(也就是说，有一个真正的问题)将 hyperopt 与 TensorFlow 一起使用，并且并没有花费太多精力.API 在某些方面有点奇怪，文档也不是很彻底，但它确实有效并且似乎正在积极开发中，可能会有更多优化算法和改编(例如，专门针对神经网络).但是，正如之前链接的博客文章中所建议的那样，Scikit-Optimize 可能同样出色，而且 SigOpt 看起来很容易使用(如果它适合您).

Out of these, I have only really (that is, with a real problem) used hyperopt with TensorFlow, and it didn't took too much effort. The API is a bit weird at some points and the documentation is not terribly thorough, but it does work and seems to be under active development, with more optimization algorithms and adaptations (e.g. specifically for neural networks) possibly coming. However, as suggested in the previously linked blog post, Scikit-Optimize is probably as good, and SigOpt looks quite easy to use if it fits you.

这篇关于Tensorflow 的超参数调整的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！