在Heroku上运行Scipy [英] Running Scipy on Heroku

问题描述

我在Heroku上运行了Numpy和Matplotlib，我也试图安装Scipy。但是，Scipy需要安装BLAS [1]，这在Heroku平台上没有提供。在联系Heroku支持后，他们建议我将BLAS构建为一个静态库来部署，并设置必要的环境变量。

所以，我编译了libblas.a 64位Linux box，并按[2]中所述设置以下变量：

  $ heroku config 
 BLAS => .heroku / vendor / lib / libfblas.a 
 LD_LIBRARY_PATH => .heroku / vendor / lib 
 LIBRARY_PATH => .heroku / vendor / lib 
 PATH => bin：/ usr / local / bin：/ usr / bin：/ bin 
 PYTHONUNBUFFERED => true 
  

在我的requirements.txt中添加scipy == 0.10.1后，推送仍然失败。

 文件scipy / integrate / setup.py，第10行，在配置中
 
 blas_opt = get_info（'blas_opt'，notfound_action = 2）
 
文件/tmp/build_h5l5y31i49e8/lib/python2.7/site-packages/numpy/distutils/system_info.py，第311行，在get_info 
 
 return cl（）。get_info（notfound_action）
 
文件/tmp/build_h5l5y31i49e8/lib/python2.7/site-packages/numpy/distutils/system_info.py，第462行，在get_info 
 
中引发self.notfounderror（self.notfounderror .__ doc__）
 
 numpy.distutils.system_info.BlasNotFoundError：
 
 Blas（ http://www.netlib.org/blas/）库找不到。 
 
可以在
 
 numpy / distutils / site.cfg文件（部分[blas]）中指定搜索库的目录，也可以通过设置
 
 BLAS环境变量。 
  

似乎pip并不知道BLAS环境变量，所以我使用heroku run python：
$ b $ pre $ （venv）bash-3.2 $ heroku运行python
运行python附加到终端... import up ，run.1
Python 2.7.2（默认，2011年10月31日，16:22:04）
[linux gcc 4.4.3]
键入help，copyright， 信用或许可证以获取更多信息。
>>> import os
>>> os.system（'bash'）
〜$ echo $ BLAS
.heroku / vendor / lib / libfblas.a
〜$ ls .heroku / vendor / lib / libfblas.a
.heroku / vendor / lib / libfblas.a
〜$


看来精细。现在我不知道如何解决这个问题。

[1] http：/ /www.netlib.org/blas/
[2] http://www.scipy.org/ Installing_SciPy / Linux

解决方案

我设法通过将numpy和scipy作为bdist离线构建雪松堆栈然后修改heroku python buildpack将其直接解压到dyno的vendor / venv区域。您还可以使用buildpack来设置持续存在的环境变量。

Heroku尚未正式发布buildpack - 为更多thirdparty / heroku搜索'heroku buildpacks'信息。

我的python构建包的分支是这里：
https：//[e-mail]/wyn/heroku-buildpack-python.git

这些变化在bin / compile，其中我提供了两个新步骤，一个scipy / numpy步骤和一个openopt步骤。这两个步骤的脚本位于bin / steps / npscipy和bin / steps / openopt中。我还添加了一些变量bin / release。请注意，我假设通过setup.py文件进行安装，而不是通过requirements.txt方法进行安装（参见 https：// devcenter .heroku.com / articles / python-pip＃traditional_distributions ）。

我还下载了用于构建的blas / lapack / atlas / gfortran二进制文件numpy / scipy到dyno上，因为有c扩展需要链接到它们。建立一切离线和下载的原因是，pip安装numpy / scipy要求你有一个fortran编译器+相关的开发环境，这使我的slu too太大。

它似乎工作，slu size大小现在是35mb，缩放也似乎很快。除了一个numpy测试外，所有的scipy测试都通过了。

这对我来说还在进行中，但我认为我会分享。

>

I got Numpy and Matplotlib running on Heroku, and I'm trying to install Scipy as well. However, Scipy requires BLAS[1] to install, which is not presented on the Heroku platform. After contacting Heroku support, they suggested me to build BLAS as a static library to deploy, and setup the necessary environment variables.

So, I compiled libblas.a on a 64bit Linux box, and set the following variables as described in [2] :

$ heroku config
BLAS             => .heroku/vendor/lib/libfblas.a
LD_LIBRARY_PATH  => .heroku/vendor/lib
LIBRARY_PATH     => .heroku/vendor/lib
PATH             => bin:/usr/local/bin:/usr/bin:/bin
PYTHONUNBUFFERED => true

After adding scipy==0.10.1 in my requirements.txt, the push still fails.

     File "scipy/integrate/setup.py", line 10, in configuration

       blas_opt = get_info('blas_opt',notfound_action=2)

     File "/tmp/build_h5l5y31i49e8/lib/python2.7/site-packages/numpy/distutils/system_info.py", line 311, in get_info

       return cl().get_info(notfound_action)

     File "/tmp/build_h5l5y31i49e8/lib/python2.7/site-packages/numpy/distutils/system_info.py", line 462, in get_info

       raise self.notfounderror(self.notfounderror.__doc__)

   numpy.distutils.system_info.BlasNotFoundError:

       Blas (http://www.netlib.org/blas/) libraries not found.

       Directories to search for the libraries can be specified in the

       numpy/distutils/site.cfg file (section [blas]) or by setting

       the BLAS environment variable.

It seem that pip is not aware of the BLAS environment variable, so I check the environment using heroku run python:

(venv)bash-3.2$ heroku run python
Running python attached to terminal... import up, run.1
Python 2.7.2 (default, Oct 31 2011, 16:22:04) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system('bash')
~ $ echo $BLAS
.heroku/vendor/lib/libfblas.a
~ $ ls .heroku/vendor/lib/libfblas.a
.heroku/vendor/lib/libfblas.a
~ $ 

And it seems fine. Now I have no idea how to solve this.

[1] http://www.netlib.org/blas/ [2] http://www.scipy.org/Installing_SciPy/Linux

解决方案

I managed to get this working on the cedar stack by building numpy and scipy offline as bdists and then modifying the heroku python buildpack to unzip these onto the dyno's vendor/venv areas directly. You can also use the buildpack to set environment variables that persist.

Heroku haven't officially published buildpacks yet - search for 'heroku buildpacks' for more thirdparty/heroku ones and information.

My fork of the python build pack is here: https://[e-mail]/wyn/heroku-buildpack-python.git

The changes are in the bin/compile where I source two new steps, a scipy/numpy step and an openopt step. The scripts for the two steps are in bin/steps/npscipy and bin/steps/openopt. I also added some variables to bin/release. Note that I am assuming installation through a setup.py file rather than the requirements.txt approach (c.f. https://devcenter.heroku.com/articles/python-pip#traditional_distributions).

I also download the blas/lapack/atlas/gfortran binaries that were used to build numpy/scipy onto the dyno as there are c extensions that need to link through to them. The reason for building everything offline and downloading is that pip-installing numpy/scipy requires you to have a fortran compiler + associated dev environment and this made my slugs too big.

It seems to work, the slug size is now 35mb and scaling seems fast too. All but one of the numpy tests pass and all of the scipy tests pass.

This is still work in progress for me but I thought I'd share.

这篇关于在Heroku上运行Scipy的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！