使用本地依赖项部署 GCP Cloud 功能 [英] Deploy GCP Cloud function with local dependencies

问题描述

我一直在尝试部署具有一些私有依赖项 (pyodbc) 的 Cloud Function，因为我无法通过 requirements.txt 使其正常工作.请注意，我不想在这里使用 Docker.所以我在这里构建的所有内容都在文件下方，

I have been trying to deploy a Cloud Function with some private dependencies (pyodbc) as I couldn't get it working thru requirements.txt. Please note, I don't want to use Docker here. So all I have built here is below files,

1. main.py
2. process.py ( this one use pyodbc to connect to teradata)
3. libs (folder)
   3.1 pyodbc-4.0.30.dist-info (package)
   3.2 pyodbc (python extension module)
   3.3 __init.py__ ( this is to make this folder as module)
4.requirements.txt

我还更新了 process.py 文件以导入 pyodbc 模块，如下所示，

I also updated process.py file to import pyodbc module as below,

import libs.pyodbc

请注意:我使用 GCP 文档来安装 pyodbc 包并使用 https://cloud.google.com/functions/docs/writing/specifying-dependencies-python最重要的是，我也将 requirements.txt 作为默认导入.

Please note: I used GCP docs to install the pyodbc package and put in libs using https://cloud.google.com/functions/docs/writing/specifying-dependencies-python On top this, I am also requirements.txt to import as default.

但我仍然收到如下模块错误.

But I am still getting module error as below.

Error message: Code in file main.py can't be loaded.
Did you list all required modules in requirements.txt?
Detailed stack trace: Traceback (most recent call last):
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 305, in check_or_load_user_function
    _function_handler.load_user_function()
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 184, in load_user_function
    spec.loader.exec_module(main)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/user_code/main.py", line 9, in <module>
    from process import process
  File "/user_code/process.py", line 6, in <module>
    import libs.pyodbc
ModuleNotFoundError: No module named 'libs.pyodbc'

非常感谢这里的任何线索或帮助.我在这里试图实现的是，从 GCP 存储桶中读取 CSV 文件并通过加载到 teradata 的数据帧处理它，并将输出文件生成回另一个 GCP 存储桶.我正在尝试仅使用 Cloud Functions 来实现所有目标.谢谢

Any leads or help from here is really appreciated. All I am trying to achieve here is, Read CSV files from GCP bucket and process it thru dataframe which loads into teradata and generate output file back into another GCP bucket. I am trying to achieve all with Cloud Functions only. Thank you

推荐答案

pyodbc 项目在这里可能有点特殊，因为:

The pyodbc project might be a bit of a special case here, because:

1. 该项目需要一些特定于平台的代码；
2. 他们尚未发布最新版本的源代码发行版(仅内置发行版).

这是我为使其正常工作所做的工作.从一个空的 libs 目录开始，首先下载最新的可用源代码分发:

Here's what I did to get this to work. Starting with an empty libs directory, first download the latest available source distribution:

$ pip download pyodbc --no-binary :all:

为模块创建一个目录:

$ mkdir libs/pyodbc

将源代码解压到模块中:

Untar the source distribution into the module:

$ tar xf pyodbc-4.0.28.tar.gz -C libs/pyodbc

然后，在函数中你可以:

Then, in the function you can do:

import libs.pyodbc

这篇关于使用本地依赖项部署 GCP Cloud 功能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！