将 AWS Glue Python 与 NumPy 和 Pandas Python 包结合使用 [英] Use AWS Glue Python with NumPy and Pandas Python Packages

问题描述

在 AWS 上名为 Glue 的新 ETL 工具中使用 NumPy 和 Pandas 等包的最简单方法是什么?我在 Python 中有一个完整的脚本，我想在利用 NumPy 和 Pandas 的 AWS Glue 中运行.

What is the easiest way to use packages such as NumPy and Pandas within the new ETL tool on AWS called Glue? I have a completed script within Python I would like to run in AWS Glue that utilizes NumPy and Pandas.

推荐答案

我认为目前的答案是你不能.根据 AWS Glue 文档:

I think the current answer is you cannot. According to AWS Glue Documentation:

只能使用纯 Python 库.尚不支持依赖 C 扩展的库，例如 pandas Python 数据分析库.

Only pure Python libraries can be used. Libraries that rely on C extensions, such as the pandas Python Data Analysis Library, are not yet supported.

但即使我尝试在 S3 中包含一个普通的 Python 编写的库，由于某些 HDFS 权限问题，Glue 作业失败了.如果您找到解决此问题的方法，也请告诉我.

But even when I try to include a normal python written library in S3, the Glue job failed because of some HDFS permission problem. If you find a way to solve this, please let me know as well.

这篇关于将 AWS Glue Python 与 NumPy 和 Pandas Python 包结合使用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！