如何在PySpark中运行脚本 [英] How to run a script in PySpark

问题描述

我正在尝试在pyspark环境中运行脚本，但到目前为止，我还无法运行.如何在pyspark中运行类似python script.py的脚本?谢谢

I'm trying to run a script in the pyspark environment but so far I haven't been able to. How can I run a script like python script.py but in pyspark? Thanks

推荐答案

您可以执行以下操作:./bin/spark-submit mypythonfile.py

You can do: ./bin/spark-submit mypythonfile.py

从Spark 2.0开始，不支持通过pyspark运行python应用程序.

Running python applications through pyspark is not supported as of Spark 2.0.

这篇关于如何在PySpark中运行脚本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！