spark-2.1.0-bin-hadoop2.7\python: CreateProcess error=5，访问被拒绝 [英] spark-2.1.0-bin-hadoop2.7\python: CreateProcess error=5, Access is denied

问题描述

我试图在 pyspark 上运行这个简单的代码，但是当我收集时，得到一个错误访问被拒绝.我不明白出了什么问题，我认为我拥有所有权利.

I tried to run this simple code on pyspark, however when I do the collect a get an error acces denied. I don't understand what's wrong I think i have all the rights.

x = sc.parallelize([("a", 1), ("b", 1), ("a", 1), ("a", 1),("b", 1), ("b", 1), ("b", 1), ("b", 1)], 3)
y = x.reduceByKey(lambda accum, n: accum + n)
for v in y.collect():
    print(v)

在本地，但我有一个错误:

in local but i have an error :

CreateProcess error=5, Access is denied

    17/04/25 10:57:08 ERROR TaskSetManager: Task 2 in stage 0.0 failed 1 times; aborting job
    Traceback (most recent call last):
      File "C:/Users/rubeno/PycharmProjects/Pyspark/Twiiter_ETL.py", line 40, in <module>
        for v in y.collect():
      File "C:\Users\rubeno\Documents\spark-2.1.0-bin-hadoop2.7\python\pyspark\rdd.py", line 809, in collect
        port = self.ctx._jvm.PythonRDD.collectAndServe(self._jrdd.rdd())
      File "C:\Users\rubeno\Documents\spark-2.1.0-bin-hadoop2.7\python\lib\py4j-0.10.4-src.zip\py4j\java_gateway.py", line 1133, in __call__
      File "C:\Users\rubeno\Documents\spark-2.1.0-bin-hadoop2.7\python\pyspark\sql\utils.py", line 63, in deco
        return f(*a, **kw)
      File "C:\Users\rubeno\Documents\spark-2.1.0-bin-hadoop2.7\python\lib\py4j-0.10.4-src.zip\py4j\protocol.py", line 319, in get_return_value
    py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
    : org.apache.spark.SparkException: Job aborted due to stage failure: Task 2 in stage 0.0 failed 1 times, most recent failure: Lost task 2.0 in stage 0.0 (TID 2, localhost, executor driver): java.io.IOException: Cannot run program "C:\Users\\rubeno\Documents\spark-2.1.0-bin-hadoop2.7\python": CreateProcess error=5, Access is denied
        at java.lang.ProcessBuilder.start(Unknown Source)

推荐答案

需要设置整个pyspark目录的权限.

You need to set the permissions to the whole pyspark directory.

右键单击目录->属性 ->安全选项卡并设置完全控制"为每个人"并启用继承.

Right click on the directory -> Properties -> Security tab and set "Full control" for "Everyone" and enabled inheritance.

这篇关于spark-2.1.0-bin-hadoop2.7\python: CreateProcess error=5，访问被拒绝的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！