我在CentOS上工作,我已经设置了$SPARK_HOME,还添加了$PATH中bin的路径。

我可以在任何地方运行pyspark。

但是当我试图创建python文件并使用此语句时from pyspark import SparkConf, SparkContext

它抛出以下错误python pysparktask.py

Traceback (most recent call last):

File "pysparktask.py", line 1, in

from pyspark import SparkConf, SparkContext

ModuleNotFoundError: No module named 'pyspark'

我试图使用pip重新安装它。pip install pyspark

它也给出了这个错误。Could not find a version that satisfies the requirement pyspark (from versions: )

No matching distribution found for pyspark

编辑

根据答案,我更新了代码。

错误是Traceback (most recent call last):

File "pysparktask.py", line 6, in

from pyspark import SparkConf, SparkContext

File "/opt/mapr/spark/spark-2.0.1/python/pyspark/__init__.py", line 44, in

from pyspark.context import SparkContext

File "/opt/mapr/spark/spark-2.0.1/python/pyspark/context.py", line 33, in

from pyspark.java_gateway import launch_gateway

File "/opt/mapr/spark/spark-2.0.1/python/pyspark/java_gateway.py", line 31, in

from py4j.java_gateway import java_import, JavaGateway, GatewayClient

ModuleNotFoundError: No module named 'py4j'

Logo

腾讯云面向开发者汇聚海量精品云计算使用和开发经验,营造开放的云计算技术生态圈。

更多推荐