将数据从pyspark写入天蓝色的blob? [英] Write data from pyspark to azure blob?

问题描述

我想将数据帧从pyspark写入到azure blob吗?有任何建议或代码怎么做?

I want to write dataframe from pyspark to azure blob? Any suggestions or code how to do it?

我有Blob的位置和密钥

I have location and key of blob

在此处输入图片描述

推荐答案

您可以遵循

You could follow this tutorial to connector your spark dataframe with Azure Blob Storage.

设置连接信息:

session.conf.set(
    "fs.azure.account.key.<storage-account-name>.blob.core.windows.net",
    "<your-storage-account-access-key>"
)

然后将数据写入Blob存储:

Then write data into blob storage:

sdf = session.write.parquet(
    "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/<prefix>"
)

此外，您可以参考这种情况:将pyspark写入wasb blob存储容器

Also,you could refer to this case:pyspark write to wasb blob storage container

这篇关于将数据从pyspark写入天蓝色的blob?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！