使用pySpark在Azure Databricks中消耗EventHub中的事件 [英] Consume events from EventHub In Azure Databricks using pySpark

问题描述

我可以看到火花接头&在Azure Databricks中使用Scala从事件中心使用事件的准则.

I could see spark connectors & guidelines for consuming events from Event Hub using Scala in Azure Databricks.

但是，如何使用pySpark从蔚蓝的数据块中消耗事件中心中的事件?

任何建议/文档详细信息都会有所帮助.谢谢

any suggestions/documentation details would help. thanks

推荐答案

下面是用于从蔚蓝数据块上的pyspark从事件中心读取事件的代码段.

Below is the snippet for reading events from event hub from pyspark on azure data-bricks.

// With an entity path 
val with = "Endpoint=sb://SAMPLE;SharedAccessKeyName=KEY_NAME;SharedAccessKey=KEY;EntityPath=EVENTHUB_NAME"


# Source with default settings
connectionString = "Valid EventHubs connection string."
ehConf = {
  'eventhubs.connectionString' : connectionString
}

df = spark \
  .readStream \
  .format("eventhubs") \
  .options(**ehConf) \
  .load()

readInStreamBody = df.withColumn("body", df["body"].cast("string"))
display(readInStreamBody)

这篇关于使用pySpark在Azure Databricks中消耗EventHub中的事件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！