在加载之前，来自databrick的ADLS中存在检查文件 [英] Ho to check file exists in ADLS from databrick before load

问题描述

如何在加载前检查dadsricks（scala）中ADLS中存在的文件 

How to check file exists in ADLS in databricks (scala) before loading 

var yltPaths：Array [String] = new Array [String]（layerCount）

var yltPaths: Array[String] = new Array[String](layerCount)

  for（i< - 0 to（layerCount-1））

  &NBSP; {

  &NBSP; &NBSP; &NBSP; &NBSP; layerKey = layerArr（i）.getInt（0）  &NBSP; &NBSP; &NBSP;   

  &NBSP; &NBSP; &NBSP;    yltPaths（i）= s""" adl：//xxxxxxxxxxxxxxxxxxxxxxxxx / testdata/loss/13/2/dylt/loss_$layerKey.parquet"""

  for(i <- 0 to (layerCount-1))
    {
          layerKey =layerArr(i).getInt(0)          
          yltPaths(i) = s"""adl://xxxxxxxxxxxxxxxxxxxxxxxxx/testdata/loss/13/2/dylt/loss_$layerKey.parquet"""

}

var fexs =  yltPaths.filter（p => {< check file exists >}）

var fexs = yltPaths.filter(p=> {<check file exists >})

var ylt = spark.read.parquet（fexs：_ *）

var ylt = spark.read.parquet(fexs:_*)

推荐答案

请给出这个
方法一试。这应该工作。如果您遇到任何错误，请告诉我您尝试过的代码

please give this method a try. This should work. If you face any error please let me know with the code which you tried

这篇关于在加载之前，来自databrick的ADLS中存在检查文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！