从创建的胶水客户端Scala中提取AWS Glue凭证 [英] Extract AWS Glue credentials from created glue client Scala

问题描述

我正在尝试引用此问题来提取嵌入式连接凭证。
但出现以下错误。

I am attempting to extract emmbedded connection credentials in reference to this question. But I get the following error.

import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.types._  //import everything from the package    
import org.apache.spark.sql.expressions.{Window => W} 
import org.apache.spark.sql.{functions => F}
import org.apache.spark.SparkContext
import com.amazonaws.services.glue.GlueContext
import com.amazonaws.services.glue.util.GlueArgParser 

import com.amazonaws.services.glue.DynamicFrame

import com.amazonaws.regions.Regions
import com.amazonaws.services.glue.model._

import com.amazonaws.services.glue.{AWSGlue, AWSGlueClient}
import scala.collection.JavaConverters.{mapAsJavaMapConverter, seqAsJavaListConverter}
import com.amazonaws.services.sagemaker.sparksdk.IAMRole

val sc = spark.sparkContext
val glueContext: GlueContext = new GlueContext(sc)

val region = Regions.fromName("us-east-1")

// Function to create AWS glue client
def glueClient(region: Regions):
AWSGlue = AWSGlueClient.builder().withRegion(region).build()

val glue = glueClient(region =region)

glue.getConnection("{Name: name-of-embedded-connection,HidePassword: False}")

遇到错误：：71：错误：类型不匹配；找到：String（ {{名称：preprod-samtec-redw，HidePassword：False}）必需）：
com.amazonaws.services.glue.model.GetConnectionRequest
gel.getConnection（ {名称：嵌入连接的名称，HidePassword：False}））

An error was encountered: :71: error: type mismatch; found : String("{Name: preprod-samtec-redw,HidePassword: False}") required: com.amazonaws.services.glue.model.GetConnectionRequest glue.getConnection("{Name: name-of-embedded-connection,HidePassword: False}")

推荐答案

如错误所示，您需要向 getConnection 方法提供 GetConnectionRequest 对象，例如：

As the error indicates, you need to provide a GetConnectionRequest object to the getConnectionmethod, something like:

import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.types._  //import everything from the package    
import org.apache.spark.sql.expressions.{Window => W} 
import org.apache.spark.sql.{functions => F}
import org.apache.spark.SparkContext
import com.amazonaws.services.glue.GlueContext
import com.amazonaws.services.glue.util.GlueArgParser 

import com.amazonaws.services.glue.DynamicFrame

import com.amazonaws.regions.Regions
import com.amazonaws.services.glue.model._

import com.amazonaws.services.glue.{AWSGlue, AWSGlueClient}
import scala.collection.JavaConverters.{mapAsJavaMapConverter, seqAsJavaListConverter}
import com.amazonaws.services.sagemaker.sparksdk.IAMRole

val sc = spark.sparkContext
val glueContext: GlueContext = new GlueContext(sc)

val region = Regions.fromName("us-east-1")

// Function to create AWS glue client
def glueClient(region: Regions):
AWSGlue = AWSGlueClient.builder().withRegion(region).build()

val glue = glueClient(region =region)

val getConnectionRequest: GetConnectionRequest = new GetConnectionRequest()
  .withName("name-of-embedded-connection")
  .withHidePassword(false)

val getConnectionResult: GetConnectionResult =  glue.getConnection(getConnectionRequest)

val connection: Connection = getConnectionResult.getConnection()

val connectionProperties = connection.getConnectionProperties()

// Now you can extract as many properties as you need (see https://github.com/aws/aws-sdk-java/blob/2d73c9847a327eea2f673a4cd2b3449e433649e5/aws-java-sdk-glue/src/main/java/com/amazonaws/services/glue/model/Connection.java#L687)
// For instance, the username or password
val username = connectionProperties.get("USERNAME")
val password = connectionProperties.get("PASSWORD")

这篇关于从创建的胶水客户端Scala中提取AWS Glue凭证的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！