EC2上的AWS Lambda + Tinkerpop/Gremlin + TitanDB +云中的AWS DynamoDB [英] AWS Lambda + Tinkerpop/Gremlin + TitanDB on EC2 + AWS DynamoDB in cloud

问题描述

我正在尝试执行以下流程:

I am trying to execute following flow:

1. 用户点击了AWS Gateway(REST)，

1. user hits AWS Gateway (REST),

它触发AWS Lambda，

it triggers AWS Lambda,

连接到

EC2上的TitanDB，它使用

TitanDB on EC2, that uses

AWS DynamoDB作为后端在云中(不在EC2上).

AWS DynamoDB in cloud (not on EC2) as backend.

现在，我已经设法在EC2上创建完全正常工作的TitanDB实例，该实例将数据存储在云中的DynamoDB中. 我也只能通过Tinkerpop/Gremlin BUT通过这种方式从AWS Lambda连接到EC2:

Right now I have managed to crete fully working TitanDB instance on EC2, that stores data in DynamoDB in cloud. I am also able to connect from AWS Lambda to EC2 through Tinkerpop/Gremlin BUT only this way:

Cluster.build()
       .addContactPoint("10.x.x.x") // ip of EC2
       .create()
       .connect()
       .submit("here I type my query as string and it will work");

这可行，但是我强烈希望使用"Criteria API"(GremlinPipeline)而不是普通的Gremlin语言. 换句话说，我需要ORM或类似的东西. 我知道，Tinkerpop包含了它. 我意识到，我需要的是Graph类的对象. 这是我尝试过的:

And this works, however I strongly prefer to use "Criteria API" (GremlinPipeline) instead of plain Gremlin language. In other words, I need ORM or something like that. I know, that Tinkerpop includes it. I have realized, that what I need is object of class Graph. This is what I have tried:

Graph graph = TitanFactory
            .build()
            .set("storage.hostname", "10.x.x.x")
            .set("storage.backend", "com.amazon.titan.diskstorage.dynamodb.DynamoDBStoreManager")
            .set("storage.dynamodb.client.credentials.class-name", "com.amazonaws.auth.DefaultAWSCredentialsProviderChain")
            .set("storage.dynamodb.client.credentials.constructor-args", "")
            .set("storage.dynamodb.client.endpoint", "https://dynamodb.ap-southeast-2.amazonaws.com")
            .open();

但是，它抛出找不到实现类:com.amazon.titan.diskstorage.dynamodb.DynamoDBStoreManager". 当然，计算机是正确的，因为IntelliJ IDEA也找不到它.

However, it throws "Could not find implementation class: com.amazon.titan.diskstorage.dynamodb.DynamoDBStoreManager". Of course, computer is correct, as IntelliJ IDEA also cannot find it.

我的依赖项:

//
// aws
compile 'com.amazonaws:aws-lambda-java-core:+'
compile 'com.amazonaws:aws-lambda-java-events:+'
compile 'com.amazonaws:aws-lambda-java-log4j:+'
compile 'com.amazonaws:aws-java-sdk-dynamodb:1.10.5.1'
compile 'com.amazonaws:aws-java-sdk-ec2:+'
//
// database
// titan 1.0.0 is compatible with gremlin 3.0.2-incubating, but not yet with 3.2.0
compile 'com.thinkaurelius.titan:titan-core:1.0.0'
compile 'org.apache.tinkerpop:gremlin-core:3.0.2-incubating'
compile 'org.apache.tinkerpop:gremlin-driver:3.0.2-incubating'

我的目标是:拥有完全正常工作的Graph对象

What is my goal: have fully working Graph object

我的问题是什么:我没有DynamoDBStoreManager类，并且我不知道必须添加什么依赖项.

What is my problem: I don't have DynamoDBStoreManager class, and I do not know what dependency I have to add.

我的另一个问题是:为什么通过Cluster类进行连接只需要IP并可以工作，而TitanFactory需要像我在EC2的gremlin-server上使用过的属性一样? 我不想创建第二个服务器，我只想作为客户端连接到它并获取Graph对象.

My additional question is: why connecting through Cluster class requires only IP and works, but TitanFactory requires properties like those I have used on gremlin-server on EC2? I do not want to create second server, I just want to connect as client to it and take Graph object.

添加解析器后，它将在输出中生成多个内容:

After adding resolver, it builds, in output I get multiple:

13689 [TitanID(0)(4)[0]]警告com.thinkaurelius.titan.diskstorage.idmanagement.ConsistentKeyIDAuthority-获取ID块时出现临时存储异常-在PT2.4S中重试:com.thinkaurelius.titan.diskstorage .TemporaryBackendException:为PT0.342S中的ID块[1，51)编写了声明=>太慢，阈值为:PT0.3S

13689 [TitanID(0)(4)[0]] WARN com.thinkaurelius.titan.diskstorage.idmanagement.ConsistentKeyIDAuthority - Temporary storage exception while acquiring id block - retrying in PT2.4S: com.thinkaurelius.titan.diskstorage.TemporaryBackendException: Wrote claim for id block [1, 51) in PT0.342S => too slow, threshold is: PT0.3S

并且执行挂在open()方法上，因此不允许我执行任何查询.

and execution hangs on open() method, so does not allow me to execute any queries.

推荐答案

对于DynamoDBStoreManager类，您需要此然后针对DynamoDBLocal问题，尝试添加此解析器:

Then for the DynamoDBLocal issue, try adding this resolver:

resolvers += "AWS DynamoDB Local Release Repository" at "http://dynamodb-local.s3-website-us-west-2.amazonaws.com/release"

我不清楚这是什么意思-标准API"而不是普通的Gremlin语言.我猜您是要使用Java与图形进行交互，而不是将Gremlin作为字符串传递给正在运行的Titan/Gremlin Server?如果是这种情况，那么您根本不需要启动Titan/Gremlin Server(上面的步骤4).编写一个AWS Lambda程序(上面的步骤2-3)，该程序通过TitanFactory创建直接的Titan客户端连接，其中所有Titan配置属性都适用于您的DynamoDB实例(上面的步骤5).

I'm not entirely clear on what this means -- "Criteria API" instead of plain Gremlin language. I'm guessing that you mean that you want to interact with the graph using Java rather than passing Gremlin as a string over to a running Titan/Gremlin Server? If this is the case, then you don't need to start a Titan/Gremlin Server at all (step 4 above). Write an AWS Lambda program (step 2-3 above) that creates a direct Titan client connection via TitanFactory, where all of the Titan configuration properties are for your DynamoDB instance (step 5 above).

这篇关于EC2上的AWS Lambda + Tinkerpop/Gremlin + TitanDB +云中的AWS DynamoDB的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！