DynamoDB - 本地二级索引

某些应用程序仅使用主键执行查询,但某些情况会受益于备用排序键.通过创建单个或多个本地二级索引,允许您的应用程序选择.

复杂的数据访问要求,例如组合数百万个项目,使得必须执行更有效的查询/扫描.本地二级索引为分区键值提供备用排序键.它们还包含所有或某些表属性的副本.它们按表分区键组织数据,但使用不同的排序键.

使用本地二级索引不需要进行整个表扫描,并允许使用排序进行简单快速的查询密钥.

所有本地二级索引必须满足某些条件 :

  • 相同的分区键和源表分区键.

  • 只有一个标量属性的排序键.

  • 投影作为非键的源表排序键属性.

所有本地二级索引都自动保存父表中的分区和排序键.在查询中,这意味着有效收集投影属性,以及检索未投影的属性.

本地二级索引的存储限制仍为每个分区键值10GB,包括所有表项和共享分区键值的索引项.

投影属性

由于复杂性,某些操作需要过多的读/取.这些操作会消耗大量吞吐量. Projection允许您通过隔离这些属性来避免代价高昂的提取并执行丰富的查询.请记住,投影由复制到二级索引中的属性组成.

创建二级索引时,指定投影的属性.回想一下DynamoDB提供的三个选项: KEYS_ONLY,INCLUDE和ALL .

在投影中选择某些属性时,请考虑相关的成本权衡 :

  • 如果只投影一小组必要属性,则可以大大降低存储成本.

  • 如果您投影经常访问的非关键属性,则会将扫描成本与存储成本相抵消.

  • 如果您投影最多或者所有非关键属性,这可以最大限度地提高灵活性并降低吞吐量(无需检索);但是,存储成本会上升.

  • 如果您为频繁的写入/更新和不常见的查询投影KEYS_ONLY,它会最小化大小,但会保持查询准备.

创建本地二级索引

使用CreateTable的 LocalSecondaryIndex 参数创建一个单个或多个本地二级索引.您必须为排序键指定一个非键属性.在创建表时,您将创建本地二级索引.删除时,删除这些索引.

具有本地二级索引的表必须遵守每个分区键值大小为10GB的限制,但可以存储任意数量的项.

本地二级索引查询和扫描

当索引中的多个项共享排序键值时,对本地二级索引的查询操作将返回具有匹配分区键值的所有项.匹配项目不按特定顺序返回.对本地二级索引的查询使用最终或强一致性,使用强一致性读取提供最新值.

扫描操作返回所有本地二级索引数据.扫描要求您提供表和索引名称,并允许使用过滤器表达式来丢弃数据.

项目写入

创建在本地二级索引中,指定排序键属性及其数据类型.当您编写项目时,如果项目定义了索引键的属性,则其类型必须与键架构的数据类型匹配.

DynamoDB不强制实施任何一对一关系要求表项和本地二级索引项.具有多个本地二级索引的表比较少的表具有更高的写入成本.

本地二级索引中的吞吐量注意事项

读取查询的容量消耗取决于数据访问的性质.查询使用最终或强一致性,使用一个单位进行强一致性读取,而最终一致性读取使用半个单位.

结果限制包括最大1MB大小.结果大小来自匹配的索引项目大小的总和,四舍五入到最接近的4KB,匹配的表项大小也向上舍入到最接近的4KB.

写入容量消耗保持在配置单位内.通过查找表写入中消耗单位的总和以及更新指数中的消耗单位来计算总预配成本.

您还可以考虑影响成本的关键因素,其中一些可以是 :

  • 当您编写定义索引属性的项目或更新项目以定义未定义的索引属性时,单个写入操作发生.

  • 当表更新更改索引键属性值时,会发生两次写入删除然后 - 添加项目.

  • 当写入导致删除索引属性时,会发生一次写入以删除旧项目投影.

  • 如果在更新之前或之后索引中不存在项目,则不会发生写入.

本地二级索引存储

在表项目写入上,DynamoDB会自动将正确的属性集复制到所需的本地二级索引.这会向您的帐户收费.使用的空间来自表主键字节大小,索引键属性字节大小,任何当前投影属性字节大小和每个索引项的开销100字节之和.

估计值通过估计平均索引项目大小并乘以表项数量来获得存储.

使用Java处理本地二级索引

创建本地二级索引index首先创建一个DynamoDB类实例.然后,使用必要的请求信息创建CreateTableRequest类实例.最后,使用createTable方法.

示例

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient( 
   new ProfileCredentialsProvider()));
String tableName = "Tools";  
CreateTableRequest createTableRequest = new 
   CreateTableRequest().withTableName(tableName);
   
//Provisioned Throughput
createTableRequest.setProvisionedThroughput (
   new ProvisionedThroughput()
   .withReadCapacityUnits((long)5)
   .withWriteCapacityUnits(( long)5));
   
//Attributes 
ArrayList<AttributeDefinition> attributeDefinitions = 
   new ArrayList<AttributeDefinition>();
   attributeDefinitions.add(new AttributeDefinition()
   .withAttributeName("Make")
   .withAttributeType("S"));
   
attributeDefinitions.add(new AttributeDefinition()
   .withAttributeName("Model")
   .withAttributeType("S"));
   
attributeDefinitions.add(new AttributeDefinition()
   .withAttributeName("Line")
   .withAttributeType("S"));
   
createTableRequest.setAttributeDefinitions(attributeDefinitions);

//Key Schema 
ArrayList<KeySchemaElement> tableKeySchema = new 
   ArrayList<KeySchemaElement>();
   
tableKeySchema.add(new KeySchemaElement()
   .withAttributeName("Make")
   .withKeyType(KeyType.HASH));                    //Partition key
   
tableKeySchema.add(new KeySchemaElement()
   .withAttributeName("Model")
   .withKeyType(KeyType.RANGE));                   //Sort key
   
createTableRequest.setKeySchema(tableKeySchema);
ArrayList<KeySchemaElement> indexKeySchema = new 
   ArrayList<KeySchemaElement>();
   
indexKeySchema.add(new KeySchemaElement()
   .withAttributeName("Make")
   .withKeyType(KeyType.HASH));                   //Partition key
   
indexKeySchema.add(new KeySchemaElement()
   .withAttributeName("Line")
   .withKeyType(KeyType.RANGE));                   //Sort key
   
Projection projection = new Projection()
   .withProjectionType(ProjectionType.INCLUDE);

ArrayList<String> nonKeyAttributes = new ArrayList<String>(); 
nonKeyAttributes.add("Type"); 
nonKeyAttributes.add("Year"); 
projection.setNonKeyAttributes(nonKeyAttributes);  

LocalSecondaryIndex localSecondaryIndex = new LocalSecondaryIndex() 
   .withIndexName("ModelIndex")
   .withKeySchema(indexKeySchema)
   .withProjection(p rojection);  

ArrayList<LocalSecondaryIndex> localSecondaryIndexes = new 
   ArrayList<LocalSecondaryIndex>(); 

localSecondaryIndexes.add(localSecondaryIndex); 
createTableRequest.setLocalSecondaryIndexes(localSecondaryIndexes);  
Table table = dynamoDB.createTable(createTableRequest); 
System.out.println(table.getDescription());

使用describe方法检索有关本地二级索引的信息.只需创建一个DynamoDB类实例,创建一个Table类实例,并将表传递给describe方法.

示例

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient( 
   new ProfileCredentialsProvider()));
   
String tableName = "Tools";
Table table = dynamoDB.getTable(tableName);
TableDescription tableDescription = table.describe();

List<LocalSecondaryIndexDescription> localSecondaryIndexes = 
   tableDescription.getLocalSecondaryIndexes();
   
Iterator<LocalSecondaryIndexDescription> lsiIter = 
   localSecondaryIndexes.iterator();
   
while (lsiIter.hasNext()) {  
   LocalSecondaryIndexDescription lsiDescription = lsiIter.next(); 
   System.out.println("Index info " + lsiDescription.getIndexName() + ":"); 
   Iterator<KeySchemaElement> kseIter = lsiDescription.getKeySchema().iterator(); 
   
   while (kseIter.hasNext()) { 
      KeySchemaElement kse = kseIter.next(); 
      System.out.printf("\t%s: %s\n", kse.getAttributeName(), kse.getKeyType()); 
   }
   
   Projection projection = lsiDescription.getProjection(); 
   System.out.println("\tProjection type: " + projection.getProjectionType()); 
   
   if (projection.getProjectionType().toString().equals("INCLUDE")) { 
      System.out.println("\t\tNon-key projected attributes: " + 
         projection.getNonKeyAttributes()); 
   } 
}

使用与表查询相同的步骤执行查询.仅创建DynamoDB类实例,Table类实例,Index类实例,查询对象,并使用查询方法.

示例

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient( 
   new ProfileCredentialsProvider()));
   
String tableName = "Tools";  
Table table = dynamoDB.getTable(tableName); 
Index index = table.getIndex("LineIndex");  
QuerySpec spec = new QuerySpec() 
   .withKeyConditionExpression("Make = :v_make and Line = :v_line") 
   .withValueMap(new ValueMap() 
   .withString(":v_make", "Depault") 
   .withString(":v_line", "SuperSawz"));
      
ItemCollection<QueryOutcome> items = index.query(spec);
Iterator<Item> itemsIter = items.iterator();

while (itemsIter.hasNext()) { 
   Item item = itemsIter.next(); 
   System.out.println(item.toJSONPretty()); 
}

您还可以查看以下示例.

注意 去;以下示例可以假定先前创建的数据源.在尝试执行之前,获取支持库并创建必要的数据源(具有所需特征的表或其他引用的源).

以下示例还使用Eclipse IDE,AWS凭证文件,以及Eclipse AWS Java项目中的AWS Toolkit.

示例

import java.util.ArrayList;
import java.util.Iterator;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;

import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Index;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;
import com.amazonaws.services.dynamodbv2.document.PutItemOutcome;
import com.amazonaws.services.dynamodbv2.document.QueryOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.spec.QuerySpec;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;

import com.amazonaws.services.dynamodbv2.model.AttributeDefinition;
import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.KeySchemaElement;
import com.amazonaws.services.dynamodbv2.model.KeyType;
import com.amazonaws.services.dynamodbv2.model.LocalSecondaryIndex;
import com.amazonaws.services.dynamodbv2.model.Projection;
import com.amazonaws.services.dynamodbv2.model.ProjectionType;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;
import com.amazonaws.services.dynamodbv2.model.ReturnConsumedCapacity;
import com.amazonaws.services.dynamodbv2.model.Select;

public class LocalSecondaryIndexSample {  
   static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient( 
      new ProfileCredentialsProvider()));  
   public static String tableName = "ProductOrders";  
   
   public static void main(String[] args) throws Exception {  
      createTable();
      query(null); 
      query("IsOpenIndex"); 
      query("OrderCreationDateIndex"); 
   }
   public static void createTable() { 
      CreateTableRequest createTableRequest = new CreateTableRequest() 
         .withTableName(tableName) 
         .withProvisionedThroughput(new ProvisionedThroughput() 
         .withReadCapacityUnits((long) 1) 
         .withWriteCapacityUnits((long) 1));
         
      // Table partition and sort keys attributes 
      ArrayList<AttributeDefinition> attributeDefinitions = new 
         ArrayList<AttributeDefinition>(); 
      
      attributeDefinitions.add(new AttributeDefinition() 
         .withAttributeName("CustomerID") 
         .withAttributeType("S"));
         
      attributeDefinitions.add(new AttributeDefinition() 
         .withAttributeName("OrderID") 
         .withAttributeType("N"));
         
      // Index primary key attributes 
      attributeDefinitions.add(new AttributeDefinition() 
         .withAttributeName("OrderDate") 
         .withAttributeType("N"));
         
      attributeDefinitions.add(new AttributeDefinition() 
         .withAttributeName("OpenStatus") 
         .withAttributeType("N"));  
      createTableRequest.setAttributeDefinitions(attributeDefinitions);
      
      // Table key schema 
      ArrayList<KeySchemaElement> tableKeySchema = new
         ArrayList<KeySchemaElement>(); 
      tableKeySchema.add(new KeySchemaElement()  
         .withAttributeName("CustomerID") 
         .withKeyType(KeyType.HASH));                    //Partition key
         
      tableKeySchema.add(new KeySchemaElement() 
         .withAttributeName("OrderID") 
         .withKeyType(KeyType.RANGE));                   //Sort key
         
      createTableRequest.setKeySchema(tableKeySchema);  
      ArrayList<LocalSecondaryIndex> localSecondaryIndexes = new 
         ArrayList<LocalSecondaryIndex>();  
      
      // OrderDateIndex 
      LocalSecondaryIndex orderDateIndex = new LocalSecondaryIndex() 
         .withIndexName("OrderDateIndex");
         
      // OrderDateIndex key schema 
      ArrayList<KeySchemaElement> indexKeySchema = new 
         ArrayList<KeySchemaElement>(); 
      indexKeySchema.add(new KeySchemaElement() 
         .withAttributeName("CustomerID") 
         .withKeyType(KeyType.HASH));                   //Partition key
         
      indexKeySchema.add(new KeySchemaElement() 
         .withAttributeName("OrderDate") 
         .withKeyType(KeyType.RANGE));                   //Sort key
      orderDateIndex.setKeySchema(indexKeySchema);
      
      // OrderCreationDateIndex projection w/attributes list 
      Projection projection = new Projection() 
         .withProjectionType(ProjectionType.INCLUDE); 
      
      ArrayList<String> nonKeyAttributes = new ArrayList<String>(); 
      nonKeyAttributes.add("ProdCat"); 
      nonKeyAttributes.add("ProdNomenclature"); 
      projection.setNonKeyAttributes(nonKeyAttributes);
      orderCreationDateIndex.setProjection(projection);  
      localSecondaryIndexes.add(orderDateIndex);  
      
      // IsOpenIndex 
      LocalSecondaryIndex isOpenIndex = new LocalSecondaryIndex() 
         .withIndexName("IsOpenIndex");  
      
      // OpenStatusIndex key schema 
      indexKeySchema = new ArrayList<KeySchemaElement>(); 
      indexKeySchema.add(new KeySchemaElement() 
         .withAttributeName("CustomerID") 
         .withKeyType(KeyType.HASH));                   //Partition key
         
      indexKeySchema.add(new KeySchemaElement() 
         .withAttributeName("OpenStatus") 
         .withKeyType(KeyType.RANGE));                   //Sort key
         
      // OpenStatusIndex projection 
      projection = new Projection() .withProjectionType(ProjectionType.ALL);  
      OpenStatusIndex.setKeySchema(indexKeySchema); 
      OpenStatusIndex.setProjection(projection);  
      localSecondaryIndexes.add(OpenStatusIndex);  
      
      // Put definitions in CreateTable request 
      createTableRequest.setLocalSecondaryIndexes(localSecondaryIndexes);  
      System.out.println("Spawning table " + tableName + "..."); 
      System.out.println(dynamoDB.createTable(createTableRequest));  
      
      // Pause for ACTIVE status 
      System.out.println("Waiting for ACTIVE table:" + tableName); 
      try { 
         Table table = dynamoDB.getTable(tableName);
         table.waitForActive(); 
      } catch (InterruptedException e) { 
         e.printStackTrace(); 
      } 
   }
   public static void query(String indexName) {  
      Table table = dynamoDB.getTable(tableName);  
      System.out.println("\n*************************************************\n"); 
      System.out.println("Executing query on" + tableName);  
      QuerySpec querySpec = new QuerySpec() 
         .withConsistentRead(true) 
         .withScanIndexForward(true) 
         .withReturnConsumedCapacity(ReturnConsumedCapacity.TOTAL);
      
      if (indexName == "OpenStatusIndex") {  
         System.out.println("\nEmploying index: '" + indexName 
            + "' open orders for this customer.");
            
         System.out.println( 
            "Returns only user-specified attribute list\n"); 
         Index index = table.getIndex(indexName); 
             
         querySpec.withKeyConditionExpression("CustomerID = :v_custmid and 
            OpenStatus = :v_openstat") 
            .withValueMap(new ValueMap() 
            .withString(":v_custmid", "jane@sample.com") 
            .withNumber(":v_openstat", 1));  
         
         querySpec.withProjectionExpression( 
            "OrderDate, ProdCat, ProdNomenclature, OrderStatus"); 
            ItemCollection<QueryOutcome> items = index.query(querySpec); 
            Iterator<Item> iterator = items.iterator();  
            System.out.println("Printing query results...");  
            
         while (iterator.hasNext()) { 
            System.out.println(iterator.next().toJSONPretty()); 
         }  
      } else if (indexName == "OrderDateIndex") { 
         System.out.println("\nUsing index: '" + indexName 
            + "': this customer's orders placed after 05/22/2016."); 
         System.out.println("Projected attributes are returned\n"); 
         Index index = table.getIndex(indexName); 
             
         querySpec.withKeyConditionExpression("CustomerID = :v_custmid and OrderDate 
            >= :v_ordrdate") 
            .withValueMap(new ValueMap() 
            .withString(":v_custmid", "jane@sample.com") 
            .withNumber(":v_ordrdate", 20160522));
               
         querySpec.withSelect(Select.ALL_PROJECTED_ATTRIBUTES);  
         ItemCollection<QueryOutcome> items = index.query(querySpec); 
         Iterator<Item> iterator = items.iterator();  
         System.out.println("Printing query results...");  
            
         while (iterator.hasNext()) { 
            System.out.println(iterator.next().toJSONPretty()); 
         }  
      } else { 
         System.out.println("\nNo index: All Jane's orders by OrderID:\n"); 
         querySpec.withKeyConditionExpression("CustomerID = :v_custmid") 
            .withValueMap(new ValueMap()
            .withString(":v_custmid", "jane@example.com"));  
         
         ItemCollection<QueryOutcome> items = table.query(querySpec); 
         Iterator<Item> iterator = items.iterator();  
         System.out.println("Printing query results...");  
         
         while (iterator.hasNext()) { 
            System.out.println(iterator.next().toJSONPretty()); 
         } 
      } 
   } 
}