Apache Ignite区域(机架)感知的parititons [英] Apache Ignite zone(rack)-aware parititons

问题描述

我正在努力配置Apache Ignite以区域感知方式分配分区.我有Ignite 2.8.0，其中4个节点作为GKE 1.14中的StatefulSet容器运行，分为两个区域.我遵循了

然后，我将节点ID映射到节点报告的AVAILABILITY_ZONE env var中的值，并得到以下结果:

  AA146954 us-central1-a3943ECC8 us-central1-cF7B7AB67 us-central1-aA94EE82C us-central1-c 

很容易看出，分区0 pri/bak驻留在节点3943ECC8和A94EE82C上，它们都在同一区域中.我想让它工作什么?

另一个奇怪的事情是，将分区号指定为低(例如2或4)，仅使用4个节点中的3个).当使用1024个分区时，所有节点都被使用，但问题仍然存在-1024个分区中的346个的主/备份位于同一区域中.

这是我的节点配置XML:

 < beans xmlns ="http://www.springframework.org/schema/beans"xmlns:xsi ="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation =http://www.springframework.org/schema/beanshttp://www.springframework.org/schema/beans/spring-beans.xsd">< bean class ="org.apache.ignite.configuration.IgniteConfiguration"><！-启用Apache Ignite永久存储.-><属性名称="dataStorageConfiguration">< bean class ="org.apache.ignite.configuration.DataStorageConfiguration"><属性名称="defaultDataRegionConfiguration">< bean class ="org.apache.ignite.configuration.DataRegionConfiguration"><属性名称="persistenceEnabled"值="true"/></bean></property></bean></property><！-明确配置TCP发现SPI以提供初始节点列表.-><属性名称="discoverySpi">< bean class ="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"><属性名称="ipFinder"><！-启用Kubernetes IP finder并设置自定义名称空间和服务名称.->< bean class ="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder"><属性名称=名称空间"值=点燃"/></bean></property></bean></property><属性名称="cacheConfiguration">< list>< bean id ="zone-aware-cache-template" abstract ="true" class ="org.apache.ignite.configuration.CacheConfiguration"><！-通过XML配置创建模板时，必须在模板名称上添加星号-><属性名称=名称" value =区域感知缓存*"/><属性名称="cacheMode"值="PARTITIONED"/><属性名称="atomicityMode"值="ATOMIC"/><属性名称=备份" value ="1"/><属性名称="readFromBackup" value ="true"/><属性名称="partitionLossPolicy" value ="READ_WRITE_SAFE"/><属性名称="copyOnRead"值="true"/><属性名称="eagerTtl"值="true"/><属性名称="statisticsEnabled"值="true"/><属性名称="affinity">< bean class ="org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction"><属性名称="partitions" value ="2"/><！-仅用于调试！-><属性名称="excludeNeighbors" value ="true"/><属性名称="affinityBackupFilter">< bean class ="org.apache.ignite.cache.affinity.rendezvous.ClusterNodeAttributeAffinityBackupFilter">< constructor-arg>< array value-type ="java.lang.String"><！-备份必须转到不同的可用区->< value> AVAILABILITY_ZONE</value></array></constructor-arg></bean></property></bean></property></bean></list></property></bean></beans> 

更新:最终 excludeNeighbors 否/true会破坏区域意识.我不确定为什么以前对我来说 excludeNeighbors = false 不起作用.我编写了一些脚本来自动化测试.现在可以确定是 excludeNeighbors 设置.都在这里: https://github.com/doitintl/ignite-gke .无论如何，我还使用IGNITE Jira打开了一个错误: https://issues.apache.org/jira/browse/IGNITE-12896 .非常感谢@alamar的建议.

解决方案

我建议将 excludeNeighbors 设置为 false .在您的情况下，它是 true ，不需要，当我将其设置为 false 时，我会获得正确的分区映射(当然，我也在本地运行所有四个节点)

环境属性就足够了，不需要手动将其添加到用户属性中.

I'm battling to configure Apache Ignite to distribute partitions in zone-aware manner. I have Ignite 2.8.0 with 4 nodes running as StatefulSet pods in GKE 1.14 split in two zones. I followed the guide, and the example:

• Propagated zone names into pod under AVAILABILITY_ZONE env var.
• Then using Web-Console I verified that this env var was loaded correctly for each node.
• I setup cache template in node XML config as in the below and created a cache from it using GET /ignite?cmd=getorcreate&cacheName=zone-aware-cache&templateName=zone-aware-cache (I can't see affinityBackupFilter settings in UI, but other parameters from the template got applied, so I assume it worked)

To simplify verification of partition distribution, I the partition number is set to just 2. After creating the cache I observed the following partition distribution:

Then I mapped nodes ids to values in AVAILABILITY_ZONE env var, as reported by nodes, with the following results:

AA146954 us-central1-a
3943ECC8 us-central1-c
F7B7AB67 us-central1-a
A94EE82C us-central1-c

As one can easily see, partition 0 pri/bak resides on nodes 3943ECC8 and A94EE82C which both are in the same zone. What am I missing to make it work?

Another odd thing, is then specifying partition number to be low (e.g. 2 or 4), only 3 out of 4 nodes are used). When using 1024 partitions, all nodes are utilized, but the problem still exists - 346 out of 1024 partitions had their primary/backup colocated in the same zone.

Here is my node config XML:

<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="
  http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans.xsd">

  <bean class="org.apache.ignite.configuration.IgniteConfiguration">

    <!-- Enabling Apache Ignite Persistent Store. -->
    <property name="dataStorageConfiguration">
      <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
        <property name="defaultDataRegionConfiguration">
          <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
            <property name="persistenceEnabled" value="true"/>
          </bean>
        </property>
      </bean>
    </property>

    <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
    <property name="discoverySpi">
      <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
        <property name="ipFinder">
          <!-- Enables Kubernetes IP finder and setting custom namespace and service names.  -->
          <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
            <property name="namespace" value="ignite"/>
          </bean>
        </property>
      </bean>
    </property>

    <property name="cacheConfiguration">
      <list>
        <bean id="zone-aware-cache-template" abstract="true" class="org.apache.ignite.configuration.CacheConfiguration">
          <!-- when you create a template via XML configuration, you must add an asterisk to the name of the template -->
          <property name="name" value="zone-aware-cache*"/>
          <property name="cacheMode" value="PARTITIONED"/>
          <property name="atomicityMode" value="ATOMIC"/>
          <property name="backups" value="1"/>
          <property name="readFromBackup" value="true"/>
          <property name="partitionLossPolicy" value="READ_WRITE_SAFE"/>
          <property name="copyOnRead" value="true"/>
          <property name="eagerTtl" value="true"/>
          <property name="statisticsEnabled" value="true"/>
          <property name="affinity">
            <bean class="org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction">
              <property name="partitions" value="2"/>  <!-- for debugging only! -->
              <property name="excludeNeighbors" value="true"/>
              <property name="affinityBackupFilter">
                <bean class="org.apache.ignite.cache.affinity.rendezvous.ClusterNodeAttributeAffinityBackupFilter">
                  <constructor-arg>
                    <array value-type="java.lang.String">
                      <!-- Backups must go to different AZs -->
                      <value>AVAILABILITY_ZONE</value>
                    </array>
                  </constructor-arg>
                </bean>
              </property>
            </bean>
          </property>
        </bean>
      </list>
    </property>

  </bean>
</beans>

Update: Eventually excludeNeighbors false/true makes or breaks zone awareness. I'm not sure why it didn't work with excludeNeighbors=false previously for me. I made some scripts to automate my testing. And now it's definite that it's the excludeNeighbors setting. It's all here: https://github.com/doitintl/ignite-gke. Regardless I also opened a bug with IGNITE Jira: https://issues.apache.org/jira/browse/IGNITE-12896. Many thanks to @alamar for his suggestions.

解决方案

I recommend setting excludeNeighbors to false. It is true in your case, it is not needed, and I get correct partitions mapping when I set it to false (of course, I also run all four nodes locally).

Environment property was enough, did not need to add it manually to user attributes.

这篇关于Apache Ignite区域(机架)感知的parititons的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！