作业窃取配置在Apache Ignite中不起作用 [英] Job Stealing Configuration not working in Apache Ignite

问题描述

我有以下配置文件

    <bean abstract="true" id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
    <property name="peerClassLoadingEnabled" value="true"/>

    <property name="includeEventTypes">
        <list>
            <!--Task execution events-->
            <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
            <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
            <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
        </list>
    </property>

    <property name="metricsUpdateFrequency" value="10000"/>
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                    <property name="addresses">
                        <list>
                            <!-- In distributed environment, replace with actual host IP address. -->
                            <value>127.0.0.1:47500..47509</value>
                            <value>127.0.0.1:48500..48509</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>

      <!-- Enabling the required Failover SPI. -->
    <property name="failoverSpi">
        <bean class="org.apache.ignite.spi.failover.jobstealing.JobStealingFailoverSpi"/>
    </property>

    <property name="collisionSpi">
        <bean class="org.apache.ignite.spi.collision.jobstealing.JobStealingCollisionSpi">
            <property name="activeJobsThreshold" value="50"/>
            <property name="waitJobsThreshold" value="0"/>
            <property name="messageExpireTime" value="1000"/>
            <property name="maximumStealingAttempts" value="10"/>
            <property name="stealingEnabled" value="true"/>
        </bean>
    </property>
</bean>

闭包将按预期在网格中的服务器节点上执行.

The closure gets executed over the server nodes in the grid as expected.

在执行闭包期间通过在网格中执行以下命令来添加新节点时

When we add a new node by executing the below command to the grid during the execution of closure

现有节点确认在网格中添加了新节点，但是闭包未分发到新添加的节点.

The existing nodes acknowledge the addition of the new node in the grid but the closure is not distributed to the newly added node.

下面是我的闭包实现

@Override
public AccruedSimpleInterest apply(SimpleInterestParameter simpleInterestParameter) {

    BigDecimal si = simpleInterestParameter.getPrincipal()
            .multiply(new BigDecimal(simpleInterestParameter.getYears()))
            .multiply(new BigDecimal(simpleInterestParameter.getRate())).divide(SimpleInterestClosure.HUNDRED);

    System.out.println("Calculated SI for id=" + simpleInterestParameter.getId() + " SI=" + si.toPlainString());
    return new AccruedSimpleInterest(si, simpleInterestParameter);
}

下面是主要课程

public static void main(String... args) throws IgniteException, IOException {
    Factory<SimpleInterestClosure> siClosureFactory = FactoryBuilder.factoryOf(new SimpleInterestClosure());

    ClassPathResource ress = new ClassPathResource("example-ignite-poc.xml");
    File file = new File(ress.getPath());

    try (Ignite ignite = Ignition.start(file.getPath())) {
        System.out.println("Started Ignite Cluster");
        IgniteFuture<Collection<AccruedSimpleInterest>> igniteFuture = ignite.compute()
                .applyAsync(siClosureFactory.create(), createParamCollection());
        Collection<AccruedSimpleInterest> res = igniteFuture.get();
        System.out.println(res.size());
    }nter code here

推荐答案

据我所知，Job Stealing SPI需要您实现一些其他API才能正常工作.

As far as my understanding goes, Job Stealing SPI requires you to implement some additional APIs in order to work.

请参见关于用户列表的讨论:

关于窃取SPI的一些评论:

Some remarks about job stealing SPI:

1)您有一些节点可以执行某些计算作业的任务.

1)You have some nodes that can proceed the tasks of some compute job.

2)默认情况下，任务将在公共线程池中执行: https://apacheignite.readme.io/docs/thread-pools#section-public-pool

2)Tasks will be executed in public thread pool by default: https://apacheignite.readme.io/docs/thread-pools#section-public-pool

3)如果某些节点线程池繁忙，则可以执行一些计算任务在其他节点上执行.

3)If some node thread pool is busy then some task of compute job can be executed on other node.

在接下来的情况下，它将不起作用:

In next cases it will not work:

1)如果您为计算任务选择特定的节点

1)In case if you choose specific node for your compute task

2)如果您进行相似性调用(与上面相同，但是节点将是通过亲和力映射选择)

2)In case if you do affinity call (the same as above but node will be choose by affinity mapping)

这篇关于作业窃取配置在Apache Ignite中不起作用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！