Spring-添加低优先级的多线程服务(不影响生产性能) [英] Spring - add a low priority multi threaded service (no impact to production performance)
问题描述
我们有一个Spring应用程序，我想添加一个服务，该服务将处理具有多个线程的10K ID，但将作为后台进程，而不会实时影响生产.
We have a Spring application, I want to add a service that will handle 10Ks IDs with multiple threads but will be as background process without impact production realtime.
服务将更新数据库并发送外部提供商请求.
Service will update database and send external providers requests.
我不希望服务影响/影响生产性能/时间安排，我想以较低的优先级对每个ID执行操作
I don't want service to impact/effect production performance/timing, I want to execute operation on each ID in a low priority
我阅读了以前的帖子关于在Executer中设置优先级，但我希望可以降低此特定Executer范围之外的所有其他线程的优先级.
I read previous post about setting priority in Executer, but I want low priority to all other threads that can be outside this specific Executer scope.
使用ThreadPoolExecutor回答是否与我的案子更相关?
Is answer using ThreadPoolExecutor more relevant to my case?
ThreadPoolExecutor threadPool = new ThreadPoolExecutor(1, numOfWorkerThreads, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>());
threadPool.setThreadFactory(new OpJobThreadFactory(Thread.NORM_PRIORITY-2));
public final static class OpJobThreadFactory implements ThreadFactory {
private int priority;
public OpJobThreadFactory(int priority) {
this(priority, true);
}
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r, namePrefix + threadNumber.getAndIncrement());
t.setDaemon(daemon);
t.setPriority(priority);
}
}
- 甚至可以使用
Thread.MIN_PRIORITY - maybe even use
Thread.MIN_PRIORITY
或者我可以使用Executors.newCachedThreadPool()
创建一个线程池，该线程池根据需要创建新线程，但是将在可用时重用以前构造的线程.这些池通常会提高执行许多短期异步任务的程序的性能.
还我应该使用Spring bean吗?，因为我需要按需/请求创建池，因此似乎不需要/错误
Also should I use Spring bean? because I need to create pool on demand/request so it seems not needed/wrong
编辑 我是否应该使用弹簧执行器来获得此任务或其他监视工具?
EDIT Should I use Spring Actuator to get this task or other monitoring tool?
Spring Boot Actuator模块通过提供生产就绪型功能(如运行状况检查，审计，度量收集，HTTP跟踪等)来帮助您监视和管理Spring Boot应用程序.所有这些功能都可以通过JMX或HTTP端点进行访问.
Spring Boot Actuator module helps you monitor and manage your Spring Boot application by providing production-ready features like health check-up, auditing, metrics gathering, HTTP tracing etc. All of these features can be accessed over JMX or HTTP endpoints.
推荐答案
我想澄清一下这个问题
什么是线程优先级?根据Java SE文件
what is a thread priority? According to java SE Docs
每个线程都有一个优先级.具有较高优先级的线程优先于具有较低优先级的线程执行.
Every thread has a priority. Threads with higher priority are executed in preference to threads with lower priority.
即使您创建具有优先级的线程，也不能完全保证优先级较低的线程先执行，您可能不得不阻止具有较低优先级的线程，直到执行其他线程为止
Even though you create threads with priority it does not completely guarantee that threads with lower priority get executed first you may have to block the thread with lower priority until other threads are executed
对于小型Java程序，您可以自己处理线程执行，但对于较大的程序，建议您使用来自java.util.concurrent包的vanilla Java中的Executor框架，或者使用spring TaskExecutor.框架，您可以异步执行任务，这些任务是作为主要任务的一部分在后台执行的.
For small java programs you can handle the thread execution by yourself but for larger programs, it's recommended either you use the Executor Framework from vanilla Java which is from the package java.util.concurrent or use the spring TaskExecutor.By using both frameworks you can execute tasks asynchronously that is executing them in the background as part of the main task.
对生产的影响
例如，主要任务将是调用其余端点(即/account )，并在调用帐户端点时要向客户发送欢迎电子邮件，这是第三方API调用，可以使用Executor Framework或Spring TaskExecutor异步执行它们，以异步方式执行它们，即作为后台进程，它们不会对当前API产生影响，但是由于您正在同一JVM中运行线程，因此它们肯定会对生产服务器产生影响他们共享共同的记忆.如果创建了多个线程且未将其破坏，则服务器肯定会关闭.
The main task, for example, will be a call to your rest endpoint i.e /account and on calling the account endpoint you want to send welcome emails to customers which are a third party API call which can be executed asynchronously either using Executor Framework or Spring TaskExecutor on executing them asynchronously i.e as background process they will not have an impact on the current API but it will surely have an impact on the production server since you are running the threads within the same JVM and they share common memory. if there are a number of threads created and not destroyed .you server will surely go down.
因此，使用Executor Framework或Spring TaskExecutor不能保证您不会影响当前的生产，它肯定会提高所调用的其余API的性能.因为它是异步执行的，并且在您遇到其他问题时
So using Executor Framework or Spring TaskExecutor does not guarantee you that it will not affect your current production it will surely increase the performance of the rest API that is called. since it's executed asynchronously and on your other questions
我可以使用Executors.newCachedThreadPool()
是，如果您有许多短期任务，例如更新数据库中的单个列或仅触发一次休息端点，并且不适合批量加载或执行某些后端作业，因为该作业会更新10000条记录，为每个任务创建更多数量的线程，您肯定会遇到内存问题.
yes if you have a number of the short-lived task such as updating a single column in a database or triggering a rest endpoint only once and it's not suitable for bulk loading or executing some backend job which updates 10000 records because it will create larger number of threads for each task and you will surely face memory problems.
另外，我应该使用Spring bean吗?因为我需要按需/请求创建一个池，所以似乎不需要/错误
Also, should I use Spring bean? because I need to create a pool on-demand/request so it seems not needed/wrong
是的，您可以使用ThreadPoolTaskExecutor使用 https://egkatzioura.com/2017/10/25/spring-and-async/这是一个很好的开始
yes you can ThreadPoolTaskExecutor use https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/concurrent/ThreadPoolTaskExecutor.html as per the docs Setting "queueCapacity" to 0 mimics Executors.newCachedThreadPool() .so you can use either Spring TaskExecutor or use Executor Framework from vanilla Java I personally recommend using Spring TaskExecutor which has more feature for a good start on using Spring you can refer the tutorial https://egkatzioura.com/2017/10/25/spring-and-async/ which is a good start
最终结论:如果您希望仅异步或后台执行任务，则可以使用java的Executor Framework或Spring TaskExecutor，但是这两者都会对生产产生影响，因为它们使用相同的JVM.根本不想影响生产，那么我建议在不同的服务器上创建单独的Spring Boot应用程序，并从新应用程序进行数据库调用或服务调用，并将其公开为其余端点，并使用Spring Task从您的主应用程序异步调用这些端点执行者.
final verdict: if you are looking to execute the task only asynchronous or a background process you can use either use Executor Framework from java or Spring TaskExecutor but both will have an impact on production since they use the same JVM .if you do not want to impact production at all then I recommend creating separate spring boot app on a different server and make the database calls or service call from the new app and expose it as a rest endpoint and invoke these endpoints asynchronously from your main app using Spring Task Executor.
春季任务执行器: https://egkatzioura.com/2017/10/25/spring-and-async/
Spring Task Executor: https://egkatzioura.com/2017/10/25/spring-and-async/
Java执行器框架: https://stackabuse.com/concurrency-in-java-the-executor-framework/
Java Executor Framework : https://stackabuse.com/concurrency-in-java-the-executor-framework/
用于使用低优先级的线程: https://medium.com/@daniyaryeralin/弹簧框架中基于优先级的线程池d74b91b51dcb
for using threads with low priority : https://medium.com/@daniyaryeralin/priority-based-thread-pooling-in-spring-framework-d74b91b51dcb
对不起，如果答案太长:)
Sorry if the answer is too long :)
这篇关于Spring-添加低优先级的多线程服务(不影响生产性能)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
