在Thread实现代码不能正常工作之后 [英] After Thread implementation code not working as assumed
问题描述
我有一个循环，里面我放置了Execute方法，负责向某个服务器发送Web请求。它的工作正常，但它需要太长时间的请求响应。我决定改为实现线程。实现代码后不能正常工作。让我先解释一下代码在做什么。我的执行方法采用了几个变量：第一个采用xmlsoap文件请求路径，第二个文件路径将为每个元素存储结果，第三个是库存元素名称，第四个Timeout文件和我发送这些请求的最后一个服务器。工作方式是Execute方法正在取第一个参数并发出请求然后如果结果正在将其保存到第二个参数文件（带有元素名称的DONE文件）并进入下一个。有可能对于特定请求元素将是超时，这意味着结果未出现，那么元素的名称将保存到Timeout文件中（保存在catch子句中完成）并且DONE文件未创建。如果我同步这样做，一切都很完美。但是在实现解决方案之后，线程结果非常快（好）的新结果（每个可能的检索结果元素的DONE文件）出现在我的文件夹中。但是......从调查中我发现有太多的元素名称会转到TimeOut文件，甚至其中一些也是重复的。方法是如果元素X转到TimeOut文件，则不会创建包含结果的DONE文件，因为有超时等于没有结果。如果它确实存在某些元素的超时，那么元素的名称应该被放到TimeOutFile并且只被放置一次。第二件事我看到一些元素在TimeOut文件中，并且也创建了Done文件响应（奇怪）。你能帮我解决一下在这种情况下会出现什么问题吗？希望解释得足够好。
这是我使用的方法：
public static void 执行（ string xmlFileRequest， string xmlFileResult， string InventoryElement， string TimeOutFile，Server server）
{
HttpWebRequest request = null ;
尝试
{
System.Net.ServicePointManager.DefaultConnectionLimit = 100 ;
request =（HttpWebRequest）WebRequest.Create（server.ServerIP）;
// request.ServicePoint.ConnectionLimit = 1000;
request.Headers。添加（ @ SOAP：Action）;
request.ContentType = text / xml; charset = \utf-8 \< /跨度>;
request.Accept = text / xml;
request.Method = POST;
request.KeepAlive = false ;
var soapEnvelopeXml = new XmlDocument（）;
soapEnvelopeXml.Load（xmlFileRequest）;
尝试
{
使用（流媒体流） = request.GetRequestStream（））
{
soapEnvelopeXml.Save（stream）;
}
}
catch （例外情况）
{
}
尝试
{
使用（WebResponse response = request.GetResponse（））
{
使用（ var rd = new StreamReader（response.GetResponseStream（）））
{
string soapResult = rd.ReadToEnd（）;
var xdoc = new XmlDocument（）;
xdoc.LoadXml（soapResult）;
xdoc.Save（xmlFileResult）;
}
}
}
catch （WebException e）
{
Monitor.Enter （锁定装置）;
{
使用（StreamWriter sw = File.AppendText（TimeOutFile））
{
sw.WriteLine（InventoryElement） ;
}
}
Monitor.Exit（locker）;
}
}
catch
{
}
最后
{
if （request！= null ）
{
request = null ;
}
}
}
和该方法的Thread循环：
列表<线程> myThreads = new List< Thread>（）;
foreach （MakeRequest item in makeRequestList）
{
Thread thread = new 线程（（）= > 执行（item.file_READYTOTEST，item.file_DONEFILE，item.Inventory_name ，IO.TimeOutFP，item.server））;
thread.Start（）;
myThreads.Add（thread）;
}
foreach （ var thread < span class =code-keyword> in myThreads）
thread.Join（）;
如果您需要更多信息或有些不明确的地方请写信给我。谢谢。
在我看来，主要问题不在您的代码中，而是在服务器端。从你的问题，我可能会认为，如果你按顺序执行相同的操作，你的客户端就可以找到。请证实。如果您没有使用新代码明确尝试，请执行此操作。最简单的尝试方法是：确保makeRequestList只有一个元素。更好的是，只需在调用线程中直接在循环下调用执行。如果它运行缓慢并且（慢慢地）给出令人满意的结果，服务器端几乎肯定是一个问题。
现在，让我们看看你的线程可能给你什么。映像您并行执行多个线程，每个线程使用不同的远程主机。在这种情况下，增益是显而易见的：瓶颈要么在网络中间某处的网络流量中，要么在服务中。在所有情况下，您的CPU使用率都不是瓶颈：客户端系统很少在其他地方广泛使用网络，因此您的线程在等待状态中花费大量时间，等待网络流中的更多数据，从而不会花费CPU资源。一切都会好的。
如果您轮询相同的服务主机，会发生什么？这得看情况。如果该服务是一个具有负载平衡和其他强大功能的大型Web工厂，那么您的线程方法将起作用。但是如果主机只是一台拥有2-4个内核的主机，那么Web服务/站点/应用程序就会成为瓶颈，因此大多数线程都会被浪费掉。你的案子是什么？请求列表中有什么？如果这些是对同一主机的请求，那么拥有所有线程可能不是解决方案。由于线程本身的开销，这种解决方案可能会使事情变得更慢。您可能需要一组顺序请求，或者一次只需要几个活动线程，比如2-6。您可以尝试找出接近最佳的数字（当然，当您的功能得到修复时）。
但是你的问题更严重。超时。如果我在上面第一段中提出的假设是正确的，那么服务器端就是一个问题。你用太多的请求压倒了主机，响应花了太多时间进行这样的设置;你得到很多超时。除非您解决服务上的问题，否则您无法处理这种情况。即使您使数据下载更顺序，其他一些客户端也可能会压倒服务器部分。这是不对的。
现在，关于代码的一些问题。首先，在显示的第二个代码片段中创建所有线程并不好。你最好使用线程池中的线程，或者重用一些固定的线程集，我个人更喜欢这样做。一个线程在阻塞调用时休眠（例如使用EventWaitHandle）并在任务准备好启动时唤醒。综合方法是使用System.Collections.Concurrent.BlockingCollectio< code> n< t>< / t>。另请参阅我的文章，其中我解释了所有重要细节并提供了有趣的使用示例：用于线程通信和线程间调用的简单阻塞队列 [ ^ ]。
我已经很感激你将多个参数传递给一个线程了。但是，我知道更好的方法，使用线程包装器。请看我过去的答案：
如何将ref参数传递给线程 [ ^ ]，
更改线程（生产者）启动后的参数 [ ^ ]，
C＃中的多线程 [ ^ ];
参见：在webservice中运行一个永远不会终止的作业/线程/进程（asp.net） [ ^ ]，
使代码线程安全 [ ^ ]。
我会质疑一般的代码设计，你从文件中获取数据，放到其他文件，许多文件......但是，我不太清楚你的问题，也许这是有意义的。
最大的问题是使用... catch {}来阻止异常传播。很少有这种情况有意义，主要是如果你必须修复第三方代码中的某些缺陷，这些缺陷无法修补。
最后，我希望你能理解任何硬代码立即常量，例如100，@SOAP：Action，依此类推，不好;使用显式声明的常量（甚至是资源，数据文件等，在其他情况下）要好得多。
-SA
I got a loop which inside i've placed Execute method which is responsible for sending web request towards some server. Its working fine but its taking too long foreach request response. I decided to implement threads instead. Unfortunetly after implementation code is not working as expected. Let me explain you firstly what code is doing. My execute method taking couple variables: first one takes xmlsoap file request path, second file path which result will be stored for each element, third is for inventory element name, fourth Timeout file and last one server towards which i am sending those requests. The way of working is Execute method is taking first parameter and sending out request then if the result is comming up saves it to second parameter file (DONE file with the element name) and going next. It could be possible that for specific request element will be timeout that means result is not comming out then the name of element is saving into Timeout file (saving is done in catch clause) and DONE file is not creating. All works perfectly if i am doing it synchronusly. But after implemented solution Threads results comming up very fast (good) new results cames around (DONE file for each possitive retreived result element) were appearing in my folder . BUT... From investigation i've found out that too many elements names are going to TimeOut file and even some of them are duplicated there. The way is if element X went to TimeOut file then DONE file containing result is not created because there was timeout which is equal to no result. If its really true there was timeout for some element the name of element should be putted to TimeOutFile and only once. Second thing i saw some elements are inside TimeOut file and Done file response is created as well (strange). Could you please help me out what can be wrong in this situation? Hope explained you enough good.
This is the method i am using:
public static void Execute(string xmlFileRequest, string xmlFileResult, string InventoryElement, string TimeOutFile, Server server)
{
HttpWebRequest request = null;
try
{
System.Net.ServicePointManager.DefaultConnectionLimit = 100;
request = (HttpWebRequest)WebRequest.Create(server.ServerIP);
//request.ServicePoint.ConnectionLimit = 1000;
request.Headers.Add(@"SOAP:Action");
request.ContentType = "text/xml;charset=\"utf-8\"";
request.Accept = "text/xml";
request.Method = "POST";
request.KeepAlive = false;
var soapEnvelopeXml = new XmlDocument();
soapEnvelopeXml.Load(xmlFileRequest);
try
{
using (Stream stream = request.GetRequestStream())
{
soapEnvelopeXml.Save(stream);
}
}
catch (Exception ex)
{
}
try
{
using (WebResponse response = request.GetResponse())
{
using (var rd = new StreamReader(response.GetResponseStream()))
{
string soapResult = rd.ReadToEnd();
var xdoc = new XmlDocument();
xdoc.LoadXml(soapResult);
xdoc.Save(xmlFileResult);
}
}
}
catch (WebException e)
{
Monitor.Enter(locker);
{
using (StreamWriter sw = File.AppendText(TimeOutFile))
{
sw.WriteLine(InventoryElement);
}
}
Monitor.Exit(locker);
}
}
catch
{
}
finally
{
if (request != null)
{
request = null;
}
}
}
and the Thread loop over that method:
List<Thread> myThreads = new List<Thread>();
foreach (MakeRequest item in makeRequestList)
{
Thread thread = new Thread(() => Execute(item.file_READYTOTEST, item.file_DONEFILE, item.Inventory_name, IO.TimeOutFP, item.server));
thread.Start();
myThreads.Add(thread);
}
foreach (var thread in myThreads)
thread.Join();
If you need more information or something is not clear please write me question. Thank you.
It seems to me that major problem is not in your code, but on the server side. From your question, I probably can assume that if you do the same operations sequentially, your client works find. Please confirm it. If you did not explicitly tried it with your new code, please do it. The simplest way to try it is this: make sure yourmakeRequestListhas only one element. Better yet, simply callExecuteunder the loop directly, in the calling thread. If it works slowly and gives your satisfactory results (slowly), the server side is almost certainly a problems.
Now, let's see what your threading can possibly give you. Imaging you execute several threads in parallel, each working with different remote host. In this case, the gain is apparent: the bottleneck is either in the network traffic somewhere in the middle of the network, or at the service. In all cases, your CPU usage is not a bottleneck: the client system rarely extensively use the network elsewhere, so your threads spend considerable time in wait state, waiting for more data in networks streams, thus not spending CPU resources in vain. Everything would be good.
What happens if you poll the same very service host? Well, it depends. If the service is a big Web factory with load balancing and other powerful things, your threading approach will work. But if the host is just a single host computer with the 2-4 cores, the Web service/site/application can become a bottleneck, so most of your threads would be wasted. What is your cases? what's in the request lists? If those are the requests to the same host, having all your threads might not be a solution. Such solution can make things even slower, because of the overhead of the threading itself. You might need a sequential set of request, or just a few active threads at a time, say, 2-6. You can try it out to figure out the close-to-optimum number (when your functionality is fixed, of course).
But your problem is more serious. Timeouts. If my assumption I formulated above in my first paragraph is correct, the server side is a problem. You overwhelm the host with too many request, the responses take too much time for such settings; and you get many timeouts. Not much you can do with this situation, unless you address the problem on the service. Even if you make your data downloads "more sequential", some other client may overwhelm the server part. This is not right.
Now, about some problems of your code. First, creating all threads in the second fragment of code shown is not good. You should better use the thread from thread pool, or reuse some fixed set of threads, which I personally prefer to do. A thread is sleeping at the blocking call (such as with the use ofEventWaitHandle) and awaken when a task is ready to start. On comprehensive approach is using theSystem.Collections.Concurrent.BlockingCollectio<code>n<t></t>. See also my article where I explain all the important detail and provide interesting usage samples: Simple Blocking Queue for Thread Communication and Inter-thread Invocation[^].
I already appreciated your way of passing multiple parameters to a thread. However, I know even better way, with a thread wrapper. Please see my past answers:
How to pass ref parameter to the thread[^],
Change parameters of thread (producer) after it is started[^],
MultiThreading in C#[^];
see also: Running exactly one job/thread/process in webservice that will never get terminated (asp.net)[^],
Making Code Thread Safe[^].
I would question the general code design, where you get data from files, put to other files, many files… However, I don't know your problem well enough to be sure, maybe it makes sense.
The big problem would be the blocking of the exception propagation using… catch { }. There are few cases when it makes sense, mostly if you have to fix some defects in 3rd-party code which is not accessible for patching.
And, finally, I hope you understand that hard-coding any immediate constants like your100,@"SOAP:Action", and so on, is not good; it's much better to use explicitly declared constants (and even resources, data files, etc., in other cases).
—SA
这篇关于在Thread实现代码不能正常工作之后的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
