XPages HTTP线程挂起 [英] XPages HTTP Threads hanging

问题描述

我在这里打开了一个关于提高XPage中30gb工作流应用程序性能的问题。有很多建议，但大多数都涉及回收，改进代码等。实际上已经解决了速度问题的实际问题 - 应用程序属性中的高级选项卡（请参阅我的上一篇文章）

现在我的应用程序运行得非常好，速度很快，人们很高兴，但服务器仍然会定期崩溃。或者我应该说，HTTP变得没有响应，并且在极端情况下运行100％的CPU，因此Domino也很迟钝但仍在运行。

我一直用<正在监视HTTP线程/ p>

告诉http show thread state

在大多数情况下，我看到80个空闲的http线程，或做某事但很快就释放了。自上次更新应用程序以来，我们更专注于回收SSJS中的notes对象，我想我们会看到挂起的http线程的结束，但它仍然存在。

我几乎肯定这不是一个导致这个问题的无限循环，因为我已经与最终用户确认的2个案例完全不同，并且没有循环据我所知。

1. 用户正在编辑文档，按工作流程按钮批准并将其发送给下一个用户。他们正在使用Chrome。 Chrome选项卡上的旋转圆圈启动，然后服务器应运行工作流代理，发送电子邮件，然后在浏览器上关闭页面。我注意到有2或3个挂起的http线程在一小时后没有被释放所以我联系了用户，她告诉我页面没有刷新，但旋转的圆圈仍在旋转，表明服务器正在做某事。我检查了日志和工作流代理HAS运行，电子邮件已发送并且文档已更新。她刷新了页面，现在可以看到它已被更新，但无论出于何种原因，Chrome坐在那里耐心等待，从未收到LS代理已运行的消息。我使用notesAgent.runOnServer并返回结果整数以确认代理是否已运行。如果它返回1（我认为）那么该页面应该关闭，否则它应该显示一条消息。页面永远不会刷新，所以它没有显示任何内容，但代理确实完成了。




2. 晚上的用户最终得到了大约15个挂起的http线程。在日志中我可以看到她试图多次重新加载页面。然后搜索了她想要的文档，然后尝试打开它。当我检查她说她搜索了文件时，搜索页面显示了结果（重复控制），每次她点击文件打开它都没有发生。所以她甚至没有进入文档，但每次尝试后线程都挂了。我从笔记日志中获取了URL并尝试了它，文档打开正常。我运行相同的搜索，文档打开正常。我直接向她发送了一份文件链接，并为她打开了。奇怪!!

有没有办法诊断这种行为，因为现在我必须打开多米诺骨牌管理员整天运行tell http show ....命令，密切关注它以确保线程不会挂起。它通常到午餐时间，服务器需要重启，这是垃圾。

请帮助我理智： - ）

解决方案

我用了一些Java转储分析他们的内存泄漏，虽然我在那里发现了大量的信息，不知道我在寻找什么意味着我没有得出任何结论。

纯粹是偶然的，我需要在系统中使用随机导致这些线程锁的文档，并且我手动运行在处理这些文档时调用的Workflow代理。我花了一分钟才意识到发生了什么，但是当试图根据文档内容生成一个mime电子邮件时，代理似乎陷入了循环。

当预定的代理卡在Domino中的无限循环中时，只需通过查看Admin Client中的代理管理器就可以轻松发现它。您将看到它不断消耗CPU并且永远不会完成。

当XPage调用一个卡住的代理时，没有任何线索（至少在我的情况下）代理管理器正在运行且HTTP Server任务不运行表明它正在做任何与众不同的事情。这就是为什么我最初认为没有无限循环，但我完全错了！

我添加了一些代码来计算mime电子邮件中已经达到的循环次数生成器例程，如果它达到某个任意高的值，则表示它被卡在循环中，并添加了一个中断。 EtVoilà！没有更多的挂起的http线程！

这是一个很好的借口，可以通过整个系统并修复一些旧的（差）代码，并整理一切。最终虽然是代理人而不是导致问题的xpage。谢谢大家的建议。

I've had a question open on here about improving the performance of my 30gb workflow application in XPages. There were lots of suggestions but most involve recycling, improving code etc. and what actually fixed the issues with speed are not often talked about - the advanced tab in the application properties (see my last post)

Now I have an application that runs really well, it is fast and people are happy BUT the server still periodically crashes. Or I should say, HTTP becomes unresponsive and in extreme cases runs 100% CPU so Domino is also sluggish but still running.

I've been monitoring the HTTP threads with

tell http show thread state

And in most cases I see 80 http threads that are idle, or doing something but quickly released. Since the last update to the application where we have been way more focused on recycling the notes objects in SSJS, I thought we would see the end of the hung http thread but it's still there.

I'm almost positive that it's not an infinite loop that's causing this problem because 2 cases that I have confirmed with the end users are completely different and there are no loops as far as I can tell.

1. User is editing a document, presses a workflow button to approve and send it on to the next user. They are using Chrome. The spinning circle on the chrome tab starts, the server is then supposed to run the workflow agent, send emails, and then close the page on the browser. I noticed that there were 2 or 3 hung http threads that hadn't been released after an hour so I contacted the user and she told me the page hadn't refreshed but the spinning circle was still spinning in chrome suggesting the server was doing something. I checked the logs and the workflow agent HAS run, emails have been sent and the document is updated. She refreshed the page and can now see that it's been updated but for whatever reason Chrome sat there waiting patiently and never received the message that the LS Agent had run. I use notesAgent.runOnServer and return the resulting integer to confirm if the agent has run or not. If it returns 1 (i think) then the page is supposed to close, otherwise it should display a message. The page never refreshed so it didn't display anything, but the agent did complete.

2. A user in the evening ended up with about 15 hung http threads. In the logs I could see she was trying to reload the page multiple times. Then there was a search for the document she wanted, and then more attempts to open it. When I checked she said she searched for the document, the search page showed the results (in a repeat control), and every time she clicked the document to open it nothing happened. So she didn't even get in to the document yet the threads were hanging after each attempt. I got the URL from the notes log and tried it, document opens fine. I ran the same search, document opens fine. I send her a link to the document directly and it opens fine for her. Weird!!

Is there ANY way to diagnose this sort of behaviour because right now I have to have domino admin open running the tell http show.... command all day keeping an eye on it to make sure threads aren't hanging. It usually gets to lunch time and the server needs a reboot, which is rubbish.

Please help my sanity :-)

解决方案

I used some of the Java dumps to analyse them for memory leaks and while I found a ton of information in there, not really knowing what I was looking for meant I found no conclusions.

Purely by chance I needed to work with a document in the system that randomly causes these thread locks and I manually ran the Workflow agent that gets called when these documents are processed. It took me a minute to realise what was going on but the agent appeared to be getting stuck in a loop when trying to generate a mime email based on the content of the document.

When a scheduled agent gets stuck in an infinite loop in Domino it's easy to spot just by looking at Agent Manager in the Admin Client. You will see it constantly consuming CPU and never finishing.

When an XPage calls an agent that gets stuck however, there is no clue (at least in my case) that agent manager is running and the HTTP Server task doesn't show that it's doing anything out of the ordinary. This is why I initially thought there were no infinite loops but I was completely wrong!

I added some code to count the number of loops that had been reached in the mime email generator routine and added a break if it reached some arbitrarily high value indicating it was stuck in a loop. Et Voilà! No more hung http threads!

This was a great excuse to go through the entire system and fix some of the old (poor) code in there, and tidy everything up. Ultimately though it was the agent not the xpage that was causing the issue. Thanks for everyone's suggestions though.

这篇关于XPages HTTP线程挂起的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！