使用PhantomJS时出现错误消息，随机中断 [英] Error message when using PhantomJS, breaks at random intervals

问题描述

我不断收到的错误消息如下:

The error message that I keep getting is the following:

assert.js:92
  throw new assert.AssertionError({
        ^
AssertionError: abnormal phantomjs exit code: -1073741819
    at Console.assert (console.js:102:23)
    at ChildProcess.<anonymous> (C:\Users\file_path...\node_modules\phantom\phantom.js:132:28)
    at ChildProcess.emit (events.js:98:17)
    at Process.ChildProcess._handle.onexit (child_process.js:810:12)
Program node app.js exited with code 8

这种中断是随机发生的，有时是在postgreSQL中插入一千多行之后，有时是在少数行之后.

The break happens at random, sometimes after inserting over a thousand rows into postgreSQL, sometimes after just a handful of rows.

根据我在整个代码中放置的许多不同的console.logs，我相当确定错误是在代码中的以下函数中发生的.另外，我认为assert.js:92来自柴:

I'm fairly sure that the error is occuring in the following function inside of my code, based on a lot of different console.logs that I have put throughout the code. Also, I think that assert.js:92 is from Chai:

function getNetworkTraffic(networkUrl,senderEmail) {
    phantom.create(function (ph) {
        ph.createPage(function (page) {
            page.set("onResourceRequested", function (req) {
                referrerValue = "";
                referrerName = "";
                linkRedirectUrl="";
                console.log('Fetching network traffic...')
                for (i in req.headers) {
                    allReferrals = req.headers[i]
                    if (allReferrals.name == "Referer"){
                        referrerName = allReferrals.name
                        referrerValue = allReferrals.value
                    }
                }
                linkUrl = req.url
                if(req.redirectURL){
                    linkRedirectUrl = redirectURL             
                }
                singleReq = {"referrerName":referrerName,"referrerValue":referrerValue,"requestUrl":linkUrl,"redirectURL":linkRedirectUrl, "parent_url":networkUrl, "source": "email", "senderEmail":senderEmail}
                // insertNetworkTrafficPg(singleReq)
            });
            page.set("onResourceReceived", function (res) {
                linkRedirectUrl = "";
                responseUrl = res.url
                if(res.redirectURL){
                    linkRedirectUrl = res.redirectURL      
                }
                singleRes = {"responseUrl":responseUrl,"redirectURL":linkRedirectUrl,"parent_url":networkUrl,"source": "email", "senderEmail":senderEmail}
                // insertNetworkTrafficPg(singleRes)
            });
            try{
                page.open(networkUrl, function (status) {
                    if (status !== 'success') {
                        console.log('FAIL to load the address');
                    }
                    console.log('Opening web address...');
                    ph.exit();
                });
            } catch(err) {
                console.log(err)
            }
        });
    }, {
        dnodeOpts: {
            weak: false
        }
    });    
}

推荐答案

可以将phantomjs与node一起使用，但请记住，这是一座桥梁.另外，它实际上并不是要进行大量刮取的操作，因此，也许您打开了太多线程并导致了堆栈溢出，但又出现了另一条消息.您可能想尝试使用类似 python-shell 之类的方法来运行python脚本您的抓取内容.

It's possible to use phantomjs with node, but keep in mind that this is a bridge. Also, it's really not intended for a lot of scraping, so perhaps you are opening too many threads and its a stackoverflow, but breaking with another message. You might want to try using something like python-shell to run a script for python to do your scraping.

这篇关于使用PhantomJS时出现错误消息，随机中断的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！