Cron和噩梦 [英] Cron and nightmarejs

问题描述

我无法设法用噩梦来运行我的cron。

I can't manage to run my cron with nightmarejs.

函数get_data（）的第一次迭代工作效果很好，但之后cron重新启动，函数不会再次触发。

The first iteration of the function get_data() works great but after that the cron restarts and the function will not be triggered again.

也不会记录抓取已结束。

Also "crawl ended" is never logged.

您知道我的代码有什么问题吗？

Do you know what's wrong with my code?

日志

1
cron
data fetched
2
cron
3
cron

-

var Nightmare = require('nightmare')
var nightmare = Nightmare({
  typeInterval: 300,
  show: true,
  executionTimeout: 120000,
  gotoTimeout: 120000
});
let data = ""

-

var get_data = function(){
  return new Promise(function(resolve, reject) {
    nightmare
    .goto('https://url.com')
    .type('[name=email]', '')
    .wait(1000)
    .type('[name=email]', 'myemail')
    .wait(1000)
    .type('[name=password]', '')
    .wait(1000)
    .type('[name=password]', 'mypassword')
    .click('[type=submit]')
    .wait(5000)
    .goto('https://url.com')
    .wait(25000)

    .evaluate(function (page, done) {

      return document.body.innerText
      done()
    })
    .end()
    .then(function (result) {
      data = result
    })
    .then(function(data){
      return fs.writeFile("./data.txt", data, function(err) {
        if(err) {
          console.log(err)
          reject(err)
        }
        resolve(data)
      });
    })
    .catch(function(error){
      reject(error)
    })
  })
}

-

var i = 0
var job = new CronJob('0 */20 * * * *', function() {
    ++i
    console.log(i)
    console.log("cron")
    get_data()
  }, function () {
    console.log("crawl ended")
  },
  true
);

job.start();

推荐答案

p>

Couple of things that jump out right away.

.evaluate(function (page, done) {

      return document.body.innerText
      done()
    })

并且可能永远不会返回并导致超时错误。您不是传递页面的参数，这意味着 done 将是未定义的。将以上更改为：

This won't do what you expect it to do, and will likely never return and cause a timeout error. You're not passing in an argument for page, which means done will be undefined. Change the above to:

.evaluate(function (done) {

      return document.body.innerText
      done()
    })

>

Second, this:

.then(function(data){
      return fs.writeFile("./data.txt", data, function(err) {
        if(err) {
          console.log(err)
          reject(err)
        }
        resolve(data)
      });
    })

...重新定义 data 。我不认为你在上一次输出数据变量设置，这应该总是输出 undefined ，我想。

... redefines data. I don't think you're putting out the data variable set in the previous then, this should always output undefined, I'd think. Be careful with your closures.

第三，也许最重要的是：

Third, and perhaps most importantly:

.evaluate(function (page, done) {

      return document.body.innerText
      done()
    })
    .end() // <== this might be a problem
    .then(function (result) {
      data = result
    })

由于噩梦仅定义一次，因此您将结束唯一的实例。如果您尝试在循环的第二次迭代中对已结束的实例执行操作，它将不会被重新创建，并且无法正常工作。或者取 .end（），并将其移动到脚本的结尾，或者为每次迭代创建一个新的Nightmare实例。

Since nightmare is only defined once, you're ending the only instance you have. It will not be recreated, and will not work properly if you try to execute actions on the ended instance in the second iteration of your loop. Either take the .end() out and move it to the end of your scripts, or create a new Nightmare instance for every iteration.

这篇关于Cron和噩梦的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！