我应该在后台使用puppeteer吗? [英] Should I use puppeteer as part of my backend?

问题描述

让我们说我想创建一个前端，多个用户可以在该前端向服务器发送请求，以将某个网站上的某些链接抓取，伪造者能够同时/至少足够快地处理该请求，或者我应该考虑使用其他方法?

Lets say I want to create a front-end where multiple users can send a request to a server that scrapes some links off a website, would puppeteer be able to process it concurrently/atleast fast enough or should I consider a different method?

还有，有没有可能在移动设备上的无头浏览器实例(启用js)中加载页面的方法?如果可能的话，我该如何使用javascript编写自己的无头浏览器?

Also, is there any possible way to load a page in a headless browser instance(with js enabled) on a mobile device? How could I go about coding my own headless browser in javascript if its possible?

推荐答案

您始终可以通过PM2部署node.js实例，并同时生成多个进程来处理传入的负载.您应将总进程限制为您的设备可用的内核总数，否则可以正常工作.

You can always deploy your node.js instance via PM2 and spawn multiple processes concurrently to handle the incoming load. You should restrict the total processes to the total number of cores available to your box, but otherwise this would work fine.

这是否可以处理您的负载取决于服务器，工作负载以及预期的整体性能.您需要进行一些负载测试才能确定您的系统.

Whether or not this could handle your load depends on your server, the workload, and the expected throughout. You'd need to do some load testing to make that determination for your system.

这篇关于我应该在后台使用puppeteer吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！