如何使用Selenium优化Java中的向下滚动代码 [英] How to optimize the scroll-down code in java using selenium

问题描述

我正在使用 Java 在 MAVEN 中进行项目. 我必须获取一个URL，向下滚动它们，并获取此给定网页中其他项目的所有链接.

I am working a project in MAVEN using Java. I have to get a URL, scroll them down ,and get all the links of other items in this given web page.

到目前为止，我使用Selenium动态获取页面，并将其向下滚动，并获取链接.但这需要太多时间.请帮助我进行优化.

Till now, I get the page dynamically using Selenium , and scrolling them down, and fetch the links also. But it takes too much time. Please help me in optimize that.

示例:-，我正在处理一个页面，其链接为此处.

Example:-, I am working on a page , whose link is here.

我的问题:-

1. 使用硒滚动网页非常慢.我该如何优化呢? (建议使用其他任何方法
   做同样的事情或帮助我优化这一点)

1. Scrolling web page using selenium is very slow. How can I optimize this? (Suggest any other method
   to do the same or help me to optimize this one)

先谢谢了.寻找您的友善回应.

Thanks in advance. Looking for your kind response.

用于动态获取和滚动页面的代码:-

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import java.io.IOException;
import com.google.common.collect.*;
import java.io.File;
import java.util.ArrayList;
import java.util.Date;
import org.apache.commons.io.FileUtils;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxProfile;

/**
 *
 * @author jhamb
 */
public class Scroll_down {

    private static FirefoxProfile createFirefoxProfile() {
        File profileDir = new File("/tmp/firefox-profile-dir");
        if (profileDir.exists()) {
            return new FirefoxProfile(profileDir);
        }
        FirefoxProfile firefoxProfile = new FirefoxProfile();
        File dir = firefoxProfile.layoutOnDisk();
        try {
            profileDir.mkdirs();
            FileUtils.copyDirectory(dir, profileDir);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return firefoxProfile;
    }



    public static void main(String[] args)  throws InterruptedException{
        String url1 = "http://www.jabong.com/men/shoes/men-sports-shoes/?source=home-leftnav";
        System.out.println("Fetching %s..." + url1);
        WebDriver driver = new FirefoxDriver(createFirefoxProfile());


        driver.get(url1);  

        JavascriptExecutor jse = (JavascriptExecutor)driver;
        jse.executeScript("window.scrollBy(0,250)", "");
        for (int second = 0;; second++) {
            if (second >= 60) {
                break;
            }
            jse.executeScript("window.scrollBy(0,200)", "");
            Thread.sleep(1000);
        }
            String hml = driver.getPageSource();
        driver.close();


        Document document = Jsoup.parse(hml);

            Elements links = document.select("div");

        for (Element link : links) {
            System.out.println(link.attr("data-url"));
        }
    }
}

推荐答案

Selenium滚动基于Javascript.我不知道您对硒的目标，但是您没有断言可以比较代码中的任何内容? 当您确定自己的数据获取速度如此之快时，请不要使用任何睡眠方法. 睡眠方法会使硒变慢，但是是的，它正在等待元素被正确加载..... 由您决定，但要测试什么

Well Selenium scrolling is based on Javascript. I dont know your goal with selenium though, you have no assertion to compare anything in your code ? When you are so sure that your data fetching so fast then don't use any sleep methode. Sleep methods makes selenium slower, but yeah it is waiting until the element is properly loaded ..... It's up to you, what to test though

这篇关于如何使用Selenium优化Java中的向下滚动代码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！