使用JSOUP登录ConEd网站 [英] Using JSOUP to Login to ConEd website

问题描述

我已经广泛阅读了这个和我如何尝试了许多不同的变体，但我无法让它工作。

I have read extensively about how do this and I have tried a number of different variations, but I can't get it to work.

基本上，我只想登录ConEdison网站并查看我的结算历史记录。以下是我的内容：

Basically, I just want to login to the ConEdison website and scrape my billing history. Here is what I have:

Connection.Response loginForm = Jsoup.connect("https://apps.coned.com/cemyaccount/NonMemberPages/Login.aspx?lang=eng")
                    .data("_LASTFOCUS","")
                    .data("_EVENTTARGET","")
                    .data("_EVENTARGUMENT","")
                    .data("_VIEWSTATE", viewState)
                    .data("_EVENTVALIDATION", eventValidation)
                    .data("ctl00$Main$Login1$UserName", username)
                    .data("ctl00$Main$Login1$Password", password)
                    .data("ctl00$Main$Login1$LoginButton", "Sign In")
                    .userAgent("Mozilla/5.0")
                    .method(Method.POST)
                    .execute();

            Map<String, String> loginCookies = loginForm.cookies();

            Document document = Jsoup.connect("https://apps.coned.com/CEMyAccount/CSOL/BillHistory.aspx?lang=eng")
                    .cookies(loginCookies)
                    .get();

            Elements data = document.select("table.ctl00_Main_lvBillHistory_Table1");

            //checking if it found the right page
            System.out.println("document: " + document);
            //checking if it found the table
            System.out.println("data: " + data);

我知道信息是正确的（虽然我不知道我是否真的需要传递数据没有值的参数）。

I know the information is correct (though I don't know if I really need to pass the data parameters with no values).

我没有收到任何错误，只是打印出登录页面（ https://apps.coned.com/cemyaccount/NonMemberPages/Login.aspx?lang=eng ）

I am not getting any errors, just printing out the login page (https://apps.coned.com/cemyaccount/NonMemberPages/Login.aspx?lang=eng)

我们非常感谢任何帮助。

Any help would be greatly appreciated.

谢谢

编辑

所以，我现在确信我无法访问内部页面，因为在POST之后 https://apps.coned.com/cemyaccount/NonMemberPages/Login.aspx?lang=eng ，设置了3个cookie，但随后它将GET请求发送到 https://apps.coned.com/cemyaccount/SessionTransfer.aspx?dir=2asp&url=https://apps.coned.com/csol/MainHome.asp?src=DOTNET 然后 https://apps.coned.com/csol/SessionTransfer.asp?dir=2asp&guid=3c413f48-d2eb-434a-896b -f9c4eb100714& url = https://apps.coned.com/csol/MainHome.asp?src = DOTNET& frm = 了解其他Cookie，然后再访问主页

So, I am now convinced that I was not able to get to the internal page, because after the POST to https://apps.coned.com/cemyaccount/NonMemberPages/Login.aspx?lang=eng, 3 cookies are set, but then it sends GET requests to https://apps.coned.com/cemyaccount/SessionTransfer.aspx?dir=2asp&url=https://apps.coned.com/csol/MainHome.asp?src=DOTNET then to https://apps.coned.com/csol/SessionTransfer.asp?dir=2asp&guid=3c413f48-d2eb-434a-896b-f9c4eb100714&url=https://apps.coned.com/csol/MainHome.asp?src=DOTNET&frm= for additional cookies before going to the homepage

有谁知道我如何能够遵循所有这些重定向并最终获得cookie？

Does anyone know how I can follow all these redirects and get the cookies in the end?

这是我目前拥有的，但我无法获得cookie来自POST电话。

Here is what I currently have, but I cannot get the cookies from the POST call.

Response response = Jsoup
                        .connect("https://apps.coned.com/cemyaccount/NonMemberPages/Login.aspx?lang=eng")
                        .method(Method.GET)
                        .execute();

                Map<String, String> cookies = response.cookies();
                cookies.put("NSC_DpoFe_Bqqt-TTM-pme", response.cookie("NSC_DpoFe_Bqqt-TTM-ofx"));
                System.out.println("response cookies: " + cookies);

                response = Jsoup
                        .connect("https://apps.coned.com/cemyaccount/NonMemberPages/Login.aspx?lang=eng")
                        .header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8")
                        .header("Accept-Encoding", "gzip, deflate")
                        .header("Accept-Language", "en-US,en;q=0.8")
                        .header("Connection", "keep-alive")
                        .cookies(cookies)
                        .header("Host", "apps.coned.com")
                        .referrer("https://apps.coned.com/cemyaccount/NonMemberPages/Login.aspx?lang=eng&login=0")
                        .userAgent("Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36")
                        .data("_LASTFOCUS", "")
                        .data("_EVENTTARGET", "")
                        .data("_EVENTARGUMENT", "")
                        .data("_VIEWSTATE", viewState)
                        .data("_EVENTVALIDATION", eventValidation)
                        .data("ctl00$Main$Login1$UserName", username)
                        .data("ctl00$Main$Login1$Password", password)
                        .data("ctl00$Main$Login1$LoginButton", "Sign In")
                        .followRedirects(false)
                        .method(Method.POST)
                        .execute(); 

                System.out.println("post cookies: " + response.cookies());
                cookies.putAll(response.cookies());
                System.out.println("response cookies: " + cookies);

                response = Jsoup
                        .connect("https://apps.coned.com/cemyaccount/SessionTransfer.aspx?dir=2asp&url=https:"
                                + "//apps.coned.com/csol/MainHome.asp?src=DOTNET")
                        .cookies(cookies)
                        .followRedirects(false)
                        .method(Method.GET)
                        .execute();

                cookies.putAll(response.cookies());
                System.out.println("response cookies: " + cookies);
                String guid = response.header("location");

                response = Jsoup
                        .connect("https://apps.coned.com/csol/SessionTransfer.asp?dir=2asp&guid="
                                + guid + "&url=https://apps.coned.com/csol/MainHome.asp"
                                + "?src=DOTNET&frm=")
                        .cookies(cookies)
                        .method(Method.GET)
                        .execute();

                cookies.putAll(response.cookies());
                System.out.println("response cookies: " + cookies);

                Document dataPage = Jsoup
                        .connect("https://apps.coned.com/CEMyAccount/CSOL/BillHistory.aspx?lang=eng")
                        .cookies(cookies)
                        .get();

                System.out.println("data page: " + dataPage);

                Elements data = dataPage.select("table.ctl00_Main_lvBillHistory_Table1");

                System.out.println("data: " + data);

在输出中，我得到所有的cookie，但POST的cookie是空白的。

In the output I get all the cookies, except the POST cookies which are blank.

推荐答案

答案很简单 - 带下划线的标题有2个下划线，我只用了1. Doh

The answer was painfully simple - the headers with the underscores had 2 underscores, I was only using 1. Doh

这篇关于使用JSOUP登录ConEd网站的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！