我需要删除Google警报网址 [英] I need to strip a Google Alerts URL

问题描述

作为序言，我知道与此类似，但我使用的是C#，而不是Java，Python，Php.一些线程为单个URL提供了一种解决方案，这不是通用的.感谢您不对我进行举报.

To preface, I know there are similar threads about this, but I am using C#, not java, or python, or Php. Some threads provide a solution for a single URL, which is not universal. Thanks for not flagging me.

因此，我正在使用Google快讯通过电子邮件获取文章的链接.我已经写了一个程序，可以将URL从电子邮件中剥离出来，另外还有一个程序可以对网站进行抓取.我的问题是Google警报电子邮件中的链接看起来像这样:

So I am using Google Alerts to get links to articles via email. I have already written a program that can strip the URLs out of the email as well as another program to scrape the websites. My issue is that the links in the google alerts email look like this:

https://www.google.com/url?rct=j& sa = t& url = http://www.foxnews.com/health/2016/08/19/virtual-reality-treadmills-help-prevent-falls-in-elderly.html& ct = ga& cd = CAEYACoTOTc2NjE4NjYyNzMzNzc3NDcyODIaYzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzDzTz usg = AFQjCNGyK2EyVBLoKnNkdxIBDf8a_B3Ung .是的，丑陋.

因为这会通过Google重定向到实际文章，所以我的抓取程序不适用于这些链接.我从这里的问题和其他来源尝试了一百万种不同的RegEx.我设法剥离了所有内容，直到实际文章的http://为止，但它的尾端仍然将其拧紧.到目前为止，这就是我所拥有的.它们现在看起来像:

Because this redirects to the actual article through google, my scraping program does not work on these links. I have tried a million different RegExs from questions here and other sources. I managed to strip off everything up until the http:// of the actual article but it still has the tail end that screws it up. Here is what I have so far. They now look like:

http://www.foxnews.com/health/2016/08/19/virtual-reality-treadmills-help-prevent-falls-in-elderly.html&ct= ga& cd = CAEYACoTOTc2NjE4NjYyNzMzNzc3NDcyODIaODk2NWUwYzRjMzdmOGI4Nzpjb206ZW46VVM& usg = AFQjCNGyK2EyVBLoKnNkdxIBDf8a_>

http://www.foxnews.com/health/2016/08/19/virtual-reality-treadmills-help-prevent-falls-in-elderly.html&ct=ga&cd=CAEYACoTOTc2NjE4NjYyNzMzNzc3NDcyODIaODk2NWUwYzRjMzdmOGI4Nzpjb206ZW46VVM&usg=AFQjCNGyK2EyVBLoKnNkdxIBDf8a_B3Ung

    private List<string> GetLinks(string message)
    {
        List<string> list = new List<string>();
        Regex urlRx = new Regex(@"((http|ftp|https):\/\/[\w\-_]+(\.[\w\-_]+)+([\w\-\.,@?^=%&amp;:/~\+#]*[\w\-\@?^=%&amp;/~\+#])?)", RegexOptions.IgnoreCase);

        MatchCollection matches = urlRx.Matches(message);
        foreach (Match match in matches)
        {
            if(!match.ToString().Contains("news.google.com/news") && !match.ToString().Contains("google.com/alerts"))
            {
                string find = "=http";
                int ind = match.ToString().IndexOf(find);                    
                list.Add(match.ToString().Substring(ind+1));
            }                
        }
        return list;
    }        

一些摆脱结尾的帮助会很棒，无论是新的RegEx还是一些额外的代码.预先感谢.

Some help getting rid of the endings would be awesome, be it a new RegEx or some extra code. Thanks in advance.

推荐答案

您可以使用HttpUtility.ParseQueryString检索查询字符串的url部分.它位于System.Web命名空间中(需要引用).

You can use HttpUtility.ParseQueryString to retrieve the url part of the query string. It is located in the System.Web namespace (reference required).

var uri = new Uri("https://www.google.com/url?rct=j&sa=t&url=http://www.foxnews.com/health/2016/08/19/virtual-reality-treadmills-help-prevent-falls-in-elderly.html&ct=ga&cd=CAEYACoTOTc2NjE4NjYyNzMzNzc3NDcyODIaODk2NWUwYzRjMzdmOGI4Nzpjb206ZW46VVM&usg=AFQjCNGyK2EyVBLoKnNkdxIBDf8a_B3Ung");
var queries = HttpUtility.ParseQueryString(uri.Query);
var foxNews = queries["url"]; //http://www.foxnews.com/health/2016/08/19/virtual-reality-treadmills-help-prevent-falls-in-elderly.html

这篇关于我需要删除Google警报网址的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！