从MySQL表自动进行URL检查 [英] Automated URL checking from a MySQL table

问题描述

好的，我在MySQL表中有一个URL列表.我希望脚本自动检查表中的每个链接是否为404，然后再希望它存储URL是否为404，以及存储最后一次检查的时间.

Okay, I have a list of URLs in a MySQL table. I want the script to automatically check each link in the table for 404, and afterward I want it to store whether the URL was 404'd or not, as well as store a time for last checked.

即使没有人运行脚本，这甚至有可能自动执行吗?也就是说，几天没有人访问该页面，但是即使没有人访问该页面，它也会自动运行测试.

Is this even possible to do automatically, even if no one runs the script? ie, no one visits the page for a few days, but even with no one visiting the page, it automatically ran the test.

如果可能的话，我该怎么做才能做到这一点?

If its possible, how could I go about making a button to do this?

推荐答案

无需使用CURL，如果请求失败(除2xx以外的任何其他HTTP代码)，file_get_contents($url);将返回false，这对于执行以下操作可能更有用您正在尝试做的事，例如:

No need to use CURL, file_get_contents($url); will return false if the request fails (any other HTTP code other than 2xx), which might be more useful for what you're trying to do, an example:

function urlExists($url)
{
    return (bool) @file_get_contents($url);
}

如果URL返回有用的内容，则返回true，否则返回false.

Will return true if the URL returns useful content, false otherwise.

编辑:这是一种更快的方法(仅请求标头)，并且第一个字节而不是整个页面:

EDIT: Here is a faster way (it only requests the headers) and the first byte instead of the whole page:

function urlExists($url)
{
    return (bool) @file_get_contents($url, false, null, 0, 1);
}

urlExists('https://stackoverflow.com/iDontExist'); // false

---

但是，与结合使用问题使用这样的内容可能更明智:

---

However, in combination with your other question it may be wiser to use something like this:

function url($url)
{
    return @file_get_contents($url);
}

$content = url('https://stackoverflow.com/');

// request has failed (404, 5xx, etc...)
if ($content === false)
{
    // delete or store as "failed" in the DB
}

// request was successful
else
{
    $hash = md5($content); // md5() should be enough but you can also use sha1()

    // store $hash in the DB to keep track of changes
}

或者，如果您使用的是PHP 5.1+，则只需执行以下操作:

Or if you're using PHP 5.1+ you only have to do:

$hash = @md5_file($url);

当URL加载失败时，

$hash将为false，否则将返回内容的MD5哈希.

$hash will be false when the URL fails to load, otherwise it will return the MD5 hash of the contents.

从@Jamie . =)

这样，您只需要发出一个请求即可，而不是两个. =)

This way you only have to make one request instead of two. =)

这篇关于从MySQL表自动进行URL检查的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！