Skip to content
ReDock ReDock help

Find broken links on your sites with ReDock Web

About once a week ReDock walks each connected site the way a visitor would. It reads the pages and follows the links, then lists every link that answered with an error, with the page it was found on, so you can fix it before your client’s customers run into it.

Nothing needs setting up. The walk is an ordinary visit to the public site, so it works whether or not the site has ReDock Connect on it.

Read the list

Section titled “Read the list”

  1. Open Broken links, under Work in the sidebar. The line under the heading adds up the broken links across the sites walked, and the ones ReDock could not check.

    The Broken links page, open from the Work group of the sidebar. Under the heading a line reads 3 broken links across 2 sites walked, and 1 we could not check. One site’s card is marked Failed and says it was walked 3 hours ago, 18 pages read and 121 links checked, and that the walk stopped early because the site is bigger than the 20 pages one walk reads, with a Walk this site now button. Below it are three tables. Broken lists three links with the page each is on, what is wrong (a 503, a 404, and a page WordPress marks as a 404) and when each was first seen. Could not check lists one link that gave no answer within 8 seconds. Long redirects lists one link that works after 4 redirects.

  2. Read a site’s card. It says when the site was walked and how much of it was read, then lists what was found under up to three headings.

    • Broken: the link, the page it is on with the link’s own words under it, what is wrong, and when it was first seen.
    • Could not check: links that gave no answer. They are not broken and never counted as broken, because the cause is as likely to be on ReDock’s side as the site’s.
    • Long redirects: links that work after four or more hops, which is slow for a visitor and usually worth updating.

    A broken link keeps the date it was first seen for as long as it stays broken, so you can tell a client how long it has been that way. The same dead link on four pages makes four rows, one for each place it needs fixing.

  3. When a site has more findings than its card shows, See all opens that site’s own page, with every finding and a table of its last walks. ReDock keeps the last eight walks of each site.

    The last walks table on one site’s own page, marked 1 of 8 kept, with columns for when, pages, links, broken, could not check, how it ended and why it ran. Its one row reads 3 hours ago, 18 pages, 121 links, 3 broken, 0 could not check, Stopped early, On schedule.

  4. Fix the link on the page it names. The next walk that reads that page and finds the link working takes it off the list.

Walk a site now

Section titled “Walk a site now”

Walk this site now moves a site to the front of the queue, and its card then says the walk runs within the hour. A newly connected site gets its first walk within the hour anyway. After that each site waits its turn, about six days, and the one that has waited longest goes first.

A panel headed Nothing walked yet, saying ReDock walks each site the way a visitor would, about once every 6 days, and lists every broken link with the page it is on, that the first walk of each site starts within the hour, and that one site can be moved to the front with the button on its line. Below it a site card marked Not used yet reads not walked yet, with a Walk this site now button.

What counts as broken

Section titled “What counts as broken”

The What this check does panel. It says the walk starts at the homepage and the sitemap, reads up to 20 pages and checks up to 150 links, that a 404 or a server error is broken, that a page WordPress marks as not found is broken too, that a link working after four or more redirects is listed as slow, that no answer at all is could not check and never broken, that a site which will not let it in at the front page is not walked further, and that it reads robots.txt first and waits 300 milliseconds between requests to the same server. A second list says what it deliberately does not do.

A link is broken when it answers 404 or a server error. So is a page on the site that answers 200 while WordPress itself marks it as page not found. If the only sign is the page’s title, the list says so and quotes the title, because a title is weaker evidence than WordPress’s own mark.

These are not counted:

  • A link that gave no answer, a server that asked ReDock to slow down, or one that timed its own request out. They go under Could not check.
  • A page on somebody else’s site that answers 200. ReDock asks other sites for their headers only, never for the page, so it never guesses from someone else’s content.
  • A theme’s own friendly “not found” page that answers 200 without WordPress’s mark. Telling that apart from a real page would mean guessing at words.
  • A missing address that redirects to a real page. Whether it is the right page is something no crawler can know.

When a walk stops early

Section titled “When a walk stops early”

One walk reads up to 20 pages and checks up to 150 links. A bigger site is covered over several walks: its card says the walk stopped early, that the list is what was found among the pages it reached, and that the rest is read next time. A walk that stopped early never takes a link off the list, because it has not looked at the pages it did not reach.

When a site does not let ReDock in

Section titled “When a site does not let ReDock in”

Its robots.txt asks crawlers to stay out. ReDock obeys, so there is nothing to report, and the card says so and tells you how to let the check run: allow ReDock-LinkCheck in robots.txt. An address robots.txt rules out is never fetched and never reported.

A site card marked Off, saying the site’s robots.txt asks crawlers not to read it, that ReDock obeys that so there is nothing to report here, and that to let this check run you allow ReDock-LinkCheck in robots.txt. It was last tried 5 hours ago, and there is a Walk this site now button.

Its front page asks for a password, as a staging site or a firewall might. The walk stops at the front door and says so. A site behind a password is not reported as a site full of broken links.

How gentle the walk is

Section titled “How gentle the walk is”

One request at a time, never two at once, with at least 300 milliseconds between requests to the same server. When a site’s robots.txt asks for more room with Crawl-delay, it gets it. Links to other sites are asked for their headers, not their pages.

If a hosting company asks what it is

Section titled “If a hosting company asks what it is”

The checker names itself in every request as ReDock-LinkCheck, along with the address of a public page, app.redock.xyz/crawler. That page says what the checker is, what it fetches and how often, and how to stop it. Nobody needs an account to read it.

A public page headed The ReDock link checker. It says that if you found ReDock-LinkCheck in your access log, this page is what it is. It explains that the checker looks for broken links with an ordinary visit that changes nothing and signs in to nothing, then lists roughly once a week per site, at most 20 pages and 150 link checks in one visit, one request at a time with at least 300 milliseconds between requests, robots.txt and Crawl-delay obeyed, and HEAD requests wherever a server accepts them. Under How to stop it are two lines to add to robots.txt.

To stop the check on one site, add this to that site’s robots.txt:

User-agent: ReDock-LinkCheck
Disallow: /

It stops on the next run, and the card for that site then says its robots.txt asks crawlers not to read it.

Open ReDock Web, or create an account if you do not have one yet.