Limitations of link checking software

Ben Logan - Published: 26th Feb 2009 11:45 GMT

Wouldn't it be nice to press a button and scan an entire website and be presented with a full list of pages and content types, in order to help you establish a starting point for an accessibility or functionality review?

Sadly the scanning tools we have come across require a degree of manual interpretation in order to get more accurate results.

Why would you scan a website?

When working on a testing project, or even being able to quote for a testing project for a client, you need to know the potential scope of work in terms of number of pages and assets.

What can appear to be a small site when browsing through a web browser can actually turn out to be a rather large site with complex multi-stage forms and login processes as well as functionality that might not be immediately apparent.

Link checking with Xenu's Link Sleuth

One of the best link checking tools we have come across is Xenu's Link Sleuth (I know its a funny name!) written by Tilman Hausherr and available to download for free

We have downloaded and tested several link checking tools, but have found this the best one to work with.

Working with Xenu

Xenu is straightforward to use. You simple enter the URL of a web page and click OK. It's typically very fast to check all of the links but sometimes this can run for over an hour on a large scale site with thousands of pages and assets.

Mapping large sites is a major drain for Xenu. If you want a site map, run the scan without external links, setting your depth to something modest.

Screenshot

Interpreting the results

Xenu does not necessarily check all types of links and in some respects is limited to conventional hypertext links. We have seen it struggle with dynamic pages such as ASPX. It does not, for example, check links in <link> or <meta> elements.

Screenshot

Once the scanning is complete you have the option to view a report, or work with the results set window.

Links that show up as broken aren't always bad - it may be that the connection was down at the time the link was checked or that the timeout (set by the PC's TCP/IP settings and not controllable by the program) was too short. You can re-run the link check on the broken links only as a further verification or double click on a link to open it in your browser. Connection problems are quite common and you may need to check a link over a period of days to be sure it really is gone.

As well as the tabular display of information, Xenu can produce a report listing broken and redirected links, sorted by page. I have only one problem with this report; it lists the URL but not the title of the page, which can sometimes make it hard to figure out exactly which link on a page is bad - to get that information you have to go back to the tabular display.

back to top