Perl programmer for hire: download my resume (PDF).
John Bokma MexIT
freelance Perl programmer

Is there an infinite number of web pages?

Saturday, July 26, 2008 | 0 comments

Today I read an article on a Dutch web site, which was based on a blog entry on the official Google blog. At first I thought that the translator had come up with the following silly statement, but it turned out to be in the post on the official Google blog as well:

Strictly speaking, the number of pages out there is infinite [...]

Which sounds impressive, especially coming from two Software Engineers of Google's Web Search Infrastructure Team, but it's (strictly) provable false.

First, for all practical purposes there is a (relative low) limit on the maximum length of the unique address of a web page; the uniform resource locator or URL for short. But even if there wasn't such a (relative low) limit, the following URL would be a valid one: http://example.com/ followed by 1,000,000,000,000 times the letter 'a'. To obtain that page, a web browser has to transfer over 1,000,000,000,000 characters to the web server since the web server needs the address to return the right web page. Assuming that each character has a size of one byte, this means that one terabyte of data has to be transferred from the browser to the web server. Impractical of course.

But if that example doesn't convince you, note that an infinite number of web pages implies that one can keep adding letters to the aformentioned URL. And hence it's quite easy to construct a URL that's so huge that the sending of the request for that page from the web browser to the web sever will never finish in a life time, even if you never got disconnected.

An example given by the two Google software engineers to somewhat justify their claim is:

[...] for example, web calendars may have a "next day" link, and we could follow that link forever, each time finding a "new" page.

Based on my explanation above, one can't follow a "next day" link forever, because after a while it will take forever to even send the address of the next page to the web server, because that is what following a link means: sending the URL of the link to the web server.

So dear Jesse Alpert and Nissan Hajaj, how impressive it might sound on a the official Google Blog; there is no infinite number of web pages.

Infinity related

Also today

Please post a comment | read 0 comments | RSS feed