The Perl program on this page is an example, for educational purposes only, of how to use the LWP::UserAgent, HTTP::Request::Common modules to download a web page and using HTML::TreeBuilder to extract information from it.
Overture's Search Term Suggestion Tool (also known as the keyword selector tool) is a tool that can be used by for SEO (Search Engine Optimization) purposes, for example to assist in improving the content of a website. The Perl program given below is a simple interface to this search term suggestion tool.
Since the result page is cluttered with a lot of visual markup I use HTML::TreeBuilder to create a parse tree of the page. The score and term information is in the second table, so it is obtained from the parse tree and the information is extracted from each table data cell. Because the information is prefixed with , which is stored as a character with code value 225, substr is used to get rid of it.
# suggestion.pl - Overture Search Term Suggestion Tool interface
#
# © Copyright, 2004-2005 By John Bokma, http://johnbokma.com/
#
# $Id$
use strict;
use warnings;
use LWP::UserAgent;
use HTML::TreeBuilder;
unless ( @ARGV ) {
print "usage: suggestion.pl term\n";
print " example: suggestion.pl perl script\n";
exit( 1 );
}
my $ua = LWP::UserAgent->new;
my $response = $ua->post(
'http://inventory.overture.com/d/searchinventory/suggestion/',
[
'mkt value' => "us",
'lang value' => "en_US",
'term' => join ( ' ', @ARGV ),
]
);
$response->is_success or die $response->status_line;
# parse the content into a tree
my $root = HTML::TreeBuilder->new_from_content( $response->content );
# get the second table (if any)
my $table = ( $root->look_down( '_tag' => 'table' ) )[ 1 ];
if ( $table ) {
# get all table data and remove the first character
# ( , stored as char 225)
my ( @data ) = map { substr $_->as_text, 1 }
$table->look_down( '_tag' => 'td' );
# @data consist of score, term, score, term tuples
while ( my ( $score, $term ) = splice @data, 0, 2 ) {
printf "%8d - %s\n", $score, $term;
}
} else {
print "No suggestions\n";
}
# clean up
$root->delete;Example of program usage and output:
suggestion.pl perl script
5775 - perl script
667 - free perl script
282 - perl cgi script
94 - free perl cgi script
70 - perl sample script
64 - ftp perl script
59 - perl script tutorial
57 - perl who is script
54 - cisco perl script
51 - example perl script
50 - java perl script
47 - perl script email
42 - download perl script
38 - perl run script
35 - perl script using vb
34 - perl script simple
34 - perl script window
30 - perl script upload
27 - free page perl php script
26 - book guest perl script
26 - free login perl script
26 - perl script test
25 - form perl script