John Bokma Perl
freelance Perl programmer

Overture Search Term Suggestion Tool

Example program | 0 comments

The Perl program on this page is an example, for educational purposes only, of how to use the LWP::UserAgent, HTTP::Request::Common modules to download a web page and using HTML::TreeBuilder to extract information from it.

Overture's Search Term Suggestion Tool (also known as the keyword selector tool) is a tool that can be used by for SEO (Search Engine Optimization) purposes, for example to assist in improving the content of a website. The Perl program given below is a simple interface to this search term suggestion tool.

Since the result page is cluttered with a lot of visual markup I use HTML::TreeBuilder to create a parse tree of the page. The score and term information is in the second table, so it is obtained from the parse tree and the information is extracted from each table data cell. Because the information is prefixed with  , which is stored as a character with code value 225, substr is used to get rid of it.

# suggestion.pl - Overture Search Term Suggestion Tool interface
#
#  Copyright, 2004-2005 By John Bokma, http://johnbokma.com/
#
# $Id: suggestion.pl 1097 2008-09-30 19:19:53Z john $ 

use strict;
use warnings;

use LWP::UserAgent;
use HTML::TreeBuilder;


unless ( @ARGV ) {

    print "usage: suggestion.pl term\n";
    print "    example: suggestion.pl perl script\n";
    exit( 1 );
}

my $ua = LWP::UserAgent->new;

my $response = $ua->post(

    'http://inventory.overture.com/d/searchinventory/suggestion/',
    [
        'mkt value'  => "us",
        'lang value' => "en_US",
        'term'       => join ( ' ', @ARGV ),
    ]
);

$response->is_success or die $response->status_line;

# parse the content into a tree
my $root = HTML::TreeBuilder->new_from_content( $response->content );

# get the second table (if any)
my $table = ( $root->look_down( '_tag' => 'table' ) )[ 1 ];

if ( $table ) {

    # get all table data and remove the first character
    # ( , stored as char 225)
    my ( @data ) = map { substr $_->as_text, 1 }
        $table->look_down( '_tag' => 'td' );

    # @data consist of score, term, score, term tuples
    while ( my ( $score, $term ) = splice @data, 0, 2 ) {

        printf "%8d - %s\n", $score, $term;
    }

} else {

    print "No suggestions\n";
}

# clean up
$root->delete;

Example of usage

Example of program usage and output:

suggestion.pl perl script
    5775 - perl script
     667 - free perl script
     282 - perl cgi script
      94 - free perl cgi script
      70 - perl sample script
      64 - ftp perl script
      59 - perl script tutorial
      57 - perl who is script
      54 - cisco perl script
      51 - example perl script
      50 - java perl script
      47 - perl script email
      42 - download perl script
      38 - perl run script
      35 - perl script using vb
      34 - perl script simple
      34 - perl script window
      30 - perl script upload
      27 - free page perl php script
      26 - book guest perl script
      26 - free login perl script
      26 - perl script test
      25 - form perl script

Search term related

Please post a comment | read 0 comments | RSS feed