John Bokma

Tunneling an API callback to development

April 24, 2017

A project I am currently working on uses a web based API for keyword research. Because results are not available right away there are two options: polling for results or providing a callback URL. Of course, the latter is more efficient. So today I looked into setting up a reverse tunnel over SSH from a virtual private server I rent to my development machine, a 2014 Mac mini.

On the development side I had written a small Perl Web Server Gateway Interface (PSGI) application based on Toby Inkster's A Simple Plack/DBI Example blog post. The small test program I wrote is similar to the one given below:

# -*- mode: cperl -*-

use strict;
use warnings;

use Plack::Request;

my $app = sub {

    my $req = 'Plack::Request'->new( shift );
    if ($req->method eq 'POST') {
        print "POST method called\n";
        my @names = qw( foo bar baz qux );
        for my $name ( @names ) {
            my $value = $req->parameters->{ $name };
            if ( defined $value ) {
                print "$name: '$value'\n";
            }
        }
    }
    else {
       print "Unauthorized access\n";
    }
    my $res = $req->new_response( 200 );
    
    return $res->finalize();
};

The API uses the POST method to callback and provide the program with result data.

Next, I used plackup to run the PSGI Perl Program. I made it listen on port 9090, a port number that I was sure about not being in use.

plackup --host 127.0.0.1 --port 9090 callback.psgi

I built the reverse tunnel to my VPS as follows:

ssh -f -N -R 9090:localhost:9090 user@example.com

Due to firewalling and configuration of the sshd daemon on the VPS this does not open port 9090 on the Internet facing side of the VPS.

Based on instructions read in nginx config for http/https proxy to localhost:3000 I added the following lines to the server entry in file virtual.conf located in /etc/nginx/conf.d:

server {

    :
    :
    :

    location /cb/ {
        proxy_pass          http://localhost:9090;
        proxy_set_header    Host             $host;
        proxy_set_header    X-Real-IP        $remote_addr;
        proxy_set_header    X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_set_header    X-Client-Verify  SUCCESS;
        proxy_set_header    X-Client-DN      $ssl_client_s_dn;
        proxy_set_header    X-SSL-Subject    $ssl_client_s_dn;
        proxy_set_header    X-SSL-Issuer     $ssl_client_i_dn;
        proxy_read_timeout 1800;
        proxy_connect_timeout 1800;
    }
}

Note that this already assumes the callback URL is accessed over https which is not yet the case in my test set up. After restarting nginx I did the following tests:

curl http://127.0.0.1:9090/

Which, being a GET request and not POST, reported:

Unauthorized access

Next, I tried the POST method:

curl http://127.0.0.1:9090/ --data 'foo=3.14'

Which, as expected, reported:

POST method called
foo: '3.14'

Next, I verified that port 9090 was not directly accessible on the VPS. As expected I got a "Connection refused.":

curl http://example.com:9090/ --data 'foo=9&bar=20'
curl: (7) Failed to connect to example.com port 9090: Connection refused

Finally, I used curl to POST data to the callback URL as follows:

curl http://example.com/cb/ --data 'baz="Hello, World!"'

As expected, the output was as follows:

POST method called
baz: '"Hello, World!"'

home - blog - email

I am a freelance Perl programmer for hire, download my up-to-date resume (PDF).