Python: subprocess and pid-falls

Today I needed to be able to enter a description of an image in a Python program. I hadn't enough experience with PyQt: the Python bindings for Qt, yet; I had only browsed a few chapters in Mark Summerfield's excellent (so far) "Rapid GUI Programming with Python and Qt". I did, however, have quite some experience with WxPerl, the WxWidgets bindings for Perl. Moreover, I had been using wxPython in action by Noel Rappin and Robin Dunn - another excellent book in my opinion - as a great help while doing WxPerl coding...

So while I probably could have coded a much more elegant solution using wxPython I decided to keep things simple and just start an external program from the Python program I was writing, read the description via a prompt, and kill the external program.

Since I am using Python 2.5.2 I browsed the 2.5.2 Global Module Index. First, I considered to use os.system but shortly after my eye fell upon the subprocess module:

The subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to replace several other, older modules and functions, such as:

• os.system
• os.spawn*
• os.popen*
• popen2.*
• commands.*

So my first subprocess attempt became, using "The Eye of Gnome" (eog) as the program to display the picture:

from subprocess import Popen
:
:
p = Popen('eog ' + filename, shell=True)
description = raw_input("Enter description: ")

Which worked, the Eye of Gnome displayed the image I had specified in the variable filename and I was able to enter a description.

Next step: killing the Eye of Gnome. From the documentation of Popen I had already gathered that there was a way to obtain the process ID (pid) by accessing the pid attribute of the Popen object p I had created. However, a kill method didn't seem to be part of Popen in Python 2.5.2... After some Googling I learned that I had to write my own kill code using kill from the os module:

import os
import signal
from subprocess import Popen
:
:
p = Popen('eog ' + filename, shell=True)
description = raw_input("Enter description: ")
os.kill(p.pid, signal.SIGTERM)

However, when I tested the above code it did not work as expected; it didn't kill the Eye of Gnome...

First thing I did was adding an additional statement to the program reporting the process id (pid). And after some playing with the ps and kill command at the shell I discovered that the pid returned by the Popen object was one less than the pid of the process it started: eye of gnome.

And then, after some Googling, I found a post by wolfy: [issue4855] Popen with Shell=True should return the command's PID, not the shell's which described exactly the issue I was having with my code, and why: since I had set the shell parameter to the value of True, a shell was indeed created, and it's pid was returned, not the pid of the process I started in that shell. Well, to be honest, that all sounded very logical to me. The reply to wolfy's message, written by Alexander Belopolsky, made clear to me that this was clearly by design. So I decided to remove the shell=True part:

p = Popen('eog ' + filename

which resulted in

OSError: [Errno 2] No such file or directory

I replaced eog with the absolute path I had found using which eog, but still the same error. Next I also used the absolute path of the file I wanted to open; still the same error.

Since the reply by Belopolsky had some examples, I decided to try the list syntax used in his examples, instead of the string contatenation I used. And this worked:

import os
import signal
from subprocess import Popen
:
:
p = Popen(['eog', filename])
description = raw_input("Enter description: ")
os.kill(p.pid, signal.SIGTERM)

Note 1: if you are modifying your own code based on this blog post make sure that you don't leave the space after the command - eog in my case - since this will also result in the aforementioned OSError.

Note 2: one example by Belopolsky uses the kill method which seems to be available in more recent versions of subprocess. So if you use a Python more recent than what I am using, which is 2.5.2, you might be able to use p.kill() instead of os.kill(p.pid, signal.SIGTERM)

Looking back, while obvious, I would recommend to have a line added to the pid entry of the Popen Objects documentation:

Note that if you set the shell argument to True this attribute is set to the value of the process ID of the spawned shell.

Related

• subprocess module documentation - Python 2.5.2

Also today

• Python calling (overview)