TIL that /bin/sh -c has subtle portability issues

I tried to use a Python auto-reloader program and found that it ran correctly under macOS but was failing to correctly restart the monitored program under Linux (Debian, to be specific).

After some investigation, I found that it used subprocess.Popen with shell=True; this basically runs /bin/sh -c <specified command>, which itself runs the command. We initially get the following process hierarchy:

[PID X] python
 \_ [PID Y] /bin/sh -c <specified command>

But then things start to differ depending on shell used. If /bin/sh is zsh (default under macOS) or bash, the shell directly replaces itself with the command being run (using exec), and we get:

[PID X] python
 \_ [PID Y] <specified command>

But if /bin/sh is dash (default under Debian), it runs the command in a subprocess and waits for it to exit. This gives:

[PID X] python
 \_ [PID Y] /bin/sh -c <specified command>
     \_ [PID Z] <specified command>

And this is were things break if if the Python program calls kill() to kill the started process. In both cases, it will send a KILL signal to PID Y; under macOS, this will kill the process, but under Debian it will kill the intermediate shell and leave the program running!

Moral of the story:

• Portability is hard
• This is yet another reason to not use shell=True in Python programs!