Invalid `SHELL` in `Makefile` for AIX builds?

[sideeffect42]

Bug report

Bug description:

When trying to build CPython on AIX, the make steps fails immediately with the following error message:

$ ./configure
[...]
$ make       
/bin/sh -e: not found

make: 1254-004 The error code from the last command is 1.


Stop.

AIX make(1) interprets whatever is assigned to the SHELL variable as a file path to the shell. It thus tries to use a shell named "/bin/sh -e":

execve("/bin/sh -e", 0x0000000110004D28, 0x0FFFFFFFFFFFFB90) Err#2  ENOENT

The -e was added by #100220, however I wonder if this is the right approach.

Going by POSIX I think -e should be the default unless it is explicitly disabled:

The execution line shall then be executed by a shell as if it were passed as the argument to the system() interface, except that if errors are not being ignored then the shell -e option shall also be in effect. If errors are being ignored for the command (as a result of the -i option, a '-' command prefix, or a .IGNORE special target), the shell -e option shall not be in effect.

I assume the -e was added for GNU make, which does not set -e unless it is set to POSIX-conforming mode (GNU make: Choosing the shell).

If the Makefile is defined to be a POSIX Makefile, wouldn't it be better to define .POSIX: in the makefile?

Quick test with GNU make:

$ cat Makefile
.POSIX:

SHELL = /bin/sh

test:
	false; echo after false
$ make --version
GNU Make 4.4.1
Built for powerpc64le-unknown-linux-musl
Copyright (C) 1988-2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ make test
false; echo after false
make: *** [Makefile:4: test] Error 1

CPython versions tested on:

CPython main branch

Operating systems tested on:

Other

Linked PRs

• gh-149021: Fix AIX build by dropping -e from SHELL in Makefile #149022

[picnixz]

1. We have AIX build bots so I'm surprised that this wasn't caught.
2. This happens to me on non-AIX platforms sometimes because of a non-clean configure step or whatever. So just to be sure, your ./configure call was successful and was from a clean build right?
3. I'm not entirely sure that we assume GNU Make actually. I know that because it's not GNU make we don't use wildcard targets or %-based targets as it's not portable. Is .POSIX portable?
4. We could limit this change to AIX only with an additional configure step (where we detect the existing make + platform and decide how to include the -e option).

@aisk I think you have access to AIX, can you check please? (if not sorry for the ping)

[sideeffect42]

@picnixz, thank you for looking at this issue so quickly.

1. Looking at some buildbot logs it seems to me that the AIX builders use GNU make.
   GNU make can be installed on AIX from the so called "Linux Toolbox".

2. It happens every time and after I patch the SHELL variable (and apply another tiny Makefile patch) manually, the build goes through successfully.

3. I thought that Python could be assuming GNU make silently; that's why I opened an issue first and didn't submit a PR already.
   It would be a shame though, because the Makefile looks like it works with other makes fine otherwise.

   I would consider .POSIX: portable, it's defined by the POSIX standard afterall.
   I tested my little shell snippet successfully with /usr/bin/make on AIX, FreeBSD, OpenBSD.
   NetBSD prints after false no matter if .POSIX and/or SHELL = /bin/sh -e is defined and requires a special .SHELL target to configure the shell.

   The only thing I could see possibly making problems would be Windows.

4. Depending on how one interprets POSIX, one could make the point that the SHELL variable is meant to only contain a path and adding options is non-standard:

   The SHELL macro shall be treated specially. It shall be provided by make and set to the pathname of the shell command language interpreter (see sh). The SHELL environment variable shall not affect the value of the SHELL macro. If SHELL is defined in the makefile or is specified on the command line, it shall replace the original value of the SHELL macro, but shall not affect the SHELL environment variable. Other effects of defining SHELL in the makefile or on the command line are implementation-defined.

   If GNU make and NetBSD do not feel like conforming to the standard (the -e was added due to some issues on Gentoo Linux popping up which uses GNU make), I think the configure checks should be the other way round.