[Buildroot] [PATCH v2 1/2] package/python-pybind: fix upgrade to version 2.6.1

[Yann E. MORIN]

Arnout, All,

On 2021-12-07 09:04 +0100, Arnout Vandecappelle spake thusly:
> On 06/12/2021 22:57, Yann E. MORIN wrote:
> >Guillaume, All,
> >
> >On 2021-12-06 20:31 +0100, guillaume.bressaix at gmail.com spake thusly:
> >>From: "Guillaume W. Bres" <guillaume.bressaix at gmail.com>
> >>
> >>The python 'setup.py' script needs header files
> >>in $(@D)/pybind11 to work since v2.6.1, and these
> >>files are generated by an internal minimalist cmake build.
> >>
> >>Fixes
> >>    http://autobuild.buildroot.net/results/b89f1de64b308dffa73675f1f31ccb0b7be5a10d
> >>    http://autobuild.buildroot.net/results/d0287b7f64f206b0f074908c5780a3632e0cb799
> >>    http://autobuild.buildroot.net/results/27efb545a5a719a5581c8f746d3a3555ff4216ce
> >>    http://autobuild.buildroot.net/results/d2f0a0ad8f6c7178517df109e7d885dac9134c3a
> >>    http://autobuild.buildroot.net/results/b57e9a3279260dae4a590f9421238fcabb2f7cab
> >>    http://autobuild.buildroot.net/results/515e6f2fc6b5780260d98d6bb52b541ce4bf1afe
> >>    http://autobuild.buildroot.net/results/d89c4ecc81222d4f80c951da2232d2e393fa1c69
> >>
> >>Signed-off-by: Guillaume W. Bres <guillaume.bressaix at gmail.com>
> >I am sorry, but I fail to see how this actually fixes things.
[--SNIP--]
> >Also, since pybind is supposed to be a headers'-only library, there is
> >nothing supposed to go in target, so it should be;
> >     PYTHON_PYBIND_INSTALL_TARGET = NO
>  Note that this is only relevant to make sure that pybind11-config doesn't
> appear on target. The headers themselves are removed automatically by
> Buildroot.

It also installs a buncha stuff in /usr/lib/python3.9/site-packages/pybind11/
which are totally useless because that needed only for the packaging of
pybind11, not at runtime (as far as I could understand).

> >Furthermore, /usr/bin/pybind11-config is a python script which hardcodes
> >its shabang to #!/usr/bin/python, and this is incorrect:
> >
> >   1. /usr/bin/python might be python2 or python3, we can't know
> >   2. anyway, it would not use our host python, so it would miss the
> >     proper modules search paths.
> 
>  Yeah, this is something that needs to be fixed - in particular, there may
> not be a /usr/bin/python at all.
> 
>  However, I think that is kind of orthogonal to this patch. This patch fixes
> a build failure of pybind itself. Your proposed change would be to fix the
> build of a dependency of pybind.

Well, we can also easily fix it with just this:

    PYTHON_PYBIND_CONFIGURE_CMDS = true
    PYTHON_PYBIND_BUILD_CMDS = true
    PYTHON_PYBIND_INSTALL_TARGET_CMDS = true
    PYTHON_PYBIND_INSTALL_STAGING_CMDS = true
    HOST_PYTHON_PYBIND_CONFIGURE_CMDS = true
    HOST_PYTHON_PYBIND_BUILD_CMDS = true
    HOST_PYTHON_PYBIND_INSTALL_CMDS = true

Of course, this won't be very useful... Fixing the build of pybind in a
way that does not make it usable by its dependents is useless, imho. If
we fix it, we must fix it so that 1. it indeed builds and installs, and
2. that it is usable by its dependents, which means it should install
properly.

>  Note also that scipy depends on host-pybind, not pybind itself. I'm not
> sure how *that* is supposed to work - although, being header-only, it
> probably doesn't really matter if you link with the host or the target
> version of the library, and the host-installed pybind11-config makes sure
> that you pick up the host headers (which are normally not in include path).
> So conceptually wrong but works in practice.

And if we keep it like that without proper explanations why that is, or
if we do not fix it so that it behaves properly, it is going to become
a maintenance minefield down the road... Already is, anyway... :-]

>  In conclusion: I think this patch is going in the right direction (modulo
> BR2_CMAKE_HOST_DEPENDENCY). It's still "conceptually wrong but works"
> because it doesn't set the toolchain file and other cmake options, but that
> happens to work because it's header-only so it doesn't actually need any of
> the cmake options. So it's not entirely the correct thing to do, but it does
> guarantee to fix things. I'm not sure if that means we should merge it or
> not...

For me, no. But since we can't continue having build failures, and
becasue we have no package that depends on pybind, if we eventualyl do
not get a solution, then we can still remove it (it's broken!).

>  The proper fix, I think, would require patching setup.py so we can pass in
> additional cmake options to it.

Or as we've been discussing in another mail and IRC, by providing a
cmake wrapper in $(@D)/bin/cmake (not unlike some packages for which we
do a python symlink in $(@D)/bin/pyhon).

But now that the cmake issue has been identified, I think that we are
narrowing down on to a proper solution (Famous Last Words™).

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'