In Python 3.12, distutils is gone. Well, sorta. setuptools nowadays provides a distutils package to provide backwards compatibility—but `import distutils` will fail if `setuptools` hasn't been imported first, which means the AutoInstaller will fail to install a variety of packages. Specifically: Installing MarkupSafe-1.1.1... Traceback (most recent call last): File "/tmp/markupsafe-311492/MarkupSafe-1.1.1/setup.py", line 6, in <module> from distutils.errors import CCompilerError ModuleNotFoundError: No module named 'distutils' Installing zope-interface-5.1.0... Traceback (most recent call last): File "/tmp/zope.interface-311492/zope.interface-5.1.0/setup.py", line 25, in <module> from distutils.errors import CCompilerError ModuleNotFoundError: No module named 'distutils' Installing selenium-3.141.0... Traceback (most recent call last): File "/tmp/selenium-311492/selenium-3.141.0/setup.py", line 20, in <module> from distutils.command.install import INSTALL_SCHEMES ModuleNotFoundError: No module named 'distutils' Installing jeepney-0.7.1... Traceback (most recent call last): File "/tmp/jeepney-311492/jeepney-0.7.1/setup.py", line 4, in <module> from distutils.core import setup ModuleNotFoundError: No module named 'distutils' Installing cffi-1.15.1... Traceback (most recent call last): File "/tmp/cffi-311492/cffi-1.15.1/setup.py", line 148, in <module> ask_supports_thread() File "/tmp/cffi-311492/cffi-1.15.1/setup.py", line 75, in ask_supports_thread config = get_config() ^^^^^^^^^^^^ File "/tmp/cffi-311492/cffi-1.15.1/setup.py", line 68, in get_config from distutils.core import Distribution ModuleNotFoundError: No module named 'distutils' Installing entrypoints-0.3.0... Traceback (most recent call last): File "/tmp/entrypoints-311492/entrypoints-0.3/setup.py", line 4, in <module> from distutils.core import setup ModuleNotFoundError: No module named 'distutils' Some of these have upstream releases which fix this, but some don't. And with setup.py deprecated, many of these packages are moving further away from what we support in the AutoInstaller today (c.f. bug 261082). There's a few possible paths forward, in decreasing levels of hackiness: * Move to relying on pip to download/build/install packages, and get out of all of those games entirely. * Implement a PEP 517 compliant builder (i.e., fix bug 261082). * Hardcode usage of (the default PEP 517 builder) setuptools.build_meta.__legacy__ to build all packages (which will solve the above problems, as the setuptools legacy builder keeps them working). * Tokenize/untokenize setup.py files to add a setuptools import at the top.
(In reply to Sam Sneddon [:gsnedders] from comment #0) > There's a few possible paths forward, in decreasing levels of hackiness: I did, of course, mean "increasing levels of hackiness".
<rdar://problem/115287945>
Ahhhh, I'd failed to realised how this worked with setuptools's vendored distutils (via _distutils_hack). So setuptools installs a `distutils-precedence.pth`, which makes setuptools' provided distutils get exposed as distutils. Thus, naturally, you'd expect this to work, but pth files are only processed for site directories (i.e., site-packages directories), and _not_ for any paths added to sys.path via the PYTHONPATH environment variable (as we use to invoke setup.py). This feels like another reason to move to using a venv for the AutoInstaller. See https://github.com/web-platform-tests/wpt/blob/9eb816722861f5f12f46738c9591f4c88aa0e995/tools/wpt/virtualenv.py#L99-L137 for code that roughly self-activates a venv within the running interpreter. I guess there's no reason why we can't just do this on Python 3 in the near future (where the venv module already exists), though it will trigger reinstalls (again). Might be worth to land alongside Bug 224669 to avoid multiple cases of reinstalling the world.