Python Compiler Error during package installation?

2 years ago

Noah Thompson

8 minutes

An error that frequently occurs while installing

An error that frequently occurs while installing Python modules is the No such file or directory error. This error can be deceiving as it typically does not imply a missing file or directory from the package being installed. Rather, this error occurs when Python attempts to access your system compiler during the module installation process. The reason behind this is that Python has the paths to your system compiler hardcoded within itself and is unable to locate the necessary compiler files. In this tutorial, we will demonstrate an instance of this error and outline the necessary steps to resolve it on various platforms.

Compiler errors that are not appearing or being identified correctly.

When installing Python packages, it is common to use the pip package manager along with the pip install command. After selecting a package, pip will display a list of additional dependencies needed and a lengthy output during the installation process. Occasionally, the installer may encounter an error and provide a message similar to the following towards the end of the output.

Output

x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.10 -I/usr/local/lib/python3.10/dist-packages/numpy/core/include -I/usr/include/python3.10 -c radiomics/src/_cmatrices.c -o build/temp.linux-x86_64-3.10/radiomics/src/_cmatrices.o error: command ‘x86_64-linux-gnu-gcc’ failed: No such file or directory [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure × Encountered error while trying to install package. ╰─> pyradiomics note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure.

I encountered this specific error while attempting to install pd-dwi, a Python library utilized in chemotherapy studies, using the pip install pd-dwi command.

Certain Python libraries, particularly those utilized for scientific computing, may require compiling additional code directly on your device following installation. Python is an interpreted language that operates effectively with just the presence of the Python interpreter. However, lower-level languages like C or Rust, which are sometimes incorporated within Python libraries for improved performance, must first be compiled and optimized before they can be executed. If your system lacks a compiler, the installation process will not be successful.

In most modern platforms, when you install pip, which is Python’s package manager, it will automatically configure a compiler environment along with the necessary packages. Nevertheless, there are instances where this may not occur. This could be due to accidental uninstallation or the compiler never being installed in the first place. Additionally, unlike Linux, system package managers on Mac or Windows typically do not handle the installation of Python packages, leading to potential issues.

This tutorial will outline the necessary steps for installing and confirming a Python-friendly compiler on Ubuntu/Debian Linux, Red Hat/Rocky Linux, Windows, and macOS.

Compiler packages available for Ubuntu and Debian systems.

In Ubuntu, you have the option to install build-essential, a package that includes all the necessary packages for a modern and well-maintained compiler environment. This meta-package does not specifically refer to any single package, but ensures that various essential compiler tools are installed by including them as dependencies.

Another option is to install libpython3-dev. This package, which has been part of the Ubuntu/Debian ecosystem for a long time, connects the compiler to Python and configures the necessary backend settings for invoking the compiler seamlessly from Python or pip. Normally, it is automatically installed together with pip when using the package manager, but if you install pip without the package manager, you might not have it installed.

Use apt to install the packages.

sudo apt install build-essential libpython3-dev

Once the installation is complete, several requirements will be installed. You can confirm if a compiler is accessible by checking for the presence of the make command on your system using the which command.

which make

Output

/usr/bin/make

gcc, the widely-used open-source compiler, utilizes the “make” command to analyze a Makefile, which contains compilation instructions for each software package. Therefore, if you currently have a make version installed on your path, please attempt to reinstall your Python module using pip.

Red Hat and Rocky Linux offer compiler packages.

You can utilize the group feature of the dnf package manager on Red Hat and Rocky Linux to install a collection of packages that provide a stable compiler environment. The package group you will install is known as “Development Tools”.

To install the package group, utilize two dnf commands.

sudo dnf groups mark install “Development Tools”
sudo dnf groupinstall “Development Tools”

Furthermore, a variety of dependencies will be installed as well. You can then proceed to install python3-devel, which has long been a package in the Red Hat ecosystem. Essentially, this package connects the compiler to Python by providing the necessary backend configuration. With python3-devel, your compiler can be called automatically from either Python or pip.

sudo dnf install python3-devel

You can check if a compiler is available by using the which command to verify the existence of the make command on your system afterwards.

which make

Output

/usr/bin/make

The command “make” is used by gcc, the widely-used open-source compiler, to analyze a Makefile, which contains compilation instructions for each package. If you have currently installed make on your path, attempt to reinstall your Python module using pip.

Compiler environments for the Windows operating system.

Windows compiler problems can be more complicated as there are numerous methods for installing Python, and each of them requires specific compilers to be available.

If you use Python with WSL2, it’s just like running Python under Linux, so you can follow the troubleshooting instructions for your distro (Ubuntu by default).
If you use Python with Anaconda, it will provide its own compiler packages within the conda environment, usually avoiding any of these errors in the first place.
If you use Python on Windows natively, there are a few other considerations. By default, Python on Windows tries to use the Microsoft Visual Studio Build Tools. This is a very large install and adds many Windows ecosystem packages that may be unfamiliar if you mostly work in the cloud, but should work automatically after installation, like installing make on Linux.
If you already have a working version of open-source gcc and make build tooling installed in your Windows environment using MinGW or Chocolatey, you can tell Python to use this compiler on Windows instead, by creating a file at Lib/distutils/distutils.cfg relative to your Python install path and adding the following contents:

distutils.cfg file in the Lib/distutils directory

[build]
compiler=mingw32

[build_ext]
compiler=mingw32

If you encounter difficulties while installing a compiler on Windows, you may attempt to install a precompiled wheel package for the library instead. However, this method is less convenient compared to pip installation and such packages are typically accessible on a temporary basis.

Compiler environments for macOS.

macOS includes its compiler toolchain in XCode, Apple’s development suite. Similar to Visual Studio on Windows, XCode is a comprehensive development environment with its own interface. However, you don’t necessarily have to use XCode for compiling Python packages. Instead, you just need to ensure that the XCode packages are installed by executing the command xcode-select –install.

xcode-select –install

Firstly, you will be asked to initiate the installation process, after which you will be asked once more to agree to the software license. Subsequently, the tools will automatically download and install.

In conclusion,

The Python ecosystem is robust and inclusive for developers of all levels. However, it can be bewildering when you come across gaps in its tools. In this guide, you discovered how to resolve errors that may occur due to the absence of compiler packages and when Python requires compiling low-level code during module installation.

Afterwards, it might be worth your while to take a look at our series on Python programming.