• 製造業出身のデータサイエンティストがお送りする記事

今回はメモとして、ライブラリーのバージョン管理方法を残しておきます。

はじめに

自分で作成したスクリプトを関係者へ渡す際に、一緒に動作環境も提供するかと思います。その際、最近はdockerを使っているのですが、ライブラリーのバージョンを管理する方法をメモとして残しておきます。

pip freezeで一覧を見る

今回、dockerで構築した環境は下記です。

FROM ubuntu:18.04

ENV PYTHON_VERSION 3.8.6
ENV PYTHON_ROOT /tmp/Python/python-$PYTHON_VERSION
ENV PATH $PYTHON_ROOT/bin:$PATH
ENV PYENV_ROOT /tmp/.pyenv

WORKDIR /usr/src/app
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update \
    && apt-get install -y locales \
    && locale-gen ja_JP.UTF-8 \
    && echo "export LANG=ja_JP.UTF-8" >> ~/.bashrc \
    && apt install -y --no-install-recommends \
    build-essential \
    ca-certificates \
    libssl-dev \
    zlib1g-dev \
    libbz2-dev \
    libreadline-dev \
    libsqlite3-dev \
    libncurses5-dev \
    libncursesw5-dev \
    libffi-dev \
    liblzma-dev \
    vim \
    ssh \
    wget \
    xz-utils \
    tk-dev \
    git \
    && apt clean \
    && rm -rf /var/lib/apt/lists/*

# # Python環境構築
RUN git clone https://github.com/pyenv/pyenv.git $PYENV_ROOT
RUN $PYENV_ROOT/plugins/python-build/install.sh
RUN /usr/local/bin/python-build -v $PYTHON_VERSION $PYTHON_ROOT
RUN rm -rf $PYENV_ROOT

RUN pip install --upgrade pip

COPY requirements.txt ./
RUN pip install -r requirements.txt

WORKDIR home/work

version: "3"
services:
  jupyter:
    build: .
    volumes:
      - ./:/work
    working_dir: /work
    ports:
      - 8888:8888
    command: jupyter lab --ip=0.0.0.0 --allow-root --no-browser --NotebookApp.token=''

numpy==1.20.1
pandas==1.2.4
jupyterlab==3.0.12
matplotlib==3.3.4
seaborn==0.11.1
scikit-learn==0.24.1
plotly==4.14.3

実際にdockerで構築した環境でpip freezeでライブラリー一覧を確認してみた結果が下記です。

anyio==3.0.1
argon2-cffi==20.1.0
async-generator==1.10
attrs==21.2.0
Babel==2.9.1
backcall==0.2.0
bleach==3.3.0
certifi==2020.12.5
cffi==1.14.5
chardet==4.0.0
cycler==0.10.0
decorator==5.0.8
defusedxml==0.7.1
entrypoints==0.3
idna==2.10
ipykernel==5.5.5
ipython==7.23.1
ipython-genutils==0.2.0
jedi==0.18.0
Jinja2==3.0.0
joblib==1.0.1
json5==0.9.5
jsonschema==3.2.0
jupyter-client==6.1.12
jupyter-core==4.7.1
jupyter-packaging==0.7.12
jupyter-server==1.7.0
jupyterlab==3.0.12
jupyterlab-pygments==0.1.2
jupyterlab-server==2.5.1
kiwisolver==1.3.1
MarkupSafe==2.0.0
matplotlib==3.3.4
matplotlib-inline==0.1.2
mistune==0.8.4
nbclassic==0.2.8
nbclient==0.5.3
nbconvert==6.0.7
nbformat==5.1.3
nest-asyncio==1.5.1
notebook==6.3.0
numpy==1.20.1
packaging==20.9
pandas==1.2.4
pandocfilters==1.4.3
parso==0.8.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.2.0
plotly==4.14.3
prometheus-client==0.10.1
prompt-toolkit==3.0.18
ptyprocess==0.7.0
pycparser==2.20
Pygments==2.9.0
pyparsing==2.4.7
pyrsistent==0.17.3
python-dateutil==2.8.1
pytz==2021.1
pyzmq==22.0.3
requests==2.25.1
retrying==1.3.3
scikit-learn==0.24.1
scipy==1.6.3
seaborn==0.11.1
Send2Trash==1.5.0
six==1.16.0
sniffio==1.2.0
terminado==0.9.5
testpath==0.4.4
threadpoolctl==2.1.0
tornado==6.1
traitlets==5.0.5
urllib3==1.26.4
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==0.59.0

複数のパッケージの依存関係で様々なライブラリーが一緒にインストールされていることが分かります。
普段はこれをrequirements.txtに書きます。

pip-chillを使ってみた

今回はpip-chillというライブラリーを使ってみました。
まずは、インストールします。

$ pip install pip-chill

使い方はpip freezeと似ておりますので、以下のコマンドで実行してみます。

$ pip-chill

#jupyterlab==3.0.12
#pip-chill==1.0.1
#plotly==4.14.3
#scikit-learn==0.24.1
#seaborn==0.11.1

自分でインストールしたライブラリーがわかります。
requirements.txtで書いたものと少し異なりますが、細かい部分は今回は省略します。

次にパッケージ名だけを出力させます。

$ pip-chill --no-version

#jupyterlab
#pip-chill
#plotly
#scikit-learn
#seaborn

最後に、ライブラリーの依存関係を確認することもできます。

$ pip-chill -v

jupyterlab==3.0.12
pip-chill==1.0.1
plotly==4.14.3
scikit-learn==0.24.1
seaborn==0.11.1
# anyio==3.0.1 # Installed as dependency for jupyter-server
# argon2-cffi==20.1.0 # Installed as dependency for jupyter-server, notebook
# async-generator==1.10 # Installed as dependency for nbclient
# attrs==21.2.0 # Installed as dependency for jsonschema
# babel==2.9.1 # Installed as dependency for jupyterlab-server
# backcall==0.2.0 # Installed as dependency for ipython
# bleach==3.3.0 # Installed as dependency for nbconvert
# certifi==2020.12.5 # Installed as dependency for requests
# cffi==1.14.5 # Installed as dependency for argon2-cffi
# chardet==4.0.0 # Installed as dependency for requests
# cycler==0.10.0 # Installed as dependency for matplotlib
# decorator==5.0.8 # Installed as dependency for ipython
# defusedxml==0.7.1 # Installed as dependency for nbconvert
# entrypoints==0.3 # Installed as dependency for nbconvert
# idna==2.10 # Installed as dependency for anyio, requests
# ipykernel==5.5.5 # Installed as dependency for notebook
# ipython==7.23.1 # Installed as dependency for jupyterlab, ipykernel
# ipython-genutils==0.2.0 # Installed as dependency for jupyter-server, traitlets, nbformat, notebook
# jedi==0.18.0 # Installed as dependency for ipython
# jinja2==3.0.0 # Installed as dependency for jupyter-server, jupyterlab-server, notebook, nbconvert, jupyterlab
# joblib==1.0.1 # Installed as dependency for scikit-learn
# json5==0.9.5 # Installed as dependency for jupyterlab-server
# jsonschema==3.2.0 # Installed as dependency for jupyterlab-server, nbformat
# jupyter-client==6.1.12 # Installed as dependency for jupyter-server, nbclient, ipykernel, notebook
# jupyter-core==4.7.1 # Installed as dependency for jupyter-server, jupyter-client, notebook, nbformat, nbconvert, jupyterlab
# jupyter-packaging==0.7.12 # Installed as dependency for jupyterlab
# jupyter-server==1.7.0 # Installed as dependency for nbclassic, jupyterlab, jupyterlab-server
# jupyterlab-pygments==0.1.2 # Installed as dependency for nbconvert
# jupyterlab-server==2.5.1 # Installed as dependency for jupyterlab
# kiwisolver==1.3.1 # Installed as dependency for matplotlib
# markupsafe==2.0.0 # Installed as dependency for jinja2
# matplotlib==3.3.4 # Installed as dependency for seaborn
# matplotlib-inline==0.1.2 # Installed as dependency for ipython
# mistune==0.8.4 # Installed as dependency for nbconvert
# nbclassic==0.2.8 # Installed as dependency for jupyterlab
# nbclient==0.5.3 # Installed as dependency for nbconvert
# nbconvert==6.0.7 # Installed as dependency for jupyter-server, notebook
# nbformat==5.1.3 # Installed as dependency for jupyter-server, nbconvert, nbclient, notebook
# nest-asyncio==1.5.1 # Installed as dependency for nbclient
# notebook==6.3.0 # Installed as dependency for nbclassic
# numpy==1.20.1 # Installed as dependency for matplotlib, scipy, scikit-learn, seaborn, pandas
# packaging==20.9 # Installed as dependency for jupyterlab, bleach, jupyterlab-server, jupyter-packaging
# pandas==1.2.4 # Installed as dependency for seaborn
# pandocfilters==1.4.3 # Installed as dependency for nbconvert
# parso==0.8.2 # Installed as dependency for jedi
# pexpect==4.8.0 # Installed as dependency for ipython
# pickleshare==0.7.5 # Installed as dependency for ipython
# pillow==8.2.0 # Installed as dependency for matplotlib
# prometheus-client==0.10.1 # Installed as dependency for jupyter-server, notebook
# prompt-toolkit==3.0.18 # Installed as dependency for ipython
# ptyprocess==0.7.0 # Installed as dependency for terminado, pexpect
# pycparser==2.20 # Installed as dependency for cffi
# pygments==2.9.0 # Installed as dependency for ipython, nbconvert, jupyterlab-pygments
# pyparsing==2.4.7 # Installed as dependency for packaging, matplotlib
# pyrsistent==0.17.3 # Installed as dependency for jsonschema
# python-dateutil==2.8.1 # Installed as dependency for jupyter-client, matplotlib, pandas
# pytz==2021.1 # Installed as dependency for pandas, babel
# pyzmq==22.0.3 # Installed as dependency for jupyter-server, jupyter-client, notebook
# requests==2.25.1 # Installed as dependency for jupyterlab-server
# retrying==1.3.3 # Installed as dependency for plotly
# scipy==1.6.3 # Installed as dependency for scikit-learn, seaborn
# send2trash==1.5.0 # Installed as dependency for jupyter-server, notebook
# six==1.16.0 # Installed as dependency for websocket-client, argon2-cffi, cycler, plotly, retrying, python-dateutil, bleach, jsonschema
# sniffio==1.2.0 # Installed as dependency for anyio
# terminado==0.9.5 # Installed as dependency for jupyter-server, notebook
# testpath==0.4.4 # Installed as dependency for nbconvert
# threadpoolctl==2.1.0 # Installed as dependency for scikit-learn
# tornado==6.1 # Installed as dependency for jupyter-server, jupyter-client, notebook, terminado, jupyterlab, ipykernel
# traitlets==5.0.5 # Installed as dependency for jupyter-server, jupyter-client, notebook, ipython, nbclient, nbformat, nbconvert, jupyter-core, matplotlib-inline, ipykernel
# urllib3==1.26.4 # Installed as dependency for requests
# wcwidth==0.2.5 # Installed as dependency for prompt-toolkit
# webencodings==0.5.1 # Installed as dependency for bleach
# websocket-client==0.59.0 # Installed as dependency for jupyter-server

これでライブラリー同士の依存関係がわかります。
依存関係があるライブラリーに関してはコメントアウトされて出力されます。

さいごに

最後まで読んで頂き、ありがとうございました。
簡単ですが、ライブラリーのバージョンを管理する方法をメモとして残しておきました。

訂正要望がありましたら、ご連絡頂けますと幸いです。