How to get a list of python modules (with version for each module) and create a requirements.txt file ?

Examples of commands I used to get a list of python modules with corresponding versions. If you have better options to do that, your are welcome to add a comment or edit the note.

Get a list of python modules available in my current environment

To get a list of python modules available, a solution is to use freeze:

pip freeze

returns for example:

absl-py==0.9.0
alabaster==0.7.10
anaconda-client==1.6.9
anaconda-navigator==1.7.0
anaconda-project==0.8.2
appnope==0.1.0
appscript==1.0.1
asn1crypto==0.24.0
astor==0.8.1
astroid==1.6.1
astropy==2.0.3
attrs==17.4.0
Babel==2.5.3
backports.shutil-get-terminal-size==1.0.0
beautifulsoup4==4.6.0
bitarray==0.8.1
bkcharts==0.2
blaze==0.11.3
bleach==2.1.2
bokeh==0.12.13
boto==2.48.0
Bottleneck==1.2.1
cachetools==3.1.1
certifi==2018.1.18
cffi==1.11.4
chardet==3.0.4
click==6.7
cloudpickle==0.5.2
clyent==1.2.2
colorama==0.3.9
conda==4.8.3
conda-build==3.4.1
conda-package-handling==1.7.0+0.g7c4a471.dirty
conda-verify==2.0.0
contextlib2==0.5.5
cryptography==2.1.4
cycler==0.10.0
Cython==0.27.3
cytoolz==0.9.0
dask==0.16.1
datashape==0.5.4
decorator==4.2.1
distributed==1.20.2
docutils==0.14
entrypoints==0.2.3
et-xmlfile==1.0.1
fastcache==1.0.2
filelock==2.0.13
Flask==0.12.2
Flask-Cors==3.0.3
geoip2==2.9.0
gevent==1.2.2
glob2==0.6
gmpy2==2.0.8
google-api-python-client==1.7.11
google-auth==1.6.3
google-auth-httplib2==0.0.3
greenlet==0.4.12
h5py==2.7.1
heapdict==1.0.0
html5lib==1.0.1
httplib2==0.14.0
idna==2.6
imageio==2.2.0
imagesize==0.7.1
ipykernel==4.8.0
ipython==6.2.1
ipython-genutils==0.2.0
ipywidgets==7.1.1
isort==4.2.15
itsdangerous==0.24
jdcal==1.3
jedi==0.11.1
Jinja2==2.10
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.2.2
jupyter-console==5.2.0
jupyter-core==4.4.0
jupyterlab==0.31.5
jupyterlab-launcher==0.10.2
lazy-object-proxy==1.3.1
llvmlite==0.21.0
locket==0.2.0
lxml==4.1.1
MarkupSafe==1.0
matplotlib==2.1.2
maxminddb==1.4.1
mccabe==0.6.1
mistune==0.8.3
mpmath==1.0.0
msgpack-python==0.5.1
multipledispatch==0.4.9
navigator-updater==0.1.0
nbconvert==5.3.1
nbformat==4.4.0
networkx==2.1
nltk==3.2.5
nose==1.3.7
notebook==5.4.0
numba==0.36.2
numexpr==2.6.4
numpy==1.14.0
numpydoc==0.7.0
oauth2client==4.1.3
odo==0.5.1
olefile==0.45.1
openpyxl==2.4.10
packaging==16.8
pandas==0.22.0
pandocfilters==1.4.2
parso==0.1.1
partd==0.3.8
path.py==10.5
pathlib2==2.3.0
patsy==0.5.0
pdf2image @ file:///home/conda/feedstock_root/build_artifacts/pdf2image_1588278609827/work
pep8==1.7.1
pexpect==4.3.1
pickleshare==0.7.4
Pillow==5.0.0
pkginfo==1.4.1
pluggy==0.6.0
ply==3.10
prompt-toolkit==1.0.15
protobuf==3.12.4
psutil==5.4.3
ptyprocess==0.5.2
py==1.5.2
pyasn1==0.4.7
pyasn1-modules==0.2.6
pycodestyle==2.3.1
pycosat==0.6.3
pycparser==2.18
pycrypto==2.6.1
pycurl==7.43.0.1
pyflakes==1.6.0
Pygments==2.2.0
pylint==1.8.2
pyodbc==4.0.22
pyOpenSSL==17.5.0
pyparsing==2.2.0
PySocks==1.6.7
pytest==3.3.2
python-dateutil==2.6.1
pytz==2017.3
PyWavelets==0.5.2
PyYAML==3.12
pyzmq==16.0.3
QtAwesome==0.4.4
qtconsole==4.3.1
QtPy==1.3.1
requests==2.18.4
rope==0.10.7
rsa==4.0
ruamel-yaml==0.15.35
scikit-image==0.13.1
scikit-learn==0.19.1
scipy==1.0.0
seaborn==0.8.1
Send2Trash==1.4.2
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.12.0
snowballstemmer==1.2.1
sortedcollections==0.5.3
sortedcontainers==1.5.9
Sphinx==1.6.6
sphinxcontrib-websupport==1.0.1
spyder==3.2.6
SQLAlchemy==1.2.1
statsmodels==0.8.0
sympy==1.1.1
tables==3.4.2
tblib==1.3.2
tensorflow-docs @ git+https://github.com/tensorflow/docs@cb886cfdd16d66ff7f8d1430676ff395b02910e6
terminado==0.8.1
testpath==0.3.1
toolz==0.9.0
tornado==4.5.3
tqdm==4.46.0
traitlets==4.3.2
typing==3.6.2
unicodecsv==0.14.1
uritemplate==3.0.0
urllib3==1.22
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.14.1
widgetsnbextension==3.1.0
wrapt==1.10.11
xlrd==1.1.0
XlsxWriter==1.0.2
xlwings==0.11.5
xlwt==1.2.0
zict==0.1.3

To create a requirements.txt file from freeze:

pip freeze > requirements.txt

Modules using in a python script

To find all modules used in a python script (for example test.py), a solution is to use ModuleFinder:

from modulefinder import ModuleFinder

finder = ModuleFinder()
finder.run_script('test.py')

print('Loaded modules:')
for name, mod in finder.modules.items():
    print('%s: ' % name, end='')
    print(','.join(list(mod.globalnames.keys())[:3]))

print('-'*50)
print('Modules not imported:')
print('\n'.join(finder.badmodules.keys()))

Note that It seems not to work as expected with anaconda (I got a list of all available modules not the modules used in the script only).

Modules using in a jupyter notebook

In a jupyter notebook to get a list of python modules used, a solution is to enter in a cell the following lines:

import types

def imports():
    for name, val in globals().items():
        if isinstance(val, types.ModuleType):
            yield val.__name__

list(imports())

returns for example:

['builtins',
 'builtins',
 'types',
 'tensorflow',
 'tensorflow_docs',
 'tensorflow_docs',
 'matplotlib.pyplot',
 'matplotlib',
 'matplotlib.cm',
 'pandas',
 'numpy',
 'numpy.ma',
 'seaborn',
 'os',
 'warnings',
 'tensorflow_core.keras',
 'tensorflow_core.keras.layers']

Note that

!pip freeze

returns all the modules that can be imported in the notebook:

absl-py==0.9.0
alabaster==0.7.10
anaconda-client==1.6.9
anaconda-navigator==1.7.0
anaconda-project==0.8.2
appnope==0.1.0
appscript==1.0.1
asn1crypto==0.24.0
astor==0.8.1
astroid==1.6.1
astropy==2.0.3
attrs==17.4.0
Babel==2.5.3
backports.shutil-get-terminal-size==1.0.0
beautifulsoup4==4.6.0
bitarray==0.8.1
bkcharts==0.2

References

• How to create a requirements.txt?
• modulefinder
• Robin's Blog
• How can I see all installed Python modules in Jupyter Lab (like pip freeze) with Python 3.7 or newer?
• Package for listing version of packages used in a Jupyter notebook - python