How to read a file from a python package subdirectory ?

Example of how to read a file from a python package subdirectory:

Table of contents

Introduction
(Option 1) Create a function that read a file from a python package subdirectory
(Option 2) Create a function that read a file from a python package subdirectory
Test if the file is read
References

Introduction

Let's assume that we want to create a new python package (called "ideas4eo") from scratch and that we store some data in a json file located in a subdirectory (ideas4eo/data/user):

ideas4eo/
    data/user/
        __init__.py
        config.json
    data_order/
        __init__.py
        nasa_laads_daac.py

the config.json file (that contain a user token):

{
        "nasa_laads_daac": {
                "token": "123456789"
        }
}

and we need to read the json file from a python script called nasa_laads_daac.py located in the subdirectory ideas4eo/data_order.

(Option 1) Create a function that read a file from a python package subdirectory

If you try to read the json file from nasa_laads_daac.py using a relative path ('../data/user/data.json') it is not going to work:

import json

with open('../data/user/data.json') as json_data:
        config_data = json.load(json_data)
        token = config_data['nasa_laads_daac']['token']

To read the json file, a solution is to use pkg_resources from setuptools, example

import pkg_resources
import json

def get_token():

        err_msg = []

        try:
                data_path = pkg_resources.resource_filename('ideas4eo.data.user', 'config.json')
                with open(data_path) as json_data:
                        config_data = json.load(json_data)
                token = config_data['nasa_laads_daac']['token']
        except:
                err_msg.append("Token Not found")
                token = -1

        return err_msg, token

(Option 2) Create a function that read a file from a python package subdirectory

Another possible solution is to use importlib.resources – Resources. See stackoverflow: How to read a (static) file from inside a Python package? to learn more.

Test if the file is read

To test if it works, start python outside the package (for example just above ideas4eo reperestory) and enter

>>> import ideas4eo.data_order.nasa_laads_daac as t
>>> t.get_token()
'123456789'

References

Hi, I'm Ben, the Founder of moonbooks.org. I work as a research scientist specializing in Earth satellite remote sensing, particularly focusing on Fire detection using VIIRS and ABI sensors. My areas of expertise include Python, Machine Learning, and Open Science. Feel free to reach out if you notice any errors on the site. I am constantly striving to enhance it. Thank you!