How to read a file from a python package subdirectory ?

Example of how to read a file from a python package subdirectory:

Table of contents

Introduction
(Option 1) Create a function that read a file from a python package subdirectory
(Option 2) Create a function that read a file from a python package subdirectory
Test if the file is read
References

Introduction

Let's assume that we want to create a new python package (called "ideas4eo") from scratch and that we store some data in a json file located in a subdirectory (ideas4eo/data/user):

ideas4eo/
    data/user/
        __init__.py
        config.json
    data_order/
        __init__.py
        nasa_laads_daac.py

the config.json file (that contain a user token):

{
        "nasa_laads_daac": {
                "token": "123456789"
        }
}

and we need to read the json file from a python script called nasa_laads_daac.py located in the subdirectory ideas4eo/data_order.

(Option 1) Create a function that read a file from a python package subdirectory

If you try to read the json file from nasa_laads_daac.py using a relative path ('../data/user/data.json') it is not going to work:

import json

with open('../data/user/data.json') as json_data:
        config_data = json.load(json_data)
        token = config_data['nasa_laads_daac']['token']

To read the json file, a solution is to use pkg_resources from setuptools, example

import pkg_resources
import json

def get_token():

        err_msg = []

        try:
                data_path = pkg_resources.resource_filename('ideas4eo.data.user', 'config.json')
                with open(data_path) as json_data:
                        config_data = json.load(json_data)
                token = config_data['nasa_laads_daac']['token']
        except:
                err_msg.append("Token Not found")
                token = -1

        return err_msg, token

(Option 2) Create a function that read a file from a python package subdirectory

Another possible solution is to use importlib.resources – Resources. See stackoverflow: How to read a (static) file from inside a Python package? to learn more.

Test if the file is read

To test if it works, start python outside the package (for example just above ideas4eo reperestory) and enter

>>> import ideas4eo.data_order.nasa_laads_daac as t
>>> t.get_token()
'123456789'

References

Developer in my spare time, I’m interested in Earth observation and satellite-based fire detection. Passionate about Python, machine learning, and open science.