# Python Package Template Project
[![image](https://img.shields.io/pypi/v/py-package-template.svg)](https://pypi.org/project/py-package-template/)
[![Build Status](https://travis-ci.org/AlexIoannides/py-package-template.svg?branch=master)](https://travis-ci.org/AlexIoannides/py-package-template)
The py-template-project package allows users to download the contents of this [GiHub repository](https://github.com/AlexIoannides/py-package-template), containing a skeleton Python package project to be used as a template for kick-starting development of **any** type of Package; destined for upload to PyPI, or just for local install using Pip. The downloaded package includes the following components to aid rapid development without having to spend time cloning existing set-ups from other projects:
- a minimal `setup.py` file;
- testing with PyTest;
- documentation (HTML and PDF) generated using Sphinx with auto-documentation setup;
- an entry-point that allows the package to execute functions directly from the command line - e.g. to start a server, interact with a user, download a GitHub repository, etc.; and,
- automated testing and deployment using Travis CI.
A description of how to work with (and modify) each of these components, is provided in more detail in the sections that follow-on below, as well as in the documentation and within the example code bundled with the package.
This is obviously a opinionated view of how a Python package project ought to be structured, based largely on my own experiences and requirements. Where I have needed guidance on this subject, I have leant heavily on the advice given by the [Python Packaging Authority (PyPA)](https://packaging.python.org/guides/distributing-packages-using-setuptools/) and used the excellent [Requests](https://github.com/requests/requests) and [Flask](https://github.com/pallets/flask) projects as references for 'best practices'.
## Installing
Install and update using [pip](https://pip.pypa.io/en/stable/quickstart/):
```bash
pip3 install py-template-project
```
## Downloading a Python Package Template Project
To down load the latest version of the Python Package Template project located in [this GiHub repository](https://github.com/AlexIoannides/py-package-template), execute the following command from the command line:
```bash
py-package-template install
```
This will be downloaded to the current directory and will contain the following directory structure:
```bash
py-package-tempate/
|-- docs/
|-- |-- build_html/
|-- |-- build_latex/
|-- |-- source/
|-- py-pkg/
|-- |-- __init__.py
|-- |-- __version__.py
|-- |-- curves.py
|-- |-- entry_points.py
|-- tests/
|-- |-- test_data/
|-- | |-- supply_demand_data.json
|-- | __init__.py
|-- | conftest.py
|-- | test_curves.py
|-- .env
|-- .gitignore
|-- Pipfile
|-- Pipfile.lock
|-- README.md
|-- setup.py
```
## The Python Package Template Project
We now describe the various components of the template project and the workflows associated with it. The template package project contains two modules to get things started:
- `curves.py`
- `entry_points.py`
The `curves.py` module contains sample code for modelling economic supply and demand curves and makes for a useful demonstration of how Python type annotation and interface definition via abstract base classes, can make code easier to read, document and reason about (I am a big fan). The test suite for this module is contained in the `tests` folder and demonstrates how to get up-and-running with PyTest.
The `entry_points.py` module is referenced in the `setup.py` file via the `entry_points` definitions:
```python
entry_points={
'console_scripts': ['py-package-template=py_pkg.entry_points:main'],
}
```
It enables the declared entry point - `py_pkg.entry_points.main` - to be invoked when `py-package-template` is called from the command line. This is what enables the template project to be downloaded programmatically (check the code for the full details). This could easily be extended to start a server (e.g. using Flask), or run any other type of script.
### Project Dependencies
We use [pipenv](https://docs.pipenv.org) for managing project dependencies and Python environments (i.e. virtual environments). These dependencies are **not** to be confused with the package installation dependencies for the package under developement - i.e. those that need to be defined in the `install_requires` section of `setup.py`. All of the direct packages dependencies required to run the project's code (e.g. NumPy for tensors), as well as all the packages used during development (e.g. flake8 for code linting and IPython for interactive console sessions), are described in the `Pipfile`. Their precise downstream dependencies are crystallised in `Pipfile.lock`, which is used to guarentee repeatable (i.e. deterministic) builds.
#### Installing Pipenv
To get started with Pipenv, first of all download it - assuming that there is a 'global' version of Python available on your system and on the PATH, then this can be achieved by running the following command,
```bash
pip3 install pipenv
```
For more information, including advanced configuration options, see the [official pipenv documentation](https://docs.pipenv.org).
#### Installing this Projects' Dependencies
Make sure that you're in the project's root directory (the same one in which `Pipfile` resides), and then run,
```bash
pipenv install --dev
```
This will install all of the direct project dependencies as well as the development dependencies (the latter a consequence of the `--dev` flag). To add and remove dependencies as required for your new project, use `pipenv install` and `pipenv uninstall` as required, using the `--dev` flag for development-only dependencies.
#### Running Python and IPython from the Project's Virtual Environment
In order to open a Python REPL using within an environment that precisely mimics the one the project is being developed with, use Pipenv from the command line as follows,
```bash
pipenv run python3
```
The `python3` command could just as well be `ipython3`.
#### Automatic Loading of Environment Variables
Pipenv will automatically pick-up and load any environment variables declared in the `.env` file, located in the package's root directory. For example, adding,
```bash
SPARK_HOME=applications/spark-2.3.1/bin
```
Will enable access to this variable within any Python program, via a call to `os.environ['SPARK_HOME']`. Note, that if any security credentials are placed here, then this file **must** be removed from source control - i.e. add `.env` to the `.gitignore` file to prevent potential security risks.
#### Pipenv Shells
Prepending `pipenv` to every command you want to run within the context of your Pipenv-managed virtual environment, can get (very) tedious. This can be avoided by entering into a Pipenv-managed shell,
```bash
pipenv shell
```
Which is equivalent to 'activating' the virtual environment. Any command will now be executed within the virtual environment. Use `exit` to leave the shell session.
### Running Unit Tests
All test have been written using the [PyTest](https://docs.pytest.org/en/latest/) package. Tests are kept in the `tests` folder and can be run from the command line by - e.g. by invoking,
```bash
pipenv run pytest
```
The test suite is structured as an independent Python package as follows:
```bash
tests/
|-- test_data/
| |-- supply_demand_data.json
| __init__.py
| conftest.py
| test_curves.py
```
The `conftest.py` module is used by PyTest - in this particular instance for loading test data and building objects that will then be used by potentially many other tests. These are referred to as 'fixtures' in PyTest - more details can be found [here](https://docs.pytest.org/en/latest/fixture.html).
### Linting Code
I prefer to use [flake8](http://flake8.pycqa.org/en/latest/) for style guide enforcement. This can be invoked from the command line