Development Environment

Setting up a development environment

First, clone the repo from Github.

git clone git@github.com:ska-sa/katgpucbf.git

A setup script (dev-setup.sh) is included for your convenience to get going.

cd katgpucbf
source dev-setup.sh

The script will perform the following actions:

  • Create a fresh Python virtual environment.

  • Install all the requirements for running, developing and building this documentation.

  • Install the katgpucbf package itself, in editable mode.

  • Build this documentation.

  • Install pre-commit to help with keeping things tidy.

Sourcing the script instead of executing it directly will keep your virtual environment active, so you can get going straight away. Next time you want to work, navigate into the katgpucbf directory and source the virtual environment directly:

source .venv/bin/activate

And you are ready to start developing with katgpucbf!

Tip

I don’t recommend using the dev-setup.sh for anything other than initial setup. If you run it again, the requirements will be re-installed, and the module will be re-installed in editable mode. It’s unlikely that any of this will be harmful in any way, but it will use up a few minutes. You probably won’t want to do that every time.

Pre-commit

katgpucbf is configured with pre-commit for auto-formatting Python code. Pre-commit runs whenever anything is committed to the repository.

For more detailed information, please consult the pre-commit documentation. The installation and initialisation of the pre-commit flow is handled in dev-setup.sh.

Configuration Files

This repo contains the following configuration files for the pre-commit flow to monitor Python development.

  • .pre-commit-config.yaml for pre-commit specifies which git hooks will be run before committing to the repo.

  • pyproject.toml dictates the configuration of utilities such as black and isort.

  • .flake8 for flake8, a tool for enforcing PEP 8-based style guide for Python.

  • .pydocstyle.ini for pydocstyle, a tool for enforcing PEP 257-based docstring style guides for Python.

  • mypy.ini file for mypy, a static type checker (or lint-like tool) for type annotations in the Python code - according to PEP 484 and PEP 526 notation.

Installation Prerequisites

Although black, flake8, pydocstyle and mypy are used, the only prerequisite is the pre-commit Python library. That is, the YAML configuration file is set up so that when the pre-commit hooks are installed, all dependencies are automatically installed. (Note, they won’t be available to you in your Python environment, they will be used only by pre-commit. If you want to use them separately, you will need to install them separately with pip.)

Light-weight installation

There are a few cases where it is unnecessary (and inconvenient) to install CUDA, such as for building the documentation or launching a correlator on a remote system. If one does not use dev-setup.sh but installs manually (in a virtual environment) using pip install -e ., then only a subset of dependencies are installed. There are also some optional extras that can be installed, such as pip install -e ".[doc]" to install necessary dependencies for building the documentation. Refer to setup.cfg to see what extras are available.

This is not recommended for day-to-day development, because it will install whatever is the latest version at the time, rather than the known-good versions pinned in requirements.txt.

Boiler-plate files

The module contains the following boiler-plate files:

  • Dockerfile for generating repeatable container images which are capable of running this package.

  • Jenkinsfile for a Jenkins Continuous Integration (CI) server to run unit tests automatically. Comments in the file document hardware requirements.

  • requirements.in and requirements-dev.in specify the Python prerequisites for running and developing with this package respectively. They are used as inputs to pip-compile.

  • requirements.txt and requirements-dev.txt list complete pinned requirements, to ensure repeatable operation. These are the output of the pip-compile process mentioned above. These should be passed to pip install with the -r flag to install the requirements either to run or develop. Development requires an additional set of packages which are not required for users to run the software (such as pytest). Note that developers should install both sets of requirements, not just the development ones.

  • setup.cfg and setup.py allow setuptools to install this package.

  • pyproject.toml is a standard file included with many Python projects. It is used to store some configuration for pre-commit (as described above), some configuration options for pytest, and other configuration as described here.

Preparing to raise a Pull Request

Pre-commit compliance

Contributors who prefer to develop without pre-commit enabled will be required to ensure that any submissions pass all the checks described here before they can be accepted and merged.

No judgement, we know pre-commit can be annoying if you’re not used to it. This is in place in order to keep the code-base consistent so we can focus on the work at hand - rather than maintaining code readability and appearance.

Module documentation updates

katgpucbf holds documentation within its code-base. sphinx-apidoc provides a manner to generate module documentation as reStructuredText. If you, the developer, add or remove a module or file, execute the full sphinx-apidoc command below to regenerate the module documentation with your updates. The incantation below is run from the root katgpucbf directory.

sphinx-apidoc -efo doc/ src/

Note

The above command will likely generate a modules.rst file, which is not necessary to commit.