Development Environment
Setting up a development environment
First, clone the repo from Github.
git clone git@github.com:ska-sa/katgpucbf.git
A setup script (dev-setup.sh) is included for your convenience to get going.
cd katgpucbf
source dev-setup.sh
The script will perform the following actions:
Create a fresh Python virtual environment.
Install all the requirements for running, developing and building this documentation.
Install the
katgpucbf
package itself, in editable mode.Build this documentation.
Install pre-commit to help with keeping things tidy.
Sourcing the script instead of executing it directly will keep your virtual
environment active, so you can get going straight away. Next time you want to
work, navigate into the katgpucbf
directory and source the virtual
environment directly:
source .venv/bin/activate
And you are ready to start developing with katgpucbf
!
Tip
I don’t recommend using the dev-setup.sh for anything other than initial setup. If you run it again, the requirements will be re-installed, and the module will be re-installed in editable mode. It’s unlikely that any of this will be harmful in any way, but it will use up a few minutes. You probably won’t want to do that every time.
Pre-commit
katgpucbf
is configured with pre-commit for auto-formatting Python code.
Pre-commit runs whenever anything is committed to the repository.
For more detailed information, please consult the pre-commit documentation. The installation and initialisation of the pre-commit flow is handled in dev-setup.sh.
Configuration Files
This repo contains the following configuration files for the pre-commit flow to monitor Python development.
.pre-commit-config.yaml
for pre-commit specifies which git hooks will be run before committing to the repo.pyproject.toml
dictates the configuration of utilities such as black and isort..flake8
for flake8, a tool for enforcing PEP 8-based style guide for Python..pydocstyle.ini
for pydocstyle, a tool for enforcing PEP 257-based docstring style guides for Python.mypy.ini
file for mypy, a static type checker (or lint-like tool) for type annotations in the Python code - according to PEP 484 and PEP 526 notation.
Installation Prerequisites
Although black, flake8, pydocstyle and mypy are used, the only prerequisite is the pre-commit Python library. That is, the YAML configuration file is set up so that when the pre-commit hooks are installed, all dependencies are automatically installed. (Note, they won’t be available to you in your Python environment, they will be used only by pre-commit. If you want to use them separately, you will need to install them separately with pip.)
Light-weight installation
There are a few cases where it is unnecessary (and inconvenient) to install
CUDA, such as for building the documentation or launching a correlator on a
remote system. If one does not use dev-setup.sh but installs
manually (in a virtual environment) using pip install -e .
, then only a
subset of dependencies are installed. There are also some optional extras that
can be installed, such as pip install -e ".[doc]"
to install necessary
dependencies for building the documentation. Refer to setup.cfg
to see what
extras are available.
This is not recommended for day-to-day development, because it will install whatever is the latest version at the time, rather than the known-good versions pinned in requirements.txt.
Boiler-plate files
The module contains the following boiler-plate files:
Dockerfile
for generating repeatable container images which are capable of running this package.Jenkinsfile
for a Jenkins Continuous Integration (CI) server to run unit tests automatically. Comments in the file document hardware requirements.requirements.in
andrequirements-dev.in
specify the Python prerequisites for running and developing with this package respectively. They are used as inputs to pip-compile.requirements.txt
andrequirements-dev.txt
list complete pinned requirements, to ensure repeatable operation. These are the output of thepip-compile
process mentioned above. These should be passed topip install
with the-r
flag to install the requirements either to run or develop. Development requires an additional set of packages which are not required for users to run the software (such as pytest). Note that developers should install both sets of requirements, not just the development ones.setup.cfg
andsetup.py
allow setuptools to install this package.pyproject.toml
is a standard file included with many Python projects. It is used to store some configuration for pre-commit (as described above), some configuration options forpytest
, and other configuration as described here.
Preparing to raise a Pull Request
Pre-commit compliance
Contributors who prefer to develop without pre-commit enabled will be required to ensure that any submissions pass all the checks described here before they can be accepted and merged.
No judgement, we know pre-commit can be annoying if you’re not used to it. This is in place in order to keep the code-base consistent so we can focus on the work at hand - rather than maintaining code readability and appearance.
Module documentation updates
katgpucbf
holds documentation within its code-base.
sphinx-apidoc provides a manner to generate
module documentation as reStructuredText. If you, the developer, add or remove
a module or file, execute the full sphinx-apidoc
command below to regenerate
the module documentation with your updates. The incantation below is run from the
root katgpucbf
directory.
sphinx-apidoc -efo doc/ src/
Note
The above command will likely generate a modules.rst
file, which is
not necessary to commit.