Extending¶
Open Forms (“the backend”) is implemented with a pluggable architecture. Modules and plugins are Python packages with some Django bindings. They get called by the Open Forms core functionality (forms, submissions…).
We support two ways to add new plugins:
Adding new plugins directly in Open Forms. These are extensions that serve the greater community and can be contributed as a pull request to Open Forms (more information can be found in the CONTRIBUTING.md).
Adding ‘external’ plugins. These are extensions that are extremely specific or can’t be open-sourced. Instructions on how to implement such ‘external plugins’ are described below.
Creating an external plugin¶
An extension must be implemented as a Django package, which is a stricter form of a regular Python package. They can be developed independently from the open-formulieren Github organization.
The extensions can be loaded at deployment time through the environment variable OPEN_FORMS_EXTENSIONS
which
specifies the Python name of the extensions to load. Open Forms will extract this configuration value and
load the referenced extension packages. For an extension to be loaded, it needs to be present in the PYTHONPATH
.
This means that it needs to be either:
In the
src/
directory of Open Forms. It is then automatically picked up.Anywhere on the file path, but the
PYTHONPATH
environment variable is modified to include the path to the extension.In the relevant site-packages directory, similarly to when an Open Forms dependency is installed with
pip install
andvirtualenv
. In order for this to work, the extension should be published on PyPi and added to therequirements/extensions.in
file.
Building and distributing the extended Open Forms¶
You can build a custom Docker image extending an Open Forms release with a custom
Dockerfile
, in which the extension source code is added to the desired location.
See Testing and distributing with Docker for more details.
Another option is to mount the source code as a volume in a custom docker-compose.yml
file or as part of your Kubernetes manifests, depending on how you deploy your
instance(s).
The demo extension is an example
of an extension plugin. It implements a demo registration backend that prints the
specified configuration variables to the console. It uses a custom Dockerfile
to extend the (latest) Open Forms version.
Steps to implement an extension plugin¶
The default-app can be used as a template for an empty django package. Follow the instructions in the
README.rst
file to start a new Django project.Implement the plugin in the
<project_name>
directory. This will require at least updating/adding the files below. Refer to the plugin section for further details.
Extension code is allowed to import from Open Forms’ public API.
Any additional initialisation that the extension module might need can be implemented in the
extension.apps.ExtensionConfig.ready
hook. This hook is called when Django bootstraps and it is the mechanism used
by Open Forms to register plugins (refer to the plugin section for further details).
Configurable options¶
Extensions cannot require modifications to the Django settings of Open Forms. Any run-time configuration option can be specified as:
Deployment time environment variables. See plugin.py:35.
Dynamic options using
django-solo
, with the advantage that configuration can be modified at runtime through the Admin interface. See models.py and admin.py in the demo plugin. If you add a solo model, you will have to generate migrations. See the section below for details.For certain types of plugins, like for registration backends, it is also possible to add form-specific options. These can be specified with a serializer. See config.py for details.
Testing locally¶
In order to test your external plugin locally, you can use symlinks. Inside the src/
directory of Open Forms,
create a symlink to your package. For example, in the case of the demo extension package:
ln -s /path/to/plugin/demo_extension .
Now add an environment variable OPEN_FORMS_EXTENSIONS
with the name of your package. The variable is a
comma-separated list of valid python identifiers (i.e. the python package names). For example:
export OPEN_FORMS_EXTENSIONS=demo_extension,another_extension
If you need to generate migrations for your package, you can now do it as follows (from within the Open Forms directory):
python src/manage.py makemigrations demo_extension
python src/manage.py migrate
If you created a solo model, the configuration page should be available in the admin interface automatically under the “miscellaneous” group. Currently it’s not possible to configure these groups from an extension as they are reset on every deploy.
Note
If your demo-extension is a demo feature (Plugin.is_demo_plugin = True
), you must
ensure that demo plugins are enabled in the admin interface for them to be available.
Under Configuratie > Algemene configuratie scroll to the bottom of the page and click on Tonen next to Feature flags, test- en ontwikkelinstellingen. Then, check the box Demo plugins inschakelen and save the changes.
Testing and distributing with Docker¶
The recommended way to create a container image is to extend the Open Forms base image,
and set up a docker-compose.yml
locally to test with this custom image. You can
find examples of both in the demo extension repository.
Dockerfile structure
We recommend using a two-stage Dockerfile approach, where the first stage is used to
install any additional Python dependencies (if relevant). This stage should inherit
from the same Python base image of Open Forms to keep the Python version identical.
Open Forms itself also applies this principle, so you can look at the upstream
Dockerfile
for inspiration.
The second stage is meant for the production image and should extend from the Open Forms
version you are extending, e.g. open-formulieren/open-forms:1.0.0
. You can copy
the dependencies from your build stage and the extension source code into the final
image here.
Building and tagging the image
From within your extension repository, build the image and give it a name and tag of your choice, for example:
docker build -t myorg/open-forms:1.0.0 .
or use the relevant docker-compose command variants.
Running all the services with docker-compose
You can create your own docker-compose.yml
inspired by the Open Forms docker-compose
configuration, or use the docker-compose.override.yml
approach. Typically you will want to modify the image names and any additional
environment variables your extensions require.
docker-compose up
Testing in CI¶
There are multiple approaches to testing in CI. These include:
Using git submodules
Using symlinks
Git submodules
You can include Open-Forms as a git submodule in the repository of the extension with:
git submodule add https://github.com/open-formulieren/open-forms.git
Then, stage and commit the updated files.
In order for the extension to be able to import from openforms
, the path to the openforms
package needs to be present in
the PYTHON_PATH
. If it is not present, it can be added with:
export PYTHON_PATH=$PYTHON_PATH:<path to extension repo>/open-forms/src
Open-Forms (in the git submodule) should then be set up in the same way as when installing Open-Forms for development. See Installation for more information.
With this set up, from within the extension repository it is possible to test the extension. This can be done
using the manage.py
file from open-forms:
open-forms/src/manage.py test extension_package
In Github Actions, one can then create an action to run the tests with a similar logic. This action requires a postgres
and a redis
service. It is possible to checkout the repository with the Open-Forms submodule using:
- uses: actions/checkout@v3
with:
submodules: true
Then, after setting up the Open-Forms backend with the maykinmedia/setup-django-backend@v1.1
action, one can run the
tests as follows. Note that both the directory containing the Open-Forms manage.py
and the working directory are
automatically added to the path (so there is no need to update the PYTHON_PATH
).
- name: Run tests
run: |
python open-forms/src/manage.py compilemessages
coverage run --source=extension_package open-forms/src/manage.py test extension_package
coverage xml -o coverage-extension.xml
env:
DJANGO_SETTINGS_MODULE: openforms.conf.ci
OPEN_FORMS_EXTENSIONS: extension_package
For an example of how to set up github action with this method, look at the demo extension.
The advantages of using this approach include:
If you are not actively developing Open-Forms locally, then this approach keeps the extension nicely contained in one folder.
Setting up CI with Github actions is more straightforward.
Symlinks
This approach involves symlinking the extension package inside Open-Forms. The idea behind it is described in Testing locally.
In order to create a Github Action to test with this approach, you need to checkout both the extension AND the Open-Forms repositories:
- name: Checkout Open Forms
uses: actions/checkout@v3
with:
repository: open-formulieren/open-forms
path: open-forms
- name: Checkout extension
uses: actions/checkout@v3
with:
path: extension
Then, the Open-Forms backend needs to be set-up. You can use the maykinmedia/setup-django-backend@v1.1
action,
specifying the working-directory: open-forms
and the nvmrc-custom-dir: open-forms
arguments in addition to
all the other arguments that are normally used to set up the Open-Forms backend.
Next, the extension package needs to be symlinked in the Open-Forms repository:
- name: Make symlink in OF to the extension
run: |
ln -s ${{ github.workspace }}/extension/extension_package ${{ github.workspace }}/open-forms/src
The tests can be run in the same way as for the previous approach, but the working-directory
needs to be specified:
- name: Run tests
run: |
python open-forms/src/manage.py compilemessages
coverage run --source=extension_package open-forms/src/manage.py test extension_package
coverage xml -o coverage-extension.xml
env:
DJANGO_SETTINGS_MODULE: openforms.conf.ci
OPEN_FORMS_EXTENSIONS: extension_package
working-directory: ${{ github.workspace }}/open-forms
In order to get CodeCov working properly, the working-directory
and the root_dir
parameters both need to be
specified:
- name: Publish coverage report
uses: codecov/codecov-action@v3.1.4
with:
root_dir: ${{ github.workspace }}/extension
working-directory: ${{ github.workspace }}/open-forms
files: ./coverage-extension.xml
For examples of how to set up Github Actions with this approach, look at open-forms-ext-haalcentraal-hr and open-forms-ext-token-exchange.
The advantages of using this approach are:
If you are developing Open-Forms locally, you do not need to have an extra copy of the repository inside the extension repository.
If you want to test multiple extensions at the same time, this can be easily achieved by adding more symlinks to Open-Forms.