Running tests¶
The Open Forms backend comes with an extensive test suite to prevent regressions and also describe intended behaviour.
Running the backend tests¶
The testsuite uses Django’s standard test mechanism:
python src/manage.py test src --exclude=e2e
will run all the tests discovered in the src
directory, excluding
End-to-end tests.
You can also limit the tests to run by python path:
python src/manage.py test openforms.registrations.contrib.demo
or even by certain tags:
python src/manage.py test src --tag=gh-2418
There’s a tag dangerous
for tests that point out breaking behaviour.
For all options, see src/manage.py test --help
.
Cheatsheet for speeding up tests¶
--keepdb
to skip running all migrations every time--parallel <number>
to break the testsuite into<number>
parts and run them in parallel.--reverse
to scan for test isolation problemscoverage run src/manage.py test src <options> && coverage html
to measure code coverage
Frontend tests¶
There is (a limited) set of tests for the frontend code used in the backend. Tests are run with Jest.
npm run test
You can also run jest in watch mode or pass any other flags:
npm run test -- --watch
End-to-end tests¶
We have some end-to-end and Javascript smoke tests with Playwright for the custom JS used in the admin.
These tend to be slower and harder to debug/maintain.
Installation
After installing the dependencies, install the browsers locally:
playwright install
Running only the E2E tests
python src/manage.py test src --tag=e2e
Configuration
Configuration is done through environment variables:
NO_E2E_HEADLESS=<anything>
: will open an actual browser window so you can see what’s happening. By default, tests are run in headless mode.E2E_DRIVER=chromium
: specifies which browser is used for the selenium tests, defaults to Chromium. Available options:chromium
,firefox
andwebkit
.
Example custom command:
NO_E2E_HEADLESS=1 E2E_DRIVER=firefox python src/manage.py test src --tag=e2e
Note
Only the presence of the NO_E2E_HEADLESS
is checked, not the value
Known issues¶
AssertionError: Database queries to ‘default’ are not allowed in SimpleTestCase subclasses.
These are often caused by django-solo SingletonModel
sucblasses that are being
called somewhere, e.g. GlobalConfiguration.get_solo
. Sometimes they fetch from
cache, sometimes there is a cache miss and a database query is needed (e.g. when running
tests in reverse).
This is typically a test-isolation smell and the root cause should be fixed. This may
also be caused indirectly if you have LOG_REQUESTS
set to True
in your local
.env
, as it also results in a django-solo lookup.
The preferred approach to mitigate these kind of issues is to mock the get_solo
call
to prevent cache or DB hits:
@unittest.mock.patch(
"path.to.module.using_the_model.GlobalConfiguration.get_solo",
return_value=GlobalConfiguration(...),
)
def test_something(self, mock_get_solo):
...