Test helpers

Open Forms tests are built with the core Django testing helpers defined in django.test, extended with some third party libraries and project-specific helpers.

Third party packages

• django-webtest: acts more like a browser without being a full-blown browser. Very useful for admin tests, especially when combined with pyquery.

• hypothesis: property based testing, very good for generating fuzzy data to catch edge cases you never would think of. See Custom Hypothesis strategies.

Project helpers

HTML assertions

class openforms.utils.tests.html_assert.HTMLAssertMixin

Mixin HTML-related assertions.

assertHTMLValid(html_text)

check basic HTML syntax validity

assertNotTagWithTextIn(tag, text, document_str)

check for html tags and their content while ignoring tag attributes

assertTagWithTextIn(tag, text, document_str)

check for html tags and their content while ignoring tag attributes

openforms.utils.tests.html_assert.strip_all_attributes(document: str) → str

Reduce an HTML document to just the tags, stripping any attributes.

Useful for testing with self.assertInHTML without having to worry about class names, style tags etc.

Taken and adapted from https://stackoverflow.com/a/7472003

class openforms.utils.tests.webtest_base.WebTestPyQueryMixin

Mixin PyQuery assertions for WebTest tests.

Frontend redirects

class openforms.frontend.tests.FrontendRedirectMixin

A mixin providing a helper method checking frontend redirects.

assertRedirectsToFrontend(response: HttpResponse, frontend_base_url: str, action: SDKAction, action_params: dict[str, str] | None = None, **kwargs: Any) → None

Assert that a response redirected to a specific frontend URL.

Parameters:

• response – The response to test the redirection on.

• frontend_base_url – The base URL of the frontend.

• action – The SDK action performed.

• action_params – Optional parameters for the action.

• **kwargs – Additional kwargs to be passed to django.test.SimpleTestCase.assertRedirects().

Formio assertions

class openforms.formio.tests.assertions.FormioMixin

assertFormioComponent(configuration: JSONObject, key: str, properties_map: dict[str, JSONValue]) → None

Assert that the formio component with specified key has the expected properties.

Parameters:

• configuration – Formio form configuration

• key – the unique key of the component to check

• properties_map – a mapping of formio property name to expected property value. Note that the dict keys can be dotted paths for nested properties.

Recording HTTP traffic

Open Forms configuration and wrapper around vcr.py.

There are essentially two approaches for testing the interaction with third party services:

• Mocking the requests using requests_mock

• Recording the real interactions and replaying those during tests

The tooling here assists with the second approach.

Advantages of this approach

• no bugs/mistakes in mock data/setup, as you are talking to a real service

• no need to manually update mocks or create them, you can automatically (re-)record the interactions

• if/when the remote service changes, you just run the same tests again while re-recording to capture the new behaviour

Disadvantages

• you need access to a real implementation and sharing those credentials in an open source project is a challenge

• there is a risk of exposing data you didn’t mean to expose

Strategies to avoid leaking information

The host/domain of a particular service provided by a client to test with should not be exposed, as these are internal domains. You can use mitmproxy to hide this, e.g.:

mitmdump --mode reverse:https://example.com --ssl-insecure

If it is a SOAP service that requires client certificates for in-band messages, the --set client_certs=DIRECTORY|FILE flag of mitmproxy to setup mTLS can’t help. E.g. the Suwinet client (and prefill plugin) tests rewrite urls in requests and responses with a url from an environment variable (see dotenv-example).

Specifying the record mode

You can set the environment variable VCR_RECORD_MODE to any of the supported record modes.

Note

When you use VCR for tests with obfuscated URLs or credentials (or any sensitive data in general), you must document this information (in Taiga) so that other people have all the necessary information/steps at hand to re-record cassettes.

Re-recording is done as part of the release process.

class openforms.utils.tests.vcr.OFVCRMixin

Mixin to use VCR in your unit tests.

Using this mixin will result in HTTP requests/responses being recorded.

openforms.utils.tests.vcr.with_setup_test_data_vcr(cls: type) → Iterator[None]

Context manager to explicitly use VCR (inside setUpTestData for instance)

Parameters:

• base_path – The base directory for VCR test files.

• class_name – The qualified name of the test class.

Custom Hypothesis strategies

General purpose strategies

openforms.tests.search_strategies.json_collections(values, keys_strategy=text()) → SearchStrategy[dict[str, JSONValue] | list[JSONValue]]

openforms.tests.search_strategies.json_primitives(text_strategy=text()) → SearchStrategy[JSONPrimitive]

openforms.tests.search_strategies.json_values(*, max_leaves: int = 15) → SearchStrategy[JSONValue]

openforms.tests.search_strategies.jsonb_primitives() → SearchStrategy[JSONPrimitive]

openforms.tests.search_strategies.jsonb_text() → SearchStrategy[str]

openforms.tests.search_strategies.jsonb_values(*, max_leaves: int = 15) → SearchStrategy[JSONValue]

openforms.tests.search_strategies.no_null_byte_characters(**kwargs)

openforms.tests.search_strategies.valid_key(key: str) → bool

Formio component strategies

Expose hypothesis derived strategies to work with Formio.js data structures.

Tip

Use hypothesis strategies to generate random Formio form configurations to test implementation robustness.

All the formio component definitions must be JSON-serializable. JSON in itself can handle NULL bytes inside strings (they turn into ), but JSONB as used by postgresql and Django’s model.JSONField - where we persist these component definitions - cannot handle NULL bytes. That’s why we opt for jsonb_text rather than the plain st.text strategy.

The component definitions are by no means complete and it is possible hypothesis generates some combinations with optional fields that semantically make little sense (like setting both deriveStreetName and deriveCity to true in the textfield component). This shall have to be iterated on.

openforms.formio.tests.search_strategies.address_nl_component()

openforms.formio.tests.search_strategies.any_component = deferred(lambda: st.one_of(textfield_component(), email_component(), date_component(), datetime_component(), time_component(), phone_number_component(), file_component(), textarea_component(), number_component(), select_component(), checkbox_component(), selectboxes_component(), currency_component(), radio_component(), iban_component(), license_plate_component(), bsn_component(), address_nl_component(), np_family_members_component(), signature_component(), cosign_v2_component(), map_component(), edit_grid_component(), content_component(), columns_component(), fieldset_component(), postcode_component(), cosign_v1_component()))

A search strategy returning any possible/supported Formio component dictionary.

openforms.formio.tests.search_strategies.bsn_component()

openforms.formio.tests.search_strategies.checkbox_component()

openforms.formio.tests.search_strategies.columns_component()

openforms.formio.tests.search_strategies.content_component()

openforms.formio.tests.search_strategies.cosign_v1_component()

openforms.formio.tests.search_strategies.cosign_v2_component()

openforms.formio.tests.search_strategies.currency_component()

openforms.formio.tests.search_strategies.data_sources()

openforms.formio.tests.search_strategies.date_component()

openforms.formio.tests.search_strategies.datetime_component()

openforms.formio.tests.search_strategies.edit_grid_component()

openforms.formio.tests.search_strategies.email_component()

openforms.formio.tests.search_strategies.fieldset_component()

openforms.formio.tests.search_strategies.file_component()

openforms.formio.tests.search_strategies.formio_key()

A search strategy that produces valid Formio.js key values.

Formio.js keys must start and end with an alphanumeric character. Dashes and dots as separators are allowed. A value like foo..bar is valid, in the resulting data structure empty strings are used as keys: {"foo": {"": {"": {"bar": $value}}}}

See openforms.formio.validators.variable_key_validator() for the validator implementation.

This strategy differs slightly from the validator - it will generate keys with a maximum length of 100 chars.

openforms.formio.tests.search_strategies.iban_component()

openforms.formio.tests.search_strategies.license_plate_component()

openforms.formio.tests.search_strategies.map_component()

openforms.formio.tests.search_strategies.nested_components()

openforms.formio.tests.search_strategies.np_family_members_component()

openforms.formio.tests.search_strategies.number_component()

openforms.formio.tests.search_strategies.option()

openforms.formio.tests.search_strategies.phone_number_component()

openforms.formio.tests.search_strategies.postcode_component()

openforms.formio.tests.search_strategies.radio_component()

openforms.formio.tests.search_strategies.select_component()

openforms.formio.tests.search_strategies.selectboxes_component()

openforms.formio.tests.search_strategies.signature_component()

openforms.formio.tests.search_strategies.textarea_component()

openforms.formio.tests.search_strategies.textfield_component()

openforms.formio.tests.search_strategies.time_component()