Runtime Validations

Click here to return to the Advanced Ludwig table of contents.

What are Runtime Validations?

In addition to design-time validation, Ludwig supports runtime validation. A validation is a type of function that tests a property of your code, and if any portion of the code fails that test, it will not compile and cannot be executed. A runtime validation is a validation that is uploaded to the Conductor and that the Conductor enforces at runtime. In contrast, a design-time validation is checked by the Ludwig compiler (lwc), locally and at compile-time.

How Runtime Validations Work

Validation modules must be uploaded to the Conductor with the fugue policy validation-add command. At that time, the validation is tested against currently running processes, and if any running process fails validation, the validation module is not created on the Conductor, and the CLI returns an error message.

After the validation module has been successfully created on the Conductor, when a composition is run, the Conductor checks it against any validation modules that have been uploaded to the Conductor. If the composition passes validation, the process is created as normal. If the composition fails validation of any validation module, the Conductor returns an error message and the process is not created.

Design-Time vs. Runtime Validations

There are a few other differences between design-time and runtime validation:

  • A design-time validation is handled by the Ludwig compiler, locally, and does not require an installed Conductor.
    • A runtime validation requires an installed Conductor.
  • A design-time validation is stored in a Ludwig files called a module or library. For a composition to be validated, the validation must either be imported into the composition or specified with the --validation-modules option in lwc. Note: Importing a validation is a best practice for local development, but using --validation-modules may be preferable in certain CI/CD scenarios.
    • A runtime validation is stored on the Conductor. The Conductor tests all current and future processes for validation, because the validation module is present on the Conductor. Note: While it’s not strictly necessary to import the validation module in your composition, it’s better to do so for ease of local development.
  • A design-time validation is “opt-in,” because you can choose whether to include it in your composition.
    • A runtime validation is implemented no matter what’s in your composition.
Design-Time vs. Runtime Validations.

Design-Time vs. Runtime Validation

Maximum Validations

The maximum number of validation modules on a Conductor is 1,000.

Writing Runtime Validations

The process for writing a runtime validation is similar to writing a design-time validation. First, you write the validation function itself. Then, you need to register the validation by using the validate keyword, a step that allows the compiler to apply the validation to all relevant types in scope. Once you’ve done that, you’re free to write the rest of your code as usual. Applying the validation module to the Conductor is the final step.

We’ll use the same type of validation we discussed in Design-Time Validations to demonstrate the process.

Writing the Validation Function

Once again, if you need a refresher on how functions work, check out our Functions Tutorial.

This validation ensures that compositions only run in the us-west-2 region. If a composition uses a region other than us-west-2, it will fail validation.

First, we’ll import the Fugue.AWS module. This module contains the region types.

import Fugue.AWS as AWS

Next comes the actual function.

fun usWest2Only(region: AWS.Region) -> Validation:
  case region of
  | AWS.Us-west-2  -> Validation.success
  | _              -> Validation.error {message: "Infrastructure is allowed in us-west-2 only."}

If this validation looks familiar, it’s because it’s exactly the same as the one we used in the Design-Time Validations example. To recap, any instance of AWS.Region in a composition will be checked for the value AWS.Us-west-2, and if it’s there, the validation succeeds. If the AWS.Region value is anything else, the validation fails.

Registering the Validation

As before, in order to apply our validation to all AWS.Region occurrences in scope at compile time, we must register the validation by using the validate keyword and the name of the validation function:

validate usWest2Only

Applying a Validation Module to the Conductor

We’ve written a validation! As you can see, it is identical to the validation used for design-time validation. The next step is where the process really differs.

With design-time validation, the validation module must be imported into any composition it validates (or it may all be written in the same file). But with runtime validation, the validation module must be uploaded to the Conductor. Once it has been uploaded to the Conductor, the validation module can be downloaded, listed, or deleted. These actions can be accomplished by executing policy validation-* commands through the Fugue CLI.

There are four policy validation-* commands:

For more details on each of these commands, see the policy page.

For an in-depth walkthrough of how to add, remove, and list runtime validation modules, along with how to test them, see Enforcing Policy with Runtime Validations.

Best Practices for Authoring Runtime Validations

When authoring runtime validations, keep in mind:

Validations should not use @override bindings.

The use of @override is an anti-pattern for validations and can lead to unwanted side effects at runtime. Given the appropriate user permissions in Fugue, validations with @override bindings can be altered at runtime to allow behavior that violates the intention of the validation. Fugue disallows @override bindings in some, but not all, conditions. Avoid this issue entirely by writing runtime validations without the use of @override bindings.

Running Compositions

When a composition that is run passes runtime validation, the process is created as usual. If the composition fails runtime validation, however, you’ll see an error message determined by the code in the validation function. The full message returned by the Conductor might look like this:

[ fugue run ] Running /Users/main-user/projects/HelloWorld.lw

Run Details:
    Account: default
    Alias: n/a

Compiling Ludwig file /Users/main-user/projects/HelloWorld.lw
[ OK ] Successfully compiled. No errors.

Uploading compiled Ludwig composition to S3...
[ OK ] Successfully uploaded.

Requesting the Conductor to create and run process based on composition ...
[ ERROR ] ludwig (validation error):
  "/tmp/921678480/composition/src/HelloWorld.lw" (line 13, column 11):
  Validations failed:

    13|   region: AWS.Us-west-1,

    - Infrastructure is allowed in us-west-2 only.
      (from UsWest2OnlyConductorValidation.usWest2Only)

The output highlights the erroneous code and lists its line and column number. It also includes the error message written in the validation function and the validation module it was found in. In the above case, UsWest2OnlyConductorValidation.usWest2Only indicates that the original filename of the validation module is UsWest2OnlyConductorValidation.lw and the name of the validation function inside it is usWest2Only.

The output also includes a file location stating where the composition snapshot is saved on the Conductor – in this case, /tmp/921678480/composition/src/HelloWorld.lw. To learn more about snapshots, see the next section.

Runtime Validations and Snapshots

lwc, the Ludwig compiler, is capable of compiling compositions into snapshots. A snapshot is a gzipped tarball containing several files:

File containing the name of the validation module and the version of lwc.
JSON file containing environment variables and file mapping.
A folder containing a copy of the Fugue Standard Library modules and any other modules required for validations.
A folder containing the entry point for the validation.

When you fugue run or fugue update a composition, or when you create a validation module on the Conductor with fugue policy validation-add, lwc compiles the composition into a snapshot and the Fugue CLI uploads the snapshot to S3. The Conductor then downloads the snapshot from S3 and extracts it locally.

That’s why the error message for a composition that fails runtime validation includes the /tmp/ file location of the extracted snapshot. In the example in above, it was /tmp/921678480/composition/src/HelloWorld.lw, a location on the Conductor instance.

Snapshots and Differing Libraries

Say that the runtime validation MyValidation.lw and the composition MyComposition.lw both import the library MyLibrary. Because a snapshot includes Ludwig libraries as they are at compilation time, it’s possible for the contents of MyLibrary to differ between the validation snapshot and the composition snapshot. (For example, perhaps MyLibrary was changed after the runtime validation was created but before MyComposition.lw was written.) In this case, the Conductor honors the version of the library contained in the validation snapshot.

The exception is Fugue.* libraries, which are included in the Conductor itself. The Fugue.* library version on the Conductor will always take precedence over the Fugue.* libraries packaged with validations or compositions.

Overloaded Environment Variables on the Conductor

On the Conductor, certain environment variables are used by Ludwig or the compilation service and cannot be overriden by a user. For example, environment variables prefixed with FUGUE_RUNTIME_ are explicitly overloaded on the Conductor. Since these environment variables are used to inject runtime information into the compilation, the user may overload them locally but the values will be discarded on the Conductor. Another example of an environment variable the user can overload locally but not on the Conductor is LUDWIG_PATH.

Further Reading

This is just a simple example of how validations can be used, but the sky’s the limit. For a more complex demonstration, see our policy-as-code Github repo. You can also see a collection of validation modules in our validation-examples-docs Github repo.