Guide to Writing Validations, Part 1

This two-part guide explains how to write validations. You’ll learn how to interpret the Standard Library Reference documentation to determine which fields to validate. You’ll get to know the basic three-step process to writing the validation. And we’ll use real-world examples throughout the guide, starting simple in Part 1 and covering more complex topics in Part 2.

What is a Validation?

A validation is a type of function that tests a property of your code, and if any portion of the code fails that test, it will not compile and cannot be executed. By writing validations that enforce internal requirements and external regulations, you can ensure that your infrastructure always adheres to security and compliance policies.

For example, a validation can verify that your AWS S3 buckets are configured to store only encrypted files. If a user attempts to run a composition with an S3 bucket that is not properly configured, the validation will prevent the composition from running. As a result, only compliant processes are running in your account at any given time.

A runtime validation is a validation that is uploaded to the Conductor and enforced at runtime. In contrast, a design-time validation is checked by the Ludwig compiler (lwc), locally and at compile-time.

This guide focuses on writing validations, with less emphasis on applying them. All validations can be used at design-time or runtime.

Note: If you’re looking for information about NodeStream validations, which validate across types and values, see NodeStream Validations Across Types and Values.

The Importance of Validations

It’s true that type safety goes a long way. Ludwig is statically typed, so the compiler catches type errors. For example, if a composition declares an EC2.Vpc but omits the required cidrBlock field, the Ludwig compiler will throw an error. Or, if the cidrBlock field contains an integer instead of a string, lwc catches that, too.

But what if cidrBlock contains a nonsensical string? Ludwig’s type safety system can only check that the type is String – it doesn’t check the contents. Whether your cidrBlock is "10.0.0.0/16" or "cats" is irrelevant.

A validation here can help. You can write a validation that uses regular expressions to determine whether a string is formatted as a CIDR block value.

You can also use validations for things like restricting, limiting, or requiring certain properties. For example, a validation could restrict the type of EC2 instance used in a composition by throwing an error if it’s not T2.micro. Or it could limit the number of S3 buckets declared in a composition. A validation could also require VPC flow logging to be enabled in all VPCs in a composition. Validations are an important tool in maintaining security and compliance – they’re guard rails for your infrastructure.

Now that we’ve discussed some reasons you might want to write a validation, let’s get down to business: writing your first validation.

Prerequisites

  1. You’ll need to install the Fugue Client Tools. If you haven’t done so yet, it just takes a few quick steps.
  2. We recommend writing your validation in VSCode using our plug-in and language server to get error highlighting, hover text, symbol searches, and more. For setup information, see Ludwig LSP Server. (We also offer simpler Ludwig plug-ins for other text editors.)
  3. The Standard Library Reference is necessary for knowing how our resources are defined. You’ll find documentation for all of Fugue’s modules and core types there.
  4. In order to test your validation out, it’s useful to have a terminal open so you can run lwc, the Ludwig compiler.

An easy field validation

There are three basic steps to writing just about any validation:

  1. Determine the resource you want to validate.
  2. Obtain the resource field (or other info) that needs to be validated.
  3. Perform the check on the field.

We’ll start by writing a simple validation on a simple field of a resource: checking whether an AWS Classic Load Balancer is in the correct region.

Determine the resource we want to validate

Open up your text editor and create a Ludwig file by saving it with a .lw extension. We’re going to call ours MyValidation.lw.

The first thing we’ll need to do is import the appropriate libraries. Since we’re writing a validation on a load balancer, and the LoadBalancer Ludwig type is in the ELB service, we import Fugue.AWS.ELB. We also import the Fugue.AWS module to access region definitions.

import Fugue.AWS.ELB as ELB
import Fugue.AWS as AWS

The “as ELB” part gives the whole module an alias, or a shorter name. If we skipped that, we’d have to write the full module name every time we reference it, which gets tedious very fast. Now we can just write ELB.LoadBalancer instead of Fugue.AWS.ELB.LoadBalancer.

Validations are Ludwig functions that determine whether or not a resource is valid. We need to give our validation function a name, which must start with a lowercase character. Here’s our updated code, in full:

import Fugue.AWS.ELB as ELB
import Fugue.AWS as AWS

fun mustBeInUsEast1(lb: ELB.LoadBalancer) -> Validation:
  Validation.success

Let’s dissect that a little:

  • mustBeInUsEast1 is the name of the function.
  • The function has one argument, lb, of the type ELB.LoadBalancer. Ludwig types start with an uppercase character and variable names with a lowercase character.
  • After the arguments is the return type. For all validations, this will just be Validation.
  • Following the return type is a colon : and then the body of the function.

The body of the function is really the logic of your validation. This is where we’ll determine whether or not the resource is valid. We’ve used Validation.success for now, which is a validation that always succeeds – we’ll add more in a bit.

You might be wondering where the Validation type came from, since we didn’t import it like we did with ELB. Ludwig imports a few carefully chosen standard modules, like Ludwig.List, Ludwig.Int, and Ludwig.Validation by default.

Now we can test this validation by compiling it with lwc, the Ludwig compiler. Run this command in your terminal, changing the filename if necessary:

lwc MyValidation.lw

For now, we will see a warning that we are not using the lb argument, which is correct: that’s what we’ll get to now!

Obtaining the fields that we want to validate

The next step in writing a validation is extracting the field we want to validate. Since our validation focuses on the load balancer’s region, we need to look at the LoadBalancer type and find the relevant field.

If we look at the Standard Library Reference, we can easily see that a LoadBalancer is created by the ELB.LoadBalancer.new function.

But we’re not interested in creating one – we want to validate them. Under the “Returns:” section of the documentation, we can see that the new function lists LoadBalancer as the return type. If you click on the return type, you’ll be taken to the exact definition of the type. That looks a bit like this:

type LoadBalancer:
  | LoadBalancer
      loadBalancerName: String
      # ... Many more fields ...
      region: Region

Bingo! That’s the information we’re looking for – it’s a list of all the parameters that make up the LoadBalancer type. In Ludwig, all cloud resources are represented as sum types. Very informally, that means that after the type name ( type LoadBalancer: ) there may be multiple constructors. In this case, we only have one constructor, conveniently also called LoadBalancer. Types with single constructors usually have the same name as their constructor, by convention. Types with more than one constructor are less common, so we’ll talk about them later in this guide.

The LoadBalancer constructor (denoted with a pipe | above) contains a record, and that record holds the parameters comprising the LoadBalancer type. A record is a type of binding that holds multiple fields. We’ve omitted most of the fields above, but you can see the required region field, which is of the AWS.Region type. That’s the field we need for our validation.

(For more on bindings, records, and fields, see the Ludwig Syntax Guide.)

Now we’re going to create a local variable to hold the region of the LoadBalancer. Our validation ends up looking like this:

import Fugue.AWS.ELB as ELB
import Fugue.AWS as AWS

fun mustBeInUsEast1(lb: ELB.LoadBalancer) -> Validation:
  let AWS.Region region: lb.(ELB.LoadBalancer).region
  Validation.success

Let’s break it down again.

  • let is the keyword to introduce a new local variable.
  • We’ve declared the type of our local variable, namely AWS.Region. Declaring this type is optional because the compiler figures it out on its own, so we could have used let region: instead.
  • We use a dot . to access things. If there is only one constructor, we can use the name of that constructor to access its “guts.” From the internals, we then want to pick the region field. So, the syntax lb.(ELB.LoadBalancer).region just means that in the parameter lb (which is of the ELB.LoadBalancer type), we want to drill down to the ELB.LoadBalancer constructor, and from the constructor’s record, select the region field.

If all went well, you should again be able to compile your file using lwc:

lwc MyValidation.lw

Now we get a warning that region is unused, but that’s okay – we’ll use it in the next section!

Performing the check

Now that we’ve got all the required information, we can swap out Validation.success for the validation that we really want: verifying the region.

Most validations in Ludwig can be written using Validation.error and a condition. For simple conditions, we can use the when parameter of Validation.error. The when parameter is a Bool (Boolean) that tells Ludwig when to trigger the error. Our validation now looks like this:

import Fugue.AWS.ELB as ELB
import Fugue.AWS as AWS

fun mustBeInUsEast1(lb: ELB.LoadBalancer) -> Validation:
  let AWS.Region region: lb.(ELB.LoadBalancer).region
  Validation.error {
    when: region != AWS.Us-east-1,
    message: "LoadBalancer must be in us-east-1"
  }

We’ve replaced Validation.success with Validation.error, which has two parameters:

  • when specifies the condition that triggers the error. It’s the logic that decides whether or not a resource is valid.
  • message is required. It’s the string that users will see when they trigger the validation error.

In this case, we want to raise the error whenever the region is not us-east-1. In Ludwig, we can compare most things using == for equality and != for inequality. The result of that comparison yields a Bool, which is exactly what the when parameter expects. If that Bool is True, the error is triggered.

So, if a composition declares a LoadBalancer in a region that isn’t us-east-1, our validation raises an error and the composition fails compilation.

And that’s our first validation! Before you can try it out on a composition, you must register it by using the validate keyword:

import Fugue.AWS.ELB as ELB
import Fugue.AWS as AWS

fun mustBeInUsEast1(lb: ELB.LoadBalancer) -> Validation:
  let AWS.Region region: lb.(ELB.LoadBalancer).region
  Validation.error {
    when: region != AWS.Us-east-1,
    message: "LoadBalancer must be in us-east-1"
  }

validate mustBeInUsEast1

(You can download this validation from Github.)

Registering the validation essentially tells the compiler to turn the validation on.

Now you can declare a legal (or illegal!) load balancer in a composition and see if it passes the validation.

Testing the validation

There are two ways to apply validation modules: at design-time or at runtime. Design-time validations are enforced locally at compile-time. Runtime validations are enforced by the Conductor at runtime. You can read more at Design-Time vs. Runtime Validations.

Since we’re working locally, it makes sense to test our validation locally, at design-time.

If you need a fresh copy of our validation module, you can download it from Github. Then you’ll need to download this ELB composition, because we’re going to import our validation into it and compile it. The ELB.lw composition declares a load balancer and associated infrastructure in the AWS.Us-west-2 region, which you can see in line 53.

To test your validation, you’ll need to import the validation module. Now, edit ELB.lw and add the following line near the other imports (line 9):

import MyValidation

If you named your validation module something different, use that instead of MyValidation (and be sure to omit the .lw extension).

Now you can compile the composition:

lwc ELB.lw

You should see this output:

ludwig (validation error):
  "ELB.lw" (line 22, column 17):
  Validation failed:

    22| exampleDemoElb: ELB.LoadBalancer.new {
    23|   loadBalancerName: "exampleElbElb",
    24|   subnets: [exampleSubnet],
    25|   healthCheck: ELB.HealthCheck.tcp {
    26|     port: 3000,
    27|     interval: 15,
    28|     timeout: 3,
    29|     unhealthyThreshold: 3,
    30|     healthyThreshold: 3
    31|   },
    32|   securityGroups: [exampleSecurityGroup],
    33|   listeners: [exampleDemoListener],
    34| }

  LoadBalancer must be in us-east-1

  (from MyValidation.mustBeInUsEast1)

Great! That means our validation works as expected – it raised an error because the load balancer wasn’t in us-east-1.

If you like, you can change the region declared in the VPC on line 53 to AWS.Us-east-1 and try compiling the composition again. This time, you’ll see no output, which means the composition passed validation and successfully compiled.

Note

It’s a best practice to import the validation into the composition, but you can use lwc’s --validation-modules option to specify the validation modules to be used. Every validation registered in those modules will be used:

lwc ELB.lw --validation-modules MyValidation

Once again, don’t include the .lw extension.

Dealing with optional fields

In the last section, we introduced the when parameter. There are many, many ways to implement the logic for it. We’ll review the most common ones, including numbers, strings and regexes, and working with lists. But first, we’re going to take a small detour to see how we can conveniently deal with optional fields.

A lot of fields in Ludwig types are not required by the AWS API. These fields represent optional types. An optional type is noted with Optional<t> syntax, where t is the type that is optional, like Int or String. For example, the idleTimeout field in ELB.LoadBalancer.new is marked Optional<Int>, which indicates that the field takes an optional integer. That means it’s OK if you don’t include the field when you declare a new LoadBalancer, but if you do, its value must be an Int.

In general, there are two ways to write logic around optional fields:

  1. If they’re missing, assign them a default value.
  2. Take one action if they’re missing and a different action if they’re present.

Let’s dig into the first method.

Assigning defaults

The easiest way to handle optional fields is by assigning defaults. We can use a simple Ludwig operator called ?|. It sits between an optional field and a default value, like so: Optional("Hello") ?| "Ignored".

The ?| operator checks if the optional field on the left is present, and uses the default value on the right if it isn’t. So the example None ?| "Ignored" returns the default value "Ignored", because there isn’t an optional field on the left. And the example Optional("Hello") ?| "Ignored" returns "Hello", the optional field.

As an example, let’s see if we can use this to validate an optional field for our LoadBalancer. If we look at the LoadBalancer type again in the Standard Library Reference, we see the following:

type LoadBalancer:
  | LoadBalancer
      loadBalancerName: String
      ...
      tags: Optional<List<Tag>>

With the knowledge we already have, we can easily write a validation to make sure the load balancer’s tags are present. We use the same technique we used earlier to obtain the data we want – the tags field.

import Fugue.AWS.ELB as ELB
import Fugue.AWS as AWS

fun mustBeTagged(lb: ELB.LoadBalancer) -> Validation:
  let tags: lb.(ELB.LoadBalancer).tags ?| []
  Validation.error {
    when: List.isEmpty(tags),
    message: "LoadBalancer must be tagged"
  }

validate mustBeTagged

(You can download this validation from Github.)

Our validation checks a LoadBalancer to see if the tags field is present. If it is, the local variable tags – which is a List<AWS.Tag> – is assigned the value of the tags field in the composition. If the load balancer’s tags field is not present, the local variable tags is assigned an empty list [].

The key differences from the last validation are:

  • We are extracting the tags field, and that’s also the name of our local variable. Last time, we declared the local variable’s type, but this time we’ve left it out since the compiler infers the type automatically.
    • If we’d left it in, it would look like this: let List<AWS.Tag> region:
  • We now use the ?| operator to assign a default value of [] if the tags field isn’t present on an ELB.LoadBalancer.
  • Our when condition uses the function List.isEmpty, which tests whether a list is empty, as you might expect.

If you’re wondering why the local variable tags has the type List<AWS.Tag> instead of Optional<List<AWS.Tag>>, it’s because of the ?| operator, which is shorthand for the Optional.unpack function. ?| unpacks the Optional by looking inside it to check if a value is present. Once the function answers the question “is the value there or not?” then the answer is either “yes” (and the function returns the unpacked value) or “no” (and the function returns a default value) – there’s no longer a “maybe” (Optional). So here, the return type is List<AWS.Tag> instead of Optional<List<AWS.Tag>>, and that’s what local variable tags is too.

If you want to read up on Optional, check out Ludwig.Optional in the Standard Library Reference, our Advanced Ludwig Syntax guide, or the validation in Validations to Enforce Limits on Security Group Rules, which uses the same logic we used here.

Branching logic

Now we’re going to look at the second way to validate on optional fields.

Sometimes there is no good way to assign a default, or we just want to do something entirely different if there isn’t a value. In these cases, we can use a case expression. A case statement is similar to an if-else statement in other programming languages. Ludwig has an if-else statement, too, but the case statement is often more useful when it comes to optionals.

We’ll come back to case statements later when we look at validating other sum types. After that, the usage and benefit of these statements should become much clearer.

Informally, a case statement lets you check a what a value looks like and do different things depending on the answer. If we update the example from above to use a case statement, it would look like this:

import Fugue.AWS.ELB as ELB
import Fugue.AWS as AWS

fun mustBeTagged(lb: ELB.LoadBalancer) -> Validation:
  let optTags: lb.(ELB.LoadBalancer).tags
  case optTags of
  | None ->
    Validation.error {
      message: "No tags are set!"
    }
  | Optional tags ->
    Validation.error {
      when: List.isEmpty(tags),
      message: "LoadBalancer must be tagged"
    }

validate mustBeTagged

(You can download this validation from Github.)

The changes are:

  • We did not use ?|, which means that optTags is of the type Optional<List<AWS.Tag>> rather than List<AWS.Tag>. As such, we can’t use List.isEmpty on it.
  • We use a case statement to determine what optTags is.
    • In the simplest case, it is absent. That means the value of optTags is None. In that case, we have a Validation.error.
    • In the other case, it is present and it is of the form Optional tags. Here, tags is just a name that we can choose – very similar to a local variable.
  • The variable tags is only available in the second branch and it is of the type List<AWS.Tag> rather than Optional<List<AWS.Tag>>. This means we can use List.isEmpty on it.

When we write case statements, we must write logic around all possible cases. If we were to end the validation after the None case, the compiler would warn us that the “pattern matching is not exhaustive” because we haven’t accounted for every possible outcome. But since we’ve handled the only two possibilities – remember, an Optional value is either present, or it isn’t – our validation compiles without a problem.

It might seem a bit strange at first to account for all cases, but it’ll make more sense soon, especially after we look at validations on proper sum types. The important thing to note is that Ludwig places a lot of emphasis on safety by design. The compiler requires us to handle all cases, which prevents a whole class of issues called null pointer references that persists in other languages.

But wait, there’s more!

Up next, in Part 2, we’ll show you common validations you can use on fields, and we’ll explain sum types, list comprehensions, and other useful ways to write validation logic. We’ll also include a full example that puts together everything we’ve learned.

Ready? Head on over to Guide to Writing Validations, Part 2!