Building A Network: VPC Peering

Overview

This example shows you how to create three VPCs with two VPC peering connections in Amazon Web Services using Fugue.

Prerequisites

You’ll need to have the Fugue CLI set up and the Conductor installed before you can run this example. If you haven’t done so yet, it just takes a few quick steps.

This walkthrough is designed to come after Building A Network. Although it’s not necessary to start there, we’d recommend doing so for a fuller understanding of the material here.

What We’ll Do In This Example

We’ll cover how to use Ludwig to create additional VPCs and two peering connections.

What We’ll Have When We’re Done

Three VPCs in AWS – the example-vpc from Building A Network, plus another Fugue-managed VPC and one non-Fugue-managed VPC. We’ll also have two VPC peering connections. The first one connects the example-vpc to the other Fugue-managed VPC, and the second one connects the example-vpc to the non-Fugue-managed VPC. In both cases, the connected VPCs will be set up with routing between their private subnets only.

How Long It Will Take

About 30 minutes.

Download

You can download the source code for this example here. You’ll need to edit it manually when you get to a later step.

Get editor plug-ins here.

Let’s Go!

Boilerplate

As before, we’ll want to declare that our file is a composition (using the composition keyword). We’ll also import the base AWS library, as well as the EC2 library.

composition

import Fugue.AWS as AWS
import Fugue.AWS.EC2 as EC2

Adding configuration variables

When we create the non-Fugue-managed VPC in the AWS Management Console a little later in this walkthrough, we’ll need a way to reference its ID and the account ID in which it was created. Don’t worry about these values for now – just leave them commented out and we’ll cross that bridge when we get to it.

# Added config variables
# non-managed-vpc-id: "vpc-1234abcd"  # <VPCID>
# non-managed-acc-id: "012345678912"  # <AccID>

Making a VPC

We’re ready to start declaring infrastructure. If you completed the Building A Network example, this should look pretty familiar. The only difference is that we’ve enabled DNS support (3rd line from bottom) and DNS hostnames (2nd line from bottom). That allows us to use the EC2.PeeringOptions.allowDnsResolution option for each VPC in the peering connection we’ll declare later.

example-vpc: EC2.Vpc.new {
  cidrBlock: "10.0.0.0/16",
  tags: [
    AWS.tag("Name", "My Cool VPC")
  ],
  region: AWS.Us-west-2,
  enableDnsSupport: True,  # Enabled DNS support
  enableDnsHostnames: True  # Enabled DNS hostnames
}

Making a public subnet

This should look pretty familiar, too. The only difference from the composition in the Building A Network example is that we’ve added a tag so it’s easier to tell our VPCs’ infrastructure apart.

public-10-0-1-0: EC2.Subnet.new {
  cidrBlock: '10.0.1.0/24',
  vpc: example-vpc,
  availabilityZone: AWS.A,
  mapPublicIpOnLaunch: True,
  defaultForAz: False,
  tags: [
    AWS.tag("Name", "My Cool VPC Public Subnet")
  ]  # Added tag
}

Adding a private subnet

Previously, we’d included a second public subnet, but for this example it’s not necessary. Instead, we’ll create a private subnet, since we’ll need that for our peering connections. It’ll be in the same availability zone as the public subnet.

# Replaced second public subnet with private subnet
private-10-0-2-0: EC2.Subnet.new {
  cidrBlock: '10.0.2.0/24',
  vpc: example-vpc,
  availabilityZone: AWS.A,
  tags: [
    AWS.tag("Name", "My Cool VPC Private Subnet")
  ]
}

Adding an internet gateway and public route

We’ll need an internet gateway (IGW) and a public route, as before. The only difference here is that we’ve added a tag to the IGW, and we’ve changed the name of the public route (again, to differentiate from other VPCs’ public routes).

example-igw: EC2.InternetGateway.new {
  vpc: example-vpc,
  tags: [
    AWS.tag("Name", "My Cool VPC IGW")
  ]  # Added tag
}

# Added "example" to name to differentiate from other VPC routes
example-public-route: EC2.Route.new {
  destinationCidrBlock: "0.0.0.0/0",
  target: EC2.GatewayTarget(example-igw)
}

Creating a peer route from our example VPC to a Fugue-managed VPC

We haven’t created the Fugue-managed VPC yet, but when we do, we’ll need to create a VPC peering route so that traffic can flow from example-vpc to the Fugue-managed VPC (which we’ll name managed-peer-vpc).

The peer route points from example-vpc to the managed-vpc-peering connection we’ll create further down, using the CIDR block of managed-peer-vpc.

# Added VPC peer route from example-vpc to managed-vpc-peering
example-vpc-peer-route: EC2.Route.new {
  destinationCidrBlock: managed-peer-vpc.(EC2.Vpc).cidrBlock,
  target: EC2.VpcPeeringConnectionTarget(managed-vpc-peering),
}

Add VPC public route table and private peer route table

The public route table here is the same as the one we used in the previous example, but we’ve renamed it for clarity. We also deleted the reference to the second public subnet, since we removed it from this composition.

Below that, you’ll see that we’ve created a private route table for the example-vpc-peer-route, associated with our private subnet, private-10-0-2-0.

# Added "example-vpc" to name to differentiate from other VPC route tables
example-vpc-public-route-table: EC2.RouteTable.new {
  vpc: example-vpc,
  routes: [example-public-route],  # Updated name
  associations: [public-10-0-1-0],  # Deleted second public subnet
  tags: [
    AWS.tag("Name", "My Cool VPC Route Table")
  ]  # Added tag
}

# Added VPC peer route table
example-vpc-peer-route-table: EC2.RouteTable.new{
  vpc: example-vpc,
  routes: [example-vpc-peer-route],
  associations: [private-10-0-2-0],
  tags: [
    AWS.tag("Name", "My Cool VPC Peer Route Table")
  ]
}

Security groups and IP permissions

Nothing new here! We still need security groups for example-vpc, so we’ve used the same ones defined in Building A Network.

example-elb-sg: EC2.SecurityGroup.new {
  description: "Allow http/s traffic from the Internet",
  ipPermissions: [
    inet-HTTP,
    inet-HTTPS
  ],
  ipPermissionsEgress: None,
  vpc: example-vpc
}

inet-HTTP: EC2.IpPermission.new {
  ipProtocol: "tcp",
  fromPort: 80,
  toPort: 80,
  target: EC2.IpRanges([
    EC2.IpRange {cidrIp: "0.0.0.0/0"}
  ])
}

# Note the short form of this ^ is:
#   EC2.IpPermission.http(EC2.IpPermission.Target.all)
# Similar functions exist for common protocols. We've used the long form in this example to be more illustrative.

inet-HTTPS: EC2.IpPermission.new {
  ipProtocol: "tcp",
  fromPort: 443,
  toPort: 443,
  target: EC2.IpRanges([
    EC2.IpRange {cidrIp: "0.0.0.0/0"}
  ])
}

example-app-sg: EC2.SecurityGroup.new {
  description: "Allow traffic from the internet-facing SG",
  ipPermissions: [
    EC2.IpPermission.tcp(3000, EC2.IpPermission.Target.securityGroup(example-elb-sg)),
    # This ^ is the shorthand for both IP permissions and targets.
  ],
  vpc: example-vpc
}

Creating a Fugue-managed VPC

Now it’s time to create our other Fugue-managed VPC, managed-peer-vpc. AWS stipulates that the peer VPC “cannot have a CIDR block that overlaps with the requester VPCs CIDR block,” so we’ll assign it a CIDR block of 10.1.0.0/16, in comparison with the example-vpc CIDR block of 10.0.0.0/16. We’ll put both VPCs in the same region, AWS.Us-west-2, though they could just as easily reside in separate regions.

### Fugue-managed VPC to peer with example-vpc
managed-peer-vpc: EC2.Vpc.new {
  cidrBlock: "10.1.0.0/16",
  region: AWS.Us-west-2,
  enableDnsSupport: True,
  enableDnsHostnames: True,
  tags: [
    AWS.tag("Name", "Fugue-Managed Peer VPC")
  ]
}

Adding public and private subnets to the second VPC

Just like example-vpc, managed-peer-vpc needs a public subnet and a private subnet...

# Public subnet for managed-peer-vpc
public-10-1-1-0: EC2.Subnet.new {
  cidrBlock: '10.1.1.0/24',
  vpc: managed-peer-vpc,
  availabilityZone: AWS.A,
  mapPublicIpOnLaunch: True,
  tags: [
    AWS.tag("Name", "Fugue-Managed Peer VPC Public Subnet")
  ]
}

# Private subnet for managed-peer-vpc
private-10-1-2-0: EC2.Subnet.new {
  cidrBlock: '10.1.2.0/24',
  vpc: managed-peer-vpc,
  availabilityZone: AWS.A,
  tags: [
    AWS.tag("Name", "Fugue-Managed Peer VPC Private Subnet")
  ]
}

Adding an IGW to the second VPC

...And an IGW...

# IGW for managed-peer-vpc
managed-peer-vpc-igw: EC2.InternetGateway.new {
  vpc: managed-peer-vpc,
  tags: [
    AWS.tag("Name", "Fugue-Managed Peer VPC IGW")
  ]
}

Adding routes and route tables to the second VPC

...And a public route, peer route, public route table, and private route table.

Similar to how example-vpc-peer-route works, managed-peer-vpc-peer-route points from managed-peer-vpc to the managed-vpc-peering connection we’ll create a little later, using the CIDR block of example-vpc.

# Public route for managed-peer-vpc
managed-peer-vpc-public-route: EC2.Route.new {
  destinationCidrBlock: "0.0.0.0/0",
  target: EC2.GatewayTarget(managed-peer-vpc-igw),
}

# Peer route from managed-peer-vpc to managed-vpc-peering
managed-peer-vpc-peer-route: EC2.Route.new {
  destinationCidrBlock: example-vpc.(EC2.Vpc).cidrBlock,
  target: EC2.VpcPeeringConnectionTarget(managed-vpc-peering),
}

# Public route table for managed-peer-vpc
managed-peer-vpc-public-route-table: EC2.RouteTable.new {
  vpc: managed-peer-vpc,
  routes: [managed-peer-vpc-public-route],
  associations: [public-10-1-1-0],
  tags: [
    AWS.tag("Name", "Fugue-Managed Peer VPC Public Route Table")
  ]
}

# Private route table for managed-peer-vpc
managed-peer-vpc-private-route-table: EC2.RouteTable.new {
  vpc: managed-peer-vpc,
  routes: [managed-peer-vpc-peer-route],
  associations: [private-10-1-2-0],
  tags: [
    AWS.tag("Name", "Fugue-Managed Peer VPC Peering Route Table")
  ]
}

Adding a peering connection from example-vpc to managed-peer-vpc

Finally, we’re about to create our peering connection, managed-vpc-peering. This sets up example-vpc to be the control VPC and managed-peer-vpc to be the peer VPC. You can read more about VPC peering connections in the Fugue.AWS.EC2.VpcPeeringConnection portion of the Fugue Standard Library Reference.

### example-vpc to managed-peer-vpc peering connection
managed-vpc-peering: EC2.VpcPeeringConnection.new {
  vpc: example-vpc,
  options: EC2.PeeringOptions.allowDnsResolution,
  peerVpc: EC2.ManagedVpc {
    vpc: managed-peer-vpc,
    options: EC2.PeeringOptions.allowDnsResolution,
  },
}

Adding a peering connection from example-vpc to a non-Fugue-managed VPC

The last item in the composition is currently commented out. This bit of code will eventually create a peering connection from example-vpc to a non-Fugue-managed VPC, which you’ll create in your account shortly. We’re going to run the composition as it is first, and then we’ll uncomment this section and update the process. But for now, go ahead and leave the hash marks where they are.

### example-vpc to non-managed-peer-vpc peering connection
# non-managed-peer-vpc-peering: EC2.VpcPeeringConnection.new {
#   vpc: example-vpc,
#   options: EC2.PeeringOptions.allowDnsResolution,
#   peerVpc: EC2.UnmanagedVpc {
#     vpcIdentifier: non-managed-vpc-id,  # <VPCID>
#     owner: non-managed-acc-id  # <AccID>
#   },
# }

Starting the Fugue process

All right, that’s our composition! Now we’re ready to launch the composition as a process by running it.

$ fugue run VPCPeering.lw -a peering

You’ll see output that looks like this:

[ fugue run ] Running VPCPeering.lw

Run Details:
    Account: default
    Alias: peering

Compiling Ludwig file /Users/user/projects/VPCPeering.lw
[ OK ] Successfully compiled. No errors.


Uploading compiled Ludwig composition to S3...
[ OK ] Successfully uploaded.

Requesting the Conductor to create and run process based on composition ...
[ DONE ] Process created and running.

State    Updated    Created    Account              FID                                   Alias    Last Message    Next Command
-------  ---------  ---------  -------------------  ------------------------------------  -------  --------------  --------------
Running  9:01pm     9:01pm     fugue-1505320402664  7c3a6ac3-c57a-405d-9771-dd92edc137b3  peering                  run

[ HELP ] Run the "fugue status" command to view details and status for all Fugue processes.

After a couple minutes, run fugue status to see how your process is doing. You should see a SUCCEEDED message, like so:

Fugue Status Report for user/xxxxxxxxxxxx - Sun Nov 27 2016 9:04pm

State    Updated    Created    Account              FID                                   Alias    Last Message    Next Command
-------  ---------  ---------  -------------------  ------------------------------------  -------  --------------  --------------
Running  9:01pm     9:01pm     fugue-1505320402664  7c3a6ac3-c57a-405d-9771-dd92edc137b3  peering  SUCCEEDED

A quick check of the AWS Console confirms that we have the VPCs as described.

My Cool VPC, or example-vpc.

My Cool VPC, or example-vpc.

Fugue-Managed Peer VPC, or managed-peer-vpc.

Fugue-Managed Peer VPC, or managed-peer-vpc.

Let’s take a look for that VPC peering connection now. In the sidebar of the VPC Dashboard, click “Peering Connections,” and you should see something like this:

The peering connection between example-vpc and managed-peer-vpc.

The peering connection between example-vpc and managed-peer-vpc.

Pretty cool, right? But wait, there’s more!

Creating a non-Fugue-managed VPC

We’re going to create a non-Fugue-managed VPC from the AWS Console in order to show off Fugue’s ability not only to connect two Fugue-managed VPCs (example-vpc and managed-peer-vpc), but also to connect a Fugue-managed VPC (example-vpc) and a non-Fugue-managed VPC.

Go back to the VPC Dashboard and ensure you’re still in the Oregon region. Select “Your VPCs” in the left sidebar, then click on the “Create VPC” button. Give your new VPC a name tag (we opted for non-fugue-managed-vpc), and then a CIDR block that doesn’t overlap with the other two VPCs’ CIDR blocks (we went with 10.2.0.0/16). Leave “Tenancy” as “Default” and click “Yes, Create.” You should see your new VPC pop up in the list of VPCs.

Action required! Updating the Ludwig code

Now, copy the new VPC’s ID (found underneath the “VPC ID” column), and let’s go back to lines 7 and 8 in VPCPeering.lw.

# non-managed-vpc-id: "vpc-1234abcd"  # <VPCID>
# non-managed-acc-id: "012345678912"  # <AccID>

Uncomment these lines by removing the hash marks, then paste the copied VPC ID in line 7 and enter your account number in line 8. (You can find your account number by visiting the Support Center and checking the upper-right corner of the screen, or simply by copying it from the fugue status output.)

Note

Yes, this means you can connect VPCs across accounts! For this exercise, we’re focusing on peering connections between VPCs within the same account, but for cross-account VPC peering, all you need is the other account ID – and the account owner’s permission, of course.

The last thing we need to do with VPCPeering.lw is to uncomment the lines at the very bottom, 204-211. Once you remove the hash marks, we’re ready to update our process and deploy our second and final VPC peering connection.

Updating the Fugue process

Now that we’ve uncommented lines 7-8 and 204-211, as well as filling in lines 7-8 with our information, we can execute fugue update on our process:

$ fugue update -y peering VPCPeering.lw

You should see output like this:

[ fugue update ] Updating process with Alias: peering with /Users/user/projects/VPCPeering.lw

Compiling Ludwig file /Users/user/projects/VPCPeering.lw ...
[ OK ] Successfully compiled. No errors.

Uploading compiled Ludwig composition to S3 ...
[ OK ] Successfully uploaded.

Requesting the Conductor to update process with composition ...
[ DONE ] Process with Alias: peering updated with composition.

[ HELP ] Run the 'fugue status' command to view details and status of this process update.

In a moment, if you go back to the AWS Console and click on “Peering Connections” again, you should see a new peering connection with the status “Pending Acceptance.”

The pending peering connection between example-vpc and non-fugue-managed-vpc.

The pending peering connection between example-vpc and non-fugue-managed-vpc.

Click on the “Action” drop-down menu, then select “Accept Request,” and if the information looks correct, click “Yes, Accept.”

The VPC peering connection from example-vpc to non-fugue-managed-vpc should now show up successfully under the name non-managed-peer-vpc-peering.

The successful peering connection between example-vpc and non-fugue-managed-vpc.

The successful peering connection between example-vpc and non-fugue-managed-vpc.

Ta-dah! We just used Fugue to create two VPCs and two different VPC peering connections, one between two Fugue-managed VPCs and one between a Fugue-managed VPC and a non-Fugue-managed VPC. Not bad for a few minutes’ work!

Killing The Fugue Process

Now that we’re done, we don’t want extra VPCs or peering connections hanging out in our account, so it’s time to kill the process, like so:

$ fugue kill -y peering

Within a few minutes, all the Fugue-managed infrastructure created as part of this composition will be gone. You’ll need to delete the non-fugue-managed-vpc manually, since it’s non-Fugue-managed.

Next Steps

If you’re eager to build more with Fugue, check out the next example. You can also try on your own to build service-specific examples, like a DynamoDB table, a Lambda function, a CloudFormation stack, or more. And as always, feel free to reach out to support@fugue.co with any questions.