Migrating Infrastructure with Fugue

How does migration work?

Fugue’s governance, visibility, and control capabilities can be applied to existing cloud resources that were not created by Fugue by migrating those resources to enable Fugue management. Migrating resources to Fugue is a two-step process.

  • First, Composer or Transcriber is used to generate a composition that reflects the existing resources in an account.
  • Next, the import feature instructs Fugue to begin managing those resources.

Both Composer (via Generate Composition) and Transcriber can scan your AWS account for existing infrastructure and generate a Ludwig composition representing your resources. After using Composer or Transcriber to analyze your AWS accounts and generate the appropriate Ludwig compositions, the import function then matches resources between your Ludwig composition and your existing account. To execute the import function, you can use Composer’s Create Process feature (with the “Import Existing Resources” box checked) or the Fugue CLI’s fugue run --import command. Once the process has been successfully created, Fugue treats it like any other process – you can configure enforcement, you can apply validations, you can write policy-as-code around it, you can update or kill the process, and so on. Visit this link for a detailed example.

Note: Ludwig compositions used for migration include resourceId fields which can be created by hand, or generated using Composer or Transcriber – both included in the Fugue Client Tools. In general, we recommend using Composer or Transcriber for this process, but if you have specific questions reach out to support@fugue.co.

Using Fugue to Migrate Infrastructure.

Using Fugue to Migrate Infrastructure.

Using the import feature

Import (e.g. fugue run <sample.lw> --import) is the second step in the Fugue migration process. It allows you to import and manage existing resources as if they were provisioned by Fugue.

How does import work?

Fugue import matches resource definitions in a user-created or auto-generated Ludwig composition with your existing resources in AWS. An import is activated by passing the --import flag to fugue run, enabling Fugue’s Conductor to match resourceIDs specified within your composition to resources that exist in your account. You can also use Composer’s Create Process feature and check the box for “Import Existing Resources.”

For example:

import-this-vpc: EC2.Vpc.new {
  cidrBlock: “10.0.0.0/16”,
  region: AWS.Us-east-2,
  resourceId: “vpc-1234abcd”
}

Supported services for import

Currently supported services include:

Name of service Supported for import
ASG AutoScaling Groups aws-autoscaling-autoscaling-groups
ASG AutoScaling Launch Configurations aws-autoscaling-launch-configurations
ASG AutoScaling Scaling Policies aws-autoscaling-scaling-policies
CloudFormation Stacks aws-cloudformation-stacks
CloudFront Web Distribution aws-cloudfront-distributions
Cloudtrail Trails aws-cloudtrail-trails
CloudWatch Alarms aws-cloudwatch-alarms
CloudWatch LogGroup aws-cloudwatch-log-groups
CloudWatch Metric Filters aws-cloudwatch-metric-filters
DynamoDB Tables aws-dynamodb-tables
EC2 Customer Gateways aws-ec2-customer-gateways
EC2 DHCP Options aws-ec2-dhcpoptions
EC2 Elastic IPs aws-ec2-elastic-ip-addressess
EC2 FlowLog aws-ec2-flow-logs
EC2 Instances* aws-ec2-instances
EC2 Internet Gateways aws-ec2-internet-gateways
EC2 Nat Gateway aws-ec2-nat-gateways
EC2 Network ACLs aws-ec2-network-acls
EC2 Network Interfaces aws-ec2-network-interfaces
EC2 Route Tables aws-ec2-route-tables
EC2 Security Groups aws-ec2-security-groups
EC2 Subnets aws-ec2-subnets
EC2 Volumes aws-ec2-volumes
EC2 VPC Endpoints aws-ec2-vpc-endpoints
EC2 VPC Peering aws-ec2-vpc-peering-connections
EC2 VPCs aws-ec2-vpcs
EC2 VPN Gateways aws-ec2-vpn-gateways
ECS Cluster aws-ecs-clusters
ECS Service aws-ecs-services
ECS Task Definition aws-ecs-task-definitions
Elasticache Cache Cluster aws-elasticache-cache-clusters
Elasticache Cache Subnet Group aws-elasticache-cache-subnet-groups
Elasticache Replication Group aws-elasticache-replication-groups
ELB Load Balancers aws-elasticloadbalancing-loadbalancers
ELBv2 Load Balancers aws-elasticloadbalancing-loadbalancers
ELBv2 Target Groups aws-elasticloadbalancing-target-groups
IAM Group aws-iam-groups
IAM Instance Profiles aws-iam-instance-profiles
IAM Managed Policies aws-iam-managed-policies
IAM Roles aws-iam-roles
IAM User aws-iam-users
Lambda Aliases aws-lambda-aliases
Lambda Event Source Mappings aws-lambda-event-sources
Lambda Functions aws-lambda-functions
RDS Clusters aws-rds-clusters
RDS Cluster Parameter Group aws-rds-cluster-parameter-groups
RDS Instances** aws-rds-instances
RDS Subnet Groups aws-rds-subnet-groups
Route53 Resource Record Set aws-route53-resource-record-sets
S3 Buckets aws-s3-buckets
SNS Subscriptions aws-sns-subscriptions
SNS Topics aws-sns-topics
SQS Queues aws-sqs-queues

More services are forthcoming. Contact support@fugue.co with any questions.

*EC2 Instances require additional steps to transcribe when custom instance stores are used. Read more about the details here. **For configurations that include an RDS database some details around password management should be considered. Read more about those details here.

Using Composer and the import feature together

The combination of Composer and the import feature simplify the process of bringing your infrastructure into Fugue. Use Composer’s “Generate Composition” feature to scan the desired account, and it will output a composition that includes the resource IDs of each resource it scans. The composition can then be run with Composer’s “Create Process” feature (using the “Import Existing Resources” option) or with the Fugue CLI’s fugue run --import command in order to import your infrastructure.

Composer and Transcriber may be used interchangeably in the migration process. Transcriber is a command-line tool that scans an AWS account and creates a composition representing the resources it finds. Composer is a composition visualization tool and editor that offers a Generate Composition feature powered by Transcriber and a Create Process feature powered by the Fugue CLI. Visit this link for a detailed example.

Best practices

In order to optimize your experience with Composer or Transcriber, and the import feature, consider the following best practices.

  • While you can write the migration composition yourself, we recommend using Composer to generate the migration composition. It’s simpler, faster, and more accurate. (Note: Transcriber can be used directly from the command line as well.)
  • Instead of migrating all resources within an account into a single Fugue process, we recommend creating multiple Fugue processes during migration. Processes can be used to separate concerns based on your organization’s needs. For example, processes could group resources by type (e.g., network, compute), team, or use case (e.g., dev/test/prod). Transcriber works best when scanning 40 resources or less.
  • Tag your resources when possible. Transcriber and Composer can include or exclude resources by tags when scanning an account, which means less time spent processing the output prior to importing it. For example, the -k or --tag-key option filters on a given tag key, and the -p or --tag-pair option filters on a given tag key and value.
  • Transcribed compositions should be inspected and processed before migration. Because Composer, the import feature, and Fugue make a very powerful team, it’s best to make sure the transcription doesn’t include anything you don’t want Fugue to manage (such as your IAM login credentials or default VPC).
  • Always do a dry-run before migrating. Simply execute fugue run <composition.lw> --dry-run --import. This is a good way to catch certain errors that could impact your migration. Dry-runs are also a best practice for executing other process management commands, such as update, resume, kill, or non-migration run.
  • Use Composer to visualize your infrastructure. After you use the Generate Composition feature to create a composition of the resources in your account, you can explore a diagram of the resources and see how they interrelate before you bring them under Fugue’s control.
  • Tip: You can import existing infrastructure, modify mutable attributes of the existing infrastructure, and create new infrastructure in the same composition. One fugue run --import handles it all, and so does Composer’s Create Process feature.

Caveats and things to avoid

Accidentally killing migrated resources

Killing a process destroys the process’s infrastructure, including migrated resources. If you use Transcriber or Composer to prepare a composition for import, make sure you remove anything from the transcription that you don’t want Fugue to manage, such as your default VPC with its associated resources, and IAM resources such as login credentials.

Workaround: If you experience a failed run on a migrated process for any reason, don’t kill the process. The safest course of action is to release the process, edit the composition as needed, and then run it again with --import.

Importing AWS-created default resources

When you use Transcriber or Composer to prepare a migration composition, check for default resources and delete or edit them out of the composition. If you import AWS-created default resources (as opposed to resources you create explicitly), you’ll be unable to kill the process. For example, when you create a VPC, AWS generates a default security group and default route table. Transcriber and Composer will include these defaults in its generated composition, and you can successfully import them, but because AWS doesn’t allow you to delete the default security group or route table, the process will be stuck in a Killing state.

Workaround: Suspend the process, release it, and transcribe the infrastructure again, this time excluding the default resource. When you run the composition with --import again, you’ll be able to kill the resulting process.

Importing a resource before it has been fully created

Ensure that resources have been created before you attempt to import them. Due to eventual consistency, an import may fail if the resources to be imported are not yet fully created.

Workaround: If you have just created a resource, wait a little longer before you try to import it. Ensure that you can view or access the resource in the AWS Management Console or the AWS CLI. Resources that have an explicit state should be in the “created” state, not the “creating” state.

Importing a resource but not its dependencies

When you prepare to import a resource, be sure all of its non-default dependencies are included in the transcribed composition. Say you’ve created a VPC with two subnets via the AWS Management Console – we’ll call them VPC-A, subnet-1, and subnet-2. Let’s also say you’ve transcribed the account, and in the process of editing the generated composition, you’ve accidentally deleted the subnet-2 binding. If you then use the --import flag to run the composition, migrating VPC-A and subnet-1 but not subnet-2, you won’t be able to kill the process. On kill, Fugue will attempt to delete VPC-A but will fail because deleting a resource also deletes its dependencies, and since subnet-2 is a dependency of VPC-A, and Fugue does not manage subnet-2, the process will be stuck in a Killing state.

Workaround: Suspend the process, release it, and transcribe the infrastructure again, this time keeping subnet-2. When you run the composition with --import again, you’ll be able to kill the resulting process.

Forgetting the --import flag

Don’t forget to use the --import flag when you run a composition whose resources you intend to migrate. If you do a normal fugue run on it, Fugue ignores the resourceId fields and creates all new infrastructure.

Workarounds:

  • If this occurs and your composition does not contain idempotent resources (S3, IAM, etc.), the process will successfully run and create a complete copy of your infrastructure. You’ll be able to kill the process.
  • If this occurs and your composition references existing idempotent resources, the process run will fail but you’ll be able to kill it.

Executing an import run too soon after a dry-run

Executing an import run immediately after executing an import dry-run, or executing two import dry-runs in a row, may fail with an error stating that resources are already managed by another process.

Workaround: Wait 30 seconds and try again.

Using EC2.InstanceBlockDeviceMapping

EC2 instances can’t be imported if the Ludwig declaration uses EC2.InstanceBlockDeviceMapping, which is deprecated. Instead, rewrite the instance binding using the new style of specifying EBS volumes on EC2 instances. For more information, see A New Way of Specifying EBS Volumes on EC2 Instances.

Tip: Transcriber and Composer use the new style when they transcribe EC2 instances in your account, so if you intend to import a transcribed composition, you’re all set.

Trying to import resources that don’t support externals

Certain resources do not support external references. When you transcribe an account prior to migration, the generated composition may contain external resources, especially if you exclude some AWS services from transcription. Running such a composition will produce a validation error, whether it’s an --import run or a non-import run. For more information, see Referring to External Resources.

Omitting the path field in an IAM role

In one particular case, updating a migrated process can fail if an IAM role in the original composition did not specify a path field. This case only arises if you are following this particular workflow:

  1. Write a composition, A1, containing an IAM role with no path
  2. Run A1
  3. Release the resulting process
  4. Transcribe the released resources in a composition, T
  5. Run T using the --import flag to migrate the resources to Fugue
  6. Update the migrated process with a composition, A2, that is based on the original composition, A1, and contains the same IAM role

In this situation, the update will fail because the original composition does not specify a role path, whereas AWS assigns a path of / when the role is created, triggering an EntityAlreadyExists error upon an update.

Note: This caveat does not apply to simply transcribing and then importing infrastructure. It also does not apply to transcribing, importing, and updating infrastructure when the update is based on composition T. The issue only arises when an IAM role is declared with no path in a composition A1 that is then run, released, transcribed, imported, and updated with a composition based on A1.

Workaround: If you intend to follow the workflow above, declare an IAM role path of / in the initial composition rather than leaving the field blank. Here’s an example IAM.Role.new declaration:

role: IAM.Role.new {
  roleName: 'myRoleToBeImported',
  assumeRolePolicyDocument: IAM.Policy.AssumeRole.ec2,
  path: "/"
}

Other problems not described above

If you are experiencing an issue that isn’t described here, reach out to support@fugue.co.

Use cases

  • Use either Composer or Transcriber with the import feature to update mutable properties on migration. If a bad actor gives public access to non-Fugue-managed S3 buckets, you can transcribe the resources and change the bucket policies by adding the policy to the transcribed composition. When you run the composition with the --import flag, or use Composer’s Create Process feature with the “Import Existing Resources” option, the bucket policies will be corrected and the buckets will be under Fugue management.
  • Use either Composer or Transcriber with the import feature to separate applications into different processes. If a company has 20 apps in an AWS account and wants to separate them so each app has its own process, and all infrastructure attached to an app is tagged with the app name, you can filter on that tag when running Transcriber and then import the infrastructure for that app. You’ll need to repeat the process for each separate app tag. When you’re done, all apps are contained in separate processes and all app resources are managed by Fugue.
  • Use either Composer or Transcriber with the import feature to bring existing infrastructure into compliance with a policy. If you have existing, noncompliant resources in your account, you can transcribe the resources, apply a validation module to the composition, and then edit the Ludwig in accordance with the policy. When you import the infrastructure, the resources will be compliant and managed by Fugue. Note: You can view an example of this use case here.

FAQ

What information does my composition need to contain to successfully import?

Before you run import you must have a valid composition with the required resource definitions and resourceIDs.

Note: If a resourceID isn’t present for every resource you are attempting to import, Fugue will create a new version of that resource, which is, in general, not the desired behavior. To avoid this scenario we recommend using Composer or Transcriber to generate the Ludwig for your composition.

What services can I import?

See the list of supported services on this page - here. For details on supported services using Transcriber - refer here.

What is a resourceId?

The resourceId is an identifier used to uniquely identify a resource in an AWS account. Where possible, the resourceId is the ARN, but many resources do not have ARNs, in which case the resourceId is some other unique identifier associated with the resource.

Note: Due to the difficulty of determining the resourceId for a given resource, we recommend using Composer or Transcriber to generate Ludwig compositions to import, rather than writing them by hand.

What determines which services I have permission to import?

Composer and Transcriber simply require read-only access. For import the AWS profile/role associated with your Fugue account will need permission to view and modify any resources you want to import. Click here to read more about permissions.

What is the relationship between import and Composer/Transcriber?

Composer or Transcriber analyzes AWS accounts and generates Ludwig compositions that can be used to import the resources in those accounts. Import references the Ludwig compositions and brings the infrastructure into Fugue as processes for management.

Do you have an example of how the migration process works?

Yes we do, the example is available here.

What if I have comments or questions?

You can reach out to us at support@fugue.co.