Fugue Transcriber

Fugue Transcriber is a command-line tool that scans your AWS account for existing infrastructure and generates a Ludwig composition representing your resources.

Transcriber auto-generates compositions. As with all code-generation tools, it can be a challenge to manage lots of automatically generated compositions from a large environment. There are some features, like filtering, that make it easier to scale down the transcribed environment. For help with transcribing large or complex environments, contact support@fugue.co.

Transcriber is packaged with the Fugue Client Tools. Create a Fugue account, if you haven’t already, to download the Fugue Client Tools from the Download Portal. Find installation instructions at the Fugue Quick Setup.

Note: If you prefer to work in a GUI, Fugue Composer has a “Generate Composition” feature that offers the same major options as the Transcriber command-line tool. Find more information at Fugue Composer.

Usage

fugue-transcriber [OPTIONS] OUT

Options

--region TEXT
Region value to substitute for AWS::Region. Defaults to profile setting or us-east-1.
--profile TEXT
Specify an AWS profile for credentials and region. Uses environment defaults if not specified.
-k | --tag-key TEXT
Limit scan to AWS resources that have this tag key. Use one of --tag-key or --tag-pair.
-t | --tag-pair TEXT...
Two arguments to limit scan to AWS resources that have this tag key and value. Example: --tag-pair Name my-vpc. Use one of --tag-key or --tag-pair.
-i | --include-service TEXT
Include only this service when querying the cloud. May be used more than once.
-x | --exclude-service TEXT
Exclude this service when querying the cloud. May be used more than once.
-f | --filter-file FILENAME
EXPERIMENTAL: Load a filter file to specify the included services and resources when querying the cloud.
--include-fugue-resources
Include AWS resources that are managed by Fugue. (Excluded by default.)
--include-fugue-runtime
Include AWS resources that are part of the Fugue runtime. (Excluded by default.)
-q | --quiet
Quiet mode
--debug
Debug mode
-v | --version
Print the current version number and exit.
-l | --list-services
List services covered by Transcriber and exit.
-p | --server-port TEXT
Set the port number for the Fugue API Server. Use this option or FUGUE_API_PORT environment variable. Default is 8080.
-h | --help
Show help message and exit.

Definition

Transcriber generates Ludwig compositions from AWS account resources. Each resource declaration in the composition is preceded by a comment that indicates the type and ID of the resource that it represents.

By default, Transcriber ignores AWS resources that belong to the Fugue runtime. To include resources that are part of the Conductor, use --include-fugue-runtime.

By default, Transcriber uses the Fugue API Server, if available, to filter out Fugue-managed AWS resources. The server port number defaults to 8080, but a different port may be specified with the -p or --server-port option or with the $FUGUE_API_PORT environment variable. If the server is running and you wish to include resources managed by Fugue, use the option --include-fugue-resources. If no server is available, Transcriber includes Fugue-managed resources in the generated composition.

Transcriber scans the resources in the default AWS region, which is specified in your AWS CLI configuration file (usually located at ~/.aws/config on macOS or Linux or at C:\Users\USERNAME\.aws\config on Windows). To specify a different region, use the --region option. You can also use the aws configure command to update your default region.

Transcriber output is sent to OUT. For OUT, use a filename to save to that file or - to send results to standard output.

To view available services and for a list of valid services for the --include-service and --exclude-service options, use the -l or --list-services option.

Supported Services

Currently supported services include:

Name of service Usage for Transcriber
ASG AutoScaling Groups aws-autoscaling-autoscaling-groups
ASG AutoScaling Launch Configurations aws-autoscaling-launch-configurations
ASG AutoScaling Scaling Policies aws-autoscaling-scaling-policies
CloudFormation Stacks aws-cloudformation-stacks
CloudFront Web Distribution aws-cloudfront-distributions
Cloudtrail Trails aws-cloudtrail-trails
CloudWatch Alarms aws-cloudwatch-alarms
CloudWatch LogGroup aws-cloudwatch-log-groups
CloudWatch Metric Filters aws-cloudwatch-metric-filters
DynamoDB Tables aws-dynamodb-tables
EC2 Customer Gateways aws-ec2-customer-gateways
EC2 DHCP Options aws-ec2-dhcpoptions
EC2 Elastic IPs aws-ec2-elastic-ip-addressess
EC2 FlowLog aws-ec2-flow-logs
EC2 Instances* aws-ec2-instances
EC2 Internet Gateways aws-ec2-internet-gateways
EC2 Nat Gateway aws-ec2-nat-gateways
EC2 Network ACLs aws-ec2-network-acls
EC2 Network Interfaces aws-ec2-network-interfaces
EC2 Route Tables aws-ec2-route-tables
EC2 Security Groups aws-ec2-security-groups
EC2 Subnets aws-ec2-subnets
EC2 Volumes aws-ec2-volumes
EC2 VPC Endpoints aws-ec2-vpc-endpoints
EC2 VPC Peering* aws-ec2-vpc-peering-connections
EC2 VPCs aws-ec2-vpcs
EC2 VPN Connections aws-ec2-vpn-connections
EC2 VPN Gateways aws-ec2-vpn-gateways
ECS Cluster aws-ecs-clusters
ECS Service aws-ecs-services
ECS Task Definition aws-ecs-task-definitions
Elasticache Cache Cluster aws-elasticache-cache-clusters
Elasticache Cache Subnet Group aws-elasticache-cache-subnet-groups
Elasticache Replication Group aws-elasticache-replication-groups
ELB Load Balancers aws-elasticloadbalancing-loadbalancers
ELBv2 Load Balancers aws-elasticloadbalancing-loadbalancers
ELBv2 Target Groups aws-elasticloadbalancing-target-groups
IAM Group aws-iam-groups
IAM Instance Profiles aws-iam-instance-profiles
IAM Managed Policies aws-iam-managed-policies
IAM Roles aws-iam-roles
IAM User aws-iam-users
Lambda Aliases aws-lambda-aliases
Lambda Event Source Mappings aws-lambda-event-sources
Lambda Functions aws-lambda-functions
RDS Clusters aws-rds-clusters
RDS Cluster Parameter Group aws-rds-cluster-parameter-groups
RDS Instances* aws-rds-instances
RDS Subnet Groups aws-rds-subnet-groups
Route53 Resource Record Set aws-route53-resource-record-sets
S3 Buckets aws-s3-buckets
SNS Subscriptions aws-sns-subscriptions
SNS Topics aws-sns-topics
SQS Queues aws-sqs-queues

More services are forthcoming.

*Refer to notes below for known issues and exceptions.

Note: EC2 instances

EC2 Instances require additional steps to transcribe when custom instance stores are used. Read more about the details here.

Note: VPC peering connections

Transcriber does not support managed VPC peering connections. If a managed VPC peer is transcribed, it will be transcribed as an unmanaged VPC peer using an external reference, even if the peer VPC has been transcribed in the same composition. It will need to be manually updated with the correct reference.

Note: RDS databases

Password management

For configurations that include an RDS database some details around password management should be considered. Read more about those details here.

Database name validations

Validations for RDS database names follow the constraints in AWS documentation. However, in certain cases the documented constraints are overly strict, and a transcribed RDS database with a valid name in the dbName field may trigger a validation error:

Invalid database name for MariaDB. Must contain 1 to 64 letters or numbers. Cannot be a word reserved by the specified database engine

If you see an error message similar to the above after transcribing an RDS instance and compiling the composition, contact support@fugue.co.

Note: VPCs with a secondary CidrBlock

Transcriber does not currently support configurations that include a VPC with a secondary CidrBlock. Configurations containing these components will result in a Validation failed/Invalid subnet message. Support for this functionality will be implemented in a subsequent release, timing is still TBD.

Transcriber and AWS credentials

Transcriber scans the AWS account associated with the values of AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, if those environment variables are set. Otherwise, Transcriber scans the account associated with the default profile of the AWS CLI credentials file (generally located at ~/.aws/credentials), and if that profile is not present, it uses the default profile of the AWS CLI configuration file (generally located at ~/.aws/config).

To have Transcriber scan an account using a different profile, use the --profile option to specify the profile name as it appears in the AWS CLI credentials file or configuration file.

Alternatively, export the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables associated with the desired account. Or, make another profile the default account by setting the AWS_DEFAULT_PROFILE environment variable:

$ export AWS_DEFAULT_PROFILE=user2

The order of precedence for credential sources is:

  1. --profile option
  2. Environment variables
  3. default profile in AWS CLI credentials file (~/.aws/credentials)
  4. default profile in AWS CLI configuration file (~/.aws/config)

IAM Policies for Transcriber

The AWS credentials used to run Transcriber simply need read-only permissions for each AWS service to be scanned. The two IAM policies created when installing the Fugue Conductor are not used by Transcriber.

If you prefer, you may create a new IAM user to use with Transcriber. To do so, visit the IAM Management Console and select “Add user.” Enter a name, check the box to enable programmatic access, and select the read-only policies for the services you want Transcriber to scan. (For example, the AWS-managed policy AmazonEC2ReadOnlyAccess enables read-only access to EC2, and ReadOnlyAccess enables read-only access to all AWS services.)

Once you’ve created the user, download the auto-generated credentials and set them as described above. When you execute a Transcriber command, Transcriber will use the permissions associated with those credentials to scan your account.

Examples

Check out a full page of examples here.

FAQ

What is Transcriber?

Fugue Transcriber is a command-line tool, packaged with the Fugue Client Tools that scans your AWS account for existing infrastructure and generates a Ludwig composition representing your resources.

How do I install Transcriber?

After you select the package for your platform the Fugue Transcriber is installed as part of the Fugue Client Tools. Complete installation details are available in the Fugue Quick Setup.

How do I uninstall Transcriber?

Fugue Transcriber can be uninstalled along with the Fugue Client Tools. Details about removing Fugue are available here.

How do I upgrade Transcriber?

Fugue Transcriber will be upgraded as part of the Fugue Client Tools, and any feature announcements, upgrades, or new releases of Fugue are available through our Download Portal.

What platforms are Transcriber supported on?

Transcriber is currently supported on the same platforms as the Fugue Client Tools and includes:

  • macOS El Capitan (10.11.*), macOS Sierra (10.12.*), macOS High Sierra (10.13.*)
  • Ubuntu (14.04 LTS, 16.04 LTS)
  • Amazon Linux (2016.03.3)
  • RHEL 6 & 7.2 (Yum/RPM)
  • Microsoft Windows (Windows 7, 10) Note: For Windows users we recommend using PowerShell 5 and $env:var syntax. To determine your version of PowerShell you can use echo $PSVersionTable.PSVersion. If you have additional questions reach out to support@fugue.co.

Do I need to have a Conductor installed before I can use Transcriber?

No, Fugue Transcriber does not require a Conductor to operate. You will only need the Fugue CLI to issue commands for Transcriber.

What services can I transcribe?

To see the full list of supported services simply issue the fugue-transcriber --list-services command. You can also see the full list at Supported Services.

What determines which services I have permission to transcribe?

Aside from simply using the --include or --exclude options, the scope of the services Transcriber has permission to scan or transcribe is determined by the permissions granted by the AWS credentials used. For more information about IAM policies for Transcriber, see Transcriber and AWS credentials. Further details about AWS permissions are available here.

What if I have comments or questions?

You can reach out to us at support@fugue.co.