init

Usage

fugue [global options] init [options] <ami_id>

Arguments

<ami_id>

Required: The AMI ID of the Fugue Conductor to install (you can find the ID for each release at the Download Portal).

Example: fugue init ami-e1d64ff6

Options

Global options are detailed here.

-p | --profile PROFILE

Specify the name of the AWS credential profile in ~/.aws/credentials to use when searching for credentials. Defaults to default.

Example: fugue init -p myprofile ami-e1d64ff6

--creds-check
Check that the configured AWS credentials have sufficient permissions to use the Fugue CLI and install the Fugue Conductor.
-h | --help
Show help text. The help flag is available throughout the CLI in both an application-level and command-level context. It enables a user to view help text for any command within the Fugue CLI.

Definition

The init command initializes a project in the current directory.

The initialization process consists of setting AWS credentials and Conductor version, and then recording the Conductor version in a fugue.yaml file. The name of the credential profile is also recorded in fugue.yaml if it is not the default profile.

fugue.yaml also holds the fields userId and userSecret, which are generated as part of fugue install. The file may optionally contain other settings that guide the fugue CLI through its various operations.

Note

Anytime the Fugue CLI executes a command, it searches for the fugue.yaml file to verify Fugue user credentials and load configuration settings. The CLI first searches for fugue.yaml in a project-level directory, then in a user-level directory, and finally in a system-level directory.

In Mac and Linux systems, these folders are:

  1. The current working directory
  2. ~/.fugue/fugue.yaml
  3. /etc/fugue/fugue.yaml

In Windows, these folders are:

  1. The current working directory
  2. %AppData%\Local\Fugue\fugue.yaml
  3. %ProgramData%\Fugue\fugue.yaml

Fugue will use the first file it finds in the search path described above.

init must be executed prior to a fugue install. Once the Fugue Conductor has been installed, the changes init makes to the fugue.yaml settings (i.e., AWS credentials and Conductor version) won’t take effect until the Fugue Conductor is uninstalled and reinstalled. Note: You may use upgrade to change the Conductor AMI without having to uninstall and reinstall Fugue (for example, fugue upgrade ami-e1d64ff6).

Before you run fugue init, you should know:

  • Which AWS credentials do I want to use for this project?
  • Which Conductor AMI do I want to run, if one isn’t already running in my account?

Determining Credentials To Use For The Project

When init is executed, the CLI searches for credentials in the following order, using the first set it finds:

  1. The environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY; then,
  2. The default profile in ~/.aws/credentials, unless the init -p <profile> option is specified; then,
  3. The default profile in ~/.aws/config, unless the init -p <profile> option is specified; then finally,
  4. The instance metadata service (IMDS), which is present only on EC2 instances and provides credentials reflecting the IAM Role of the instance.

The easiest of these to use for a developer desktop is usually the credential profile option. The best way to configure credential profiles is to use the AWS CLI. You can read more about quickly setting up AWS credentials with both Fugue and the AWS CLI here.

Note: The Fugue CLI does not support the use of AWS root account credentials. If you attempt to configure Fugue to use your AWS root account, init will fail. AWS recommends creating an IAM user for everyday use and using its credentials instead, because root credentials cannot be restricted. If administrative-level permissions are desired, you can grant the IAM user full access, allowing you to modify or revoke permissions later if necessary. Use the IAM user credentials with Fugue instead. For more information, see AWS’s documentation on root account vs. IAM user credentials.

While you can attach the AdministratorAccess policy to the IAM user associated with your credentials, we’ve made available two JSON policies with the minimum required permissions to run the CLI as an installer and as a user. For more information, see AWS Permissions and the Fugue CLI.

Determining Which Conductor AMI To Use For The Project

The AMI ID is required for the init command. Fugue distributes versions of the Conductor with specific AMI IDs, so that you get a pre-built, working machine image instead of software you have to install on your own OS. Fugue Conductors are built on Debian Linux. You can find the ID for each release at the Download Portal. Generally, you will want to use an AMI that contains a version of the Conductor that is compatible with the Fugue CLI you plan to use.

How init Works

When the user executes fugue init, the CLI validates the provided Conductor AMI ID, then creates the fugue.yaml file.

If a fugue.yaml file already exists, the CLI renames it fugue.yaml.old and creates a new fugue.yaml file with the new settings.

If a fugue.yaml.old file already exists, the CLI deletes it, renames the existing fugue.yaml to fugue.yaml.old, and creates a new fugue.yaml file with the new settings.

For example, when no fugue.yaml file exists, and AWS credentials are stored in a default profile in ~/.aws/credentials, executing fugue init ami-f9baedee produces the following output:

[ fugue init ] Initializing Fugue project with the following configuration:

Fugue Conductor AMI ID: ami-f9baedee
AWS Credentials: Profile (default)

Validating Fugue Conductor AMI ID ...
[ OK ] Provided AMI ID is valid.

Creating new fugue.yaml file ...

[ Done ] Project initialized.

Likewise, when no fugue.yaml file exists, and AWS credentials are stored in environment variables, executing fugue init ami-f9baedee produces the following output:

[ fugue init ] Initializing Fugue project with the following configuration:

Fugue Conductor AMI ID: ami-f9baedee
AWS Credentials: Environment variables

Validating Fugue Conductor AMI ID ...
[ OK ] Provided AMI ID is valid.

Creating new fugue.yaml file ...

[ Done ] Project initialized.

Note: The default CLI timeout is 15 minutes. This timeout setting only applies when fugue init is being run for the first time or there are no existing timeout settings in the fugue.yaml file.

Contents of fugue.yaml

Running fugue init ami-482ae35e produces the following fugue.yaml file:

conductor:
  ami: ami-482ae35e

Running fugue init --profile myProfile ami-482ae35e produces the following fugue.yaml file:

aws:
  credentialProfile: myProfile
conductor:
  ami: ami-482ae35e

After running fugue install, the fugue.yaml file might look like this:

aws:
  credentialProfile: myProfile
conductor:
  ami: ami-482ae35e
user:
  userId: root
  userSecret: gIowKW6AnAM4fgBqEXAMPLEEXAMPLEEXAMPLEEXAMPLE

fugue.yaml Line-By-Line

All of the fields listed below are optional, except for ami, userId, and userSecret. Manually editing the fugue.yaml file should not be necessary during normal operation, but you may wish to review these settings when troubleshooting or learning about Fugue.

If you ran fugue init -p <profile>, then the aws block contains one field, credentialProfile, specifying that AWS credential profile. However, if you’re using the default profile, the aws block is optional and does not appear.

aws Lines
aws:
  credentialProfile: myProfile

The conductor block governs how the CLI installs Fugue in an AWS account and interacts with it.

conductor Lines
conductor:
  region: us-east-1
  ami: ami-f00ba5
  instanceType: m4.large
  keyName: null
  requestQueue: fugue-demarc-requests
  responseQueue: fugue-cli-resp-ycbctybtqxqwofwrvhom
  compositionBucket: fugue-xxxxxxxxxxxx-us-east-1
  installAZ:
    - us-east-1c
    - us-east-1d
region
Specifies the AWS region in which the fugue binary operates for this project. All calls to the AWS API that the CLI makes are directed to this region. This influences all commands, including install.
ami (required)
Specifies the Amazon Machine Image that the EC2 Fugue Conductor will launch with. Different AMIs may have different versions of the Fugue Conductor installed. This is set with the AMI ID argument. The fugue binary version and Conductor version are generally a matched set, and compatibility between different versions is not guaranteed, so this setting should not ever be changed for production use.
instanceType
Specifies the instance type that the EC2 Fugue Conductor launches with. Currently, the only supported size is m4.large, so do not change this setting.
keyName
Allows you to specify an SSH keypair to associate with the Fugue Conductor EC2 instance at launch. Fugue launches the Conductor with no support for inbound TCP connections of any kind by default, so to log in to the instance via SSH, you can specify a keypair with this setting, and the CLI will add a security group ingress rule allowing access to the Conductor on port 22 from the IP address where fugue install is run.
installAZ
A list of Availability Zones (AZs) to use for the Fugue Conductor installation in your account. The Fugue Conductor installation uses an Autoscaling Group that straddles two AZs to ensure availability. Normally, the fugue install command will figure out two suitable AZs for the installation on its own. However, in some narrow cases this detection mechanism can get false positives from the AWS API (causing the Fugue CloudFormation stack to fail), or you may need to specifically control the AZs that Fugue runs in. In such cases, provide a list in this field with one or two AZs for Fugue to use (you may put more in the list, but only the first two will be used). Note that if you supply only one AZ, you will see a warning in the CLI log file from the fugue install command.

The requestTimeoutSecs field governs how long the CLI waits for a response from the Conductor for requests.

requestTimeoutSecs Line
requestTimeoutSecs: 5

The retry block governs how the CLI handles commands that don’t work perfectly the first time and exceed the requestTimeoutSecs value without a response. Most of these should not be modified during normal operation. However, for troubleshooting or experimentation, there are several settings you can change:

retry Lines
retry:
  initialDelayMs: 200
  backoff: constant
  maxDelayMs: null
  cutoffDelayMs: null
  cutoffRetries: 5
initialDelayMs
Sets the initial wait time between requests in milliseconds.
backoff
Determines whether the wait time increases between requests, and if so, by how much. Valid values are constant, which keeps the delay at the initial setting; fibonacci, which increases the delay using a Fibonacci formula seeded with the initial delay; and exponential, which increases the delay exponentially with each failed request.
maxDelayMs
Sets a cap on delay between retries in milliseconds. The delay will always be the lesser of this value or that determined by the fibonacci or exponential backoff algorithms. The request will not be terminated when this cap is reached, the delay will just not increase beyond it.
cutoffDelayMs
Sets a delay at which the CLI gives up on the request. When this delay value is reached, the request attempt terminates.
cutoffRetries
Sets a maximum number of attempts to make a request, after which it gives up on the request.

The lwcOptions field is for internal or support use only. It enables the user to override fugue run and fugue update with lwc (Ludwig compiler) options, which are passed directly to lwc. An example would look like this:

lwcOptions Line
lwcOptions: --timeout=30 --no-color

The above example would set the target file’s compilation timeout at 30 seconds and disable colorized output. For more information about lwc options, see Ludwig Compiler Options.

The user block contains information about the current Fugue user. This block is added to fugue.yaml during the install process.

user Lines
user:
  userId: root
  userSecret: gIowKW6AnEXAMPLEEXAPMLEEXAPMLEEXAMPLEEXAMPLE
userId (required)
Field is generated during fugue install. Specifies the current Fugue user (root by default). You can use the user command to change the user, or you can change it here manually. Just remember to also change the corresponding userSecret.
userSecret (required)
Field is generated during fugue install. A symmetric access token that the CLI uses to sign requests to the Conductor, authenticating the user. Each user must have a corresponding userSecret to use the CLI. To generate a secret, use the policy generate-secret <user_id> command (see policy). Note: As with any secret, the userSecret should be guarded carefully. Do not check your fugue.yaml file into source control.

Warning

Do not lose your root credentials. They cannot be replaced with the policy generate-secret command. If you have lost your root credentials, see Troubleshooting or contact support@fugue.co.

Most of the remaining settings should not be modified during normal operation. However, for troubleshooting or experimentation, there are several settings you can change that govern how the CLI finds and interacts with the Fugue Conductor:

requestQueue
Is the queue to which the CLI sends requests. Note: This setting should not ever be changed for production use.
responseQueue
Is the queue from which the CLI reads responses. Note: This setting should not ever be changed for production use.
compositionBucket
Is the name of the bucket used to store loaded and running Fugue compositions. It is calculated as a composite of the string fugue-, your AWS account number, and the value of the region field. This allows the bucket name to be globally unique, which is important since all S3 buckets across AWS share one namespace.

Examples

Changing the AWS credential profile the Fugue CLI uses

If you have configured the Fugue CLI to use a particular set of AWS credentials but you need to specify a different set, you can run fugue init to change which AWS profile Fugue uses in the ~/.aws/credentials file. (Recall that root account credentials are not supported.)

For example, if you’ve configured the Fugue CLI to use your personal AWS account, but you need to reconfigure it to use your corporate AWS account, you may first run aws configure --profile CORPORATE_PROFILE, and enter the AWS access key ID and secret access key associated with the corporate account. Then, run fugue init AMI -p CORPORATE_PROFILE to switch Fugue to that profile in the local fugue.yaml.

As usual, the Fugue CLI checks to make sure the target account has the correct permissions to run the Fugue CLI and install the Fugue Conductor.

Note: If you have already executed fugue install but need to change your AWS account credentials, you must fugue uninstall, fugue init with the new account credentials, and fugue install again for the credential settings to take effect.

Changing The Region The Conductor Is Running In

While we anticipate changing this in the future, at present Fugue only supports running Conductors in the us-east-1 region.

Changing The Size Of The Conductor Instance

The Conductor instance type is m4.large by default. Currently, we do not support changing the size of the Conductor instance. If you believe you need a different size, email support@fugue.co.

Checking AWS credentials with init

You can use the --creds-check option to validate your AWS credentials. The CLI checks to see if the IAM policy associated with the configured AWS credentials has the required permissions to run the Fugue CLI and install the Fugue Conductor.

When you execute fugue init --creds-check <ami_id>, the CLI does the following:

  1. The CLI searches for AWS credentials according to the order listed here.
  2. The CLI checks the credentials to ensure the user is authorized to use the Fugue CLI and install the Fugue Conductor.
  3. If the credentials do not have sufficient permissions, the CLI displays an error.
  4. If the credentials do have sufficient permissions, the CLI validates the provided Conductor AMI ID and creates the fugue.yaml file.

Example error message for credentials missing one or more required IAM permissions:

[ fugue init ] Initializing Fugue project with the following configuration:

Fugue Conductor AMI ID: ami-f9baedee
AWS Credentials: Profile (no-access-creds-test)

Checking your AWS Credentials for Fugue CLI use ...
[ ERROR ] There was a problem executing this command.
   Reason: An error occurred (AccessDenied) when calling the SimulatePrincipalPolicy operation: User: arn:aws:iam::xxxxxxxxxxxx:user/no-access-creds-test is not authorized to perform: iam:SimulatePrincipalPolicy on resource: arn:aws:iam::xxxxxxxxxxxx:user/no-access-creds-test

Example error message for erroneous credentials:

[ fugue init ] Initializing Fugue project with the following configuration:

Fugue Conductor AMI ID: ami-f9baedee
AWS Credentials: Profile (mistyped-creds)

Checking your AWS Credentials for Fugue CLI use ...
[ ERROR ] There was a problem executing this command.
   Reason: An error occurred (InvalidClientTokenId) when calling the GetCallerIdentity operation: The security token included in the request is invalid.

Manually Setting Installation Availability Zones

If you have problems installing Fugue due to an Availability Zone (AZ) being unavailable for new resources in your account, or wish to customize the AZs used by Fugue for installation, you can manually specify the AZs that Fugue uses for the Fugue Conductor installation in your fugue.yaml. These AZs will be used when you run the fugue install command. To do this, add an entry in fugue.yaml under the conductor heading called installAZ as a list of two known-good availability zones in your account. For example:

conductor:
  installAZ:
    - us-east-1c
    - us-east-1d

Next time you run fugue install, the command will install the Fugue Conductor VPC with subnets in the first two AZs listed.

There are two caveats you should keep in mind if you have to use this feature. Firstly, bear in mind that you can specify one or two AZs. If you specify one, the CLI will log a warning during installation, but Fugue will install and run. If you specify two, installation should proceed as normal. If you specify more than two, only the first two will be used, and all subsequent entries in the list will be ignored.

Secondly, the AZs you list will be assumed valid. If you provide an invalid or unavailable AZ for your account, installation will fail with a CloudFormation stack error, like:

[ ERROR ] AWS CloudFormation stack creation failed

Note that you can get a list of AZs available to you using the AWS CLI command describe-availability-zones, but this is the same command that Fugue uses, and it sometimes returns unreliable data. As a rule of thumb, you should look for a pair of AZs that are listed as available to you, but that you don’t heavily utilize.