fugue [global options] init [options] <ami_id>
Global options are detailed here.
Specify the name of the AWS credential profile in
~/.aws/credentialsto use when searching for credentials. Defaults to
fugue init -p myprofile ami-e1d64ff6
- Check that the configured AWS credentials have sufficient permissions to use the Fugue CLI and install the Fugue Conductor.
- Show help text. The help flag is available throughout the CLI in both an application-level and command-level context. It enables a user to view help text for any command within the Fugue CLI.
init command initializes a project in the current directory.
The initialization process consists of setting AWS credentials and
Conductor version, and then recording the Conductor version in a
fugue.yaml file. The name of the credential profile is also recorded
fugue.yaml if it is not the
fugue.yaml also holds the fields userId and
userSecret, which are generated as part
fugue install. The file may
optionally contain other settings that
fugue CLI through its various operations.
Anytime the Fugue CLI executes a command, it searches for the
fugue.yaml file to verify Fugue user credentials and load configuration settings. The CLI first searches for
fugue.yaml in a project-level directory, then in a user-level directory, and finally in a system-level directory.
In Mac and Linux systems, these folders are:
- The current working directory
In Windows, these folders are:
- The current working directory
Fugue will use the first file it finds in the search path described above.
init must be executed prior to a
fugue install. Once the Fugue
Conductor has been installed, the changes
init makes to the
fugue.yaml settings (i.e., AWS credentials and Conductor version)
won’t take effect until the Fugue Conductor is uninstalled and
reinstalled. Note: You may use upgrade to change the
Conductor AMI without having to uninstall and reinstall Fugue (for
fugue upgrade ami-e1d64ff6).
Before you run
fugue init, you should know:
- Which AWS credentials do I want to use for this project?
- Which Conductor AMI do I want to run, if one isn’t already running in my account?
Determining Credentials To Use For The Project¶
init is executed, the CLI searches for credentials in the
following order, using the first set it finds:
- The environment variables
~/.aws/credentials, unless the
init -p <profile>option is specified; then,
~/.aws/config, unless the
init -p <profile>option is specified; then finally,
- The instance metadata service (IMDS), which is present only on EC2 instances and provides credentials reflecting the IAM Role of the instance.
The easiest of these to use for a developer desktop is usually the credential profile option. The best way to configure credential profiles is to use the AWS CLI. You can read more about quickly setting up AWS credentials with both Fugue and the AWS CLI here.
Note: The Fugue CLI does not support the use of AWS root account
credentials. If you attempt to configure Fugue to use your AWS root
init will fail. AWS recommends creating an IAM user for
everyday use and using its credentials instead, because root credentials
cannot be restricted. If administrative-level permissions are desired,
you can grant the IAM user full access, allowing you to modify or revoke
permissions later if necessary. Use the IAM user credentials with
Fugue instead. For more information, see AWS’s documentation on root
account vs. IAM user
While you can attach the
AdministratorAccess policy to the IAM user
associated with your credentials, we’ve made available two JSON policies
with the minimum required permissions to run the CLI as an installer and
as a user. For more information, see AWS Permissions and the Fugue CLI.
Determining Which Conductor AMI To Use For The Project¶
The AMI ID is required for the
init command. Fugue distributes
versions of the Conductor with specific AMI IDs, so that you get a
pre-built, working machine image instead of software you have to install
on your own OS. Fugue Conductors are built on Debian Linux. You can find
the ID for each release at the Download
Portal. Generally, you will want to use
an AMI that contains a version of the Conductor that is compatible with
the Fugue CLI you plan to use.
When the user executes
fugue init, the CLI validates the provided
Conductor AMI ID, then creates the
fugue.yaml file already exists, the CLI renames it
fugue.yaml.old and creates a new
fugue.yaml file with the new
fugue.yaml.old file already exists, the CLI deletes it, renames
fugue.yaml.old, and creates a new
fugue.yaml file with the new settings.
For example, when no
fugue.yaml file exists, and AWS credentials are
stored in a default profile in
fugue init ami-f9baedee produces the following output:
[ fugue init ] Initializing Fugue project with the following configuration: Fugue Conductor AMI ID: ami-f9baedee AWS Credentials: Profile (default) Validating Fugue Conductor AMI ID ... [ OK ] Provided AMI ID is valid. Creating new fugue.yaml file ... [ Done ] Project initialized.
Likewise, when no
fugue.yaml file exists, and AWS credentials are
stored in environment variables, executing
fugue init ami-f9baedee
produces the following output:
[ fugue init ] Initializing Fugue project with the following configuration: Fugue Conductor AMI ID: ami-f9baedee AWS Credentials: Environment variables Validating Fugue Conductor AMI ID ... [ OK ] Provided AMI ID is valid. Creating new fugue.yaml file ... [ Done ] Project initialized.
Note: The default CLI timeout is 15 minutes. This timeout setting only
fugue init is being run for the first time or there are
no existing timeout settings in the
fugue init ami-482ae35e produces the following
conductor: ami: ami-482ae35e
fugue init --profile myProfile ami-482ae35e produces the
aws: credentialProfile: myProfile conductor: ami: ami-482ae35e
fugue install, the
fugue.yaml file might look like
aws: credentialProfile: myProfile conductor: ami: ami-482ae35e user: userId: root userSecret: gIowKW6AnAM4fgBqEXAMPLEEXAMPLEEXAMPLEEXAMPLE
All of the fields listed below are optional, except for
userSecret. Manually editing the
should not be necessary during normal operation, but you may wish to
review these settings when
troubleshooting or learning about
If you ran
fugue init -p <profile>, then the
aws block contains
credentialProfile, specifying that AWS credential
profile. However, if you’re using the
default profile, the
block is optional and does not appear.
aws: credentialProfile: myProfile
conductor block governs how the CLI installs Fugue in an AWS
account and interacts with it.
conductor: region: us-east-1 ami: ami-f00ba5 instanceType: m4.large keyName: null requestQueue: fugue-demarc-requests responseQueue: fugue-cli-resp-ycbctybtqxqwofwrvhom compositionBucket: fugue-xxxxxxxxxxxx-us-east-1 installAZ: - us-east-1c - us-east-1d
- Specifies the AWS region in which the
fuguebinary operates for this project. All calls to the AWS API that the CLI makes are directed to this region. This influences all commands, including
- Specifies the Amazon Machine Image that the EC2 Fugue Conductor will
launch with. Different AMIs may have different versions of the Fugue
Conductor installed. This is set with the AMI ID argument. The
fuguebinary version and Conductor version are generally a matched set, and compatibility between different versions is not guaranteed, so this setting should not ever be changed for production use.
- Specifies the instance type that the EC2 Fugue Conductor launches
with. Currently, the only supported size is
m4.large, so do not change this setting.
- Allows you to specify an SSH keypair to associate with the Fugue
Conductor EC2 instance at launch. Fugue launches the Conductor with
no support for inbound TCP connections of any kind by default, so to
log in to the instance via SSH, you can specify a keypair with this
setting, and the CLI will add a security group ingress rule allowing
access to the Conductor on port 22 from the IP address where
fugue installis run.
- A list of Availability Zones (AZs) to use for the Fugue Conductor
installation in your account. The Fugue Conductor installation uses
an Autoscaling Group that straddles two AZs to ensure availability.
fugue installcommand will figure out two suitable AZs for the installation on its own. However, in some narrow cases this detection mechanism can get false positives from the AWS API (causing the Fugue CloudFormation stack to fail), or you may need to specifically control the AZs that Fugue runs in. In such cases, provide a list in this field with one or two AZs for Fugue to use (you may put more in the list, but only the first two will be used). Note that if you supply only one AZ, you will see a warning in the CLI log file from the
requestTimeoutSecs field governs how long the CLI waits for a
response from the Conductor for requests.
retry block governs how the CLI handles commands that don’t work
perfectly the first time and exceed the
without a response. Most of these should not be modified during normal
operation. However, for troubleshooting or experimentation, there are
several settings you can change:
retry: initialDelayMs: 200 backoff: constant maxDelayMs: null cutoffDelayMs: null cutoffRetries: 5
- Sets the initial wait time between requests in milliseconds.
- Determines whether the wait time increases between requests, and if
so, by how much. Valid values are
constant, which keeps the delay at the initial setting;
fibonacci, which increases the delay using a Fibonacci formula seeded with the initial delay; and
exponential, which increases the delay exponentially with each failed request.
- Sets a cap on delay between retries in milliseconds. The delay will
always be the lesser of this value or that determined by the
exponentialbackoff algorithms. The request will not be terminated when this cap is reached, the delay will just not increase beyond it.
- Sets a delay at which the CLI gives up on the request. When this delay value is reached, the request attempt terminates.
- Sets a maximum number of attempts to make a request, after which it gives up on the request.
lwcOptions field is for internal or support use only. It
enables the user to override
fugue run and
fugue update with
lwc (Ludwig compiler) options, which are passed
lwc. An example would look like this:
lwcOptions: --timeout=30 --no-color
The above example would set the target file’s compilation timeout at 30
seconds and disable colorized output. For more information about
options, see Ludwig Compiler
user block contains information about the current Fugue user.
This block is added to
fugue.yaml during the install
user: userId: root userSecret: gIowKW6AnEXAMPLEEXAPMLEEXAPMLEEXAMPLEEXAMPLE
- Field is generated during
fugue install. Specifies the current Fugue user (
rootby default). You can use the user command to change the user, or you can change it here manually. Just remember to also change the corresponding
- Field is generated during
fugue install. A symmetric access token that the CLI uses to sign requests to the Conductor, authenticating the user. Each
usermust have a corresponding
userSecretto use the CLI. To generate a secret, use the
policy generate-secret <user_id>command (see policy). Note: As with any secret, the
userSecretshould be guarded carefully. Do not check your
fugue.yamlfile into source control.
Most of the remaining settings should not be modified during normal operation. However, for troubleshooting or experimentation, there are several settings you can change that govern how the CLI finds and interacts with the Fugue Conductor:
- Is the queue to which the CLI sends requests. Note: This setting should not ever be changed for production use.
- Is the queue from which the CLI reads responses. Note: This setting should not ever be changed for production use.
- Is the name of the bucket used to store loaded and running Fugue
compositions. It is calculated as a composite of the string
fugue-, your AWS account number, and the value of the
regionfield. This allows the bucket name to be globally unique, which is important since all S3 buckets across AWS share one namespace.
Changing the AWS credential profile the Fugue CLI uses¶
If you have configured the Fugue CLI to use a particular set of AWS
credentials but you need to specify a different set, you can run
fugue init to change which AWS profile Fugue uses in the
~/.aws/credentials file. (Recall that root account credentials
are not supported.)
For example, if you’ve configured the Fugue CLI to use your personal AWS
account, but you need to reconfigure it to use your corporate AWS
account, you may first run
aws configure --profile CORPORATE_PROFILE, and enter the AWS access
key ID and secret access key associated with the corporate account.
fugue init AMI -p CORPORATE_PROFILE to switch Fugue to
that profile in the local
As usual, the Fugue CLI checks to make sure the target account has the correct permissions to run the Fugue CLI and install the Fugue Conductor.
Note: If you have already executed
fugue install but need to
change your AWS account credentials, you must
fugue init with the new account credentials, and
again for the credential settings to take effect.
Changing The Region The Conductor Is Running In¶
While we anticipate changing this in the future, at present Fugue only
supports running Conductors in the
Changing The Size Of The Conductor Instance¶
The Conductor instance type is m4.large by default. Currently, we do not support changing the size of the Conductor instance. If you believe you need a different size, email email@example.com.
Checking AWS credentials with
You can use the
--creds-check option to validate your AWS
credentials. The CLI checks to see if the IAM policy associated with the
configured AWS credentials has the required permissions to run the Fugue
CLI and install the Fugue Conductor.
When you execute
fugue init --creds-check <ami_id>, the CLI does the
- The CLI searches for AWS credentials according to the order listed here.
- The CLI checks the credentials to ensure the user is authorized to use the Fugue CLI and install the Fugue Conductor.
- If the credentials do not have sufficient permissions, the CLI displays an error.
- If the credentials do have sufficient permissions, the CLI
validates the provided Conductor AMI ID and creates the
Example error message for credentials missing one or more required IAM permissions:
[ fugue init ] Initializing Fugue project with the following configuration: Fugue Conductor AMI ID: ami-f9baedee AWS Credentials: Profile (no-access-creds-test) Checking your AWS Credentials for Fugue CLI use ... [ ERROR ] There was a problem executing this command. Reason: An error occurred (AccessDenied) when calling the SimulatePrincipalPolicy operation: User: arn:aws:iam::xxxxxxxxxxxx:user/no-access-creds-test is not authorized to perform: iam:SimulatePrincipalPolicy on resource: arn:aws:iam::xxxxxxxxxxxx:user/no-access-creds-test
Example error message for erroneous credentials:
[ fugue init ] Initializing Fugue project with the following configuration: Fugue Conductor AMI ID: ami-f9baedee AWS Credentials: Profile (mistyped-creds) Checking your AWS Credentials for Fugue CLI use ... [ ERROR ] There was a problem executing this command. Reason: An error occurred (InvalidClientTokenId) when calling the GetCallerIdentity operation: The security token included in the request is invalid.
Manually Setting Installation Availability Zones¶
If you have problems installing Fugue due to an Availability Zone (AZ)
being unavailable for new resources in your account, or wish to
customize the AZs used by Fugue for installation, you can manually
specify the AZs that Fugue uses for the Fugue Conductor installation in
fugue.yaml. These AZs will be used when you run the
fugue install command. To do this, add an entry in
conductor heading called
installAZ as a list of two
known-good availability zones in your account. For example:
conductor: installAZ: - us-east-1c - us-east-1d
Next time you run
fugue install, the command will install the Fugue
Conductor VPC with subnets in the first two AZs listed.
There are two caveats you should keep in mind if you have to use this feature. Firstly, bear in mind that you can specify one or two AZs. If you specify one, the CLI will log a warning during installation, but Fugue will install and run. If you specify two, installation should proceed as normal. If you specify more than two, only the first two will be used, and all subsequent entries in the list will be ignored.
Secondly, the AZs you list will be assumed valid. If you provide an invalid or unavailable AZ for your account, installation will fail with a CloudFormation stack error, like:
[ ERROR ] AWS CloudFormation stack creation failed
Note that you can get a list of AZs available to you using the AWS CLI command describe-availability-zones, but this is the same command that Fugue uses, and it sometimes returns unreliable data. As a rule of thumb, you should look for a pair of AZs that are listed as available to you, but that you don’t heavily utilize.