fugue [global options] init [options] <region>
Required: The region in which to install the Fugue Conductor.
fugue init us-east-1
/Note/: The Fugue Conductor may only be installed in the following
Global options are detailed here.
Override the AMI to use in the specified <region>
fugue init --ami ami-1234567 us-east-1
Specify the name of the AWS credential profile in
credentialsto use when searching for credentials. Defaults to
fugue init -p myprofile us-east-1
- Specify the Conductor type to use:
- Check that the configured AWS credentials have sufficient permissions to use the Fugue CLI and install the Fugue Conductor.
- Show help text. The help flag is available throughout the CLI in both an application-level and command-level context. It enables a user to view help text for any command within the Fugue CLI.
init command initializes a project in the current directory.
The initialization process consists of setting the Conductor version
and recording the Conductor AMI ID in a
fugue.yaml file. The AMI ID
is selected by Fugue unless the user overrides it with the
option. The name of the AWS credential profile is also recorded in
fugue.yaml if it is not the
When a user issues the
fugue install command the fields
secret are generated and stored in Fugue
fugue install also generates a default profile name, for
[default-53812345123-us-east-1], typically based on the
account number and region.
Note: In earlier versions of Fugue the fields
userSecret were stored in the
fugue.yaml file. Those field are
now stored in your current working directory in
secret but correspond to the same values, they just
vary based on your version of Fugue. Otherwise, Fugue will still check
for these values in both the
fugue.yaml file and the
file to maintain backwards compatibility.
The file may
optionally contain other settings that
fugue CLI through its various operations.
Anytime the Fugue CLI executes a command, it searches and verifies Fugue user credentials both
credentials and in
fugue.yaml (for backwards compatibility) and loads the applicable configuration settings. The CLI first searches for
fugue.yaml in a project-level directory, then in a user-level directory, and finally in a system-level directory.
In Mac and Linux systems, these folders are:
- The current working directory
In Windows, these folders are:
- The current working directory
Fugue will use the first file it finds in the search path described above.
init must be executed prior to a
fugue install. Once the Fugue
Conductor has been installed, the changes
init makes to the
fugue.yaml settings (i.e., AWS credentials and Conductor version)
won’t take effect until the Fugue Conductor is uninstalled and
reinstalled. Note: You may use upgrade to change the
Conductor AMI without having to uninstall and reinstall Fugue (for
fugue upgrade ami-e1d64ff6).
Before you run
fugue init, you should know:
- Which AWS credentials do I want to use for this project?
- Which region do I want to run a Conductor in for this project?
Determining AWS Credentials To Use For The Project¶
init is executed, the CLI searches for credentials in the
following order, using the first set it finds:
- The environment variables
~/.aws/credentials, unless the
init -p <profile>option is specified; then,
~/.aws/config, unless the
init -p <profile>option is specified; then finally,
- The instance metadata service (IMDS), which is present only on EC2 instances and provides credentials reflecting the IAM Role of the instance.
The easiest of these to use for a developer desktop is usually the credential profile option. The best way to configure credential profiles is to use the AWS CLI. You can read more about quickly setting up AWS credentials with both Fugue and the AWS CLI here.
Note: The Fugue CLI does not support the use of AWS root account
credentials. If you attempt to configure Fugue to use your AWS root
init will fail. AWS recommends creating an IAM user for
everyday use and using its credentials instead, because root credentials
cannot be restricted. If administrative-level permissions are desired,
you can grant the IAM user full access, allowing you to modify or revoke
permissions later if necessary. Use the IAM user credentials with
Fugue instead. For more information, see AWS’s documentation on root
account vs. IAM user
Fugue generates IAM policies and roles for you during the
process. For more information, see AWS Permissions and the Fugue CLI.
Determining Which Region To Use For The Project¶
The region is required for the
init command. Fugue distributes
versions of the Conductor with specific AMI IDs, so that you get a
pre-built, working machine image instead of software you have to install
on your own OS. Fugue Conductors are built on Debian Linux. You can find
the ID for each release at the Download
When the user executes
fugue init, the CLI validates the Conductor
AMI ID, if provided, or selects a Conductor AMI ID for the user, then
fugue.yaml file already exists, the CLI renames it
fugue.yaml.old and creates a new
fugue.yaml file with the new
fugue.yaml.old file already exists, the CLI deletes it, renames
fugue.yaml.old, and creates a new
fugue.yaml file with the new settings.
For example, when no
fugue.yaml file exists, and AWS credentials are
stored in a default profile in
fugue init --ami ami-f9baedee us-east-1 produces the following
[ fugue init ] Initializing Fugue project with the following configuration: Fugue Conductor AMI ID: ami-f9baedee AWS Credentials: Profile (default) Region: us-east-1 Validating Fugue Conductor AMI ID ... [ OK ] Provided AMI ID is valid. Creating new fugue.yaml file ... [ Done ] Project initialized.
Likewise, when no
fugue.yaml file exists, and AWS credentials are
stored in environment variables, executing
fugue init --ami ami-f9baedee us-east-1 produces the following
[ fugue init ] Initializing Fugue project with the following configuration: Fugue Conductor AMI ID: ami-f9baedee AWS Credentials: Environment variables Validating Fugue Conductor AMI ID ... [ OK ] Provided AMI ID is valid. Creating new fugue.yaml file ... [ Done ] Project initialized.
Note: The default CLI timeout is 15 minutes. This timeout setting only
fugue init is being run for the first time or there are
no existing timeout settings in the
fugue init us-east-1 produces the following
conductor: ami: ami-482ae35e region: us-east-1
fugue init --profile myProfile us-east-1 produces the
aws: credentialProfile: myProfile conductor: ami: ami-482ae35e region: us-east-1
fugue install, the
fugue.yaml file might look like
conductor: ami: ami-482ae35e region: us-east-1 secretsKeyId: alias/fugue/rbac/secrets
Note: In earlier versions of Fugue the
fugue.yaml may contain the
userId and userSecret fields. These details are now stored in
credentials but are respected regardless of location.
All of the fields listed below are optional, except for
region. Manually editing the
fugue.yaml file should not be
necessary during normal operation, but you may wish to review these
settings when troubleshooting or
learning about Fugue.
If you ran
fugue init -p <profile>, then the
aws block contains
credentialProfile, specifying that AWS credential
profile. However, if you’re using the
default profile, the
block is optional and does not appear.
aws: credentialProfile: myProfile
conductor block governs how the CLI installs Fugue in an AWS
account and interacts with it.
conductor: region: us-east-1 ami: ami-f00ba5 instanceType: m4.large keyName: null requestQueue: fugue-demarc-requests responseQueue: fugue-cli-resp-ycbctybtqxqwofwrvhom compositionBucket: fugue-xxxxxxxxxxxx-us-east-1 installAZ: - us-east-1c - us-east-1d
- Specifies the AWS region in which the
fuguebinary operates for this project. All calls to the AWS API that the CLI makes are directed to this region. This influences all commands, including
- Specifies the Amazon Machine Image that the EC2 Fugue Conductor will
launch with. Different AMIs may have different versions of the Fugue
Conductor installed. This is set with the AMI ID argument. The
fuguebinary version and Conductor version are generally a matched set, and compatibility between different versions is not guaranteed, so this setting should not ever be changed for production use.
- Specifies the instance type that the EC2 Fugue Conductor launches
with. Currently, the only supported size is
m4.large, so do not change this setting.
- Allows you to specify an SSH keypair to associate with the Fugue
Conductor EC2 instance at launch. Fugue launches the Conductor with
no support for inbound TCP connections of any kind by default, so to
log in to the instance via SSH, you can specify a keypair with this
setting, and the CLI will add a security group ingress rule allowing
access to the Conductor on port 22 from the IP address where
fugue installis run.
- A list of Availability Zones (AZs) to use for the Fugue Conductor
installation in your account. The Fugue Conductor installation uses
an Autoscaling Group that straddles two AZs to ensure availability.
fugue installcommand will figure out two suitable AZs for the installation on its own. However, in some narrow cases this detection mechanism can get false positives from the AWS API (causing the Fugue CloudFormation stack to fail), or you may need to specifically control the AZs that Fugue runs in. In such cases, provide a list in this field with one or two AZs for Fugue to use (you may put more in the list, but only the first two will be used). Note that if you supply only one AZ, you will see a warning in the CLI log file from the
requestTimeoutSecs field governs how long the CLI waits for a
response from the Conductor for requests.
retry block governs how the CLI handles commands that don’t work
perfectly the first time and exceed the
without a response. Most of these should not be modified during normal
operation. However, for troubleshooting or experimentation, there are
several settings you can change:
retry: initialDelayMs: 200 backoff: constant maxDelayMs: null cutoffDelayMs: null cutoffRetries: 5
- Sets the initial wait time between requests in milliseconds.
- Determines whether the wait time increases between requests, and if
so, by how much. Valid values are
constant, which keeps the delay at the initial setting;
fibonacci, which increases the delay using a Fibonacci formula seeded with the initial delay; and
exponential, which increases the delay exponentially with each failed request.
- Sets a cap on delay between retries in milliseconds. The delay will
always be the lesser of this value or that determined by the
exponentialbackoff algorithms. The request will not be terminated when this cap is reached, the delay will just not increase beyond it.
- Sets a delay at which the CLI gives up on the request. When this delay value is reached, the request attempt terminates.
- Sets a maximum number of attempts to make a request, after which it gives up on the request.
lwcOptions field is for internal or support use only. It
enables the user to override
fugue run and
fugue update with
lwc (Ludwig compiler) options, which are passed
lwc. An example would look like this:
lwcOptions: --timeout=30 --no-color
The above example would set the target file’s compilation timeout at 30
seconds and disable colorized output. For more information about
options, see Ludwig Compiler
Note: In previous versions of Fugue, the
user block contained
information about the current Fugue user. This block was added to
fugue.yaml during the install process. Current versions
of Fugue store this information in
credentials and the fields
operate as follows:
- Field is generated during
fugue install. Specifies the current Fugue user (
rootby default). You can use the user command to change the user, or you can change it in
credentialsmanually. Just remember to also change the corresponding
- Field is generated during
fugue install. A symmetric access token that the CLI uses to sign requests to the Conductor, authenticating the user. Each
usermust have a corresponding
secretto use the CLI. To generate a secret, use the
policy generate-secret <user_id>command (see policy). Note: As with any secret, the
secretshould be guarded carefully. Do not check your
credentialsfile into source control.
experimental block contains information about experimental user
settings. These settings generally will only take effect if the
FUGUE_CLI_PREVIEW environment variable is set to
In general, do not use
experimental user settings unless instructed
to do so. Changing these settings can have unintended consequences.
experimental: varsSystemTable: fugue-vars-headless-store-v2
- Specifies the name of the Conductor Vars DynamoDB table, which Vars
uses for key/value storage. Overrides the default table name,
fugue-vars-headless-store. For more information, see Changing the System Vars DDB Table Prior to Installation and Changing the System Vars DDB Table During Upgrade.
Changing your Vars table can have severe consequences, including dropping your entire dataset. Do not change your Vars table unless you have been instructed to do so by Fugue Support.
Most of the remaining settings should not be modified during normal operation. However, for troubleshooting or experimentation, there are several settings you can change that govern how the CLI finds and interacts with the Fugue Conductor:
- Is the queue to which the CLI sends requests. Note: This setting should not ever be changed for production use.
- Is the queue from which the CLI reads responses. Note: This setting should not ever be changed for production use.
- Is the name of the bucket used to store loaded and running Fugue
compositions. It is calculated as a composite of the string
fugue-, your AWS account number, and the value of the
regionfield. This allows the bucket name to be globally unique, which is important since all S3 buckets across AWS share one namespace.
Changing the AWS credential profile the Fugue CLI uses¶
If you have configured the Fugue CLI to use a particular set of AWS
credentials but you need to specify a different set, you can run
fugue init to change which AWS profile Fugue uses in the
~/.aws/credentials file. (Recall that root account credentials
are not supported.)
For example, if you’ve configured the Fugue CLI to use your personal AWS
account, but you need to reconfigure it to use your corporate AWS
account, you may first run
aws configure --profile CORPORATE_PROFILE, and enter the AWS access
key ID and secret access key associated with the corporate account.
fugue init <region> -p CORPORATE_PROFILE to switch Fugue
to that profile in the local
As usual, the Fugue CLI checks to make sure the target account has the correct permissions to run the Fugue CLI and install the Fugue Conductor.
Note: If you have already executed
fugue install but need to
change your AWS account credentials, you must
fugue init with the new account credentials, and
again for the credential settings to take effect.
Changing The Region The Conductor Is Running In¶
While we anticipate changing this in the future, at present Fugue only
supports running Conductors in
us-gov-west-1. To specify a region
for the Conductor use
init and provide the the desired region.
Changing The Size Of The Conductor Instance¶
The Conductor instance type is m4.large by default. Currently, we do not support changing the size of the Conductor instance. If you believe you need a different size, email firstname.lastname@example.org.
Checking AWS credentials with
You can use the
--creds-check option to validate your AWS
credentials. The CLI checks to see if the IAM policy associated with the
configured AWS credentials has the required permissions to run the Fugue
CLI and install the Fugue Conductor.
When you execute
fugue init --creds-check <region>, the CLI does the
- The CLI searches for AWS credentials according to the order listed here.
- The CLI checks the credentials to ensure the user is authorized to use the Fugue CLI and install the Fugue Conductor.
- If the credentials do not have sufficient permissions, the CLI displays an error.
- If the credentials do have sufficient permissions, the CLI
validates the provided Conductor AMI ID and creates the
Example error message for credentials missing one or more required IAM permissions:
[ fugue init ] Initializing Fugue project with the following configuration: Fugue Conductor AMI ID: ami-f9baedee AWS Credentials: Profile (no-access-creds-test) Checking your AWS Credentials for Fugue CLI use ... [ ERROR ] There was a problem executing this command. Reason: An error occurred (AccessDenied) when calling the SimulatePrincipalPolicy operation: User: arn:aws:iam::xxxxxxxxxxxx:user/no-access-creds-test is not authorized to perform: iam:SimulatePrincipalPolicy on resource: arn:aws:iam::xxxxxxxxxxxx:user/no-access-creds-test
Example error message for erroneous credentials:
[ fugue init ] Initializing Fugue project with the following configuration: Fugue Conductor AMI ID: ami-f9baedee AWS Credentials: Profile (mistyped-creds) Checking your AWS Credentials for Fugue CLI use ... [ ERROR ] There was a problem executing this command. Reason: An error occurred (InvalidClientTokenId) when calling the GetCallerIdentity operation: The security token included in the request is invalid.
Manually Setting Installation Availability Zones¶
If you have problems installing Fugue due to an Availability Zone (AZ)
being unavailable for new resources in your account, or wish to
customize the AZs used by Fugue for installation, you can manually
specify the AZs that Fugue uses for the Fugue Conductor installation in
fugue.yaml. These AZs will be used when you run the
fugue install command. To do this, add an entry in
conductor heading called
installAZ as a list of two
known-good availability zones in your account. For example:
conductor: installAZ: - us-east-1c - us-east-1d
Next time you run
fugue install, the command will install the Fugue
Conductor VPC with subnets in the first two AZs listed.
There are two caveats you should keep in mind if you have to use this feature. Firstly, bear in mind that you can specify one or two AZs. If you specify one, the CLI will log a warning during installation, but Fugue will install and run. If you specify two, installation should proceed as normal. If you specify more than two, only the first two will be used, and all subsequent entries in the list will be ignored.
Secondly, the AZs you list will be assumed valid. If you provide an invalid or unavailable AZ for your account, installation will fail with a CloudFormation stack error, like:
[ ERROR ] AWS CloudFormation stack creation failed
Note that you can get a list of AZs available to you using the AWS CLI command describe-availability-zones, but this is the same command that Fugue uses, and it sometimes returns unreliable data. As a rule of thumb, you should look for a pair of AZs that are listed as available to you, but that you don’t heavily utilize.