RDS Password Encryption with Vars

Overview

Encrypted passwords are a common requirement for a variety of tools. This example is designed to teach you how to use the Fugue software, including Vars, and the Ludwig language to run a composition that points to an encrypted password.

This example demonstrates, for a slightly more experienced user, how to set up an Amazon Relational Database Service (RDS) (in this example a SQL database) and access that database with an encrypted password. We’ll explain how to point to a Vars encrypted key in a Ludwig declaration so that the value of the key can be used as a password field in a Ludwig composition.

Before you get started, if you need a quick review, check out:

Finally, if you’d like to take a look at the specific module we’re working with, you can view it here: Fugue.AWS.RDS.DBInstance

Prerequisites

This example assumes the following:

For this example we’re only going to be using MySql as a client to connect to the RDS instance.

Note: If you’re using Homebrew, installing MySql is as simple as running brew install mysql.

What We’ll Do In This Example

We’ll show you how to use Vars put to apply an encrypted password and then access an RDS instance from the Fugue CLI with the password you created.

What We’ll Have When We’re Done

A running RDS instance in AWS with encrypted password access.

How Long It Will Take

About 15 minutes.

Download

You can view/download the source code here in Github. Save it to the directory in which you ran fugue init during your Fugue setup.

Let’s Go!

Launching Vars

In this example we’re going to launch our vars daemon in AWS region us-east 1.

$ vars -v daemon -r us-east-1

When the Vars output message says “Starting web service on port 4444”, Vars is ready to go.

{
  "component": "vars",
  "log_level": "info",
  "message": "Using aws region: us-east-1",
  "timestamp": "2016-07-13T18:33:38.798173"
}
{
  "component": "vars",
  "log_level": "info",
  "message": "Starting vars daemon",
  "timestamp": "2016-07-13T18:33:38.985732"
}
{
  "component": "vars",
  "log_level": "warning",
  "message": "Cannot get to instance metadata service. The node ID and queue key will be: external-08-6d-41-be-bc-fe",
  "timestamp": "2016-07-13T18:33:39.992699"
}
{
  "component": "vars",
  "log_level": "info",
  "message": "Starting web service on port 4444",
  "timestamp": "2016-07-13T18:33:41.754954"
}

Creating a KMS Key

Create a new Key Management Service (KMS) key. Details about creating a key are available here.

Note: All passwords require a minimum of 8 characters.

Log in to your AWS account and navigate to the IAM portion of the portal. From here you can select the Encryption Keys option to launch the wizard and create a new KMS key including its permissions, and an alias.

Note: when defining the key usage permissions, make sure you include the IAM roles for both the Fugue Conductor and your intended Vars client.

Our example here uses the password alias testpass.

The IAM Landing Page.

The IAM Landing Page

New KMS Key

A new KMS Key

Create a Password and Launch an RDS Instance

Run the vars put command to encrypt your password (value) and associate it with your AWS KMS master key. In this example after confirming our installation with fugue status, we use vars put to create the password and then issue the fugue run command to launch the sample composition rds_pw.lw and confirm the creation of a running RDS process with the alias varstest.

$ fugue status

Fugue Status Report for user/xxxxxxxxxxxx - Mon Oct 17 2016 3:22pm

State    Updated    Created    Account    FID    Alias    Last Message    Next Command
-------  ---------  ---------  ---------  -----  -------  --------------  --------------
Nothing to see here. Go create something! :-)
$ vars put /rds/password/ testpassword -e alias/testpass
testpassword
$ fugue run rds_pw.lw -a varstest
[ fugue run ] Running /Users/user/test-fugue/rds_pw.lw

Run Details:
    Account: default
    Alias: varstest

Compiling Ludwig file /Users/carrie/test-fugue/rds_pw.lw
[ OK ] Successfully compiled. No errors.

Uploading compiled Ludwig composition to S3...
[ OK ] Successfully uploaded.

Requesting the Conductor to create and run process based on composition ...
[ DONE ] Process created and running.


State    Updated    Created    Account              FID                                   Alias     Last Message    Next Command
-------  ---------  ---------  -------------------  ------------------------------------  --------  --------------  --------------
Running  3:36pm     3:36pm     fugue-1505320402664  2f884072-b534-4a92-a337-fdccab5fdadb  varstest                  run

[ HELP ] Run the "fugue status" command to view details and status for all Fugue processes.
$ fugue status


Fugue Status Report for user/xxxxxxxxxxxx - Mon Oct 17 2016 3:41pm


State    Updated    Created    Account              FID                                   Alias     Last Message    Next Command
-------  ---------  ---------  -------------------  ------------------------------------  --------  --------------  --------------
Running  3:36pm     3:36pm     fugue-1505320402664  2f884072-b534-4a92-a337-fdccab5fdadb  varstest  SUCCEEDED

Confirm Your Instance and Find Your Endpoint

From the Amazon AWS Portal, under the RDS Dashboard, selecting instances displays your newly created RDS instance. This quick check confirms the infrastructure you created exists. On the Security and Network tab there are details about your instance, including your endpoint.

RDS Instance

Your RDS Instance.

Connecting to the Database

After determining your endpoint (e.g. dboneinstanceid.c2b0zbacnoqy.us-west-2.rds.amazonaws.com), you can run the following command (our example assumes the installation of MySql) to connect to your RDS instance and supply the encrypted password you created.

$ mysql -u DBOneMaster -p -h <endpoint>
Enter password:

Once you’ve successfully provided the password you should be able to confirm the database you created (DBOneDBName) from the sample composition.

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 21
Server version: 5.6.27-log MySQL Community Server (GPL)
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| DBOneDBName        |
| innodb             |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
6 rows in set (0.09 sec)
mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+-------------------------------+
| Tables_in_mysql               |
+-------------------------------+
| columns_priv                  |
| db                            |
| event                         |
| func                          |
| general_log                   |
| help_category                 |
| help_keyword                  |
| help_relation                 |
| help_topic                    |
| host                          |
| innodb_index_stats            |
| innodb_table_stats            |
| ndb_binlog_index              |
| plugin                        |
| proc                          |
| procs_priv                    |
| proxies_priv                  |
| rds_configuration             |
| rds_global_status_history     |
| rds_global_status_history_old |
| rds_heartbeat2                |
| rds_history                   |
| rds_replication_status        |
| servers                       |
| slave_master_info             |
| slave_relay_log_info          |
| slave_worker_info             |
| slow_log                      |
| tables_priv                   |
| time_zone                     |
| time_zone_leap_second         |
| time_zone_name                |
| time_zone_transition          |
| time_zone_transition_type     |
| user                          |
+-------------------------------+
35 rows in set (0.09 sec)


mysql>

Changing the Password

To change the password associated with your KMS key you only need to make two small changes. First, simply revise your composition on the `masterUserPassword` line third from the bottom. Below is the RDS portion of the source code provided in Github.

#########################
# RDS
#########################

my-db-subnet-group: RDS.DBSubnetGroup.new {
  name: "DBSubnetGroupOne",
  description: "DBSubnetGroupOne description...",
  subnets: network.publicSubnets,
  tags: [mvc-rds-tag]
}

my-db-instance: RDS.DBInstance.new {
  dbInstanceIdentifier: 'DBOneInstanceID',
  dbInstanceClass: RDS.DB_T2_SMALL,
  engine: RDS.MySQL,
  masterUsername: "DBOneMaster",
  dbName: "DBOneDBName",
  allocatedStorage: 1000,
  securityGroups: [mvc-rds-sg],
  availabilityZone: None,
  dbSubnetGroup: my-db-subnet-group,
  multiAZ: True,
  publiclyAccessible: True,
  storageType: EC2.Standard,
  port: mvc-rds-instance-port,
  masterUserPassword: Vars.VarsKey({key: "/rds/password"}),
  tags: [mvc-rds-tag],
}

And here is the updated RDS portion of the composition where `masterUserPassword`, third line from the bottom, reflects the creation of a new password.

#########################
# RDS
#########################

my-db-subnet-group: RDS.DBSubnetGroup.new {
  name: "DBSubnetGroupOne",
  description: "DBSubnetGroupOne description...",
  subnets: network.publicSubnets,
  tags: [mvc-rds-tag]
}

my-db-instance: RDS.DBInstance.new {
  dbInstanceIdentifier: 'DBOneInstanceID',
  dbInstanceClass: RDS.DB_T2_SMALL,
  engine: RDS.MySQL,
  masterUsername: "DBOneMaster",
  dbName: "DBOneDBName",
  allocatedStorage: 1000,
  securityGroups: [mvc-rds-sg],
  availabilityZone: None,
  dbSubnetGroup: my-db-subnet-group,
  multiAZ: True,
  publiclyAccessible: True,
  storageType: EC2.Standard,
  port: mvc-rds-instance-port,
  masterUserPassword: Vars.VarsKey({key: "/rds/newpassword"}),
  tags: [mvc-rds-tag],
}

Second, you will need to issue another vars put command to encrypt a new password (value) and associate it with the existing AWS KMS master key.

$ vars put /rds/newpassword/ newtestpassword -e alias/testpass
newtestpassword

After successfully issuing the vars put command you will need to issue a fugue update with your revised composition.

$ fugue update [FID/Alias] rds_pw_revised.lw

One thing to note is that the changes won’t fully take effect until AWS updates the Master password through it’s own system, which is separate from Fugue, and can take a minute or more to propagate.

Killing The Fugue Process

We don’t want the RDS instance running indefinitely, so we’ll clean this one up by killing the process. It’s very easy to do:

$ fugue kill varstest
[ fugue kill ] Killing process with Alias: varstest

[ WARN ] Are you sure you want to kill the process with Alias: varstest? [y/N]: y
Requesting the Conductor to kill Suspended composition with Alias: varstest...
[ Done ] The conductor is killing the process with Alias: varstest

Within a few minutes, you should see varstest gone from your account. The information may take a few minutes longer to disappear from the AWS console.

Next Steps

Now that you’ve completed this example, why not take a look at the Fugue.AWS.RDS.DBInstance Module or explore the Vars Guide or head back to the Fugue By Example section to Build a Network or deply a Lambda function!