Vars Tutorial

Overview

Vars is Fugue’s distributed variable service. This tutorial teaches the basics of using Vars and demonstrates its item encryption and watch features.

Prerequisites

Vars is included in the Fugue Client Tools, so you’ll need to download and install the tools locally.

You’ll also need to have the Conductor installed. If you haven’t done so yet, it just takes a few quick steps.

What We’ll Do In This Example

We’ll cover how to install Vars on an EC2 instance and how to use vars daemon, vars put, vars list, vars get, and vars delete. We’ll also introduce Vars watches and encrypted keys.

What We’ll Have When We’re Done

A server IP address and an encrypted database password synchronized between a local Vars client and a Vars client on an EC2 instance.

How Long It Will Take

About 25 minutes.

Let’s Go!

Launch an EC2 Instance

Vars enables you to synchronize values across a fleet of instances, so we’ll demonstrate this by launching an EC2 instance and installing Vars on it, then using your local Vars client to store a value and using the remote Vars client to retrieve it. First, we need to launch an EC2 instance.

Log in to the AWS Management Console and head to the IAM Dashboard. We need to create an IAM policy and attach it to a new IAM role. This way, the EC2 instance profile will contain our new role and pass the attached policy to the EC2 instance we launch.

Click on “Policies” in the left sidebar, then click “Create policy.” Select “Create Your Own Policy.” Give it a name (we’ll use vars-policy) and a description. Then, download the beta Vars IAM policy template here and copy/paste the policy into the Policy Document field.

Warning

This policy document is a beta IAM template. While it isn’t officially supported yet, you’re welcome to try it out. If you find problems, let us know at support@fugue.co.

Click the “Create Policy” button. In the left sidebar, click on “Roles,” then “Create new role.”

Under “AWS Service Role,” select “Amazon EC2.” On the Attach Policy screen, find vars-policy (or whatever you named your policy) and select “Next Step.” Give your new role a name – we went with vars-role – and click “Create role.”

Now, head to the EC2 Dashboard in the N. Virginia region.

Click “Launch Instance,” and on the next screen, select the free-tier eligible Amazon Linux AMI.

Ensure the t2.micro instance type is selected and click “Next: Configure Instance Details.”

On the Configure Instance Details screen, find vars-role underneath the IAM role drop-down menu, then click “Next: Add Storage.”

Keep the defaults on the Add Storage screen and click “Next: Add Tags.” Add tags if you wish and click “Next: Configure Security Group.”

On the Configure Security Group screen, select the drop-down menu under “Source” and choose “My IP.” The settings should be as follows:

  • Type: SSH
  • Protocol: TCP
  • Port Range: 22
  • Source: My IP

Click “Review and Launch.” Review the information and click “Launch.” Select an existing key pair or create a new one. Make sure you have access to your key pair, or you won’t be able to SSH into the instance and complete this exercise! Then click “Launch Instances.”

Finally, click “View Instances” to see the status of your new instance.

Install Vars on the Instance

Once your instance is up and running, you’ll need to SSH into it. Open a new terminal window (which we’ll refer to as the “remote terminal”) and enter the following command, replacing "admin-login.pem" with your key pair name and ec2-54-202-48-243 with your instance IP address:

ssh -i "admin-login.pem" ec2-user@ec2-54-202-48-243.us-west-2.compute.amazonaws.com

To find the exact command you’ll need to use, go back to the EC2 Dashboard and highlight your instance, then click “Connect.” Amazon will display a customized command for you to enter.

Next, we’ll install Vars on the instance. Head to the Download Portal and scroll down to “Additional Downloads,” then select “Download Vars Client.” Download the Amazon Linux RPM.

Switch back to your original terminal window (which we’ll refer to as the “local terminal”) and execute the scp -i command, using the name of your key pair, the name of the Vars RPM file, and the address of your EC2 instance, with :~ at the end. That will look something like this:

scp -i admin-login.pem fugue-vars-*.rpm ec2-user@ec2-54-202-48-243.us-west-2.compute.amazonaws.com:~

In the remote terminal, execute ls and you should see the Vars RPM file in your home directory. Install Vars with the following command:

sudo rpm -ivh fugue-vars*.rpm

Start the Daemon on the EC2 Instance

We’re ready to start the Vars daemon on the EC2 instance! We’ll run it in the background with the & operator so we can continue to use the remote terminal to put and get. We’ll also use the -vv option to turn on verbose, debug-level logging.

vars -vv daemon &

Once you see this output...

{"component":"vars","log_level":"info","message":"Cache Ready","timestamp":"2017-05-01T16:31:15.551395"}

...then you can hit the Enter key to return to your command prompt.

Start the Daemon on the Local Instance

We’re about to start the local Vars daemon. But first, we need to create a watch file, which we’ll implement later in this exercise. For now, it’ll just be an empty file. Switch back to your local terminal and enter the following command:

touch watch.config

Now, we can start the daemon:

vars -vv daemon --watch=watch.config &

As before, this runs the daemon in verbose mode, in the background. This time, though, we’ve used the --watch=watch.config option, which tells Vars where to find the watch file.

Note the PID in the output – it’ll look something like [1] 66525. We’ll need that number to kill the daemon at the end of the exercise.

When you see this output...

{"component":"vars","log_level":"info","message":"Cache Ready","timestamp":"2017-05-01T16:37:19.186451"}

...then you can hit Enter to return to the command prompt.

Store a Value in Vars

It’s time to store a value in Vars! Back in the remote terminal, use the vars put <KEY> <VALUE> command:

vars put appserver_ip 127.0.0.1

You’ll see the value echoed in the final line of the output:

{"command":"put","component":"vars","form":{},"ip":"127.0.0.1","log_level":"debug","message":"SUCCESSFUL REQUEST","method":"POST","params":[{"Key":"key","Value":"/appserver_ip"}],"query":{},"response_code":200,"timestamp":"2017-05-01T20:17:32.918925"}
127.0.0.1

This stores the appserver_ip key with the value 127.0.0.1 in Vars. The EC2 instance’s Vars client synchronizes with the Vars client running on the Conductor, which then distributes the information to all your other Vars clients.

We’ll prove this in the next step.

Retrieve a Value From Vars

Now, switch to the local terminal, and use the vars get <KEY> command:

vars get appserver_ip

You’ll see the key’s value in the final line of the output:

{"command":"get","component":"vars","form":{},"ip":"127.0.0.1","log_level":"debug","message":"SUCCESSFUL REQUEST","method":"GET","params":[{"Key":"key","Value":"/appserver_ip"}],"query":{},"response_code":200,"timestamp":"2017-05-01T20:18:40.063251"}
127.0.0.1

Pretty cool, right? You’ve effortlessly synchronized your server’s IP address between an EC2 instance in Oregon and your local machine. If you have a fleet of instances running Vars, each instance will receive the same information, too.

You can see a list of your Vars keys by executing the list command:

vars list

The output should look similar to this:

{"command":"list","component":"vars","form":{},"ip":"127.0.0.1","log_level":"debug","message":"SUCCESSFUL REQUEST","method":"OPTIONS","params":[{"Key":"prefix","Value":"/"}],"query":{},"response_code":200,"timestamp":"2017-05-02T18:14:58.759094"}
/vars/instances/external-34-36-3b-62-c5-52/status
/vars/instances/conductor_vars_userland/status
/vars/instances/i-0cfcf5d8e8bd62a78/status
/appserver_ip

After the success message are four keys, including your shiny new appserver_ip key. The other three keys contain the status of the Vars daemons on your local machine, the Conductor, and the remote instance.

Set a Watch to Append to a Log

Wouldn’t it be great if you could have an arbitrary command fire whenever a certain key is updated in Vars? Good news – you can!

In the local terminal, edit the watch.config file to add the following line:

/appserver_ip echo "$(date): Server IP updated to $(vars get $WATCHED_KEY)" >> appserver_ip.log

This means when the appserver_ip key is updated, Vars will create the appserver_ip.log file if it doesn’t already exist, and it will append a timestamped message to the end of the file. $(vars get $WATCHED_KEY) retrieves the updated value of the key, which is logged in the message.

Still in the local terminal, update appserver_ip to 1.1.1.1:

vars put appserver_ip 1.1.1.1

You’ll see the following output:

{"command":"put","component":"vars","form":{},"ip":"127.0.0.1","log_level":"debug","message":"SUCCESSFUL REQUEST","method":"POST","params":[{"Key":"key","Value":"/appserver_ip"}],"query":{},"response_code":200,"timestamp":"2017-05-05T21:41:31.121186"}
{"component":"vars","log_level":"debug","message":"echo \"$(date): Server IP updated to $(vars get $WATCHED_KEY)\" \u003e\u003e appserver_ip.log; exit $?;","timestamp":"2017-05-05T21:41:31.121572"}
1.1.1.1
{"component":"vars","log_level":"debug","message":"Started: [bash]","timestamp":"2017-05-05T21:41:31.127727"}

Now, view the appserver_ip.log file that Vars created for you:

cat appserver_ip.log

You’ll see the message “Server IP updated” with a timestamp and the updated appserver_ip value:

Fri May  5 17:41:31 EDT 2017: Server IP updated to 1.1.1.1

This example demonstrates using a watch to update a log, but Vars watches can be used with any command. Note: Watches are supported on Windows, but are limited to simple commands. This particular watch command will only work on Unix machines because it uses a Bash redirection operator.

Encrypt a Value in Vars

Let’s say you’re running a fleet of EC2 instances and you need to ensure all instances have access to a shared credential, like a database password. For the last portion of this tutorial, we’ll demonstrate how to store an encrypted value in Vars.

First, we’re going to create a KMS master key, which will be used to encrypt values in Vars. This is a one-time only step. Log in to the AWS console and head to the IAM Dashboard, then click on “Encryption Keys.”

Click “Create Key” and enter a name, such as vars-test-key. Click “Next Step” and add tags if you wish. On the next screen, “Define Key Administrative Permissions,” select the Conductor’s IAM role. It’ll look something like this: fugue-FugueIam-QIZ2X4G6EB0W

On the next screen, “Define Key Usage Permissions,” select the Conductor’s IAM role again, and also select vars-role.

Finally, click “Finish.”

Return to your local terminal and enter the following command:

vars put prod.db.password password1234 -e alias/vars-test-key

As you know, this stores the key prod.db.password with the value password1234. The -e option signals that the value will be encrypted with the vars-test-key master key. Don’t forget to prefix your master key name with alias/!

Let’s peek inside Vars’ DynamoDB table to prove that the value has successfully been encrypted. Head to the DynamoDB Dashboard in the North Virginia region. Click “Tables” in the left sidebar, and select fugue-vars-store. Click the “Items” tab. Find the key /prod.db.password and click on it. You’ll see a window like this one:

An encrypted value stored in Vars' DynamoDB table.

An encrypted value stored in Vars’ DynamoDB table.

As you can see, the prod.db.password value password1234 has successfully been encrypted and stored!

Now, it’s time to retrieve the encrypted value and decrypt it on the EC2 instance. Switch to the remote terminal and enter the following command:

export DB_PASS=$(vars get prod.db.password)

vars get prod.db.password instructs Vars to retrieve the password, and Vars does the work of decrypting it for you. With this command, the password is now stored in the environment variable $DB_PASS. You can view it like so:

echo $DB_PASS

You’ll see this output:

password1234

Ta-dah! You’ve securely stored the encrypted database password in Vars and distributed it to your “fleet” of instances. Now any instance running a Vars client can safely retrieve and decrypt the value.

Delete Keys in Vars

We don’t need the test keys hanging around in our Vars cache anymore, so let’s delete the prod.db.password key:

vars delete prod.db.password

You’ll see output like this:

{"command":"delete","component":"vars","form":{},"ip":"127.0.0.1","log_level":"debug","message":"SUCCESSFUL REQUEST","method":"DELETE","params":[{"Key":"key","Value":"/prod.db.password"}],"query":{},"response_code":200,"timestamp":"2017-05-02T18:12:30.622037"}

Next, delete the appserver_ip key:

vars delete appserver_ip

You’ll see similar output.

Now, when you execute the list command, you’ll see a shorter list of keys, showing that prod.db.password and appserver_ip have successfully been deleted:

vars list
{"command":"list","component":"vars","form":{},"ip":"127.0.0.1","log_level":"debug","message":"SUCCESSFUL REQUEST","method":"OPTIONS","params":[{"Key":"prefix","Value":"/"}],"query":{},"response_code":200,"timestamp":"2017-05-02T18:17:07.816385"}
/vars/instances/external-34-36-3b-62-c5-52/status
/vars/instances/conductor_vars_userland/status
/vars/instances/i-0cfcf5d8e8bd62a78/status

Shut It All Down

We’ve reached the end of the exercise, so it’s time to terminate the EC2 instance. Head to the EC2 Dashboard in the Oregon region, click on “Instances” in the left sidebar, and find your instance in the list. In the “Actions” drop-down menu, go to “Instance State” and select “Terminate.” Click the “Yes, Terminate” button.

In your local terminal, kill the Vars daemon running in the background by replacing the PID below with your Vars daemon’s PID:

kill 66525

You can also use ps | grep vars to find the PID if you forgot to write it down earlier.

Next Steps

Now that you’ve got the hang of Vars, try our RDS Password Encryption with Vars walkthrough to see how you can use Vars with Fugue to spin up an RDS instance with an encrypted password. Or, read about other Vars commands such as listget, status, or vacuum. You can also return to the Vars Guide to learn more about features and use cases. And as always, reach out to support@fugue.co with any questions.