Trigger Terraform in the cloud with Terraform-deck

For a few weeks now, I've been working on Terraform-deck, an API to kick off Terraform workspaces remotely. I'm using it on a project a for some time and, if it is not easy, I've started to turn it into an open source. It has been a real opportunity. A lot of people I love are supportive and some have already contributed more than I had expected. I'm very grateful. It is a lot of talks, energy and passion!

This first article introduces the tool and shows how you can put an hand on it.

What is Terraform-deck?

If you are using Terraform, I don't have to explain its principles... You can spawn or destroy hundreds of resources in a few minutes. If you need to scale and make them more robust, Terraform Enterprise is the way to go: it is serious; it integrates registries to share your code; it manages parameters; it helps enforce policies; it provides a nice and easy UI; it help you integrates your CI/CD pipelines. Plus, it is worth and fair to reward the fantastic people at Hashicorp for their contributions to our life to be better ops working in the cloud.

Terraform-deck is for fun! Operations are challenging. It is not boring but it is serious. It is about keeping stacks secure, reliable and being able to scale. It means dealing with changes, managing performance and anticipating problems. It is about being on time and moving on. It requires commitment. I like operations, mostly because I'm good at it. If you think it is fun, you are sick! I am famous for what I've broken which, by the way, are my failures. Only few know my magic. That sort of the rule for Ops too. Memories for the specials.

What I call fun for Ops is helping developers experimenting and drafting things fast. Enabling a product team or some business to launch new ideas, and getting rewards. THAT is fun! If you are good at operations, there is near zero risks there but being fast and available is difficult. To say it in another way, because we are so deadly serious, it is hard to be fun.

Terraform-deck helps to be fast. It helps to scale our work to address developer needs. It is an API you can use to kick off projects in the Cloud. It can be used to integrate your stacks into chat bots and applications. I use it myself with Slack and you can easily integrate it any tool. It has replaced my large set of makefile and scripts. Before, I had to help developers installing my tools on their laptop. I had to make them compatible with Linux, Mac and Windows. I had to deal with SSH setup, provider configurations, auditing procedures.

Now, the developers I work with just request a stack to a Slack user bot, and they get it. If they want to upgrade it, they send another command to apply master or whatever tag/branch they are using. And if I want to destroy 10 environments in a row, I just need to kick off one script and I can review the status or the logs later from my phone without any UI... just from Slack.

The way it works is quite simple :
  • It is an API developed with NodeJS with a Go Client 
  • It stores its data in a Couchbase database for now at least
  • It has a pool of docker images that are used to connect to the provider, update the project and run terraform commands

How to test it?

The project comes with a basic Consul demonstration: it is lightweight; it is used internally to test the API; Consul is used as a provider but also to store Terraform state.

If you want to use it, it is pretty simple:
  • Clone the project:
git clone https://github.com/gregoryguillou/terraform-deck.git
cd terraform-deck
  • Start the docker compose, that will download the prebuilt images from the docker hub
cd stack/docker
docker-compose up -d
docker-compose logs -f
  • Download the CLI of your choice from the project release page and configure it. For instance, with Linux, you should run :
~/bin
curl -L -o deck \
   https://github.com/gregoryguillou/terraform-deck/releases/download/v0.1.4/deck-linux-amd64
chmod +x deck
deck configure
Note: The default token for the demonstration stack is notsosecretadminkey
  • Use the CLI to check the stack
deck show -p demonstration -w staging
{
  "creation": 1521760774840,
  "lastEvents": [],
  "project": "demonstration",
  "ref": "branch:master",
  "state": "new",
  "type": "workspace",
  "workspace": "staging"
}
  • You can also query Consul to check for keys. It should be empty
curl --silent http://0.0.0.0:8500/v1/kv/environment/staging/?recurse=true \
  | jq -r '.[] | {Value: .Value, Key: .Key}'
  • Create the stack on master:
deck apply -p demonstration -w staging
  • You can check Consul again and see keys have been created by Terraform
curl --silent http://0.0.0.0:8500/v1/kv/environment/staging/?recurse=true \
  | jq -r '.[] | {Value: .Value, Key: .Key}'
  • You can destroy the stack and stop the docker-compose
deck destroy -p demonstration -w staging
docker-compose kill
docker-compose rm -f 

To continue...

If you want some better explanations about how to use the API with your project, the architecture, the deployment or anything, have a look to the documentation. There are many features yet to come and your comments are welcome. But with some work, you can already:
  • use it to schedule environment by provisioning/deprovisioning from a script in seconds and, by it, cut your provider costs
  • provision environment for demonstration or training
  • build a SaaS offe
  • ...
I'm looking forward to working on richer feature with this API and I'll keep you updated. It already planned ;-).

Comments

Popular posts from this blog

Installing Oracle Database 12.1 in Command Line and "Silent Mode"

Introduction to Oracle Linux 7 Network

Oracle database 12c with Oracle Linux 7.x and LXC