Infrastructure Terraform

TF-CIX as a new approach to share information between terraform stacks - Part 3

By Daniel Ciaglia
TF-CIX as a new approach to share information between terraform stacks - Part 3

Photo by Tim Mossholder Unsplash

This post is Part 3 in the TIER cloud infrastructure series. Jump to Part 2 and Part 1 for more information.

Preface

Don’t share the state – this is the pattern of Terraform Configuration Inter eXchange (TF-CIX) we implemented in order to exchange information between our different Terraform stacks.

Problem solving

When sharing information between stacks, you may face several issues, eg.:

  1. Sensitive Data in State
  2. Full access to all data of a stack
  3. Hard dependencies between stacks

👉 There is one simple rule to follow – Don’t share the state between teams

What to do in this case? Here’s a solution: use an intermediary data storage. The source stack writes SSM parameters in a defined structure while a TF-CIX module reads and exposes the data.

TIER implementation

Let’s take a look at the ring-0/core-infra stack to see how we use this approach.

The stack creates:

Then there is another stack eg. teamFoo/serviceBar. The new stack may need some basic resources

Share it

$> cat ring-0/core-infra/tf-cix.tf

locals {
  ssm_prefix = "/tf-cix/${var.metadata["service"]}/${var.region}"
}

resource "aws_ssm_parameter" "main_db_networks" {
  description = "The ID of the _db_networks_"
  name        = "${local.ssm_prefix}/main-vpc/db_networks"
  tags        = local.aws_tags
  type        = "String"
  value       = jsonencode(var.network_config.db_networks)
  overwrite   = true
}

resource "aws_ssm_parameter" "aaa" { }
resource "aws_ssm_parameter" "bbb" { }
resource "aws_ssm_parameter" "ccc" { }
resource "aws_ssm_parameter" "ddd" { }


----
SSM Parameter: /tf-cix/core-infra/eu-central-1/main-vpc/db_networks
Value:         ["10.0.56.0/21","10.0.120.0/21","10.0.184.0/21"]

Behind the scenes

$> cat TF-CIX/core-infra/main.tf

data "aws_ssm_parameter" "vpc_id" {
  name = "/tf-cix/${var.service}/${var.region}/${var.name}/id"
}

data "aws_ssm_parameter" "aaa" { }
data "aws_ssm_parameter" "bbb" { }


$> cat TF-CIX/core-infra/outputs.tf

output "vpc_id" {
  description = "The ID of the _main-vpc_"
  value       = data.aws_ssm_parameter.vpc_id.value
}

output "aaa" { }
output "bbb" { }

Consume it

$> cat teamFoo/serviceBar/data.tf

module "core-data" {
  source  = "your.terraform.registry.local/core-infra"
  version = "~> 1.0"

  service = "core-infra"
  region  = "eu-central-1"
  name    = "main-vpc"
}

$> cat teamFoo/serviceBar/service.tf

module "db" {
  source  = "your.terraform.registry.local/tier/postgresql/aws"
  version = "~> 1.0"

  name                       = "db"
  subnets                    = module.core-data.vpc_aux_networks
  instance_type              = "db.t3.micro"
  engine_version             = "11.8"
  vpc_id                     = module.core-data.vpc_id
  allowed_security_groups    = [module.core-data.eks-worker-security-group-id]
  vault_enabled              = true
  [...]
}

Conclusion

To wrap up, exchanging information in a standardized and abstracted way allows to layer the infrastructure without the risk of sharing sensitive information between teams.

The actual implementation of the shared resources can be changed without breaking their dependencies when using an API.

The series

Loading open positions...
TF-CIX as a new approach to share information between terraform stacks - Part 3
Older post

Building TIER cloud infrastructure as code - Part 2
Newer post

The Boring Registry

TF-CIX as a new approach to share information between terraform stacks - Part 3