Skip to main content
Data Storage Modules 0.26.0

Redshift Module

View SourceRelease Notes

This module creates an Amazon Redshift cluster that you can use as a data warehouse. The cluster is managed by AWS and automatically handles leader nodes, worker nodes, backups, patching, and encryption.

Redshift architectureRedshift architecture

Features​

  • Deploy a fully-managed data warehouse

  • Scalable worker nodes and storage

  • Automatic nightly snapshots

Learn​

Note

This repo is a part of the Gruntwork Infrastructure as Code Library, a collection of reusable, battle-tested, production ready infrastructure code. If you’ve never used the Infrastructure as Code Library before, make sure to read How to use the Gruntwork Infrastructure as Code Library!

Core concepts​

Deploy​

Non-production deployment (quick start for learning)​

If you just want to try this repo out for experimenting and learning, check out the following resources:

  • examples folder: The examples folder contains sample code optimized for learning, experimenting, and testing (but not production usage).

Production deployment​

If you want to deploy this repo in production, check out the following resources:

  • (coming soon)

Manage​

Day-to-day operations​

Major changes​

Sample Usage​

main.tf

# ------------------------------------------------------------------------------------------------------
# DEPLOY GRUNTWORK'S REDSHIFT MODULE
# ------------------------------------------------------------------------------------------------------

module "redshift" {

  source = "git::git@github.com:gruntwork-io/terraform-aws-data-storage.git//modules/redshift?ref=v0.26.0"

  # ----------------------------------------------------------------------------------------------------
  # REQUIRED VARIABLES
  # ----------------------------------------------------------------------------------------------------

  # The instance type to use for the db (e.g. dc2.large)
  instance_type = <INPUT REQUIRED>

  # The name used to namespace all resources created by these templates, including
  # the DB instance (e.g. drupaldb). Must be unique for this region. May contain
  # only lowercase alphanumeric characters, hyphens.
  name = <INPUT REQUIRED>

  # The number of nodes in the cluster
  number_of_nodes = <INPUT REQUIRED>

  # A list of subnet ids where the database should be deployed. In the standard
  # Gruntwork VPC setup, these should be the private persistence subnet ids.
  subnet_ids = <INPUT REQUIRED>

  # The id of the VPC in which this DB should be deployed.
  vpc_id = <INPUT REQUIRED>

  # ----------------------------------------------------------------------------------------------------
  # OPTIONAL VARIABLES
  # ----------------------------------------------------------------------------------------------------

  # A list of CIDR-formatted IP address ranges that can connect to this DB. Should
  # typically be the CIDR blocks of the private app subnet in this VPC plus the
  # private subnet in the mgmt VPC. This is ignored if create_subnet_group=false.
  allow_connections_from_cidr_blocks = []

  # A list of Security Groups that can connect to this DB.
  allow_connections_from_security_groups = []

  # Indicates whether major version upgrades (e.g. 9.4.x to 9.5.x) will ever be
  # permitted. Note that these updates must always be manually performed and will
  # never automatically applied.
  allow_major_version_upgrade = true

  # A list of CIDR-formatted IP address ranges that this DB can connect. Use this if
  # the database needs to connect to certain IP addresses for special operation
  allow_outbound_connections_from_cidr_blocks = []

  # Indicates that minor engine upgrades will be applied automatically to the DB
  # instance during the maintenance window. If set to true, you should set
  # var.engine_version to MAJOR.MINOR and omit the .PATCH at the end (e.g., use 5.7
  # and not 5.7.11); otherwise, you'll get Terraform state drift. See
  # https://www.terraform.io/docs/providers/aws/r/db_instance.html#engine_version
  # for more details.
  auto_minor_version_upgrade = true

  # The description of the aws_db_security_group that is created. Defaults to
  # 'Security group for the var.name DB' if not specified.
  aws_db_security_group_description = null

  # The name of the aws_db_security_group that is created. Defaults to var.name if
  # not specified.
  aws_db_security_group_name = null

  # How many days to keep backup snapshots around before cleaning them up. Must be 1
  # or greater to support read replicas.
  backup_retention_period = 21

  # The description of the cluster_subnet_group that is created. Defaults to 'Subnet
  # group for the var.name DB' if not specified.
  cluster_subnet_group_description = null

  # The name of the cluster_subnet_group that is created, or an existing one to use
  # if cluster_subnet_group is false. Defaults to var.name if not specified.
  cluster_subnet_group_name = null

  # If false, the DB will bind to aws_db_subnet_group_name and the CIDR will be
  # ignored (allow_connections_from_cidr_blocks)
  create_subnet_group = true

  # Timeout for DB creating
  creating_timeout = "75m"

  # A map of custom tags to apply to the RDS Instance and the Security Group created
  # for it. The key is the tag name and the value is the tag value.
  custom_tags = {}

  # The name for your database of up to 8 alpha-numeric characters. If you do not
  # provide a name, Amazon RDS will not create a database in the DB cluster you are
  # creating.
  db_name = "dev"

  # Timeout for DB deleting
  deleting_timeout = "40m"

  # If true , enhanced VPC routing is enabled. Forces COPY and UNLOAD traffic
  # between the cluster and data repositories to go through your VPC.
  enhanced_vpc_routing = false

  # The name of the final_snapshot_identifier. Defaults to var.name-final-snapshot
  # if not specified.
  final_snapshot_name = null

  # A list of IAM Role ARNs to associate with the cluster. A Maximum of 10 can be
  # associated to the cluster at any time.
  iam_roles = null

  # The ARN of a KMS key that should be used to encrypt data on disk. Only used if
  # var.storage_encrypted is true. If you leave this blank, the default RDS KMS key
  # for the account will be used.
  kms_key_arn = null

  # Configures logging information such as queries and connection attempts for the
  # specified Amazon Redshift cluster. If enable is set to true. The bucket_name and
  # s3_key_prefix must be set. The bucket must be in the same region as the cluster
  # and the cluster must have read bucket and put object permission.
  logging = {"bucket_name":null,"enable":false,"s3_key_prefix":null}

  # The weekly day and time range during which system maintenance can occur (e.g.
  # wed:04:00-wed:04:30). Time zone is UTC. Performance may be degraded or there may
  # even be a downtime during maintenance windows.
  maintenance_window = "sun:07:00-sun:08:00"

  # The password for the master user. If var.snapshot_identifier is non-empty, this
  # value is ignored. Required unless var.replicate_source_db is set.
  master_password = null

  # The username for the master user. Required unless var.replicate_source_db is
  # set.
  master_username = null

  # Name of a Redshift parameter group to associate.
  parameter_group_name = null

  # The port the DB will listen on (e.g. 3306)
  port = 5439

  # WARNING: - In nearly all cases a database should NOT be publicly accessible.
  # Only set this to true if you want the database open to the internet.
  publicly_accessible = false

  # Determines whether a final DB snapshot is created before the DB instance is
  # deleted. Be very careful setting this to true; if you do, and you delete this DB
  # instance, you will not have any backups of the data!
  skip_final_snapshot = false

  # If non-null, the name of the cluster the source snapshot was created from.
  snapshot_cluster_identifier = null

  # If non-null, the Redshift cluster will be restored from the given Snapshot ID.
  # This is the Snapshot ID you'd find in the Redshift console, e.g:
  # rs:production-2015-06-26-06-05.
  snapshot_identifier = null

  # Required if you are restoring a snapshot you do not own, optional if you own the
  # snapshot. The AWS customer account used to create or copy the snapshot.
  snapshot_owner_account = null

  # Specifies whether the DB instance is encrypted.
  storage_encrypted = true

  # Timeout for DB updating
  updating_timeout = "75m"

}

Reference​

Required​

instance_typestringrequired

The instance type to use for the db (e.g. dc2.large)

namestringrequired

The name used to namespace all resources created by these templates, including the DB instance (e.g. drupaldb). Must be unique for this region. May contain only lowercase alphanumeric characters, hyphens.

number_of_nodesnumberrequired

The number of nodes in the cluster

subnet_idslist(string)required

A list of subnet ids where the database should be deployed. In the standard Gruntwork VPC setup, these should be the private persistence subnet ids.

vpc_idstringrequired

The id of the VPC in which this DB should be deployed.

Optional​

A list of CIDR-formatted IP address ranges that can connect to this DB. Should typically be the CIDR blocks of the private app subnet in this VPC plus the private subnet in the mgmt VPC. This is ignored if create_subnet_group=false.

[]

A list of Security Groups that can connect to this DB.

[]

Indicates whether major version upgrades (e.g. 9.4.x to 9.5.x) will ever be permitted. Note that these updates must always be manually performed and will never automatically applied.

true

A list of CIDR-formatted IP address ranges that this DB can connect. Use this if the database needs to connect to certain IP addresses for special operation

[]

Indicates that minor engine upgrades will be applied automatically to the DB instance during the maintenance window. If set to true, you should set engine_version to MAJOR.MINOR and omit the .PATCH at the end (e.g., use 5.7 and not 5.7.11); otherwise, you'll get Terraform state drift. See https://www.terraform.io/docs/providers/aws/r/db_instance.html#engine_version for more details.

true

The description of the aws_db_security_group that is created. Defaults to 'Security group for the name DB' if not specified.

null

The name of the aws_db_security_group that is created. Defaults to name if not specified.

null

How many days to keep backup snapshots around before cleaning them up. Must be 1 or greater to support read replicas.

21

The description of the cluster_subnet_group that is created. Defaults to 'Subnet group for the name DB' if not specified.

null

The name of the cluster_subnet_group that is created, or an existing one to use if cluster_subnet_group is false. Defaults to name if not specified.

null

If false, the DB will bind to aws_db_subnet_group_name and the CIDR will be ignored (allow_connections_from_cidr_blocks)

true
creating_timeoutstringoptional

Timeout for DB creating

"75m"
custom_tagsmap(string)optional

A map of custom tags to apply to the RDS Instance and the Security Group created for it. The key is the tag name and the value is the tag value.

{}
db_namestringoptional

The name for your database of up to 8 alpha-numeric characters. If you do not provide a name, Amazon RDS will not create a database in the DB cluster you are creating.

"dev"
deleting_timeoutstringoptional

Timeout for DB deleting

"40m"

If true , enhanced VPC routing is enabled. Forces COPY and UNLOAD traffic between the cluster and data repositories to go through your VPC.

false
final_snapshot_namestringoptional

The name of the final_snapshot_identifier. Defaults to name-final-snapshot if not specified.

null
iam_roleslist(string)optional

A list of IAM Role ARNs to associate with the cluster. A Maximum of 10 can be associated to the cluster at any time.

null
kms_key_arnstringoptional

The ARN of a KMS key that should be used to encrypt data on disk. Only used if storage_encrypted is true. If you leave this blank, the default RDS KMS key for the account will be used.

null
loggingobject(…)optional

Configures logging information such as queries and connection attempts for the specified Amazon Redshift cluster. If enable is set to true. The bucket_name and s3_key_prefix must be set. The bucket must be in the same region as the cluster and the cluster must have read bucket and put object permission.

object({
    enable        = bool
    bucket_name   = string
    s3_key_prefix = string
  })
{
  bucket_name = null,
  enable = false,
  s3_key_prefix = null
}
maintenance_windowstringoptional

The weekly day and time range during which system maintenance can occur (e.g. wed:04:00-wed:04:30). Time zone is UTC. Performance may be degraded or there may even be a downtime during maintenance windows.

"sun:07:00-sun:08:00"
master_passwordstringoptional

The password for the master user. If snapshot_identifier is non-empty, this value is ignored. Required unless replicate_source_db is set.

null
master_usernamestringoptional

The username for the master user. Required unless replicate_source_db is set.

null
parameter_group_namestringoptional

Name of a Redshift parameter group to associate.

null
portnumberoptional

The port the DB will listen on (e.g. 3306)

5439

WARNING: - In nearly all cases a database should NOT be publicly accessible. Only set this to true if you want the database open to the internet.

false

Determines whether a final DB snapshot is created before the DB instance is deleted. Be very careful setting this to true; if you do, and you delete this DB instance, you will not have any backups of the data!

false

If non-null, the name of the cluster the source snapshot was created from.

null
snapshot_identifierstringoptional

If non-null, the Redshift cluster will be restored from the given Snapshot ID. This is the Snapshot ID you'd find in the Redshift console, e.g: rs:production-2015-06-26-06-05.

null

Required if you are restoring a snapshot you do not own, optional if you own the snapshot. The AWS customer account used to create or copy the snapshot.

null
storage_encryptedbooloptional

Specifies whether the DB instance is encrypted.

true
updating_timeoutstringoptional

Timeout for DB updating

"75m"