AWS Community Day CPH 2024 - Three problems of Terraform

Terragrunt, Terraspace, Terramate, terra... whatever. What is wrong with Terraform so people keep on creating wrappers and solutions around it? How OpenTofu will affect this dynamic? In this presentation, we will look into the fundamental driving forces behind a zoo of wrappers. Moreover, we are going to put together a wrapper ourselves so you can make an educated decision if you need one.

1. NORDICS
DGI Byen’s CPH Conference
2024

2. NORDICS
The three problems of Terraform
Andrey Devyatkin | 2024-05-07

3. www.fivexl.io | hello@fivexl.io

4. www.fivexl.io | hello@fivexl.io

5. www.fivexl.io | hello@fivexl.io

https://www.grc.com/sn/sn-923-notes.pdf

6. www.fivexl.io | hello@fivexl.io

Do it.
Do it better.
Do it right.
Alex Lindsay

7. www.fivexl.io | hello@fivexl.io

Andrey Devyatkin
Co-Host @ DevSecOps Talks
podcast
Principal AWS Consultant
AWS Community Builder
Security and Identity
Co-Founder @ FivexL
AWS User Group Leader
UG Las Palmas de GC

8. www.fivexl.io | hello@fivexl.io

https://www.istockphoto.com/es/foto/mano-de-gato-levantada-gm914509428-251700990

10. www.fivexl.io | hello@fivexl.io

11. www.fivexl.io | hello@fivexl.io

three
conceptual
problems
Dynamic state location
Deploying the same configuration to multiple
environments
Environment specific parameters
A way to address differences between environments
Cross-state resources lookup
A need to reference resources from different states

12. www.fivexl.io | hello@fivexl.io

Assumptions
AWS

13. www.fivexl.io | hello@fivexl.io

https://www.primevideo.com/detail/Silicon-Valley/0PHZ6LOP10TB423SCOM0BMCFMM

14. www.fivexl.io | hello@fivexl.io

15. www.fivexl.io | hello@fivexl.io

$ ls
README.md
.terraform
main.tf
terraform.tfstate

16. www.fivexl.io | hello@fivexl.io

terraform {
backend "s3" {
bucket = "my-cool-startup-infra-state"
key = "terraform/main.tfstate"
region = "us-east-1"
}
}

17. www.fivexl.io | hello@fivexl.io

Assumptions
AWS
S3 backend

18. www.fivexl.io | hello@fivexl.io

So far so good
No need for the wrapper

19. www.fivexl.io | hello@fivexl.io

https://www.amazon.co.uk/Silicon-Valley-Season-2-DVD/dp/B018I8RFZS

20. www.fivexl.io | hello@fivexl.io

We need to deploy the app to
the second environment

21. www.fivexl.io | hello@fivexl.io

Assumptions
AWS
dev/production
S3 backend

22. www.fivexl.io | hello@fivexl.io

We need to change
backend configuration
depending on env

23. www.fivexl.io | hello@fivexl.io

terraform {
backend "s3" {
bucket = "my-cool-startup-infra-state"
key = "terraform/main.tfstate"
region = "us-east-1"
}
}

24. www.fivexl.io | hello@fivexl.io

https://github.com/hashicorp/terraform/issues/17288
https://github.com/opentofu/opentofu/issues/1042

25. www.fivexl.io | hello@fivexl.io

Can we use Terraform
workspaces?

26. www.fivexl.io | hello@fivexl.io

https://developer.hashicorp.com/terraform/cli/workspaces
As of 2023-05-18

27. www.fivexl.io | hello@fivexl.io

https://developer.hashicorp.com/terraform/cli/workspaces#when-not-to-use-multiple-workspaces
As of 2023-05-18
Okay, what is the real life use case then?
??????

28. www.fivexl.io | hello@fivexl.io

https://developer.hashicorp.com/terraform/cli/workspaces#alternatives-to-workspaces
As of 2023-05-18

29. www.fivexl.io | hello@fivexl.io

30. www.fivexl.io | hello@fivexl.io

https://terragrunt.gruntwork.io/docs/features/keep-your-terragrunt-architecture-dry/

31. www.fivexl.io | hello@fivexl.io

But why so many
directories? Can’t we just
use the same directory
somehow? ??????

32. www.fivexl.io | hello@fivexl.io

https://developer.hashicorp.com/terraform/language/settings/backends/configuration#partial-configuration
As of 2023-05-18

33. www.fivexl.io | hello@fivexl.io

terraform {
backend "s3" {
bucket = "my-cool-startup-infra-state"
key = "terraform/main.tfstate"
region = "us-east-1"
}
}

34. www.fivexl.io | hello@fivexl.io

terraform {
backend "s3" {}
}
terraform init \
-backend-config "bucket=my-cool-startup-infra-state" \
-backend-config "key=terraform/main.tfstate" \
-backend-config "region=us-east-1"

35. www.fivexl.io | hello@fivexl.io

Do we share S3 bucket
between environments?

36. www.fivexl.io | hello@fivexl.io

Assumptions
AWS
dev/production
S3 backend
bucket/state per env with
predefined name

37. www.fivexl.io | hello@fivexl.io

terraform {
backend "s3" {}
}
terraform init \
-backend-config "infra-state-7YYYYYYYYY62" \
-backend-config "key=terraform/main.tfstate" \
-backend-config "region=us-east-1"

38. www.fivexl.io | hello@fivexl.io

Why did we name s3
bucket this way? ??????

39. www.fivexl.io | hello@fivexl.io


format("infra-state-%s",
data.aws_caller_identity.current.account_id)

40. www.fivexl.io | hello@fivexl.io

Would exposing the account
id get us into trouble? ??????

41. www.fivexl.io | hello@fivexl.io

As of 2024-05-07
https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-can-make-your-aws-bill-explode-934a383cb8b1

42. www.fivexl.io | hello@fivexl.io

https://www.lastweekinaws.com/blog/are-aws-account-ids-sensitive-information/

43. www.fivexl.io | hello@fivexl.io

Control Tower naming convention

44. www.fivexl.io | hello@fivexl.io

# debatable
format("infra-state-%s-%s",
data.aws_caller_identity.current.account_id,
data.aws_region.r.name)

# paranoid edition
format("infra-state-%s", sha1(
format(
"%s-%s",data.aws_caller_identity.current.account_id
data.aws_region.r.name)))

45. www.fivexl.io | hello@fivexl.io

Why not just use env
suffix like -prod or -dev?
??????

46. www.fivexl.io | hello@fivexl.io

If we are using the same dir
then how to be with .terraform?
??????

47. www.fivexl.io | hello@fivexl.io

https://developer.hashicorp.com/terraform/cli/config/environment-variables#tf_data_dir
As of 2023-05-18

48. www.fivexl.io | hello@fivexl.io

49. www.fivexl.io | hello@fivexl.io

50. www.fivexl.io | hello@fivexl.io

AWS_DEFAULT_REGION
env variable?
Use aws-vault for env setup

51. www.fivexl.io | hello@fivexl.io

$ ls
README.md
.terraform.7YYYYYYYYY62
.terraform.8XXXXXXXXX28
main.tf

52. www.fivexl.io | hello@fivexl.io

53. www.fivexl.io | hello@fivexl.io

How do we specify different
parameters for different
environments?

54. www.fivexl.io | hello@fivexl.io

https://developer.hashicorp.com/terraform/language/values/variables#assigning-values-to-root-module-variables
As of 2023-05-18

55. www.fivexl.io | hello@fivexl.io

56. www.fivexl.io | hello@fivexl.io

$ ls
README.md
7YYYYYYYYY62.tfvars
8XXXXXXXXX28.tfvars
.terraform.7YYYYYYYYY62
.terraform.8XXXXXXXXX28
main.tf

$ cat 7YYYYYYYYY62.tfvars
# dev
instance_type = "t4g.micro"

$ cat 8XXXXXXXXX28.tfvars
# prod
instance_type = "t4g.large"

57. www.fivexl.io | hello@fivexl.io

58. www.fivexl.io | hello@fivexl.io

59. www.fivexl.io | hello@fivexl.io

Do we really need a
wrapper for this?

60. www.fivexl.io | hello@fivexl.io

61. www.fivexl.io | hello@fivexl.io

What if we add more
applications?

62. www.fivexl.io | hello@fivexl.io

https://infrastructure-as-code.com/book/2018/03/28/defining-stacks.html

63. www.fivexl.io | hello@fivexl.io

https://www.youtube.com/watch?v=wgzgVm7Sqlk

64. www.fivexl.io | hello@fivexl.io

How do I get VPC id from network
stack to my application stack?

65. www.fivexl.io | hello@fivexl.io

https://developer.hashicorp.com/terraform/language/state/remote-state-data

66. www.fivexl.io | hello@fivexl.io

Terragrunt remote state resoltion

67. www.fivexl.io | hello@fivexl.io

AWS SSM
Parameters
AWS S3
Self-containing
modules
Are the other ways?
Tooling

68. www.fivexl.io | hello@fivexl.io

Self-contained
modules? ??????
Create resources
Look up resources
Provide policies
…

69. www.fivexl.io | hello@fivexl.io

70. www.fivexl.io | hello@fivexl.io

two
more
problems
Licensing
Are you competing with HashiCorp?
Team work
How do you work together with the same state?

71. www.fivexl.io | hello@fivexl.io

OpenTofu
Not a big gap at the
moment
Unclear longevity
Community
requested features
Has a momentum

72. www.fivexl.io | hello@fivexl.io

Terraform is a cli tool
That operates on a shared resources

73. www.fivexl.io | hello@fivexl.io

Conventional CI/CD
vs
TACOS
Terraform Automation and Collobaration Systems

74. www.fivexl.io | hello@fivexl.io

Commit
UnitTest
Lint
Build
Deploy Test Promote
CI/CD Server
TACOS
Server

75. www.fivexl.io | hello@fivexl.io

Benefits of using
TACOS
Lock down access to
the state
Better visibility
Less shoulders
bumping

Extra features
https://www.reddit.com/r/Terraform/comments/lkylzk/scalr_vs_spacelift_vs_atlantis_vs_env0_bake_off/

76. www.fivexl.io | hello@fivexl.io

Conclusion and
recap

77. www.fivexl.io | hello@fivexl.io

three
conceptual
problems
Dynamic state location
Deploying the same configuration to multiple
environments
Environment specific parameters
A way to address differences between environments
Cross-state resources lookup
A need to reference resources from different states

78. www.fivexl.io | hello@fivexl.io

two
more
problems
Licensing
Are you competing with HashiCorp?
Team work
How do you work together with the same state?

79. www.fivexl.io | hello@fivexl.io

80. www.fivexl.io | hello@fivexl.io

Tools like Terragrunt offer nice extra
features

Which are making going back harder

81. www.fivexl.io | hello@fivexl.io

conventions vs wrappers

82. www.fivexl.io | hello@fivexl.io

83. www.fivexl.io | hello@fivexl.io

https://www.youtube.com/channel/UCiJjTS8EiCKlbT85It_0e6g

84. Thank you
https://github.com/Andrey9kin/3-terraform-problems
https://twitter.com/andrey9kin
https://www.linkedin.com/in/andreydevyatkin/
andrey.devyatkin@fivexl.io
https://devsecops.fm/

85. NORDICS
Thank you!