Terraform S3 Having Problem with Leading Slash

Tags: May 28, 2020 12:32 PM

Problem with S3 Leading Slash in Terraform

S3 accept leading slash "/" and automatically strip them off. When we use it in Terraform as a S3 key it may looks fine until we use it in another object. See example below.

# This bucket is used to store Lambda function and layer
resource "aws_s3_bucket" "deno" {
  bucket = var.default_bucket
  acl = "private"
  tags = var.default_tags

# Upload the layer to S3
resource "aws_s3_bucket_object" "deno_func" {
  bucket = aws_s3_bucket.deno.id
  tags = var.default_tags
  # / in front "deno-custom-runtime/function.zip" below creating problem
  key = "/deno-custom-runtime/function.zip"
  source = "${path.module}/../build/function.zip"
  etag = filemd5("${path.module}/../build/function.zip")

# Deno Layer
resource "aws_lambda_layer_version" "deno" {
  layer_name = "TeknocerdasDenoRuntime"
  s3_bucket = aws_s3_bucket.deno.id
  s3_key = aws_s3_bucket_object.deno_layer.key
  s3_object_version = aws_s3_bucket_object.deno_layer.version_id
  compatible_runtimes = ["provided"]
  description = "Custom Deno runtime by TeknoCerdas.com"
  source_code_hash = filebase64sha256("${path.module}/../build/layer.zip")
When applying the resources we should get error below.
Error: Error creating lambda layer: InvalidParameterValueException: Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist.
  RespMetadata: {
    StatusCode: 400,
    RequestID: "888bed7e-5345-4d5e-ab0e-0d8c683f49b2"
  Message_: "Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist.",
  Type: "User"

Solution to S3 Leading Slash in Terraform

The solution is simply remove the leading slash from the key or filename. So instead of writing /deno-custom-runtime/function.zip use deno-custom-runtime/function.zip.

Problem solved. Simple and stupid.


Share on Facebook Twitter

How to Escape Character when Running SSH Remote Command

Tags: April 8, 2020 4:18 AM

Solution of Escaping Character on SSH

The solution is DO NOT escape it. Use cat HEREDOC to build the string of commands and then pass it to SSH.

$ cat <<EOF | ssh user@myserver bash
echo "This is complex command"
echo "Another command that take a $VARIABLE."
sudo mkdir /tmp/foo && echo "SUCCESS"

If you have a lot of $ dollar signs in your script and do not want local shell to interpret it then use single quote HEREDOC.

$ cat <<'EOF' | ssh user@myserver bash
echo "This is complex command"
echo "Another command that take a $VARIABLE name."
sudo mkdir /tmp/foo && echo "SUCCESS"

On command above the variable $VARIABLE value is "SSH VAR" since it is took the value from remote server not from local machine.

Share on Facebook Twitter

Variable Variables in Shell

Tags: March 10, 2020 8:12 PM

Subtitute Variable inside Variable in Bash

Example below is using Bash for variable variables subtitution.

$ hello="Hello World"
$ foobar="hello"
$ echo "${!foobar}"
Hello World

Subtitute Variable inside Variable using eval

This is for other shell which do not recognize "${!}" syntax. It utilise eval so use it with caution.

$ hello="Hello World"
$ foobar="hello"
$ eval echo "\$${foobar}"
Hello World

References for Variable Variables in Shell

Share on Facebook Twitter

Terraform: Force Destroy Resource when prevent_destroy is true

Tags: March 3, 2020 6:55 AM

How to Force Destroy Resource in Terraform

Terraform resource that having lifecycle prevent_destroy = true can not be destroyed. You need to manually edit the file inplace and change the value prevent_destroy to false manually each time you want to destroy the resource. Instead of having to edit manually and make git status dirty we can automate this using simple shell script.

Automate Force Destroy Resource in Terraform

The idea is simple.

  1. Search all *.tf files and look the value of prevent_default = true to prevent_default = false
  2. Run terraform destroy command
  3. Revert the changes back to prevent_default = true
Here is the implementation in Bash.

$ find . -name '*.tf' -type f \
-exec perl -i -pe 's@prevent_destroy = true@prevent_destroy = false@g' {} \;

# Run terraform destroy
[ "$IS_PLAN" = "yes" ] && terraform plan -destroy || terraform destroy $@

# Revert the changes
$ find . -name '*.tf' -type f \
 -exec perl -i -pe 's@prevent_destroy = false@prevent_destroy = true@g' {} \;
Save the file with name e.g: terraform-force-destroy.sh. To issue terraform plan -destroy command use the following.
$ IS_PLAN=yes bash terraform-force-destroy.sh
To force destroy the resource use the following command.
$ bash terraform-force-destroy.sh -auto-approve
You can give normal terraform's arguments just like the original terraform destroy.

References for Terraform Force Destroy

Share on Facebook Twitter