Python3.9

README.md

@@ -1,35 +1,88 @@
 # terraform-aws-lambda
-terraform module for provisioning a lambda function 
+This repo contains Terraform modules to manage Lamdbas:
 
-For most scenarios I would reccomend using the "create_empty_function" argument, rather than using terraform to deploy the function code. Once all infrastructure, functions, and permisisons have been provisioned using terraform, you should use your CI/CD tooling to deploy the function code, typically with and **aws lambda update** command.
+| Directory          | Module Description                                 |
+| ------------------ | -------------------------------------------------- |
+| lambda_function/   | Lambda Function and IAM, Trigger, and CI resources |
+| lambda_layer/      | Lambda Layer and CI resources                      |
 
+These modules are primarily designed to deploy Lambda functions and layers with _placeholder_ code and then use an
+external CI/CD process to manage the function and layer code independently of Terraform. It is also possible to point
+the CI/CD process to a specific feature branch using the git_branch variable. By default, this value is set to *master*.
 
-# Arguments
-Many of the module arguments map directly to the aws_lambda_function resource arguments:
+You can optionally provide a GitHub repo containing your function or layer code and the modules will create a simple
+CodeBuild job to deploy it.
+
+## Arguments
+
+### Common
+
+| argument                  | Description                                                               | Default      |
+| ------------------------- | --------------------------------------------------------------------------| ------------ |
+| github_url                | GitHub URL of function or layer code.  Enables CodeBuild.  Assumes buildspec.yml at root of repo.  Requires github_token_ssm_param | "" |
+| codebuild_credential_arn  | AWS Codebuild source credential for accessing github                      | ""           |
+| build_timeout             | Codebuild Timeout in minutes.                                             | "60"           |
+
+### lambda_function
+Many of the module arguments map directly to the [aws_lambda_function](https://www.terraform.io/docs/providers/aws/r/lambda_function.html) resource arguments:
 * function_name 
+* git_branch
 * filename
 * description
 * runtime
 * handler
 * timeout
+* layers
 * memory_size
 * environment_variables
 * tags
 * vpc_config
 * reserved_concurrent_executions
 * publish
 
-Additional arguments are:
-* **create_empty_function** - (Required) (bool) - Create an empty lambda function without the actual code if set to true
-* **policies** - (Required) (list) - The module automatically creates a base IAM role for each lambda, This is a list of statement policies to add to that role. The contents are converted to json using the jsonencode() function.
-* **permissions** - (Optional) (list) - A list of external resources which can invoke the lambda function such as s3 bucket / sns topic. Properties are:
-  * statement_id
-  * action
-  * principal
-  * source_arn
+Additional arguments:
+
+| argument                  | Description                                                               | Default      |
+| ------------------------- | --------------------------------------------------------------------------| ------------ |
+| create_empty_function     | Create an empty lambda function without the actual code if set to true    | True         |
+| policies                  | List of statement policies to add to module-manageg Lambda IAM role role. | []           |
+| permissions               | map of external resources which can invoke the lambda function            | { enabled = false } |
+
+### lambda_layer
+Many of the module arguments map directly to the [aws_lambda_layer_version](https://www.terraform.io/docs/providers/aws/r/lambda_layer_version.html) resource arguments:
+* layer_name
+* filename
+* description
+* runtime
+
+
+Additional arguments:
 
+| argument                  | Description                                                               | Default      |
+| ------------------------- | --------------------------------------------------------------------------| ------------ |
+| create_empty_layer        | Create an empty lambda layer without the actual code if set to true       | True         |
+| codebuild_image           | Specify Codebuild's [image](https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-available.html) | "aws/codebuild/standard:1.0" |
+| privileged_mode           | Run the docker container with [privilege](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities)               | False         |
+| codebuild_can_run_integration_test | Specifies whether or not codebuild job can invoke lambda function and is passed through to the job as an env variable (run_integration_test) | False
 
-# Event trigger arguments
+# CodeBuild
+
+This module will optionally create a CodeBuild job and trigger webhook to deploy your Lambda function or layer from a
+GitHub repository.
+
+To enable creation of a CodeBuild job you must:
+    * Supply the github_url module argument
+    * Import a GitHub credential using [awscli](https://docs.aws.amazon.com/cli/latest/reference/codebuild/import-source-credentials.html)
+    or [Terraform](https://www.terraform.io/docs/providers/aws/r/codebuild_source_credential.html).
+    This credential must have admin access to your repository to create the webhook.
+
+---
+  > **_NOTE:_**  At the time of this writing, [each AWS account is limited to one GitHub CodeBuild credential](https://forums.aws.amazon.com/thread.jspa?threadID=308688&tstart=0).
+  >
+  > The module will try to construct the ARN of the CodeBuild credential as arn:aws:codebuild:<REGION_ID>:<ACCOUNT_ID>:token/github.  You can optionally override this using the module's codebuild_credential_arn argument.
+---
+
+# Function Event trigger arguments
 
 ## SNS topic trigger
 * **sns_topic_subscription** (Optional) (map) - The SNS topic ARN which trigger the lambda function`
@@ -49,9 +102,11 @@ In addition to the trigger, make sure you
  * Add sufficient permissions to the lambda role to interact with s3 (E.g s3:GetObject)
  * Add the source resource has permissions to invoke the lambda (see **permissions** argument)
 
-* **bucket_trigger** - (Optional) (map) - Configures the lambda function to trigger on s3 bucket ObjectCreated events. Has two properties:
+* **bucket_trigger** - (Optional) (map) - Configures the lambda function to trigger on s3 bucket ObjectCreated events:
     * enabled (bool) - true | false
     * bucket (string) - The bucket name only (Not the full bucket arn!)
+    * filter_prefix (string) - Only trigger for objects with this prefix (must be "" if no filter)
+    * filter_suffix (string) - Only trigger for objects with this suffix (must be "" if no filter)
 
 
 ## SQS trigger
@@ -66,3 +121,19 @@ Ensure you add the following permissions to the lambda role
   * event_source_arn (string) - arn of the event source
   * batch_size (int) - The largest number of records that Lambda will retrieve from your event source at the time of invocation
 
+
+
+[]: https://www.terraform.io/docs/providers/aws/r/lambda_function.html
+
+## Codebuild and Integration Testing
+
+If invoking this module within an environment where Integration testing makes sense as part of CI, by setting the "codebuild_can_run_integration_test" argument to true
+ * The codebuild job that accompanies lambda ci is now able to invoke the lambda function
+ * The codebuild job will know if it's appropriate to perform integration testing in the environment it's running in according to env variable "run_integration_test"
+
+For an example implementation of a lambda-codebuild job setup to conditionally run integration tests see this buildspec.yml excerpt:
+
+    if [ "$run_integration_test" = true ]; then
+          aws lambda wait function-updated --function-name $lambda_name;
+          aws lambda invoke --function-name $lambda_name --payload file://tests/testEvent.json response.json | jq -e 'has("FunctionError")|not';
+    fi

examples/lambda-s3-bucket-event-trigger/main.tf

@@ -37,6 +37,8 @@ module "lambda_s3_trigger" {
   bucket_trigger = {
     enabled = true
     bucket  = "${aws_s3_bucket.example.bucket}"
+    filter_prefix  = "images/"
+    filter_suffix  = ""
   }
 
   permissions = {

lambda_function/bucket_trigger.tf

@@ -1,9 +1,11 @@
 resource "aws_s3_bucket_notification" "bucket_notification" {
-  count  = "${var.bucket_trigger["enabled"] ? 1 : 0}"
-  bucket = "${var.bucket_trigger["bucket"]}"
+  count  = var.bucket_trigger["enabled"] ? 1 : 0
+  bucket = var.bucket_trigger["bucket"]
 
   lambda_function {
-    lambda_function_arn = "${aws_lambda_function.lambda.arn}"
+    lambda_function_arn = aws_lambda_function.lambda.arn
     events              = ["s3:ObjectCreated:*"]
+    filter_prefix       = var.bucket_trigger["filter_prefix"]
+    filter_suffix       = var.bucket_trigger["filter_suffix"]
   }
 }

lambda_function/chicken-egg.tf

@@ -1,5 +1,5 @@
 data "archive_file" "lambda_placeholder" {
-  count       = "${var.create_empty_function ? 1 : 0}"
+  count       = var.create_empty_function ? 1 : 0
   type        = "zip"
   output_path = "${path.module}/placeholder.zip"
 

lambda_function/ci.tf

@@ -0,0 +1,115 @@
+data "aws_region" "current" {}
+data "aws_caller_identity" "current" {}
+
+resource "aws_iam_role" "codebuild" {
+  count = var.github_url == "" ? 0 : 1
+
+  name = "codebuild_${var.function_name}"
+
+  assume_role_policy = <<EOF
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Principal": {
+        "Service": "codebuild.amazonaws.com"
+      },
+      "Action": "sts:AssumeRole"
+    }
+  ]
+}
+EOF
+}
+
+resource "aws_iam_role_policy" "codebuild" {
+  count = var.github_url == "" ? 0 : 1
+  role = aws_iam_role.codebuild[0].name
+  policy = data.aws_iam_policy_document.policy.json
+}
+
+data "aws_iam_policy_document" "policy" {
+  statement {
+    effect = "Allow"
+    resources = ["*"]
+    actions = [
+      "logs:CreateLogGroup",
+      "logs:CreateLogStream",
+      "logs:PutLogEvents"]
+  }
+  statement {
+    effect = "Allow"
+    resources = [
+      aws_lambda_function.lambda.arn]
+    actions = [
+      "lambda:UpdateFunctionCode",
+      "lambda:ListVersionsByFunction",
+      "lambda:UpdateAlias"
+    ]
+  }
+  dynamic "statement" {
+    for_each = var.codebuild_can_run_integration_test ? ["allow_invoke"] : []
+    content {
+      effect = "Allow"
+      resources = [aws_lambda_function.lambda.arn]
+      actions = ["lambda:InvokeFunction", "lambda:GetFunctionConfiguration"]
+    }
+  }
+
+}
+
+resource "aws_codebuild_project" "lambda" {
+  count = var.github_url == "" ? 0 : 1
+
+  name          = var.function_name
+  build_timeout = var.build_timeout
+  service_role  = aws_iam_role.codebuild[0].arn
+
+  artifacts {
+    type = "NO_ARTIFACTS"
+  }
+
+  environment {
+    compute_type                = "BUILD_GENERAL1_SMALL"
+    image                       = "aws/codebuild/standard:4.0"
+    type                        = "LINUX_CONTAINER"
+    image_pull_credentials_type = "CODEBUILD"
+    environment_variable {
+      name = "run_integration_test"
+      value = var.codebuild_can_run_integration_test
+    }
+    environment_variable {
+      name  = "lambda_function_name"
+      value = var.function_name
+    }
+  }
+
+  source {
+    type            = "GITHUB"
+    location        = var.github_url
+    git_clone_depth = 1
+
+    auth {
+      type     = "OAUTH"
+      resource = var.codebuild_credential_arn == "" ? "arn:aws:codebuild:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:token/github" : var.codebuild_credential_arn
+    }
+  }
+}
+
+resource "aws_codebuild_webhook" "lambda" {
+  count = var.github_url == "" ? 0 : 1
+
+  project_name = aws_codebuild_project.lambda[0].name
+
+  filter_group {
+    filter {
+      type    = "EVENT"
+      pattern = "PUSH"
+    }
+
+    filter {
+      type    = "HEAD_REF"
+      pattern = var.git_branch
+    }
+  }
+}

lambda_function/event_source_mapping.tf

@@ -1,8 +1,8 @@
 # Process events from SQS queue, DynamoDB, Kinesis
 resource "aws_lambda_event_source_mapping" "lambda" {
-  count            = "${length(var.source_mappings)}"
-  batch_size       = "${lookup(var.source_mappings[count.index], "batch_size")}"
-  event_source_arn = "${lookup(var.source_mappings[count.index], "event_source_arn")}"
-  enabled          = "${lookup(var.source_mappings[count.index], "enabled")}"
-  function_name    = "${aws_lambda_function.lambda.arn}"
+  count            = length(var.source_mappings)
+  batch_size       = lookup(var.source_mappings[count.index], "batch_size")
+  event_source_arn = lookup(var.source_mappings[count.index], "event_source_arn")
+  enabled          = lookup(var.source_mappings[count.index], "enabled")
+  function_name    = aws_lambda_function.lambda.arn
 }

lambda_function/iam.tf

@@ -33,9 +33,9 @@ EOF
 }
 
 resource "aws_iam_role_policy" "lambda_policy" {
-  count = "${length(var.policies) == 0 ? 0 : 1}"
+  count = length(var.policies) == 0 ? 0 : 1
   name  = "${var.function_name}-lambda-policy"
-  role  = "${aws_iam_role.lambda.id}"
+  role  = aws_iam_role.lambda.id
 
   policy = <<EOF
 {
@@ -46,6 +46,6 @@ EOF
 }
 
 resource "aws_iam_role_policy_attachment" "lambda_role_policy_attachment" {
-  role       = "${aws_iam_role.lambda.name}"
+  role       = aws_iam_role.lambda.name
   policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
 }

lambda_function/lambda_function.tf

@@ -1,29 +1,33 @@
 resource "aws_lambda_function" "lambda" {
-  function_name                  = "${var.function_name}"
-  description                    = "${var.description}"
-  role                           = "${aws_iam_role.lambda.arn}"
-  handler                        = "${var.handler}"
-  runtime                        = "${var.runtime}"
-  filename                       = "${var.create_empty_function ? "${path.module}/placeholder.zip" : var.filename}"
-  timeout                        = "${var.timeout}"
-  memory_size                    = "${var.memory_size}"
-  reserved_concurrent_executions = "${var.reserved_concurrent_executions}"
-  publish                        = "${var.publish}"
+  function_name                  = var.function_name
+  description                    = var.description
+  role                           = aws_iam_role.lambda.arn
+  handler                        = var.handler
+  runtime                        = var.runtime
+  filename                       = var.create_empty_function ? "${path.module}/placeholder.zip" : var.filename
+  timeout                        = var.timeout
+  memory_size                    = var.memory_size
+  reserved_concurrent_executions = var.reserved_concurrent_executions
+  publish                        = var.publish
+  layers                         = var.layers
 
+  ephemeral_storage {
+    size = var.ephemeral_storage
+  }
   vpc_config {
-    subnet_ids         = ["${var.vpc_config["subnet_ids"]}"]
-    security_group_ids = ["${var.vpc_config["security_group_ids"]}"]
+    subnet_ids         = var.vpc_config["subnet_ids"]
+    security_group_ids = var.vpc_config["security_group_ids"]
   }
 
   environment {
-    variables = "${var.environment_variables}"
+    variables = var.environment_variables
   }
 
-  tags = "${var.tags}"
+  tags = var.tags
 
   lifecycle {
     ignore_changes = [
-      "filename",
+      filename,
     ]
   }
 }

lambda_function/outputs.tf

@@ -1,3 +1,19 @@
-output "lambda_arn" {
-  value = "${aws_lambda_function.lambda.arn}"
+output "arn" {
+  value = aws_lambda_function.lambda.arn
+}
+
+output "role" {
+  value = aws_iam_role.lambda
+}
+
+output "function_name" {
+  value = aws_lambda_function.lambda.function_name
+}
+
+output "invoke_arn" {
+  value = aws_lambda_function.lambda.invoke_arn
+}
+
+output "codebuild_role" {
+  value = aws_iam_role.codebuild
 }