- * [HTTP APIs](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-metrics.html) })
- ```json title="JSON Log Format for REST API"
- {
- "accountId": "$context.accountId",
- "requestId": "$context.requestId",
- "authorizerClaimsProperty": "$context.authorizer.claims.property",
- "extendedRequestId": "$context.extendedRequestId",
- "identitySourceIp": "$context.identity.sourceIp",
- "identityCaller": "$context.identity.caller",
- "identityUser": "$context.identity.user",
- "requestTime": "$context.requestTime",
- "status": "$context.status",
- "routeKey": "$context.routeKey",
- "apiId": "$context.apiId",
- "domainPrefix": "$context.domainPrefix",
- "httpMethod": "$context.httpMethod",
- "identityClientCertSerialNumber": "$context.identity.clientCert.serialNumber",
- "identityUserAgent": "$context.identity.userAgent",
- "path": "$context.path",
- "protocol": "$context.protocol",
- "resourceId": "$context.resourceId",
- "responseOverrideStatus": "$context.responseOverride.status",
- "authorizeError": "$context.authorize.error",
- "resourcePath": "$context.resourcePath",
- "authorizeLatency": "$context.authorize.latency",
- "authorizeStatus": "$context.authorize.status",
- "authorizerError": "$context.authorizer.error",
- "authorizerIntegrationStatus": "$context.authorizer.integrationStatus",
- "authorizerIntegrationLatency": "$context.authorizer.integrationLatency",
- "authorizerLatency": "$context.authorizer.latency",
- "authorizerPrincipalId": "$context.authorizer.principalId",
- "authorizerRequestId": "$context.authorizer.requestId",
- "authorizerStatus": "$context.authorizer.status",
- "authenticateError": "$context.authenticate.error",
- "authenticateLatency": "$context.authenticate.latency",
- "authenticateStatus": "$context.authenticate.status",
- "connectedAt": "$context.connectedAt",
- "connectionId": "$context.connectionId",
- "domainName": "$context.domainName",
- "errorMessage": "$context.error.message",
- "errorResponseType": "$context.error.responseType",
- "errorValidationErrorString": "$context.error.validationErrorString",
- "eventType": "$context.eventType",
- "identityAccountId": "$context.identity.accountId",
- "identityPrincipalOrgId": "$context.identity.principalOrgId",
- "identityUserArn": "$context.identity.userArn",
- "identityApiKey": "$context.identity.apiKey",
- "identityApiKeyId": "$context.identity.apiKeyId",
- "integrationError": "$context.integration.error",
- "integrationIntegrationStatus": "$context.integration.integrationStatus",
- "integrationLatency": "$context.integration.latency",
- "integrationRequestId": "$context.integration.requestId",
- "integrationStatus": "$context.integration.status",
- "contextIntegrationLatency": "$context.integrationLatency",
- "responseLatency": "$context.responseLatency",
- "responseLength": "$context.responseLength",
- "xrayTraceId": "$context.xrayTraceId",
- "requestTimeEpoch": "$context.requestTimeEpoch",
- "stage": "$context.stage",
- "messageId": "$context.messageId",
- "wafResponseCode": "$context.wafResponseCode",
- "wafError": "$context.waf.error",
- "wafLatency": "$context.waf.latency",
- "wafStatus": "$context.waf.status",
- "webaclArn": "$context.webaclArn"
- }
- ```
-
+
+
* Enable Access logs for HTTP APIs by referring to [AWS documentation](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-logging.html#http-api-enable-logging) and when you specify the `Log format` field use the below JSON.
JSON Log Format for REST API
+ + ```json + { + "accountId": "$context.accountId", + "requestId": "$context.requestId", + "authorizerClaimsProperty": "$context.authorizer.claims.property", + "extendedRequestId": "$context.extendedRequestId", + "identitySourceIp": "$context.identity.sourceIp", + "identityCaller": "$context.identity.caller", + "identityUser": "$context.identity.user", + "requestTime": "$context.requestTime", + "status": "$context.status", + "routeKey": "$context.routeKey", + "apiId": "$context.apiId", + "domainPrefix": "$context.domainPrefix", + "httpMethod": "$context.httpMethod", + "identityClientCertSerialNumber": "$context.identity.clientCert.serialNumber", + "identityUserAgent": "$context.identity.userAgent", + "path": "$context.path", + "protocol": "$context.protocol", + "resourceId": "$context.resourceId", + "responseOverrideStatus": "$context.responseOverride.status", + "authorizeError": "$context.authorize.error", + "resourcePath": "$context.resourcePath", + "authorizeLatency": "$context.authorize.latency", + "authorizeStatus": "$context.authorize.status", + "authorizerError": "$context.authorizer.error", + "authorizerIntegrationStatus": "$context.authorizer.integrationStatus", + "authorizerIntegrationLatency": "$context.authorizer.integrationLatency", + "authorizerLatency": "$context.authorizer.latency", + "authorizerPrincipalId": "$context.authorizer.principalId", + "authorizerRequestId": "$context.authorizer.requestId", + "authorizerStatus": "$context.authorizer.status", + "authenticateError": "$context.authenticate.error", + "authenticateLatency": "$context.authenticate.latency", + "authenticateStatus": "$context.authenticate.status", + "connectedAt": "$context.connectedAt", + "connectionId": "$context.connectionId", + "domainName": "$context.domainName", + "errorMessage": "$context.error.message", + "errorResponseType": "$context.error.responseType", + "errorValidationErrorString": "$context.error.validationErrorString", + "eventType": "$context.eventType", + "identityAccountId": "$context.identity.accountId", + "identityPrincipalOrgId": "$context.identity.principalOrgId", + "identityUserArn": "$context.identity.userArn", + "identityApiKey": "$context.identity.apiKey", + "identityApiKeyId": "$context.identity.apiKeyId", + "integrationError": "$context.integration.error", + "integrationIntegrationStatus": "$context.integration.integrationStatus", + "integrationLatency": "$context.integration.latency", + "integrationRequestId": "$context.integration.requestId", + "integrationStatus": "$context.integration.status", + "contextIntegrationLatency": "$context.integrationLatency", + "responseLatency": "$context.responseLatency", + "responseLength": "$context.responseLength", + "xrayTraceId": "$context.xrayTraceId", + "requestTimeEpoch": "$context.requestTimeEpoch", + "stage": "$context.stage", + "messageId": "$context.messageId", + "wafResponseCode": "$context.wafResponseCode", + "wafError": "$context.waf.error", + "wafLatency": "$context.waf.latency", + "wafStatus": "$context.waf.status", + "webaclArn": "$context.webaclArn" + } + ``` +})
- ```json title="JSON Log Format for HTTP API"
- {
- "requestId": "$context.requestId",
- "extendedRequestId": "$context.extendedRequestId",
- "identitySourceIp": "$context.identity.sourceIp",
- "identityCaller": "$context.identity.caller",
- "identityUser": "$context.identity.user",
- "requestTime": "$context.requestTime",
- "httpMethod": "$context.httpMethod",
- "resourcePath": "$context.resourcePath",
- "status": "$context.status",
- "protocol": "$context.protocol",
- "responseLength": "$context.responseLength",
- "accountId": "$context.accountId",
- "authorizerProperty": "$context.authorizer.property",
- "routeKey": "$context.routeKey",
- "responseLatency": "$context.responseLatency",
- "integrationErrorMessage": "$context.integrationErrorMessage",
- "apiId": "$context.apiId",
- "authorizerClaimsProperty": "$context.authorizer.claims.property",
- "authorizerError": "$context.authorizer.error",
- "authorizerPrincipalId": "$context.authorizer.principalId",
- "awsEndpointRequestId": "$context.awsEndpointRequestId",
- "awsEndpointRequestId2": "$context.awsEndpointRequestId2",
- "customDomainBasePathMatched": "$context.customDomain.basePathMatched",
- "dataProcessed": "$context.dataProcessed",
- "domainName": "$context.domainName",
- "domainPrefix": "$context.domainPrefix",
- "errorMessage": "$context.error.message",
- "errorResponseType": "$context.error.responseType",
- "identityAccountId": "$context.identity.accountId",
- "identityCognitoAuthenticationProvider": "$context.identity.cognitoAuthenticationProvider",
- "identityCognitoAuthenticationType": "$context.identity.cognitoAuthenticationType",
- "identityCognitoIdentityId": "$context.identity.cognitoIdentityId",
- "identityCognitoIdentityPoolId": "$context.identity.cognitoIdentityPoolId",
- "identityPrincipalOrgId": "$context.identity.principalOrgId",
- "identityClientCertClientCertPem": "$context.identity.clientCert.clientCertPem",
- "identityClientCertSubjectDN": "$context.identity.clientCert.subjectDN",
- "identityClientCertIssuerDN": "$context.identity.clientCert.issuerDN",
- "identityClientCertSerialNumber": "$context.identity.clientCert.serialNumber",
- "identityClientCertValidityNotBefore": "$context.identity.clientCert.validity.notBefore",
- "identityClientCertValidityNotAfter": "$context.identity.clientCert.validity.notAfter",
- "identityUserAgent": "$context.identity.userAgent",
- "identityUserArn": "$context.identity.userArn",
- "integrationError": "$context.integration.error",
- "integrationIntegrationStatus": "$context.integration.integrationStatus",
- "integrationLatency": "$context.integration.latency",
- "integrationRequestId": "$context.integration.requestId",
- "integrationStatus": "$context.integration.status",
- "contextIntegrationLatency": "$context.integrationLatency",
- "contextIntegrationStatus": "$context.integrationStatus",
- "path": "$context.path",
- "requestTimeEpoch": "$context.requestTimeEpoch",
- "stage": "$context.stage"
- }
- ```
+
+
* Enable Access logs for WebSocket APIs by referring to [AWS documentation](https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-logging.html#set-up-access-logging-using-console) and when you specify the `Log format` field use the below JSON.
JSON Log Format for HTTP API
+ + ```json + { + "requestId": "$context.requestId", + "extendedRequestId": "$context.extendedRequestId", + "identitySourceIp": "$context.identity.sourceIp", + "identityCaller": "$context.identity.caller", + "identityUser": "$context.identity.user", + "requestTime": "$context.requestTime", + "httpMethod": "$context.httpMethod", + "resourcePath": "$context.resourcePath", + "status": "$context.status", + "protocol": "$context.protocol", + "responseLength": "$context.responseLength", + "accountId": "$context.accountId", + "authorizerProperty": "$context.authorizer.property", + "routeKey": "$context.routeKey", + "responseLatency": "$context.responseLatency", + "integrationErrorMessage": "$context.integrationErrorMessage", + "apiId": "$context.apiId", + "authorizerClaimsProperty": "$context.authorizer.claims.property", + "authorizerError": "$context.authorizer.error", + "authorizerPrincipalId": "$context.authorizer.principalId", + "awsEndpointRequestId": "$context.awsEndpointRequestId", + "awsEndpointRequestId2": "$context.awsEndpointRequestId2", + "customDomainBasePathMatched": "$context.customDomain.basePathMatched", + "dataProcessed": "$context.dataProcessed", + "domainName": "$context.domainName", + "domainPrefix": "$context.domainPrefix", + "errorMessage": "$context.error.message", + "errorResponseType": "$context.error.responseType", + "identityAccountId": "$context.identity.accountId", + "identityCognitoAuthenticationProvider": "$context.identity.cognitoAuthenticationProvider", + "identityCognitoAuthenticationType": "$context.identity.cognitoAuthenticationType", + "identityCognitoIdentityId": "$context.identity.cognitoIdentityId", + "identityCognitoIdentityPoolId": "$context.identity.cognitoIdentityPoolId", + "identityPrincipalOrgId": "$context.identity.principalOrgId", + "identityClientCertClientCertPem": "$context.identity.clientCert.clientCertPem", + "identityClientCertSubjectDN": "$context.identity.clientCert.subjectDN", + "identityClientCertIssuerDN": "$context.identity.clientCert.issuerDN", + "identityClientCertSerialNumber": "$context.identity.clientCert.serialNumber", + "identityClientCertValidityNotBefore": "$context.identity.clientCert.validity.notBefore", + "identityClientCertValidityNotAfter": "$context.identity.clientCert.validity.notAfter", + "identityUserAgent": "$context.identity.userAgent", + "identityUserArn": "$context.identity.userArn", + "integrationError": "$context.integration.error", + "integrationIntegrationStatus": "$context.integration.integrationStatus", + "integrationLatency": "$context.integration.latency", + "integrationRequestId": "$context.integration.requestId", + "integrationStatus": "$context.integration.status", + "contextIntegrationLatency": "$context.integrationLatency", + "contextIntegrationStatus": "$context.integrationStatus", + "path": "$context.path", + "requestTimeEpoch": "$context.requestTimeEpoch", + "stage": "$context.stage" + } + ``` +})
- ```json title="JSON Log Format for WebSocket API"
- {
- "apiId": "$context.apiId",
- "authorizeError": "$context.authorize.error",
- "authorizeLatency": "$context.authorize.latency",
- "authorizeStatus": "$context.authorize.status",
- "authorizerError": "$context.authorizer.error",
- "authorizerIntegrationLatency": "$context.authorizer.integrationLatency",
- "authorizerIntegrationStatus": "$context.authorizer.integrationStatus",
- "authorizerLatency": "$context.authorizer.latency",
- "authorizerRequestId": "$context.authorizer.requestId",
- "authorizerStatus": "$context.authorizer.status",
- "authorizerPrincipalId": "$context.authorizer.principalId",
- "authorizerProperty": "$context.authorizer.property",
- "authenticateError": "$context.authenticate.error",
- "authenticateLatency": "$context.authenticate.latency",
- "authenticateStatus": "$context.authenticate.status",
- "connectedAt": "$context.connectedAt",
- "connectionId": "$context.connectionId",
- "domainName": "$context.domainName",
- "errorMessage": "$context.error.message",
- "errorResponseType": "$context.error.responseType",
- "errorValidationErrorString": "$context.error.validationErrorString",
- "eventType": "$context.eventType",
- "extendedRequestId": "$context.extendedRequestId",
- "identityAccountId": "$context.identity.accountId",
- "identityApiKey": "$context.identity.apiKey",
- "identityApiKeyId": "$context.identity.apiKeyId",
- "identityCaller": "$context.identity.caller",
- "identityCognitoAuthenticationProvider": "$context.identity.cognitoAuthenticationProvider",
- "identityCognitoAuthenticationType": "$context.identity.cognitoAuthenticationType",
- "identityCognitoIdentityId": "$context.identity.cognitoIdentityId",
- "identityCognitoIdentityPoolId": "$context.identity.cognitoIdentityPoolId",
- "identityPrincipalOrgId": "$context.identity.principalOrgId",
- "identitySourceIp": "$context.identity.sourceIp",
- "identityUser": "$context.identity.user",
- "identityUserAgent": "$context.identity.userAgent",
- "identityUserArn": "$context.identity.userArn",
- "integrationError": "$context.integration.error",
- "integrationIntegrationStatus": "$context.integration.integrationStatus",
- "integrationLatency": "$context.integrationLatency",
- "integrationRequestId": "$context.integration.requestId",
- "integrationStatus": "$context.integration.status",
- "messageId": "$context.messageId",
- "requestId": "$context.requestId",
- "requestTime": "$context.requestTime",
- "requestTimeEpoch": "$context.requestTimeEpoch",
- "routeKey": "$context.routeKey",
- "stage": "$context.stage",
- "status": "$context.status",
- "wafError": "$context.waf.error",
- "wafLatency": "$context.waf.latency",
- "wafStatus": "$context.waf.status"
- }
- ```
+
+
5. To Export logs, refer to [Manually subscribe AWS Kinesis Firehose stream to an existing CloudWatch Log Group](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-logs-source/#manually-subscribeaws-kinesis-firehose-stream-to-an-existing-cloudwatch-log-group).
### Collect AWS API Gateway CloudTrail logs
-To your Hosted Collector, add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source.md) using the instructions below.
-
-#### Collect CloudTrail Lambda data events
+:::note
+CloudTrail data events will be collected under this source.
+:::
-To configure a CloudTrail Source, perform these steps:
+#### Prerequisites
1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
2. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html). For more information on what events are logged, refer to the [API Gateway API calls documentation](https://docs.aws.amazon.com/apigateway/latest/developerguide/cloudtrail.html).
3. Confirm that logs are being delivered to the Amazon S3 bucket.
-4. Add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source.md) to Sumo Logic. While configuring the source, add an **account** field and assign it a value that is a friendly name/alias to your AWS account from which you are collecting logs. This name will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability). Logs can be queried via the `account` field.
- JSON Log Format for WebSocket API
+ + ```json + { + "apiId": "$context.apiId", + "authorizeError": "$context.authorize.error", + "authorizeLatency": "$context.authorize.latency", + "authorizeStatus": "$context.authorize.status", + "authorizerError": "$context.authorizer.error", + "authorizerIntegrationLatency": "$context.authorizer.integrationLatency", + "authorizerIntegrationStatus": "$context.authorizer.integrationStatus", + "authorizerLatency": "$context.authorizer.latency", + "authorizerRequestId": "$context.authorizer.requestId", + "authorizerStatus": "$context.authorizer.status", + "authorizerPrincipalId": "$context.authorizer.principalId", + "authorizerProperty": "$context.authorizer.property", + "authenticateError": "$context.authenticate.error", + "authenticateLatency": "$context.authenticate.latency", + "authenticateStatus": "$context.authenticate.status", + "connectedAt": "$context.connectedAt", + "connectionId": "$context.connectionId", + "domainName": "$context.domainName", + "errorMessage": "$context.error.message", + "errorResponseType": "$context.error.responseType", + "errorValidationErrorString": "$context.error.validationErrorString", + "eventType": "$context.eventType", + "extendedRequestId": "$context.extendedRequestId", + "identityAccountId": "$context.identity.accountId", + "identityApiKey": "$context.identity.apiKey", + "identityApiKeyId": "$context.identity.apiKeyId", + "identityCaller": "$context.identity.caller", + "identityCognitoAuthenticationProvider": "$context.identity.cognitoAuthenticationProvider", + "identityCognitoAuthenticationType": "$context.identity.cognitoAuthenticationType", + "identityCognitoIdentityId": "$context.identity.cognitoIdentityId", + "identityCognitoIdentityPoolId": "$context.identity.cognitoIdentityPoolId", + "identityPrincipalOrgId": "$context.identity.principalOrgId", + "identitySourceIp": "$context.identity.sourceIp", + "identityUser": "$context.identity.user", + "identityUserAgent": "$context.identity.userAgent", + "identityUserArn": "$context.identity.userArn", + "integrationError": "$context.integration.error", + "integrationIntegrationStatus": "$context.integration.integrationStatus", + "integrationLatency": "$context.integrationLatency", + "integrationRequestId": "$context.integration.requestId", + "integrationStatus": "$context.integration.status", + "messageId": "$context.messageId", + "requestId": "$context.requestId", + "requestTime": "$context.requestTime", + "requestTimeEpoch": "$context.requestTimeEpoch", + "routeKey": "$context.routeKey", + "stage": "$context.stage", + "status": "$context.status", + "wafError": "$context.waf.error", + "wafLatency": "$context.waf.latency", + "wafStatus": "$context.waf.status" + } + ``` +})
+:::note
+Namespace for **AWS API Gateway** service is **AWS/ApiGateway**.
+:::
+
+Follow the steps below to collect logs for AWS API Gateway:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.})
+ 1. After adding fields, check their status indicators:
+ * }){kind=small}
A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * }){kind=small}
An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
### Centralized AWS CloudTrail log collection
@@ -470,7 +508,7 @@ _sourceCategory=aws/observability/cloudtrail/logs
#### Parse Expression
-Enter a parse expression to create an `account` field that maps to the alias you set for each sub account. For example, if you used the `“dev”` alias for an AWS account with ID `"528560886094"` and the `“prod”` alias for an AWS account with ID `"567680881046"`, your parse expression would look like this:
+Enter a parse expression to create an `account` field that maps to the alias you set for each sub account. For example, if you used the `dev` alias for an AWS account with ID `528560886094` and the `prod` alias for an AWS account with ID `567680881046`, your parse expression would look like this:
```sumo
| json "recipientAccountId"
@@ -489,7 +527,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
- 1. Keep in mind:
- * A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
- * An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as **Dropped**.
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
-### Collect access logs
+### Collect AWS Application Load Balancer Access logs
#### Prerequisites
@@ -86,30 +95,64 @@ Before you begin to use the AWS Elastic Load Balancing (ELB) Application app, co
2. [Enable Application Load Balancer logging](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html#enable-access-logging) in AWS.
3. Confirm that logs are being delivered to the Amazon S3 bucket.
-#### Collecting access Logs for AWS Application Load Balancer
+Follow the steps below to collect access logs for AWS Application Load Balancer:
+1. Configure the [Access Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-sources/#create-an-aws-source).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
-1. Configure a Application Load Balancing (ALB) [Access Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-sources/#create-an-aws-source).
-1. **Metadata**. Click the **+Add Field** link to add custom log metadata [fields](/docs/manage/fields). Define the fields you want to associate, each field needs a name (key) and value. The following **Fields** are to be added in the source:
- 1. Add an **account** field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- 1. Add a **region** field and assign it the value of respective AWS region where the Load Balancer exists.
- 1. Add an **accountId** field and assign it the value of the respective AWS account id which is being used.
- 1. Keep in mind:
- * A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
- * An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as **Dropped**.
+### Collect AWS Application Load Balancer CloudTrail logs
-### Collect Cloudtrail logs
+#### Prerequisites
-1. Configure a Application Load Balancing (ALB) [Cloudtrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
-1. **Metadata**. Click the **+Add Field** link to add custom log metadata [Fields](/docs/manage/fields). Define the fields you want to associate, each field needs a name (key) and value.
- 1. Add an **account** field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- 1. Keep in mind:
- * A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
- * An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as **Dropped**.
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+2. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+3. Confirm that logs are being delivered to the Amazon S3 bucket.
:::note
-Namespace for AWS Application Load Balancer Service is AWS/ApplicationELB.
+Namespace for **AWS Application Load Balancer** service is **AWS/ApplicationELB**.
:::
+Follow the steps below to collect logs for AWS Application Load Balancer:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+### Centralized AWS CloudTrail log collection
+
+In case you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following field extraction rule to map proper AWS account(s) friendly name/alias. You'll need to create it if not already present or update it as required.
+
+```sumo
+Rule Name: AWS Accounts
+Applied at: Ingest Time
+Scope (Specific Data):
+_sourceCategory=aws/observability/cloudtrail/logs
+```
+
+#### Parse Expression
+
+Enter a parse expression to create an `account` field that maps to the alias you set for each sub account. For example, if you used the `dev` alias for an AWS account with ID `528560886094` and the `prod` alias for an AWS account with ID `567680881046`, your parse expression would look like this:
+
+```sumo
+| json "recipientAccountId"
+// Manually map your aws account id with the AWS account alias you setup earlier for individual child account
+| "" as account
+| if (recipientAccountId = "528560886094", "dev", account) as account
+| if (recipientAccountId = "567680881046", "prod", account) as account
+| fields account
+```
+
## Installing the AWS Application Load Balancer app
Now that you have set up collection for AWS Application Load Balancer, install the Sumo Logic App to use the pre-configured searches and dashboards that provide visibility into your environment for real-time analysis of overall usage.
@@ -118,7 +161,9 @@ import AppInstall from '../../reuse/apps/app-install-index-apps-v2.md';
- 1. Keep in mind:
- * A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
- * An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as **Dropped**.
+Sumo Logic supports collecting metrics using one of the following source types:
-### Collect access logs
+* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (**recommended**)
+* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
-#### Prerequisites
+ :::note
+ Namespace for **AWS Classic Load Balancer** service is **AWS/ELB**.
+ :::
-Before you can begin to use the AWS Classic Load Balancing (ELB) App, complete the following steps:
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+### Collect AWS Classic Load Balancer Access logs
+
+#### Prerequisites
+
+Before you begin to use the AWS Elastic Load Balancing (ELB) Application app, complete the following steps:
1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
2. [Enable Application Load Balancer logging](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html#enable-access-logging) in AWS.
3. Confirm that logs are being delivered to the Amazon S3 bucket.
-#### Collecting access logs for AWS Classic Load Balancer
+Follow the steps below to collect access logs for AWS Classic Load Balancer:
+1. Configure the [Access Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-sources/#create-an-aws-source).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
-1. Configure a Classic Load Balancing (CLB) [Access Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-sources/#create-an-aws-source).
-1. **Metadata**. Click the **+Add Field** link to add custom log metadata [fields](/docs/manage/fields). Define the fields you want to associate, each field needs a name (key) and value. The following **Fields** are to be added in the source:
- 1. Add an **account** field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- 1. Add a **region** field and assign it the value of respective AWS region where the Load Balancer exists.
- 1. Add an **accountId** field and assign it the value of the respective AWS account id which is being used.
- 1. Keep in mind:
- * A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
- * An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as **Dropped**.
-### Collect Cloudtrail logs
+### Collect AWS Classic Load Balancer CloudTrail logs
-1. Configure a Classic Load Balancing (CLB) [Cloudtrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
-1. **Metadata**. Click the **+Add Field** link to add custom log metadata [fields](/docs/manage/fields). Define the fields you want to associate, each field needs a name (key) and value.
- 1. Add an **account** field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- 1. Keep in mind:
- * A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
- * An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as **Dropped**.
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+2. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+3. Confirm that logs are being delivered to the Amazon S3 bucket.
:::note
-Namespace for **AWS Classic Load Balancer** Service is **AWS/ELB**.
+Namespace for **AWS Classic Load Balancer** service is **AWS/ELB**.
:::
+Follow the steps below to collect logs for AWS Classic Load Balancer:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+### Centralized AWS CloudTrail log collection
+
+In case you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following field extraction rule to map proper AWS account(s) friendly name/alias. You'll need to create it if not already present or update it as required.
+
+```sumo
+Rule Name: AWS Accounts
+Applied at: Ingest Time
+Scope (Specific Data):
+_sourceCategory=aws/observability/cloudtrail/logs
+```
+
+#### Parse Expression
+
+Enter a parse expression to create an `account` field that maps to the alias you set for each sub account. For example, if you used the `dev` alias for an AWS account with ID `528560886094` and the `prod` alias for an AWS account with ID `567680881046`, your parse expression would look like this:
+
+```sumo
+| json "recipientAccountId"
+// Manually map your aws account id with the AWS account alias you setup earlier for individual child account
+| "" as account
+| if (recipientAccountId = "528560886094", "dev", account) as account
+| if (recipientAccountId = "567680881046", "prod", account) as account
+| fields account
+```
+
## Installing the AWS Classic Load Balancer app
Now that you have set up a collection for AWS Classic Load Balancer, install the Sumo Logic app to use the pre-configured searches and dashboards that provide visibility into your environment for real-time analysis of overall usage.
@@ -120,7 +163,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
+
### Sample queries
@@ -78,37 +79,54 @@ account=dev namespace=aws/dynamodb region=us-east-1 "\"eventSource\":\"dynamodb.
| limit 20
```
-### Collect Metrics for Amazon DynamoDB
-
-Sumo Logic supports collecting metrics using two source types:
-* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (Recommended); or
-* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
-
-Namespace for **Amazon DynamoDB** Service is **AWS/DynamoDB**.
+## Collect logs and metrics for Amazon DynamoDB
-* **Metadata**. Add an **account** field to the source and assign it a value that is a friendly name/alias to your AWS account from which you are collecting metrics. Metrics can be queried via the “account field”.
+### Configure Hosted Collector
+When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector).
-### Collect Amazon DynamoDB CloudTrail Logs
+### Collect Amazon DynamoDB CloudWatch metrics
-1. To your Hosted Collector, add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source.md).
- * **Name**. Enter a name to display the new Source.
- * **Description**. Enter an optional description.
- * **S3 Region**. Select the Amazon Region for your **Amazon DynamoDB** S3 bucket.
- * **Bucket Name**. Enter the exact name of your **Amazon DynamoDB** S3 bucket.
- * **Path Expression**. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (`*`) in this string. (DO NOT use a leading forward slash. See [Amazon Path Expressions](/docs/send-data/hosted-collectors/amazon-aws/amazon-path-expressions).) The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression
- * **Source Category**. Enter `aws/observability/cloudtrail/logs`
- * **Fields**. Add an **account** field and assign it a value that is a friendly name/alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- * **Access Key ID and Secret Access Key**. Enter your Amazon [Access Key ID and Secret Access Key](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html). Learn how to use Role-based access to AWS [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources)
- * **Log File Discovery -> Scan Interval**. Use the default of 5 minutes. Alternately, enter the frequency. Sumo Logic will scan your S3 bucket for new data. Learn how to configure **Log File Discovery** [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources).
- * **Enable Timestamp Parsing**. Select the **Extract timestamp information from log file entries** check box.
- * **Time Zone**. Select **Ignore time zone from the log file and instead use**, and select **UTC** from the dropdown.
- * **Timestamp Format.** Select **Automatically detect the format**.
- * **Enable Multiline Processing**. Select the **Detect messages spanning multiple lines** check box, and select **Infer Boundaries**.
-2. Click **Save**.
+Sumo Logic supports collecting metrics using one of the following source types:
+* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (**recommended**)
+* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
-### Centralized AWS CloudTrail Log Collection
+ :::note
+ Namespace for **Amazon DynamoDB** service is **AWS/DynamoDB**.
+ :::
+
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.Sample CloudTrail Log Message
```json -{ - "eventVersion":"1.05", - "userIdentity":{ - "type":"IAMUser", - "principalId":"AIDAIBF5TU7HNYUE7V676", - "arn":"arn:aws:iam::568388783903:user/ankit", - "accountId":"568388783903", - "accessKeyId":"ASIAI3Q5RU4FIZFHFJZA", - "userName":"ankit", - "sessionContext":{ - "attributes":{ - "mfaAuthenticated":"false", - "creationDate":"2017-10-10T23:01:45+0000" - } - }, - "invokedBy":"signin.amazonaws.com" - }, - "eventTime":"2017-10-10T23:01:45+0000", - "eventSource":"dynamodb.amazonaws.com", - "eventName":"DescribeTable", - "awsRegion":"us-east-1", - "sourceIPAddress":"38.99.50.98", - "userAgent":"signin.amazonaws.com", - "requestParameters":{ - "tableName":"users3" - }, - "responseElements":null, - "requestID":"AIFQQ1I27ASKDSAQ4L9L4DTQPVVV4KQNSO5AEMVJF66Q9ASUAAJG", - "eventID":"f2bec08c-a56a-4f04-be92-0cac7aaabe9b", - "eventType":"AwsApiCall", - "apiVersion":"2012-08-10", - "recipientAccountId":"568388783903" -} + { + "eventVersion":"1.05", + "userIdentity":{ + "type":"IAMUser", + "principalId":"AIDAIBF5TU7HNYUE7V676", + "arn":"arn:aws:iam::568388783903:user/ankit", + "accountId":"568388783903", + "accessKeyId":"ASIAI3Q5RU4FIZFHFJZA", + "userName":"ankit", + "sessionContext":{ + "attributes":{ + "mfaAuthenticated":"false", + "creationDate":"2017-10-10T23:01:45+0000" + } + }, + "invokedBy":"signin.amazonaws.com" + }, + "eventTime":"2017-10-10T23:01:45+0000", + "eventSource":"dynamodb.amazonaws.com", + "eventName":"DescribeTable", + "awsRegion":"us-east-1", + "sourceIPAddress":"38.99.50.98", + "userAgent":"signin.amazonaws.com", + "requestParameters":{ + "tableName":"users3" + }, + "responseElements":null, + "requestID":"AIFQQ1I27ASKDSAQ4L9L4DTQPVVV4KQNSO5AEMVJF66Q9ASUAAJG", + "eventID":"f2bec08c-a56a-4f04-be92-0cac7aaabe9b", + "eventType":"AwsApiCall", + "apiVersion":"2012-08-10", + "recipientAccountId":"568388783903" + } ``` +
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+### Collect Amazon DynamoDB CloudTrail logs
+
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+2. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+3. Confirm that logs are being delivered to the Amazon S3 bucket.
+
+:::note
+Namespace for **Amazon DynamoDB** service is **AWS/DynamoDB**.
+:::
+
+Follow the steps below to collect logs for Amazon DynamoDB:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+### Centralized AWS CloudTrail log collection
In case you have a centralized collection of CloudTraillogs and are ingesting them from all accounts into a single Sumo Logic CloudTraillog source, create following Field Extraction Rule to map proper AWS account(s) friendly name/alias. Create it if not already present / update it as required.
```sql
@@ -118,7 +136,7 @@ Scope (Specific Data):
_sourceCategory=aws/observability/cloudtrail/logs
```
-**Parse Expression**
+#### Parse Expression
Enter a parse expression to create an “account” field that maps to the alias you set for each sub-account. For example, if you used the `“dev”` alias for an AWS account with ID `"528560886094"` and the `“prod”` alias for an AWS account with ID `"567680881046"`, your parse expression would look like this:
```sumo
@@ -138,7 +156,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
+
### Sample queries
@@ -101,56 +103,62 @@ account={{account}} region={{region}} namespace={{namespace}} eventname eventsou
| count as count by error_code | sort by count, error_code asc | limit 10
```
+## Collecting logs and metrics for AWS EC2
-### AWS EC2 CloudWatch Metrics
+### Configure Hosted Collector
-AWS EC2 automatically monitors functions on your behalf, reporting [AWS EC2 metrics](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch.html) through Amazon CloudWatch. These metrics are collected by our Hosted Collector by configuring the Amazon CloudWatch source.
+When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector).
-The Sumo Logic app for AWS EC2 (CloudWatch Metrics) allows you to collect your EC2 instance metrics and display them using predefined dashboards. The app provides dashboards to analyze EC2 instance metrics for CPU, disk, network, EBS, and Health Status Check.
+### Collect AWS EC2 CloudWatch metrics
+Sumo Logic supports collecting metrics using one of the following source types:
-### CloudTrail EC2 Data Events
+* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (**recommended**)
+* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
-[CloudTrail EC2 Data Events](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-management-and-data-events-with-cloudtrail.html#logging-data-events) allow you to continuously monitor the execution activity of your EC2 instance and record details of all the related events.
+ :::note
+ Namespace for **Amazon EC2** service is **AWS/EC2**.
+ :::
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.Sample CloudTrail Log
+ +```json { "eventVersion":"1.08", "userIdentity":{ @@ -68,6 +69,7 @@ For details on the metrics of AWS EC2, see [here](https://docs.aws.amazon.com/AW } } ``` +
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
-### Collect Amazon CloudWatch EC2 Metrics
+AWS EC2 automatically monitors functions on your behalf, reporting [AWS EC2 metrics](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch.html) through Amazon CloudWatch. These metrics are collected by our Hosted Collector by configuring the Amazon CloudWatch source.
-Sumo Logic supports collecting metrics using two source types:
-* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) **(recommended)** or
-* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
-:::note
-Namespace for **Amazon EC2** Service is **AWS/EC2**.
-:::
+The Sumo Logic app for AWS EC2 (CloudWatch Metrics) allows you to collect your EC2 instance metrics and display them using predefined dashboards. The app provides dashboards to analyze EC2 instance metrics for CPU, disk, network, EBS, and Health Status Check.
-* **Metadata**: Add an **account** field to the source and assign it a value which is a friendly name / alias to your AWS account from which you are collecting metrics. Metrics can be queried through the **account** field.
-})
+### Collect AWS EC2 CloudTrail logs
-### Collect CloudTrail EC2 Data Events
+:::note
+CloudTrail data events will be collected under this source.
+:::
-To configure a CloudTrail Source, perform these steps:
+#### Prerequisites
1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
-2. [Configure DataEvents with CloudTrail](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/using-cloudtrail.html) in your AWS account.
+2. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
3. Confirm that logs are being delivered to the Amazon S3 bucket.
-4. Add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source) to Sumo Logic.
- 1. **Name**. Enter a name to display the new Source.
- 2. **Description**. You may skip the description as it's optional.
- 3. **S3 Region**. Select the Amazon Region for your API Gateway S3 bucket.
- 4. **Bucket Name**. Enter the exact name of your API Gateway S3 bucket.
- 5. **Path Expression**. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard `*` in this string.
- :::note
- DO NOT use a leading forward slash. See [Amazon Path Expressions](/docs/send-data/hosted-collectors/amazon-aws/amazon-path-expressions). The S3 bucket name is not part of the path. Don’t include the S3 bucket name when you are setting the Path Expression.
- :::
-5. **Source Category**. Enter `aws/observability/cloud trail/logs`.
-6. **Fields**. Add an **account** field and assign it a value that is a friendly name/alias to your AWS account from which you are collecting logs. Logs can be queried through the **account** field. })
-7. **Access Key ID and Secret Access Key**. Enter your [Access Key ID and Secret Access Key](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html). Learn how to use Role-based access to AWS [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources).
-8. **Log File Discovery -> Scan Interval**. Use the default of 5 minutes. Alternately, enter the frequency. Sumo Logic will scan your S3 bucket for new data. Learn how to configure Log File Discovery [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources).
-9. **Enable Timestamp Parsing**. Select the **Extract timestamp information from log file entries** check box.
-10. **Time Zone**. Select **Ignore time zone from the log file and instead use**, and select **UTC** from the dropdown.
-11. **Timestamp Format.** Select **Automatically detect the format**.
-12. **Enable Multiline Processing**. Select the **Detect messages spanning multiple lines** check box, and select **Infer Boundaries**.
-13. Click **Save**.
+
+:::note
+Namespace for **Amazon EC2** service is **AWS/EC2**.
+:::
+
+Follow the steps below to collect logs for AWS API Gateway:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
### Centralized AWS CloudTrail log collection
@@ -162,8 +170,7 @@ Applied at: Ingest Time
Scope (Specific Data): _sourceCategory=[**Classic UI**](/docs/get-started/sumo-logic-ui-classic). In the main Sumo Logic menu, select **Manage Data > Logs > Fields**. -1. Search for the `instanceid` field. -1. If not present, create it. Learn how to create and manage fields [here](/docs/manage/fields#manage-fields). - -
})
+Host metrics are gathered by the open-source [SIGAR library](https://github.com/hyperic/sigar). The metrics that are collected are described in [Host Metrics for Installed Collectors](/docs/send-data/installed-collectors/sources/host-metrics-source#collected-metrics).
+The Host Metrics (EC2) app relies upon an Installed Collector with a [Host Metrics Source](/docs/send-data/installed-collectors/sources/host-metrics-source) on each of your AWS EC2 hosts. This section describes the data sources for the Host Metrics (EC2) app and has instructions for setting up metric collection.
### Configure Host Metrics sources
@@ -59,7 +47,7 @@ Perform these steps for each EC2 host:
* Add a field named **account**, and set it to your AWS account alias.
* Add a field named **namespace** and set it to **aws/ec2**.
-})
+
3. Set the **Scan Interval** (the frequency at which the Source is scanned) to 1 minute.
@@ -67,6 +55,8 @@ A default Scan Interval of 1 minute is recommended. You can set it to a higher o
You can also build your EC2 AMI machine image with these fields and settings. For instructions, see [this blog](https://www.sumologic.com/blog/packer-and-sumo-logic). Here’s a sample sources.json file that you can include in your AMI.
+
+
-
-### AWS Metadata
+#### AWS metadata
Collectors running on AWS EC2 instances can optionally collect AWS Metadata such as EC2 tags to make it easier to search for Host Metrics. Only one AWS Metadata Source for Metrics is required to collect EC2 tags from multiple hosts. For more information, see [AWS Metadata Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/aws-metadata-tag-source).
-
## Install the Host Metrics (EC2) App
Now that you have set up the collection for Host Metrics (EC2) metrics, install the Sumo Logic App to use the pre-configured searches and dashboards that provide visibility into your environment for real-time analysis of overall usage.
@@ -125,7 +114,13 @@ import AppInstall from '../../reuse/apps/app-install.md';
Click to expand
```json { @@ -110,13 +100,12 @@ You can also build your EC2 AMI machine image with these fields and settings. Fo } } ``` +
- 1. Keep in mind:
- * A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
- * An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as **Dropped**.
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
-### Collect ECS events using CloudTrail
+### Collect Amazon ECS CloudTrail logs
-1. Configure a [AWS CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
-1. **Metadata**. Click the **+Add Field** link to add custom log metadata [Fields](/docs/manage/fields). Define the fields you want to associate, each field needs a name (key) and value.
- 1. Add an **account** field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- 1. Keep in mind:
- * A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
- * An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as **Dropped**.
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+2. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+3. Confirm that logs are being delivered to the Amazon S3 bucket.
+ :::note
+ Namespace for **Amazon ECS** service is **AWS/ECS**.
+ :::
+
+Follow the steps below to collect logs for Amazon ECS:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+### Centralized AWS CloudTrail log collection
+
+In case you have a centralized collection of CloudTraillogs and are ingesting them from all accounts into a single Sumo Logic CloudTraillog source, create following Field Extraction Rule to map proper AWS account(s) friendly name/alias. Create it if not already present / update it as required.
+```sql
+Rule Name: AWS Accounts
+Applied at: Ingest Time
+Scope (Specific Data):
+_sourceCategory=aws/observability/cloudtrail/logs
+```
+
+#### Parse Expression
+
+Enter a parse expression to create an “account” field that maps to the alias you set for each sub-account. For example, if you used the `“dev”` alias for an AWS account with ID `"528560886094"` and the `“prod”` alias for an AWS account with ID `"567680881046"`, your parse expression would look like this:
+```sumo
+| json "recipientAccountId"
+// Manually map your aws account id with the AWS account alias you setup earlier for individual child account
+| "" as account
+| if (recipientAccountId = "528560886094", "dev", account) as account
+| if (recipientAccountId = "567680881046", "prod", account) as account
+| fields account
+```
## Installing the Amazon ECS app
@@ -325,7 +368,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
+
### Sample queries
@@ -105,35 +107,56 @@ account={{account}} region={{region}} namespace={{namespace}} "\"eventSource\":\
| sort by _timeslice
```
-## Collect Logs and Metrics for Amazon ElastiCache
+## Collecting logs and metrics for Amazon ElastiCache
+
+### Configure Hosted Collector
+
+When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector).
+
+### Collect Amazon ElastiCache CloudWatch metrics
+
+Sumo Logic supports collecting metrics using one of the following source types:
-* Sumo Logic supports collecting metrics using two source types:
- * Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (recommended); or
- * Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
- * Namespace for **Amazon ElastiCache** service is **AWS/ElastiCache**
- * **Metadata**: Add an **account** field to the source and assign it a value which is a friendly name / alias to your AWS account from which you are collecting metrics. Metrics can be queried via the “account field”.
+* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (**recommended**)
+* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
+ :::note
+ Namespace for **Amazon ElastiCache** service is **AWS/ElastiCache**.
+ :::
-### Collect Amazon ElastiCache CloudTrail Logs
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.Sample CloudTrail Log Message
+ +```json { "eventVersion":"1.05", "userIdentity":{ @@ -72,7 +74,7 @@ The Amazon ElastiCache app uses the following logs and metrics: "recipientAccountId":"123456789038" } ``` - +
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
-1. To your Hosted Collector, add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source.md).
- * **Name**. Enter a name to display for the new Source.
- * **Description**. Enter an optional description.
- * **S3 Region**. Select the Amazon Region for your **ElastiCache** S3 bucket.
- * **Bucket Name**. Enter the exact name of your **ElastiCache** S3 bucket.
- * **Path Expression**. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string. (DO NOT use a leading forward slash. See [Amazon Path Expressions](/docs/send-data/hosted-collectors/amazon-aws/amazon-path-expressions).) The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression.
- * **Source Category**. Enter aws/observability/cloudtrail/logs
- * **Fields**. Add an **account** field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- * **Access Key ID and Secret Access Key**. Enter your Amazon [Access Key ID and Secret Access Key](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html). Learn how to use Role-based access to AWS [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources)
- * **Log File Discovery -> Scan Interval**. Use the default of 5 minutes. Alternately, enter the frequency. Sumo Logic will scan your S3 bucket for new data. Learn how to configure **Log File Discovery** [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources).
- * **Enable Timestamp Parsing**. Select the **Extract timestamp information from log file entries** check box.
- * **Time Zone**. Select **Ignore time zone from the log file and instead use**, and select **UTC** from the dropdown.
- * **Timestamp Format.** Select **Automatically detect the format**.
- * **Enable Multiline Processing**. Select the **Detect messages spanning multiple lines** check box, and select **Infer Boundaries**.
-2. Click **Save**.
+### Collect Amazon ElastiCache CloudTrail logs
+#### Prerequisites
-### Centralized AWS CloudTrail Log Collection
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+2. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html). For more information on what events are logged, refer to the [ElastiCache API calls documentation](https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/logging-using-cloudtrail.html).
+3. Confirm that logs are being delivered to the Amazon S3 bucket.
+
+:::note
+Namespace for **Amazon ElastiCache** service is **AWS/ElastiCache**.
+:::
+
+Follow the steps below to collect logs for Amazon ElastiCache:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+### Centralized AWS CloudTrail log collection
In case you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following Field Extraction Rule to map a proper AWS account(s) friendly name / alias. Create it if not already present / update it as required.
@@ -143,8 +166,9 @@ Applied at: Ingest Time
Scope (Specific Data): _sourceCategory=aws/observability/cloudtrail/logs
```
-**Parse Expression**. Enter a parse expression to create an “account” field that maps to the alias you set for each sub account. For example, if you used the `“dev”` alias for an AWS account with ID `"528560886094"` and the `“prod”` alias for an AWS account with ID `"567680881046"`, your parse expression would look like:
+#### Parse Expression
+Enter a parse expression to create an `account` field that maps to the alias you set for each sub account. For example, if you used the `dev` alias for an AWS account with ID `528560886094` and the `prod` alias for an AWS account with ID `567680881046`, your parse expression would look like:
```sumo
| json "recipientAccountId"
@@ -155,7 +179,6 @@ Scope (Specific Data): _sourceCategory=aws/observability/cloudtrail/logs
| fields account
```
-
## Installing the Amazon ElastiCache app
This section has instructions for installing the Sumo Logic app for **Amazon ElastiCache** and descriptions of each of the app dashboards along with associated use cases.
@@ -166,7 +189,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
})
-### CloudTrail Lambda Data Events
+### AWS Lambda CloudTrail logs
[CloudTrail Lambda Data Events](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-management-and-data-events-with-cloudtrail.html#logging-data-events) allow you to continuously monitor the execution activity of your Lambda functions and to record details on when and by whom an Invoke API call was made.
@@ -37,7 +37,7 @@ The Sumo Logic App for AWS Lambda provide insights into the Lambda Functions inv
})
-### AWS Lambda CloudWatch Metrics
+### AWS Lambda CloudWatch metrics
AWS Lambda automatically monitors functions on your behalf, reporting [AWS Lambda metrics](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-functions-metrics.html) through Amazon CloudWatch. These metrics are collected by our Hosted Collector by configuring the Amazon CloudWatch source.
@@ -46,7 +46,10 @@ The Sumo Logic App for AWS Lambda provide insights into the Lambda Functions inv
### Sample log messages
This section provides sample Amazon CloudWatch Logs and CloudTrail Lambda Data Events log messages.
-```json title="Amazon CloudWatch Log"
+
+
+
+Amazon CloudWatch Log
+ +```json { "id":"32563142671071560797760688825700039436306340248688066573", "timestamp":1511808906799, @@ -56,8 +59,12 @@ This section provides sample Amazon CloudWatch Logs and CloudTrail Lambda Data E "logGroup":"/aws/lambda/AWSlambda1" } ``` +
+
### Sample queries
@@ -137,81 +144,101 @@ account={{account}} region={{region}} Namespace={{namespace}}
namespace=aws/lambda metric=Errors statistic=Sum account=* region=* functionname=* Resource=* | sum
```
-## Collecting logs for the AWS Lambda App
+## Collecting logs and metrics for AWS Lambda
-This section provides instructions for setting up log and metric collection.
+### Configure Hosted Collector
-### Collect Amazon CloudWatch Logs
+When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector).
-Sumo supports several methods for collecting Lambda logs from Amazon CloudWatch. You can choose any of them to collect logs.
-- **AWS Kinesis Firehose for Logs**. Configure an [AWS Kinesis Firehose for Logs](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-logs-source/#create-an-aws-kinesis-firehose-for-logssource) (Recommended).
-- **Lambda Log Forwarder**. Configure a collection of Amazon CloudWatch Logs using our AWS Lambda function using a Sumo Logic provided CloudFormation template, as described in [Amazon CloudWatch Logs](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/) or configure collection without using CloudFormation, see [Collect Amazon CloudWatch Logs using a Lambda Function](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/collect-with-lambda-function/).CloudTrail Lambda Data Events
-```json title="CloudTrail Lambda Data Events" +```json { "eventVersion":"1.06", "userIdentity":{ @@ -98,7 +105,7 @@ This section provides sample Amazon CloudWatch Logs and CloudTrail Lambda Data E "recipientAccountId":"111111111111" } ``` - ++### Collect AWS Lambda CloudWatch logs -* While configuring the CloudWatch log source, the following Fields can be added in the source: - * Add an **account** field and assign it a value that is a friendly name/alias to your AWS account from which you are collecting logs. Logs can be queried via the **account** field. - * Add a **region** field and assign it the value of the respective AWS region where the Lambda function exists. - * Add an **accountId** field and assign it the value of the respective AWS account ID being used. +Sumo Logic supports collecting Lambda logs from Amazon CloudWatch using one of the following methods: +- **AWS Kinesis Firehose for Logs**. Configure an [AWS Kinesis Firehose for Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-logs-source/#create-an-aws-kinesis-firehose-for-logssource) (**Recommended**) +- **Lambda Log Forwarder**. There are two ways to set up the Lambda Log Forwarder: + - **With CloudFormation**. Configure the collection of Amazon CloudWatch logs using Sumo Logic-provided CloudFormation template, as described in [Amazon CloudWatch Logs](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/). + - **Without CloudFormation**. Configure the collection of Amazon CloudWatch Logs using a Lambda function, as described in [Collect Amazon CloudWatch Logs using a Lambda Function](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/collect-with-lambda-function/). -
})
+Follow the steps below to add custom fields when configuring the CloudWatch log source:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+1. Add a `region` field and assign it the value of the respective AWS region where the Lambda function exists.
+1. Add an `accountId` field and assign it the value of the respective AWS account ID being used.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
-### Collect CloudTrail Lambda Data Events
+### Collect AWS Lambda CloudTrail logs
+
+:::note
+CloudTrail data events will be collected under this source.
+:::
-To configure a CloudTrail Source, perform these steps:
+#### Prerequisites
1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
2. [Configure DataEvents with CloudTrail](https://docs.aws.amazon.com/lambda/latest/dg/logging-using-cloudtrail.html) in your AWS account.
3. Confirm that logs are being delivered to the Amazon S3 bucket.
-4. Add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source.md) to Sumo Logic.
-5. While configuring the cloud trail log source, the following field can be added to the source:
- * Add an **account** field and assign it a value that is a friendly name/alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
-
+Follow the steps below to collect logs for AWS Lambda:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source.md).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+### Collect AWS Lambda CloudWatch metrics
-### Collect Amazon CloudWatch Metrics
+Sumo Logic supports collecting metrics using one of the following source types:
-Sumo Logic supports collecting metrics using two source types:
-
-* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (Recommended)
- or
+* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (**recommended**)
* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
-:::note
-Namespace for **Amazon Lambda** Service is **AWS/Lambda**.
-:::
-
-* **Metadata**. Add an **account** field to the source and assign it a value that is a friendly name/alias to your AWS account from which you are collecting metrics. The **account** field allows you to query metrics.
+ :::note
+ Namespace for **AWS ElastiCache** service is **AWS/ElastiCache**.
+ :::
-
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
Continue with the process of [enabling Provisioned Concurrency configurations](#enable-provisioned-concurrency-configurations-for-lambda-functions) for Lambda functions, as needed.
-
### Enable Provisioned Concurrency configurations for Lambda functions
AWS Lambda provides Provisioned Concurrency for greater control over the start-up time for Lambda functions. When enabled, [Provisioned Concurrency](https://docs.aws.amazon.com/lambda/latest/dg/provisioned-concurrency.html) keeps functions initialized and hyper-ready to respond in double-digit milliseconds. AWS Lambda provides additional metrics for provisioned concurrency with CloudWatch.
To collect the metrics in Sumo Logic, follow the steps below:
-1. Jump to the [Collect Amazon CloudWatch Metrics](#collect-amazon-cloudwatch-metrics) section and complete the steps as described.
+1. Jump to the [Collect Amazon CloudWatch Metrics](#collect-aws-lambda-cloudwatch-metrics) section and complete the steps as described.
2. Configure Provisioned Concurrency while creating a Lambda function in the AWS Management Console, as shown in the following example.
})
Once Provisioned Concurrency is enabled and you start collecting CloudWatch metrics, the following new metrics will be available:
-| Metric | Description |
-|:-----------|:-------------------------|
-| **ProvisionedConcurrentExecutions** | Concurrent Executions using Provisioned Concurrency |
-| **ProvisionedConcurrencyUtilization** | Fraction of Provisioned Concurrency in use |
-| **ProvisionedConcurrencyInvocations** | Number of Invocations using Provisioned Concurrency |
+| Metric | Description |
+|:--|:--|
+| **ProvisionedConcurrentExecutions** | Concurrent Executions using Provisioned Concurrency |
+| **ProvisionedConcurrencyUtilization** | Fraction of Provisioned Concurrency in use |
+| **ProvisionedConcurrencyInvocations** | Number of Invocations using Provisioned Concurrency |
| **ProvisionedConcurrencySpilloverInvocations** | Number of Invocations that are above Provisioned Concurrency |
These metrics can then be queried using Sumo Logic [Metrics queries](/docs/metrics/metrics-queries), as shown in the following example:
})
-### Centralized AWS CloudTrail Log Collection
+### Centralized AWS CloudTrail Log collection
In case you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following Field Extraction Rule to map the proper AWS account(s) friendly name/alias. Create it if not already present / update it as required.
@@ -221,9 +248,9 @@ Applied at: Ingest Time
Scope (Specific Data): _sourceCategory=
+
### Sample queries
@@ -22,33 +85,95 @@ The AWS Network Load Balancer app uses AWS Network Load Balancer metrics.
account=* region=* LoadBalancer=* Namespace=aws/NetworkELB metric=ActiveFlowCount Statistic=Sum | sum by account, region, namespace, LoadBalancer
```
-## Collecting logs and metrics for the AWS Network Load Balancer
+```sql title="Successful Events Details"
+account=* region=* "\"eventsource\":\"elasticloadbalancing.amazonaws.com\"" "2015-12-01"
+| json "userIdentity", "eventSource", "eventName", "awsRegion", "sourceIPAddress", "userAgent", "eventType", "recipientAccountId", "requestParameters", "responseElements", "requestID", "errorCode", "errorMessage", "apiVersion" as userIdentity, event_source, event_name, region, src_ip, user_agent, event_type, recipient_account_id, requestParameters, responseElements, request_id, error_code, error_message, api_version nodrop
+| where event_source = "elasticloadbalancing.amazonaws.com" and api_version matches "2015-12-01"
+| where namespace matches "aws/networkelb" or isEmpty(namespace)
+| json field=userIdentity "accountId", "type", "arn", "userName" as accountid, type, arn, username nodrop
+| parse field=arn ":assumed-role/*" as user nodrop
+| parse field=arn "arn:aws:iam::*:*" as accountid, user nodrop
+| json field=requestParameters "name" as networkloadbalancer nodrop
+| if (isBlank(accountid), recipient_account_id, accountid) as accountid
+| where (tolowercase(networkloadbalancer) matches tolowercase("*")) or isBlank(networkloadbalancer)
+| if (isEmpty(error_code), "Success", "Failure") as event_status
+| where event_status= "Success"
+| if (isEmpty(username), user, username) as user
+| count as event_count by event_name
+| sort by event_count, event_name asc
+```
+
+## Collecting logs and metrics for AWS Network Load Balancer
+
+### Configure Hosted Collector
When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector).
-### Collect metrics
+### Collect AWS Network Load Balancer CloudWatch metrics
-1. Sumo Logic supports collecting metrics using two source types:
- * Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (recommended); or
- * Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
-1. **Metadata**. Click the **+Add Field** link to add custom log metadata [fields](/docs/manage/fields). Define the fields you want to associate, each field needs a name (key) and value.
- 1. Add an **account** field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.Sample CloudTrail Log Message
+ +```json +{ + "eventVersion": "1.11", + "userIdentity": { + "type": "AssumedRole", + "principalId": "AROATIK2E7SUFL6GB4G44:1782467664281479421", + "arn": "arn:aws:sts::224064240808:assumed-role/pdet-eks-irsa-prod-aws-lb-controller/1782467664281479421", + "accountId": "224064240808", + "accessKeyId": "ASIATIK2E7SUH6GUXFK4", + "sessionContext": { + "sessionIssuer": { + "type": "Role", + "principalId": "AROATIK2E7SUFL6GB4G44", + "arn": "arn:aws:iam::224064240808:role/pdet-eks-irsa-prod-aws-lb-controller", + "accountId": "224064240808", + "userName": "pdet-eks-irsa-prod-aws-lb-controller" + }, + "webIdFederationData": { + "federatedProvider": "arn:aws:iam::224064240808:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/0499F131BE8B24AAE70BF8AD8EB16D3A", + "attributes": {} + }, + "attributes": { + "creationDate": "2026-06-26T09:54:24Z", + "mfaAuthenticated": "false" + } + } + }, + "eventTime": "2026-06-26T09:54:25Z", + "eventSource": "elasticloadbalancing.amazonaws.com", + "eventName": "DescribeLoadBalancers", + "awsRegion": "us-west-2", + "sourceIPAddress": "44.241.82.204", + "userAgent": "aws-sdk-go-v2/1.36.3 ua/2.1 os/linux lang/go#1.24.5 md/GOOS#linux md/GOARCH#amd64 api/elasticloadbalancingv2#1.45.0 elbv2.k8s.aws/v2.13.4 m/C,E", + "requestParameters": { + "loadBalancerArns": [ + "arn:aws:elasticloadbalancing:us-west-2:224064240808:loadbalancer/net/k8s-gloosyst-gatewayp-9e3a2f18b7/262e2df5d81d69e3" + ] + }, + "responseElements": null, + "requestID": "b231b530-2877-467d-9a0b-eb9b0fed0f39", + "eventID": "7800ac19-806e-434e-b2b0-aec11ad7d312", + "readOnly": true, + "eventType": "AwsApiCall", + "apiVersion": "2015-12-01", + "managementEvent": true, + "recipientAccountId": "224064240808", + "eventCategory": "Management", + "tlsDetails": { + "tlsVersion": "TLSv1.3", + "cipherSuite": "TLS_AES_128_GCM_SHA256", + "clientProvidedHostHeader": "elasticloadbalancing.us-west-2.amazonaws.com" + } +} +``` +
- 1. Keep in mind:
- * A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
- * An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as **Dropped**.
+Sumo Logic supports collecting metrics using one of the following source types:
-### Collect Cloudtrail logs
+* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (**recommended**)
+* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
-1. Configure a Network Load Balancing (NLB) [Cloudtrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
-1. **Metadata**. Click the **+Add Field** link to add custom log metadata [fields](/docs/manage/fields). Define the fields you want to associate, each field needs a name (key) and value.
- 1. Add an **account** field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- 1. Keep in mind:
- * A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
- * An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as **Dropped**.
+ :::note
+ Namespace for **AWS Network Load Balancer** service is **AWS/NetworkELB**.
+ :::
-:::note
-Namespace for AWS Network Load Balancer Service is AWS/NetworkELB.
-:::
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+### Collect AWS Network Load Balancer CloudTrail logs
+
+#### Prerequisites
+
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+1. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+1. Confirm that logs are being delivered to the Amazon S3 bucket.
+
+ :::note
+ Namespace for **AWS Network Load Balancer** service is **AWS/NetworkELB**.
+ :::
+
+Follow the steps below to collect logs for AWS Network Load Balancer (NLB):
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+### Centralized AWS CloudTrail log collection
+
+In case you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following Field Extraction Rule to map a proper AWS account(s) friendly name / alias. Create it if not already present / update it as required.
+
+```sql
+Rule Name: AWS Accounts
+Applied at: Ingest Time
+Scope (Specific Data): _sourceCategory=aws/observability/cloudtrail/logs
+```
+
+#### Parse Expression
+
+Enter a parse expression to create an `account` field that maps to the alias you set for each sub account. For example, if you used the `dev` alias for an AWS account with ID `528560886094` and the `prod` alias for an AWS account with ID `567680881046`, your parse expression would look like:
+
+```sumo
+| json "recipientAccountId"
+// Manually map your aws account id with the AWS account alias you setup earlier for individual child account
+| "" as account
+| if (recipientAccountId = "528560886094", "dev", account) as account
+| if (recipientAccountId = "567680881046", "prod", account) as account
+| fields account
+```
## Installing the AWS Network Load Balancer app
@@ -58,7 +183,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
-
Click to expand
+Sample CloudTrail Log Message
-```json title="CloudTrail" +```json { "eventVersion":"1.05", "userIdentity": @@ -128,9 +129,9 @@ The Amazon RDS app uses the following logs and metrics: ### Sample Database CloudWatch logs
-
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
### Collect Amazon RDS CloudTrail logs
-1. Add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source.md) to your Hosted Collector.
- * **Name**. Enter a name to display the new Source.
- * **Description**. Enter an optional description.
- * **S3 Region**. Select the Amazon Region for your **Amazon RDS** S3 bucket.
- * **Bucket Name**. Enter the exact name of your **Amazon RDS** S3 bucket.
- * **Path Expression**. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string. (DO NOT use a leading forward slash. See [Amazon Path Expressions](/docs/send-data/hosted-collectors/amazon-aws/amazon-path-expressions)). The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression
- * **Source Category**. Enter `aws/observability/cloudtrail/logs`.
- * **Fields**. Add an **account** field and assign it a value that is a friendly name/alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- * **Access Key ID and Secret Access Key**. Enter your Amazon [Access Key ID and Secret Access Key](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html). Learn how to use Role-based access to AWS [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources)
- * **Log File Discovery** > **Scan Interval**. Use the default of 5 minutes. Alternately, enter the frequency. Sumo Logic will scan your S3 bucket for new data. Learn how to configure **Log File Discovery** [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources).
- * **Enable Timestamp Parsing**. Select the **Extract timestamp information from log file entries** check box.
- * **Time Zone**. Select **Ignore time zone from the log file and instead use**, and select **UTC** from the dropdown.
- * **Timestamp Format.** Select **Automatically detect the format**.
- * **Enable Multiline Processing**. Select the **Detect messages spanning multiple lines** check box, and select **Infer Boundaries**.
-2. Click **Save**.
+#### Prerequisites
+
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+1. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+1. Confirm that logs are being delivered to the Amazon S3 bucket.
+
+ :::note
+ Namespace for **Amazon RDS** service is **AWS/RDS**.
+ :::
+
+Follow the steps below to collect logs for Amazon RDS:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
### Collect Amazon RDS CloudWatch logs
-Make sure you enable the following parameters before collecting the Amazon RDS CloudWatch Logs.
-
-#### MySQL
-- Amazon RDS [MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MySQL.html#USER_LogAccess.MySQLDB.PublishtoCloudWatchLogs) supports [publishing the following MySQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.MySQLDB.PublishtoCloudWatchLogs.html):
- - Error (enabled by default)
- - SlowQuery
- - Audit
- - General
-- You can enable the following additional parameters at [DB Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithDBInstanceParamGroups.html) for better slow query and general log monitoring:
- - `log_slow_admin_statements`
- - `log_slow_slave_statements`
- - `log_replica_updates`
- - `log_queries_not_using_indexes`
- - `log_output to FILE`
- - `general_log` (to enable, set value to `1`)
-- You can configure [DB Cluster Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.html) to enable audit logs:
- - `server_audit_logging`
- - `server_audit_logs_upload`
- - `server_audit_events`
-
-#### PostgreSQL
-
-- Amazon RDS [PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.html) supports [publishing the following PostgreSQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.html#USER_LogAccess.Concepts.PostgreSQL.PublishtoCloudWatchLogs):
- - postgresql.log
-- You can enable the following additional parameters at [DB parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithDBInstanceParamGroups.html) or [DB Cluster Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.html) for slow query, connection, and query execution timing related logs.
- - `log_connections`
- - `log_duration`
- - `log_min_duration_statement` to a value (in milliseconds) over which statements will be logged for any query taking more time than the given value.
-:::note
-We recommend not setting `log_statement` to any value other than none (default value), since it will slow query logs and ingestion will increase significantly.
-:::
-
-#### MSSQL
-
-- Amazon RDS [MSSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.SQLServer.html) supports [publishing the following MSSQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.SQLServer.html#USER_LogAccess.SQLServer.PublishtoCloudWatchLogs):
- - Agent
- - Error
-
-#### Oracle
-
-- Amazon RDS [Oracle](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.Oracle.html) supports [publishing the following Oracle logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.Oracle.html#USER_LogAccess.Oracle.PublishtoCloudWatchLogs):
- - Alert logs
- - Audit files
- - Listener logs
-
-#### Proxy
-- Amazon RDS [Proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-setup.html) supports [publishing the following Proxy logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-creating.html):
- - Enhanced logs
- :::note
- The log group for an AWS RDS Proxy is created automatically. You do not need to create it manually. When you create an RDS Proxy, AWS automatically creates a CloudWatch Log Group to store logs related to the proxy’s activity.
- :::
-
-Sumo Logic supports several methods for collecting logs from Amazon CloudWatch. You can choose either of them to collect logs:
-
-- **AWS Kinesis Firehose for Logs**. Configure an [AWS Kinesis Firehose for Logs](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-logs-source/#create-an-aws-kinesis-firehose-for-logssource) (Recommended); or
-- **Lambda Log Forwarder**. Configure a collection of Amazon CloudWatch Logs using our AWS Lambda function using a Sumo Logic provided CloudFormation template, as described in [Amazon CloudWatch Logs](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/) or configure collection without using CloudFormation, see [Collect Amazon CloudWatch Logs using a Lambda Function](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/collect-with-lambda-function/).
- -- While configuring the CloudWatch log source, the following fields can be added in the source: - - Add an **account** field and assign it a value which is a friendly name/alias to your AWS account from which you are collecting logs. Logs can be queried via the **account** field. - - Add a **region** field and assign it the value of the respective AWS region where the RDS exists. - - Add an **accountId** field and assign it the value of the respective AWS account ID that is being used. - -
+#### Prerequisites
+
+Esure you enable the following parameters before collecting the Amazon RDS CloudWatch Logs.
+
+* **MySQL**
+
+ - Amazon RDS [MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MySQL.html#USER_LogAccess.MySQLDB.PublishtoCloudWatchLogs) supports [publishing the following MySQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.MySQLDB.PublishtoCloudWatchLogs.html):
+ - Error (enabled by default)
+ - SlowQuery
+ - Audit
+ - General
+ - You can enable the following additional parameters at [DB Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithDBInstanceParamGroups.html) for better slow query and general log monitoring:
+ - `log_slow_admin_statements`
+ - `log_slow_slave_statements`
+ - `log_replica_updates`
+ - `log_queries_not_using_indexes`
+ - `log_output to FILE`
+ - `general_log` (to enable, set value to `1`)
+ - You can configure [DB Cluster Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.html) to enable audit logs:
+ - `server_audit_logging`
+ - `server_audit_logs_upload`
+ - `server_audit_events`
+
+* **PostgreSQL**
+
+ - Amazon RDS [PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.html) supports [publishing the following PostgreSQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.html#USER_LogAccess.Concepts.PostgreSQL.PublishtoCloudWatchLogs):
+ - postgresql.log
+ - You can enable the following additional parameters at [DB parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithDBInstanceParamGroups.html) or [DB Cluster Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.html) for slow query, connection, and query execution timing related logs.
+ - `log_connections`
+ - `log_duration`
+ - `log_min_duration_statement` to a value (in milliseconds) over which statements will be logged for any query taking more time than the given value.
+ :::note
+ We recommend not setting `log_statement` to any value other than none (default value), since it will slow query logs and ingestion will increase significantly.
+ :::
+
+* **MSSQL**
+
+ - Amazon RDS [MSSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.SQLServer.html) supports [publishing the following MSSQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.SQLServer.html#USER_LogAccess.SQLServer.PublishtoCloudWatchLogs):
+ - Agent
+ - Error
+
+* **Oracle**
+
+ - Amazon RDS [Oracle](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.Oracle.html) supports [publishing the following Oracle logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.Oracle.html#USER_LogAccess.Oracle.PublishtoCloudWatchLogs):
+ - Alert logs
+ - Audit files
+ - Listener logs
+
+* **Proxy**
+
+ - Amazon RDS [Proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-setup.html) supports [publishing the following Proxy logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-creating.html):
+ - Enhanced logs
+ :::note
+ The log group for an AWS RDS Proxy is created automatically. You do not need to create it manually. When you create an RDS Proxy, AWS automatically creates a CloudWatch Log Group to store logs related to the proxy’s activity.
+ :::
+
+Sumo Logic supports collecting logs from Amazon CloudWatch using one of the following methods:
+- **AWS Kinesis Firehose for Logs**. Configure an [AWS Kinesis Firehose for Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-logs-source/#create-an-aws-kinesis-firehose-for-logssource) (**Recommended**)
+- **Lambda Log Forwarder**. There are two ways to set up the Lambda Log Forwarder:
+ - **With CloudFormation**. Configure the collection of Amazon CloudWatch logs using Sumo Logic-provided CloudFormation template, as described in [Amazon CloudWatch Logs](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/).
+ - **Without CloudFormation**. Configure the collection of Amazon CloudWatch Logs using a Lambda function, as described in [Collect Amazon CloudWatch Logs using a Lambda Function](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/collect-with-lambda-function/).
+
+Follow the steps below to add custom fields when configuring the CloudWatch log source:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+1. Add a `region` field and assign it the value of the respective AWS region where the Lambda function exists.
+1. Add an `accountId` field and assign it the value of the respective AWS account ID being used.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
### Centralized AWS CloudTrail log collection
@@ -434,7 +467,7 @@ Applied at: Ingest Time
Scope (Specific Data): _sourceCategory=aws/observability/cloudtrail/logs
```
-**Parse Expression**:
+#### Parse Expression
Enter a parse expression to create an “account” field that maps to the alias you set for each sub-account. For example, if you used the `“dev”` alias for an AWS account with ID `"528560886094"` and the `“prod”` alias for an AWS account with ID `"567680881046"`, your parse expression would look like:
@@ -455,7 +488,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
})
-5. Click **Save**.
-
-### Collecting Amazon SNS Events using CloudTrail
-
-1. Add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source.md) to your Hosted Collector.
- * **Name**. Enter a name to display for the new Source.
- * **Description**. Enter an optional description.
- * **S3 Region**. Select the Amazon Region for your SNS S3 bucket.
- * **Bucket Name**. Enter the exact name of your SNS S3 bucket.
- * **Path Expression**. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string.
- * DO NOT use a [leading forward slash](/docs/send-data/hosted-collectors/amazon-aws/amazon-path-expressions).
- * The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression.
- * **Source Category**. Enter a source category. For example, enter `aws/observability/CloudTrail/logs`.
- * **Fields**. Add an account field and assign it a value that is a friendly name/alias to your AWS account from which you are collecting logs. Logs can be queried using the **account** field.
})
- * **Access Key ID and Secret Access Key**. Enter your Amazon [Access Key ID and Secret Access Key](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html). Learn how to use Role-based access to AWS [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources).
- * **Log File Discovery -> Scan Interval**. Use the default of 5 minutes. Alternately, enter the frequency. Sumo Logic will scan your S3 bucket for new data. Learn how to configure Log File Discovery [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources).
- * **Enable Timestamp Parsing**. Select the **Extract timestamp information from log file entries** check box.
- * **Time Zone**. Select **Ignore time zone from the log file and instead use**, and select **UTC** from the dropdown.
- * **Timestamp Format.** Select **Automatically detect the format**.
- * **Enable Multiline Processing**. Select the **Detect messages spanning multiple lines** check box, and select **Infer Boundaries**.
-2. Click **Save**.
-
-## Centralized AWS CloudTrail Log Collection
+## Collecting logs and metrics for Amazon SNS
+
+### Configure Hosted Collector
+
+When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector).
+
+### Collect Amazon SNS CloudWatch metrics
+
+Sumo Logic supports collecting metrics using one of the following source types:
+
+* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (**recommended**)
+* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
+
+ :::note
+ Namespace for **Amazon SNS** service is **AWS/SNS**.
+ :::
+
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+### Collect Amazon SNS CloudTrail logs
+
+#### Prerequisites
+
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+1. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+1. Confirm that logs are being delivered to the Amazon S3 bucket.
+
+ :::note
+ Namespace for **Amazon SNS** service is **AWS/SNS**.
+ :::
+
+Follow the steps below to collect logs for Amazon SNS:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+## Centralized AWS CloudTrail log collection
In case, you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following **Field Extraction Rule** to map a proper AWS account(s) friendly name/alias. Create it if not already present or update it as required.
* **Rule Name**: AWS Accounts
* **Applied at**: Ingest Time
* **Scope (Specific Data)**: `_sourceCategory=aws/observability/cloudtrail/logs`
-* **Parse Expression**: Enter a parse expression to create an “account” field that maps to the alias you set for each sub account. For example, if you used the “dev” alias for an AWS account with ID "528560886094" and the “prod” alias for an AWS account with ID "567680881046", your parse expression would look like:
+
+### Parse Expression
+
+Enter a parse expression to create an “account” field that maps to the alias you set for each sub account. For example, if you used the “dev” alias for an AWS account with ID "528560886094" and the “prod” alias for an AWS account with ID "567680881046", your parse expression would look like:
```sumo
| json "recipientAccountId"
@@ -128,7 +154,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
+### Collect Amazon SQS CloudTrail logs
-### Collect Amazon SQS Events using CloudTrail
+#### Prerequisites
-1. To your Hosted Collector, add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source).
- * **Name**. Enter a name to display for the new Source.
- * **Description**. Enter an optional description.
- * **S3 Region**. Select the Amazon Region for your SQS S3 bucket.
- * **Bucket Name**. Enter the exact name of your SQS S3 bucket.
- * **Path Expression**. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string. (DO NOT use a leading forward slash. See [Amazon Path Expressions](/docs/send-data/hosted-collectors/amazon-aws/amazon-path-expressions).
- :::note
- The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression.
- :::
- * **Source Category**. Enter aws/observability/CloudTrail/logs.
- * **Fields**. Add an account field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- * **Access Key ID and Secret Access Key**. Enter your Amazon [Access Key ID and Secret Access Key](https://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html).
- * **Log File Interval > Scan Interval**. Use the default of 5 minutes. Alternately, enter the frequency Sumo Logic will scan your S3 bucket for new data.
- * **Enable Timestamp Parsing**. Select the **Extract timestamp information from log file entries** check box.
- * **Time Zone**. Select **Ignore time zone from the log file and instead use**, and select **UTC** from the dropdown.
- * **Timestamp Format.** Select **Automatically detect the format**.
- * **Enable Multiline Processing**. Select the **Detect messages spanning multiple lines** check box, and select **Infer Boundaries**.
-2. Click **Save**.
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+1. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+1. Confirm that logs are being delivered to the Amazon S3 bucket.
-## Centralized AWS CloudTrail Log Collection
+ :::note
+ Namespace for **Amazon SQS** service is **AWS/SQS**.
+ :::
+
+Follow the steps below to collect logs for Amazon SQS:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+## Centralized AWS CloudTrail log collection
In case you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following **Field Extraction Rule** to map a proper AWS account(s) friendly name/alias. Create it if not already present/update it as required.
* **Rule Name**: AWS Accounts
* **Applied at**: Ingest Time
* **Scope (Specific Data)**: _sourceCategory=aws/observability/cloudtrail/logs
-* **Parse Expression**: Enter a parse expression to create an “account” field that maps to the alias you set for each sub account. For example, if you used the “dev” alias for an AWS account with ID "528560886094" and the “prod” alias for an AWS account with ID "567680881046", your parse expression would look like:
+
+### Parse Expression
+
+Enter a parse expression to create an “account” field that maps to the alias you set for each sub account. For example, if you used the “dev” alias for an AWS account with ID "528560886094" and the “prod” alias for an AWS account with ID "567680881046", your parse expression would look like:
```sumo
| json "recipientAccountId"
@@ -152,7 +171,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
/` while creating the log group.
+ * API Gateway - Enable Access Logs for each respective API by following instructions in Step 3 of [Collect access logs for AWS API Gateway](/docs/integrations/amazon-aws/api-gateway/#collect-aws-api-gateway-access-logs). Make sure you have the following prefix `/aws/apigateway//` while creating the log group.
* AWS Lambda - If you are exporting logs to your custom log group, make sure you have the following prefix `/aws/lambda/` while creating the log group.
* The AWS Solution does not enable detailed or enhanced metrics collection by default.
* ECS - Enable enhanced metrics for respective cluster. Refer to [AWS documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/deploy-container-insights-ECS-cluster.html).
diff --git a/docs/observability/aws/integrations/aws-api-gateway.md b/docs/observability/aws/integrations/aws-api-gateway.md
index c6b6ad5cb4..a7e78d94f0 100644
--- a/docs/observability/aws/integrations/aws-api-gateway.md
+++ b/docs/observability/aws/integrations/aws-api-gateway.md
@@ -10,6 +10,7 @@ import useBaseUrl from '@docusaurus/useBaseUrl';
[Amazon API Gateway](https://aws.amazon.com/api-gateway/) service allows you to create RESTful APIs, HTTP APIs, and WebSocket APIs for real-time two-way communication applications in containerized and serverless environments, as well as web applications.
The Sumo Logic AWS API Gateway app provides insights into API Gateway tasks while accepting and processing concurrent API calls throughout your infrastructure, including traffic management, CORS support, authorization and access control, throttling, monitoring, and API version management.
+
## Log and metrics types
The AWS API Gateway app uses the following logs and metrics:
@@ -195,7 +196,7 @@ Use these dashboards to:
### Access Logs
Access logs contains information about who has accessed your API and how the caller accessed the API.
-To populate the dashboards, you must explicitly [enable access logs](/docs/integrations/amazon-aws/api-gateway/#collect-access-logs-for-aws-api-gateway).
+To populate the dashboards, you must explicitly [enable access logs](/docs/integrations/amazon-aws/api-gateway/#collect-aws-api-gateway-access-logs).
#### AWS API Gateway - Access Logs - Overview
diff --git a/docs/reuse/apps/app-install-v2.md b/docs/reuse/apps/app-install-v2.md
index f79d28345e..55b981e068 100644
--- a/docs/reuse/apps/app-install-v2.md
+++ b/docs/reuse/apps/app-install-v2.md
@@ -13,7 +13,7 @@ Next-Gen App: To install or update the app, you must be an account administrator
1. **Field Name**. If you already have collectors and sources set up, select the configured metadata field name (eg _sourcecategory) or specify other custom metadata (eg: _collector) along with its metadata **Field Value**.
1. Click **Next**. You will be redirected to the **Preview & Done** section.
-**Post-installation**
+### Post-installation
Once your app is installed, it will appear in your **Installed Apps** folder, and dashboard panels will start to fill automatically.
Click to expand
+Recent Warning Events (Error Logs - MySQL)
-```json title="Recent Warning Events (Error Logs - MySQL)" +```json { "timestamp":1682606169000, "message":"2023-04-27 14:36:09 14487 [Warning] Access denied for user 'dev'@'1.2.3.4' (using password: YES)", @@ -332,97 +333,129 @@ account=* region=* namespace=aws/rds proxyname=* _sourceHost=/aws/rds/proxy/* "D | fields time, proxyname, dbidentifier, db_host, db_port, db_version ``` -## Collecting logs and metrics for the Amazon RDS app +## Collecting logs and metrics for Amazon RDS + +### Configure Hosted Collector + +When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector). + +### Collect Amazon RDS CloudWatch metrics + +Sumo Logic supports collecting metrics using one of the following source types: -Sumo Logic supports collecting metrics using two source types: -* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (Recommended); or +* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (**recommended**) * Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics) -* Namespace for **Amazon RDS** Service is **AWS/RDS**. - * **Metadata**. Add an **account** field to the source and assign it a value that is a friendly name/alias to your AWS account from which you are collecting metrics. Metrics can be queried via the “account field”. + + :::note + Namespace for **Amazon RDS** service is **AWS/RDS**. + ::: + +Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics: +1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value. +1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
### Collect Amazon RDS CloudTrail logs
-1. Add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source.md) to your Hosted Collector.
- * **Name**. Enter a name to display the new Source.
- * **Description**. Enter an optional description.
- * **S3 Region**. Select the Amazon Region for your **Amazon RDS** S3 bucket.
- * **Bucket Name**. Enter the exact name of your **Amazon RDS** S3 bucket.
- * **Path Expression**. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string. (DO NOT use a leading forward slash. See [Amazon Path Expressions](/docs/send-data/hosted-collectors/amazon-aws/amazon-path-expressions)). The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression
- * **Source Category**. Enter `aws/observability/cloudtrail/logs`.
- * **Fields**. Add an **account** field and assign it a value that is a friendly name/alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- * **Access Key ID and Secret Access Key**. Enter your Amazon [Access Key ID and Secret Access Key](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html). Learn how to use Role-based access to AWS [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources)
- * **Log File Discovery** > **Scan Interval**. Use the default of 5 minutes. Alternately, enter the frequency. Sumo Logic will scan your S3 bucket for new data. Learn how to configure **Log File Discovery** [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources).
- * **Enable Timestamp Parsing**. Select the **Extract timestamp information from log file entries** check box.
- * **Time Zone**. Select **Ignore time zone from the log file and instead use**, and select **UTC** from the dropdown.
- * **Timestamp Format.** Select **Automatically detect the format**.
- * **Enable Multiline Processing**. Select the **Detect messages spanning multiple lines** check box, and select **Infer Boundaries**.
-2. Click **Save**.
+#### Prerequisites
+
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+1. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+1. Confirm that logs are being delivered to the Amazon S3 bucket.
+
+ :::note
+ Namespace for **Amazon RDS** service is **AWS/RDS**.
+ :::
+
+Follow the steps below to collect logs for Amazon RDS:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
### Collect Amazon RDS CloudWatch logs
-Make sure you enable the following parameters before collecting the Amazon RDS CloudWatch Logs.
-
-#### MySQL
-- Amazon RDS [MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MySQL.html#USER_LogAccess.MySQLDB.PublishtoCloudWatchLogs) supports [publishing the following MySQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.MySQLDB.PublishtoCloudWatchLogs.html):
- - Error (enabled by default)
- - SlowQuery
- - Audit
- - General
-- You can enable the following additional parameters at [DB Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithDBInstanceParamGroups.html) for better slow query and general log monitoring:
- - `log_slow_admin_statements`
- - `log_slow_slave_statements`
- - `log_replica_updates`
- - `log_queries_not_using_indexes`
- - `log_output to FILE`
- - `general_log` (to enable, set value to `1`)
-- You can configure [DB Cluster Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.html) to enable audit logs:
- - `server_audit_logging`
- - `server_audit_logs_upload`
- - `server_audit_events`
-
-#### PostgreSQL
-
-- Amazon RDS [PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.html) supports [publishing the following PostgreSQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.html#USER_LogAccess.Concepts.PostgreSQL.PublishtoCloudWatchLogs):
- - postgresql.log
-- You can enable the following additional parameters at [DB parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithDBInstanceParamGroups.html) or [DB Cluster Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.html) for slow query, connection, and query execution timing related logs.
- - `log_connections`
- - `log_duration`
- - `log_min_duration_statement` to a value (in milliseconds) over which statements will be logged for any query taking more time than the given value.
-:::note
-We recommend not setting `log_statement` to any value other than none (default value), since it will slow query logs and ingestion will increase significantly.
-:::
-
-#### MSSQL
-
-- Amazon RDS [MSSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.SQLServer.html) supports [publishing the following MSSQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.SQLServer.html#USER_LogAccess.SQLServer.PublishtoCloudWatchLogs):
- - Agent
- - Error
-
-#### Oracle
-
-- Amazon RDS [Oracle](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.Oracle.html) supports [publishing the following Oracle logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.Oracle.html#USER_LogAccess.Oracle.PublishtoCloudWatchLogs):
- - Alert logs
- - Audit files
- - Listener logs
-
-#### Proxy
-- Amazon RDS [Proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-setup.html) supports [publishing the following Proxy logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-creating.html):
- - Enhanced logs
- :::note
- The log group for an AWS RDS Proxy is created automatically. You do not need to create it manually. When you create an RDS Proxy, AWS automatically creates a CloudWatch Log Group to store logs related to the proxy’s activity.
- :::
-
-Sumo Logic supports several methods for collecting logs from Amazon CloudWatch. You can choose either of them to collect logs:
-
-- **AWS Kinesis Firehose for Logs**. Configure an [AWS Kinesis Firehose for Logs](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-logs-source/#create-an-aws-kinesis-firehose-for-logssource) (Recommended); or
-- **Lambda Log Forwarder**. Configure a collection of Amazon CloudWatch Logs using our AWS Lambda function using a Sumo Logic provided CloudFormation template, as described in [Amazon CloudWatch Logs](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/) or configure collection without using CloudFormation, see [Collect Amazon CloudWatch Logs using a Lambda Function](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/collect-with-lambda-function/).- -- While configuring the CloudWatch log source, the following fields can be added in the source: - - Add an **account** field and assign it a value which is a friendly name/alias to your AWS account from which you are collecting logs. Logs can be queried via the **account** field. - - Add a **region** field and assign it the value of the respective AWS region where the RDS exists. - - Add an **accountId** field and assign it the value of the respective AWS account ID that is being used. - -
+#### Prerequisites
+
+Esure you enable the following parameters before collecting the Amazon RDS CloudWatch Logs.
+
+* **MySQL**
+
+ - Amazon RDS [MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MySQL.html#USER_LogAccess.MySQLDB.PublishtoCloudWatchLogs) supports [publishing the following MySQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.MySQLDB.PublishtoCloudWatchLogs.html):
+ - Error (enabled by default)
+ - SlowQuery
+ - Audit
+ - General
+ - You can enable the following additional parameters at [DB Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithDBInstanceParamGroups.html) for better slow query and general log monitoring:
+ - `log_slow_admin_statements`
+ - `log_slow_slave_statements`
+ - `log_replica_updates`
+ - `log_queries_not_using_indexes`
+ - `log_output to FILE`
+ - `general_log` (to enable, set value to `1`)
+ - You can configure [DB Cluster Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.html) to enable audit logs:
+ - `server_audit_logging`
+ - `server_audit_logs_upload`
+ - `server_audit_events`
+
+* **PostgreSQL**
+
+ - Amazon RDS [PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.html) supports [publishing the following PostgreSQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.html#USER_LogAccess.Concepts.PostgreSQL.PublishtoCloudWatchLogs):
+ - postgresql.log
+ - You can enable the following additional parameters at [DB parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithDBInstanceParamGroups.html) or [DB Cluster Parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.html) for slow query, connection, and query execution timing related logs.
+ - `log_connections`
+ - `log_duration`
+ - `log_min_duration_statement` to a value (in milliseconds) over which statements will be logged for any query taking more time than the given value.
+ :::note
+ We recommend not setting `log_statement` to any value other than none (default value), since it will slow query logs and ingestion will increase significantly.
+ :::
+
+* **MSSQL**
+
+ - Amazon RDS [MSSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.SQLServer.html) supports [publishing the following MSSQL logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.SQLServer.html#USER_LogAccess.SQLServer.PublishtoCloudWatchLogs):
+ - Agent
+ - Error
+
+* **Oracle**
+
+ - Amazon RDS [Oracle](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.Oracle.html) supports [publishing the following Oracle logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.Oracle.html#USER_LogAccess.Oracle.PublishtoCloudWatchLogs):
+ - Alert logs
+ - Audit files
+ - Listener logs
+
+* **Proxy**
+
+ - Amazon RDS [Proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-setup.html) supports [publishing the following Proxy logs to CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-creating.html):
+ - Enhanced logs
+ :::note
+ The log group for an AWS RDS Proxy is created automatically. You do not need to create it manually. When you create an RDS Proxy, AWS automatically creates a CloudWatch Log Group to store logs related to the proxy’s activity.
+ :::
+
+Sumo Logic supports collecting logs from Amazon CloudWatch using one of the following methods:
+- **AWS Kinesis Firehose for Logs**. Configure an [AWS Kinesis Firehose for Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-logs-source/#create-an-aws-kinesis-firehose-for-logssource) (**Recommended**)
+- **Lambda Log Forwarder**. There are two ways to set up the Lambda Log Forwarder:
+ - **With CloudFormation**. Configure the collection of Amazon CloudWatch logs using Sumo Logic-provided CloudFormation template, as described in [Amazon CloudWatch Logs](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/).
+ - **Without CloudFormation**. Configure the collection of Amazon CloudWatch Logs using a Lambda function, as described in [Collect Amazon CloudWatch Logs using a Lambda Function](/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/collect-with-lambda-function/).
+
+Follow the steps below to add custom fields when configuring the CloudWatch log source:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+1. Add a `region` field and assign it the value of the respective AWS region where the Lambda function exists.
+1. Add an `accountId` field and assign it the value of the respective AWS account ID being used.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
### Centralized AWS CloudTrail log collection
@@ -434,7 +467,7 @@ Applied at: Ingest Time
Scope (Specific Data): _sourceCategory=aws/observability/cloudtrail/logs
```
-**Parse Expression**:
+#### Parse Expression
Enter a parse expression to create an “account” field that maps to the alias you set for each sub-account. For example, if you used the `“dev”` alias for an AWS account with ID `"528560886094"` and the `“prod”` alias for an AWS account with ID `"567680881046"`, your parse expression would look like:
@@ -455,7 +488,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
+
### Sample queries
@@ -73,43 +77,65 @@ account={{account}} region={{region}} namespace={{namespace}} "\"eventsource\":\
account={{account}} region={{region}} namespace={{namespace}} TopicName={{topicname}} metric=NumberOfMessagesPublished Statistic=Sum | sum
```
-## Collecting logs and metrics for the Amazon SNS app
-
-### Collecting Metrics for Amazon SNS
-
-1. Configure a [Hosted Collector](/docs/send-data/hosted-collectors/configure-hosted-collector).
-2. Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics) or [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (Recommended).
-3. Namespaces. Select **aws/sns**.
-4. **Metadata**. Add an **account** field to the source and assign it a value that is a friendly name/alias to your AWS account from which you are collecting metrics. The **account** field allows you to query metrics.Sample CloudTrail Log Message
+ +```json { -resourceArn:"arn:aws:sns:us-east-1:956882708938:testnull-SumoCWEmailSNSTopic-1NV3GQ8XZ4DFY" -}, -responseElements:null, -requestID:"d8eee5b8-a894-5db4-994c-bef20b57fc0b", -eventID:"2156cf7f-f18d-47f4-b7ba-7b8a6907390a", -readOnly:true, -eventType:"AwsApiCall", -managementEvent:true, -recipientAccountId:"956882708938", -eventCategory:"Management" + eventVersion:"1.08", + userIdentity: + {...}, + eventTime:"2022-07-14T23:06:43Z", + eventSource:"sns.amazonaws.com", + eventName:"ListTagsForResource", + awsRegion:"us-east-1", + sourceIPAddress:"config.amazonaws.com", + userAgent:"config.amazonaws.com", + requestParameters: + { + resourceArn:"arn:aws:sns:us-east-1:956882708938:testnull-SumoCWEmailSNSTopic-1NV3GQ8XZ4DFY" + }, + responseElements:null, + requestID:"d8eee5b8-a894-5db4-994c-bef20b57fc0b", + eventID:"2156cf7f-f18d-47f4-b7ba-7b8a6907390a", + readOnly:true, + eventType:"AwsApiCall", + managementEvent:true, + recipientAccountId:"956882708938", + eventCategory:"Management" } ``` +
-5. Click **Save**.
-
-### Collecting Amazon SNS Events using CloudTrail
-
-1. Add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source.md) to your Hosted Collector.
- * **Name**. Enter a name to display for the new Source.
- * **Description**. Enter an optional description.
- * **S3 Region**. Select the Amazon Region for your SNS S3 bucket.
- * **Bucket Name**. Enter the exact name of your SNS S3 bucket.
- * **Path Expression**. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string.
- * DO NOT use a [leading forward slash](/docs/send-data/hosted-collectors/amazon-aws/amazon-path-expressions).
- * The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression.
- * **Source Category**. Enter a source category. For example, enter `aws/observability/CloudTrail/logs`.
- * **Fields**. Add an account field and assign it a value that is a friendly name/alias to your AWS account from which you are collecting logs. Logs can be queried using the **account** field.
- * **Access Key ID and Secret Access Key**. Enter your Amazon [Access Key ID and Secret Access Key](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html). Learn how to use Role-based access to AWS [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources).
- * **Log File Discovery -> Scan Interval**. Use the default of 5 minutes. Alternately, enter the frequency. Sumo Logic will scan your S3 bucket for new data. Learn how to configure Log File Discovery [here](/docs/send-data/hosted-collectors/amazon-aws/aws-sources).
- * **Enable Timestamp Parsing**. Select the **Extract timestamp information from log file entries** check box.
- * **Time Zone**. Select **Ignore time zone from the log file and instead use**, and select **UTC** from the dropdown.
- * **Timestamp Format.** Select **Automatically detect the format**.
- * **Enable Multiline Processing**. Select the **Detect messages spanning multiple lines** check box, and select **Infer Boundaries**.
-2. Click **Save**.
-
-## Centralized AWS CloudTrail Log Collection
+## Collecting logs and metrics for Amazon SNS
+
+### Configure Hosted Collector
+
+When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector).
+
+### Collect Amazon SNS CloudWatch metrics
+
+Sumo Logic supports collecting metrics using one of the following source types:
+
+* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (**recommended**)
+* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
+
+ :::note
+ Namespace for **Amazon SNS** service is **AWS/SNS**.
+ :::
+
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+### Collect Amazon SNS CloudTrail logs
+
+#### Prerequisites
+
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+1. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+1. Confirm that logs are being delivered to the Amazon S3 bucket.
+
+ :::note
+ Namespace for **Amazon SNS** service is **AWS/SNS**.
+ :::
+
+Follow the steps below to collect logs for Amazon SNS:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+## Centralized AWS CloudTrail log collection
In case, you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following **Field Extraction Rule** to map a proper AWS account(s) friendly name/alias. Create it if not already present or update it as required.
* **Rule Name**: AWS Accounts
* **Applied at**: Ingest Time
* **Scope (Specific Data)**: `_sourceCategory=aws/observability/cloudtrail/logs`
-* **Parse Expression**: Enter a parse expression to create an “account” field that maps to the alias you set for each sub account. For example, if you used the “dev” alias for an AWS account with ID "528560886094" and the “prod” alias for an AWS account with ID "567680881046", your parse expression would look like:
+
+### Parse Expression
+
+Enter a parse expression to create an “account” field that maps to the alias you set for each sub account. For example, if you used the “dev” alias for an AWS account with ID "528560886094" and the “prod” alias for an AWS account with ID "567680881046", your parse expression would look like:
```sumo
| json "recipientAccountId"
@@ -128,7 +154,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';
+
### Sample queries
@@ -90,50 +93,66 @@ account=* region=* namespace=aws/sqs eventname eventsource "sqs.amazonaws.com"
| top 10 username by event_count, username asc
```
-## Collecting logs and metrics for the Amazon SQS app
+## Collecting logs and metrics for Amazon SQS
+
+### Configure Hosted Collector
+
+When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector).
+
+### Collect Amazon SQS CloudWatch metrics
-### Collect Metrics for AmazonSQS
+Sumo Logic supports collecting metrics using one of the following source types:
-Sumo Logic supports collecting metrics using two source types:
+* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (**recommended**)
+* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
-1. Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source). (recommended) Or
-2. Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics).
+ :::note
+ Namespace for **Amazon SNS** service is **AWS/SQS**.
+ :::
- :::note
- Namespace for **Amazon SQS** Service is **AWS/SQS**
- :::
+Follow the steps below to add custom metadata [fields](/docs/manage/fields) with your metrics:
+1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and metrics can be queried using the `account` field.Sample CloudTrail Log Message
+ ```json { "eventVersion":"1.08", @@ -63,6 +65,7 @@ The app uses SQS logs and metrics for: "sessionCredentialFromConsole":"true" } ``` +
+1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
-**Metadata**: Add an account field to the source and assign it a value which is a friendly name / alias to your AWS account from which you are collecting metrics. Metrics can be queried via the “account” field.
+### Collect Amazon SQS CloudTrail logs
-### Collect Amazon SQS Events using CloudTrail
+#### Prerequisites
-1. To your Hosted Collector, add an [AWS CloudTrail Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source).
- * **Name**. Enter a name to display for the new Source.
- * **Description**. Enter an optional description.
- * **S3 Region**. Select the Amazon Region for your SQS S3 bucket.
- * **Bucket Name**. Enter the exact name of your SQS S3 bucket.
- * **Path Expression**. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string. (DO NOT use a leading forward slash. See [Amazon Path Expressions](/docs/send-data/hosted-collectors/amazon-aws/amazon-path-expressions).
- :::note
- The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression.
- :::
- * **Source Category**. Enter aws/observability/CloudTrail/logs.
- * **Fields**. Add an account field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
- * **Access Key ID and Secret Access Key**. Enter your Amazon [Access Key ID and Secret Access Key](https://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html).
- * **Log File Interval > Scan Interval**. Use the default of 5 minutes. Alternately, enter the frequency Sumo Logic will scan your S3 bucket for new data.
- * **Enable Timestamp Parsing**. Select the **Extract timestamp information from log file entries** check box.
- * **Time Zone**. Select **Ignore time zone from the log file and instead use**, and select **UTC** from the dropdown.
- * **Timestamp Format.** Select **Automatically detect the format**.
- * **Enable Multiline Processing**. Select the **Detect messages spanning multiple lines** check box, and select **Infer Boundaries**.
-2. Click **Save**.
+1. [Grant Sumo Logic access](/docs/send-data/hosted-collectors/amazon-aws/grant-access-aws-product) to an Amazon S3 bucket.
+1. [Create a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html).
+1. Confirm that logs are being delivered to the Amazon S3 bucket.
-## Centralized AWS CloudTrail Log Collection
+ :::note
+ Namespace for **Amazon SQS** service is **AWS/SQS**.
+ :::
+
+Follow the steps below to collect logs for Amazon SQS:
+1. Configure a [CloudTrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
+1. Add custom metadata [fields](/docs/manage/fields) with your logs:
+ 1. Click **+Add Field** under **Metadata**. Each field consists of a name (key) and a corresponding value.
+ 1. Create a field named `account` and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the [AWS Observability view](/docs/dashboards/explore-view/#aws-observability), and logs can be queried using the `account` field.
+ 1. After adding fields, check their status indicators:
+ * A green check mark indicates the field exists and is enabled in the Fields table schema.
+ * An orange exclamation icon indicates the field does not exist or is disabled in the schema.
+ * You will have the option to automatically add or enable the field.
+ * If a field is sent but not present or enabled in the schema, it is ignored and marked as **Dropped**.
+
+## Centralized AWS CloudTrail log collection
In case you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following **Field Extraction Rule** to map a proper AWS account(s) friendly name/alias. Create it if not already present/update it as required.
* **Rule Name**: AWS Accounts
* **Applied at**: Ingest Time
* **Scope (Specific Data)**: _sourceCategory=aws/observability/cloudtrail/logs
-* **Parse Expression**: Enter a parse expression to create an “account” field that maps to the alias you set for each sub account. For example, if you used the “dev” alias for an AWS account with ID "528560886094" and the “prod” alias for an AWS account with ID "567680881046", your parse expression would look like:
+
+### Parse Expression
+
+Enter a parse expression to create an “account” field that maps to the alias you set for each sub account. For example, if you used the “dev” alias for an AWS account with ID "528560886094" and the “prod” alias for an AWS account with ID "567680881046", your parse expression would look like:
```sumo
| json "recipientAccountId"
@@ -152,7 +171,9 @@ import AppInstall from '../../reuse/apps/app-install-v2.md';